AI meets sampling: Coalescence uses machine learning to reimagine what a sampler plugin can do

Dillon Bastan's latest Max for Live device is a "neural concatenative multisampler" that uses neural networks to organize, analyze and play back sample slices

Electronic musician and software developer Dillon Bastan's Max for Live devices are among some of the most creative tools we've seen developed for the platform. 

Whether it's a sequencer based on the emulation of a bouncy ball or a synth that lets a singing flock of birds loose in your Ableton Live set, all of Bastan's tools take a refreshingly unique approach to music creation.

Bastan's latest project is no exception. Coalescence is a multi-sampler that uses the power of machine learning to reimagine what a sampler plugin can do. Load in up to 2000 samples, and Coalescence will slice them automatically and run these slices through its neural network.

Individual sample slices are then analyzed by the neural network and organized into a visual map on an XY grid before being grouped together in clusters based on a property of sound chosen by the user, such as tonality or timbre. Once organized and plotted on the map, samples are colour-coded with reference to that property and can be played back using three different modes. The map can hold up to 2500 sample slices.

Point Mode works much like a classic sampler; choose a single point on the map (each of which represents a sample slice) and incoming MIDI notes will trigger the sample and repitch it based on the note information.  

Rings Mode does things a little differently. The user is able to plot several rings on the map of samples, each of which can be expanded to contain any number of sample slices. Each ring corresponds to a MIDI note, and when that MIDI note is triggered a randomly chosen sample from within the ring's radius will be played. Rings can be set up manually or created automatically based on Coalescence's cluster detection algorithm. 

Finally, Paths Mode creates playback paths that are assigned to individual pitches within the range of an octave. For each note of the octave, the user can plot several points on the sample map that will be played consecutively when that note is triggered. Each path can play in one-shot mode, on loop, or backwards and forwards, and the path playback can be set to jump between points or play all of the samples that occur along the lines of each path.

Coalescence also offers many of the controls and features you'd find in a conventional sampler plugin (volume, pitch, glide, reverse, and transient sensitivity, etc) along with a modulation section equipped with two LFOs and two envelopes that can be routed to two destinations each. There are several filter options including both ladder and formant filters.

Perhaps the most interesting feature of Coalescence, though, is its ability to use incoming audio to determine its output. The input section features a pitch detector and an envelope follower, so that incoming audio can control the pitch and modulation of the sampler. What's more, the sample slice being played at any given moment can be determined by the analysis of incoming audio. 

Route audio into Coalescence, and its neural network will analyse the sound in real time, selecting and playing back the samples that it deems to be the closest match to the incoming audio. This is a pretty wild feature that could make for all sorts of interesting uses; load a set of drum samples into Coalescence, for example, send a different drum loop through the audio input, and the plugin will recreate the loop using your sample set. 

Coalesence is a Max For Live device that works with Ableton Live 10 and up. It's priced at $30 and is available now from Dillon Bastan's website.