An experimental prototype device that uses a Raspberry Pi and some technology from a company called Neutone that does ‘timbral sampling’ using AI and lets you apply the timbral characteristics of one sound to the pitch/envelope of another.
Also a Sonicstate interview with the Roland future Design Lab people that has some additional background. Heavy emphasis from them on the DIY possibilities of this.
This is where it’s going, the whole heaving mass of this industry.
Here’s Neutone’s site with some additional detail. The plugin (morpho) is free and they sell the AI timbral models (although it comes with some free ones).
Interestingly they offer the possibility of using your own sounds to train the AI model, BUT you retain ownership of the resulting model weights and it’s private to you. So rather than being fed into a giant interactive AI and resold to everyone like OpenAI does, it’s privately finetuning an existing model so you can download the weights, which would otherwise require you to have a $5000 GPU and some advanced programming skills.
I think this has a lot in common with the Roland V-synth, but rather than doing a relatively simple spectral analysis of one sound and applying that to another, it’s also modeling the changes in the time domain so it’s a lot more sophisticated than a simple transfer function.
So, basically a vocoder with some AI fluff thrown in for marketing purposes?
Whoosh.
No, vocoders only operate in the frequency domain. Operating in the temporal domain means capturing characteristics of the source material over time - how formants change, if there’s characteristic portamento behavior etc. I don’t think this is ‘AI fluff for marketing purposes’, but I’d have to write a few thousand words to describe modern neural networks, vectorization, and so forth.
This is the key technical paper (pdf) underlying most modern AI systems, you can kind of get an idea od what does on from looking at the signal flow diagrams on page 2 but it’s too much for me to try to break down in an Elektronauts comment. Essentially you’d be feeding the transformer model an hour’s worth of spectrogram data and letting it build up a model of the ‘system’ that produced those sounds. That’s why training is a fundamentally offline process, but once the model is built you can interact with it in near real time.
makers of next-generation AI tools
But can it play in sync with other stuff? 
And so it begins.
Haha. But I think we can unfortunately expect that modulating through latent space comes with a latency penalty … at least for now. Someone who’s tried it please correct me.
The main story here seems to be the Neutone Morpho, the hardware is literally just a little computer and some MIDI control. Roland’s contribution to the concept seems… kind of minimal?
I think the potential for the actual effect is very interesting, great weird sound design potential, stuff that would be very difficult if not impossible to do without machine learning tech. Going to have to try something like this out at some point, I’d be more interested in something that can fully locally though, even if it’s more lo-fi.
Seems to be another offer for “if you can’t do your thing yourself ask AI to take over”.
Honestly - if I want to have a Djembe sound in my beat instead of a Kick, I procure some nice samples and proceed with the production … no AI needed for this (was an example on the plugin web site 
).
It’s nice to know that an AI algorithm could analyse my original sound and replace parts of it with something quite different. I would also not rule out that at some point the result could be quite different and great. But wouldn’t this be more a lucky incidence rather than the ability to craft a sound out of my mind? I prefer crafting, even if it’s old school
I think AI stuff like this is on the cusp of an anteres auto-tune moment.
Some young subgenre finds a way to abuse it that sounds cool and it will take off
No. It will be off by 8 or 50ms 
This seems like it would be interesting to pair with a 303 for new acid sounds.
This 100%
Oh yeah!
There was that holly herndon album from before AI became cringe and oversaturated where she trained an AI on her own voice, this was like 2019 before everyone had access to the tech. I’ve wondered if that album is going to get critically re-evaluated in a good way or bad way. It’s probably still too early to tell.
It’ll be everything people say for good and bad. Lots of interesting ways for it to get used by creative folks unafraid to get technical but I bet a lot of people will just be using it to mimic synths they can’t afford then tell other people how they don’t need an oberheim because their cheap digital synth of choice plus this AI timbral modeler is “just as good.”