I realise using AI for anything is a controversial topic. But please:
HYPOTHETICALLY IF USING AI WAS OKAY (maybe one day)
It would be great to have an app that could convert (as closely as possible), a patch from one synth to any other synth.
Yes, I know the true answer is use your ears and your brain and do it yourself.
I’d still like this to be possible! Maybe the app could have a massive database of synths and it could be open-source.
This doesn’t need two-letters-both-vowels to do. I know it has been done already several times in SW and i think HW for both HW and SW synths. A spectral analysis over time is sufficient, as well as resynthesis, and to some extent some granular synthesis.
ADDED : A first step to this is separating a piece of music into separate components and there is plenty of SW that accomplishes this without TLBV.
Perhaps a more likely approach would be to have it analyse the audio from the original patch and iterate towards getting the same result by tweaking CCs ?
Reminds me of Synplant, not sure if that uses AI to achieve it
EDIT: oh from one synth to another, naw this doesnt do that >.<
I’m thinking, the app could analyse both the patch itself (if possible), and an audio sample of the patch.
Then use both of those produce it’s output patch.
I’d be willing to pay for an app that does that, as long as it had a wide database of synth types!
I do own this, it’s very fun to use!
I’m a layman but that sounds similar to diffusion but instead of noise you have these 50 or so ccs you twist and you just play hot and cold until you can’t get closer.
I love this. So far IP and creative theft has only been able to flourish in the digital realm. Let’s really bring this out into meat space and make sure these things can train on real equipment and sap what curious edge cases might still be out there in that space.
I’ve also been feeling like there’s just too much diversity across synth tones and character. Really nice idea for an effort to close that gap and and embrace the gray mush.
I’m being an ass but I’m just doing my part to keep this true as long as I can.
Isn’t that how that aphex twin dx7 thingy worked? I remember it blowing my mind in like 2013 but nothing ever came of it?
I am not against modest use of AI for support purposes; it is overuse and over-reliance that concerns me (e.g. people using it to code in a language, such as Swift for iOS, that they cannot understand). One of the uses I think is acceptable is for translation between human languages, because we understand that this is difficult even for expert humans, and it is imperfect and often incorrect in subtle ways.
With that in mind, I think something like this may be possible, perhaps not so much between any pairs of synths, but between ones that are close (e.g. the FM engines in Digitone II and M8). Stretching it too far may increase the error rate.
In computer science, when translating from one language to another (“transpiling”, or simply compiling), it is often useful to have an intermediate representation (IR) which abstracts away concrete details. Then it becomes a two-stage process: first language to IR, then IR to second language. The issue is usually how much inefficiency is introduced. A human expert can usually make more drastic leaps between separate language features.
Efficiency may not matter so much in patch terms (if all the LFOs are used up, does it really matter?). So this might be an approach that, with AI assistance, could pay off.
If the musical end goal is to have a close copy of the original sound, that begs the question, why not just sample the original sound?
If this is more of an academic question about what is possible, I suppose a computer could employ a brute-force algorithm, making successive parametric changes, then performing spectral analysis on each of those changes, sloughing off bad decisions and keeping the good ones. I assume, also, that machine learning could improve at this task over time, making more fruitful guesses, eliminating bad guesses.
I suppose another application of AI resynthesis could be to evaluate gear. We typically rely on others’ subjective valuations of gear, which is pretty sketchy. But, given a specific sonic “target”, then determining how close a certain bit of gear can get to that target, could be helpful in choosing gear.
So like Sonic Charge Synplant 2 (Genopatch) there is also MicroMusic AI (for Vital), both front end analysis for those specific sound generators.
But really you could use your own favorite system, feed it what you want and tell it what you want it to give you back. That’s going to slop something back at you, how good that is and what you can do with it is more a matter of you applying it and tweaking it.
sorry didn’t read the whole thread but machine learning and audio analysis is already pretty neat.
some people from ircam in paris if i remember right, already showed a prototype some years ago at ars electronica, where a system noodled out every possible potentiometer position on a virtual synth
and you could make a sound into a mic and it would try to recreate the sound with the synth they learned.
but yeah, make ai go through every possible bleeb of a synth. then play another synth patch into it, and let it recreate that patch.
should work much better today with the latest system. i think i saw that in 2019
what i thought would be more interesting for me is to make ai learn old hardware synths, and let it create impossible sounds with older gear.
i think the mistakes it makes would make interesting sounds
i also liked this project that was shown at ars. pretty much an analog vocoder
As a software developer, our programming languages, libraries and frameworks have already been on a very high abstraction level. I’d argue that quite few programmers really understand what happens under the hood—especially when we go down to C/ASM/machine code level. I see natural language programming as another layer of abstraction, but a very powerful one which DOES highlight the importance of having general software development skills.
If you can program in one language, I don’t see it important to really know specific programming languages. You do need to have a solid grasp of the underlying concepts, and you need to have a healthy amount of scepticism to be able to ask the right questions.
i wish there was an ios version of this… looks like such a fun synth!
Because that’s far from the same as I’m sure you must realise… having full control of the sound etc. Sampling a sound is very different from resynthesis. Useful, but in a different way.
That’s a fascinating take on it, thanks for that!
Vibe coding is here to stay, it seems. For good or bad!
kind of linked - kind of not linked. Made this tool a while back (Codex assist) that attempts to turn audio into a DX7 patch. Mixed results, short tonal sounds work better, but some interesting textures from breaks/noise etc.
I believe there were a couple of small threads. IRCAM and some other folks released similar plugins. In both cases it’s neural-work based, but not LLM (and practically even the smallest open-source LLMs a bit too large to be shipped with the software, but on the other hand, we all got used to 100Gb sound libraries)