Sound resynthesis is a vast topic. I’ve been studying it for years, and I’ve spent the last few months building a dedicated music tracker based entirely on resynthesis—and let me tell you, it’s a real pain in the ass. There are plenty of analyzers that might give you great results for some instruments, while producing total garbage for others. So, you really have to view it more as a source of inspiration and interpretation rather than a direct solution to the problem. Actually, additive synthesis is a highly underrated technology in this regard; it’s far deeper than it might seem.
3 Likes
Here are examples of full resynthesis. There is no processing here.
3 Likes
First nit fond of ai for this but:
Depends on how close it should be. Sone sound cant be reproduced. How to decide when its “as good as it gets” in a solution space as broad is a none trivial task. If roughly ballpark between similar synths thats easily solvable by software.
If not? Reinforcement machine learning could work.
The generall ai understanding now (llms/transformer) would be an unelegant brute force solution thats unrelyable but would work sometimes.
Unlike generative ai that synthesises the output, here it would need to synthesize the parameters to generate an output, and ulike programming where its already in the form that llms understand (text) and with endless documentation and examples online to train model, we dont have those luxurys with “synth presets->sound” and the ai has no conceptual understanding of it.
My question becomes more: whats the goal? Why not just ise the synth it was made with? And if the goal is to emulate all synths, … I understand this to make everything as plain simply to be universal.
The beauty of an additive synthesizer is that the sounds created within it act like modeling clay—they can be blended together, allowing for actual sound morphing. Take a look at the Visco plugin; it’s not a sampler, but rather a resynthesis tool based on a truly ingenious design.
By the way, I think the engine from this device could easily have been used in the Model:Samples. But Elektron likely lacks the boldness for a device like that.
1 Like
General curiosity – What are the compute demands of additive synthesis / resynthesis in general? Is it reasonable that something with the CPU power of the Model line to implement this in a sonically pleasing way?
For modern (processors? We had va synths since the 90s. The microkorg was from was an entry level poly va synth from 2002.
Now what some see as pleasing is a good hard question. Adding some better filtering, 32bit float processing would be easy and cheap today.
Add some feature to add harmonic distortion and you are probably indistinguishable.
Or heck. Look at the Roland Aira S-1 comparisons. Its an 4 voice sh101 emulation with solid delay/reverb/chorus effects and a competent sequencer, and the abc simulation is widely accepted as indistuingishable in a mix from the real deal as far as i understood. (At least i could not confident say i hearthe difference in comparisons)
It all depends on the model and the number of partials in the source material. Analyzing the model is the most resource-intensive part, but with a well-configured optimizer, you can certainly achieve highly usable results. The analysis phase—which forms the foundation of your model—is crucial, given that handling transients and elements with rapid decay is one of the weak points of additive synthesis.
This presents a trade-off or a choice of approaches: upon loading the snapshot, you could convert it to PCM (which was my initial solution) for higher quality and lower CPU load—though this requires more RAM and complicates morphing—or you could design the model to “re-synthesize” parts of the signal, specifically the harmonically rich elements. The team at Forever 89 did a tremendous job in this regard, achieving excellent results with a low processing load. I imagine their approach involves a complex blend of various engineering ideas.
2 Likes
Well, I see it a bit more broadly than just a virtual equivalent. To me, spectral resynthesis is essentially a replacement for the standard PCM sampler—which takes up tons of space and, at best, can only layer sounds. That’s just too boring for me.
Hm i think im a bit lost. Are we still talking about transitioning presets from one synth to another, or are we talking about digital aditive synthesis or about recreating the quirks oh analog synthesis?
I threw my two cents into the discussion. It’s such cool technology—it immediately sparks my curiosity about what more could be achieved if I had access to full sound resynthesis.
Sounds great!
1 Like