Music Synthesizer Technologies made using AI Methods

The Lounge General Discussion
#21

You can train an AI in your voice, or whatever tonal sound, and then perform with that machine learned voice.

Holly Herndon did a short TED talk introducing this technology, and demonstrates her spawned ( not sampled ) voice in this video.

She calls that AI voice Holly+. Part of this process is called timbre transfer which can transfer a live performance in a performers voice, into a synthetic, though very real sounding AI machine learned voice.

You teach an AI a sound in order to generate an entirely new sound.

Yes there have been similar things done in the past, like in particular the Yamaha Vocaloid technology. But be aware of the improvements in sound quality, capabilities and the process involved with this newer technology.

My understanding is that as a part of this that Herndon is using technology from Never Before Heard Sounds. They have a quick demo in this video. ( Not as good a watch as the TED Talk viceo above. )

Also not linked is a Holly+ performance of Jolene, but go ahead and Google it.

ADDED : Herndon is also using AI vocal software from Voctro Labs. They were recently acquired by another AI vocal company VoiceMod. This second company has a long term experience with this sort of thing, having worked on the Vocaloid in the past.

1 Like
The Evolution of Yamaha's Vocaloid
Hardware voice / speech / song synth?
#22

Would love to have a sampler or an update for existing sampler (DT, MPC ) where it’s possible to create audio samples on the fly with a command prompt, same way as we create images with playground ai, midjourney etc.

like „one shot low kick mix between cannon gun and 909 kick“

or „12 seconds long luxury noise transition from heavy metal to cats purring“

also more complicated „sound of silver coin falling on the ceramic plate, followed by rain drops and stretched in time 200x“

This would increase creativity incredibly and make process of working with a sampler great again : )

1 Like
#23

There was a group of companies that got together at NAMM 2023 that made presentations and showed their products dealing with AI uses in audio. The article in Sonic State gives an overview, as well as has a video done with a representative from GPU Audio.

I fixed the links at the end of that article which were mostly garbage :

There is also this video with Mntra .

Recursion Studio -- Video Synthesizer from Entropy & Sons
#24

They run through the details and suggests specific products to use to do your own.

I think you could do the same with your own voice, or with someone you know too.

#25

You know the movie Wishmaster? Better to choose your words more carefully :slight_smile:

Back to the topic: yes, I think what you’re asking is already possible with today’s technology. Isn’t it much more fun to create those sounds yourself though, in particular the luxury noise on metal and with cats purring?

Maybe you should be able to ask the “AI” for less real stuff: “one shot of adrenaline running through a 909 into a kick drum”.

#26

AI drum samples generator.

Emergent Drums by Audialab - Generate infinite, royalty-free drum samples…

1 Like
#28

I think this is a great idea. Even a simple version of this where you might select a few existing patches and it would generate something like those patches, without the natural language interface would work well. Generate ten patches and pick ones you like. And then maybe repeat the process.

It’s not necessarily confined to use with the Osmose but given the difficulty making patches for that synth “by hand” it would be high on my list.

So what’s the first step in approaching such a project ?

2 Likes
#29

The first step of such an approach is defining the problem in both business terms and data terms. Then collecting and labelling the data.

1 Like
#30

Anyone using this? It seems really interesting plugin.

#31

Small world, I bought one of those rebranded Creamware VA boxes off eBay that came from the Ohio Players (it came with a little certificate!)

The sound was cool but the 90s UI just doesn’t work with my ADHD.

I like Synplant’s recreations of various noises for patch generation, not a huge fan of the majority of “AI” branded apps.

I’ll use targeted apps like Stemroller and sit out the glut of crap until the bubble pops, the noise dies down and I can investigate working, useful tools.