I fixed Elektron´s timestretching (proof of concept vid)

Dear Elektron, please hire this guy:)

43 Likes

That sounds very good… Need to start using timestretching more in my music!

“How to apply to an employer that actually is interesting.”

Brilliant in multiple ways.

3 Likes

Whoa, if this was added to the Digitakt (2), and actually playable (e.g. parameters controllable in real-time), I’d buy one in no time. Really good work. Elektron, please let this guy help you implement good time-stretch (and possibly more!).

8 Likes

Good thing: @dialectrics is somewhere in here, too! :nerd_face:

Brilliant demo!

(and although he didn’t invent it, I still recall the good 'ol days of LFO timestretching on the OG DT, which he helped promoting very well before Elektron took some time to implement something)

5 Likes

Sounds crunchy, but not actually hifi. I like the tone of it, and can see many applications for it, but from listening i wouldnt guess it wouldnt allow a seamless stretch for 10-20% up /down.

1 Like

Yes, he is amazing power user, I´ve also bookmarked all his vids. I believe that the Werp function on DT is basically the LFO timestretching trick with dedicated interface, or at least it sounds like it. His compressor video is also priceless.

2 Likes

The extreme sampling method that makes this possible does inherently give samples a crunchy lofi almost tape like sound, especially at high compression ratios

This idea starter out as data compression first and foremost, the time stretching is a natural consequence of the sparse representation. So it’s a double edge sword. It will alway sound a little crunchy, but the crunch is what makes cheap operations possible

There’s also many unexplored advantages to the sparse domain. One I’ve messed with is applying distortion to the sparse points then reconstructing. Basically you can get massive massive distortion with no aliasing.

15 Likes

it would be an honor and my crowning achievement if I was allowed to put this into the OG digitakt in one final update that multiplies the ram and fixes the time stretch.

Either way I’m making my own looper box prototype with this tech. The inverse digitakt if you will

21 Likes

You may alread have done it, but I want to stress it a bit more: please make Elektron aware of this.

2 Likes

i want to stress… i’m not criticising whats being done here… i just had a few general questions.

obviously i have no knowledge of how powerful the processors are in elektron hardware , or on the eurorack box but i was interested to understand how you quantify the eurorack processor compared to the elektron processor,.

without understanding how processes are distributed across any cpu , or any bandwidth limits , memory access times etc … i assume its hard to quantify …
eg.
running 16 x stereo sample playback systems , each running envelopes, filters , lfo’s and lots of track based processing all locked in sync across the range of bpm from 50 - 200 (or whatever the range is on dt) .
maintaining the ui processing to keep it responsive.
processing all of the 16 tracks of sequence data , inc tha max amount of parameter locks with step based conditional triggering…
plus any encoding required to push everything via usb
global procsessing cpu for delays, reverbs etc,
controlling the reat time performance of ctrl - all , potentially updated 16 tracks (all of which could be updated the entire sound parameters per step) while altering thing with ctrl .

i guess i’m wondering … unless you know how everything is distrubuted across the processers (and im sure ui might be on a different processor to audio / data processing) … , and therefor how little processing is allocated to playing a stereo waveform at 48k etc… how can you judge a eurorack processor that is effectively a single track ?? (mono) sample playback is ‘cheaper’ or more efficient while producing better audible results.

i have no idea if what has been demo’d is well made / efficient or more efficient that what elektron has produced … but on the surface i also presume the eurorack sample stretching is generally ‘playing back a sample’ (and the required processing the pitch/repeat samples in order to simulate ‘stretching’ ) … but is a lot less when compared the what an elektron device might be doing 'behind the scenes on the cpu’s ’ .

just a bit of final context … lets say an elektron device is running on 1 processor / thread etc … with everything going on (ui, sequencer, fx, lfo’s , 16 tracks, usb … basically everything turned ‘on’ on every track) … the actual ‘audio/machine’ processing might be restricted to 10% to ensure no stuttering on audio … without knowing the bandwidth that is reserved for each ‘machine’. … i assume its hard to judge performance and where the compromises have had to be made… if might be perfectly possible to do a better stretching algorithm if allocating more bandwidth to a machine … … but the result might be only 8 tracks , 4 tracks or removal of lfo’s in order to maintain ‘worst case’ of an elektron device being fully stressed .

3 Likes

All totally fair and valid questions, I will do my best to address what you’re asking below.

My performance metric of ~5% on the electrosmith daisy is obtained by timing the clock cycles required for my algorithm to block process audio at 48khz. For reference, 5% is about what a stereo phaser or auto wah from Daisy’s built in libraries costs. The reason I say it’s “cheaper” is because I know my way is faster than any FFT or autocorrelation based method, which is generally what “better” timestretch is based on.

How do i compare Daisy to Digitakt? Daisy uses a STM32M7 processor at 480MHz which to be fair is one of the better ARM chips out there. Compared to the OG digitakt its more powerful and has a floating point unit which offers a large advantage (although my code can operate on fixed point). Compared to the DT2, daisy is weak. Digitakt 2 has the same processor as the OG but also has a DSP unit operating around 1GHZ, specialized for parallel processing. This is a massive advantage over Daisy. Basically means whole buffers get processed in one clock cycle.

As for the other overhead for LFOs, sequencing, overbridge, USB audio, MIDI, screen updates, etc - what you’re describing is essentially why I believe the Digitakt doesn’t have a better timestretch. Always having to account for the worst case scenario (every step filled with 128x retrigs while streaming overbridge, etc) its not possible to justify expensive operations like FFT or autocorrelation.

This is precisely why my method is well suited to limited resource embedded devices. Its essentially all cubic interpolation that could be reduced to linear interpolation if cubic is too expensive. There’s no extra memory required, in fact it makes buffers smaller by design.

Anyways I could talk on this for hours, this algorithm has been my life for the past idk how long but hopefully that addresses your questions. Cheers

19 Likes

my basic takeaway from your explanaition is that… 'you know what you’re talking about ’ … so thanks for the explanation …
its far beyond anything i understand … my background is video games … ive seen many decades of people comparing orange and apples because they dont understand how cpu/gpu / framerates etc work … thanks for your response.

5 Likes

lmao I’m dying :rofl:

Different jargon for different fields. The video game world interests me a lot too, sadly in this day and age many game devs forgo optimization entirely. Luckily we still have Valve and Kojima showing people how its done. cheers m8

1 Like

well, i might disagree … optimization / defining efficient pipeline is essential from day 1 … but i do see games coming out with the typical unreal shader stutter etc … projects should stop adding content many many months before release just to allow art/code/tech etc to finally understand the game theyve made and fine tune the art / draw calls / bandwidth / lodding systems / streaming optimizations for HD / DVD etc etc … PLUS doing all this suitable to work across multiple platforms for various resolutions / drivers / tech expectations … blah blah . … just to add its kojima’s team … not him … i doubt he’s creating lodding systems for character rigs , making background LOD systems to align with shader optimizations … i know he’s the headline name but its the team that makes the game … its just handy for marketing to latch everything onto a ‘personality’

all thats for another thread / forum / lifetime. …

1 Like

Timestretching sounds horrible.

The end.

1 Like

Is there a mode on Digitakt that lets you freely adjust both pitch and speed independent of each other (like your Daisy module)? I sold mine when I bought a Tonverk and I can’t remember… with Tonverk you can kind of do this with the Grainer app by using size (grain size), scan (speed), and pitch data, but obviously the results aren’t as good unless you spend the time to dial in

1 Like

Sometimes just a pinch for utility is all you need :wink: BTW here’s the paper I wrote on the method if you’re curious about the theory.

DAFx26_tmpl.pdf (6.6 MB)

10 Likes

For those that have the Daisy Patch init module, here’s a flashable binary file. The program is the same one from the video and it works as follows:

  1. start with the switch in the down position and plug in an audio source. It expects stereo ins at modular levels. If the levels aren’t hot enough it will not work right!!
  2. flip the switch to the up position to start recording. The buffer is 20 seconds long but higher compression ratio = longer record times
  3. during recording, the bottom right dial controls the compression ratio. minimum position is the highest quality max is the lowest quality.
  4. flip the switch down to stop recording and back up to replay. it will loop automatically.
  5. during playback, the top left dial controls pitch, the top right dial controls time, and the bottom left controls grain size
  6. now toggling the switch will start and stop playback of the recorded sample
  7. press the button to clear the buffer. then, when the switch goes back up recording begins again.

That’s pretty much it.

On full mixes at low quality expect about 4x compression ratios, but on harmonically simple or low frequency material it compresses a LOT. I haven’t really made use of any of the cv or gate inputs in this program but if you have suggestions for how you’d like that set up let me know.

EDIT: if you plug a USB and open the serial terminal you’ll see live CPU and compression info

SparsifierTest.bin (96.0 KB)

6 Likes

Very nice work, funny vid, and thanks for sharing! Amazing contribution :slight_smile:

It seems to me that Tonverk’s Grainer can get pretty close, I’ve had much fun faking timestretch with it ^^

1 Like