AR Uploader for Windows - Sample Chains and Batch Uploads

Elektron Gear Analog Rytm
#1

Yesterday I bought some nice sample packs (Samples From Mars) and wanted to upload them to the AR.

Well, I quickly ran into the darn C6 issue on Windows that causes batch uploads to stop prematurely.

The good news is that I developed an alternative sample upload tool that actually works.

In defence of Elektron: The root cause for the C6 upload issue is the Windows USB MIDI driver but there is a workaround, so Elektron, please implement that in C6.

In the mean time, you can use my alternative upload tool (yep, it’s free):

http://tkscript.de/files/other/ar_uploader-29Mar2016.7z (latest)
http://tkscript.de/files/other/ar_uploader-26Mar2016.7z (older)
http://tkscript.de/files/other/ar_uploader-25Mar2016.7z (older)
http://tkscript.de/files/other/ar_uploader-19Feb2016.7z (older)
http://tkscript.de/files/other/ar_uploader-14Feb2016.7z (initial release)

It’s a command line application but it comes with an easy to use utility script so you won’t need to become a shell wizard to use this.

For example, the following command line will convert all .wav files in the given directory (and all its sub-directories) to the AR sample format, then upload the samples to the AR:

$ ./upload_dir.sh /d/samples/percussion/packs/samples_from_mars/Modded\ 707\ From\ Mars\ -\ Samples\ From\ Mars/Modded\ 707\ From\ Mars\ -\ WAV/

Please make sure to follow the instructions in the “readme.txt” - precisely.
Install the recommended tools, and also make sure that your AR has a recent firmware (I tested this with 1.22b).

The uploader stresses the AR CPU more than C6 so you will not be able to use your AR while any uploads are in progress.
The upside to this is that transfers are faster than with C6 - still not much faster than a floppy disk but it’s good enough.
Just remember that watching progress bars makes things go slower, so better do something else while the upload is running.

The uploaded samples will be named after the .wav filenames (without the .wav extension), just like with C6.

Changelog:


* 29Mar2016
 - Use libanalogrytm for SysEx handling
     (see https://github.com/bsp2/libanalogrytm ) 
 - Add "-kitsyxout" cmdline option
     (use with -chain or -varichain option)
     (generate + save kit .syx file. sending it to the AR
      will update the RAM edit buffer)
     (sets STA, END, and sound name for each track)
     (sample starts can be finetuned via the
      LFO start phase ("SPH") parameter)
 - Add "-kitsmpnr" cmdline option
     (sets samplenr in generated kit .syx)


* 25Mar2016
 - Fix varichain algorithm
 - Change debug output to print STA (start) positions of varichain slices
 - Add -minpad option (varichain only)
 - Add -indev and -outdev cmdline options
      (the default devices can be configured in "setenv.sh")
 - Pass extra cmdline args to SDS script
      (see "upload_dir.sh", "upload_chain.sh", "upload_varichain.sh")


* 19Feb2016
 - Fix stereo->mono channel conversion in "upload_dir.sh" script
     (add "-c 1" SOX option)
 - Add "convert_dir.sh" script
     (convert to 48k mono 16bit but do not upload samples)
 - Add "-chain <length>" option
     (create sample chain that consists of <length> evenly spaced sub-samples)
 - Add "-varichain" option
     (create sample chain that consists of aligned but not evenly spaced sub-samples)
 - Add "-dir" option
     (prefix filenames in filelist.txt with a specific dirname)
 - Add "-single <filename.wav>" option
     (upload a single wav file instead of using filelist.txt)
 - Add "-name <samplename>" option
     (set AR sample name of single wav or sample chain file upload)
 - Add "-pad <numsamplefragments>" option
     (add padding after each sub-sample in a sample chain)
 - Add "-autodir" option
     (prefix filenames in filelist.txt with dirname of filelist.txt)
 - Add "-d" option
     (lots of debug text, plus this saves the generated sample chain to "debug.wav")
 - Add "-noupload" option
     (e.g. for generating sample chains without uploading them)
 - Add "upload_chain.sh" script
     (convert directory to 48k mono 16bit and upload as sample chain)
 - Add "upload_varichain.sh" script
     (convert directory to 48k mono 16bit and upload as vari-chain)


 * 14Feb2016
   - Initial release.
2 Likes
Chains - a Sunday afternoon math riddle
SDS Drop for Mac - Better Sample Upload
#2

cool :slight_smile:

what are you doing to stress the AR more though?
not waiting for ACKs?
what’s your speed relative to c6?

#3

Yep, not waiting for ACK.

Upload speed is ~48…80k/sec but the AR needs some time after each upload to store the sample to flash (my guess) so in average you probably won’t get more than 25k/sec (~=floppy speed).

I just left the sample pack upload running over night so the speed did not bother me that much.

#4

One last thing I forgot to mention:

You do not necessarily need to install MSYS and SOX.

At the very least, just unpack the tool somewhere, open a regular “cmd.exe” shell and type

d:\ar_uploader>tks applications\sds filelist.txt

“filelist.txt” must be a text file that simply contains the list of samples you want to upload (one filename per line).

To create a file list using the regular windows shell, type

d:\ar_uploader>dir /B /S d:\samples\packs\st-xx\wav\ST-01*.wav >filelist.txt

The “upload_dir.sh” script is just sugar on top but requires MSYS and SOX, which I highly recommend.

1 Like
#5

Very useful - is there any potential to upload into a RYTM folder structure?

#6

If someone told me how to remotely create a folder and select that for upload, I would implement that ASAP.

As far as I know, this is not possible right now.

BTW: A word of warning: Now that I’ve filled my own AR with a lot of samples (~0.5GB) sample uploads have become quite slow (~1k/sec).

That does not make this tool any less useful, of course.

If someone told me how to remotely create a folder and select that for upload, I would implement that ASAP.

As far as I know, this is not possible right now.

BTW: A word of warning: Now that I’ve filled my own AR with a lot of samples (~0.5GB) sample uploads have become quite slow (~1k/sec).

That does not make this tool any less useful, of course.

EDIT:

Before I am accused of spreading FUD, here are some actual time measurements:

57241k, 781 files, finished over night (<7h)

3228k, 121 files, ~43min => ~1,25k/sec

3337k, 1 file, 108sec => ~31k/sec

Seems that the uploads become slow if they are many small files. It does not seem to have anything to do with the fill level of the +drive (what a relief!).

#7

well that would be bad haha :smiley:
transfer time increasing exponentially the more the drive is filled - therefore making it impossible to fill the drive 100%. :stuck_out_tongue_winking_eye:

about your measurements - are these raw kilobytes over USB, or the size of the wave files?
I’d like to compare this to my SDS Drop app - maybe we should try a bunch of the same test files, then stop the time?

My app is probably slower, but you can still use the Rytm while transferring samples. It’s roughly twice as fast as c6…

#8

hehe, that would be very bad, indeed.

Regarding the transfer speeds: I measured the time between script launch (upload_dir.sh) and when the AR was 100% done with processing/saving the new sample.

That includes the 48khz conversion but since I have a fast PC, this only takes a very brief moment (maybe half a second in case of the single sample test).

The transfer speed itself, based on the duration of just the SDS packet send, sometimes goes up to 110k/sec but what really counts is the actual time, i.e. USB/MIDI transfer PLUS flash writes.

IIRC, Elektron changed something in the firmware a while ago (was it 1.02D?) that allowed SDS packets to be sent faster than in previous releases.
My guess is that the AR caches the packets in RAM if the flash memory writes cannot keep up.
That would explain the transfer speed fluctuations.

I just uploaded the single 3337k sample with C6 and it took 148sec, so my uploader is 37% faster in this case.

For obvious reasons, I could not check how long it takes to transfer those 121 small samples – C6 aborts the batch upload randomly and that’s why I wrote this alternative uploader in the first place.

1 Like
#9

Does this tool maintain the folder structure of the source files? Including subfolders?

#10

mokomo already asked this (=> no)

#11

Saw that. But, I reckoned he was asking about something different. I.e., uploading to a specific folder might be different than maintaining existing folder structures of the source files.

#12

Okay then, point taken.

Nevertheless, maintaining a folder structure would require uploads to specific folders and remote creation of these.

Unfortunately Elektron did not implement neither of that in the firmware (if they did, they did not document it).

It’s also not part of the MIDI Sample Dump Standard so it would require vendor-specific SysEx commands.

1 Like
#13

Thanks bsp for making this available to the community!

#14

@LMLMLM: no problem :slight_smile:

Here’s a new release of the AR uploader which adds some nifty features:

** SAMPLE CHAINS **

Since the February 19th release (today), this tool can also be used to create, and upload “sample chains”.

A sample chain is basically just a long sample that contains “sub samples” that can be played by adjusting the sample start parameter on the Analog Rytm.

Sub-samples in sample chains are automatically aligned so that the exact sample start can be dialed in on the Analog Rytm (without requiring any tricks like using an LFO for fine tuned start points).

Sample chains come in two flavours:
[ul]
[li]With regular sample chains, sub-samples are evenly distributed in the generated waveform. Therefore, a fixed sample start “step” (e.g. 8) can be used to address the sub-samples on the AR [/li]
[/ul]
[ul]
[li]“Varichains” use varying sub-sample lengths. Here, the sample start parameter on the AR is not a multiple of a “step” constant. However, all sub-samples are still aligned to a common block size. The advantage of vari-chains is that they require less memory, especially when some sub-samples are short, and others are quite long.[/li]
[/ul]
Last but not least, it is possible to add some silence / padding after each sample chain sub-sample.
This can be used to avoid accidental replay of the following sub-sample.

Historically, “sample chains” were first used with “Tracker” programs on the Amiga home computer in the early 90ies.The software back then had a limitation of 31 samples per song (“module”), so musicians had to find a way around that limitation – they stored multiple samples in one waveform, then used the sample offset command (9xx) to select the sub-samples.

Well, the Analog Rytm has 127 sample slots, so the big advantage of using sample-chains here is that you can swap out an entire set of samples (up to 120) by simply replacing one single waveform.


Change Log:
[ul]
[li]Fix stereo->mono channel conversion in “upload_dir.sh” script (add “-c 1” SOX option)[/li]
[li]Add “convert_dir.sh” script (convert to 48k mono 16bit but do not upload samples)[/li]
[li]Add "-chain " option (create sample chain that consists of evenly spaced sub-samples)[/li]
[li]Add “-varichain” option (create sample chain that consists of aligned, but not evenly spaced sub-samples)[/li]
[li]Add “-dir” option (prefix filenames in filelist.txt with a specific dirname)[/li]
[li]Add “-single <filename.wav>” option (upload a single wav file instead of using filelist.txt)[/li]
[li]Add "-name " option (set AR sample name of single wav or sample chain)[/li]
[li]Add "-pad " option (add padding after each sub-sample in a sample chain)[/li]
[li]Add “-autodir” option (prefix filenames in filelist.txt with dirname of filelist.txt)[/li]
[li]Add “-d” option (lots of debug text, plus this saves the generated sample chain to “debug.wav”)[/li]
[li]Add “-noupload” option (e.g. for generating sample chains without uploading them)[/li]
[li]Add “upload_chain.sh” script (convert directory to 48k mono 16bit and upload as regular sample chain)[/li]
[li]Add “upload_varichain.sh” script (convert directory to 48k mono 16bit and upload as vari-chain)[/li]
[/ul]

Download:
http://tkscript.de/files/other/ar_uploader-19Feb2016.7z

Have fun!

p.s.: some related tricks that you might find useful:

[ul]
[li]Route a zero-speed LFO to “SAMP: Start”, select the “RMP” LFO wave, set “DEP” to +1, “MOD” to “TRG”, then use the “SPH” parameter to finetune the (sub-)sample starts[/li]
[li]Plock sample starts in a pattern or use scenes that target the sample start, then use a performance controller to modify the start points relative to the plocks/scenes[/li]
[li]Use single-cycle, looped sub-samples to achieve wave/graintable like effects. To scan the wavetable, use MIDI or a performance controller mapped to both sample start and end.[/li]
[li]Place entire sample kits in a sample chain. Use the same sample type ordering (BD, SD, HH, …) and chain size for different kits => It now becomes very easy to replace kits (just replace the waveform)[/li]
[/ul]

2 Likes
#15

Great work! Meant to say earlier that I really appreciate you making this available to the community. Thanks!

#16

Here’s an update.

Changelog:


 * 25Mar2016
   - Fix varichain algorithm
   - Change debug output to print Analog Rytm STA (start) positions of varichain slices
   - Add -minpad option (varichain only)
   - Add -indev and -outdev cmdline options (the default devices can be configured in "setenv.sh")
   - Pass extra cmdline args to SDS script (see "upload_dir.sh", "upload_chain.sh", "upload_varichain.sh")

Download: http://tkscript.de/files/other/ar_uploader-25Mar2016.7z

Note: The default in/out device names are “Elektron Analog Rytm”. If you’re using the Korg USB-MIDI driver, edit the “setenv.sh” script and uncomment the lines that read
#export SDS_INDEV=“Analog Rytm in 1”
#export SDS_OUTDEV=“Analog Rytm out 1”

Here are some example samplechains:
[ul]
[li]ST-01 Amiga 8bit sampledisk (from 1987): http://tkscript.de/files/tmp/sc-st-01.7z (120 slices)[/li]
[li]Korg Mini Pops drum machine (varichain): http://tkscript.de/files/tmp/sc-minipops.7z (13 slices, see .txt file for STA params)[/li]
[/ul]

1 Like
#17

…and another update (the forum’s been offline the past few days, could not post this any earlier):

Changelog:


 * 26Mar2016
   - Use libsamplechain (see https://github.com/bsp2/libsamplechain )
   - Support pathname arguments with spaces in utility scripts
   - Add "-chainout" cmdline option
       (allows sample chain to be saved to a file other than "debug.wav")

D/L: http://tkscript.de/files/other/ar_uploader-26Mar2016.7z

1 Like
#18

Just passing by to say that I am amazed at people like you @bsp or @void that just share their good work like this…
You guys keep it making this a better place !

Cheers, and keep up the good work gentlemen !

1 Like
#19

^^ and thank you for the nice music!

void was right when he said that setting up the sample starts/ends for varichains in a kit is quite annoying.
This update adds at least some automatism for that.
You still need to load the sample manually (via C6 or this tool) but at least the kit .syx is auto-generated now.

If you have followed the instructions in the “readme.txt”, it’s as simple as this:


$ cd ar_uploader
$ . setenv.sh
$ ./upload_varichain.sh /d/samples/percussion/packs/minipops/ vc-minipops -kitsmpnr 46 -kitsyxout vc-minipops.syx -chainout vc-minipops.wav -noupload

(convert the minipops/ directory to 48Khz/16bit, create varichain .wav and matching kit .syx file)
(the generated “vc-minipops.syx” and “vc-minipops.wav” files can be found in the minipops/48k/ directory)
(remove the “-noupload” option to upload the generated .wav to the AR right away)

Changelog:


 * 29Mar2016
   - Use libanalogrytm for SysEx handling
       (see https://github.com/bsp2/libanalogrytm ) 
   - Add "-kitsyxout" cmdline option
       (use with -chain or -varichain option)
       (generate + save kit .syx file. sending it to the AR
        will update the RAM edit buffer)
       (sets STA, END, and sound name for each track)
       (sample starts can be finetuned via the
        LFO start phase ("SPH") parameter)
   - Add "-kitsmpnr" cmdline option
       (sets samplenr in generated kit .syx)

D/L: http://tkscript.de/files/other/ar_uploader-29Mar2016.7z

1 Like
#20

Hi bsp,

Many thanks again for this tool. I finally had a chance to play with it. I noticed something weird, which I doubt is related to your tool, but I am still curious to know if you have experienced anything similar.

I tried to upload about 2700 samples. The tool completed successfully, but I was only left with about 512 samples in my upload directory. After some more testing, it seems like the max number of samples you can upload to a RYTM directory is about 512.

I then tried to upload in batches and copy each batch into a main directory, but after about 1024 samples, RYTM stopped allowing me to copy files into that directory. In fact, it would not even let me delete files (and other odd behavior). I ended up having to reformat my drive.

My +Drive was definitely not full. I formatted the drive and tried again with the same result.

I know you did a test above with about 700 samples. I am wondering if you were actually really able to get all 700 samples uploaded to your upload directory, or if you had the same behavior as me.

I am going to continue uploading, but I am going to keep my directories less than 512 samples.

Curious to know your thoughts.

Thanks!

1 Like