good stuff 
that’s a low % waste too & plenty fast 
I suspect there could be different directions to come to the optimal (taking the cue from the separate padding region above) but I think it’s diminishing returns and if the waste % is that low for a few other length examples then you should be pretty chuffed
.
I’m sure you could set up a randomising routine to grind through a number of potential size combos and print the results (size+pad) out sequentially and pictorially - your eye would quickly pick up if the algorithm did anything ‘off the grid’ or you could do it with some numerical checking
yea it’s good enough for now… 
the other tricky thing is what to do with it specifically…
thought it would be easy to just send kits/sounds to the AR and it would load their samples from the +drive automatically, but.
found out that the AR will only load samples from +drive to the project if you load a sound from the pool or +drive into a track.
so for sharing chain-based kits, I’ll have to resort to trickery with storing the sample info after transfer, then tweaking the kit sysex…
also, and I think you mentioned this before, it would be really cool if there were a universal machine for sounds which works on all tracks, for storing samples. even if it were just a silent dummy machine.
cool! (and thumbs up for releasing your source, too!)
although, to be nitpicky : one of the problems with sample chains is that you have to add quite a lot of extra padding after each slice to avoid accidental triggering of the next slice.
I just spent another hour on this (which almost doubles the dev effort :D) and optimized the sample chain so that the entire 0…120 range is used (approximately). This is also done via iterative refinement (just a quick hack).
The older version stopped at STA=73. The new one creates slightly smaller chains but I’m not sure whether that increased range feature is really an improvement since it becomes a bit more diffult to dial in the sample start.
Here’s the current result of “the great varichain shootout” :
original bsp1 void1 bsp2 void2 bsp3
34004 34480 34960 38790 38010 40380
11800 12930 13110 17240 17738 16152
38356 38790 39330 43100 43078 43072
35744 38790 37145 43100 40544 40380
4834 8620 6555 12930 10136 10768
13106 17240 13110 17240 17738 18844
14846 17240 15295 21550 20272 18844
15282 17240 15295 21550 20272 21536
34004 34480 34960 38790 38010 40380
43146 47410 45885 47410 48146 48456
5704 8620 6555 12930 10136 10768
4310 4310 5384
------ ----- ------ ------ ------ ------
250826 280150 262200 318940 304080 314964
legend:
bsp1 = varichain using my original single-pass algorithm
void1 = varichain using void’s new multi-pass algorithm
bsp2 = bsp1 with extra padding (2000 sample frames = 4000 bytes per slice)
void2 = void1 with extra padding
bsp3 = coarse optimization for 0…120 range (old ver. did not use the entire range)
The numbers are all in the same ballpark. void’s version generates the smallest file (in 9255 iterations), mine is slightly larger but only requires 9 iterations. Fortunately, we don’t use 8MHz CPUs anymore
Well, now we have to see whether these “variable sample chains” are really worth it (not a coding problem).
It will take some trial and error to find the right selection of samples and not make these chains too long (unused/unplayed slices are wasted RAM, too!).
I’m also still a bit skeptical whether dialing in these highly irregular sample start offsets is really practicable.
For the test sample set void and I used, it looks like this: 0, 15, 21, 37, 52, 56, 63, 70, 78, 93, 111, 115 (void posted the STAs for his algorithm on the previous page; they look quite similar).
yep, so far I haven’t used chains at all because I found it super annoying to dial in the start/end values for selecting an element.
So for me personally it makes no difference if the numbers are regular or not, it always sucks.
my goal is to just automate this step away, i.e. generate sysex which stores the elements, and provide a small generated PDF containing info about the chain for those who like to tweak…
e.g. thing 1:
make chains which have (up to) 12 elements, create a kit syx which has one element per track. One kit, one chain.
thing 2:
make a chain with up to 120 elements, create [element count] sound syx. The problem is that there is no sound which can be loaded into any of the tracks, because machines.
I see. I’ve entertained that idea as well (using chains to store entire kits, not the sysex generation part) but I came to the conclusion that this would waste a lot of RAM since you cannot unload single slices (for obvious reasons), and I like to combine samples from different sources (e.g. 808+606), or simply browse for alternative samples (there are usually dozens of versions for e.g. an 808 snare sound).
The only things I can see me using sample chains for are
1:
song-specific sound collections that go through several iterations (upload, modify externally, re-upload, …) (probably via varichains)
2:
wavetables (via regular sample chains that store a number of single-cycle sounds)
p.s.: I have to admit that my varichain algorithm is not working correctly, void’s produces the correct results. My tests sounded ok but the math does not quite add up.
In my defense: I did not spend a lot of time on this. Maybe I’ll take another shot at this tomorrow, there’s got to be a way to solve this problem other than a brute-force numerical approach
ok good to know, I was wondering how to coerce 120-alignment out of your code and didn’t manage, so then started the iterative thing.
there’s probably room for optimization in my code, one obvious thing is the update_sum() loop at each iteration isn’t really necessary, and maybe the main loop has one or more unneeded tests…
but I’m not good at math, not sure how to cut down on number of iterations…
I agree that there has to be a better way, but tbh compared to MIDI transfer times or disk read/writes the speed is negligible, and so far it did handle any combination of sample lengths that I’ve tried reasonably well.
though I would be surprised if there weren’t edge cases where it fails terribly 
Can I play too??!!
I’m not much of a programmer, but I’ve had a crack at developing a calculator tool for this using excel. Not sure how to post a link so I put it in the files section…
Update: Thought I should add my results too 
Can i just say that you all are awesome?
@rex_mundii: looks good! although using excel kind of makes it hard to integrate this into a samplechain tool that automatically creates the chain from a set of samples
@midilifestyle: I guess we are all just crazy 
I rethought my approach this morning over coffee and now the varichain calculation is finally working correctly.
The new version takes advantage of the following aspects of the “extra padding after each slice” property:
[ul]
[li]There is no way of generating a usable samplechain that does not have a fair bit of silence after each slice [/li]
[li]The actual amount of silence is not really important, as long as a minimum amount of silence is guaranteed[/li]
[/ul]
With the test sampleset it now looks like this:
$ tks sds -varichain -noupload -pad 2000 -minpad 1000 -autodir /d/samples/packs/st-xx/wav/ST-01/sc_test/48k/filelist.txt -d
[trc] STA= 0.00 origSz= 16980 padSz= 2385 chSz= 19365 stepSz=1291
[trc] STA= 15.00 origSz= 5878 padSz= 1868 chSz= 7746 stepSz=1291
[trc] STA= 21.00 origSz= 19156 padSz= 2791 chSz= 21947 stepSz=1291
[trc] STA= 38.00 origSz= 17850 padSz= 2806 chSz= 20656 stepSz=1291
[trc] STA= 54.00 origSz= 2395 padSz= 1478 chSz= 3873 stepSz=1291
[trc] STA= 57.00 origSz= 6531 padSz= 1215 chSz= 7746 stepSz=1291
[trc] STA= 63.00 origSz= 7401 padSz= 1636 chSz= 9037 stepSz=1291
[trc] STA= 70.00 origSz= 7619 padSz= 1418 chSz= 9037 stepSz=1291
[trc] STA= 77.00 origSz= 16980 padSz= 2385 chSz= 19365 stepSz=1291
[trc] STA= 92.00 origSz= 21551 padSz= 2978 chSz= 24529 stepSz=1291
[trc] STA=111.00 origSz= 2830 padSz= 1043 chSz= 3873 stepSz=1291
[...] padNewNumSlices=114 int=114
[...] finalNumSlices=120 int=120
[...] origTotalSmpSz=125171
[...] total padding:29749 ratio to unpadded orig=123.767%
[...] ratio to padded orig=105.265%
[...] avg slice padding=1833.58
[...] totalSmpSz=154920 /120=1291
[...] (309840 bytes)
That’s still 480 bytes (0.155%) larger than void’s version (if you set the same avg pad size) but meh, it’s good enough. enough time spent on this nonsense At least it’s working now.
EDIT: I just tried this with a different sampleset (minipops). with void’s algorithm, the samplechain is 638880 bytes, with mine it’s 545520 bytes. Yet another sampleset worked more in favour of void’s algorithm with a size of 688920 bytes vs 715440 bytes with mine. In other words: Both algorithms have their advantages and it depends on the sample set which one creates the smaller samplechain.
share the minipops samples or it didn’t happen
first of all, here’s the source for my “AR Uploader”, in case anyone wants to take a look: http://tkscript.de/4s/sds_22Mar2016.tks.html (quite a hack I originally made this for my Yamaha A5000 sampler)
and here are the minipops samples (I picked them completely at random, apparently the sample set is quite old (pre 2000)): http://tkscript.de/files/tmp/minipops.7z
sweet stuff, will investigate 
.
.
Now I need some other help:
Can you send me $2000 for an urgent operation
Account 606808 Bank of Nigeria
.
.
Edit: have investigated - very sweet - big thanks !!
while we’re at it, can you post these ST-01 samples too…?
edit: nevermind, they’re here: https://archive.org/details/AmigaSoundtrackerSamplePacksst-xx
edit: they don’t seem that great
also updated my code to handle certain edge cases:
previously it could fail if an element was smaller than 1/120 of the sum…
e.g. 2 inputs: 10, 1000000.
No worries - it was more about attempting to give an explanation for the maths side of things, since I don’t really have time to muck about with matlab atm. Too bad this hadn’t come up a month ago, or I’d be able to take it further…
Anyways, carry on!
if anyone wants to have a laugh, here’s the ST-01 disk converted to a samplechain (120 slices): http://tkscript.de/files/tmp/sc-st-01.7z
8bit goodness straight from 1987
I had quite a bit of fun messing with this on the AR a few weeks ago
so ugh…
my algorithm can indeed fail horribly with certain inputs…
back to the drawing board 
…or: we could just agree on a common C interface and create an open source library (“libsamplechain” ? MIT license?) for this that implements different algorithms.
here’s what it could look like (just wrote this down, didn’t try to compile it, hopefully it works:)) :
// Opaque sample chain handle
typedef void *samplechain_t;
typedef struct {
// Query the algorithm name
const char *(*query_algorithm_name) (void);
// Allocate/init samplechain / internal data structures
// - Prepare new output chain
// - Returns new samplechain handle in 'retSc'
void (*init) (samplechain_t *_retSc, uint32_t _numSlices/*120 for AR*/);
// Set algorithm-specific parameter value (e.g. "extra_padding")
// - Returns true if parameter was set successfully, false otherwise (unknown param, value out of range, ..)
// - Requires that init() has been called
bool_t (*set_parameter_i) (samplechain_t _sc, const char *_paramName, int32_t _paramValue);
bool_t (*set_parameter_f) (samplechain_t _sc, const char *_paramName, float32_t _paramValue);
// Add a new element (waveform) to the chain
// - Invalidates the current output
// - May only be called after init() was called
// - Returns true if the element was added, false otherwise (e.g. max number of slices exceeded)
bool_t (*add) (samplechain_t _sc, size_t _numSampleFrames);
// Calculate sample chain
// - Layout sample chain elements and create new output state (for queries)
void (*calc) (samplechain_t _sc);
// Query the current number of elements in the sample chain
uint32_t (*query_num_elements) (samplechain_t _sc);
// Query total size of sample chain (number of sample frames)
// - Requires that 'calc' has been called
// - Returns 0 if something went terribly wrong (tm)
size_t (*query_total_size) (samplechain_t _sc);
// Query start offset of sample chain element (number of sample frames)
// - (note) this is just an utility fxn (could be implemented generically)
// - Requires that 'calc' has been called
size_t (*query_element_offset) (samplechain_t _sc, uint32_t _elementIdx);
// Query total size of sample chain element (number of sample frames)
// - Requires that 'calc' has been called
size_t (*query_element_total_size) (samplechain_t _sc, uint32_t _elementIdx);
// Query original size of sample chain element (waveform) (number of sample frames)
// - (note) the difference to the total size is the number of padding frames
size_t (*query_element_original_size) (samplechain_t _sc, uint32_t _elementIdx);
// Free samplechain / internal datastructures
void (*exit) (samplechain_t *_sc);
} samplechain_algorithm_t;
// Query the number of available algorithms
uint32_t samplechain_get_num_algorithms (void);
// Select an algorithm.
// - Returns algorithm vtable in 'retAlgorithm' if 'algorithmIdx' is valid, NULL otherwise.
void samplechain_select_algorithm (uint32_t _algorithmIdx, samplechain_algorithm_t *_retAlgorithm);
// Example usage:
samplechain_algorithm_t alg;
samplechain_t sc;
samplechain_select_algorithm(0, &alg);
alg.init(&sc, 120);
alg.add(sc, 34004);
alg.add(sc, 11800);
alg.add(sc, 38356);
alg.add(sc, 35744);
alg.add(sc, 4834);
alg.add(sc, 13106);
alg.add(sc, 14846);
alg.add(sc, 15282);
alg.add(sc, 34004);
alg.add(sc, 43146);
alg.add(sc, 5704);
alg.calc(sc);
printf("total samplechain size is %lu sample frames
", alg.query_total_size(sc));
alg.exit();
(EDIT: looks messed up in the forum – copy this to a textfile to make it readable…!)
(EDIT#2: https://github.com/bsp2/libsamplechain/blob/master/algorithm_interface_proposal.h )
for each sampleset, all implementations/algorithms would be called and the result of the one that produces the “best” samplechain is used.
what’s “best” should be configurable, i.e. most tightly packed, most amount of padding between samples while not exceeding a certain size threshold (e.g. more than 20% of the unpadded original), and so on.
(note by: the reason why I favour padding is that this helps to prevent accidently triggering of the next slice when the “END” parameter is constantly set to 120 so that only “STA” needs to be changed to select slices)
(to keep the library simple, the actual selection mechanism does not need to be a part of that, just the interface to query the results)
I think this would be the best approach from a user perspective, and it would save us both any more headaches.
…maybe “rex_mundii” explains how his algorithm works and we can add that to the library, too.
…maybe Google has a secret varichain department willing to step in (who knows!)
yeah guess we could do that…
hmmm what happens with your algo if you throw random inputs at it?
random number of elements (2-120) and random frame count per element?
does it work consistently?
haven’t tried that, yet.
I just set up a github repo, https://github.com/bsp2/libsamplechain/blob/master/algorithm_interface_proposal.h
EDIT: …and added you as a “collaborator”
EDIT#2: …so… are you in ?