|ot-tools-io| open source rust library for reading/writing/modifying octatrack files

Elektron Gear Octatrack
1

been working on reverse engineering the octatrack’s data files over the last few months. the result is this rust library crate which can be used to read/inspect/modify/write .work and .strd data files.

i’ve got most of the files worked out. there’s only a few missing fields left. and arrangement checksumming (which has been doing my head in).

releasing / publicising it now because i’ve got some life stuff this has been distracting me away and it’s in a decent enough state to be ‘useful’, even just as a reference. there are probably some bugs / code still needs cleaning up. so bear that in mind.

hope someone finds this helpful.

24 Likes
2

Hey, congrats, that is big. I was trying not to get pulled in the RS trend, this might change my mind. :slight_smile:

I would definiiley compile the examples and run them, I’ll post issues if anything shows up.

Very interested by the checksum computation, I had been too lazy to look into it seriously. I believe pple at Elektroid, Octalib or Octachainer would be interested too !

Thank you for the big work done.

3

I really want to be interested in this, because I’m usually really interested in Octatrack utilities. Sadly, I don’t understand any of the documentation and have no idea what any of this actually does. (that’s not a criticism; well, it is, but not of you or this, simply of me lol…I’m bad at anything more complex than CSS :laughing:)

4

Very interested by the checksum computation, I had been too lazy to look into it seriously

i think working out bank checksums was like the second or third time i’ve ever gotten really emotional over code that i’ve written (is still a very draft version, need to rework it as quite hacky).

Thank you for the big work done.

you’re welcome.

5

feel free to ask questions if you want.

have no idea what any of this actually does

yeah, i wouldn’t have a year ago either :wink: this was a good hobby project to start learning rust with … it can be a bit of an opaque language until you understand some of it.

if you focus on the ‘structs’ and ‘modules’ and read the fields in the structs you should notice stuff that’s familiar to the octatrack menus. like Part in ot_tools_io::banks::parts - Rust

6

This is rad – I just posted a rex2wav tool, maybe we can finally batch convert rx2 to OT :smiley: Can I use Recycle to prepare sliced loops for the Octatrack? - #51 by charlesv

2 Likes
7

Congrats on the release! And also on having the patience to reverse-engineer the OT binary file formats, I know from experience that that is a slog of a job.

Any plans to build out a UI for this, or will it be strictly CLI? Also, I see you have this on GitLab and GitHub, will those two repos be kept in sync?

8

And also on having the patience to reverse-engineer the OT binary file formats, I know from experience that that is a slog of a job.

yes

also, I see you have this on GitLab and GitHub, will those two repos be kept in sync?

ah yes … erm … well …

i forgot about the sync’d github release repo… i’ve hidden it for now as that was synced from an ‘older’ gitlab repo which has some ‘extras’ you’ve obviously seen … :eyes:

i’ve been figuring out copying banks between projects :eyes: – turns out to be quite complicated :joy: but i’m a decent way there with it. needs a lot of clean up and scanning the markers settings file though.

so yeah, might be more stuff coming. needs some work before i’ll be happy with it. but also don’t want to over promise anything.

Any plans to build out a UI for this, or will it be strictly CLI?

I’d only do a CLI.

I’m more interested getting additional language bindings for the io library and the ‘extra’ library for python, nodejs, c# etc – so other people can make UI based tools / apps in other languages.

2 Likes
9

FYI – could do with some assistance on this PR to finally figure out bank checksums … feel free to create some extra PRs from a fork of your own.

solving this problem pretty much opens up almost every project/file management request that people want, but can’t do yet…

e.g. copying banks between different projects.

3 Likes
10

@dijksterhuis I was looking into your code with help of AI. More specifically trying to understand the checksum story.

AI pointed me towards another project. Did you already have a look at this?

image
image713×234 11.3 KB

11

AI suggested something like this for the arranger checksum based on the other project.

12

i spent several days in a coffee shop trying to get the octarranger implementation to work. it didn’t work for all arrangement test files. it only works for some of them.

using a modified version of your pastebin dump running against a number of previously ignored test case

failures:
    arrangements::checksum_field::files::full_options
    arrangements::checksum_field::files::full_options_no_rems
    arrangements::checksum_field::files::one_halt
    arrangements::checksum_field::files::one_pattern
    arrangements::checksum_field::files::one_pattern_1_jump_1_loop
    arrangements::checksum_field::files::one_pattern_1_jump_1_loop_1_halt
    arrangements::checksum_field::files::one_pattern_1_loop
    arrangements::checksum_field::files::one_rem_row_notext
    arrangements::checksum_field::files::one_rem_row_wtext
    arrangements::checksum_field::files::two_rem_row_wtext

it may be a path towards getting it working, but it doesn’t seem to be the correct implementation just yet.

having said that, the test data for arrangements was done quite a while ago. might be worth someone sitting down an walking through them to double check they do what they say in the filename. (TIP – use ot-tools-io to create a corresponding yaml file for each test file, then read through the yaml file to confirm it does what then file name indicates).

additional note on the pastebin code – it was incorrect/unsuitable to merge in in several ways

  • the OtToolsIoError::ChecksumError enum variant doesn’t exist
  • the while loop always errors due to index out of bounds: the len is 11336 but the index is 11336
  • the bytes length check is redundant
  • inconsistent code style versus other calculate_checksum implementations
  • too heavily commented e.g. // Serialize the entire file to bytes (including current checksum placeholder) for the let bytes = bincode::serialize(&self)?; line.

at the very least, please test code submissions and confirm it works if using ‘next-word-with-highest-probability’ generators (basically, the first two bullet points)

2 Likes
13

If you’re from Netherlands, i understand why working in a coffee shop could not give a satisfying result :joy:

2 Likes
14

scotland, so helpfully there’s none of that nonsense to worry about… at least not INSIDE the coffee shop :face_with_peeking_eye:

2 Likes
15

Okay, good that you already found the Octarranger project. Bit late to the party here :sweat_smile:

16

if a repo has been tagged with elektron or octatrack on github then i’ve probably looked through it :wink:

2 Likes
17

0.6.0 is out now, with a seemingly functional implementation of bank checksum calculations thanks to @imacactus fiddling around with an LLM thingy…

which means edited bank datafiles should now be able to be loaded onto the octatrack. hopefully.

2 Likes
18

also, non-UTF-8 character parsing for project files should hopefully be fixed too. along with a more obvious error message when there’s a OS version mismatch.

2 Likes