Decoding the Digitone SysEx

Another major (and more likely) possibility is that I’m interested in the values of the SysEx so I coded my library to match what is displayed and not the NPRNs. Keep in mind that I’m building this library so I can use a ML algorithm to generate patches using the Tag data. The entire library has been built tested against the display, not the CC or NPRN data. The mapping may need a little bit of tweaking to match up with the NPRNs out.

2 Likes

I did it (vendor NDA) for a video card at one time, and a while later for an audio/midi card for PCs back in the earlier days of Linux, to write Linux drivers for those products. It has been many years since then, so I don’t know if companies still are willing to do that.

Tony

More testing on the Detune:

If the flag is not set (0), then the sysex matches the CC’s sent out:
sysex MSB = CC MSB
sysex LSB / 2 = CC LSB

I still haven’t made sense of the #'s for when the flag is set (8).

Thanks so much for your help. What do you think about moving this conversation over to an Issue page on github?

Sure, didn’t mean to hijack this thread

I started posting issues on your Github.

The way Elektron encoded the sysex drives me insane :confounded:

They are capable of outputting 2 CC’s for all their High Resolution Parameters…but in the sysex they will encode those with 3 values. On top of that, some Parameters share the location of that 3rd value, which means those 2 Parameters are also encoded together (coupled somehow). Why bother? It’s not like the size of the sysex is a commodity and needs to be as small as possible.

1 Like

Hello dhuck and thanks for your work.
I’ve tried to send this sysex message to the Digitone :
msg_array = [int(‘0x00’, 16), int(‘0x20’, 16), int(‘0x3c’, 16), int(‘0x0d’, 16), int(‘0x00’, 16),
int(‘0x6B’, 16), int(‘0x01’, 16), int(‘0x01’, 16), 0, int(‘0x00’, 16),
int(‘0x00’, 16), int(‘0x00’, 16), int(‘0x05’, 16)]

So I sent 0xf0 0x00 0x20 0x3c 0x0d 0x00 0x6b 0x01 0x01 0x00 0x00 0x00 0x05 0xf7

But it didn’t send me back any data…
Did you succeed in getting back the current patch by sending a sysex message to the Digitone. If yes, could you please give us the exact sysex message you sent to the Digitone.

Thanks a lot !

You are missing a value:

0xf0 0x00 0x20 0x3c 0x0d 0x00 0x6b 0x01 0x01 track_number_here 0x00 0x00 0x00 0x05 0xf7

4 Likes

Hello @mekohler !

I didn’t see your answer before. And it is working !!! Thanks a lot for your reply. It’s really a huge step for me to have this working !
Thanks again !

1 Like

Hello,
May be it can help someone.
I found the way Ratio B is encoded in the sysex sound patch dump.
On a midi level, Ratio B is encoded with CC16 and CC48 and covers a range of values from 0 to 360.

  • 0 when CC16 =0 and CC48 = 0 (minimum values)
  • 360 when CC16 = 2 and CC48 = 104 (maximum values)

On a sysex level :
The sysex sound patch dump is composed of 339 bytes that I put (converted into decimal) in an array : sysex[0] to sysex [338]
Ratio B is encoded with 3 bytes using sysex[82], sysex [88] and sysex [89].
And here is the formula I found :

Int ( (sysex[82] x 64) + (sysex[88] x 128) + (sysex[89] / 2) ) = range from 0 to 360

3 Likes

Hi guys,
I think I found something interesting about the way that different parameters share the same sysex location (cf question of @mekohler )

Harmonics is encoded using sysex[90], sysex[91] and sysex[92]
Detune is encoded using sysex[90], sysex[93] and sysex[94]

So the 2 parameters shares the value sysex[90]
What I found is that if you convert sysex[90] into binary, it can be seen as a binary switch. The same switches you can find on some synth to select the midi channel.
sysex[90] can be written b128 b64 b32 b16 b8 b4 b2 b1 and each of these bits can have a value of 1 or 0
If b32 = 1, it modify the way Harmonics is calculated. If b8 = 1, it modifies the way Detune is calculated and of course you can have both b32 and b8 = 1 but b32 only affects Harmonics and b8, Detune.

So here are the formula found :

Harmonics
if b32 of sysex[90]= 0
Harmonics value ={ [((sysex[91]x128) + Int(sysex[92]/2) - 4736 ) x
(5201/6657) ] -2600 } / 100

if b32 of sysex[90]= 1
Harmonics value ={ [((sysex[91]x128) + Int((sysex[92]+128)/2) - 4736 ) x
(5201/6657) ] -2600 } / 100

Detune
if b8 of sysex[90]= 0
Detune value = [ (sysex[93]x128) + Int(sysex[94]/2) ] x 127/16256

if b8 of sysex[90]= 1
Detune value = [ (sysex[93]x128) + Int((sysex[94]+128)/2) ] x 127/16256

3 Likes

I have been playing around with your Python code for the past couple of days and reading your documentation and discoveries. Thanks very much for the work you have done. I probably wouldn’t get anywhere or even start without your work.

If anyone is still interested in the subject, I think I have figured out the root cause of a few quirks, especially with regards to tags and name (which is my focus)

You have covered the subject of 7-bit Encoding of data in your documentation, but I’m not sure why you haven’t applied this to your implementation. The peculiar pattern of the tag bit flags and the random/incorrect characters in the name both can be associated to this.

Impact on Tags
Here is a representation of the bits for the first 8 tags:

      22222222 22222222 11111111 11111111 00000000 00000000
      FEDCBA98 76543210 FEDCBA98 76543210 FEDCBA98 76543210

KICK: -------- -----●-- -------- -------- -------- -------●
SNAR: -------- -----●-- -------- -------- -------- ------●-
DEEP: -------- -----●-- -------- -------- -------- -----●--
BRAS: -------- -----●-- -------- -------- -------- ----●---
STRI: -------- -----●-- -------- -------- -------- ---●----
PERC: -------- -----●-- -------- -------- -------- --●-----
HHAT: -------- -----●-- -------- -------- -------- -●------
CYMB: -----●-- -----●-- -------- -------- -------- --------

The key observation is that the bytes 0x12 (i.e. 28…2F above) actually contains the MSBs of the following 7 bytes. Therefore, the bit set for “CYMB” in 0x12 is really the MSB of 0x17 (i.e. 07 above). That means that a sequence of 32 consecutive bits are used to represent flags in the non-encoded representation.

Impact on Name
The fourth patch in Digitone’s Bank A is named:
PLUCKY EÅ

However, mere UTF-8 decoding of the byte data yields:
PL.UCKY EE

Here are the corresponding bytes:
50 4C 01 55 43 4B 59 20 45 45

Note that byte 0x01 above contains the MSBs of next 7 bytes. So the last character is not “E” (01000101), but rather “Å” (11000101)

The MSBs of the first two characters are actually in byte 0x12.

I’m completely new to SysEx and not that experienced with Python, but I’m working my way through both. I’m hoping to build a tool for managing sound names/tags more easily on the computer.

3 Likes

This is fantastic! Thanks much. I feel like I was working through this back when I was still active on the project and may have something similar to it on a local copy. I lost interest in the project due to school ramping up and the release of Overbridge for the digitone.

My original plan was to make a neural net that created patches, but I don’t think it would work very well unless I made an ungodly amount of patches by hand for the nn to work through. Since the actual human readable name of the tag wouldn’t be as important as just identifying the tag field, I probably got fast and loose with that detail.

If you wanted to add this to the repo, make a pull request and let me know and I can put your contribution in. The next thing I would need to do here is to identify the bits that got added with the operator microtuning update in OS 1.2(1?).

1 Like

From this thread and your documentation I realized your focus was on doing more interesting things with the patches. But my needs are much more simple and mundane.

Basically, given the huge number of sounds available, and the fact that the factory tag applications don’t seem accurate (as brought up here), I needed an easy way to re-tag sounds to make them more useful. And unfortunately Overbridge or Transfer don’t provide any way for managing sounds on the computer. I hope they add this capability.

For now, my plan is to export the current state in CSV format, use another application to edit the CSV, and feed the modified CSV file back into my tool and apply the changes.

I’ll see if I can submit parts of my code back to your repository. But currently I’ve just implemented something from scratch for simplicity.

By the way, I just finished implementing the 7-bit decoding for the data portion of the message (based on the formula you had documented), and the results confirm what I described earlier.

One additional thing I realized is that the name encoding is actually “Latin-1”, not “UTF-8”

I will post an update when I have something useful to share.

3 Likes

I just finished the implementation. You can find it here:

4 Likes

Hey, I’m wondering if you’ve ever looked into project structure in digitone?
I’m currently looking at the digitakt and some wierdness in the message has me confused.

When I dump a project, I get 129 messages.
The first 128 all start with:
F0 00 20 3C 0A 00 50 01 01 {00-7F}

Where {00-7F} I assume is the pattern number.

This to me looks like the same thing you found in your docs here:

However the final message starts
F0 00 20 3C 0A 00 54 01 01 00

Now the thing is, I’m not sure if this last message is the project settings, or if that would be the first one (I originally assumed the first message was project settings)

It’s strange because 54 suggests a different model ID based on your table, however based on other docs, http://www.2writers.com/eddie/tutsysex.htm
I’m thinking 54 01 01 is actually the “address” it’s writing to and the MODEL ID is just “0A 00” (or “0D 00” in the case of digitone)

What do you think? I’m only just starting to look into all this as I’m trying to see if I can recover a corrupted sysex project from digitakt as I only need 2 or 3 of the patterns.

Yer avatar…that building in Vancouver?

1 Like

Isn’t the sysex Model ID usually just one byte?

I think this is about decoding the sysex for sound dumps, not projects or patterns.

layout of the SysEx message for Digitone sounds

But this might be a good starting point to understand the project sysex better.

So, I think the final message could just be an identifier for the type of sysex data. An adress, like you say, could also make sense since that data is obviously not stored in the same place as sound data.

I never made it far into the project sysex data, since there is far too much going on. IIRC, I got the model ID information from the Rytm library, so I didn’t do much work to verify past what the digitone looked like compared to that. What your saying makes sense, but I’m far too removed from this project to answer concretely. My guess would be that the 53/54 discrepancy is there to tell the device what kind of message to expect.

I’ve never had the pleasure to dive too deeply into a digitakt, so I’m not sure how much overlap there would be between the two. Best of luck!