Tough call on which to get first. I’m not an OT fan, so my recommendation sways between A4 and Rytm.

A4:
It is true that you can make drums on A4, but unless you like real short autechrey kicks with hardly any decay, the kick is going to need its own track most of the time. Good thing is, it’s a fiercely nice sounding kick from the A4
When dedicating a Volca beats or sample for your drum work, you can make the A4 go really far with those 4 tracks. I’m consistently pleased with A4 when I let it spread its wings and not handicap its synth capabilities by losing a track or two to drums.

Rytm:
It can do great bass lines and leads from single cycle waveform samples, and has a solid compressor for a good finished sound. You have 8 voices over 12 tracks. These parts alone can make a complete song. There are limitations with doing more evolving atmospheric sounds or pads on the Rytm, due to the single LFO. I feel the Rytm get’s closer to that “complete” song in one box capability. However, my music is mostly rhythm oriented techno (and I purchased the Rytm before purchasing the A4). The combination of both scene and performance modes make it a very fun box to perform and compose live on. I can just hit record on my recorder or DAW and spend 5 minutes jamming on a main pattern with a few variations and at the end of it I have a song I can be proud of.

It probably comes down to the type of music you write, more than anything else.

Is it synth driven, with the occasional need for long evolving and atmospheric sounds, and a rich sound scape? If so, look to the A4

Is it drum & sample driven, with a need for the most realtime control of variations, and a nice finished sound? If so, look to the Rytm.

Best of luck!