Here’s what’s likely happening:
when you first turn on all your gear, the A4 doesn’t get clock data, so maybe it’s set at 120 bpm internally and waiting for new bpm info. When you hit play on your OT, it sends the new clock and the A4 corrects.

I bet it you waited 1 cycle of your pattern and then brought in the A4 it would be OK.