Try to make A4 as a master device and OT as a slave. I didn’t have issues like you described in such configuration.
By the way I decided to make A4 as master device because it has only 8 banks compared to 16 in OT. And according to the rule that says that the speed (number of banks of A4 in our case) of the slowest device in the system will define the averall speed (number of banks) of the system.
I didn’t check behaviour of A4 in case when OT is master and you change current bank to 15th (in example) which doesn’t exist in A4. But I desided not to risk with this on lives.