The batteries (there are two of them) are indeed in a holder. To get at them, use an appropriately sized metric hex wrench (2mm IIRC) to remove the six screws along the edges of the top panel. The entire panel then lifts off, along with all the knobs, etc., and the circuit board attached to the underside of the panel.
That circuit board is attached to another circuit board at the bottom of the Octatrack by three ribbon cables, each one of which has a connector at each end. These connectors pull straight out and do not require much force. The battery holder is also on that board.
Given that the oldest Octatracks are only about three years old, and that these batteries last a long time, I am guessing that the problem is grunk on the battery surface rather than that they are actually low. I would suggest removing the batteries and cleaning each surface of each battery with rubbing alcohol, then putting them back. Obviously, you should not put them in backwards
Reconnecting the ribbon cables is a little fiddly because of the tight quarters but not difficult. Good light helps. A lot. If you removed only one end of each cable (doesn’t matter which end), you should have no problem figuring out which way the connector goes. Do be aware that it is possible to seat these connectors so that some pins go into the wrong holes and other pins go into no holes at all. This is obviously a Very Bad Thing To Do.
I make no representations about the effect that doing this might have on your warranty.