If you go modular, I’d suggest to limit yourself, something like a 4MS Pod for the beginning.
If you want to expand, then you will definitely have a use for it afterwards, or easily sell it.
One misconception though is that smaller racks are more “hands on”, you either have larger HP knob-per-function modules, or multi-button-combination smaller modules.
A4 as a sequencer and modulator is a good start, but you will quickly need more modulators, mixers, utility- and logic modules. I’d suggest to start with only one fx module and a poweful modulator, like IME Kermit MK3 or Quadrax with expander. Maybe also only Maths and FX Aid, to keep it simple and fun 