Is it possible to switch between 2 sets of monitors?
yup.
As far as i remember you do the routing in your DAW, not in cuemix.
I did this a while ago to have one output pair to the speakers and a cloned output pair to an external recorder. didn’t even open the cuemix software - all it took was dragging some virtual cables (I’m using reason) but I vaguely remember Live also having ctrl room and main out…
First strange thing is the two are not recording at the same volume, OK figure gain on those outs, I assume.
What is killing me is the Headphones. They only play when I am in playback. do i have to boot the other software?
i’ve been using the ultralite mk3 for a few years now - with a macbook running live 8 and reason 7 and never experienced any of the symptoms you have mentioned - all it took was to install the software and plug it in and it worked perfectly ever since.
mind you - i tried every configuration: USB, FW with external power, FW with power thru FW - always good results.
ad different volumes:
maybe you accidentaly messed with trimming? it’s possible to change channel settings from the front panel as well as from the cue mix app so maybe you accidentally changed volumes 
ad headphones:
you can select what goes out to the headphones - it can be either the mains or the ctrl room or a submix of your preference. maybe your issue is there?