Hi there,
I made a similar experience. Do you work with a hardware mixer or do you route the audio direct into the Orion? This way you have to deal with the input and output latency as well.
For me it helped using a mixing desk. Why such a high sampling rate?
Cheers