I’d disagree, All MIDI clocks have to go through the CPU, which depending on the load will introduce latency and jitter. Though as you say USB and drivers will play a part, but not the whole

Older computers like Amgias etc had dedicated MIDI chips, it’s also down to the resolution of MIDI clock itself, when you have trackers that can output 1024 PPQN it really won’t stand up.

Here’s another interesting thread: http://www.gearslutz.com/board/so-much-gear-so-little-time/165343-how-do-i-get-rock-solid-sample-accurate-midi-timing.html