I seem to recall having the same issue. I’m on win10 with latest OB, etc. Also had weird issues when using L+R inputs. I am also using Reaper, which as you pointed out, don’t believe is officially supported yet.
Do you have the actual OB plugin enabled on one of the tracks? I stopped using the plugin altogether and that seemed to help (just have the engine running in the background). That said, I was able to remove it because I wasn’t using it, perhaps that’s not the case for you.