Yes that will work.

Just create an external instrument track in Ableton and use the OT like a sampler module without the sequencer. OT is 8 tracks so you will probly want to use sliced samples per track.