I label them with short names: A01, A02, A03, etc. It simplifies the process and as long as I know what they are, then it’s not too much of an issue. If I thought mutual exclusion of more tracks were possible or a better option I would tell you honestly to keep trying, but for me, this is the way to go.
It’s nice to solve problems, but it’s also good to save your mental energy for the real problems and put the rest towards making music 
There may be a solution to the changing of samples via the round robin lfo technique, however. Search the words round robin on this forum and you’ll see some topics about it. Here is one: