Unfortunately changing parts when the parts have different sample assignments does cause some glitches… You can sometimes get around them by carefully changing them just before a downbeat etc.
Using flex machines might yield better results for this stuff…