Color change Layer Adhesion

I have a V-core 4 500 and it works fine on single color prints. I just started to play with the Color Change feature in Prusaslicer and it seems everything I change out the filament (single extruder) and resume the print, the printer is extruding at a higher level than before the pause and I'm not getting good layer Adhesion. Any tips on how to fix this?