I feel silly suggesting it because you probably know more about it than me, but does the starting point for each layer possibly have anything to do with it, ie the options for "Nearest, Random" etc...
I can't say I colour swap with my kossel yet so haven't actually tried what you are trying to do - and unless the bridging is happening at the start of a layer, my above suggestion is even more useless