if you have a look at the SFP window which shows the DSP load, you'll notice that it monitors a dynamic process. The executing code is loaded / unloaded on demand constantly.
If a processing module happens to be loaded across DSP boundary (a part is executed on chip 2 and another on chip 3 or for - whatever is free) then there's a very slight delay in the sub-milliseconds range.
Sometimes this doesn't matter and sometimes it gets noticable, depending on the application.
On stereo channels this is generally unwanted and afaik the little button in the mixers prevents this.
But the phenomenon isn't restricted to mixer channels of course. Since SFP is extremely precise in sound processing, it is easy for an experienced listener to detect this.
On the other hand don't panic. If you move your head for 25 cm that's already close to half a millisecond (I hope I hit the right number) 'latency'

I'm absolutely shure that this is even more widespread (but unnoticed) in many native plugins and even more in cheap monitoring equipment
The dynamic allocation allows you to exploit the DSPs much better by taking the risk of cross DSP loading.
Developers can take care for this and in recent devices they actually do
On some systems (afaik Protools) the DSPs code is 'locked' to certain chips, which of course effectively prevents this situation, but may leave half of the card's power unused in certain configuration.
cheers, Tom