FWIW, I've been following the development of VNC for about a year and a half, and it's become pretty obvious to me that although the performance of VNC on CE is pretty bad, that may be more a function of a poor VNC for CE than any inherent problems in the VNC architecture.
So far as I can tell, the only VNC client available for CE is pretty old, seriously suboptimal, and does not implement some of the newer VNC improvements (such as "tight" encoding, etc.) that have made it into newer VNC distros for other platforms. (Check out http://www.tridiavnc.com for some newer and more capable clients for other platforms, or Constantin Kaplinsky's tight encoding site http://www.ce.cctpu.edu.ru/vnc for more info.)
MOre seriously, it looks to me like the real problem isn't so much the lack of better encoding as just simply that the CE VNC client is extremely inefficient and has never been optimized. I don't have CE dev tools (or the time to mess with this), but if someone does, grab the source (available at the the original ORL VNC site at http://www.uk.research.att.com/vnc/download.html ) and have at it. I would expect there's a LOT of room for optimization in there for a decent CE code hacker. Me, I'm working on getting a *real* OS running on this thing, since I want Konqueror on Qt embedded and BSD...