Re: VNC is your friend
I use TightVNC on perhaps 7 raspberry pi's, 3 odroid HC2's, half a dozen NUCs, and find it to be well, not even close to anyone's definition of slow. I use Remmina on the PC's (all this is linux, various).
Sadly the clipboard support is a little dodgy, but otherwise....nope, not slow, seems often as fast as just "being there" would be.
Maybe your experience is with (far) older versions of something, or some terrible setup options.
I generally choose medium quality and the native pixel bits for whatever the server side is.
Which is usually higher than the default, yet it's fast - maybe not selecting things that avoid transcoding is your issue?