Re: Then there's running an X session remotely.....
The problem with modern X applications working by throwing bitmaps around is actually a problem with the way that people write X client programs today, rather than a problem with X.
They've forgotten all of the short cuts and efficiencies that were built into X to allow it to work efficiently over bandwidth limited networks, as was the case when 10Mb networks were all you would expect to see.
The original font handling code, for example. It had it's limitations, but by using the font definitions stored on the X server it allowed you to transmit text into remote windows extremely efficiently (a few bytes per character compared to full bitmaps for each character), especially once they built scalable font support into X. By the problem is that you were restricted to the fonts that the server had available locally. The official solution was to create font servers, so that when you wanted to use a font that the X server didn't have, it knew how to find and download it, (although there were licensing issues with making fonts generally available on a network, but that's another story).
Instead, what people ended up doing was to locally compose to a pixmap of the entire window on the client, so only the client had to have the font definitions, and then push the whole pixmap for the window across the network.
This is inefficient (actually X even has some efficiencies that allow you to just send the deltas), and only was acceptable because of the increase in speeds in the network.
Obviously, there are problems with highly graphic intensive applications, such as video handling, but by using client and server on the same system and using shared-memory communication between the client and the server, this could be fixed, but then the 'lets throw the baby out with the bath water' brigade decided that the whole X protocol thing running locally was a waste of resource, and promptly started ripping large parts of the function out for the sake of efficiency, rather than making it work better.
Of course, abstracting the graphics acceleration layer on the local system to allow programs to directly use hardware acceleration is always going to be more efficient, but by using an intermediary interface, such as OpenGL - which the X.org implementation can use, can even make this more efficient.
And yes, there are security issues, but networking security issues have been fixed for other things, and would have been (and some have been) fixed as time passed. The computing world used to be a much more trusting environment than it is now, but people took the time to fix it.
X is an old protocol, but as has been pointed out, does things that the replacement systems simply can't do. I keep trying to explain to my wife that just because I'm not sitting in front of a system that is on in one of the other rooms in the house, it does not mean that I'm not working on that system. Most people just see the down-sides, and never the upsides.
I'm an old hack. I used to be one of the people handling X support for AIX in the UK, and I still use it frequently today, even between Linux systems. And I mean today, as I've been doing remote work on one Linux system and an AIX system at my home from my Linux laptop, and also using VNC to control a Windows system (I don't have RDP on the system, at least I don't think I have), and I can tell you the way X works is much more flexible and efficient, and more pleasant to use than VNC.