Re. "X was a horrible project"
I agree that X was designed for a different environment than personal computers running a GUI on the same system, but to brand it a "horrible project" just goes too far.
Because of it's origins (in academia), it would be fair to say that X10 and X11, particularly the client side, was one of the first "Open Source" projects (along with the original UNIX contributed software products - many of which pre-date GNU). As such, it helped define the model that enabled other open source initiatives to get off the ground. But it suffered teething problems like all new methods, particularly when it got orphaned when the original academic projects fell by the wayside.
What happened with XFree86 and X.org was messy, but ultimately necessary to wrest control back from a number of diverging proprietary implementations by the major UNIX vendors (X11 never did form part of the UNIX standards). I don't fully understand your comment of reducing bloat, unless you mean modularising the graphic display support so you only have to load what you need, rather than building specific binaries for each display type, but that is just a matter of the number of display types that needed to be supported. X11R5 and X11R6 was actually lightweight by the standards of even X.org.
But I have said this before, and I will say it again. If you don't understand what X11 is actually capable of, then you run the risk of throwing the baby out with the bath water. It would be perfectly possible to keep X11 as the underlying display method, and replace GNOME as a window manager (much as Compiz does, and does quite well). This is one of its major strengths, and would allow us die-hard X11 proponents happy. If you use one of the local communication methods (particularly shared memory) you need not necessarily have a huge communication or memory overhead, especially if you expose something like OpenGL at the client/server interface. It's higher than having the display managed as a monolithic single entity, but I don't believe that any of the major platforms do that. There is always an abstraction between the display and the various components.
Having tried Unity and 10.10 netbook on my EeePC 701, surely one of the targeted systems (small display, slow processor) for several weeks, I eventually decided that it was COMPLETELY UNUSABLE at this level of system. The rotating icons on the side of the screen were too slow, and the one you needed was never visible leading to incredible frustration as you scrolled through the available options, trying to decode what the icons actually mean while they fly up and down the screen. It appeared very difficult to customise, and I begrudged the screen space it occupied. My frustration knew virtually no bounds, and it's lucky that the 701 did not fly across the room (note to self - check out anger management courses) on several occasions.
I reverted to GNOME (by re-installing the normal desktop distro), and my 701 is now usable again, and indeed quick enough to be used for most purposes including watching video.
I know I am set in my ways, but I can do almost everything soooo much faster in the old way. I fail to see that adding gloss at the cost of reduced usability and speed helps anybody apart from the people easily dazzled by bling. To put this in context, I also find the interface on my PalmOS Treo much easier to live with than Android on my most recent phone.
I'll crawl back under my rock now, but if Unity becomes the main interface for Ubuntu, I will be switching to the inevitable Gnubuntu distribution, or even away from Ubuntu completely.