Re: When is an appliance not an appliance?
Well, a pre-configured and supported stack of commodity hardware and software is no bad thing, as long as it is properly configured and supported by the vendor.
An example would be the Erastore filers (http://shop.erastor.eu/), basically an x86 box running Nexenta. Nexenta is an OpenSolaris based distribution that runs ZFS. Rather than you building your own server and configuring a BSD on it, you get someone else to do it for you and pay them for support. If later you feel like doing it yourself, you can take over any or all of the stack. It's certainly cheaper than a NetApp if you want a NAS rather than a full blown SAN.
As for things like Exadata, if you push hard enough, quite often you can get some hefty discounts on the hardware if you wave a quote from Dell or HP for compatible boxes at them. Likewise with IBM (though not quite as effective) you can go get a quote for Mainframes or Power/AIX systems from Bull
I do agree though that there are vendors out there that just bodge stuff together with default configurations on a whitebox from SuperMicro, and charge and arm and a leg for the privilege.