fanless with high performance is moot
My first silenced PC was my home 486 @ 120 MHz, back in 1995 or so. This was at the time when CPU fans started to appear - I removed the flimsy active CPU heatsink and used a larger passive heatsink body. In addition to that, I undervolted the 80mm PSU fan from 12 to 5 V. At that point, the AT PSU already had some 4 years of service behind it, catering for a 386DX - and survived maybe 6 more years, until the 486 PC finally went to the scrapyard for moral obsolescence.
Nowadays I work as a troubleshooter in a shop selling industrial PC's, both classic 19" machines and also some fanless models. We don't make fanless PC's, we import them from TW. There are maybe 5 famous brands of such fanless IPC's in TW (famous in Europe and the U.S.). It's not something you cobble together in a miditower ATX case and unplug all the fans. We sell tightly integrated x86 machines with a die-cast or extruded aluminum outside shell, with the outside surface consisting of fins everywhere. Yes, surface is the keyword. But what's even more complicated, is proper thermocoupling of the power-hungry components on the inside, to the outer shell. If the PC maker is not pedantic enough, he cheats by not thermocoupling everything inside properly to the outer shell. Whatever lives inside and consumes electricity, runs hotter than the outside shell... As there's no forced air flow, and telepathic heat transfer doesn't work, any "uncoupled" heat sources on the inside have to rely on natural convection... not a very good prospect. Especially with first-generation fanless PC's, using Banias / Dothan / Core2 "notebook" CPU platforms with about 35 W total TDP, and with botched thermo-coupling, the PC's were plagued by overheating problems. A modern Haswell or BayTrail SoC in a generous finned enclosure, that's a very different story :-)
I've seen people build fanless PC's by taking a desktop ATX case and stuffing all fanless components inside: a fanless CPU heatsink, a fanless GPU card, a fanless PSU... such a PC is rather short-lived :-) No matter how big your heatsinks are, if you keep them closed in a Miditower case without any fan, they don't have too much effect, as it's the PC case's outside surface that matters for heat dissipation - and that surface isn't very big. Without airflow, everyting inside roasts in its own heat. Even if you just use a tall CPU heatsink with heatpipes (can even be active), or water cooling with an outside radiator, pay attention to the fact that CPU and memory VRM's (point-of-load buck converters) on ATX motherboards often *rely* on the toroidal vortex around the CPU socket, caused by a conventional CPU heatsink fan.
For small internal use, I tend to build low-power servers in retired 19"/2U IPC server cases. I use a new silent-ish high-efficiency industrial PSU (with a rear 80mm fan), I adapt the old 19" case to accept a MicroATX motherboard, I remove all the noisy 80mm chassis fans and insert a single low-RPM radial 120mm blower over the CPU heatsink, with its exhaust channeled through the rear wall. The silence is marvellous...
Generally, in home-cobbled ATX PC's and small servers, I like to use large "passive" heatsinks, but only combined with a slow fan that creates a very basic draft of fresh air though the case.