Re: "nifty software tricks to achieve similar speeds on CPUs"
My previous home server had 2 cpus (an old dell workstation) and my current rack mount server has 2 cpus, I've worked on 4 and 8 cpu rack mounts before and even more specialist blade servers can be had for around the 50k mark with 8+ cpus present.
If you're only thinking of desktop machines then of course they tend to only have the 1 cpu. But 2+ cpu systems have been available since the mid 90s (pentium Pro for example) and pretty much anything with a xeon (other server grade cpus are available) badge is designed with multi cpu in mind but for heavier IT workloads and bigger budgets.
Also with regards to having 4+ graphics cards you need more pcie lanes and more cpus also allow for that without having to add more systems.
Oh and even smaller desktop machines have been known to be dual cpu, Apple for example (ever heard of them?) did one called Powermac G5 which could be sourced with 2 cpus.
Any questions?
/rant