They needed to use Chaos Monkey from Netflix
These guys need to use Chaos Monkey from Netflix to do fault injection more often to stress test the infrastructure. They already have horizontal scaling, so this type of testing is a perfect match for them.
Now not having the systems bootup because of BIOS problems (crappy BIOS maybe) ain't fun at all. I wonder if that's an uptime issue, since they said it could be solved by pulling the complete power from the system and then rebooting.
This is why you try to have multiple motherboards/vendors in your server center to try and spread out the pain when stuff like this pops up. It's not simple, it's not cheap, and it's hard to do at times.
John