Redundant Array of Inexpensive Servers
I look forward to the royalties rolling in..
What happens if, as we saw at the launch of Facebook's Open Compute Project on Thursday, the design of servers and data centers is open sourced and completely "demystified"? If open source software is any guide, hardware infrastructure will get better and cheaper at a faster rate than it might otherwise. And someone is going …
Yes, Facebook ain't 'open', but Google is even less so.
They're almost as bad as each other, except at least Facebook doesn't promote itself as a faux-open source mecca for gullible techies and Google fan girls.
It's amusing how Google promotes itself as 'open' at every opportunity - and pity the poor souls who actually believe this...
This post has been deleted by its author
A few of these servers would really fit nicely where I work. We could kick those aging Sparc boxes out the door(they are 5yrs old).
Being practical, is someone were to package up these Mobo's into a package that didn't cost the earth could make themselves a nice lot of money. Well they would until the big players decide to go after the market themselves. To get rid of all the crap that is not needed for a server is a great idea but traditionally 2 socked Mobo's (eg from SuperMicro) are hellishly expensive. So much so, that SME's give up and buy Off the shelf 'so called server's from the likes of Dell, HP etc.
Then there is the question of the Microsoft Tax. At least by building their own Facebook don't have to pay MS a dime. It would be nice if use mere plebs could escale that as well.
It is interesting that they can get the motherboard designed and built cheaper than the can buy a standard one from Dell or HP, they must be ordering a hell of a lot of them for that to work though.
Now, you motherboard manufacturers take note ! I really really need a Micro ITX board with 6 to 8 SATA 2 interaces for my storage servers and NO I DO NOT WANT A FAN ON THE PROCESSOR :)
Looking at the pics the thing that strikes me is the low chip density per server. I am not an engineer but the approach does not seem to me to be anywhere near the needs of a very high-scale data centre as it is not commodified enough. I would expect much higher chip density on motherboards and very little local disk storage allowing scale in three dimensions: cores, RAM and disk like the HPC stuff. Making the servers higher for better cooling means that the chips are running too hot. Better TDP required. And, of course, air cooling isn't that good. How about keeping the servers in water?
1.5U allows bigger fans (more efficient) increased contained air volumes (more effective) taller heatsinks (need lower air flow rates).
More room around the other components provides more thermal isolation, so lower overall temperatures.
If the cubic feet are cheaper than the more intensive cooling you would otherwise need, you win.
If you look carefully, it's a standard motherboard with components removed. This is because motherboard design is rather expensive, especially if you're deviating a lot from the reference design by removing things that everyone will want. So chip density matters, but not enough to justify the cost of a truly custom motherboard.
Most commodity UPS convert the mains input (230v/115V/110V) to +24 or +48V to charge the lead-acid batteries that are used for backup. When the juice fails, they then have to invert it back up to the source AC voltage to power the protected equipment. The servers then convert it back down to +3.3,+5,+12,-5V for the actual electronics.
That's a waste of cost, and power, particularly as the UPS output is generally controlled to a fairly tight tolerance.
Instead, the server accepts the raw 48V from the battery (in parallel with the mains input), and converts it directly to low-voltage power. It also probably has a reasonable input voltage latitude, so the output from the batteries does not need to be stabilised.
and it's been around for at least 20 yrs. Networking equipment has had a 48vdc option forever and servers with 48vdc power supplies were around 15yrs ago.
The real surprise it that they have both a 277 (interesting number, it's a 3-phase industrial spec) AND a 48vdc. Most places just feed everything through 48vdc as there is no switching time when the AC power fails...
They say in the document on chassis and rack triplet that the chassis has a height of 1.5U and the rack is 42U tall. One column in this rack triplet can contain 30 such servers for a total of 90 servers per rack triplet.
30 x 1.5U = 45U
Even on the mechanical drawings the height of the side pane is closer to 50U.
After trying, unsuccessfuly, for several years to get my then employer to let us design our own boxes with commodity parts top to bottom with virtualisation on top and all the goodies I'm not so sure most companies will in fact do this.
There is so much fear and inertia around 'required' support from Netapp or support from Dell/Sunoracle/HP that just convincing management that not only will it work it will actually lower costs and increase reliability I think most companies won't go for it.
Hopefully that's not the case.
Most people agree the cost of the box is tiny compared to the amount spent running the thing. If you've got your own datacentre and dedicated full time staff it's fine, but most companies can't afford to pay someone full time to monitor, manage and meddle with those boxes to keep them running.
Vendors with 4 hour SLAs make much more sense for companies that aren't all about the datacentre.
Biting the hand that feeds IT © 1998–2021