Is anyone else unable to concentrate on the text and is just wondering "why are they trying to solder the middle of that capacitor"?
Content delivery network Akamai runs a distributed computing platform consisting of 250,000 servers. At this scale, every bit of IO and every Watt of power consumption matter – so instead of buying hardware off the shelf, the company designs its own. "It would be convenient if we could just go to a normal manufacturer and look …
Friday 14th June 2019 15:24 GMT disk iops
> Managing this variety of hardware is a "constant pain".
Well, quit iterating the stupid hardware then for no good reason. It's just a storage box - no different in purpose really than an S3 or EBS storage node. Doesn't require vast CPU resources because you're not using erasure coding anyway and handling the metadata isn't exactly hard.
The minimum stack used to be 3 machines but from a practical standpoint I don't think I ever saw a pod that wasn't 5 servers.
> 250,000 servers
oooh, so 2/3 (or less) the size of the global S3 farm. Admittedly S3 site locations number in the 'dozens' instead of every tom, dick, harry ISP's Point of Presence.
Sunday 16th June 2019 07:47 GMT Anonymous Coward
The impression I had was that they were forced to iterate hardware - as you said, they aren't chasing CPU performance but availablity of components becomes a major issue.
Which moves into:
"oooh, so 2/3 (or less) the size of the global S3 farm. "
They appear to build 50k servers a year based on these numbers. I would have expected an integrator to be able to give them COTS hardware that was close enough to what they required for a given price. (ie pay for a higher spec motherboard for IO but use minimum spec CPU etc with NICs/storage you specify). It's not that different to a lot of other vendors already do with white boxes. If they see value in customisation, I wonder how many others may be better off with more customisation?
Friday 14th June 2019 15:32 GMT Claptrap314
Is this news?
I'm sorry, but what exactly here is in any way news? Large buyers having custom SKUs has been a thing in many industries for a long time. Self-designed SKUs is why custom manufacturers exist at all.
I came to the conclusion that Google should be designing their own CPUs about a year before the project was announced (the limited internal announcement happened the month I was hired, roughly). When that went public it was news because it was rare for anyone to do this at that particular level.
Saturday 15th June 2019 10:45 GMT Anonymous Coward
Here's what are no doubt naive questions but as I don't know and am curious I'll ask it. If these large companies are designing their own motherboards what chip sets would they use? Would these motherboards escape the Intel Management Engine, AMD, or Arm equivalent? Have any of these big players open sourced any of these designs that could be adapted for general PC/Laptop usage?
Saturday 15th June 2019 17:01 GMT disk iops
They use the same exact chip set everybody else uses - or very, very mildly different. Plus rarely do the big corp actually design the boards. They have SuperMicro or other shops remove (mostly) unwanted parts from the otherwise standard format/layout boards. the FB and other proprietary layouts would rejigger the entire board, obviously.
These guys design SERVER boards, not piddly-ass laptop (with their rediculous contortions) / home computer boards.
Intel has chipsets without the IME but most people simply defeat the chip in BIOS or simply not place it or wire it up.
Anyway they really don't give a damn about theoretical attacks or those carried out by national actors. They are in the business of spying on YOUR workloads afterall...
Sunday 16th June 2019 12:37 GMT Anonymous Coward
Thanks for the response. I had no doubt that the server builders weren't worried about a personal use motherboard for general computing. I was curious about the importance of the IME in the overall functionality of the motherboard and hence the computer. So, by your response then it is really not important to the functionality other than for the specific functionality it adds and for many neither required nor wanted.
Sunday 16th June 2019 03:46 GMT Anonymous Coward
Quite unlike the truth ...
What a rosy picture is painted.
The reality is not so nice. I have a lot of respect for Akamai and what they do but a lot of that disappeared when I heard that at least few of the pods in this country are basically unsupported by Akamai and not working as expected.
But I would take a pod to the IX, no doubt about it, if Akamai was interested. They probably aren't if they don't care about the few pods at the largest ISPs.
Sunday 16th June 2019 09:43 GMT Anonymous Coward
Re: Quite unlike the truth ...
Not sure where you got this info from but it does not sound correct to me. There are a vast number of Akamai nodes (or "regions" in akamai lingo) in the UK which are diversely distributed, well maintained and handling a significant traffic volume. I'd recommend reaching out to your tech contacts for confirmation and clarification of this as you appear to have been given (and now sharing) duff info.
Disclaimer: I work for akamai in a senior technical function.