DIY with Akamai: What to do when no one sells the servers you need? You build your own

Content delivery network Akamai runs a distributed computing platform consisting of 250,000 servers. At this scale, every bit of IO and every Watt of power consumption matter – so instead of buying hardware off the shelf, the company designs its own. "It would be convenient if we could just go to a normal manufacturer and look …

[Fred M]

Is anyone else unable to concentrate on the text and is just wondering "why are they trying to solder the middle of that capacitor"?

[A.P. Veening]

They are trying to burn out that capacitator as it is just wasting energy at that location without being useful.

[Suricou Raven]

With a remarkably clean iron. I take good care of mine, and it doesn't look anything like that. It's almost as if it is fresh out of the box.

[Will Godfrey]

It seems to have a remarkably thin gas pipe too, and I can't see the flame vents either. It this some kind of new-fangled thing?

Hmmm, anyone know what all those tiny bumps are on that blue sheet thingy?

[Saltee]

I thought exactly the same. Wouldn't be so bad but it's surface mount.

[theblackhand]

Does this make you happier?

https://i1.wp.com/makezine.com/wp-content/uploads/2016/03/stock-photo-beautiful-woman-repair-soldering-a-printed-circuit-board-204001492.jpg?resize=620%2C1004&ssl=1

[Anonymous Coward]

Lucky it wasn't on with that hand placement. She would have ended up with a red hand in short order

[Anonymous Coward]

Surely it was on to allow her to do her job?

Surely you're not suggesting it was faked.....what ever can I even trust now?

[disk iops]

> Managing this variety of hardware is a "constant pain".

Well, quit iterating the stupid hardware then for no good reason. It's just a storage box - no different in purpose really than an S3 or EBS storage node. Doesn't require vast CPU resources because you're not using erasure coding anyway and handling the metadata isn't exactly hard.

The minimum stack used to be 3 machines but from a practical standpoint I don't think I ever saw a pod that wasn't 5 servers.

> 250,000 servers

oooh, so 2/3 (or less) the size of the global S3 farm. Admittedly S3 site locations number in the 'dozens' instead of every tom, dick, harry ISP's Point of Presence.

[John 104]

lol. like 15 different types of hardware is such a burden. That is NOTHING for most businesses that host their own kit.

Still Akamai do a great job and the network is VERY easy to work with/configure.

[Anonymous Coward]

The impression I had was that they were forced to iterate hardware - as you said, they aren't chasing CPU performance but availablity of components becomes a major issue.

Which moves into:

"oooh, so 2/3 (or less) the size of the global S3 farm. "

They appear to build 50k servers a year based on these numbers. I would have expected an integrator to be able to give them COTS hardware that was close enough to what they required for a given price. (ie pay for a higher spec motherboard for IO but use minimum spec CPU etc with NICs/storage you specify). It's not that different to a lot of other vendors already do with white boxes. If they see value in customisation, I wonder how many others may be better off with more customisation?

[Anonymous Coward]

Granted, the Akamai nodes I hosted were long ago and old, but at that time, it was just cheap white-box off-the-shelf servers.

This article makes it sound like they are doing something special. So perhaps they've changed.

[Claptrap314]

Is this news?

I'm sorry, but what exactly here is in any way news? Large buyers having custom SKUs has been a thing in many industries for a long time. Self-designed SKUs is why custom manufacturers exist at all.

I came to the conclusion that Google should be designing their own CPUs about a year before the project was announced (the limited internal announcement happened the month I was hired, roughly). When that went public it was news because it was rare for anyone to do this at that particular level.

[Anonymous Coward]

Here's what are no doubt naive questions but as I don't know and am curious I'll ask it. If these large companies are designing their own motherboards what chip sets would they use? Would these motherboards escape the Intel Management Engine, AMD, or Arm equivalent? Have any of these big players open sourced any of these designs that could be adapted for general PC/Laptop usage?

[disk iops]

They use the same exact chip set everybody else uses - or very, very mildly different. Plus rarely do the big corp actually design the boards. They have SuperMicro or other shops remove (mostly) unwanted parts from the otherwise standard format/layout boards. the FB and other proprietary layouts would rejigger the entire board, obviously.

These guys design SERVER boards, not piddly-ass laptop (with their rediculous contortions) / home computer boards.

Intel has chipsets without the IME but most people simply defeat the chip in BIOS or simply not place it or wire it up.

Anyway they really don't give a damn about theoretical attacks or those carried out by national actors. They are in the business of spying on YOUR workloads afterall...

[Anonymous Coward]

Thanks for the response. I had no doubt that the server builders weren't worried about a personal use motherboard for general computing. I was curious about the importance of the IME in the overall functionality of the motherboard and hence the computer. So, by your response then it is really not important to the functionality other than for the specific functionality it adds and for many neither required nor wanted.

[Anonymous Coward]

Quite unlike the truth ...

What a rosy picture is painted.

The reality is not so nice. I have a lot of respect for Akamai and what they do but a lot of that disappeared when I heard that at least few of the pods in this country are basically unsupported by Akamai and not working as expected.

But I would take a pod to the IX, no doubt about it, if Akamai was interested. They probably aren't if they don't care about the few pods at the largest ISPs.

[Anonymous Coward]

Re: Quite unlike the truth ...

Not sure where you got this info from but it does not sound correct to me. There are a vast number of Akamai nodes (or "regions" in akamai lingo) in the UK which are diversely distributed, well maintained and handling a significant traffic volume. I'd recommend reaching out to your tech contacts for confirmation and clarification of this as you appear to have been given (and now sharing) duff info.

Disclaimer: I work for akamai in a senior technical function.