Google details plans for 1 MW IT racks exploiting electric vehicle supply chain Google is planning for datacenter racks supporting 1 MW of IT hardware loads, plus the cooling infrastructure to cope, as AI processing continues to grow ever more energy intensive. high voltage illustration OK great, UK is building loads of AI datacenters. How are we going to power that? READ MORE At the Open Compute …
Apart from leveraging an existing mass produced technology I was wondering if there were particular engineering considerations behind 400V and DC.
I always imagined higher voltages and kHz AC (ferrite transformer cores) would be a more efficient.
Given the heat produced I imagine the systems could be arranged in a circle venting into the centre and stacking vertically to form a chimney stack the convective draw would be enough to keep the whole boiling cool. Keeping the rain and pigeons out wouldn't be a problem as they would vaporised or cooked if either managed to overcome the updraft to enter the space.
You seem to be thinking of it as a way to shrink the transformer size, but modern electronics can switch 400vDC at MHz- giving an even better improvement in size.
And as technology improves the last thing you'll want is to be stuck to a legacy kHz (or even 100kHz) standard when others can work at 1MHz or 10MHz or whatever power electronics gets to in future.
I'm not sure when line frequency would have last mattered to PC PSUs- back before the AT PSU certainly. Since then they've been SMPS so have rectified incoming power anyway.
I'm in no way an expert on EMI, but properly designing and shielding a small switching PSU to minimise noise is probably easier than shielding a whole building laced with kHz.
Lastly... it's probably easier to move to 400vDC. Your home PC may well run on 400vDC. I used to run ATX PSUs pretty commonly from >360vDC with no ill effects, and components have only got better since. Much as they're making a big deal of it the engineering effort to move individual computers to 400vDC is pretty minimal, with the special sauce being working at distribution of power at MW levels.
Line frequency has never really mattered unless you had an alarm clock.
PC PSUs have been switch mode since the very beginning and even before the PC was a thing, SMPSUs weren't that uncommon, home computers used them too.
Even then, in a decent linear PSU, line frequency is of very little importance, they'll work on 50 or 60 Hz with only minor loss of efficiency, the only thing that reallyattered was line voltage and the better ones had switchable windings on the transformer
In those ancient SMPSUs many were switchable between 120/240v and that switch actually created a simple voltage double for the poor countries inflicted with 120v line voltage to boost the internal DC primary voltage to over 300v
"they've been SMPS so have rectified incoming power anyway."
Sort of. They may have rectified incoming mains in many cases but HVDC distribution in a box or a cabinet was relatively rare for many years.
One of the reasons for that was safety. An HVDC shock (400V or so as being discussed here,, the rules change again above a few thousand volts) is much less survivable than either LVDC or HVAC. Been there seen that, not just in 1980s computer rooms where no one was allowed in on their own and those in the room had to be properly trained first aiders for post-shock procedures.
I understand the thinking behind using the high voltage infrastructure designed for EV charging for delivering power to the racks, but it's not really new.
IBM Mainframes and the larger supercomputer variants of Power systems have had three-phase power to the rack for a long time. Generally they have what is called a bulk power unit installed at the top of the rack that converts the input to a lower voltage which is then distributed through the rack.
I no longer have the technical details to hand for the Power 7 775 systems that I used to look after, but these had 100's of KW per rack, 10 years ago.
> 100's of KW per rack, 10 years ago.
I must admit I was skeptical when I read that, but you're right! Wikipedia has a blurry photo of a Power7 775 rack that claims to be 360kW, using 400V DC (350-520VDC)
"greater economies of scale, more efficient manufacturing, and improved quality and scale" is something you also get with all that 48V stuff that comes from telecoms and automotive already.
Going for 400V/800VDC as a next step just means you can continue to benefit from some of that scale.
Upside of going 400VDC & up is you don't need quite such ridiculously oversized conductors & connectors, though they'll still be substantial at the power levels involved & might even need plenty of cooling too. Big downside is how solidly hazardous a DC supply at that voltage is in multiple ways & will require some serious safety design & changes to working practices.
The other comical bit is going to be cooling the equipment at that power density, it's not exactly trivial. There's one or two fun examples from old compact supercomputer projects (taking an existing design and making it deskside sized for deployment into flying/floating use), and I've done testing myself with kit that needed ridiculously sized coolant feeds to extract the energy; amazingly easy to heat a full flow full bore watermain if you dump enough power into it. The cooling system will probably end up as a serious hazard in itself.
Maybe the better option is to accept density limits and not chase into the realm of silly engineering requirements and safety hazards?
It was a long time ago (almost 30 years now) but I worked for a company making control systems for factories, one of the jobs we had to meet a short circuit test. This basically dropped a solid lump of copper across 3 phas busbar and the test was it the machine had to stay intact for a set number of seconds and automatically shut down,
I didn’t witness the test but those who did said it was terrifying watching something that you throught was strong just buckle and destroy itself in a few seconds.
If they are talking about this much power in a rack then they must have to pass a similar the same test, or I doubt the dc operator will let them install the rack.
Yeah dropping that spanner across the bus bars would be entertaining, I guess a supply of disposable underwear would be kept near the DC doors.
Switching DC is scary enough at lower voltages, 400 volts would be right scary and finding the people qualified to work with that stuff will be the next issue.
To be fair, i'm not sure which would be more dangerous: Opening a 48V DC breaker at 10,000A or opening a 400V DC breaker at 1000A
The arc voltage goes with the parasitic inductance of the busbars and the (rate of change of) current. I'm not sure that the nominal line voltage would add much to that!
This post has been deleted by its author
I imagine that the Underwriters Laboratory will have to approve the design and do some of their own testing. Or data center operators won't allow it in due to insurance issues.
Some of the design will undoubtedly involve "dead front" removable rack units (pull them out and they unplug themselves from a relatively inaccessible cabinet power bus). Or the technicians will have to start wearing some serious PPE to guard against arc flash injury.
A fellow electrician once shorted some large gear in a high-rise with a metal fish tape.
It melted the fish tape, it melted part of the bus gear, it melted the main breakers for that bus, it melted the fuse in the transformer down the block.
The boss was less than impressed, but we did get an early lunch.
Fun indeed. 48V probably won't kill you if you touch it, 400V is a different beast and once a short gets going (and it surely will) the effects will be spectacular. Fusing is one thing but arc fault detection might also be a good idea. I sort of see why DC is attractive for final distribution though - reducing use of all those pesky heavy and lossy AC transformers.
And unlike AC, the current doesn't go back through zero (which would normally give a chance to extinguish an arc) DC just keeps on arcing
An unexpected bonus to the system, being able to use it for maintenance of the steel building frame like that.
I exaggerate of course. 400V is far more than you'd need for welding. Most units push 50-100 at arc start and stabilize at +/-20 when running, depending on electrode type and arc length.
With 400V and 1000A you could weld with 3/4" rebar as your filler.
I was calibrating a radar unit a long time ago and accidentally reached past the potentiometer while looking into the viewer (it was like a periscope goggle) and gently touched a 2000VCD (yes 2k) low current circuit.
The next thing I knew my arm was hanging by my side with every muscle hurting like crazy.
Being young, I was more embarrassed than hurt, but it definitely took me a few days to get over my instant workout.
The cooling system will probably end up as a serious hazard in itself.
This info was at the end of the article. They've been operating for years already:
"Project Deschutes, features redundant pump and heat exchanger units for greater reliability, and that this has allowed it to achieve a CDU availability of 99.999 percent since 2020."
I suggest investing in fire brigade services too.
Currently, fire suppression systems in DCs generally focus on removing oxygen. However the old school triad of oxygen, combustible material and a source of ignition looks a bit out of date.
If you are pumping 1MW (whatever that means but I imagine it will be a lot of energy) per rack you need to really rethink things as a proper engineer. This has all the hallmarks of the shit show that railway was at back in the day when massively explosive kettles started hurtling along the lines at >100mph and raced each other for prestige and profit.
Railway locomotives are indeed an interesting point of comparison, partiicularly a high end UK electric or diesel-electric one before the City gave everything away to furriners.
E.g. Class43 (Intercity 125) around 2MW each end, most of which ends up feeding traction motors on the locomotive wheels.
Did someone mention EMC (the RFI flavour not the storage array flavor).
"the old school triad of oxygen, combustible material and a source of ignition looks a bit out of date."
Sure does. You'd have thought the Lithium battery fun in recent airliners might be an example to learn from too. But then nowadays this is the age of the untrained.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
