Dell finds liquid cooling tech on eBay, now wants you to buy it Dell is getting into the water-cooled server business for hyperscalers with an offering called “Triton” that it developed for eBay. Dell says it's keen on water cooling because it's cheap: data centres nearly always have a cooling tower that lower the temperature of water so it is sensible to put that cool water to good use. …
Yeah, I'm not sure that what you want in a datacentre when things go wrong is water, connected - directly or indirectly - to a metal plate stuck inside an electrical device, connected to every other electrical device in the building.
To be honest, even if they're using some oil-based liquid instead that is only cooled by the cooling towers, and it's non-conductive etc. still the mess from one leak is going to take out an entire rack in seconds before your fancy drip alarms get a chance to go off.
I think there's a definite case here for a risk analysis that says "Yes, it'll get more oomph out of our CPUs but if ANYTHING EVER goes wrong with this untested and new tech that we've spent a fortune on, we're immediately into losses".
The mind boggles, other than it being a means of moving the decimal point on the price tag for something painted a different colour. I mean its not as if they can sell the "rugged bit" by saying the second digit of the ingress protection rating is anything special for a simple pipe.
I'd ask for the tech specs, but even for reg readers that would probably be a ... bore
350 psi water pressure in one long long water tube coming down from the ceiling and running through the length of your data center. Just don't ever touch those tubes.
I assume you could always remove a defective processor from underneath the cooler block and glue another one in place (no, don't touch the water tubes!).
"Water and electricity are never a good mix. Too many opportunities to damage a pipe leading to significant outage."
That has never seemed to stop the Americans using water based overhead fire extinguishing systems in datacentres regardless of the obvious stupidity and inherent safety hazards...
decided to "Go Green in a BIG way"
The rack was HP I think, not sure.
Fully water cooled, "plug and pray" piping to connect to the cold water, and so on.
1 / condensation is a bitch.
2 / hard water... ever passed a kidney stone ? ever passed a CPU stone ??
3 / that little 5-7 cents O-ring that was selected because it was the cheaper one for the specs ? yeah, well, did you see in the doc that you had to change them quite often ? also the bursar forgot to buy the GREASE that should be put on it.
4 / when the 3rd rack passed away, we put an ROI sheet on the manager desk explaining him how much it would cost to put all racks in a cold aisle vs buying all the racks server and storage again.
5 / my esteemed colleague (and datacenter official graybeard) asked for a training as a plumber instead of the proposed cisco cert, explaining that all together he had more water related problems than network related issues.
6 / Green It project was deemed 'too much in advance' and they bought carbon bonds instead...
...the time when some ICL1900 systems used disc drives with hydraulic actuators. There were pipes and pumps everywhere in these these things and changing an actuator was a major plumbing job, which nearly always resulted in a leak somewhere that needed fixing. Nightmare!
I was planning to change the joint between the low level cistern and WC in our downstairs loo on Saturday morning when I realised that SWMBO was running the washing machine, which is plumbed through the wall from the same cold water pip that I would have to shut off to drain the cistern.
Wouldn't like to think of the logistics of putting shut off valves on every motherboard and catching the spray and drips inevitable after dicsonnecting 350PSI water pines.
Mainframes have been water cooled for quite a few decades. I can remember, from back in the 1980s, one of the mainframes had a plumbing elbow fracture (probably due to vibration or temperature cycling causing metal fatigue of the Copper joint). The first indication of a problem was when the people working on the floor below called up to ask why there was water dripping through the ceiling. Uhoh! The next indication of a problem was a CPU over-temperature alarm, and a subsequent automatic shutdown of the machine. The service tech showed up a few minutes later, and replaced the plumbing elbow. Everyone was really dreading the need to replace the CPU, which, at that time, was going for about US$50K! Fortunately, when the machine was powered back up, it started working normally, and did so for quite a few more years until it was finally retired.
Part of the trick is that you install the CPU, motherboard, and chiller such than any condensation/leakage/etc., drips down away from the electronics, not on to them (e.g., install the stuff "upsidedown"). Next, put a drip tray under the motherboard, with an appropriate drain in it to drain any drips out of the system and into a drain.
For those cases where you absolutely cannot use water, the alternative is Fluorinert. However, make very sure that you're sitting down before you see the price.
https://en.wikipedia.org/wiki/Fluorinert
Oh, yeah, you can breathe the stuff, too. ;-)
Dave
Title says it all. Had these now for four years and not a problem, and really the chances of a leak seem remote to me. The hoses connecting the flow and return from the cold water supply under the floor are more like car radiator hoses, and the radiator in the door just looks like a large car radiator. I think the inlet temperature on the water is like 12 Celsius, so there is zero risk of condensation, and for much of the year the compressors on the roof are bypassed, as you can just blow the outside air over the heat exchangers.
We are using ColdLogik racks/doors and my view is that not doing water cooled doors at a minimum is negligent as you are just pissing your employers money away.
I have seen many (and owned several) cars where the radiator hoses just pinged off or deteriorated to the point what it was quite likely if I hadn't checked. A hoseclip between you and disaster is really not what you want. Now, that's on a moving device, but I imagine rack doors aren't designed to never open either.
The biggest problem isn't the theory but the practice. The first idiot that "WD40's" a small section of the hinges will set the seed for the rubber to deteriorate rather rapidly (and catastrophically) in the next few months.
And though you can say "that shouldn't happen", when it does the results are catastrophic, affect a much wider range of kit than just that in the cabinet, and are maybe uninsurable.
Risk = probability (low) x effect (stupendously high).
To say that absence of that is negligent? Read the stories above where water-cooling with all the funds and impetus in the world causes more problems than it ever solves.
And what happens when the inlet water is colder than expected? Oh, dear, you have an untested scenario with possibly unknown consequences.
Dell has taken the IBM approach, use aerospace quality parts rated for ridiculous pressures, and hope that the safety factor will take care of any problems. There is another approach, run the system under vacuum and never worry about leaks again.
https://www.youtube.com/watch?v=552tzND2Xx0
Dell has also have removed the CDU from the picture to save money, but the CDU will last for many generations of servers, and protects your servers from people who might screw up the facility water. Murphy's law will always get you sooner or later, so make sure that when something goes wrong, it does not result in painful downtime and that late night phone call.
Steve
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
