First, servers were deep-fried... now, engineers bring you wet ones We've seen quite a few innovative engineers who have tried to bring down data centre cooling costs, including this mad crowd who dunked theirs in a deep fryer... Now boffins at Leeds University and British startup Iceotope have invented a super cooling liquid that could create a new generation of "wet servers". They say it could …
At an IT fair last year I came across a stand with a fire extinguishing system for server rooms which boasted a 'liquid gas' made by 3M. I dunked my phone in as well, it came out cleaner and 100% functional.
Which was kinda too bad. The vendor there promised me a new Iphone if my cell ( 2 year old HTC hero ) died after being immersed.
Halon is a CFC and has been phased out; currently they supply HFC-227 or HFC-227ea which is a halocarbon (Can also be referred to as FM-200, Solkaflam 227 or MH-227). Apparently it's also used in medical inhalers such as those for asthma.
I went to a BCS event earlier in the week and the speaker was referring to the topic of power use in the data centre. He made the point that the current growth rate in power used is unsustainable and among the options might be to use the liquid cooling instead of HVAC; then use the heat from the liquid to provide heating to offices (or as per his tongue in cheek suggestion, a greenhouse to grow cannabis!)
I think your BCS speaker is slightly out of touch.
Liquid cooling is quite common - posting anonymously as I don't want to say where I work,
however the supercomputers I manage are water cooled using radiator door on the back of the racks.
I know several locations where liquid (water or C02) cooling is used.
>>I think your BCS speaker is slightly out of touch<<
I don't think so. He highlighted that water has been used in the past, but has many potential issues and is now considered "old" technology.
It's still common for cooling amongst supercomputers yes, but he was referring to any size data centre; which could include the single cabinet installation types and any size of server.
The fluid he referred to is a product with high heat conductivity properties, but very low electical conductivity properties. (Hence not water) safer for the systems and the people working on them.
Pure (deionized) water doesn't conduct electricity. Keeping it pure is another problem - for one thing, you have to be careful what metals you use in connectors (https://en.wikipedia.org/wiki/Galvanic_corrosion) and what materials you use in hoses (https://en.wikipedia.org/wiki/Biofilm).
Water has several advantages, though. It is cheap, and its huge heat capacity makes it ideal when you don't merely need to cool something down, but keep it at a constant temperature.
The last two generations of IBM IH supercomputer (the P6-575 and the P7-775) are both water cooled, with the heat being transferred out of the machine room by a customer water installation to the frame with water used as a direct coolant to the CPU and other components inside the system, as well as having a water cooled back door.
The 775's actually boast that there is a good chance that the system has a negative effect on the ambient air temperature (i.e. they cool the air).
This is all done by significant amounts of pipework within the frames, and large redundant and hot-swappable pump/water-to-water heat exchanger units. It's all very clever, and all of the connectors are made to neither leak water nor air when the connections need to me made/unmade.
I believe that some of the earlier Crays used an electrically inert liquid many years ago as well.
AC because I look after some of these systems as well, but you may be able to narrow down where I work from the Top500 list!
...which made almost the exact same demonstration 20+ years ago with a similar nonconductive (proprietary?) liquid.
File this in the flying car category (also featured on that show)... If there is going to be immersive cooling mineral oil is the only solution that has seen real action.
Having played with liquid-cooled PCs I'd have to agree. Whilst it's all fine in theory, and cools very well in practice, it all becomes a bit dicey when you want to move or replace components. Even the flexible hoses don't like being bent new ways when full of fluid. I can see this stuff being used for liquid-cooled racks, but not directly piped into servers.
If you had a PC case which looked roughly like a normal case from outside (although without fans and with cables on top rather than back), actually was a cistern open on top, with a slid-in mounts for motherboard, HDD and other bits, motherboard connectors in upward orientation, and a top cover to pass the cables through (only bottom and sides need to be watertight), I guess it would work. Although with attached heat exchanger it would probably be large, for a single PC.
The new substance doesn't conduct electricity. The main reason for the piping was to prevent electrically-conductive water from touching the electricals. Here, you could put the server in a bath of the stuff and it wouldn't care provided you made sure the fluid didn't stagnate and start spot-boiling (but because the liquid has high thermal conductivity, the heat can spread through the fluid pretty readily).
> Wonder when I'll see it in CustomPC?
Flourinert* is incredibly expensive and 3M only sell it in bulk. I have seen some cost sharing schemes but none went anywhere. I have heard of one or two people lucking out and getting the stuff second hand.
It's not really necessary though; you can use a number of cheap and easily available mineral oils instead. Plus you get great reactions when you rock up to the supermarket till with a case of baby oil and a pair of marigolds...
I thought someone would get there before me!
The Cray-2 used a 3M Fluorinert liquid for cooling. It's a fantastic liquid for the job, but being a CFC it's use was phased out. Although the 3M website doesn't seem to explicitly state it, it looks as if the Novec is a replacement. Look up "3M Novec Engineering Fluids".
The biggest issue is that you have to make the server cases liquid tight, and all the associated plumbing. A leak means the liquid gets out, the air gets in and your servers fry.
Also these liquids are not at all cheap, which is why people still work with water/glycerol (anti-freeze)
Cold climate agree. Hot climate disagree.
Even when you're trying to cool the room, how does "plumbing your PC into the central heating system" help? You're just moving the heat to some other location where you'll eventually have to expend more energy moving it outside.[1] Letting the PC dump its waste heat into its immediate environment reduces the number of necessarily not-entirely-efficient steps in evacuating that heat from the room.
The new cooling liquid is nice, but that remark by Hopton was just dumb. It can be worthwhile moving heat from a server room into other conditioned spaces (when you want to warm the latter) because there's significant excess heat in the server room - enough to justify the energy expended to move it. Unless you have some mighty dissipation from your home electronics, there's no advantage to doing the same there.
[1] Or into the groundwater, if like Jake you're using ground-sourced heat pumps.
Well indeed. So you take the heat from the device that would otherwise be radiated into the room, squish it through a radiator, and, er, radiate it into the room...
I suppose you *could* use the waste heat to heat some other room in the house, but even that is still going to require extra heat where the device is. There's no net gain, I fear.
The big problem was the flourinert was a flourocarbon which came out JUST as the big CFC-ozone connection was made. I'm pretty sure the new 3M substance isn't a flurocarbon or they wouldn't have continued pursuing it.
From my reckoning, they developed a fluid with very high thermal conductivity, so my guess is any part of the fluid that heats up quckly diffuses throughout the fluid. Plus, in the example, it's set up as an intermediary to a water-piped cooling system. My guess is they use water to help regulate the temperature and the new fluid more as a safer heat transfer medium; the water doesn't have to get close to the sensitive electricals.
The biggest problem in computers is a few very hot components. For example, the heat density of a modern CPU chip is actually higher than the Calrod element in your electric stove (i.e., the BTUs emitted per square cm per hour). Air is not a particularly effective medium for pulling that heat away - even water is many times better than air IIRC. And a liquid has more heat capacity - it takes many times more energy to heat it one degree than to heat the air one degree. Among other advantages, because it is so much better at its job, it doesn't have to be circulated at such high speeds, it takes much less volume, and so the energy cost of moving it around is much less.
All of those big power transformers you see on your power utility lines are liquid-filled. In most cases they don't even require pumps - a well-designed convection system circulates the liquid up through the transformer coils and then out and down through vanes on the sides of the 'tank'.
My mental picture when I first started on the article was, there is a tank of this stuff and you just shove the server in the tank then use a heat exchanger from the tank to elsewhere. I suppose though it may slow the hard drive rotations a tad. However, I can see that if a server was totally redesigned to accomodate this stuff, that mental picture of mine may not be so way off beam.
I think you have the right picture - the first one done AFAIK was the Cray II, which sat in a tank of Fluorinert. But as you alluded, hard drives must have to be handled differently. I think all 'normal' hard drives have a way for equalizing air pressure inside and outside - the heads 'fly' on a microscopic layer of air. So the hard drives would have to be designed to live 'underwater'. If they were, they would be one of the four components that would most benefit from liquid cooling - the CPU, GPU, power supply and hard drives generate most of the heat.
Rather than fanny about with fancy new liquids that in all honesty won't work in the real world all that well, how about transferring the excess server heat to a network of underground pipes that deliver the hot water to homes and businesses (just like the gas network) to use for heating?
The reduction in energy usage from the homes not needing to heat themselves should offset the energy required by the datacentres by a fair amount and the datacentre provider could make some cash from a waste product.
Rather than fanny about with fancy new liquids that in all honesty won't work in the real world all that well,
Citation needed.
how about transferring the excess server heat to a network of underground pipes that deliver the hot water to homes and businesses (just like the gas network) to use for heating?
And in today's news, Reg commentator reinvents the wheel.
Utilities have been delivering heat, in the form of pressurized steam, since at least the early 20th century. There's no point in delivering it as hot liquid water - even with insulated pipes you wouldn't get significant heat any distance. It's often a waste product from electricity generation.
It's only viable for short distances, and is often a money-losing proposition these days, particularly in the US with our dirt-cheap natural gas market. (That's because we have a market, and not a bunch of insane long-term contracts linked to the price of a competing commodity.) High-pressure steam plumbing requires a huge amount of maintenance, in part because water is a small, polar molecule that wreaks havoc on many substrates and likes to find its way through small gaps.
Plenty of organizations use waste heat from data centers to heat other rooms in the same building, or other nearby buildings.
So where is the justification that this saves 97% of data centre cooling costs? I suppose you might (just) make a tenuous case that it saves 97% of the associated energy costs, but given this solution appears to require special servers, storage devices and comms equipment (all of which produce lots of heat) into this thermally conductive equipment and a whole lot of new infrastructure, I would think the up-front infrastructure costs are going to be very significant. Even if it can use simple heat exchangers in the outside environment, they are still going to have to be very large as, ultimately, unless you have a convenient nearby water source, it will still ultimately depend on how fast heat can be transferred to the air.
Then there is all the practical stuff about being able to move and upgrade equipment without this liquid leaking out everywhere. It's possible to see how this might work with something large and fixed (like and old-fashioned mainframe, which used to be liquid cooled in the past), but rather problematic with smaller stuff. Maybe somebody can design a blade server of some sort with tight thermal coupling to this sort of liquid-cooled infrastructure, but it is not going to be easy.
I rather think that a far more important approach is to improve the energy efficiency of IT infrastructure as that reduces total energy costs, not just those of cooling.
The article talks about many of your doubts. Using air cooling, there are two very big costs - it is necessary to move a lot of air through, so the fans use a lot of energy (which also adds more heat). And in order to be as effective as possible, the air is usually air conditioned or chilled, which increases the energy used by another 50% or so.
A liquid coolant is thousands of times more effective at conducting heat away from components, so such as system only needs to move small amounts of liquid. Then, once the liquid is moved to the area where the heat can be transferred out of the system (fins, plates, whatever), the surface area of the heat exchanger can be much larger so again no fans are needed.
In the transformers hanging on poles outside your house, a pure convective system (no pumps) circulates liquid coolant up through the copper coils then down through the fins or tubes on the outside of the transformer, letting the heat itself do the work of circulation.
As you say, server farms are going to be the place where this is most cost-effective, but with the amount of wasted space inside a 'tower' for example (much of which is due to the need to move vast quantities of air around quickly), I think a tower or desktop designed for this, perhaps with fins on the back and/or top, might well be the same overall size as your existing machine. And with the convective flow, you would lose the noise and power loss of the fan(s).
I have occasionally wondered why laptop makers don't either put the CPU/GPU behind the screen, or run a heat pipe from the CPU up through the hinges to a radiator on the back of the screen.
Yes, it conducts heat better than air (and presumably better than water, which itself is 25,000 times better than air), the problem is dumping the heat afterwards.
Heating from a solid (the CPU, GPU or whatever) to a liquid or gas is fairly trivial, but you the need to move that now heated coolant away from the solid and cool it back down, otherwise it's pointless. Usually we do this with radiators of some kind, which usually need airflow over them to remove the heat.
In some instances, this will be a trivial concern - the mentioned submarines are surrounded by water much colder than the internal temperature, dumping the heat from the server into the submarine itself and using the fact it's immersed in colder water to balance the internal heat would seem sensible.
However, in hot climates, you need another method of heat exchange. This would be why most large data centres (where possible) are being built in colder climates - when the external ambient temperature is low enough, expensive cooling solutions become somewhat moot.
I suspect the energy savings come from the fact you don't need fans and their associated motors and heat generation. You also reduce the need for air-conning a server room because you can concentrate the heat exchange.
As for the fluid itself, if it has a high enough thermal conductivity, wouldn't its natural tendency be to diffuse the heat to nearby parts of the liquid, which would in turn diffuse to other parts of the liqud? Air is a poor conductor, so you need to constantly keep airflow to keep absorbing heat, but this liquid is an excellent conductor, so perhaps instead of a constant blast you just maintain a gentle flow (think less hurricane and more soft breeze), which you can build into the design so that if the fluid flows a certain way, it gently washes over the CPU, GPU, and so on and out the other side to be recirculated.
While I suspect that there's more to this story I'm surprised at all the negative responses. Sure it wouldn't necessarily be ideal for a situation in which you have to hotswap hardware (as fishing things in and out of the liquid would be a messy affair) but it would have it's applications would it not? Wouldn't a blade server farm with little to no serviceable parts in it be fine?
While it's applications in a home would have to be ironed out I don't see them as insurmountable. Personally, I love the idea of pumping the heat away via pipes as I live in a desert and blowing all that heat into the room proves impossible for my A/C to combat.
For a long time it's offended me that most datacentres are busy throwing away heat to the air outside, while the same building is burning fuel to keep its inside warm for the people that work there.
In fact one can source airconditioning plant that pumps heat from the air in the server farm "uphill" into the building's central heating system. However such plant has a higher capital cost, and the IT people and the estates people and the finance people don't properly talk to each other and argue over "whose budget" rather than seeing the "interest" on the capital invested (arriving by way of a reduced fuel bill). So it's rare to find such a system. Personally I'd say legislation or tax breaks should make such airconditioning either compulsory or highly tax-advantaged.
CPUs are happy running at 80C so they could be "cooled" by water from a central heating system. Trouble is that water and electricity don't mix. The tiniest leak from the CPU cooling circuit can trash the motherboard. Cue a nonconductive inert liquid which could be pumped through heatsinks above the CPUs and then through a heat-exchanger plumbed into the central heating system, with no aircon (pumping of heat uphill) required.
Looking further ahead our CPUs could be happily running at 100C or even 120C if they were designed and tested to do so. Nothing in the physics says it can't be done. They'd slow down in proportion to the increase in absolute temperature: 60C to 120C is 333K to 393K, so that's about 20% slower. In most environments a couple of extra cores on the chip would be adequate compensation for the lost MHz.
Well to be fair it is increasingly common for large industrial projects (and datacentres are just one of them) to replace explicit outdoor radiation with a more indirect route, generally involving free heating of the swimming facility next door. If they need greater radiation, they make it an outdoor pool.
Likewise there are many places that dump waste heat into centrally heating adjacent buildings - one small datacentre I know of heats the school next door.
You often may not be able to divert more than 2/3 of the energy this way, but it tends to be a mutually beneficial system - one side gets cheap heating, the other a tax writeoff.
Thinking about this, if it's as efficient as they say, it's possible that piping could be reduced to 1/4" or even 1/8" flexible tube onto a small thermal block. With quick release fittings onto the case and mated onto an external manifold and cooling is done via one mother of a radiator plant outside the building ... it could be interesting.. My old water cooled system with 1/2" pvc pipes, plastic elbows and pond pump driving into a fan-cooled Seat Ibiza heater matrix now starts to look as if it needs a makeover ...
It was a CFC, developed IIRC as a blood plasma substitute (yes this has been used as a plot device in at least one novel).
IIRC the space Shuttle fuel cell coolant was also a 3m product originally developed as a safer (less flammable) alternative to Carbon Tetrachloride and Chloroform. Sadly it was a CFC.
Novec 1230 (which I guess is what they are using) is described as a "Fluoroketone" and used as a Halon replacement as a fire suppressant. So a bit of Flourine but no Chlorine.
Thumbs up for clever thinking. Given others have suggested it before what's their Special Sauce (C Lewis Page)?
...for the first modder to refill their water cooling rig with this and discover that it eats the pump seals or somet' like that.
That said, if it turns out to have good materials compatibility it'll probably be the next big thing in overclocking. Easier than using liquid N, anyway.
The simplest way of avoiding "eating the pump seals" is to use magnetic couplers. They're not difficult to make,. especially in this age of high power magnets. Fishtank filters have used them for decades.
Given what happens if a cooling rig leaks "mere" water, it makes sense to look at this anyway.
Plugging my Computer into the central heating? What should that be good for?
In Winter (when I use the central heating), my computer heats my room just fine. Every Watt of electricity that goes in comes out again as heat plus some shifted bits in my SSD. If that heating isn't sufficient, my room thermostat kicks in hot water from the central heating, so presto, problem already solved.
So why would I reroute the computer heat through the central heating and then into my room through hot water, when the direct path works just fine?
Computer != PC. The article mentions servers, and the small setup in the video is just a demo. Hint: connecting to a pipe system allows you to store and transport the heat further than your server room (or bedroom in your case). Large scale examples of using heat from servers exist..
Liquid cooling is not new(s) on El Reg either, see the following link. They are using the same method to immerse the servers in a coolant but this is based on Mineral Oil. Alot less money, not proprietary and little to no regulations on it. Anything liquid has a much higher thermal mass than gases and mineral oil is probably less likely to vaporize, flash or cavitate than a fluoroketone would.
http://www.theregister.co.uk/2012/05/25/immersive_cooling/
Because if you air-cool into a server room, you have to then cool the air and pump the heat outside, or use some sodding great air-exchange setup (with associated dust control and maintenance)
Both stages involve shifting vast amounts of air with fans and as a result server rooms can easily sit at 85-90dBA (loud enough to not hear equipment alarms unless you have your ear next to them.
Liquids are a lot easier to shift around in small spaces. That's why UK houses use plumbing and radiators instead of american-style air ducts.
I'm in the middle of scoping a project to upgrade our cooling capacity from 60kW to something more usable and the variety of options being offered are bewildering, to say the least. When a single 4U GPU box can be dissipating 4kW there's little room for messing up.
Proper mil/aero grade boards use conduction cooling to get heat off into the mounting fast enough that even multiple of the latest version, Intel CPU's and Nvidia processors can co-exist on 6U boards without problems or fans! In a marine environment, with careful planning of the metalic interfaces, passive cooling to the water is not a problem - no moving parts = no noise to transfer.
Now put multiple of the same boards (with different metalwork) in a radar type solution in an aircraft sitting in the equatorial sun and spray cooling is the choice.
Hmmm... use a board that's designed to get rid of the heat, or pay global corp a similar amount for the liquid to cool a normal server board?
...the more they stay the same.
When I was a postgrad at Leeds in the early 90s, they did the usual tour of the campus, and one thing that stuck in my mind was the huge shallow square pond next to the Roger Stevens building. They explained that it was used in the 60s and 70s as cooling water for the powerful computer(s). Imagine that, liquid cooling computers. This is the one (looking somewhat nicer than it did when I was a student).
http://www.flickr.com/photos/27550874@N03/3716494602/
There must be more to this item than meets the eye, as I have seen liquid cooled PCs for years and years. Presumably they are going with the revolutionary "using waste heat to heat a building" concept. In domestic terms it would be far better to use the technology to pump heat from refrigerators and freezers to pre-heat water as many commercial cooling systems do.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
