Brain-inspired neuromorphic computer SpiNNaker overheated when coolers lost their chill The brain-inspired SpiNNaker machine at Manchester University in England suffered an overheating incident over the Easter weekend that will send a chill down the spines of datacenter administrators. brain Brain-inspired chips promise ultra-efficient AI, so why aren't they everywhere? READ MORE According to Professor Steve …
Oh, be fair: it was two British Bank Holidays in a row, perfectly reasonable to expect cold weather, high winds and rain. Snow at Easter isn't unknown. They probably had emergency temperature control plans: an undergraduate poised to run in with a space-heater
Yeah, and lucky this didn't happen a week later during the (could've been) end of times UK heatwave apocalypse and resultant Iberian peninsula synch fail blackout armageddon! Especially if this mouse-brain neuromorph had been a GPU-driven computational fusion energy megaspace heater of doom, rather than power-sipping spike machinery ...
Safe to say we've averted a major "New-Zealand China Syndrome" on this one, with Manchester core meltdown shooting cataclysmic jets of devastation, all over Wellington NZ. Missed it by that much!
I hope they get the remaining 20% of the Muridae-brain beastmachine back up soon, but things could have been just so much worse imho ...
Conventional heat tolerant electronics starts malfunctioning at 90 to 100 C but immediate damage doesn't happen until it's much hotter. Since the cooling fans were were running, it's surprising that all that heat didn't trigger fire suppression.
Or maybe no fire suppression either?
"electronics starts malfunctioning at 90 to 100 C"
It depends where are you referencing the temperature.
It's usually the semiconductor temperature that is more important/relevant to device life.
In a previous employment we had semiconductor packaging sitting on a 60C heater plate, but with intended semiconductor temp well in excess of 200C with a long MTBF.
Since we could not wait that long to measure MTBF, we ran them at higher temperatures, well in excess of 300 C where they lasted tens of hours.
But that was a slightly speicialist operation.....
Commercial grade devices should operate between 0 and 70 for a long time. Multiple years.
When the junction temperature goes above 100 c the device moves through the lifetime bathtub. Performance to spec will be poor (fails). High temperature canork for a thousand hours (HTOL) the arrhenius equation will give an estimate of the lifetime.
Device functionality is not guaranteed above 70.
I just test the devices not qualify them.
Every server room I've worked in has been fitted (either originally or at my insistence) with a high temp room crowbar, usually set around 40C
It's simply a thermostat linked to the emergency stop button
Hard shutting down the power is preferable to cooking the hardware and 40C room temp is usually 70-80 at the actual semiconductor junctions. AC is one of the least reliable things in a computer centre and making sure you can prevent destruction of potentially millions of pounds worth of hardware (and more importantly, data) is fairly important. Back in the old days an overheating room would regularly result in disk drive head crashes
Don't worry, Steve told me (he was getting emails from the temperature sensors) and I shut off the boards and the servers... Unfortunately, the biggest contributor to heating the room is the fans on the Chillers, so when they are not chilling, they are heating. Ideally the chillers would switch off when the water temperature is too high, but they don't. So then someone else went in an shut the power off manually.
