back to article Facebook's first data center DRENCHED by ACTUAL CLOUD

Facebook's first data center ran into problems of a distinctly ironic nature when a literal cloud formed in the IT room and started to rain on servers. Though Facebook has previously hinted at this via references to a "humidity event" within its first data center in Prineville, Oregon, the social network's infrastructure king …


This topic is closed for new posts.
  1. Wallyb132

    That is fucking hilarious... Couldnt happen to a better bunch of people...

    1. LarsG



  2. Suburban Inmate


    That'll explain the chat bar being out of action and everyone bitching about it, but isn't one benefit of a "cloud" that it can withstand the loss of service at one location and be able to at least limp along with other sites until things are straightened out?

    Admittedly, this is probably peak time of day for their servers and don't know how many locations FB has.

    1. Russ Tarbox

      Re: SPOF?

      Check the date of the story. Tonight's chat issues are nothing to do with this :)

  3. Henry Wertz 1 Gold badge

    Couldn't post...

    "But we do wish it would happen again, just so they could snap a picture"

    They probably took photos, but couldn't get them onto facebook

  4. frank ly

    I thought they'd have used heat exchangers

    But no, they draw in outside air. I assume they filter it for dust but that won't stop water vapour. Maybe now, they'll have humidity sensors and do some calculatons to give warnings, etc.

  5. rhydian

    Raindrops keep falling on my head...

    On a related note many very tall/large buildings suffer from the same problem. It wasn't unheard of for certain transmission masts to have rain inside their centre tube as the top of the mast was much colder than the bottom, and Boeing's massive construction building suffers from indoor rain on occasion.

    1. Destroy All Monsters Silver badge

      Re: Raindrops keep falling on my head...

      Also expected for The Third Reich's SUPERDOME planned for the city of Germania. Not sure what Speer intended to do about it. It would have given a suitable Wagnerian Atmosphere like in video games.

      1. Neil Barnes Silver badge

        Re: Raindrops keep falling on my head...

        Doesn't it rain inside NASA's vertical assembly building? Or is that just urban legend?

  6. Jemma

    Thank you, Westboro..

    God hates Zucks...

    1. dshan

      Re: Thank you, Westboro..

      Yes, I do.

      -- God

  7. Anonymous Coward
    Anonymous Coward

    Hating Zuck, FB and everything it stands for is a no brainer, but hats off to the back-room boys and girls who design, build and maintain these massive data centres.

    1. Number6


      I'm not taking my hat off until the rain stops.

  8. Dodgy Geezer Silver badge

    But they do have... Environmental prize for saving energy. So that's all right then...

    Can't we have an enviro fascists icon? Please? We're going to need one soon...

    1. dshan
      Thumb Down

      Re: But they do have...

      First you need to look up fascism, learn what it is and comprehend why environmentalism is nothing like it.

  9. Destroy All Monsters Silver badge

    That's a pretty cool story!

    I have just had failure of the cooling unit so far, resulting in cabling so hot that it seemed to be outgassing monomers...

  10. Phil O'Sophical Silver badge

    Deja vu?

    IT systems, leaks, rubber raincoats? It's probably Julian Assange's fault ...

    1. Destroy All Monsters Silver badge

      Re: Deja vu?

      It's not like NSAbook isn't leaking stuff through its backdoor like it's having the runs anyway...

  11. Anonymous Coward
    Anonymous Coward

    No storm in my living room

    But, with an Aircon machine at one end, pumping out cold, dry air, and a door at the other, through which hot, humid air can be let in, I'm sure it ought to be possible to create a tropical storm in the middle. This story gives me renewed hope!

  12. Gary F

    No emergency aircon?

    A well designed data centre should be able to cope with heat, fire, power outage and excessive humidity and moisture. Surely in the type of data centres that Facebook designs they'd have moisture / humidity detectors, alarms, and emergency aircon that kicks in when the ideal environment is no longer sustainable.

    Clearly there is a risk of this problem occurring but there's no protection against it - apart from rainjackets! That doesn't protect everything or avoid the risk of people getting electrocuted. I'm sure they can afford whatever it costs to install emergency aircon and powerful fans to help avoid expensive and embarrassing future outburst of indoor rain.

    1. Random K

      Re: No emergency aircon?

      I wouldn't be so sure that an HVAC system suitable for such a data-center would be cheaper than simply replacing a few hundred/thousand/whatever servers every now and then when this sort of thing happens. If a (probably very cheap) rubber gasket can reduce hardware failures by a decent percentage and they install a warning system so they can move the compute load off-site (and temporarily shut down the facility), then such an HVAC system wouldn't sense.

  13. silent_count

    My two quid says...

    it was the BOFH "testing" the sprinkler system.

  14. mike acker


    you circulate the chilled water to heat exchangers inside the buiding

    unless you want to take a bath

    1. Will Godfrey Silver badge

      Exactly! It's not rocket science. It's more like GCSE level physics.

  15. The Alpha Klutz


    i bet that cost them haha putting high end servers in the bin because of their own stupidity. its no wonder zuckerburg cant afford a suit

  16. Zoltan

    Indoors clouds as art?

    I thought indoors clouds is a form of art:

  17. jonfr

    Next time

    Next time Facebook is going to face thunderclouds and flood rains in their computer centers. That is going to be interesting once it happens.

  18. Anonymous Coward
    Anonymous Coward

    I'm sure I've read something about this kind of things before... Maybe it was just theoretical?

    1. gzuckier

      "I'm sure I've read something about this kind of things before.."

      The Great Baker House Snow Hack?

      Something tells me this story has about as much reality behind it. Or maybe somebody could explain how taking hot dry air, running it through water so that it become cool (relatively) wet air, then mixing it with more hot dry air will result in condensation? Getting perilously close to perpetual motion there.

      I suppose somebody will end up tying this to ":and therefore we see that climate change may not be manmade":

  19. P Taylor

    Fail of the Epic kind.


    That is all... Cannot say anything else, my sides are hurting too much !.

  20. Anonymous Coward
    Anonymous Coward

    --and the other problem..

    If they don't keep the relative humidity/dewpoint down below 50% or so, they will eventually get mold growing on anything tucked away...which will cause the next health emergency in the name of greenness!

This topic is closed for new posts.

Other stories you might like