back to article Twitter datacenter melted down in Labor Day heat

Earlier this month extreme heat downed a Twitter datacenter in California over the Labor Day weekend, leaving the website and app working on bare-bones infrastructure. According to a memo obtained by CNN, Twitter lost access to its Sacramento (SMF) datacenter "due to extreme weather" on Monday, September 5, 2022. The memo, …

  1. Paul Crawford Silver badge

    "If we lose one of those remaining datacenters, we may not be able to serve traffic to all Twitter's users," Fernandez warned.

    Oh dear. How sad. Never mind.

    1. heyrick Silver badge

      To be said tomorrow...

      "Ah but look, since the infrastructure is clearly wobbly, I can completely legitimately walk away from the promise I made to purchase the entire company."

      1. NoneSuch Silver badge
        Pint

        Re: To be said tomorrow...

        Thank God the backup DC's kicked in. I would've been forced to actually talk to someone otherwise.

  2. Terry 6 Silver badge

    Sacramento gets hot at the best of times. So forty odd degrees doesn't seem all that improbable from time to time, even had we not got climate change to factor in. And the UK had 40+ temperatures a few weeks back ( hard as it is to think so now).

    So not being able to function at 45 degrees in Sacramento seems sadly like very poor planning.

    1. Nick Ryan Silver badge

      True, it does feel like very poor planning. When the average temperature for July/August is 35 degrees, with regular variances +/- 5 degrees, not being able to cope with twice the regular variance is quite poor.

      1. Someone Else Silver badge

        When the average temperature for July/August is 35 degrees, with regular variances +/- 5 degrees, not being able to cope with twice the regular variance is quite poor right in line with standard American business practice.

        There, FTFY.

        After all, gotta keep those coke-snorting yuppies on Wall St. contented...

  3. Inventor of the Marmite Laser

    "If we lose one of those remaining datacenters, we may not be able to serve traffic to all Twitter's users,"

    My heart bleeds for Twatter. Positively BLEEDS.

    Not really it doesn't. It'll just stuff those companies whose customer service is so shite that they rely on (anti)social media rather than phone(not premium rate) and email (not web forms).

  4. Nick Ryan Silver badge
    Flame

    Standards...

    The international standard for temperature is Celsius. "115F" is meaningless to most individuals on the planet and is becoming steadily more meaningless.

    Please use international standards by preference at all times but if you feel it necessary to also provide the same temperature in out-dated Imperial measurements to aid readers in the three countries on the planet that still use Fahrenheit, then do so as an extra level of detail afterwards.

    Or, at the worst, use the correct Register Units, here (FTFY):

    Bitbarn suffered 'total shutdown' after 2.5 Hilton heatwave

    1. Dan 55 Silver badge

      Re: Standards...

      Credit where it's due, they're getting better, the only place where there isn't metric is the sub heading. Unless I missed an earlier version which didn't include SI/NR units.

      1. Nick Ryan Silver badge

        Re: Standards...

        True, but it's the sub-heading that catches attention!

  5. skeptical i

    such a waste

    All the leccy being used at Twaddle's three datacenters (to run the servers, the cooling, the lights, the coffee machine, &c) while other parts of California were told not to use "too much" juice in order to mitigate the chance of brownouts. Perhaps shutting down this not-essential nonsense should be the first step, before asking Grandma to not turn on the aircon until after "peak hours".

  6. Nate Amsden Silver badge

    Atlanta

    As a QTS Atlanta customer I heard on the grapevine years ago(perhaps as much as 8+ years ago) that Twitter has/had a large presence in QTS Atlanta, which is the facility I have used for a decade. If they are lucky they are still in that facility. It's spit up into many sections I never saw their equipment.

    Fantastic facility and staff. I don't think there's much chance of it ever going down. It's relatively new too, there was still a lot of construction going on when I first got there, I recall walking through a huge open area with tons of construction equipment to get to the data center, and it was several years later that they built out the official front entrance area in another part of the building. And then on top of that they added another 250k sq foot data center on the same site? So I think ~750k sq feet raised floor between the two. Just massive.

    Am assuming the Twitter Sacramento facility was perhaps dedicated to them, since I haven't noticed(not that I have looked hard) of reports or articles indicating an outage affecting other companies in that area(I live 1 hour from Sacramento).

  7. Henry Wertz 1 Gold badge

    Partial shutdown?

    Wouldn't you think a partial shutdown would be possible? Like, shut off half the servers and your cooling requirements are cut in half. *shrug*

  8. Sampler

    How long...

    ..until Musk uses this as a reason to not complete his purchase of Twitter...

  9. Ball boy
    Coat

    Twitter Twatted by Top Temp

    I'll get my coat. Not that I need it in the heat...

  10. Anonymous South African Coward Silver badge
    Unhappy

    Pity...

    ...it did not wnet full nuclear on the meltdown.

    Less twatter will make the world a better place.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like