Google Cloud's watery Parisian outage enters third week, with no end in sight

Two weeks after the unwelcome "water intrusion" inside a Parisian Google Cloud datacenter, the stricken facility remains offline – with no indication when it might resume operations. Google Cloud's europe-west9 region took a shower on April 25. As datacenters and water don't mix, outages resulted. It later emerged that the …

[Anonymous Coward]

Cloud is it

Yeah, as I've been saying, no its not.

"Google Cloud products in europe-west9-c may not be available to customers."

So why aren't they moved to a different region? Is it because you have to pay more for that?

No matter how much our MSP pressures us to go full cloud I don't think its a good idea. But no one listens to me.

[emfiliane]

Re: Cloud is it

Strip away all the hype, and the value of the cloud is betting that renting your equipment and the ability to scale up and down in five minutes is cheaper than buying it all outright plus provisioning and running costs. For greenfield projects, or a major upgrade, it's worth weighing the different paths.

But for some reason management always wants to replace the datacenter wholesale, despite all the sunk costs that have already gone into it.

[Anonymous Coward]

Re: Cloud is it

Another commentard posted a couple of years ago:

-------

cloud

/klaʊd/

noun: cloud; plural noun: clouds

1. a visible grey or white mass of condensed watery vapour floating in the atmosphere, typically high above the general level of the ground.

The day IT lost sight of what the word really meant was the day common sense started to slide down the hill and beancounters took over.

Being beancounters, they ignore the fact that the ownership* of their company's data is inversely proportional to the amount of cloud services they buy in order to save money.

* the actual cost of getting your data back home once you realise that cloud services were more expensive than what you originaly thought.

Because, no one ever made a case for estimating just how much you'd had to put down to get out of the cloud.

---

This idea still stands.

So ...

Two weeks after the incident, the data is nowhere to be seen and Google does not seem to give a rat's toss.

1. What are Google Cloud's customers going to say to whoever they provide services to?

2. How will they spin to their shareholders what a great idea it was to rely on unstable water vapour to hold their priceless data?

.

[Anonymous Coward]

Re: Cloud is it

But you always have SLAs. Companies and MSPs that support you are strict with their SLA. But we have an urgent issue "Doesn't matter, we stick to our SLAs, your one company of many we support". And there's the problem. Outsource the kit, outsource the support and you get that bollocks. At least with me still on site I can say "We have SLAs but I can see and know its urgent with that room full of people waiting. I'll fix it now but raise a ticket after".

[Anonymous Coward]

Re: Cloud is it

So why aren't they moved to a different region?

Because that's the job of the customer, just like it's their job, when using on-prem, to have a secondary site as backup.

If your workload is critical, you deploy it to multiple availability zones, or even multiple regions, so you don't have to worry if 1 AZ is down.

If you think that because it's "in the cloud", your provider will automagically take care of everything, then you haven't understood the model.

[Anonymous Coward]

Re: Cloud is it

But so many managers don't. They still think if its "the cloud" it will be always available. I've explained this, but again, not listened to.

[v13]

Re: Cloud is it

Regional or zonal services don't auto-migrate, for good reasons. That's why they're regional or zonal. Global services and other regions work perfectly well.

[Dr Who]

A fire incident has occurred

No it hasn't. You're just trying to sound official or technical or something. What has occurred is a fire, not a "fire incident". Just like it's not a "flood event" it's a flood. And when did we move from having a storm to having a "severe weather event"? Anyway, gotta go, I had a curry last night and can feel a catastrophic evacuation event coming on.

[abend0c4]

Re: A fire incident has occurred

I was more struck by "attempts at extinguishment ". At first I thought it might be an attempt to reproduce French grammar, but it seems more likely there's been a style outage in the post-authoring amendment facility...

[heyrick]

Re: A fire incident has occurred

Might be the creeping Americanisms? Just wait for "deplane" and "burglarize" to turn up, to really mangle the language.

[katrinab]

Re: A fire incident has occurred

No, it was "an unscheduled rapid oxidation event"

[herman]

Yumky

Pardon me, but I’m not going to share a scone with a bird.

[Anonymous Coward]

And the benefits of relying on someone else's computer the cloud are what again?

[Vader]

The cloud should never go down right ?

[Potemkine!]

Heat management

How is water-cooling going?

[Missing Semicolon]

"Google Cloud products in europe-west9-c may not be available to customers."

An announcement that was not followed by "so we have redeployed all of the workloads to europe-west9-q (a bunch of containers in a car-park} so that service is maintained". Odd that. Almost as if they don't have to give a s**t.

[Fred Flintstone]

Wait, let me get this right..

A cloud gets into a cloud data centre and it all falls apart?

:)

[Anonymous Coward]

Re: Wait, let me get this right..

"A cloud gets into a cloud data centre and it all falls apart?"

Strictly, it should be:

A cloud gets into a cloud data centre and it 'precipitated' ..... a continuity failure !!!

:)

[Claptrap314]

First rule of SRE: the minimum number is THREE

One for the customer, who keeps us well paid,

One for maintenance, whose plans are well laid,

One for the moon, the stars, and the sea,

Whose shifting waters none can foresee.

Seriously. Outages like this happen. Google has given users the tools for reliability, but you need an education to use them properly. If you are not in three geographically, electrically, and chronologically (as to maintenance schedules) isolated data centers, with each capable of immediately taking on 100% of your workload, you on NOT HA. And that is on you, not them.

(And, assuming your workload is big enough, you can get by with 50% or less cost increase instead of 200%.)

[Doogie Howser MD]

Re: First rule of SRE: the minimum number is THREE

You're wasting your time with the people on here. It's usually a binary argument of cloud == expensive and bad. The discussion is far more nuanced and broader than that, but it's all commentards seem to care about these days, so this website keeps pumping out the content to wind people up, maintain engagement and sell ads.

The inevitable deluge of down votes will serve to illustrate my point!