Google Cloud burst by 12-hour power outage in German region

Google Cloud apologized on Thursday after its europe-west3 region – located in Frankfurt, Germany – experienced an outage lasting half a day. The incident began at 02:30 local time on Thursday, October 24 and ended at 15:09 – a total of 12 hours and 39 minutes. "We apologize for the inconvenience this service disruption/ …

[I should coco]

99.99 errr NEIN!

What happened to Cloud being always there persistent and available? One little power failure in one part of the building and services are unavailable, massive design failure.

Not so much about availability more offering the minimum infrastructure to maximise every penny of profit.

Another great advert for data repatriation into your own DC.

[Drakon]

Re: 99.99 errr NEIN!

> What happened to Cloud being always there persistent and available?

Who said this was ever the case?

[Oneman2Many]

Re: 99.99 errr NEIN!

"What happened to Cloud being always there persistent and available?"

You realise that "the cloud " is more than one building ?

If you don't want to pay for multiple zones and architect your application to take advantage of that then cloud is no more resilient than a single datacentre on prem solution.

[Drakon]

Re: 99.99 errr NEIN!

In this case, it seems like the loss of a single zone caused a knock-on effect in the two other zones.

If you were just running VM's (as you would in your own datacenter) you would've only been affected if your VM's were in that zone (as you would if your self-hosted datacenter had these issues).

The issue I'm seeing here is that some of GCP's managed services weren't resilient to the loss of a single zone, that's a concern.

[David Hicklin]

Re: 99.99 errr NEIN!

>> What happened to Cloud being always there persistent and available?

Problem is that it was originally marketed as " always there and resilient" and the perception has stuck, and where the caveats and warnings were given they were behind the equivalent of a locked door with a "beware of the leopard" warning sign.

[Pascal Monett]

12 hours and 39 minutes

Okay, and that impacted how many companies and how many clients of those companies ?

It is high time that we get over this cloud malarky and get back to on-site IT where, if one company has an outage, it only impacts that company and its customers.

Until you can build an infrastructure that can resist little things like power outages, it's not worth it.

[John Robson]

Re: 12 hours and 39 minutes

You're making quite a few assumptions there.

[Headley_Grange]

Re: 12 hours and 39 minutes

"It is high time that we get over this cloud malarky"

It's not really high time for anything. As has always been the case, companies need to do full lifecycle costings for on-prem vs cloud based on their business needs and decided which one is best for them. I think that we've passed peak cloud-as-the-latest-must have and the real costs and performance of cloud are starting to be understood.

[Oneman2Many]

Re: 12 hours and 39 minutes

"It is high time that we get over this cloud malarky and get back to on-site IT where, "

Do you honestly think on-prem is any better managed than cloud and that apps are properly architected ?

[Gene Cash]

Google engineers implemented a fix

They booted the janitor out, unplugged his vacuum, and plugged the servers back in?

[Coastal cutie]

Re: Google engineers implemented a fix

Or finally got around to paying the electricity bill?

[An_Old_Dog]

Is It Possible ...?

Writing as someone who has created no cloud-based systems, is it possible to create a multi-zone, multi-region, multi-whatever cloud-based system which continues to work properly despite such failures?

[avakum.zahov]

Re: Is It Possible ...?

Yes, it is possible, but it is quite expensive and a limited number of people have the brain power (NI) to design it and implement it.

[pdh]

Re: Is It Possible ...?

To be fair, it's also true that only a limited number of people have the brainpower to build a set of on-prem systems that meet those requirements, and doing so is also pretty expensive.

[Headley_Grange]

Re: Is It Possible ...?

It's more likely lack of money than lack of brains that gets you poor system performance wherever your service is based. There's a gulf between cheaper and cheap.

[Anonymous Coward]

Re: Is It Possible ...?

>> To be fair, it's also true that only a limited number of people have the brainpower to build a set of on-prem systems that meet those requirements, and doing so is also pretty expensive.

Really? The server and data centre teams at our place do a pretty good job at it, if a DC goes down the stuff pops up on the other one (its all virtualised or mirrored), even do annual DR tests to prove it works. Essential as a app user you have "your" server and those 2 teams look after the OS and Infrastructure for you.

OK a problem if both DC's are blitzed but to get to that level of destruction the surrounding areas have much bigger problems!

[Tron]

Re: Is It Possible ...?

Yes. You could even set up a crowd-owned global system - anyone who offers space on a server gets a piece of the pie.

BUT, it would make you a TERRORIST and an ENEMY OF THE PEOPLE. Because it wouldn't be a sovereign cloud with sovereign data that your government could upskirt at will. Government logic decrees that encrypted data can easily be seen by enemies (China, Russia, France) the moment it crosses the DMZ. Instead you should keep your data safely within the tribe, with a back door for tribal leaders to see it. Thus protecting you from both nasty foreigners and from yourselves.

How did German schools do in the outage? If UK schools lose their tech, they have to close, as 'safeguarding' requires always-on tech. How did we survive the 70s and 80s without WiFi-based safeguarding. Was it just a miracle?

[Drakon]

Re: Is It Possible ...?

> Yes. You could even set up a crowd-owned global system - anyone who offers space on a server gets a piece of the pie.

I'm pretty sure this has been done. Unfortunately it involves cryptocurrency for some reason.

[Headley_Grange]

Re: Is It Possible ...?

Napster?

[Drakon]

Re: Is It Possible ...?

No, I don’t remember the name and I doubt it lasted long

[Stu J]

Re: Is It Possible ...?

"How did we survive the 70s and 80s without WiFi-based safeguarding. Was it just a miracle?"

In case you hadn't noticed, nonces took full advantage of the lack of safeguarding in the 70s and 80s (and earlier too no doubt), that's the whole reason WHY safeguarding exists, you dope...

[Anonymous Coward]

WiFi-Based Safeguarding

I presume you're writing about wifi/Internet-connected cameras in schools.

I don't think a video camera has ever stopped a single bullet, or even fist or foot, from striking a victim. The fist/foot-using bullies know where the cameras are, and how to avoid them. The bullet-spraying psychopaths simply don't care.

[An_Old_Dog]

Crowd-Owned Global System

How, precisely, are users of such a system supposed to keep their private data, private?!

Any owner of a computer which is part of your proposed global system could syphon off users' unencrypted data living on that computer, since that data needs to be in unencrypted form before the user could process it -- said processing taking place on computers whose hardware and operating systems are not under direct control of the user.

[Smartypantz]

Re: Is It Possible ...?

Off course not what you are asking is "is there a perfect system that never fails if i throw enough money at it?" -- NO!

Monocultures are ripe for pandemics. This principle is universal. Cloud services by their nature are monocultures. therefore they are a global disasters waiting to happen.

Is my custom setup on-prem safer?

Yes! If you know what you are doing!

[Drakon]

Re: Is It Possible ...?

A cloud setup is also safe if you know what you’re doing

[Headley_Grange]

Re: Is It Possible ...?

One of the advantages of Cloud is that the service is defined up-front and the (minimum) costs are fixed*. Everyone, in theory, knows what they are getting for their money and the service is less susceptible to being nickel and dimed once it's up and running.

On-prem, as many here will have experience of, is an easy target for cost savings when the C-Suite need to protect their bonuses.

*in the sense that they are written down in terms of performance - they might still be variable.