Gmail suffers worldwide wobbly Wednesday Google has spent most of its day on Wednesday working to address ongoing problems around the world with its Gmail service. The Google Apps status page shows that since 0816 PT (1516 UTC) on Wednesday 14 September it has been dealing with issues that have been intermittently causing some users so be unable to use the webmail …
One failure and millions of people are without a service.
I have an old Pentium-4 machine running email services for me that has only failed on me once in the last decade (It caught fire after a staple fell into the power supply, luckily everything else survived intact and I had a couple spare ATX power supplies on hand). I have a store-and-forward SMTP server sitting in a hosting data center to take over if my primary machine gives up the ghost. The machine uses a pair of 20-GB hard disks in a RAID-1 and is backed-up quite regularly onto several 32-GB USB drives (They were $10 at the local office supply store, so I bought a dozen of them).
Yup, businesses relying on Google to host their email are mad.
My old boss tried to explain this to the PHB who had just taken over. "No, he said, Google will save us X dollars per year".
Fortunately Google suffered a 4 hour failure in our part of the world before he could sign up.
Half the company seemed to use it for their personal email, and were most unhappy when we explained that not only could we not make their gmail accounts work, but when the companies email was hosted by Google we would be telling them the same thing about their company accounts.
I was genuinely surprised when I saw this article, as we had no issues yesterday - our company mail is on gmail (we're a small company, decentralised, it works for us) plus my personal side business (spot of photography) and my three personal accounts all appeared to work without issue, each receiving and sending mail over the entire day without a hitch, maybe it was a western world thing?
And in this day an age, though we've not had a problem with gmail, even if it were to go down for a few hours to a day, it's not the end of the world, we have other communication methods and the "hierarchy of communication"* means email's fairly low on the priority anyway.
* if you really, urgently, need someone, you call, then text, then IM, then email. So, if any client has anything critical, they won't be emailing and us missing out anyway.
There is no such thing as a failure-proof system. All that kit of yours would stop working if there was a power cut.... or a motherboard failure. And if your internet connection failed, then your email service wouldn't be much use either.
And then there's the small matter of what would happen if you got run over by a bus.
That's why these anecdotal stories that follow every cloud failure are irrelevant.The modern world relies on networks and power, and those two things fail from time to time. At least when I use gmail (which worked all day for me yesterday) I know that any issues will be resolved without me doing a thing... just like with the power and internet connection.
So, I'm glad you're happy, but you haven't proven that centralised services are a 'terrible idea'
A big issue for me and cloud isn't that shit fails. Is that they are constantly fucking with it. When you have control over things you generally schedule things like upgrades and code changes so at least key folks know it is happening and have the option of providing input as to changing the schedule if needed.
I haven't seen a data center power outage on two redundant circuits in 9 years (and that was a poorly managed facility that had several outages). I have seen one ISP failure in a data center in I want to say the past 15 years. (Shit ISP I think the outage was less than 30min). Obviously this happens more often in the world I just speak to data centers I have been a part of.
Doing maintenance on power or layer 1-3 networking seems to be a well understood thing. Maintenance on cloud has orders of magnitude more complexity and more prone to failure as a result.
Which is why we have things like maintenance windows and notifications. Doesn't work as well when the provider is "agile" and is constantly fucking with it.
When cloud can offer a 100% uptime SLA like Internap does for it's network services with guarantees on latency and performance(and a track record to prove it) we'll be in a good spot. But I am not holding my breath.
"what would happen if you got run over by a bus."
If I'm dead, then I have no need for email. The machine only hosts email for myself. My point was that it isn't difficult to engineer solutions that last, especially if you have access to professional grade stuff rather than the scrap-pile that I do.
Indeed. It's also not hard to grow food if you have access to land. Or make cloth if you have access to a professional grade loom. Or to make clothes if you have access to cloth and a needle and thread.
Most of us choose not to and those that do choose self-sufficiency usually have to form a cooperative to make it work (so aren't really self-sufficient).
If you have a drought then just you will starve. If you source from multiple farmers then you can survive famine. Same with the cloud; use one region at risk, use multiple suppliers for reliability.
The problem with data free narratives is that they are data free.
A simple question: Does GMail fail more or less than an in-house mail server?
Better: Does GMail have a better downtime profile, combining frequency and duration of outages?
The real question: Comparing all costs, capital, maintenance and management of the services against the cost of their outage profiles which service is best for a given business need?
For many businesses, a short email outage of an hour or two is a nuisance with little real economic cost so GMail is a great deal. If outages have real costs, you might want to spend a lot of money on your own highly reliable system. It won't be cheap. Gather the information, do the calculation.
If you use a strategy based on anecdotes you can expect that sooner or later you will be wishing you'd done a bit more structured analysis.
"The cloud is fine for some uses but for mission critical uses I would be very wary. Outages will occur to all providers with varying degrees of frequency and severity."
A-hyup. For myself I'll use a gmail account, but when it comes to a company, I'll rather have them pay the extra $$$ and host the mailserver themselves, or get somebody else to host it for them.
In some cases it may be worth having google host your emails, but it all depends on circumstances.
Cannot afford to have my own mail server at home - yet. So the googly cloud saves me from that headache and hassles.
I realize that sooner or later somebody will fat finger something and I'll lose all my emails. That is the risk I'm taking, and I accept it.
Also that sooner or later it will be possible to read somebody else's googly email sitting in their mailbox, that is another risk I'm taking.
So, for now the cloud (aka somebody else's computer array) works for me...
Affected 20% of the users in the company, but it only took a minute to apply the workaround posted right there on the Google outages page. So three years in, our only GAW incident, and the cloud is still a great idea.
Even if had affected 100% users for 1 day - a business that cannot handle that would probably soon wink out for business reasons alone.
- CTO, previously on the biz side, and have seen worse inhouse email outages in household name megacorp.
Email is a high latency communication method.
If I wnat "instant" comms I use a phone or one of the umpteen IM options.
An email outage of a few hours is not a major issue, just an irritating annoyance (be the mail cloudy, onsite or whatever) unless for some odd reason your whole buiness model revolves heavily around email.
Could give a few SLA issues e.g. if during certain time periods your company promises customers to respond to emails within x amount of time, but even then usually a bit of fineprint about situations such as outages beyond your control / acknowledgements that if there seems to be an issue with one method try another communications avenues (e.g. phone, social media if your company uses twitter or whatever to communicate "urgent problems") unaffected
E-Mail might be *intended* as a high latency option, but in reality, if customers see any form of delay (even seconds) in receiving e-mails then I assure you it's considered the end of the world and no amount of saying it's not a real time system will satisfy the mob.
By a country mile if we have any form of e-mail issue, we get to hear about it REALLY fast.
E-Mail might be *intended* as a high latency option, but in reality, if customers see any form of delay (even seconds) in receiving e-mails then I assure you it's considered the end of the world and no amount of saying it's not a real time system will satisfy the mob.
True, but yours truly was at the root of changing an X400 based system to SMTP where the original email turnaround was measured in DAYS thanks to the conversion "services" of a telco with a very short name - and that was if the email didn't fail (attachments > 2MB would be rejected, but - and I know you'll see this coming if I tell you it was a government department - of course, the customer would still be billed for the conversion).
When we swapped that "service" for a locally hosted translator to SMTP we ended up with the most memorable helpdesk calls they've ever had: people asking what was changed because email suddenly because reliable, and, most notable, instantaneous*. Users called because they couldn't believe the improvement and were worried something had gone wrong that they couldn't discover. It's quite fun to assure users that all is well in that way :).
* When compared to the old system - in reality it was on a 5 min polling cycle.
Or the more usual, oh I just found them in my Spam folder!!
Anyone who runs their own mailserver in this day and age is mad as a hatter. I've been on the wrong end of a large Exchange Server meltdown, that took two weeks to get back to normal, it wasn't pretty and after working an 80 hour week to get it working the client still haggled over the bill and took six months to pay it. No they didn't have an SLA or even a maintenance contract on it.
Everyone I deal with now uses GAW, Office365 or MessageStream, I can have a holiday, sickday and can get hit by a bus safe in the knowledge that if there is an issue it's either between the chair and the keyboard or someone's working on it already.
Everyone I deal with now uses GAW, Office365 or MessageStream
Then none of your contacts has any privacy or data protection obligations. Especially the higher level customers we have wouldn't go near us if we weren't in total control of communication. They pay for us to do it right, and not farm it out to an uncontrolled third party with a surveillance fetish and a known allergy to law and user rights.
e-mail is unreliable
It doesn't have to be, but there are far too many people that seek to introduce complexity to create themselves a job for life. This is, for instance, why I am absolutely no fan of Microsoft Exchange.
I've been involved with email from well before the Internet came into existence (as a matter of fact, I recently rediscovered my old FidoNet address in some old documentation I wrote :) ), and have converted between many systems (the nightmares involving sendmail.cf have gone, thanks for asking).
There is a certain structure to it, and if you understand that you can make email not only stable and reliable but also safe to use. This does also involve taking into account that you should avoid making life difficult for your users and plan ahead for mistakes they may make.
'The Internet .. is a worldwide system of computer networks'
E-mail is "broken" and "insecure", so cloud outages may not be a bad thing if they remind us that there may be better alternatives. If I phone or e-mail for help, usually what I'd rather be doing is secure web-chatting. For example. I prefer Ctrl-A, Ctrl-C, Ctrl-V to "Do you have a pen and paper handy?"
Being responsible for many DCs and a large enough infrastructure, I can tell you no matter how much you invest, things happen.
The larger you get the more variables there is in the equation, look into the simpliest of things for us this days, Internet connectivity. Any ISP, despite whatever people stated here, I've worked in several big enough, daily their NOCs are flooded with failures, 99.9% they aren't service affecting, or cause minor flaps, barely noticible by the end user. and then there is 0.1% of causes that causes a big mess for a while, but almost always things are recovered in a very short time if you think about it.
Some people prefer to have email servers at home, can you share your home electrical bill with us please? or have you ever though that your house could have caught fire and you would be crying at this point because your house was on fire?
And how much it costs your contingency server in some ISP as you claim? does it really worth? I don't believe your eail is that important :).
If you want to manage because you are an entosiast is fine, you can always get 2 or 3 virtual servers in different low cost providers, for the specs you present, theses days for a couple of euros the trick is done.
Availability isn't the reason Cloud is better or worth than anything, tell me 1 company that never ever had IT problems, that is operating for more than 2 years... Google, Azure AWS, Digital ocean, Banks with internal team, or even NSA that was hacked not that long ago.
Besides I think we all prefer to be screaming at someone to fix the problem, than to be screamed at us to fix a problem... and this is the real reason Cloud is so successful, as well outsourcing before of that (like millionaire IBM contracts in early 2000s). You can always blame someone else instead of yourself for your choices.
There isn't a perfect solution for everyone, so chose what makes sense for you economically, in terms of privacy and risk that suits your needs and make you happy 90% of the time and lets you sleep. All of the solutions have their risks, I wouldn't be comfortable to have a DC at home without minimum security conditions.
So stop the type of conversation, mine is bigger than yours... not all the ladies like them big!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
