Salesforce.com outage exposes cloud's dark linings Exposing the dark side of cloud computing, Salesforce.com suffered an outage that locked more than 900,000 subscribers out of crucial applications and data needed to transact business with customers. Salesforce, the 800-pound gorilla in the software-as-a-service jungle, was unreachable for the better part of an hour, beginning …
An hours outage can happen to any system.
But what if it were 24 hours?
Or what if the supplier went terminally (oops!) bankrupt over night?
The only secure way to handle data required by lots of people, widely distributed, is to have a "core" database, from which each person downloads what they need for the day, and downloads further updates as and when necessary, and transmits their own updated data to the centre as opportunity presents.
Nobody seems to wonder what happens if your sales force need to go where there is no mobile net access.
I always carry my own essential data - my Psion 3a is in my pocket.
Oh yeah, right, this is the same centralized computing outage problem we had back in the mainframe days, only extended worldwide.
The cloud is great - for cloud/service vendors and for very small companies that can't afford their own IT staff. For everyone else it is a boondoggle waiting to happen.
We used to sell old style database software but as our clients where not exactly IT skilled they routinely screwed the pooch.
We went SAAS 5 years ago and now our IT unintelligent clients cant mess up there own DB's
Outages happen, but I would rather lose connectivity for a while with a well looked after, heavily DR supported system than have a hard drive meltdown and realise I havent backed up for a week.
As for mobile net access, most sales teams sell into developed markets, there isnt a lot of scope for widget purchases in Mogadishu for many sales teams to worry about online access in places with no mobile connectivity, even then there is Sat broadband at a cost.
And what happens when some Chav hoodie mugs you and steals your Psion?
Paris, the ultimate SAAS application!
This helps lend even more weight to Microsoft's strategy of Software PLUS Services, rather than the current Software As A Service.
Having it on Premise too would allow companies to still be productive at times like that...
Billy G cos he knows how it is!
http://richfrombechtle.wordpress.com/2009/01/07/salesforcecom-outage/
El Reg writes: "Real world sales forces are generally useless without real-time access to CRM apps."
Please seperate reality from your dream -- there are no real world sales folks, just sales droids for crissake! Even they can't use SW to do anything useful, other than send bleeding escalation emails with unrealistic demands to all and sundry up your Org Chart!
I think you'll find that there is no garantuee as to the safety of your data in that 'well looked after, heavily DR supported system' of yours. You see, back-ups cost money. Things that cost money, especially things that don't have immediately apparent consequences (like not making back-ups) have an annoying tendency to get cut.
Local database, with redundancy, encrypted and off-site back-ups is still the only reasonably secure way to go.
And anybody who runs a database and fails to back-up regularly deserves to be shot anyways.
Let's all just tap the breaks. Is this outage newsworthy? Yes. Does it "expose a dark lining"? I think that's shoddy reporting. Stick to the facts:
- First major outage since 2006
- Only been 6 hours since outage occurred...give the company time to tell us what the problem was
- On-premise systems go down too...and may not come up as quickly as SFDC did today
Report the facts...but don't sensationalize. Ruins credibility. You're better than that.
The "cloud" is simply mainframe computing spread wider and with more lies. The same problems with reliability, the same "all the eggs in one basket" problems, the same problem with connectivity to said mainframe, except the connection isn't under your control anymore.
Anyone who trusts their critical business data to the cloud deserves to go bankrupt the next time something happens to any one of the multiple points of failure that are inherent in the system.
"Things that cost money, especially things that don't have immediately apparent consequences (like not making back-ups) have an annoying tendency to get cut."
Not in systems I design ... It's in the contract. If backups aren't current, and readable, when things go pear-shaped our contract is void. Works wonders on backup viability :-)
"Local database, with redundancy, encrypted and off-site back-ups is still the only reasonably secure way to go."
Indeed. I add redundant off-site backups ... Helps occasionally.
"anybody who runs a database and fails to back-up regularly deserves to be shot anyways."
Or the idiot manager in charge who refused to budget for the regular backups should be taken out behind the barn and severely thrashed, at a minimum ... Me, I fire em. On the spot. It's in the contract. I love watching a useless middle manager's face going white ... Whether the company goes thru' with the firing is up to the company's lawyers, but I love having the option.
so eventually people will get the message.
Oh well, amusing to watch those relying on this, when it just should not be done for day to day operations, you have two points of failure now. You get a power cut or remote gets one, it is that simple that is what the cloud does to you it increases your basic risks.
The problem with cloud computing from the client perspective is disaster recovery and the available mitigations. There are many more total failure scenarios than applications hosted by the company that is using them - and almost all of them are outside the influence of the end user organisation with respect to minimising them and recovering from them. Heck, I'd not know where to start if I was asked to quantify the risk.
For example, how does a company recover from a cable cut into a data centre on another continent? Or a relaying satellite being hit by debris and going offline? Yes I know, neither situation should not result in a service loss but that is just to get you thinking about how much of the infrastructure is not under your control.
Bottom line is that you'd need need to plan a fallback position that allowed you to run your operation for a minimum of a couple of working days and ideally at least a week with no access to your cloud applications.
Can companies ever truly rely on cloud applications being available 24x7? The answer is no - regardless of how 'up' a cloud actually stays, there is always the risk that you lose access, and so disaster recovery plans must cover the eventuality.
I think the cloud has a place, though. A start-up can get going on its own hardware to 'prove the concept' and then deploy to the cloud to get the business going with minimal capital outlay. When the value to the business of those cloud applications goes past a threshold (for example, once the services provided are turning a profit) then it is time to seriously think about taking ownership of the infrastructure again.
So the new buzz-word for SAAS is "Cloud" computing eh? And no one ever stopped to think that the foundations of cloud computing are about as solid as, dare I say it....a cloud? If you're going to do the sloping shoulders trick and let someone else worry about your eggs, there's no guarantee you'll be getting the basket back intact. Personally, I'd rather have a critical system like CRM in-house, at least you've got greater ability to prevent the system falling down in the first place.
Mine's the one with disks 1, 4, 7 & 8 of the Saleforce RAID array in the pockets :-P
Where are the I told you so brigade....
http://www.theregister.co.uk/2008/07/14/salesforce_sun_server_dell/
Last time they had a major outage they had to buy Sun....they then replaced it with Dell... and now look what happened..
I know, I know... we do not know what caused the outage yet...
Cloud computing is great, if...
If the cloud is more robust than my PC, because it is distributed and replicated...
If the cloud is maintained by professionals that are smarter than me, or my IT guys...
If economies of scale are used to improve robustness instead of profit...
Not discarding my old, boring, so-not-this-year IT kit any time soon.
How is it possible for Salesforce to fail this much at clustering? Where I work we have a test lab where everything is redundant, half the datcenter could be destroyed in a fire, but everything will go on as business a usual. And this is just a test lab, the production side has redundant data centers on different continents (Strung together by multiple 10Gig connections), which means even nuclear war won't bring us down.
Its great how Mr. Ozzie pretty much signed a blank check for our kit....
"I think you'll find that there is no garantuee as to the safety of your data in that 'well looked after, heavily DR supported system' of yours. You see, back-ups cost money. Things that cost money, especially things that don't have immediately apparent consequences (like not making back-ups) have an annoying tendency to get cut."
err, any SAAS seller is going to be out of business fairly fast if they dont have decent DR and real time off site data backup
"Local database, with redundancy, encrypted and off-site back-ups is still the only reasonably secure way to go."
Depends on your rate of data flow, even a loss of 24 hours between backup and failure can cripple a business.
We do SAAS for the management of Yoga/Pilates/FItness facilities around the world, none of those sectors are renowned for there IT capabilities, data backups or budget for there own hardware, servers, offsite back up etc etc. I have had people in tears on the phone who didnt bother with back ups and then had there comp stolen or destroyed, doesnt happen now.
"And anybody who runs a database and fails to back-up regularly deserves to be shot anyways."
On that we agree.
"We do SAAS for the management of Yoga/Pilates/FItness facilities around the world, none of those sectors are renowned for there IT capabilities, data backups or budget for there own hardware, servers, offsite back up etc etc. I have had people in tears on the phone who didnt bother with back ups and then had there comp stolen or destroyed, doesnt happen now."
So basically, you're saying *aas is another word for "taking candy from babies"? I'm cool with it, if you can live with yourself. I'll get my cut when I pick up the pieces ...
Me, I provide a total solution where a previously computer illiterate company can pretty much run their own systems. Seems cleaner, somehow.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
