Every developer who's decide they can just say "something went wrong" with a little twee jovial pic or phrase, and suggest to the potentially harassed and pissed off user that they just try again needs a massive slap around the chops.
OneDrive is broken: Microsoft's cloudy storage drops from the sky for EU users
It is OneDrive's turn to get a beating with the stick of fail as the service took a tumble this morning. Issues first began appearing at around 08:00 GMT as users around Europe logged in, expecting to find their files, and found instead a picture of a bicycle with a flat tyre or a dropped ice cream cone. Oh, you guys! @ …
COMMENTS
-
-
-
Friday 30th November 2018 00:59 GMT bombastic bob
" the little picture of a paper aeroplane with a snubbed nose well and truly made up for my inability to get to all my fecking files."
Is there a collection of these someplace? It might be fun to "share" it in a snarky manner around 'teh intarwebs'.
So, at Micro-shaft, do they have *ENOUGH* *TIME* to draw these 'cutesy' "ooops ME BAD" types of "excuse" pics? But NOT! ENOUGH! TIME! TO! MAKE! THEIR! SERVICES! RELIABLE!!!
I'd say they need to "re-think their priorities".
-
-
-
Thursday 29th November 2018 15:43 GMT Mr Benny
"and suggest to the potentially harassed and pissed off user that they just try again needs a massive slap around the chops."
Perhaps iits the user that needs a slap around the chops for being such a cretinous mug as to believe all always available Cloud hype in the first place. Certainly anyone who continues to rely on MS cloud with no fallback option after the succession of cockups iis suffered deserves everything they get.
-
Thursday 29th November 2018 16:51 GMT tin 2
yes, but said user will more often than not be some poor sod who's had it decreed upon them by some overinflated CIO that their company has gone cloud, and therefore they now have to keep all their files in the cloud, and that it has to be the MS flavour of it. Because cloud is good.
Poor user ;(
-
Thursday 29th November 2018 23:26 GMT eldakka
yes, but said user will more often than not be some poor sod who's had it decreed upon them by some overinflated CIO that their company has gone cloud, and therefore they now have to keep all their files in the cloud, and that it has to be the MS flavour of it. Because cloud is good.
Poor user ;(
In which case it wouldn't be "poor user", as it is no longer the individual user's problem, nothing for them to worry about. It is the corporations problem, and the individual user is perfectly entitled to sit at their desk twiddling their thumbs, reading the paper, browsing the web stress free, at their bosses expense, until the issue is sorted.
-
Friday 30th November 2018 07:46 GMT Shadow Systems
At eldakka, re: "Poor user".
A friend of mine told me this evening that's *exactly* how her day went. Her employer mandated all their files be in the cloud, all their programs hosted there, & crowed happily about the money the company would save in doing so. Right up until they have *any* issue with said cloud provider that stops their work dead in its tracks. She couldn't get to their customer files thus she couldn't interact with said customers (lest they want said changes she was unable to make) & that left her with nothing to do all day but read her (physical) book.
I asked if she was told to do make-work/busy-work in the mean time, to which she laughed "That would have required a connection to our host. I'll give you three guesses what we DIDN'T have all day and the first three don't count!"
She got paid to sit on her ass & read a book today. And so did all her coworkers in their office. All because some CxO decided "the cloud" was the way to save gobs of money. I wonder if the idled employees got paid today out of said CxO's Golden Parachute to pound the lesson home, but I'm afraid the company will simply take it out on the innocent employees rather than the person that caused the situation to happen in the first place.
-
-
-
-
Thursday 29th November 2018 11:22 GMT Pascal Monett
"a lesson on the consequences of too much dependence on the cloud"
Wrong. Those consequences have been repeatedly and frequently exposed, so that there is no one that works anywhere near IT who can claim that he didn't have a clue.
What this is compares much better to a lesson in sheep herding, with major CEOs and "decision-makers" being the sheep, herded to this position bleating gleefully about how much money would be saved (and counting their additional bonuses on the way).
Well, people, you can now take out your calculators and count the money lost, per hour, for all your employees who cannot do their jobs. You should at least return your bonuses in a token gesture to offset the cost of such short-sightedness.
-
Thursday 29th November 2018 12:36 GMT Anonymous Coward
Re: "a lesson on the consequences of too much dependence on the cloud"
You should at least return your bonuses in a token gesture to offset the cost of such short-sightedness.
The recipients would find that notion at best humorous, but not realistically entertain it as something worth considering. Personally I would agree, but only when augmented with mandatory tarring and feathering.
-
Thursday 29th November 2018 12:40 GMT Anonymous Coward
Re: "a lesson on the consequences of too much dependence on the cloud"
@Pascal Monett
In modern management theory this is not correct. The CEO/Decision Maker still made the savings promised by the snake oil cloud vendors. This can be proved by looking at the invoices. They paid less for cloud services than they would for internal storage.
The "failure" was made by the working level staff who should have been flexible and agile enough to cope with any failure.
Consequently the CEO/Decision Maker bonuses will increase, working level pay/bonuses will be curtailed.
-
Friday 30th November 2018 09:06 GMT Anonymous Coward
Re: "a lesson on the consequences of too much dependence on the cloud"
As Pascal Monett said:
"What this is compares much better to a lesson in sheep herding, with major CEOs and "decision-makers" being the sheep,..."
THIS has been the main problem with corporate computing right from the start, and Microsoft essentially built their business on those with purchasing power having that mindset. The "You can't get fired for buying IBM" attitude was what made many businesses get desktop PC's in the first place, and that meant they ended up with an operating system made by a little outfit called Microsoft simply because that's what came with the boxes they'd just bought, not because there was any analysis of whether the OS that came with it was better than other options - the hardware and the software that came with it were generally viewed as a unit buy folk who generally didn't know any better.
What has astonished me over the years is the willingness of businesses to put up with the blatant activities of Microsoft, Apple and others to try to actually force vendor lock-in on customers (rather than merely encourage it), and this despite (in MS's case, at least - I have zero experience of Apple products) software that has been shockingly bad and unreliable at times. Things like the file format wars and companies forcing GUI changes on users rather than offering them as new alternatives that people might enjoy...
How in hell anyone could, with a straight face, laud Bill Gates and MS as having made computers easy to use (as I've seen stated on several occasions over the years) without intending to be sarcastic I do not know, unless they did so with a complete ignorance of the subject of IT. And there's the fundamental problem with IT in a nutshell - it tends to be people who don't understand IT who make the major purchasing decisions, both in businesses and in the home.
-
-
Thursday 29th November 2018 12:29 GMT Lee D
Office 352 - 360 depending on where you live, I think.
Put it this way... it ain't five-nines:
SLA level of 99.999 % uptime/availability gives the following periods of potential downtime/unavailability:
Daily: 0.9s
Weekly: 6.0s
Monthly: 26.3s
Yearly: 5m 15.6s
It's closer to 2-nines or less:
SLA level of 99 % uptime/availability gives the following periods of potential downtime/unavailability:
Daily: 14m 24.0s
Weekly: 1h 40m 48.0s
Monthly: 7h 18m 17.5s
Yearly: 3d 15h 39m 29.5s
-
Thursday 29th November 2018 14:06 GMT Hans 1
One moment ... is that "Yearly" ... because ... we have exceeded 3 days of downtime over the last 30 days....
Office 365, Exchange, MFA and, what was the other, again ? For me, MFA outage lasted roughly 30 hours ... between 25 and 30 - not sure if my system was caching something, anyway ...
Yes, it is funny, not really for the poor sods who rely on it ... I am forced to use it and no, never will I rely on it ... service is too flaky and they're too nosy.
-
-
-
-
-
-
Thursday 29th November 2018 13:01 GMT Killfalcon
Re: time to fess up...
I've got to be honest, I'm curious as to what goes wrong with this sort of thing, because it's the same pretty much everywhere.
MMOs, ISPs, cloud providers, train companies, banks: the time lag between "customers notice en mass" and "the service desk are told/permitted to treat it as a major incident" is almost always achingly long.
Almost all of them should have easily visible metrics that reveal the issue: your logins drop off a cliff. All the trains are behind on their routes. Transactions/minute have doubled or halved or whatever.
So why don't they have a way for the poor grunts on the helldesk to be told "it's buggered, tell people we're working on it and will provide an update at 14:00 UTC"? Why are they left fielding angry customers armed only with "try rebooting" scripts? It's not just bad customer service, it's down-right unfair to your staff.
-
-
Thursday 29th November 2018 23:26 GMT Terry 6
Re: time to fess up...
"support orifice spent the early part of the outage in denial, asking users to clear their browser cache and cookies".
Yep. It always seems to run like that. Any of these organisations. Customers knowing that something has gone wrong at the suppier end before front line staff are told. A week or so we had a problem with VM ( tbh pretty rare). The support guy started on the usual route, then halted and said something along the lines of "actually my colleagues are getting a lot of calls about this too, hold on". Then he went away for a couple of minutes and when he returned he's spoken to his boss, who'd checked....And you can guess..
-
-
-
Thursday 29th November 2018 13:31 GMT Anonymous Coward
Re: time to fess up...
"[...] this time lag hasnt changed for years, no idea why they cant update their status update process"
UK. We recently had a power cut - seconds afterwards the wired phone on the master socket started ringing. A recorded message told me there was a known unscheduled outage. Further button pushes accessed more details. Neighbours with mobile numbers received text messages.
Apparently their web site gave an estimated time window for power to be restored - and it was actually back before that.
Now that is a useful advance in the use of technology - by Eon and/or the grid management.
Hope they have planned what to do if the outage affects enough people to overload the phone systems.
-
Thursday 29th November 2018 15:32 GMT Anonymous Custard
Re: time to fess up...
Likewise, a little while back. Albeit on only one of my landline phones - an old wired one which is kept in service for exactly that purpose.
All of the other ones (DECT cordless jobs) were of course absolutely dead as the power cut had taken out the base station so no transmission to them, and of course the phone designers (BT, or at least the phones were branded as such) hadn't thought to include a back-up battery or capacitor cell in the base station for such eventualities.
Given that those DECT phones seem to be about all that many retailers sell these days, one has to wonder in the case of such events how many people are actually effectively cut off from landline phones as well. Not that it matters so much in this age of mobiles I guess, but it's certainly a though.
-
Thursday 29th November 2018 23:40 GMT eldakka
Re: time to fess up...
and of course the phone designers (BT, or at least the phones were branded as such) hadn't thought to include a back-up battery or capacitor cell in the base station for such eventualities.
Adding such a feature would increase the cost of the phone to the 90% of people who don't need this feature.
For anyone who does need/want that feature, there is an decades-old (possibly even century+ old) industry standard for that facility, an external battery/UPS/generator backup (pick whichever is suitable for your use case) that anyone is able to install and use across multiple devices, not limited to just the phone.
-
Friday 30th November 2018 08:04 GMT Anonymous Coward
Re: time to fess up...
UK Power Networks do this with great efficiency and granularity- right down to texting a mobile no. for a local manager out-of-hours if there’s an extended delay to a notified repair. Credit to then and why tf can’t the rest of the corporate world (especially the corporates who *specialise* in comms) do as well?
-
Friday 30th November 2018 13:54 GMT Jamie Jones
Re: time to fess up...
"UK. We recently had a power cut - seconds afterwards the wired phone on the master socket started ringing. A recorded message told me there was a known unscheduled outage."
Great. So, no sooner are you plunged into darkness, but the phone rings, causing you to stumble when going to answer it :-)
And oy, my glass is still half-empty!
-
-
-
-
Thursday 29th November 2018 12:28 GMT Kobus Botes
Re: Daily dose
@Dwarf
I wonder how many other internally visible services have amber monitoring right now ???
All's green (externally, at least).
But then again, on Tuesday when Azure MFA fell over and servers were rebooted left, right and centre, Azure status reports were all green as well.
Boy, am I glad I am retired now!
-
-
Thursday 29th November 2018 13:01 GMT DaLo
I just don't understand the idea that it must be a single user's problem when there is an outage like this. Surely they have network monitoring systems that flag up within milliseconds that there is a significant problem and should let them know it isn't a user problem before the first call/tweet/letter comes in.
-
Thursday 29th November 2018 13:52 GMT johnnyblaze
It does make me laugh. People falling over themselves to get locked into these 'broken' clouds, being sold it as the best thing ever. If sysadmins had this much downtime on their on-prem networks, they'd be hauled infront of the boss, yet once you're locked into the cloud, your stuck with it. How much money must companies be losing?
-
-
Thursday 29th November 2018 18:17 GMT Anonymous Coward
Re: Modern Milli Vanilli CEO's
Jim Morrison had it right.
This is the end
Beautiful friend
This is the end
My only friend, the end
A long time before MS was even thought of.
Time to Junk MS people before you go bust relying on their (or any off premises cloud for that matter) [insert swearword of choice here]
Just where are the countless class action lawsuits eh?
We all know that there would be lawyers crawling all over this if this was an Apple service.
-
-
Thursday 29th November 2018 16:55 GMT Anonymous Coward
That's a brilliant idea!
So that's why they go over to The Cloud...
CEO: "7 days now, and we still don't have anything back up and running!"
[Sysadmin looks up from their sunglasses, BBQ'd sausages, puts down their beer]: "Oh, the Cloud service provider is right on it, they will have a status update to me by tomorrow..." [Pours another glass]...
You can't fire someone for incompetence if they don't even work for you! XD
-
-
Thursday 29th November 2018 14:17 GMT mark l 2
It does annoy me how the recent trend by web developers to change useful error messages to something akin to 'Ooops something went wrong!'
I had an issue while trying to do something on ebay and it kept reporting one of these useless error messages, I had already tried it from different PCs and browsers and even a different ISP and always got the same messages. Yet ebay support claimed their was nothing wrong with the site and it must be a problem at my end.
-
Thursday 29th November 2018 18:46 GMT Ken Hagan
"It does annoy me how the recent trend by web developers to change useful error messages to something akin to 'Ooops something went wrong!'"
It is annoying, but it is not surprising. As you fling more and more parts of your system outside your control, there comes a point where it is impossible for the software itself to figure out even whether something has gone wrong, let alone what. All that each part of the software can possibly know is that it farmed the task out to somebody else and it seems to be taking longer than expected. Worse, the only part of the software that can report back to the user is the front end, so the only reasonable message is some variation on "I sent your top-level request off to be processed and it is somewhere along a chain of handlers longer than I'd like to admit (and quite possibly longer than I'm aware of) so ... um ... fingers crossed then!".
-
-
-
-
Thursday 29th November 2018 18:50 GMT Ken Hagan
Re: Classic!
From https://www.urbandictionary.com/define.php?term=Broken%20Arrow:
"When you're aiming for a clean break/no splash, and instead, whether by distraction or an imminent sneeze, your log breaks off and sends the staff crawling back inside, only to disturb you for the rest of the day and cause immense discomfort and multiple re-wipes."
Microsoft in a nutshell, then.
-
-
Thursday 29th November 2018 14:47 GMT MrKrotos
Its NOT just OneDrive
Exchange Online - EX165763 - Can't access email
Status: Restoring service
User impact: Users may be unable to connect to the Exchange Online service via multiple protocols.
Updated: 2018-11-29 10:17 (UTC)
Start time: 2018-11-26 10:10 (UTC)
Office 365 Portal - MO165899 - Admin usage report content delays
Status: Service degradation
User impact: Admins may notice that newly created mailboxes don't show in the Exchange usage report.
Updated: 2018-11-28 06:29 (UTC)
Start time: 2018-11-09 00:00 (UTC)
-
Thursday 29th November 2018 15:01 GMT Aqua Marina
OneDrive doesn't work even when it's working. When it can replicate a windows directory character for character exactly just like DropBox can, then I'll say otherwise. As it stands now there are too many windows file-system compliant characters in file names that OneDrive can't accept, and simply refuses to back up. Come on MS, it's an MS product, at least make it character compliant with your other MS products.
-
Thursday 29th November 2018 15:35 GMT SVV
Microsoft has a wide variety of images to illustrate failure
Well, that is probably because they have a wide variety of failures in their cloud every day. Perhaps they should make their images suitable for the type of failure instead of this cute slapstick approach which is really just rubbing salt into the wound.
So, if you can't access Word files, a burning piece of A4. For Excel, some money on fire. OneDrive : a car with no wheels. Active Directory : a phone book with a padlocked chain round it. Azure : an IT manager getting sacked.
-
Thursday 29th November 2018 15:36 GMT dizwell
Working for me...
As one who has been uploading 1TB for the past three weeks on a slightly dodgy Vodafone broadband connection from darkest Nottingham, I have to confess that my uploads have been progressing steadily all morning with no apparent interruption of service at all. Much to my relief!
Then again, I'm running on Linux and using the onedrive utility from github.com/abraunegg/onedrive as my onedrive client. Maybe that makes all the difference?
-
Thursday 29th November 2018 15:44 GMT AlexMac
MY EXCHANGE E-MAIL STILL SEEMS UNABLE TO SEND. IT KEEPS TELLING ME IT CANNOT SEND:
"Your mailbox quota on the Exchange server has been exceeded. If the problem persists, contact your Exchange server administrator. Select a different outgoing mail server from the list below or click Try Later to leave the message in your Outbox until it can be sent."
Now I can only reply to people via my iPhone or iPad and AOL :-) Thank god I kept that dear old AOL relic, although that doesn't work with my Mac in recent years, for some reason..
-
-
Thursday 29th November 2018 20:29 GMT ma1010
Re: So now you know why...
...I always said that I didn't trust cloud technology like this. The problem is the single point of failure; Microsoft.
MS is not merely a single point of failure, it's a single point of so many failures these days! I remember when they used to make good stuff. I was amazed at MS Office when it first came out. I was an evangelist for Windows and MS Office back in those days.
But now I use Linux and avoid MS as much as possible. MS seems to be the death of anything it touches nowadays. Their cloud services seem to be down more and more. Their untested, buggy OS is pretty much total crap, and why, in the name of $DEITY do they insist on a short life cycle which seems to be just shoving out more bugs faster than what they used to do. They seem to destroy whatever they touch - look at Nokia and Skype. Don't know what happened internally there, but nowadays they seem to be driving off a cliff.
Maybe if they'd fire their current leadership and go back to what they used to do - produce stuff that worked - they'd be useful again.
Paris because I don't understand why MS seems to be set on committing suicide.
-
-
Thursday 29th November 2018 17:10 GMT Jeff 11
"Hi! Let's team up in getting this issue resolved. To start, can you tell us since when did the issue start? Have you tried clearing your mobile browser's cache and cookies? We'll be waiting for your response."
...code for:
"We don't have adequate monitoring, we're putting obstacles in your path to make you think it's anything but our fault, and we won't update you until you update us."
-
Friday 30th November 2018 08:28 GMT EthelAardvaark
How can they get it so wrong?
As someone who has used Google drive at home and OneDrive at work, it's always amazed me how poor OneDrive really is. It has far more difficulty synchronising and avoiding conflicts than Google, it has repeatedly lost updates to some files (possibly a symptom of the same issue).
And now it appears that network resilience is a manual process.
If i was the corporate budget holder, I know where my money wouldn't be going...
-
Friday 30th November 2018 08:30 GMT nextenso
Msoft bashing again
There you go again, sarky reporting of an Msoft problem making a meal of it instead of making a proper journalistic report of the issue.
I have heavily used 365 and OneDrive and its predecessor SkyDrive almost since its inception and rely upon it, very cautiously at first. Over many years I have rarely had service interruption and those that arose were shortlived, apart from 3.
In comparison, I have worked for a number of large companies with own server rooms/farms. Collectively I lost count of the many times server glitches and crashes brought those companies grinding to a halt, often for long periods while techies slaved trying to coax machines that were end of life back into service.
The cost of running own servers is formidable with technology and software advances making installations redundant so quickly.
In my own business I had a £4m set of Sun servers, Jukebox, storage etc. It kept chuntering on with ever increasing costly innovative care and downtime, so slow compared to new machines which we could not afford to upgrade to. Changing to Msoft 365 breathed new life into everyone's work, such a relief.
No software/hardware provision cloud or otherwise is ever going to give 100% availability, as cloud provisions get larger and larger and stress points make themselves known, new problems are discovered. For providers its still a learning game especially as ever advancing user software loads unexpected conflicts.
Please just report without all this sacasm that is almost like a vendetta against Msoft - with whom I have no connection other than being a customer grateful for a service that saved us so much cost.
-
Friday 30th November 2018 12:40 GMT russmichaels
backup your cloud storage
If you do not want to be caught out by such problems, then I suggest backing up your cloud storage to a second location.
I have GoogleDrive as well, and I sync my onedrive and googledrive storage, so if one goes down for any reason, I can still access all my data on the other one.
-
Friday 30th November 2018 15:00 GMT Hargrove
Re: backup your cloud storage
There is a practical issue with redundant cloud storage as a solution, from a system standpoint.
The 'net is an integrated end-to-end system comprising the sum of connected users, hardware, software, and information connected at any given instant. Its size, configuration and complexity, and functional state change continuously at intervals measured in nanoseconds. It is intrinsically indeterminate and interminable.
The system is unbounded. But it is not infinite. Its performance, although incomprehensibly vast, is finite. And it is often limited by local subsystem performance, the unintended consequences of which can have catastrophic effects over large parts of the overall system.
Such a system cannot be reliably operated or maintained. The constant string of reports and the discussion in El Reg are concrete evidence of that.
A brief excursion into reality to support my assertions. From an end user perspective the system has no specifications. Not many years ago IT users demanded and equipment and service providers guaranteed to deliver a minimum level of performance with a specified reliability. Rliability was commonly expressed in terms of "availability" (basically the percentage of the time that the system was guaranteed to work as specified over a given period, typically a year) and contracts provided for adjustment in cost for if they couldn't restore service in a timely manner to meet the threshold.
In this era 99% was considered poor, four 9's (99.99%) was considered reasonably achievable.
From my narrow perspective, this all changed around 2010, as federally-mandated initiatives for automatic security patches and updates and cloud computing were implemented. I saw the specified system availability in an annually-renewed service level agreement for a web hosting service I was involved with go from 99% to 90% and then disappear completely.
This was NOT due to any lack of diligence, competence, or desire on the part of the service provider. The simple fact is that dependence on third party software and services, and the unintended consequences of continuing patching and updates from a myriad of software vendors and third party service providers make it impossible to guarantee anything.
The Achilles heel of the system is bandwidth. Marketing hype notwithstanding "unlimited data" and its more egregious step-sister "infinite bandwidth" are myths. They are exercises in metaphysics, valid only with the presumption that we have all eternity to download our data.
For a few individual with a few tens of gigabytes of data, who need to retrieve specific files, redundant cloud storage and local back up may be a practical solution at the moment. On any kind of scale, however, the bandwidth simply isn't there. We are already seeing this in the massive marketing push to get users to accept "unlimited data" plans, advertising "speeds up to" under terms and conditions that do not actually guarantee ANY level of performance.
This is not going to end well for anyone.
-
Friday 30th November 2018 15:08 GMT Terry 6
Re: backup your cloud storage
I would suggest, if budget supports it,( SOHO for example, not too much data for an external HDD or two) you use the cloud account as the second/third backup. But back everything up locally first. Cloud for when the building can't be accessed/burns down as a business continuity measure. YMMV of course.
-
-
Friday 30th November 2018 13:31 GMT Anonymous Coward
What could go wrong? They said.
I feel absolutely vindicated.
I initiated a system of backing up 'One drive' on HDD's I own off-line because I was forced to use a system that I didn't trust. Everyone laughed at me for being a Luddite.
Now the boot is on the other foot. They have all come crawling to access my archives and I have been refunded my costs of the drives by the boss (No apology though).
I don't trust the 'Cloud' and I don't trust 'virtual' money because all you need to do to conquer a country is take it off-line - and we KNOW that can be done.
All I need now is a nuclear bunker - on expenses naturally...