
Think it's actually Office 363 now, as we had that outage last month.
If you sat down to work this morning and attempted to do something as routine as check your emails with Outlook, you'd be bang out of luck. According to outage tracker DownDetector, reports began coming in of users facing a 500 error and being unable to send, receive or search email through Outlook.com from about 4am UTC, …
In the rare times I am not getting a 404/timeout/500 error when trying to open an email, Office "365" is literally changing the UI for every email. Sometimes my previous/next email buttons disappear, sometimes it is the delete. Other times they get moved to the ... button.
And what has been broken since the first of the year? Read emails. you know, that functionality where when you read an email, it gets marked as read. Or at least it used to. But I can read an email, reply to it, go to the next one - and the email I just read remains flagged as unread. I have hundreds of "unread" emails in my inbox now.
The Youth of Today. We had Better On A Camel, But Will It Arrive, Better Wait In Airport, Britain’s Worst Investment Abroad, and Panned American and The Clipped Wings of Man. (For those who don’t know… BOAC, BWIA, BWIA again, BWIA yet again, yes, they were that bad, Pan Am, and Eastern.)
United was the subject of different jokes, mostly about ahem, flying united. Delta had redneck jokes, often involving banjo music after a certain Burt Reynolds movie came out.
Ah yes, I remember SleazyJet and Cryanair. I seem to recall that our internal names for Ryanair and Virgin however were <the surname of a DBA named Ryan that we'd all worked with once> and Vogon (occasionally Vorgon), respectively.
Whoever said that it was intended as 365 base 10?
For the pessimists, how about base 8 (245 decimal ), or for the contrary, base 7 (194 decimal).
BTW re acronyms for airlines below, allegedly related to passenger "care" :
Try Walking Across
Don't Even Let Them Aboard.
Oh please:
"We're applying targeted mitigations to a subset of affected infrastructure and validating that it has mitigated impact. We're also making traffic optimization efforts to alleviate user impact and expedite recovery."
Is the excuse department on piece rates? Lookout chaps, ChatGPT is coming for you.
"We're applying targeted mitigations to a subset of affected infrastructure and validating that it has mitigated impact. We're also making traffic optimization efforts to alleviate user impact and expedite recovery."
in other words: it's gone TIT5UP, AWOL and MIA...but WTS and it might be BSS.
...it's notable that MS change management happens when it's least likely to affect the home market so rest of the world gets 12 hours or so of beta testing. Or am I I ascribing maliciousness where only ignorance is in play? Maybe the people making the decision on when to apply changes don't understand that time zones around the world cover more than the 4 or so hours most "lower 48ers" have a vague understanding of?
Saw it happen with AWS many moons ago too — entirety of Europe went down for many hours (and I mean everything, including all of Amazon's own websites) with complete radio silence from Amazon for the duration. Didn't appear on a single news website that I could see.
People are overall a lot cloudier these days, but it was telling as an outage like that in a US region would've been widely reported.
:D unfortunately not — you could see the usual backlash on the nascent Twitter et al. but all we could do was sit and watch and pray. To be fair it was overnight but a good six-to-eight hour outage affecting every service.
Of course it turned out to be DNS. All service status pages were down too, but they barely logged any problems retrospectively once they were back up.
I think things are probably a hell of a lot better now, but even ten-minute cloud outages in the US were making headlines back then (probably about 2013 or 2014).
Maybe or maybe not a conspiracy. But more or less fits in with midnight US time for the change to be implement. Just about perfect for Operations Center based in IST time. Europe is still bleary eyed and in need of a coffee.
Then needs an hour or two for the failure to propagate.
And that's about 0900 CET, 0800 UK time.
To be fair (I know... its not really done on t'internet is it?) in this case the issue was intially reported by M$ as affecting North America only, however they changed their tune when Europe woke up.
"February 7, 2023 4:48 AM..... Scope of impact: Users primarily located in the North American region attempting to access Exchange may be unable to send, receive, or search email."
"February 7, 2023 6:58 AM..... Scope of impact: Users attempting to access Exchange may be unable to send, receive, or search email. This issue is not isolated to North America."
Above quotes taken directly from the Service Health portal.
But, in general, I agree; it does appear as though Microsoft test things on "the rest of the world" rather than on home turf.
Starting my working day with ZERO email! Slack was a bit busier than normal, but never opened Teams (as the Americans aren't yet awake) so great day all around.
Seeing this article wrecks my day as I was looking forward to being productive w/o Outlook!
> If you sat down to work this morning and attempted to do something as routine as check your emails with Outlook, you'd be bang out of luck.
Where I work we have to be on-prem. Cloud isnt an option.
Sometimes I like to be a bit of a jerk so...
Everythings working fine here :D
and this is just another one of them.
MS Really needs to be sued into the next century for their inability to deliver a service that customers pay for. Who will be first to file suit?
If you do then you'd better have a plan to move away from all MS products ready to go.
Proudly MS software free since 2016.
"If you do then you'd better have a plan to move away from all MS products ready to go."
Or just backtrack all MS products to the pre-cloudy versions - I'm happily running older MS products direct off my boot drive (and behind a suitable hardware firewall) and hardly ever lose any work time (mores the pity':-( ) due to some borked MS patches that have not been properly validated prior to being pushed out onto their servers for use by those who've paid through the nose for a 24/7 service.
About that, who would be fool enough to install a patch, any patch, from any vendor, in the first week after release?
You just wait for the beta testers to install it, watch the fallout in the forums and, if all seems OK(ish), you bite the bullet. If the forums go nuclear you just wait for the patching patch. Rinse and repeat.
"MS Really needs to be sued into the next century for their inability to deliver a service that customers pay for. Who will be first to file suit?"
No-one? I mean, seriously, it seems no matter how much Microsoft messes up, businesses still voluntary put their existence onto Microsoft's creaky infrastructure.
Microsoft has zero incentive to sort out their problems.
Sendmail is acknowleged to be have its problems but once configured tends to run and run. Other mail servers such as Exim, Postfix, Dovevot are known to be easier to configure and also just run and run. Combine these with rock solid, stable OSes, such as FreeBSD, and you can have systems that run happily run for decades without a break.
Then you have Exchange…
This post has been deleted by its author
extra "Outlook.com functionality such as Calendar APIs consumed by other services such as Microsoft Teams are also affected."
Is this parenthetical, or is this what caused the outage? It seems suspiciously detailed when they are still investigating. Anyway we can certainly expect plenty of Teams-related outages as that bag of poo gets more integrated with 365-Office/Microsoft/Whatever they are calling it lately.
only thing coming up is the following for Azure
Summary: Starting around 20:19 UTC on 7 February 2023, a utility power surge in the Southeast Asia region tripped all of the chiller units for one datacenter offline. While working to restore the chiller units, temperatures in the datacenter increase so we are proactively powering down compute, storage and networking resources to avoid damage to hardware. All impacted infrastructure is in the same datacenter, within one of the region’s three Availability Zones (AZs). Downstream services that have been identified as impacted include Azure App Services, Azure Backup, Azure Cosmos DB, Azure Database for MySQL & flexible server, Azure Database for PostgreSQL & flexible server, Azure Log Analytics, Azure Red Hat OpenShift, Azure Search, Azure SQL Database, and Azure Virtual Machines (VMs).
I.e. due to dodgy power and the chiller units not being on filtered power, they dropped out and now the fan heaters in the the server room are driving the temperature through the roof. Even with the doors open and large industrial fans.
So this is affecting South East Asia,
All impacted infrastructure is in the same datacenter, within one of the region’s three Availability Zones (AZs).
I know that this horse bolted years, if not decades ago, and the stable has fallen into ruins, but I hate the use of impacted to mean affected.
Unfortunately, I tend to think of it's medical meaning first. Quoting from the wiktionary definition of impaction
(medicine) A solid, immobile bulk of stool.
Hence, the phrase "an impacted bowel".
So whenever I see a particularly dreadful use of impact for effect, when what is meant is affect, my thoughts are directed towards "A solid, immobile bulk of stool.", which is probably not what the people writing impact want to conjure up.
Thankfully, being old and grumpy is a temporary condition. Soon I shall merely be grumpy.
This post has been deleted by its author