back to article Microsoft Teams starts February with a good, old-fashioned TITSUP*

Microsoft's Slack-for-Suits collaboration platform has decided that Mondays aren't for it and has gone back to bed, much to the distress of Office 365 customers around the world. Problems began at around 1pm, rapidly reaching a crescendo of wailing from users unable to while away their day on chat channels and forced instead …

  1. Aladdin Sane

    Latest twitter update says an authentication certificate has expired, which seems slightly amateurish.

    1. MatthewSt
      Boffin

      Who among us here hasn't let a certificate expire on a service your business depends on?

      1. Michael

        Do I get to cast the first stone?

        Now, I'm not saying I've never had software issues. Certificates, however, have never been an issue.

        1. Korev Silver badge
          Joke

          Re: Do I get to cast the first stone?

          I got my 10 metres swimming certificate in 1986, when do I need to apply for renewal?

          1. Aladdin Sane

            Re: Do I get to cast the first stone?

            BSC or SSC?

            Smeghead.

      2. NoneSuch Silver badge
        FAIL

        My predecessor did. I have his job now.

  2. jon909

    This is a shockingly bad break-down in process control on Microsoft's part.

    And how long will it take them to get the certificates renewed? Place your bets, place your bets.

    1. Anonymous Coward
      Anonymous Coward

      Back up and running. Didn't take long once they recognized what the issue was.

      Process breakdown indeed. A service such as this should have well defined processes and things like the expiration date of a mission critical service should be on the auto radar........

      1. Michael Wojcik Silver badge

        Indeed.

        When you get the certificate, you know when it expires. Perhaps Microsoft should sell some sort of software that provides a calendar with a reminder feature?

        You can check the certificate's expiration date at any time, since it's part of the certificate.

        You can trivially automate the process of checking a certificate's expiration date. There are any number of tools which do this.

        You can easily automate the process of renewing certificates before they expire.

        This is not the first time that Microsoft has been caught out by an expired certificate for a public service. Over the years they haven't been able to establish corporate policy and tooling to prevent this?

        1. Claptrap314 Silver badge

          Over the decades, haven't they established a corporate reputation to downhold?

  3. jon909

    Still no official advice from MS

    BTW there's still no advisory on the service's health in the Office 365 admin cpanel.

    1. Mr Humbug

      There is one on the Service Health page in the Office 365 admin portal. It says it was opened at 2.11pm GMT and it was definitiley there when I looked at 2.30pm

  4. Sykowasp

    Amateur Hour 365

    Certificate renewal reminders shouldn't be a problem in 2020. Amateur hour from Microsoft here.

    Stick your certificates into your asset management system (or whatever you use), mark them critical, and ensure that the asset expiry alerts (use a system that does this) are acted upon.

    You can't rely on some certificate authority renewal email going to the fired-ex-manager's email address and being lost. Even worse is that I'm sure that the certificate was probably issued by Microsoft themselves in this case.

    1. a_yank_lurker

      Re: Amateur Hour 365

      And Slurp wants us to sign up for their various cloudy offerings but they cannot manage a certificate. That should make one pause about using Slurp, it is below amateurish, an amateur would be aware that a certificate needs renewal.

    2. Pascal Monett Silver badge

      Re: Amateur Hour 365

      Absolutely right. Total lack of professionalism there.

    3. Fonant
      FAIL

      Re: Amateur Hour 365

      I use Nagios to double-check all my certificates on a regular basis. Warning with a month to expiry, critical with less than a week left. Just in case automatic renewal fails for any reason.

      Not rocket science...

    4. macjules

      Re: Amateur Hour 365

      Do you know that in Azure you can do exactly that? A bit weird that MSFT do not follow their own platform guidelines.

      1. Claptrap314 Silver badge

        Re: Amateur Hour 365

        Ahh, you summer's child.

        I pointed the "Progam Files" registry entry at my new hard drive. Guess which vendor was the ONLY vendor to get it wrong?

    5. nematoad
      Windows

      Re: Amateur Hour 365

      "We're reviewing systems data to determine the cause of the issue."

      Might I suggest it's because it is being run by Microsoft?

      As others have said a kindergarten level of competence on display.

  5. N2

    OK

    Lets team up and fix this...

    Lets not, just sort your own shit out cupcake

  6. Anonymous Coward
    Anonymous Coward

    Executive Hubris

    A series of non-IT execs shoved us into the O365 cloud so they could "collaborate better" without considering the down sides. They gave up an on-premise solution with a 10GB network fiber backbone, 1GB to the desktop with full blown QoS and an exceptional up-time rating.

    During the meetings, our VP told the rest of the problems inherent to cloud computing and one of the other Finance VP's wafted his hand dismissively stating, "It'll be fine..."

    That same VP was just pounding my desk an hour ago screaming for me to sort things out. An Excel document he needs for this afternoons Board of Directors meeting is in the cloud and currently inaccessible. He has just been introduced to the realities of cloud computing.

    1. robidy

      Re: Executive Hubris

      You sound bitter...maybe move on.

    2. Claptrap314 Silver badge

      Re: Executive Hubris

      Depending on the laws in your state, perhaps that moment should have been tape recorded? Playbacks are hell...

    3. WolfFan Silver badge

      Re: Executive Hubris

      I always mark important files and folders as "Always keep on this device"... if I put them on cloudy stuff in the first place, I usually park that kind of thing on a local share on a NAS. I usually have a copy of everything that's on cloudy stuff, OneDrive, iCloud, DropBox, GoogleDrive, etc., on thumb drives and/or external spinning drives somewhere or on a NAS, or some combination. (I have this peculiar liking for backups. Strange, I know.) I might have some items on more than one cloudy thing. I definitely have stuff in more than one local store. Some of the local stores might be ancient history stuff, such as tape, or might be external drives which, like tape, are offline until I want them to be. This makes the files there not only backed up, usually to multiple different media, but offline, making it hard to drop ransomware on them. Certain elements have stated that my 'obsession' with storage indicates that I might be a 'hoarder' and that perhaps we don't need all those copies. I smile sweetly and tell them that I'll be happy to exclude their files from the backups in future, but they will then be responsible for replacing any files which took a walk, for any reason, including someone fat-fingering something and deleting the wrong file/folder/entire volume.. For some reason there are no further comments. (Yes, someone in Accounting once killed an entire volume. This was after they insisted that they needed, they had to have, full admin access, over my objections. I repeated my objections in writing. They demanded access, in writing. Three weeks later the volume showed up empty. No, I didn't BOFH it, though I could have, I just waited for the inevitable to happen. It actually took longer than I'd thought it would. After I restored the volume from backups which I'd made despite being told to not waste the space and besides these were confidential files for the eyes of Accounting only, I revoked their admin access, and stated why, in writing, using phrases like 'gross incompetence' and 'should be a termination offence'. Accounting hates me. Someone tried to raise a stink about the fact that I'd made copies of their eyes-only files. I pointed out that without those copies they'd be re-entering every single one of them. All 375GB. Higher-higher told them to shut up.)

    4. JohnFen

      Re: Executive Hubris

      > A series of non-IT execs shoved us into the O365 cloud so they could "collaborate better" without considering the down sides.

      This was exactly how I got ensnared in the O365 nonsense as well.

      1. Mine's a Large One

        Re: Executive Hubris

        It was IT execs in our case.

        Same outcome.

    5. Doctor Syntax Silver badge

      Re: Executive Hubris

      "They gave up an on-premise solution with a 10GB network fiber backbone, 1GB to the desktop with full blown QoS and an exceptional up-time rating."

      Perhaps this is the time to turn up with a purchase order to reinstate that. Even better if you can come up with an innocent sounding project title which is a backronym for something along the lines of I TOLD YOU SO or YOU WERE TOLD.

  7. Eponymous Bastard
    Devil

    Sucks

    Where I work we're being encouraged to "embrace the modern office" which includes Teams. I fucking loathe it. We've managed without it for years. I hope it stays borked or can be further borked by some little dweeb out there. I don't even want to think about privacy when it comes to all the data Redmond can slurp.

    1. Mike 137 Silver badge

      Re: Sucks

      Guess who effectively runs all our businesses now (or at least can stop them in their tracks). We really have to consider carefully whether we can safely hand over our business management to unreliable third parties. We seem to have managed in the past without doing so and it was safer. It did, however, involve accepting greater responsibility than seems to be popular these days. The modern approach seems to be "outsource and be damned" - and one really can be if one does so.

    2. MyffyW Silver badge

      Re: Sucks

      I like Teams - my email inbox has never been so quiet

      1. Michael Wojcik Silver badge

        Re: Sucks

        Here the Teams rollout hasn't proved popular, at least with the people I work with. Email volumes haven't changed noticeably, and the Teams traffic is far less than even on the relatively unpopular RocketChat installation it replaced. There are days when there's no traffic at all on the 20+ channels I'm subscribed to, and days when it's nothing but robot traffic - announcements of people joining or leaving channels, auto-posts from CI systems, and that sort of thing.

        It's been nearly six months, and many people seem to be using it grudgingly and as little as possible, while many others seem to be successfully ignoring it entirely.

        1. JohnFen

          Re: Sucks

          > It's been nearly six months, and many people seem to be using it grudgingly and as little as possible, while many others seem to be successfully ignoring it entirely.

          That's been the pattern where I work -- Teams is ignored by most people when possible, because it does more harm than good. The big exception is the suit who demanded that we get rid of the old system in favor of O365, and particularly teams. He is constantly putting stuff on the company-wide Teams team (channel?) and then complains that nobody ever responds.

      2. JohnFen

        Re: Sucks

        I hate Teams, because Teams simply blows chunks.

        In the larger picture, though, I strongly prefer email over IM. IM demands immediate attention. Email can wait until I have a few moments.

  8. macjules

    TITSUP?

    If Teams isn't working them I might hazard that TITSUP could mean "Teams Isn't Totally Screwing UP.

  9. arctic_haze

    Another expired certificate?

    Microsoft mastered forgetting about that. It became a sort of tradition.

  10. Claptrap314 Silver badge

    Nobody should ever need more than two 9's

    of reliability...

  11. Anonymous Coward
    Anonymous Coward

    I guess they don't eat their own dog food anymore. Auto Cert Rollover, Microsoft?

  12. Robert Grant

    Slack plus Zoom

    That'll do, pig. That'll do.

  13. sanmigueelbeer
    Joke

    MS need to rename this product to "Microsoft Teams MAX -- where cloud and realities come crashing together. Horribly."

  14. JohnFen

    Damn, I missed it

    I wish I could have enjoyed the absence of Teams at work today.

  15. 0laf

    Teams Team screwup borks customers Teams Teams

    Why the fuck call it 'Teams'? It's fucking confusing

    And why back it onto fucking SharePoint so it screws up corporate retention all over the place.

    Why MS Why? Why not try speaking to your customers before rolling out your next wonder of fuckupppery.

    1. Michael Wojcik Silver badge

      Yes. This is far from the worst thing about Teams, but it's one of those persistent annoyances.

  16. mr_souter_Working

    Ah, that explains it.

    I half wondered why Teams suddenly threw a hissy fit and stopped working yesterday afternoon.

    such a shame it's working again today.

  17. Anonymous Tribble

    Certificate Renewal

    Some years ago the company I worked for had a new customer with a certificate that was coming up for renewal. They had already obtained a new certificate and sent it to me to be applied to their web server.

    I'd never dealt with certificates before, so I set up some test systems with a minimal copy of their web server (just the front page), DNS and a few other things. All in a totally isolated environment with no internet connection. I got it working on that, then told them that I was ready to apply it to their live system. The system was unavailable for about 30 seconds while I put the certificate in place and restarted Apache.

    I called them back as soon as I was done. They were astonished. Apparently their previous provider had messed up badly when they tried to renew the certificates and their system was unavailable for over two days.

    They were very happy and put in a recommendation that I should get a bonus, which, when it filtered down through the chain in our company, came out as "You did your job. So what.". Never mind that I spent half the weekend of my own time and own equipment at home, unpaid, testing things to make sure I'd get it right first time. :(

  18. hplasm
    Holmes

    On the job training...

    "You did your job. So what."

    " Never mind that I spent half the weekend of my own time and own equipment at home, unpaid, testing things to make sure I'd get it right first time. :("

    Here endeth the first Lesson.

    /ConneryVoice

  19. TomPhan
    FAIL

    It could be worse

    It took most of the day before the team investigating it at our company thought to look at any of the media to see if it wasn't an external issue.

  20. vogon00

    Not impressed..

    OK, Microsoft's latest global-bollocks product had a hiccaugh, human error as usual most likely (I assume MS don't use self-renewing certs from Lets Encrypt). Move along, nothing to see here, bullshit big business as usual.

    Who I'm not impressed with is the doof quoted in the article...Scott Hoag. Why so?

    Dialling in a fact of life no matter how you do it.. with Circuit-switched PSTN, the CS ISDN, or some packet switched VoIP stuff. The voice is the same*, it's just the address (E.164 vs. URI + Authentication bollocks) that's different - you even use a microphone and speaker!.

    If he *was* a savage, he'd be forced to physically meet someone to exchange voice communication....no PSTN, no ISDN, no internet, no VOIP, not even the two empty tins with some string. Twat obviously doesn't appreciate how lucky he is to have the options he does, or how foolish he is for relying on the latest mongo-scale populist BS that can be crippled by having one bit or date out of place.

    I don't get riled easily, hardy ever, and I don't usually do violence upon people, but in this case I desperately want to punch this total knobber in the face for the crass stupidity of his comment. He needs to count his blessings.....despite having an implied dig at Microsoft which is warranted in this state.

    Someone with a handle as techie as 'ciphertxt' should be more commenting with a more technical, less business development focus. Still, not as bad as it could be - the idiot** could have spelt it 'cyphertxt'.

    Grrrr x 10^3 ....

    * Circuit switched voice is actually better (Quality, Delay, Echo) , but not the current 'flavor of the decade'....,however I'm biased having worked on National scale circuit-switched voice and data, as well as the packet-switched versions.

    ** I've used 'twat' and 'idiot', because the adjectives I had originally selected wouldn't make it past the moderators. SOOOO ANGRY!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like