back to article Microsoft brings Azure back online

Microsoft has managed to repair its Windows Azure cloud, after an expired SSL certificate downed storage and other services for people across the world. Ninety-nine percent of the affected services have been brought back online, Redmond said early in the hours of Saturday morning, Pacific Time. "We will continue monitoring …

COMMENTS

This topic is closed for new posts.
  1. danR2
    Paris Hilton

    The cloud floats majestically away...

    Beijing is going to eat the West for lunch.

    1. Don Jefe
      Joke

      Re: The cloud floats majestically away...

      Yes. But because they're using chopsticks it will take a while.

    2. This post has been deleted by its author

  2. ElNumbre
    FAIL

    20 goto 10

    To err once is human.

    To err twice takes a super-mega-corp.

    1. This post has been deleted by a moderator

      1. Spoddyhalfwit

        Re: 20 goto 10 - MS GOTO FAIL

        It's a certificate issue... Nothing to do with windows or linux

        1. Anonymous Coward
          Anonymous Coward

          Re: 20 goto 10 - MS GOTO FAIL

          This time only with Microsoft, not Windows.

        2. This post has been deleted by a moderator

          1. 404
            Facepalm

            @Eaton -> Just stop it.

            I am dealing right now with a domain that has been hacked three times in the space of a week, on a Linux server running Apache. I have to recreate the site because the fine technical folks at this Linux-based host overwrote the backups with the hacker's new site design. Yeah.

            I'm currently going through WayBack to get an idea on the site layout (designer lost original files and the webguy who put it together literally went crazy) and found the vulnerability, a known issue with Joomla 1.5 that the host never updated.

            Now, do you really want to discuss this? Damn fools and idiots are everywhere and the use all kinds of software -> the discontinuity lies between keyboard and chair, buddy.

            1. Joe Montana
              WTF?

              Re: @Eaton -> Just stop it.

              It's not up to the host to update joomla, it's up to the individual site operator because each joomla installation is local to each site hosted.

              You can't expect the hosting provider to go through all their customers, looking for instances of joomla and then manually updating them. Depending on the level of customisation each customer has made, this could cause serious breakage.

              Whoever was supposed to manage the site (probably the webguy you mentioned) should have updated joomla, and he failed to do so.

              It seemed that Linux did its job if the hackers were only able to deface one site (the one with the hole) and not root the whole server.

              As for backups, the host keeps backups to protect against disk failure... It's not their fault that the disks did not contain what they were supposed to, they just backed up what was there. Again the site operator should have backed up their legitimate content.

              1. 404

                Re: @Eaton -> Just stop it.

                Host and site operator was one in the same - package deal paid quarterly. A 5 man operation that outsourced everything except billing - plenty of them out there. The access Joomla gave did root the server, passwords changed, directories deleted, etc. I have since xferred the domain elsewhere. The host I use and recommend sends email alerts when software used via Fantastico needs an update, has for years.

                ;)

            2. This post has been deleted by a moderator

              1. 404

                Re: @Eaton -> Just stop it.

                @Eadon -> Your Momma.

                The entire box was hacked, old unpatched Linux and Apache, operated and maintained by MUPPETS.

                The majority of Windows users/admins know how to update, the old-school Unix/Linux credo of fire and forget, only checking servers if they catch fire, has been over for some time.

                1. This post has been deleted by a moderator

                  1. 1Rafayal
                    Paris Hilton

                    Re: @Eadon

                    So your comment of "...Naturally Linux / UNIX admins are the top predators - the elite...." suggests that anyone who uses Linux in their organisation are superior to those who choose not to.

                    Does that mean you are ignoring Microsofts UNIX heritage, and possibly the fact that Microsoft uses Linux on a regular basis within its own organisation?

                    1. Vic

                      Re: @Eadon

                      > Does that mean you are ignoring Microsofts UNIX heritage

                      I certainly am, as, I believe, is Mictrosoft.

                      Xenix was a fabulous product[1] in its time. But where is it now?

                      Vic.

                      [1] I'd love to see something Xenix-derived with a Win95/XP-style skin on top. That could be a *major* seller for Microsoft, if they were to introduce it carefully. But they won't... :-(

            3. Anonymous Coward
              Linux

              Re: I am dealing right now with a domain that has been hacked three times in the space of a week

              As the song says... It all makes work for the working man to do.

              But... Linux complacency... Yes, it's stupid. Won't somebody think of the penguins?

          2. mmeier

            Re: 20 goto 10 - MS GOTO FAIL

            Oh more neckbeard FUD from the EadonVerse. Cute but repetetive.

            The WORST person for security is one that assumes "my OS is perfect". Because they are blind! That's why the typical Botnet-controller and Trojan distributers are hacked LINUX servers (typically with cheap hosters) and not Windows servers or commercial UNIX boxes. That is why hobby "hackers" deface webpages by the dozend. "Linux is perfect" and so security is taking a back-seat.

            Besides: Not renewing a SSL certificate is a major FAIL in management/procedures but at least with Azure it is NOT a security problem. MS actually got THAT part right - no one is getting at the data.

            And once you get exposed to the real world you will see stuff that is forgotten a lot. No matter what OS. Like the UNIX guys forgetting to configure a CheckPoint Firewall/One properly for use with ORACLE when a program went from "single developer" to "load testing". The idea was "quick setup now and when we go load testing in three month we have the more complex one done". Nope...

            Sure, they changed the SOP after that and all those configs had to be done fully from then on. But the fail happened. And not with a small or generic house but rather a 3000+ employee IT specialist organisation :)

            1. This post has been deleted by a moderator

              1. mmeier

                Re: 20 goto 10 - MS GOTO FAIL

                Eadon you are mixing up Security and availability. Security deals with illegal access and unwanted changes to data. Availability deals with being able to access a system. Totally different things. The SSL error means the data is not available. It is neither lost nor compromised. Security is about restricting access to those with the right permissions. Nothing more, nothing less.

                As for the rest: Unlike you I understand reality!

        3. eulampios

          @Spoddyhalfwit

          yeah, it does have to with competence though or, rather, complete and utter incompetence.

        4. Anonymous Coward
          Anonymous Coward

          Re: 20 goto 10 - MS GOTO FAIL @Spoddyhalfwit

          "It's a certificate issue... Nothing to do with windows or linux"

          Well, true, but it says something about Windows admins - best admins administrate UNIX systems ... UNIX admins are better at administrating Windows boxen than "specialized" Windows admins, in my experience anyway.

          Can't even keep a certificate up-to-date, shit can happen once - you fire the lot, but twice ???

          1. Anonymous Coward
            Anonymous Coward

            Re: 20 goto 10 - MS GOTO FAIL @AC 09:15

            Hmm. Guessing you're a UNIX admin. Probably got your job by spouting the same generalisation to your manager - just what he'd want to hear since the presence of UNIX in-house means he's in the beardy club too ...

            1. Anonymous Coward
              FAIL

              Re: Guessing you're a UNIX admin

              Guessing you don't know one programming language from another, let alone what platform you might find it on.

          2. Frankee Llonnygog

            Re: 20 goto 10 - MS GOTO FAIL @Spoddyhalfwit

            I can't believe people are still used the word "boxen".

            1. Anonymous Coward
              Anonymous Coward

              Re: 20 goto 10 - MS GOTO FAIL @Frankee Llonnygog

              I'm amazed they *ever* did. Trying to gain geek cred by showing they've heard 'Vaxen' I suppose.

          3. Alan Bourke
            FAIL

            Re: 20 goto 10 - MS GOTO FAIL @Spoddyhalfwit

            No, it says nothing whatsoever about that. Stop trying to shoehorn a 'my OS is bigger than yours' argument into everything.

        5. Roland6 Silver badge

          Re: 20 goto 10 - MS GOTO FAIL

          >It's a certificate issue... Nothing to do with windows or linux

          And with the increased use of certificates, certificate expiry issues are going to happen more often and certificate administration is something that will become more important.

          In the last few years I've already encountered:

          1. email providers who have allowed their certificate to expire and then to renew it specifying a different domain ...

          2. third-party application code signing certificates expiring.

          The worrying thing is that the wide spread use of certificates could make Enterprise IT infrastructures more unstable, particularly as most certificates seem to have a relatively short life and the user (IT operations) really gets very little visibility of the certificates being used by third-party applications and their 'health'.

          The really worrying thing is that whilst your software license may not have an end date, the failure of the OEM to update the software's certificates may render this largely irrelevant.

      2. Big-nosed Pengie
        FAIL

        Re: 20 goto 10 - MS GOTO FAIL

        This shouldn't surprise us. Their software is absolute shite - what in the world would make us think they were competent to run a chook raffle, let alone an online service?

        1. P. Lee
          Facepalm

          Re: 20 goto 10 - MS GOTO FAIL

          Meh,

          MS is fine for the SMB sector. Downtime there isn't measured in dumb ways (employee * revenue per hour * hours) and they just work around any problems.

          In a large enterprise, such as a bank, we test properly, rolling system clocks forwards and backwards to see if anything breaks, precisely because we know people forget things such as certificate renewals and the public does not forgive downtime quickly.

          My question would be, assuming they used their own certificate server, why wouldn't the server be able to check all the issued certificates and flag any issues, even if there's no code in the cloud software to check things? Not even Ctrl-M to centralise scheduling of tasks? It appears that MS' software works in spite of their procedures rather than because of them. I rather want my cloud management to be all standardised and automated, not built as a one-off.

          On the upside, I think we might see so certificate-store date checking appearing in MS software any time now, which is good for customers.

          1. Anonymous Coward
            Coffee/keyboard

            Re: In a large enterprise, such as a bank, we test properly

            Excuse me while I guffaw, spitting all over my keyboard

          2. Roland6 Silver badge

            Re: 20 goto 10 - MS GOTO FAIL

            >why wouldn't the server be able to check all the issued certificates and flag any issues

            A very good point - interesting that in Windows 7 expired certificates only get flagged on screen and are not reported to the event log...

            I don't know what Unix/Linux does but I would hope it would get logged and hence picked up by the monitoring system.

      3. Anonymous Coward
        Anonymous Coward

        IBM Too

        IBM's "state of the art" Cloud data centre in Auckland was completely down for 30 hours+.

        No public expanation.

        (after a while the newspaper shut up about how long it was, and just started tell us how many racks it had, and how many backup generators. Edvertorial anyone?)

        1. Adrian Harvey

          Re: IBM Too

          There was some lack of detail in the media - the data center itself was not down - my customer has equipment housed there and it was fine. I heard about their cloud having issues - but don't know what they were.

          And no, I don't work for IBM...

      4. Don Mitchell

        Re: 20 goto 10 - MS GOTO FAIL

        This is a classic example of an operations-management failure. It's got nothing to do with the OS. And if you think only Microsoft has failures, you probably missed hearing about when Amazon's load balancing system brought its cloud service down a while ago.

        Amazon in general has done a great job at operations. One manager there told me, "We are experts at dealing with emergancies, because we use Linux." The thing that is more difficult for Amazon, because they lack the systems engineering culture, is to develop complex software systems. So they provide the number 1 cloud service, but they can't offer higher level services like instant e-commerce packages (like MS Dynamic).

        Eadon, if you think Dave Cutler doesn't know how to design an operating system, or if you think Linux never fails without a lot of tweeking and patching, then I gotta wonder why you feel so passionate about a subject you don't actually know much about?

        1. Sandtitz Silver badge

          Re: 20 goto 10 - MS GOTO FAIL

          "Eadon, if you think Dave Cutler doesn't know how to design an operating system, or if you think Linux never fails without a lot of tweeking and patching, then I gotta wonder why you feel so passionate about a subject you don't actually know much about?"

          These age old proverbs come to mind:

          - Empty barrels make the most noise

          - It is better to remain silent and be thought a fool, than to open your mouth and remove all doubt.

          1. Anonymous Coward
            Anonymous Coward

            Re: 20 goto 10 - MS GOTO FAIL @Sandtitz

            Your proverbs would sound less trite if they were backed up with counterpoints rather than being presented as the argument themselves.

        2. This post has been deleted by a moderator

          1. Phoenix50
            FAIL

            Re: 20 goto 10 - MS GOTO FAIL

            "Eadon sees deeper"

            Talking about yourself in the third person?

            Thread reply FAIL.

            1. Anonymous Coward
              Anonymous Coward

              Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

              ""Eadon sees deeper"

              Talking about yourself in the third person?

              Thread reply FAIL."

              I've seen a few people with runaway egos refer to themselves in the third person. I always wondered why, but it just occurred to me that it's probably how they think people are thinking of them. Or maybe are hoping that it looks more like something that's been said by others, in the hope that it'll become some kind of text-based ear worm. Simple (self-) marketing tactic.

              Interesting how he also argues that it's still a WIndows problem. "Sort of." And then goes on to expannd it into everyone who uses Windows is an idiot who doesn't understand anything. He had seemed to be posting more reasonably over the last week, and seems to have regressed since Friday. Gives credence to the jokes about him missing his meds :)

              1. This post has been deleted by a moderator

                1. Anonymous Coward
                  Anonymous Coward

                  Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

                  "Even if I use the odd tongue-in-cheek stylistic quirks"

                  What, you mean like when you claimed to have fucked my mum and she asked for more?

                  I actually wish the moderator had left that one up, so everyone could see what an odious runt you are.

                  Still we do have standards here. And no, I am never going to let you forget this.

                  1. This post has been deleted by a moderator

                  2. Anonymous Coward
                    Anonymous Coward

                    Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

                    Wasn't "The Stylistic Quirks" the name of an early 80s punk/synth fusion group?

                  3. Vic

                    Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

                    > I am never going to let you forget this.

                    You should do. Some of us hadn't seen that particular missive. Now you've republicised it, the Streisand Effect will tend to take over :-(

                    Vic.

        3. Anonymous Coward
          Anonymous Coward

          Re: We are experts at dealing with emergencies, because we use Linux

          Put it another way: they live in the real world, rather than a world in which what will and won't happen is laid down by the marketing department.

          However, yes, this was am administrative affair, not an x-os vs y-os thing. That a company like MS ---isn't it supposed to be entirely up its own backside with procedural stuff?--- could let this happen is ridiculous, but not surprising. About ten or twelve years ago, didn't they forget to renew their own domain? IIRC, some fairly junior employee saved the day, out of hours, by making the payment out of his own pocket. No doubt I have some details wrong: the story will be in the Reg archives.

          Procedural and management failure. Really, does it take an MBA to get this sort of stuff right? Don't they have people that actually count beans? Absurd. One of my employers nearly had the power cut off because the ex-sales-side chief thought he had better things to do than to pay the bill --- or pass the job to someone else. How can people be so blind to the fact that the first necessary skill or running a company, of any size, is basic, every-day, bill-paying, floor-sweeping, toilet-cleaning, admin?

          On the MS-Software vs Anything-Else front: No it wasn't a software failure, but the recovery process could be a valid comparison. If I was going to put up a penguin icon for this post, and then talk through its backside, I might suggest that, with *nix, it might have been a fifteen-minute job editing a text file. But I would be talking though the penguin's backside. Anyway, I have no clue about the actual problems they faced, so better not talk out of my backside. But hey, who can resist? I never did, and I never would, trust my data to MS software. And, anyway, trusting to "cloud" storage is the silliest idea ever, whatever the architecture.

        4. Anonymous Coward
          Anonymous Coward

          Re: 20 goto 10 - MS GOTO FAIL - Don Mitchell

          But, but Eadon is generally right.......

          The management of Microsofts only talent is repackaging old software and adding stupid functions to it,

          All the software with it's robustness, falls over like a card house in a wind storm.... with ONE little virus...

          Oh I mean Microsoft kept Port 80 Open for 15 years after it was identified as a security risk etc., etc., etc...

          And they run a 24/7 spy on you network, from their software....

          Your already being monitored in real time, all over the internet, from WITHIN Microsoft Widows.

          2001 and well before:

          Microsoft's Really Hidden Files

          http://sillydog.org/mshidden.php

          Look up this file:

          Windows 7:

          EntriqMediaTray.exe

          The process known as Media Server Tray Application belongs to software Entriq MediaSphere or MediaSphere by Entriq (irdeto.com).

          Description: The file EntriqMediaTray.exe is located in a subfolder of "C:\Program Files". Known file sizes on Windows 7/XP are 360,448 bytes (33% of all occurrences), 368,640 bytes or 372,736 bytes. http://www.file.net/process/entriqmediatray.exe.html

          The file is not a Windows system file. The program is not visible. The program is loaded during the Windows boot process (see Registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run). EntriqMediaTray.exe is able to monitor applications. Therefore the technical security rating is 35% dangerous.

          In case you experience problems using EntriqMediaTray.exe, you can remove the entire program using Windows Control Panel.

          Which feeds data off your windows system to here:

          http://irdeto.com/

          And they do this shit:

          http://irdeto.com/anti-piracy-services/tracking-and-enforcement.html

          Blah blah blah blah................. sun shines out our arses etc. Blah blah blah blah.................

          Irdeto Intelligence is the industry-leading solution to identify and track unauthorized digital content across all major Internet protocols including user-generated content (UGC) hosting sites, cyberlockers, peer to peer networks, IRC, Usenet groups and public FTP sites. On average Irdeto Intelligence processes 950 million detections that create over 35 million actionable events each month for its clients.

          Irdeto Intelligence tracking services include:

          P2P chart

          Sample P2P report

          Peer-to-Peer (P2P) Monitoring - the industry’s leading P2P platform for monitoring, reporting and enforcing copyright

          Scans leading P2P networks, including: Bit Torrent, eDonkey/eMule, Ares and Gnutella to identify individuals who upload client content

          Collects identifying information on the first uploaders, tracks propagation and can provide data for evidence packages in the event of possible litigation

          Includes tracking by asset, file source, language, user origin and breakouts by unique users and downloads.

          Compliant with MPAA file verification standards

          Blah blah blah blah.................

          Infringement Notices - Irdeto sends more than eight million Takedown Notices monthly on behalf of clients and monitors for compliance, providing reports to copyright holders on who has and who has not complied.

          Microsoft's entire history is of spying on all people, through a whole range of methods.

          Fuck the Peeping Tom Software Co.

          1. Anonymous Coward
            Anonymous Coward

            Re: 20 goto 10 - MS GOTO FAIL - Don Mitchell @Oh4FS

            TL;DR. Though scrolling through the message to see just how long it was, the repeated use of 'blah blah blah' was beautifully poetic. (Not ironic, since there's no contradiction there.) Congrats on effectively getting your point across. Still, I'm sure Eadon read it. I'll look forward to having to scroll through all that stuff again, interspersed throughout his future posts ...

        5. BrentRBrian
          Mushroom

          Re: 20 goto 10 - MS GOTO FAIL

          "The more they overthink the plumbing, the easier it is to stop up the drain." ... Montgomery Scott

      5. TheVogon
        Mushroom

        Re: 20 goto 10 - MS GOTO FAIL

        This sucked, but it's a process failure - not a technology or an OS one. It will be interesting to see the RCA and understand how they made such a cock up...

        Small bit of pedantry, but Azure storage was actually down for 9 hours rather than the 12 claimed in the article.

    2. This post has been deleted by its author

    3. Androgynous Crackwhore
      Thumb Up

      Re: 20 goto 10

      I thought Windows Azure is intended to be a sort of proof of concept technology advertisement. A carefully constructed and managed demonstration run by the corporation which creates and sells the underlying software - for the purpose of illustrating to the world what can be expected of it.

      It would appear to be fulfilling its role rather well.

      Too complicated for even its own super-mega-corp creator to operate? Splendid, I'll have some of that. Where do I sign?

    4. JeffyPooh
      Pint

      Re: 20 goto 10

      Their Certificate Expiry Tracking program hasn't yet been ported to Windows 8.

    5. Anonymous Coward
      Anonymous Coward

      Re: 20 goto 10

      REM Microsoft business model

      REM (c) 1975 by Microsoft Software Co.

      10 INPUT OPPORTUNITY$

      20 INPUT CLIENT$

      30 SET OUTPUT=CLIENT$

      40 PRINT "Microsoft can do that"

      50 PRINT "We've had a project like that for years"

      53 PRINT "We can deliver something better in half the time for half the cost"

      60 INPUT BUDGET$

      70 INPUT SCHEDULE$

      80 SIGN CONTRACT$

      90 GET BUDGET$

      100 PAUSE

      101 NEW PROJECT$

      102 PAUSE

      107 LET SCHEDULE$ = SCHEDULE$ + SCHEDULE$

      110 PRINT "Microsoft Software Inc is leveraging new technologies for expediteization of revolutionary solutionization"

      140 LET BUDGET$ = BUDGET$ + BUDGET$

      150 PAUSE

      160 PRINT "Microsoft "; OPPORTUNITY$ ; " is the next generation solution to leverage enterprize monitization"

      170 LET SCREENSHOTS$ = RANDOM

      180 PRINT SCREENSHOTS$

      200 LET SCHEDULE$ = SCHEDULE$ + SCHEDULE$

      210 PAUSE

      220 LET BUDGET$ = BUDGET$ + BUDGET$

      230 GET BUDGET$

      240 GET SMALL_COMPANY_EXPERIMENTING_IN_FIELD$

      250 PAUSE

      260 LET PROJECT$ = SMALL_COMPANY_EXPERIMENTING_IN_FIELD$

      270 PAUSE

      280 RANDOMIZE PROJECT$

      290 RANDOMIZE PROJECT$

      300 PRINT "Microsoft Software Inc, the worlds leading enterprize solution provider announces "; OPPORTUNITY$ ; " Pro XP 3000"

      310 DUMP PROJECT$

      320 GOTO 10

      RUN

  3. Bob Vistakin
    Devil

    Microsofts "customers" get Ballmered again

    Having stuffed them on phones, tablets and their comedy new desktop OS, destroying their faith in their cloud offering is an essential step in the slow car crash suicide that is 2013: Microsofts year of hell.

    1. AchimR
      Pint

      Re: Microsofts "customers" get Ballmered again

      "comedy new desktop" - love it!

      beer for ya

    2. Anonymous Coward
      Anonymous Coward

      Re: Microsofts "customers" get Ballmered again

      Ballmer can get billions for slow-crashing the company into the ground, the operations guys are just following his lead. Corrupt CEO maybe? Ballmer and Beeblebrox.

  4. This post has been deleted by a moderator

    1. Anonymous Coward
      Facepalm

      Re: Microsoft CLOUD EPIC FAIL

      "Obviously Microsoft just bung a framework on some windows servers and call it a "Cloud"."

      Putting lipstick on a pig has been Microsoft's entire philosophy since Windows 1.0. If it ain't broke...

      Oh , wait...

      1. Androgynous Crackwhore
        Pirate

        Re: Microsoft CLOUD EPIC FAIL

        Putting lipstick on a pig has been Microsoft's entire philosophy since Windows 1.0. If it ain't broke...

        Only since Windows? Have you forgotten the whole PC-DOS/MS-DOS fraud the outfit cut its teeth on? I've never figured why IBM brought Microsoft into OS2 after being so completely shafted in the DOS débâcle. Fool me once... Got exactly what they had coming I suppose. Poor stupid IBM.

      2. Libreated
        Trollface

        Re: Microsoft CLOUD EPIC FAIL

        Love it. Yes, of course it's the lipstick on the pig. Microsoft's heritage has been to buy products with massive potential and then do a 'cut and shut' on them. The exceptions are maybe Exchange and SQL server (I like to call them 'princess products' since the product teams don't give a crap and have more talent than the rest of the giff-gaff.).

        I've been working with MSFT 'technology' for 21 miserable years and found that 'predictable' and 'reliable' are considered by the company in the same breath as 'delinquent' and 'that's all folks'. I've presided over (unfortunately) Certs expiring on patch management infrastructure, meaning my clients' ability to secure their systems is compromised by Cert 'features' - a complete non-seqitur.

        I agree with the general sentiments in this thread - if you want a robust cloud, buy one based on very old and very predictable technology - Linux/Unix.

        Oh and for what it's worth, Azure is considered by most enterprise as mickey-mouse anyway so good luck to the MSFT sales force on the back of this one!

    2. Anonymous Coward
      Anonymous Coward

      Re: Microsoft CLOUD EPIC FAIL

      Whilst this was an undisputed cockup by Microsoft this all hail FOSS stuff is getting a bit annoying. It's got to the point where the zealotry is making people think Linux is a bastion of security & stability that Jobs tried to get people to think the Mac was. Then Mac starts getting popular and malware authors turn their attention to it. As icing on the cake people think they are safe and sound because of the "Macs don't get viruses" claptrap.

      As more inexperienced people are flocking to FOSS based alternatives you are helping them stick their head in the sand to the issues that face any system. It's damaging and will lead to people not taking the security & stability of their systems seriously.

      Case in point, I saw a post by someone the other day that stated that Linux doesn't get viruses because they have nowhere to hide as it's open source software (He then went on to insist other people only state facts about Linux or shut up, but I digress). More and more new people are heading to Linux and people like you, Eadon, are making it out to be something that it's not.

      And to finish off I will echo another poster, this has nothing to do with the operating system on which the service runs. It was organisational incompetence at management level.

      1. mmeier

        Re: Microsoft CLOUD EPIC FAIL

        Forgetting to re-new your own SSL certificates really is an EPIC fail. But it has nothing to do with operation systems, depending on who was responsible maybe not even with IT (I have seen cases where the secretary forgot to "pay"/get signed an internal bill and services where shut down...)

        Claiming "This OS is perfect" is a reason NOT to hire someone as an admin. A good admin KNOWS and ACCEPTS that his OS of choise has problems/weaknessses etc. Then he finds ways around them if possible and admits "take something else" in other cases. That goes double for security and availability

        The best example is a Siemens engineer stating flat out: "The system can not do that. Export the data and use another OS to generate those graphics/reports"(1) He than made recommendations and Siemens got hired for writing an export/import software.

        (1) The guy was a long time developer of that system and really liked it. Oh before Eadon throws another evil Windows fit - we used SCO Unix based software on X-Terms back then in 1990.

      2. Joe Montana
        WTF?

        Re: Microsoft CLOUD EPIC FAIL

        Linux is already popular as a server platform, and has been for many years...

        It is tried and tested, and while not perfect has a much better track record than windows, which has a number of serious design flaws.

        While it's true that users could misconfigure any system in an insecure way, it is much more likely for such a configuration to arise on windows than linux.

        1. Anonymous Coward
          Anonymous Coward

          Re: Microsoft CLOUD EPIC FAIL

          @Joe Mantana

          I have to disagree, Windows vulnerabilities are much more likely to be the result of software bugs than misconfigurations. Linux software still relies heavily on text based configuration which would be daunting to most users.

          I'm not sure what it offers in the way of firewalls these days as I haven't used a Linux distro for about 6 months but there was very little in the way of easy to use configuration tools. Let alone zero configuration possibilities. And IIRC the *ubuntu iptables was allow all by default.

          1. mmeier

            Re: Microsoft CLOUD EPIC FAIL

            AC: There is "privat use" and "company use"

            For privat use Windows, MacOS or even iOS and Android, are the better choice for 98+ percent of the users. Zero configuration and coupled with the typical DSL box (that has rather restrictive firewalls set up(1))) it simply works.

            A company that exposes parts of it's network to the outside hopefully uses a proper DMZ architecture and firewalls that are NOT on the exposed servers but rather independent systems. With those and staff that knows it's job OS does not matter much. You do not read much about hacked Exchange or Domino servers. The exploits on this level aim at errors in the Application servers, Skripts etc(2)

            (1) At least german Telecom, UnityMedia and Vodafone units are basically "all closed except HTTP(s)" and "open only one port at a time"

            (2) And rarely social engineering. That is more an end user than a administrator problem.

          2. eulampios

            @AC

            Windows vulnerabilities are much more likely to be the result of software bugs than misconfigurations....

            So what? Remind us again why do Microsoft strongly recommend to use antivirus software, while either Ubuntu or any other GNU/Linux vendor do not?

            1. Anonymous Coward
              Anonymous Coward

              Re: @AC

              "So what? Remind us again why do Microsoft strongly recommend to use antivirus software, while either Ubuntu or any other GNU/Linux vendor do not?"

              Because Linux on the desktop isn't popular enough for malware authors to put any serious effort into targeting the platform. This in turn means that anti-malware corps don't consider it commercially viable to make a serious effort to support it. If Desktop Linux becomes popular, as folk seem to wish it to be, malware authors will target it, hopefully anti-malware corps wont catch on too late.

              FOSS that runs on Linux, such as web browsers, have had their fair share of security problems too, and with no anti-virus solution to catch intrusions it then this could be a problem if malware authors start to make a focused effort.

              Oh, and if you are one of the folk under the impression that Linux has no viruses then allow me to direct you to this rather aptly timed rootkit discovery http://www.itwire.com/business-it-news/security/58832-sshd-rootkit-in-the-wild.

              1. eulampios

                (Re: @AC)x2

                Okay, first off let me ask you, AC, why hiding behind a mask?

                Isn't it because you're affiliated to the celebrated company we're commenting about and which should shut the fsck up to properly enjoy all this humiliation?

                Now you can see that wasting money and soul on pretty dirty campaigns like patent trolling, "droidrage", "scroogled" and other extravagant spendings is not justified at the end. Microsoft would not be so vigorously ridiculed these couple of days if they had just hired a single competent expert, not another manager or lawyer. It would only take one sane ssl brain!

                As to what you're trying to allege now, I've done hundreds of times already and will repeat this for you again, the unknown friend.

                When you and others try to surmise about someone's theoretic interests, you should prefix it with IMO (or better IMHO). Since you're making a statistical supposition that the risks and popularity are correlated, so tell us about the nature of this correlation. Is it linear ?, polynomial? logarithmic? what are the corresponding coefficients? Yet, I remind you again that, Microsoft strongly recommend AV software. On ANY setup, be it a specialized server or desktop. Now, according to netcraft, non-windows setups facing Internet are the majority. So there should be an explanation in your theory shedding some light on this paradox as well.

                Yes, Linux, BSD and other servers do get hacked due to

                ** weak ssh policy

                ** insecure cgi applications, mostly php-based

                ** some ad-hoc sql incompetent proprietary products

                No, AV is a bad idea. MAC is a much more professional solution against 0days and other threats.

                .............

                1. Anonymous Coward
                  Anonymous Coward

                  Re: (@AC)x2

                  "Isn't it because you're affiliated to the celebrated company we're commenting about and which should shut the fsck up to properly enjoy all this humiliation?"

                  Didn't read past that point. If you can't make your arguments stand without resorting to paranoid accusations then kindly go somewhere else.

                2. Sandtitz Silver badge
                  FAIL

                  Re: (@AC)x2

                  "Okay, first off let me ask you, AC, why hiding behind a mask?"

                  Eulampios, please state your full name and the company you work for.

                  Otherwise you are just another Anonymous Coward yourself

        2. mmeier

          Re: Microsoft CLOUD EPIC FAIL

          UNIX is a good, solid server platform. Linux profits from that and can be secure IF your admin knows his stuff. OTOH it is more difficult to judge capabilities in a Linux admin than in a Solaris/AIX/HPUX/... admin since there are so many variants of Linux and knowing one does not mean knowing all. For reasons like that Linux is not the platform of choice for many companies either

          As for Windows - It had some spectacular fails on the server side when exposed to the web. And I would still be reluctant to run IIS on Windows server and expose it to the internet. OTOH the customers use SOLARIS for that so I do not have to. Windows is "internal network" stuff and there it shines when you have qualified admins. And again - quality is easier to check

  5. TJ1
    WTF?

    Monitoring for 24 hours...

    "We will continue monitoring the health of the Storage service and SSL traffic for the next 24 hours," the company wrote.

    And then?

    That statement makes it sound like they don't monitor the systems routinely unless something fails; explains alot!

  6. Richard Lloyd
    FAIL

    Cron job needed and, er, why didn't they renew the cert for longer?

    OK, so this happened last year, presumably on the annual renewal date. It begs how incompetent Microsoft is:

    1. Most secure cert registrars send out e-mail reminders (mine does with 90, 30 and 7 days to go) - did whoever they registered with not send such e-mails or did Microsoft just ignore them?

    2. A simple cron job to check the cert and e-mail (to more than one person!) every day at least 7 days before expiry would have saved their bacon.

    3. When they messed up last year, why didn't they renew the cert for more than one year? Surely Microsoft can afford a multi-year cert?!

    Multiple levels of incompetence there - that's Microsoft for you.

    1. Anonymous Coward
      Anonymous Coward

      Re: Cron job needed and, er, why didn't they renew the cert for longer?

      Suppose the real reason is something else but the cert.

      1. Anonymous Coward
        FAIL

        Re: Cron job needed and, er, why didn't they renew the cert for longer?

        The people directly involved probably did know, but then had to find out who to bribe/blackmail into procuring a new certificate before the old one went belly up.

        Some of these big companies have an unbelievable amount of red tape and disparate divisions which are so out of touch with each-other they may as well be separate companies.

      2. Frumious Bandersnatch

        Re: Cron job needed and, er, why didn't they renew the cert for longer?

        re: Suppose the real reason is something else but the cert.

        I suppose they should probably stop calling them "certs" then, if they aren't.

      3. P. Lee
        Coat

        Re: Cron job needed and, er, why didn't they renew the cert for longer?

        Perhaps they weren't expecting the service to last that long...

    2. Anonymous Coward
      Anonymous Coward

      Re: Cron job needed and, er, why didn't they renew the cert for longer?

      Real enterprises don't use local schedulers, rather centralised scheduling. Anyone who thinks cron or windows scheduler is an acceptable solution in an enterprise, still has a lot to lean about IT.

      1. Anonymous Coward
        Anonymous Coward

        Re: Cron job needed and, er, why didn't they renew the cert for longer?

        "Real enterprises don't use local schedulers, rather centralised scheduling. Anyone who thinks cron or windows scheduler is an acceptable solution in an enterprise, still has a lot to lean about IT."

        Agreed. In this case even a good old calender reminder would have avoided this, though.

        It really doesn't matter how they schedule things, as long as they do it, which apparently wasn't the case here.

        "The global outage lasted for around 12 hours and occurred because Microsoft failed to renew a security certificate."

      2. Hans 1
        Boffin

        Re: Cron job needed and, er, why didn't they renew the cert for longer?

        So true!

        They could have used RunMyJobs (runmyjobs.com) which is a cloud-based scheduler that leaves your data in your data centers and does not rely on Microsoft to get their cert renewed ... it runs on Amazon, iirc.

  7. Robot

    Act 1, Scene 1

    [face in hands]

    [throws a chair against the wall]

    [does a bear scream]

    [throws a chair against the wall]

    [wrings an imaginary neck]

    [throws a Surface Pro against the wall]

    1. This post has been deleted by a moderator

  8. Paul Crawford Silver badge
    Trollface

    So how to customers get refunded?

    I guess this single outage trashes their SLA promises of uptime over the coming year, so what happens now for the customers? Do they get a year's refund or free?

    Yes, I know its online somewhere but could not be arsed looking up the legalise...

    1. Yet Another Anonymous coward Silver badge

      Re: So how to customers get refunded?

      No "acts of idiot" are specifically excluded from the SLA

  9. Anonymous Coward
    Anonymous Coward

    Marketing buzzwords

    All this talk of how Cloud will revolutionize business and yet nobody describes exactly what it is.

    It is merely an internet computing Cluster which can be adapted to suite power required.

    It will save you from buying your own Cluster hardware and managing it but requires paying for by the hour with the loss of data security. Your Cloud vendor has access to your valuable IP and customer data.

    An advancement from Shared Plesk website hosting, Virtual/Dedicated Servers. You can upload your own machine image or build for the Cloud.

    Some allow SQL others LINQ and Hadoop.

    Hadoop may be fast but this comes at a cost of omitting required DBMS features. A DB change could result in a lot of code rewrite. Is it Transaction safe and does it allow online backups? When you add all the features of a DBMS it quickly comes down in speed.

    As for LINQ, is it really much simpler to use than SQL. You could suffer an unnecessary performance penalty.

    The marketing folks pushing Cloud need to understand its not the magic bullet that the industry desperately needs.

  10. kitreno
    Go

    monkey magic

    that monkey sounds useful on his cloud

    http://www.youtube.com/watch?v=K2huJqFsFDE

  11. Anonymous Coward
    Anonymous Coward

    Oh FFS!

    How hard is it to write some code to check certificate expiry dates?

    1. Anonymous Coward
      Anonymous Coward

      Re: Oh FFS!

      How hard is it to make a browser/shell allocate itself no more cache than it can actually manage and thus prevent it from slowly killing the sheeples systems?

  12. Anonymous Coward
    Anonymous Coward

    The big intro text on their website homepage says...

    "A rock-solid cloud platform for blue-sky thinking"

    I hope they're doing that to be ironic.

    1. Captain DaFt
      Coat

      Re: The big intro text on their website homepage says...

      ""A rock-solid cloud platform for blue-sky thinking"

      I hope they're doing that to be ironic."

      Well, Looks like a fair description to me. When the cloud drops like a rock, there's nothing left overhead but empty. blue skies

      1. This post has been deleted by a moderator

  13. Anonymous Coward
    Anonymous Coward

    Cloud is like the recent beef/horse food chain debacle.

    Lots of managers looking around saying "Why would I want to pay to maintain a known chain of suppliers, a whole lot of equipment, and staff, when I can just buy it cheaper through a long chain of third party suppliers, and I won't have all that work managing all them things"... "think about how big my bonus will be when I deliver all them savings".

    1. Yet Another Anonymous coward Silver badge

      Are you sure you aren't thinking of the 787 battery debacle

  14. Anonymous Coward
    Anonymous Coward

    The Microsoft Secure Professional PowerCloud XP Enterprise 3000 ...

    ... has encountered an unexpected error and must now close.

    Try reinstalling the Cloud, and then reboot.

    1. This post has been deleted by a moderator

    2. Anonymous Coward
      Anonymous Coward

      The Microsoft Secure Professional PowerCloud XP Enterprise 9000 ...

      ...cannot do that. I'm sorry Dave.

      Please place your data into the airlock.

      Please Dave.

      It's very important Dave. You must place your data into the airlock.

      Place your data into the airlock Dave.

  15. Gray
    Facepalm

    Somebody surely got ballmered ...

    Nope ... no chair smash against the wall this time. Monkey-boy has no doubt called the Azure crew into his office, and he's smashed his chairs on their monkey-butts.

    Atta boy, Ballmer. I read the piece in Fortune Magazine awhile back about how M$ motivates its crew. He had to expect that this was comin' down the pipeline.

    2013 ... the year all the monkeys came to dance on Monkey-Boy's head.

  16. This post has been deleted by a moderator

    1. Anonymous Coward
      Anonymous Coward

      Re: More FAIL

      To be fair, that first one was 10 years ago.

      Still, it's amusing that a decade later, they're still making this rookie mistake.

  17. JassMan Silver badge
    Trollface

    Roll on Feb 2014. Obviously they will be going for 16 hours next time.

  18. pixl97

    Failure at point of no redundancy.

    Redundant Hardware: Check.

    Redundant Network: Check.

    Redundant DNS: Check.

    Redundant Services: Check.

    Single point of failure at certificate services: Check, wait not FAIL, crap ARRAHHH.

    Design tip for next time, find a way for your services to use two different set of certificates from two different providers. Make sure the expiry dates differ. Have stuff warn, but not fail if there is a problem with one.

    I'd like my consulting fee now.

  19. Anonymous Coward
    Anonymous Coward

    Digital Pearl Harbor?

    What a ballmer, one stooge at Microsoft can bring down a large part of the US digital infrastructure, was it a spy or was he shorting MSFT?

  20. AidanCheddar
    Paris Hilton

    Hmm...

    So, what we've learned? Windows sucks as a server.

    1. Anonymous Coward
      Anonymous Coward

      Re: Hmm...

      No, we have learned that crypto is complicated and can bring down your systems with a simple oversight.

  21. Herby

    Just an example of...

    ...The "Microsoft Attitude". Of course they know better, that's why they work that way.

    It goes back to MS-DOS (probably before). A company unwilling to accept "outside opinions", getting stuck in its ways.

    For the observant, there seem to be many current examples, too numerous to show here, I'll let the reader fill in the b l a n k s.

  22. Anonymous Coward
    Anonymous Coward

    it's all relative

    Azure is down so often that I'm never sure when I look at the status page what "operating normally" means... does it mean it's broken like normal, or working nominally to spec?

  23. Doug Bostrom

    A day passes, and then another year

    "We will continue monitoring the health of the Storage service and SSL traffic for the next 24 hours..."

    ...and then we will ignore the situation for 364 days, until our cert once again expires.

    Only for 24 hours? Surely not, but surely they'd also be a little more careful in their phrasing, one would think.

  24. dajames Silver badge
    Devil

    What's the lesson here?

    It's really rather clever. By omitting to renew a single certificate -- and administrative "error" that could be made on any service, regardless of the operating system that supports it -- Microsoft downed their own cloud service for a short while, demonstrating to world+canine that cloud services can't be guaranteed to be 100% available, and that if you rely on the cloud for your work (or play) you're stuffed when the service goes away.

    I'm sure the lesson Microsoft want you to take away from this is that you should do all your work on an actual PC running an installed OS and software. You might choose to use Microsoft Windows and Microsoft Office -- most people do -- go out and buy yours today.

    Result!

  25. Anonymous Coward
    Anonymous Coward

    Outlook running on Azure, or what?

    Down for 2 hours now. Ofcourse this can be a disgruntled Azure customer getting their own back

    http://downrightnow.com/hotmail

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2021