back to article Sophos tells users to roll back Microsoft's Patch Tuesday run if they want PC to boot

Brit security software slinger Sophos has advised its customers to uninstall Microsoft's most recent Patch Tuesday run – the same patches that protect PCs and servers against the latest Intel cockups. In an advisory note published over the weekend, Sophos admitted the latest batch of Windows updates are causing the machines of …

  1. Baldrickk Silver badge
    FAIL

    Not a Sophos user myself, but even if I was, this wouldn't have affected me...

    The Windows update keeps failing to install all by itself.

    1. Anonymous Coward
      Anonymous Coward

      Upgrade to... XP ... I mean Linux

      Remote desktop patch installed fine on my XP system. Phhh, Win7 users ;-)

  2. chivo243 Silver badge
    Meh

    Not a Sophos client

    and the update should still be sitting on WSUS waiting for approval. It can wait until the weekend.

    1. DailyLlama

      Re: Not a Sophos client

      For the majority, yes. For the pilot groups, no.

  3. Blockchain commentard Silver badge

    But it doesn't seem to affect AVG,Avast,Norton etc. so perhaps Sophos users need to look around when their licenses need to be renewed. I know I would if it happened to me more than once.

    1. Anonymous Coward
      Anonymous Coward

      No, that was the previous Windows update that screwed PCs with Avira and others.

    2. Danny 14

      It was the same last month until a few days later other vendors also released advice. Since each monthly patch includes the previous crap then surely this will keep happening?

    3. Jove Bronze badge

      It is not limited to Sophos product.

  4. Anonymous Coward
    Anonymous Coward

    You seem to be picking on Sophos as the cause of the issue, but the last two Microsoft updates have caused the same symptoms with various anti-virus vendors.

    So who made the changes that caused to problem? Microsoft, not Sophos.

    1. Tabor

      Yes, Microsoft made changes and Sophos didn’t. However, AV vendors have been known to use undocumented hooks into the OS.

      The main thing is : Microsoft plugged a serious hole. If one AV vendor’s product stops working I would not blame the OS vendor in this case.

      1. Lusty

        Windows hasn’t had undocumented hooks for a decade or more. Security has been the focus and that means open and documented.

        I’d rather run without AV than without current patches.

        1. Anonymous Coward
          Anonymous Coward

          "Windows hasn’t had undocumented hooks"

          Maybe this is a terminology thing then.

          I would take undocumented to mean not part of the published API, but allowing access to additional functionality that may not be supported in future due to either patching or other changes in the OS.

          The question is around how much notice AV vendors had/need to make changes and what the implications are for performance.

          For servers I'd take the patch over AV as I likely can't disable file sharing on key systems. For clients, I'd take AV and firewall polices restricting inbound file sharing to server subnets over the patch, but I guess it depends on the balance between local/offsite clients.

        2. Julz Silver badge
          Paris Hilton

          How would you know if there were any undocumented hooks; read the documentation?

          Paris, because all her hooks are undocumented...

          1. Anonymous Coward
            Anonymous Coward

            "How would you know if there were any undocumented hooks; read the documentation?"

            By debugging Windows system calls, check missing values to see if they return anything, ask Microsoft for how to do X that doesn't appear possible via documented calls, debugging/examining the behaviour of malicious programs etc.

            One further point - while "well behaved" software doesn't need to use undocumented calls, if the software you are trying to detect and protect against does use those calls, there may be no other way to protect against those calls/hooks unless you use them yourself. Where things get messy is if a method needs to change, Microsoft give a date to migrate to a new "correct" API call but you still have an existing version to support/update.

            With new OS releases/service packs you get some time to address the issues and test - with patches, you have a matter of weeks.

  5. TechyLogic

    Problem confirmed, yet, works

    Well, deployed the patches before seeing this.

    2008R2 - stuck at 30%, left for 4 hours, and it completed fine. replicated on a number of 2008R2 VMs. Its a long reboot, but, it works.

    1. Danny 14

      Re: Problem confirmed, yet, works

      Identical to last month. Long reboot and long first login then fine.

      1. Anonymous Coward
        Anonymous Coward

        Re: Problem confirmed, yet, works

        'Identical to last month. Long reboot and long first login then fine.'

        Great. Unless you have a few dozen people sat in the office who can't do fuck all for hours while a 'patch' installs on their laptops. Can you bill MS for the lost productivity?

        1. Tony Paulazzo

          Re: Problem confirmed, yet, works

          Can you bill MS for the lost productivity?

          Install Linux if you want productivity & security...

          Waits for responses detailing Ubuntu updates that borked the systems...

          1. Anonymous Coward
            Anonymous Coward

            Re: Problem confirmed, yet, works

            "Waits for responses detailing Ubuntu updates that borked the systems..."

            Isn't Ubuntu the sole Linux with the blessing of Microsoft Corporation? Why would MS choose to bless a proper Linux, one of the several with a history of *not* borking the systems every time the opportunity arises? If systems "just worked", most of the IT Departments in the world would be jobless within months.

            1. jms222

              Re: Problem confirmed, yet, works

              I have seen Ubuntu screw up grub (which it seems to update every few days) and also entirely remove the kernel before so don't think you're safe for a minute. There is also no concept of last known good kernel as initrd gets rebuilt at the drop of a hat too. In fact things are so screwed up the initrd even gets rebuilt during removal of the associated kernel image.

              1. DCFusor Silver badge

                Re: Problem confirmed, yet, works

                ?? must be old news. The only time I see grub is when I set up a machine as dual boot for a new nix user, which not too much later gets set up as nix only. I and most others switched to Mint and LTS only years ago when nearly everyone else did. I have yet to have an update of any kind bork a system, in my little family of 20 or so of them -

                Updating major versions of linux - which you have to do proactively - HAS borked some oddball locally written daemons that initially used workarounds suggested on the web to keep working after systemd first came out horribly broken. And this happened for a couple years with distro updates borking the same things as things started working the old ways again (but LP would of course never admit this, having labeled all those breakages as E_WONTFIX or "then don't do things like mount shares at boot).

                Of course, the recent advent of fairly clear how-to's and dox for systemd didn't hurt.

                FWIW, most linux now has timeshift, which makes rolling back easy. Or, more usefully since one rarely needs that much - to have a couple rolling backups at different intervals of whatever in case YOU bork something and want to recover it.

        2. TechyLogic

          Re: Problem confirmed, yet, works

          the bigger concern is why you are doing in hours patching of bus-crit systems?

          patch the server the helpdesk software runs on - cant log problems if the ticket system is down ;).

          Obviously this only works in small teams. Or if you have a PFY around to blame.

    2. Hans 1 Silver badge
      Facepalm

      Re: Problem confirmed, yet, works

      This means IT IS MS' fault.

      FFS, you poor sods, I am really wondering, why do you accept this state of affairs, seriously, you keep giving this incompetent bunch over at Redmond billions and they cannot even write a proper patch installer; they have tried for years, now, every month, EVERY SINGLE MONTH, gazillion systems somewhere break and each time it turns out to be the patch installer.

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

      Again, you all are giving billions to a company that cannot even write a patch installer!

  6. fidodogbreath Silver badge

    In Sophos' defense...

    ...the difference between Windows and malware can be subtle.

    1. Anonymous Coward
      Anonymous Coward

      Re: In Window's defense...

      ...the difference between Sophos and malware can be subtle?

  7. Pascal Monett Silver badge
    Mushroom

    So, what Sophos is really saying . .

    . . is that Microsoft put together a critical patch and Sophos couldn't be arsed to test if that was going to bork its product.

    So now, you remove the critical patch that protects you in order to give Sophos time to pull its finger out and patch its own shit.

    Of course, when you have shit running on shit, you get shit service as well.

    1. Anonymous Coward
      Anonymous Coward

      Sophos couldn't be arsed to test if that was going to bork its product

      Evidence? Do Microsoft provide their OS patches to Sophos well in advance of public release? Otherwise, how can it be Sophos's fault?

      1. PTCruiserGT

        I don't know if 2+ weeks is well in advance enough for Sophos, but Microsoft does provide Preview of Monthly Rollup updates. For example, the preview for May was released April 25. My workplace has been forced into having to test these preview releases for lack of companies like Sophos doing the same (and warning their customers in advance).

        1. LDS Silver badge

          "The Preview of Monthly Rollup is product specific and addresses new non-security updates" (emphasis mine - https://support.microsoft.com/en-us/help/824684/description-of-the-standard-terminology-that-is-used-to-describe-micro)

          It would be quite dangerous to include critical vulnerabilities fixes in a preview, since it would give away what is vulnerable, and with some reverse engineering, how.

          Don't know if some "trusted" companies can have security fixes in advanced to test.

  8. Dr Gerard Bulger

    Not just Sophos?

    My ASUS Z270-P board i7-7700k machine hangs with the May update with spinning circle. I have no idea why. Multiple reboots eventually it uninstalls in safe mode and I get back my working PC. Running the MS utility to stop it trying it again, wushowhide.diagcab, seem to fail to block it, as soon the May update is back on, as is the determination of micro$oft to stuff your machine no matter what.

  9. a_yank_lurker Silver badge

    What is going on?

    It seems every month a major AV vendor gets clobber by the updates and some months several get nailed. But it is a different vendor every month. It is almost as if Slurp has decided every month which AV vendor to target with problems. I do not remember this level of problems with updates with AV stuff with earlier versions of Bloat (I might be showing some mileage though).

    1. JeffyPoooh
      Pint

      Re: What is going on?

      AYL inquired about, "...every month a major AV vendor gets clobber by the updates..."

      AV vendors are morons. All of them.

      It's merely a hypothesis, but it's been fitting the facts perfectly since abouf 2007.

    2. LDS Silver badge

      Re: What is going on?

      AV vendors do install drivers to meddle within the kernel trying to intercept what should be malicious code before it gets executed. They may employ their own researches and techniques beyond what is standard driver interfaces to try to gain an edge over competition, believing they fully understood how the kernel works. Unluckily, little changes can cause big troubles.

      I don't trust AV vendors much today, and their code.

  10. Anonymous Coward
    Terminator

    Similar Sophos screwup from April?

    Give us some credit, we all know primary source of the F^Hscrewup ..

  11. Splork

    RDS

    "permits unauthenticated remote code execution through the medium of Remote Desktop Services"

    The first thing I do, for Windows and all OS flavors in my shed, is to disable all remote access needed for some IT wonk to mess with my systems especially RDS. However, this policy is especially important for Windows as any subset of vulnerabilities that can be easily shunted is a good thing when using a spaghetti code OS like Windows. As for Windows 7, it's been sand boxed in a VM (hosted by a strong 'nixOS) and unable to access the Internet since the whole GWX debacle.

    The only good Windows is an deaf and dumb Windows. Wait, is that redundant?

    1. LDS Silver badge

      Re: RDS

      I think you got a teleporter to manage any remote system.... most of us don't manage systems on their laptop/bedroom/basement only - so they need a way to access them. Good luck, for example, to ask physical access to any machine running at a cloud provider. In some companies too accessing the data center is not easy, and it can be hundred or thousands of kilometres away...

      There are of course ways to harden remote access - and remember it is not used only for remote administration, it could also be used as a plain "PC" by a lot of people...

      1. TechyLogic

        Re: RDS

        To elaborate

        Option 1 : Lock down the RDS/RDP to specific LAN addresses and user groups. You can include your VPN scope. Add 2FA to the VPN connection first.

        Option 2: Manage via iDRAC or HPE

        Option 3 : (actually combination of all) VLAN a management NIC, ACL on the VLAN, 2FA on the VPN, and locked down to specific LAN addresses / User groups

        theres loads - this is just off the top of my head.

    2. TonyJ Silver badge

      Re: RDS

      "...The first thing I do, for Windows and all OS flavors in my shed, is to disable all remote access needed for some IT wonk to mess with my systems especially RDS..."

      No you don't. At least not for Windows, since it's been disabled by default since at least Windows 7.

  12. This post has been deleted by its author

  13. adam payne Silver badge

    Nothing has happened here yet but taking no chances on it, declined updates.

  14. Anonymous Coward
    Anonymous Coward

    Let's take a look in the Swedish Chef's window...

    Bork

    Bork

    Bork...

  15. Anonymous Coward
    Anonymous Coward

    Ahhhhhh

    I work in an NHS Trust using Win 7 and Sophos AV. We've been having a horrible time with people stuck on 30% for hours, the best part of a day sometimes. The help desk has been jammed!

    Jolly Times!!!!

    Anon to project the the innocent and the guilty and "Think of the Children"!

  16. Jove Bronze badge

    Migration incentive ...

    ... to get users onto safe and more robust platforms.

  17. Anonymous Coward
    Anonymous Coward

    Its advice on what to do is pretty blunt: uninstall the Windows update. Specifically, revert KB4499164 (May's full-fat Patch Tuesday) and KB4499165, the security-only update ...

    Hands up anybody who still thinks roll-up patch bundles, which a customer can either take or leave as a whole, is still a good idea? A patch for an pretty esoteric hardware bug, which has a low likelihoodod being exploited relative to the overall threat landscape, effectively blocking customers from patching against a catastrophic and trivial to exploit software vulnerability in one of the core components. Yes, that's going to make things secure.

    As a less disastrous example, an aging but still perfectly functioning AMD FX-based system I have here fails to install every other monthly rollup. From the update trace logs, it looks like it tries to install an Intel microcode update (which it obviously does not need), fails, and then reverts the entire rollup. Next month's rollup goes through without a hitch. Why that system and not the others? I have no clue. Can I just block the failing patch, and let the rest apply? No, sirrah - you must wait for the next rollup. May be it will install. May be it wont. I could try rebuilding it, but there is no guarantee it'll help either ...

  18. Anonymous Coward
    Anonymous Coward

    The AV problems I get. These things happen sometimes, quality control yes yes yes.

    But this from a company who announced 4% of staff would be made redundant last week. That's what really sucks.

    1. Anonymous Coward
      Anonymous Coward

      Source?

      Do you have a source for that announcement? Not finding anything about it.

      1. PTCruiserGT

        Re: Source?

        It's not an official source, but there's this: https://www.thelayoff.com/t/Z57YGGs

      2. Anonymous Coward
        Anonymous Coward

        Re: Source?

        I work for Sophos and have just lost several colleagues. I am chosing to post anonymously. I'm not a lawyer but I understand it's below the any thresholds that require a plc to make a public statement.

  19. herman Silver badge

    So, all you have to do is find a Windows machine running Sophos, with remote control enabled and then you have good old worm on your hands...

    1. Sabot

      No, just running Windows 7 or Windows Server 2008r2

      And you have less than 36 hours before Microsoft will have fixed it.

  20. Sabot

    The real cause of the issue turned out to be Windows Defender ATP...

    Status today:

    Microsoft is aware of the issue and is rolling out a fix for Windows Defender ATP over the coming 36 hours.

    1. Anonymous Coward
      Anonymous Coward

      Re: The real cause of the issue turned out to be Windows Defender ATP...

      https://community.sophos.com/products/sophos-central/f/sophos-central/112936/sophos-notification-following-the-microsoft-windows-14th-may-update-some-machines-hang-on-boot/405085#405085 Has the info.

  21. Anonymous Coward
    Anonymous Coward

    Microsoft issue confirmed

    https://community.sophos.com/products/sophos-central/f/sophos-central/112936/sophos-notification-following-the-microsoft-windows-14th-may-update-some-machines-hang-on-boot/405107#405107 Has an update to confirm any issues with the May security updates, stopping at 30%, is a Microsoft issue.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020