back to article Nasty regreSSHion bug in OpenSSH puts roughly 700K Linux boxes at risk

Glibc-based Linux systems are vulnerable to a new bug (CVE-2024-6387) in OpenSSH's server (sshd) and should upgrade to the latest version. Infosec researchers at Qualys published their findings today, revealing that sshd is vulnerable to a race condition that could allow an unauthenticated attacker to achieve remote code …

  1. Doctor Syntax Silver badge

    Debian & Devuan patches also landed this morning.

    1. Anonymous Coward
      Anonymous Coward

      Debian & Devuan patches also landed this morning.

      Beat me to it. 8^)

      A beer for you Dr. S (on me) but not till Friday.

      .

      1. Doctor Syntax Silver badge
        Pint

        Mythic Beasts beat both of us to it. First I heard of it was an email from them telling me of an urgent update to a server.

        So - to them (but not before they've finished all their patches) >>>

    2. chasil

      RHEL not vulnerable.

      I am hearing that all RHEL variants use the -D option to log to stdout/err, avoiding syslog().

      RHEL9 is based on OpenSSH 8.7p1, but is not vulnerable even though the syslog() bug was introduced in this release.

      1. chasil

        Re: RHEL not vulnerable.

        ...actually, RHEL9 is reported as vulnerable by RedHat.

        https://access.redhat.com/security/cve/cve-2024-6387

        I *think* that adding -e as a startup option shuts down calls to syslog.

        -e : Write debug logs to standard error instead of the system log.

        1. Pacman950

          Re: RHEL not vulnerable.

          On RHEL 9.4 it's patched:

          yum list --cve CVE-2024-6387 | grep openssh

          openssh.x86_64 8.7p1-38.el9 @anaconda

          openssh-clients.x86_64 8.7p1-38.el9 @anaconda

          openssh-server.x86_64 8.7p1-38.el9 @anaconda

          openssh-askpass.x86_64 8.7p1-38.el9 updates-appstream

          openssh-keycat.x86_64 8.7p1-38.el9 updates-baseos

          1. T. F. M. Reader

            Re: RHEL not vulnerable.

            I can confirm that CentOS 9 Stream and Fedora 40 are not vulnerable

          2. jailbird

            Re: RHEL not vulnerable.

            Actually, no, RHEL9 and CentOS 9 Stream are NOT patched, I just checked. In fact, the RHEL9 package hasn't been updated since April and the CentOS 9 Stream package was updated a month ago. So there's no way they're patched.

            Your "yum" command is invalid and will produce a list of every available package. the dnf man page straight out says:

            --cve=<cves>, --cves=<cves>

            Include packages that fix a CVE (Common Vulnerabilities and Ex‐

            posures) ID (http://cve.mitre.org/about/), Eg. CVE-2201-0123.

            Applicable for the install, repoquery, updateinfo, upgrade and

            offline-upgrade (dnf-plugins-core) commands.

            Notice 'list' isn't listed.

            # dnf repoquery --cve=CVE-2024-6387

            Will return.. nothing

      2. Grogan Silver badge

        Re: RHEL not vulnerable.

        Ahh, thanks. I checked today and noticed I didn't get a package update yet (Alma Linux 9) so I created a few rules to block the port for all IP addresses but mine. I guess I didn't really need to do that then, but that's OK. I would expect it to be fixed regardless, though.

        P.S. Oops, I see that info is not quite correct. Thanks anyway :-)

    3. This post has been deleted by its author

  2. Korev Silver badge
    Pirate

    While the consequences of a successful exploit could be dire, actually doing so would take some patience. According to OpenSSH and its release notes for version 9.8, which includes the fix for CVE-2024-6387, in lab conditions it took between six and eight hours to beat the race condition.

    I'm not sure I agree, you could finish work for the day and your machine is pwned by the time you start work the next day....

    1. Paul Crawford Silver badge

      Generally we also use the UFW rate-limiting option on SSH access. Yes, sometimes you can't login as others have borked it for you through attacks, but in this sort of scenario it keeps the Barbarians at bay.

      Just checked and 20.04 is not impacted, nor is the version of Rasbian I'm using.

      1. Joe W Silver badge

        I first agreed and thought I was in the clear. Then thought again and edited my post. I'm pretty sure a timeout is not a failed login attempt. Rate limiting logins (your idea) still works, IP blacklisting the way I do it won't.

  3. Richard Tobin

    glibc??

    ".... anything running glibc is probably vulnerable. ... The notable exception here is OpenBSD"

    Surely the BSDs don't use glibc anyway?

    Updated: according to https://www.qualys.com/2024/07/01/cve-2024-6387/regresshion.txt they haven't tested any other libc implementations.

    1. Anonymous Coward
      Anonymous Coward

      Re: glibc??

      You point out a fact, and get downvoted by 4 people.

      I struggle to see their reasoning.

      Shrug

      1. Grogan Silver badge

        Re: glibc??

        Forget reasoning, they are ignorant downvotes. There must be people that sit here and go "nope, nope, nope, la, la, la" because I see a lot of wisdom that gets downvotes in this peanut gallery.

        1. Anonymous Coward
          Anonymous Coward

          Re: glibc??

          How dare you contract my cherished beliefs with facts /s

      2. Anonymous Coward
        Anonymous Coward

        Re: glibc??

        Have a downvote for trying to make sense of the nonsensical!

    2. Grogan Silver badge

      Re: glibc??

      It's not a vulnerability on the BSD's because their signal handling isn't implemented the same way (they don't use glibc... except for some Linux ports that may need to link to it. That won't be OpenSSH)

    3. bazza Silver badge

      Re: glibc??

      glibc was also implicated in the recent attempt to subvert openssh by the attack on libzma via its build system. IFUNC was used by the attack to redirect calls to openssh's authentication to the attacker's own routines. IFUNC is part of glibc, but for any executable that performs security checks and depends on a third party library IFUNC sounds like a terrible, asking-for-trouble piece of functionality. It supposed to be used by developers to be able to have several different versions of a function loaded at once, and to be able to select (at run time) which one is used. I've never ever heard of a developer doing that, the gnu documentation on it seems a bit vague on the matter and anyway confesses to there being shortfalls in the documentation. Given what nearly happened to libzma / openssh, and the strong possibility that others are trying (and may be have succeeded) with other commonly used library dependencies, getting rid of ifunc or getting rid of glibc sounds like a good idea.

      I don't know if other C libraries out there have a similar feature...

    4. chasil

      Debian on FreeBSD

      A port of Debian that runs on the FreeBSD kernel has existed in the past.

      https://www.debian.org/ports/kfreebsd-gnu/

  4. b0llchit Silver badge

    Fail2ban helps timeline?

    I,m wondering. Having installed fail2ban with exponential block time, wouldn't that mean it takes, on average, a significant amount longer to penetrate (3 failures within 8 hours starts at 4 hours blocking and doubling every next block event)?

    Sure, the first exploit hit could already be fatal, but if you play the odds, it may give you some breathing room to patch, wouldn't it?

    (BTW, already patched my servers)

    1. sten2012

      Re: Fail2ban helps timeline?

      I'm not sure if a timed out login attempt would be flagged by fail2ban as a "failed" login per se and therefore might not be blocked. So might need some config tweaking to recognise it (or at least verifying) before relying on it.

      Sorry, I haven't checked so probably should shut up, but wanted to comment in case someone sees your question and decides fail2ban is "probably good enough"!

      1. Anonymous Coward
        Anonymous Coward

        Re: Fail2ban helps timeline?

        I was wondering the same thing. Does timeout count?

        We just changed the firewalls to an even more strict policy on white-listed IP's that can connect to SSH and commenced patching. The patches are easy to apply via the standard terminal update procedures. Hats off to Canonical for having the patch ready in such a timely manner.

  5. Anonymous Coward
    Anonymous Coward

    Just like Oracle's slogan "Oracle database runs best on..."

    "OpenSSH runs best on... OpenBSD"

  6. steven_t
    Alert

    Mitigation

    Canonical/Ubuntu have published the following mitigation, which may be useful if you can apply this change faster than you can install the patch:

    "Set LoginGraceTime to 0 in /etc/ssh/sshd_config. This makes sshd vulnerable to a denial of service (the exhaustion of all MaxStartups connections), but it makes it safe from this vulnerability."

    Copied from here: https://ubuntu.com/security/CVE-2024-6387

    1. Grogan Silver badge

      Re: Mitigation

      A better option would be to:

      # iptables -A INPUT -p tcp --dport 22 ! -s a.b.c.d -j DROP

      or for example, a network:

      # iptables -A INPUT -p tcp --dport 22 ! -s a.b.c.d/24 -j DROP

      Where a.b.c.d is the correct IP address you wish to accept, or a.b.c.d/24 is the correct network and subnet.

      The ! of course means "not" DROP from that source in this case. (doing it that way saves needing two rules for DROP and ACCEPT)

      1. Grogan Silver badge

        Re: Mitigation

        Fixed for me now on all boxes with non-rolling distros (e.g. Arch had this fixed before I went to be this morning). The one I was most concerned about was an Alma Linux 9 box in a datacenter but those packages just hit my repos (Alma specific nomenclature in the package names while same base package and version)

      2. Grogan Silver badge

        Re: Mitigation

        You people who downvote are ridiculous... moreover, one of the recommendations by Qualsys was to mitigate this using "network based access controls". Since I don't have access to the routers upstream from me, that prevents any attack from reaching a service listening on the port. This has nothing to do with logins. Not sure what your problem is, but you're pathetic.

        1. Claptrap314 Silver badge

          Re: Mitigation

          I'm running a server that accepts data from almost 100 different corporations. If I want do drop everything outside a single CIDR, that CIDR will have to be 0.0.0.0/0.

          Whitelisting is what they are talking about.

          And yes, We use Google on the business side, so I've set up an SSO process that logs my IP address & updates our (AWS) security groups to whatever doctor's office or grocery store I'm working in while my wife is at an appointment. It wasn't particularly hard to set up.

    2. Ian 55

      Re: Mitigation

      Looking at the Ubuntu servers and local boxes, it could be that Canonical actively pushed this update - on several, it happened without me doing anything.

      According to dpkg.log, it happened at the same time as updates I did agree to or initiate.

  7. Bendacious Silver badge
    Coat

    2024 finally Year of the Windows desktop

    I have to use Linux at work but this has given me the push to try and go Windows at home. I'll need one of the simpler distros to get started though. I hear good things about Windows 7 for people used to Ubuntu. Might need to go dual-boot for a while until I find replacements for all my apps. Wish me luck guys.

    1. Anonymous Coward
      Anonymous Coward

      Re: 2024 finally Year of the Windows desktop

      Watch out for the butthurt Linux fanboi downvotes!

      There are one or two of the humorless souls around here!

      1. Doctor Syntax Silver badge

        Re: 2024 finally Year of the Windows desktop

        It's OK, we grok the joke - including how long a similar problem on Windows would take to be detected, buried within Microsoft, fixed, held back to the next Patch Tuesday, applied, rebooted and the resulting breakage fixed the following month - maybe. Did you miss that bit?

        1. Anonymous Coward
          Anonymous Coward

          Re: 2024 finally Year of the Windows desktop

          I never put you down as one of the butthurt fanbois. I expected better of you.

          Yes, yes, we all know Windows is far far worse, and I would never use it, but it was a joke, and I'm not a cultist.

          1. Doctor Syntax Silver badge

            Re: 2024 finally Year of the Windows desktop

            "I never put you down as one of the butthurt fanbois"

            And what part of "we grok the joke" would make you think otherwise? One of Bandacious's early upvotes was mine.

        2. Anonymous Coward
          Anonymous Coward

          Re: 2024 finally Year of the Windows desktop

          I'd print this out and hang it in my cubicle, but the latest update has broken my printer. Again. (Kidding.)

      2. Agamemnon

        Re: 2024 finally Year of the Windows desktop

        Dunno if I'm a FanBoi, but I use Linux to talk to my real UNIX machines. Works pretty good.

        That aid, I upvoted because that was damned funny.

        1. Anonymous Coward
          Anonymous Coward

          Re: 2024 finally Year of the Windows desktop

          Original AC (for other personal reasons) here.

          I am also 100% Linux and other Unix systems. Some have even called me a fanboi, but like you, I thought this was a funny comment.

          You and I aren't the butthurt fanbois I was talking about - the ones so into the coolaid that instead of just ignoring it if they didn't think it was funny, they had to actively downvote it, and then assumed by my reaction to that that I wasn't myself a linux/unix fanboi, but some windows cultist.

          That's the sad part I was pointing out. That level of cultism does our community no good at all.

          1. Joe W Silver badge

            Re: 2024 finally Year of the Windows desktop

            That level of cultism does noone any good at all.

            Something we all need to be reminded of sometimes.

        2. Doctor Syntax Silver badge

          Re: 2024 finally Year of the Windows desktop

          "I use Linux to talk to my real UNIX machines"

          Back in the day many of us have even used Windows to do that.

          1. MrBanana

            Re: 2024 finally Year of the Windows desktop

            Myself, I only used Putty to stop the window panes falling out of the frames.

          2. karlkarl Silver badge

            Re: 2024 finally Year of the Windows desktop

            My colleague only stopped with Windows because the Prolific RS-232 drivers got all fscky.

            (A fantastic story of cheap clones causing upstream vendor to change chip slightly, rendering their driver inoperable with clones and Window's inability to cleanly support multiple proprietary driver versions)

    2. JamesTGrant Bronze badge

      Re: 2024 finally Year of the Windows desktop

      I suggest XP SP1. Since I installed it for my Grandma after she was having some trouble with some apt repo in Ubuntu, she never calls me anymore.

      1. STOP_FORTH Silver badge
        Headmaster

        Re: 2024 finally Year of the Windows desktop

        SP2 James.

      2. A Known Coward

        Re: 2024 finally Year of the Windows desktop

        ... she disowned you? I mean, I know it's XP, but that's a bit harsh!

    3. Hubert Thrunge Jr.
      Coat

      Re: 2024 finally Year of the Windows desktop

      I'm sorry, I only work in command line.

      I use PC-DOS5.

    4. david 12 Silver badge

      Re: 2024 finally Year of the Windows desktop

      No joy there -- the Windows version of SSH is based on OpenSSH.

      1. DavidRa
        Meh

        Re: 2024 finally Year of the Windows desktop

        Yes it is OpenSSH but it's not like Windows uses glibc and UNIX signals. I suspect that might make it *slightly* harder to exploit on Windows.

  8. Anonymous Coward
    Anonymous Coward

    sshd restart required after upgrade

    Be aware that if you upgrade (rather than install) a machine running OpenSSH sshd to version 9.8 you need to restart the ssh daemon otherwise you will not be able to login via it.

    More info: https://gitlab.archlinux.org/archlinux/packaging/packages/openssh/-/issues/5

    1. really_adf

      Re: sshd restart required after upgrade

      A useful warning but it's worth noting (as described in the Arch package issue you linked) that's because of an unrelated change between 9.7 and 9.8, so won't apply where packages have a backported fix, nor if, unlike Arch, the restart is done for you by package scripts (a choice with pros and cons).

      Also, in this case I think the potential for exploit is after the long-running sshd process has exec'd itself meaning the fix is effective without restarting sshd, but the conservative assumption and general case is that a restart is necessary so this should be standard practice anyway.

    2. Doctor Syntax Silver badge

      Re: sshd restart required after upgrade

      The usual upgrade scripts for Debian and its children include restarting daemons.

  9. FuzzyTheBear
    Happy

    Patch

    It's already patched in Mint 21.3

    Got to say something about updates : they rock. Unless you really don't pay attention and don't do the updates. Most of the time when i read about a critical update either it's in the updates queue or already been updated. Mint's update management really is easy.

    Nothing but good words about Mint and Linux in general. Switching 23 years ago was the best decision i ever made.

    Good day.

  10. Gerhard den Hollander

    Less of an issue for 64bit ?

    According to sans

    https://isc.sans.edu/diary/rss/31046

    Exploitation for AMD64 appears to be not practical at this time.

    (which doesn't mean you shouldn't patch your 64bit machines when patches become available)

  11. elahav

    Signal Handlers

    So Linux is also insecure by design? ;-)

    The real issue here is that, what is supposed to be the epitome of secure code, makes a rookie mistake in using signal handlers. Not only that, it ends up calling a non-async-signal-safe function buried under layers of indirection, making any future updates susceptible to repeating the same mistake.

    Signal handlers should be avoided if possible. If not possible, the most the signal handler should do is set some global variable and then let the main context do the heavy lifting. Keeping track of which library functions are async-signal-safe on every C library implementation is doomed to failure.

    1. bazza Silver badge

      Re: Signal Handlers

      I suspect they've used a signal handler for the purposes of being very portable, but even then there's no particular reason to implement the desired timeout functionality with SIGALARM.

      For a Linux implementation, there's signalfd - which is where signals are delivered via a file descriptor. You can include this in a call to select() or epoll(), along with other fd's. It's a very nice way of handling signals, because you're doing so at your convenience, in the main thread context, with no limitations on what can be safely called having read the fd. Far better than a fully async handler where dangerous things to do can be easily done!

      For a more portable timeout method, openssh could perhaps have used select(); it sounds like it's waiting for input at the relevant point of code, and select() called with a timeout is perfect for that.

      1. Ken Shabby Bronze badge
        Boffin

        Re: Signal Handlers

        Not if you are waiting on over 1024 file descriptors, seems bakrd into Linux

        1. bazza Silver badge

          Re: Signal Handlers

          You can edit /etc/security/limits.conf to set the limits to whatever you want.

  12. martinusher Silver badge

    Simple bugfix but potentially further compromises

    Whenever you get a problem like this -- a simple problem with an even simpler fix ("Don't use non thread safe code by default unless you really, really, know its thread safe") -- the advice is always "Upgrade to the latest version. Now!".

    This carries with it a number potential problems because unless this update fixes this bug, and only this bug, there's no guarantee that the slew of other changes would not introduce further bugs. Obviously this can be checked by regression testing -- but, seriously, the amount needed suggests that its unlikely that the coverage would be 100%.

    I'm used to controlled firmware fixes for certified embedded products where only the bug cited is fixed in that branch of the release. The reason for this is obvious. It also suggests that fixing bugs should be done in a less scattergun approach. Constant harping about "the latest" might keep a bunch of programmers churning away productively but it doesn't produce stable products.

    1. Anonymous Coward
      Anonymous Coward

      Re: Simple bugfix but potentially further compromises

      Still on Ubuntu 20.04, not yet 22 or 24, because God told me to wait for this bug to be discovered and fixed before upgrading.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like