back to article Log4j RCE: Emergency patch issued to plug critical auth-free code execution hole in widely used logging utility

An unauthenticated remote code execution vulnerability in Apache's Log4j Java-based logging tool is being actively exploited, researchers have warned after it was used to execute code on Minecraft servers. Infosec firm Randori summarised the vuln in a blog post, saying: "Effectively, any scenario that allows a remote …

  1. Androgynous Cupboard Silver badge

    A failure of forethought.

    For anyone not paying attention, this is a steaming pile of shit.

    Forget the bug itself. The issue is that a logging framework which must be - above all other APIs - simple, stable, predictable, beyond reproach - is doing things which invoke a class loading operation. It's come as a surprise to me that they would do this, even though I've never understood the love for Log4J . It's a fairly typical Apache Java project - everything put in, nothing taken out, keep throwing interfaces at it until it works: design be damned, we'll crush the problem to death under the size of the API. Programmers of the world, listen to me: limit your scope. Do one thing well, it's enough.

    Once again, the scientists were so enthralled with the fact that they could, they never stopped to think if they should...

    1. Anonymous Coward
      Anonymous Coward

      Re: A failure of forethought.

      Well, in order to output the object you've received for logging, you have to load the class definition for that object if you don't already have it.

      I certainly can see the sense to it; logging objects is common in debugging and defect analysis.

      1. Androgynous Cupboard Silver badge

        Re: A failure of forethought.

        The object you're logging is, by definition, in a class that's already loaded. And if you're logging to a file, for example, you're going to use the system classloader to load java.io.FileOutputStream. That's how Java works.

        This is different as it involves a loading a class from a URL. That's behaviour I would not expect from a logging framework, so I didn't look for it. Neither did anyone else, apparently. It's not called the "Loggging and network class loading framework for Java".

      2. TheMeerkat

        Re: A failure of forethought.

        No, you don’t.

        The object is passed into the logging library. The library itself is not supposed to load anything.

    2. Warm Braw Silver badge

      Re: A failure of forethought.

      I don't know much about Log4j, specifically, but there's always a danger that the more "pluggable" you make your framework the greater the danger of plugging in an exploit.

      However, the bit I find surprising is that news of JNDI exploits have been cropping up on a regular basis since at least 2016 [PDF] and that LDAP-related code loading is disabled by default in later versions of the JDK and yet here we are again.

      1. Anonymous Coward
        Anonymous Coward

        Re: A failure of forethought.

        Exactly. As well as your pithy phrase on plugging, I'd add that dependencies leave you dependent. I've been writing Java for 25 years now and I'm well aware of the holes in the language - I've plugged them with code, or lived with them.

        What I haven't done is download 900 Jars of unknown scope and add them to my build environment. I hate this aspect of the industry, I really do. There are vast tracts of poorly written code out there with big, trustworthy-sounding names on it ("Apache", "Eclipse") that developers are pulling in in the hope they will solve their problems, but they just create different problems because the API designer has... well, they haven't designed. There's XML everywhere, public methods everywhere, requirements on common-everything.

        Every single method in an API is something you have to learn to use it, and when you get an anonymous lambda function calling into a class like this you have no hope of understanding the code. Most of the methods are undocumented, the rest will be deleted in the next release, and Maven is constantly pulling down new versions without your knowledge... I find it simply incredible that people tolerate this.

        As an application developer, if you take this approach you build on sand.

        Here endeth the rant. I've had quite a week, can you tell?

        1. Anonymous Coward
          Anonymous Coward

          Re: A failure of forethought.

          "This class should have been called ServletContext, but this would have cause confusion with ServletContext."

          Brilliant.

          Oh and Maven, the systemd of build systems. Is that open sewer still a thing?

        2. Anonymous Coward
          Anonymous Coward

          Re: A failure of forethought.

          The main corollary of it all: Java needs to die a fast but still painful death. It's a convoluted, shitty mess. Most of the devs are OK with the paradigm of throwing more code into any project (most of the time without checking the implications of adding it) and creating resource-eating monsters which ultimately enlarge the attack surface.

      2. Tom 7 Silver badge

        Re: A failure of forethought.

        Whats that old Unix philosophy: a program should do one job and do it well.

        1. oiseau Silver badge
          Facepalm

          Re: A failure of forethought.

          Whats that old time proven Unix philosophy: a program should do one job and do it well.

          There you go, sounds better this way.

          As a philosophy, it may not be perfect.

          But when properly implemented, it does work perfectly well.

          O.

        2. Anonymous Coward
          Anonymous Coward

          Re: A failure of forethought.

          No. Itt does lead to other kind of issues like the Tesla 500 Error because software becomes a brittle pile of unrelated layer glued somehow one on top of another. It was a philosophy only useful when systems had so little resources you had to chain applications to get the end result and they could only communicate piping data somehow - and error management meant parsing some textual information written somewhere. I see too much software performing too much logging just because it can't trap and process errors throw by some code deep in the stack of calls across many languages, frameworks, and shells.

          The gazillion libraries software today use is a corollary of the same issue - software does use too many dependencies, and most of them are used blindly just to save some development time.

          1. Warm Braw Silver badge

            Re: A failure of forethought.

            most of them are used blindly just to save some development time

            I think there's a deeper general problem (I'm not saying it's related to this package) that the presently fashionable software methodology seems to be that you start in front of a screen, possibly with a second individual, and you begin writing code (or by writing a test for the code you have yet to write) without having considered how the code might actually work, or how it might relate to the code that you might write tomorrow.

            The idea of "design" has been deprecated and replaced by a process of successive approximation to a working solution ("refactoring") on a piecemeal basis. The blind reuse of random code is just one symptom of this: it isn't seen as being inherently wrong because there can always be another refactoring in which it is replaced - but of course that's someone else's responsibility. Development has been decoupled from responsibility and authority (deliberately as it notionally gives more status to 'elite' programmer bros) and I can't help feeling that we are going to regret it.

            1. Anonymous Coward
              Anonymous Coward

              Re: A failure of forethought.

              Random net quote:

              The Waterfall methodology—also known as the Waterfall model—is a sequential development process that flows like a waterfall through all phases of a project (analysis, design, development, and testing, for example), with each phase completely wrapping up before the next phase begins.

              But that doesn't work either. It's the getting the "right" simultaneous balance of top down design and bottom up iterative implementation and testing that is the secret sauce.

              Human intelligence as we know it is a result of an iterative test and hack process. God is too distracted by playing Minecraft to see the big picture.

              1. Warm Braw Silver badge

                Re: A failure of forethought.

                I wasn't advocating any specific alternative. The problem with "methodologies" in general is that the goals are sacrificed to the worship of the method. It's like insisting that the only tool you're allowed is a hammer.

                1. Ken G Silver badge

                  Re: A failure of forethought.

                  When the only tool you have is a hammer, every problem begins to look like someone who needs to be hit with a hammer - Thor (2011)

                  1. Anonymous Coward
                    Anonymous Coward

                    Re: A failure of forethought.

                    Every tool has a hammer side - Eric O. (South Main Auto Repair)

                    Is this why lots of people around me look like tools that need to be hit with a hammer...?

        3. david 12 Silver badge

          Re: A failure of forethought.

          What's that old unix philosophy? To do any job, string together a bunch of unrelated executables with idiosyncratic API's, all copied from the example provided by a guru?

    3. Anonymous Coward
      Anonymous Coward

      Typical Java libraries...

      Java developers don't think they've completed the job unless they can get a plugin framework, shims, proxies, factories, a domain-specific language and a whole hierarchy of random and badly debugged class loaders into every single product.

      It's why the in-memory size of a java.lang.Boolean instance is currently weighing in at 647k on current processors: 1k to hold the boolean value and 646k to hold the plugins, framework and web server.

    4. Anonymous Coward
      Anonymous Coward

      Re: A failure of forethought.

      The enterpriseyness of Java strikes again. You'd assume, that if you passed a string to a logger, it would write that string to a text file somewhere. Not so, with log4j. Apparently[1] it:

      1. A ResourceRef overrides the normal JNDI reference behavior with a set of key=value pairs intended to populate a Bean via setters

      2. Java stil lets you specify a custom factor, if that factory is in the classpath.

      3. BeanFactory works with ResourceRefs to call the setters and populate the new bean.

      4. the forceString directive in a ResourceRef lets you override the name of a setter from setFoo to anything you want; when assigning foo it will call that name.

      5. the ELProcessor class works as a gadget, since its eval method is a valid bean setter (though named incorrectly).

      6. Create a forceString rule to set "x" by calling "eval" - x doesn't even have to be a real field of ELProcessor

      7. Try to set x to a string

      8. BeanFactory calls ELProcessor.eval() on the string, because forceString makes eval() the setter for x

      9. Execute arbitrary EL

      Some BeanFactory setter JNDI lookup LDAP evaluator gets involved and fucks up somehow, because obviously it's Java so you can't just write a text string to a file.

      [1] https://github.com/drahosj/log4shell_poc

    5. mistersaxon

      Re: A failure of forethought.

      TBH the idea that patching a logging service would need production DOWNTIME is beyond ridiculous.

      Bit like a PDF viewer app patch needing a system restart and who'd code something *that* dumb...?

      1. J. Cook Silver badge
        Devil

        Re: A failure of forethought.

        Bit like a PDF viewer app patch needing a system restart and who'd code something *that* dumb...?

        Adobe, for starters. But then, I wouldn't also incorporate a (poorly written) scripting engine into a PDF viewer...

  2. Androgynous Cupboard Silver badge

    Log4J 1.x not vulnerable

    By the way the article states "Whether it was or not, it affects both Log4j and its newer version, Log4j2" - but I'm not sure this is correct, as the lunasec page states it's only Log4j2:

    2.0 <= Apache log4j <= 2.14.1

    It's also worth noting that if you've upgraded your JVM fairly recently you might not be vulnerable.

    JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1 are not affected by the LDAP attack vector.

    I think that will get a lot of people off the hook.

    1. pmb00cs

      Re: Log4J 1.x not vulnerable

      Lunasec haven't checked log4j versions below 2.0 as it isn't supported software, and hasn't been for a long time. As I understand it log4j versions below 2.0 might be vulnerable to this, but even if they're not vulnerable to this they're vulnerable to plenty of other things, and they're not getting fixed.

      1. EricM

        Log4J 1.x _IS_ not vulnerable (to CVE-2021-44228)

        No, the OP is correct.

        The capability to load anything via ldap-URLs in logging strings was introduced in log4j 2.0.

        Consequently, using log4j 1.x perfectly protects against CVE-2021-44228.

        But you are correct in that the 1.x versions have other vulnerabilities, namely https://nvd.nist.gov/vuln/detail/CVE-2019-17571, that is, however, externally exploitable under only very specific circumstances and use cases and was known/handled since early 2020.

        1. JoeCool Bronze badge

          Disagree

          Here's the first part of the CVE. It does not specifically exclude 1.x versions.

          Also, LDAP isn' t the only attack vector.

          It seems that the only "immune" versions would be those without JNDI resolution:

          "Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. In previous releases (>2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by ..."

    2. PC Paul

      Re: Log4J 1.x not vulnerable

      8u191 is pretty ancient now, I suspect the others are too. Although, thanks to Oracle's unique approach to managing open source projects you need a paid licence for anything past 8u202 now. Switch to Corretto or OpenJDK ASAP!

    3. cs94njw

      Re: Log4J 1.x not vulnerable

      "Please note that the builds of Apache Kafka and Apache Zookeeper offered in MSK currently use Log4j 1.2.17, which is not affected by this issue."

      https://aws.amazon.com/security/security-bulletins/AWS-2021-006/

  3. karlkarl Silver badge

    Log4J tends to be at least 10 times larger than the actual project dragging it in. Of course this raises security concerns.

  4. Anonymous Coward
    Anonymous Coward

    Logging?

    Bricking, more likely

  5. jezza99

    Not surprised this is in a Java component.

    As a former sys admin I loathed the fact that Java insisted on ignoring all system services, using alternatives which were larger, buggy and absolutely hostile to good systems administration practice.

    1. Anonymous Coward
      Anonymous Coward

      As a Java Developer, I loathed the fact that the sysadmins tried to force us to use "alternatives" that weren't scalable, poorly structured, and disconnected from each other for the sake of following their archaic "standards."

      1. Anonymous Coward
        Anonymous Coward

        Yeah, standards schmandards.

      2. Anonymous Coward
        Alien

        Yes, always better to write your own version of all the things, so that competent people can ensure system versions are secure and patched against known vulnerabilitys and then your half-thought-out bloated half-functional changing-with-every-release equivalent brings back all the vulnerabilities people fixed and a whole lot of new ones of its own.

        Like this one.

        Java programmers: like programmers, but not so good.

        1. Hubert Cumberdale Silver badge
        2. Anonymous Coward
          Anonymous Coward

          Clearly it is better to use something poorly designed but "traditional" just because the sysadmins don't like change. Sysadmins NEVER like change - it means they have to actually WORK for once.

          1. Anonymous Coward
            Alien

            To test that hypothesis would require well-designed replacements to exist. And the things people write in Java that we are talking about here are designed by Java programmers, so they almost never are candidates.

            But are many examples in fact. Who uses CFEngine now (someone must?), and I wonder if Puppet will survive Ansible? Who uses rdist still (again, someone must)?

            Who still uses Perl? (I do, but suspect is dying away in face of less line-noisy tools).

            And so on.

          2. hoola Silver badge

            More that the SysAdmins are the ones left holding the can patching everything up because developers appear incapable of writing secure code.

            Security issue - SysAdmin problem they did not see it coming.

            Functionality issue - SysAdmin problem because they did not install the latest and greatest piece of unknown supporting tat because, unsurprisingly system availability matters.

            Compatibility issue - SysAdmin problem because they needed to remove old, unsupported software or put workarounds in place to stop it from being exploited.

            The list goes on but the nuts and bolts is that if anything goes wrong it is the fault of the SysAdmins from any quarter.

          3. Fat Guy In A Little Coat

            Everyone has glass houses

            Pipe down and take the L on this one.

          4. J. Cook Silver badge
            Mushroom

            Riiiiight.

            Like we already don't have enough to do already between planning for capacity upgrades, making sure that old crusty old legacy machine is kept on life support so it can to it's tiny little job of grinding out a company's amount of paychecks on the regular, planning for the replacement of said crusty old dinosaur, dealing with support requests of varying intelligence (or lack thereof), dealing with at least one Project manager who insists on micromanaging every last detail of the project they are working to death, dealing with random cyber attacks from outer Euraisa (or Eastasia, depending on the time of day), managing all the other critical patches, the "ZOMFG PATCH THIS RIGHT NOW!!!!111oneoenoneoen" exploits, re-building the dev boxes because some developer somehow managed to uninstall the UI on it whilst trying to install a patch for their IDE, re-re-building the dev boxes because some developer decides to try and replicate the UI removal bug intentionally, re-re-re-building the dev boxes because the machine got pwned because it was missing a patch, making sure the test lab matches production, and finally dealing with developers who demand root|admin rights on the production servers so they can screw them over as badly as the dev boxes that you just had to rebuild because their code won't run without it.

            Will that be all?

        3. sabroni Silver badge
          Happy

          re: Java programmers: like programmers, but not so good.

          Not so good what? Did you mean not as good?

          Smartarse commenters: like me, but not so smartarse.

    2. Androgynous Cupboard Silver badge

      I've got one word for you: systemd. People in glass houses, etc.

      1. Anonymous Coward
        Alien

        systemd is a crawling horror. It is just a part of the ongoing undeclared war by (some of perhaps?) people who make Linux distributions on their users. Is not the only crawling horror: look at polkit say, because ... well, polkit was not written by a person with any understanding if anything, really.

        The people who implement and deploy these slime-dripping deformed monsters are not systems administrators. Systems administrators spend their lives trying to understand what material they can use to encase them which is adamant enough that the corrosive goo that drips and sometimes sprays from them does not eat through the hull.

        1. oiseau Silver badge
          Facepalm

          ... systemd is a crawling horror. It is just a part of the ongoing undeclared war ...

          Indeed ...

          To quote another comentard on systemd, here at ElReg:

          ---

          " ... takes root in its host, eats massive quantities of resources as it grows, spreads unchecked into areas unrelated to the initial infection, and refuses to die unless physically removed from the system, all the while doing absolutely nothing of benefit to the host."

          ---

          For the longest time I have been wondering if, to find out who/why, we should not be "following the money", so to speak.

          O.

  6. Tom 7 Silver badge

    Blimey - this made it into the nrMSM

    https://www.theguardian.com/technology/2021/dec/10/software-flaw-most-critical-vulnerability-log-4-shell

    1. Bill 21

      Re: Blimey - this made it into the nrMSM

      Yes, but that's why I popped over here to find out what was actually going on. The linked article has lot's of words in it, but very little actual information.

      1. Havin_it

        Re: Blimey - this made it into the nrMSM

        Ditto. As I read it, gave the impression that Apache httpd was the source. Not cool, Grauniad.

        At least I've heard of log4j so the "brand name" (which all cool vulns need nowadays) led me to dig deeper.

  7. Norman Nescio Silver badge

    (User) input validation and sanitisation

    I'm not a Java programmer, so please excuse the naïve question: Is this another example of xkcd: 327: Exploits of a Mom, where inadequate validation and sanitisation of user input leads to untoward results?

    1. Androgynous Cupboard Silver badge

      Re: (User) input validation and sanitisation

      No, it's by design. Here's a better writeup, now the dust has settled a bit: https://blog.cloudflare.com/inside-the-log4j2-vulnerability-cve-2021-44228/.

      1. Jellied Eel Silver badge

        Re: (User) input validation and sanitisation

        Cloudflare's post regarding detected exploit attempts is interesting, especially by origin. Canada at #1? Then again, this is just a sampling, and other countries (or their citizens) are probably probing other targets.

      2. Anonymous Coward
        Anonymous Coward

        Re: (User) input validation and sanitisation

        I read through that and it is clearly a Bobby Tables moment. Any user input to log4j should have ${ stripped as well as the other usual quotes and squiggles.

        1. rcxb1

          Re: (User) input validation and sanitisation

          > Any user input to log4j should have ${ stripped as well as the other usual quotes and squiggles.

          Of course you sanitize any input you will subsequently use. But do you REALLY always sanitize all input data before LOGGING it? When a request fails, do you really never log what the raw failed request is?

          And you sanitize all the HTTP headers, too, right? Not really though of as user-input, but it is. Do you never use '$' or '{' characters in your random data for user login cookies/tokens? You always drop those if received, right? And you'd never log the fact that you received something invalid, right?

          1. DryBones

            Re: (User) input validation and sanitisation

            How about not executing it? Can we all agree that executing stuff in a logfile is dumb and all this stuff that goes into log files should be saved literally and never parsed as code to execute?

            Jimmy on a pogo stick, that's all the input sanitization you need. Just literal the write to log and move on.

    2. JoeCool Bronze badge

      Look at it this way ...

      Your design spec says

      "find and load arbitrary code from any where on the internet, by inspecting the contents of user generated log messages"

      Does that not raise a few flags that denote "Raging hemorrhoidal security hole"

    3. JTruth

      Re: (User) input validation and sanitisation

      It exactly a bobby tables moment, dev's not properly sanitising user inputs. The actual CVE doesn't matter as many languages have capabilties to execute arbitory code, it your (dev) job to not let that happen.

      1. JoeCool Bronze badge

        It's a L O G G E R.

        It's job is to LOG. If you write code to lie to the logger, you are abusing the log facility.

        Bobby tables is entirely different because that's feeding into a COMMAND PARSER.

  8. Anonymous Coward
    Anonymous Coward

    Question: would log4net.dll also be impacted? or is solely on Java implementation?

    or is solely on Java implementation? the reason i ask is the mention of Plastic in the article which is used by Unity. so log4netPlastic.dll

  9. Anonymous Coward
    Anonymous Coward

    Confused Simpleton

    Couldn't people restrict egress traffic to block LDAP queries over the internet?

    1. Anonymous Coward
      Anonymous Coward

      Re: Confused Simpleton

      It does seem to be a bit bizarre that there's a mechanism that can load and execute code from anywhere on the planet where the default state isn't locked down to localhost.

  10. Kevin McMurtrie Silver badge

    Apache

    Log4J is a very old Apache project. You need to read the source code more often if any of this news is a surprise. These projects collect so many contradicting features over time that correctness becomes logically impossible.

  11. amanfromMars 1 Silver badge

    Even Though Admittedly Still Very Odd, AI Seizes and Saves the Day and Faces with 0Days

    If an effective strings of suitable carefully chosen simple words creates, commands and controls and destroys worlds, to worry about dodgy code injections/viral systemic virtual machine infections is rather bizarre and presents yet another novel exploit field with a practically endless supply of critical vital vulnerabilities to assault with attacks or defend with crack hacks employing and enjoying Alien Instruction.

    A simple disturbing question to ask yourself is ........ Can I be immune and uninfected and remain not radically fundamentally affected whenever the masses around me so clearly are smitten?

    That’s the new hearts and minds battlefield to comprehensively conquer for those intent on inflicting sufferings and warlike defeats .... in order not to end up as you intend and contend and are content others need to be.

    A taste of one’s own medicine is a great leveller upper and AWEsome program downer.

  12. Darkk

    vmware is a mess

    If you manage a large vmware infrastructure you're in for LONG hours in patching all those vmware appliances such as vCenter and vReplication Appliance. Currently vmware don't have a patch available so have to use command line workarounds.

    So far the only saving grace is that the vsphere exsi host servers aren't affected by this.

    1. Abominator

      Re: vmware is a mess

      Looks like Citrix is in places.

      Work from home they said.

  13. Abominator

    And people said we need to use managed languages because they are more secure.....

    Managed language kills the world, through false sense of security. Oh and GC's all the frigging time.

  14. heyrick Silver badge

    Reading through the comments

    I think I can now understand why the Arduino IDE is so bloody slow (on my admittedly old PC). So it seems that Java is a double edged sword. Bonus - write once run everywhere.

    Malus - shall we talk about Java?

  15. Anonymous Coward
    Anonymous Coward

    Imagine working for a network services company when something like this happens. You have customers who run apache on their own servers. And what do they do? They contact you to ask if there's anything you can do on the network to protect them.

    Then imagine this query being repeated for multiple customers all day.

    Erm how about keeping shit patched up to date on your servers guys?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like