back to article Open source isn't the security problem – misusing it is

We're going to be cleaning up Apache Log4j security problems for months to come, but the real problem isn't that it was open-source software. It's how we track and use open-source code. When security vulnerabilities were found in the extremely popular open-source Apache Log4j logging library, we knew we were in trouble. What …

  1. Semtex451
    Facepalm

    "Is it a great time to be working in IT or what?"

    No.

    1. Joe W Silver badge

      Re: "Is it a great time to be working in IT or what?"

      Don't you hate it when people answer rhetorical questions?

      1. gobaskof
        Trollface

        Re: "Don't you hate it when people answer rhetorical questions?"

        Oh God, yes! So annoying.

        1. Norman Nescio Silver badge

          Re: "Don't you hate it when people answer rhetorical questions?"

          Replying with another question is worse, isn't it?

  2. SJA

    log4j

    To my knowledge log4j worked exactly as specified.

  3. SJA

    log4j works as specified

    To my konwledge, log4j works exactely as specified. It's just people who did not bother to check what it is and what it does and how it does it.

    1. Androgynous Cupboard Silver badge

      Re: log4j works as specified

      Exactly. None of the proposals mentioned in this article would have had any impact on it - they're not bad ideas, they just don't apply.

      The problem was scope-creep, which is not generally thought of as a security issue. It should be.

      1. Graham Dawson Silver badge

        Re: log4j works as specified

        The problem with scope creep is that developer ego is often invested in it, for reasons that I have never quite understood. One or two projects in particular come to mind...

      2. SCP

        Re: log4j works as specified

        "Exactly. None of the proposals mentioned in this article would have had any impact on it."

        If by "it" you mean the security issues arising from the use of log4j, I would agree. The article, however, is noting one of the problems of rectifying an issue once it is found - namely that it is difficult to know whether and where the software of concern is in your build.

        It is the post-event clean-up that the proposals seek to address by allowing corrective actions to be efficiently and effectively applied.

        I think SJVN [author] is right to be cautious as to when a good SBOM might be realised. To my mind the challenge will not be for the "fair wind" case, but in establishing a robust solution that will satisfy security needs (e.g. prevent falsified SBOMs).

    2. Greybearded old scrote Silver badge

      Re: log4j works as specified

      If the spec included accepting and running arbitrary code then the spec was beyond wrong.

      1. storner
        FAIL

        Re: log4j works as specified

        I would argue that the idea of putting Java bytecode in an LDAP attribute is also so horribly confused that it counts as a security issue in itself.

    3. SCP

      Re: log4j works as specified

      A problem is that, as a code production driver, functionality often over-rides all other drivers. Quality and maintainability would be lucky to get a look-in.

      The advancements that have been made in automated code analysis over the decades I was working still very rarely get a look-in - and these are quite capable of finding significant faults in code.

      (A problem [IMHO] is that the tools often require the developer to adapt their approach to design and code writing to be better suited to analysis and avoid numerous "warnings". The unwillingness to adapt to improve is reinforced by cost metrics that favour churning out quick solutions using familiar approaches.

      It is worth noting that there is a flip side in that pedantically adhering to particular strictures on design/code writing can preclude some neat and efficient solutions that would reduce the overall risks in a programme.)

  4. Ian Johnston Silver badge

    As the programmer Xe argues, "'Open Source' is broken" because even now no one is paying the developers.

    Why did they ever think that anyone would pay?

    1. spireite Silver badge

      Yeah, but arguably so is closed source.

      It's not like Developer A stays at the company for their entire life.

      Support has - almost by definition - stopped as soon as that Developer walks out the door.

      There are many times where software is written by someone, and that someone IS the only person that knows it in much detail - or indeed the platform it's written in!!

      You can say - that's what hand over is for..... but I've yet to to work anywhere in my 30-odd year career where handover has occurred in anywhere near enough detail

  5. Anonymous Coward
    Anonymous Coward

    Until we reach the point where people understand the real cost of software, this will keep on happening.

    Open Source deals with the initial cost of developing the software, and gives it away for the price of massaging the ego of the developers.

    We don't have as big an idea to keep the software working for free, until the very last person using it, no longer needs support.

    So who is happy to do that thankless task with the same vigor?

    And that great big silence is the problem that really needs addressing!

    1. Warm Braw

      People understand the real cost

      There has rarely been an industry in which the "real cost" has been understood - or, at least, paid for by the industry concerned. We have a legacy of spoil heaps, polluted watercourses, contaminated land, collapsing ecosystems, death, disease and - of most concern - a rapidly heating planet to prove it.

      Software maintenance is just another "externality" - a cost for which someone else is expected to pick up the tab.

      1. Version 1.0 Silver badge
        Joke

        Re: People understand the real cost

        I have posted this many times, it's an updated Brendan Behan quote that I think defines the open source issues very well (the original quote has always made me laugh), I see so many of Behan's views and quotes needing only a small update to describe our problems these days:

        "The big difference between software for money and software for free is that software for money usually costs a lot less."

    2. Fred Daggy Silver badge
      Alien

      I would argue that almost no one knows the real cost of anything.

      Take a bunch of carrots. Mostly, I am going to need to wash them, so, water. Store them, cool, so refrigeration. Cook them, electrickery plus perhaps water again, and depreciation on the cookware.

      I can know the price I paid, and the value I get, but never the true, final cost. The more complex the product, the less I will know about price and value, also.

  6. boblongii

    Open Source is an Opportunity

    Open Source is an opportunity to read and modify the code, or pay someone to do it for you. That's more than most paid-for software packages do for you.

    If you don't take that opportunity, that's your fault.

    1. Ian Johnston Silver badge

      Re: Open Source is an Opportunity

      A lovely idea, and quite practical if the source is a couple of hundred lines long. If it's the Linux kernel, not quite as much.

      Something wrong with your browser? Here are the dependencies for Firefox. How many lines of source there:

      lsb-release, libatk1.0-0 (>= 1.12.4), libc6 (>= 2.27), libcairo-gobject2 (>= 1.10.0), libcairo2 (>= 1.10.0), libdbus-1-3 (>= 1.9.14), libdbus-glib-1-2 (>= 0.78), libfontconfig1 (>= 2.12), libfreetype6 (>= 2.3.5), libgcc1 (>= 1:3.3), libgdk-pixbuf2.0-0 (>= 2.22.0), libglib2.0-0 (>= 2.42), libgtk-3-0 (>= 3.14), libpango-1.0-0 (>= 1.22.0), libpangocairo-1.0-0 (>= 1.14.0), libstdc++6 (>= 6), libx11-6, libx11-xcb1, libxcb-shm0, libxcb1, libxcomposite1 (>= 1:0.3-1), libxcursor1 (>> 1.1.2), libxdamage1 (>= 1:1.1), libxext6, libxfixes3, libxi6, libxrandr2 (>= 2:1.4.0), libxrender1

      1. boblongii

        Re: Open Source is an Opportunity

        What are the dependencies of MS Explorer? what are you going to do about them? What about Photoshop? Can you even see what it depends on?

        Modern software is full of deep dependencies. Open Source doesn't change that, neither does Hidden Source. What it does do is give you the chance to do something about a bug even in software which is no longer supported by whomever you got it from. As well as the ability to customise it for your own specific use, of course, which is hardly chopped liver.

        If a company that writes its own software was struck down with a bug like log4j and then claimed it wasn't their fault because they never test the code their programmers produce and just deploy it, everyone would say they were insane. Yet that is exactly what many people do with code that people that don't even work for them have produced.

        Log4j represented a lot of work each user then didn't have to do; the normal reaction is to call that 100% profit and spend all the time saved on other things, or the money saved on buns. That's their choice, but it is a choice and whining about it when it goes wrong is simply idiotic. Go with 50% profit and pay for testing/checking. Or write your own from scratch; no one owes you a living.

        The "lovely idea" is that Free Software is a collaborative enterprise and if you're not collaborating then, basically, you have no right to expect anything at all from those that are doing the work.

      2. AdamWill

        Re: Open Source is an Opportunity

        I've found and fixed bugs in situations like that, with which I was not previously familiar at all. It's not impossible; it's not even all that difficult, with experience. There are many techniques you can use to short cut the process.

        Here's one I documented a few years back, which started with me trying to rebuild a package for Python 3.6 and wound up with fixing an integer type bug in matplotlib, a giant code base I was not previously familiar with at all:

        https://www.happyassassin.net/posts/2017/01/11/the-tale-of-the-two-day-one-character-patch/

        I do this kind of thing quite a lot, and thanks to open source, I don't need anyone's special permission to dive into this sort of thing in all sorts of projects I've never seen or touched before. I couldn't do this for proprietary software.

    2. Norman Nescio Silver badge

      Re: Open Source is an Opportunity

      Open Source is an opportunity to read and modify the code, or pay someone to do it for you. That's more than most paid-for software packages do for you.

      Well, up to a point. Open Source does not necessarily give you the right to distribute modifications to the code. For that you need a 'libre' licence, or code that really, truly, is in the public domain. Open Source allows you to read the source. If you make modifications and use them privately, that's fine. Distributing the changes: copyright becomes an issue.

      Open Source also says nothing about Digital Rights Management (DRM). If the code needs to be signed to operate, you are beholden to whoever has control of the signing key.

      You really need code under a licence that allows you to read the source, make a reproducible build, distribute modified code, and run it on your own hardware without needing permission from a third party (aka DRM).

      NN

      1. AdamWill

        Re: Open Source is an Opportunity

        "Well, up to a point. Open Source does not necessarily give you the right to distribute modifications to the code."

        Yes it does.

        https://opensource.org/osd

        "3. Derived Works

        The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software."

        1. Norman Nescio Silver badge

          Re: Open Source is an Opportunity

          It's always good to have your convictions challenged, and in this case it turns out I was wrong.

          I'll say that again: I was wrong.

          So thank you for putting me straight.

          I may have been mislead by the antipathy that existed between the Open Source community and the Free Software community, as alluded to in this blog entry by Bruce Perens (the original author of the Open Source Definition).

          On Usage of The Phrase "Open Source", September 26, 2017

          It is unfortunate that for some time the Open Source Initiative deprecated Richard Stallman and Free Software, and that some people still consider Open Source and Free Software to be different things today. I never meant it to be that way. Open Source was meant to be a way of promoting the concept of Free Software to business people, who I have always hoped would thus come to appreciate Richard and his Free Software campaign. And many have. Open Source licenses and Free Software licenses are effectively the same thing.

          This random article gives a little background to the antipathy:

          WhiteSource: Open Source vs. Free Software – What is The Difference?

          But ends with:

          Why it Matters

          I have to admit, it usually doesn’t.

          While the FSF and the OSI are primarily focused on creating free and open software—oftentimes working together toward that goal—they don’t see eye-to-eye in their approach.

          While the free software fans are quick to tout their philosophy as a “purer” vision for software development and often look down on their open source counterparts, it’s important to remember one of the reasons why the OSI came into being: to aid corporations in the adoption, usage and contribution towards a larger open source software community.

          Thank you again for correcting me.

          NN

  7. Pete 2 Silver badge

    six million flies

    > the real problem isn't that it was open-source software

    No. The problem is that it takes virtually no skill to download a package and follow what every other user does with it.

    Having the knowledge and experience (and time: paid for by a person's employer) to perform a security audit, or even a risk analysis, on that software is a rarity.

    Much of the popularity of FOSS packages is the knowledge that lots of other people and companies use the same stuff. So it's gotta be OK, hasn't it?

    Whether security is a product or a process is not really relevant. The problem is that nobody takes responsibility for FOSS. Nobody has to fix problems when they arise, get written into new releases or are discovered. That lack of enforcability and the associated freedom from obligations, is the major factor behind such software being free (as in beer).

    1. Ian Johnston Silver badge

      Re: six million flies

      It's a classic example of a perverse incentive, To commercial software producers, support is a cost and there is therefor a financial incentive to receive the amount of support needed as much as possible. To free-as-in-beer software producers support is an income stream - in fact its the only income stream - so there is a strong incentive yo increase the amount of support needed.

      It doesn't always work that way, of course, but that's the financial pressure.

    2. AndrueC Silver badge
      Meh

      Re: six million flies

      No. The problem is that it takes virtually no skill to download a package and follow what every other user does with it.

      That's not a problem. That's a virtue. We want using these packages to be simple because otherwise we might as well write our own. That's the whole point of technology. To wrap complicated things up and make them available to more people.

      This problem isn't unique to software development. Every industry relies on 'off-the shelf' components and that's all a package is. What's missing from software development is accountability and a proper standards authority.

      When you buy a telephone from your local supplier you can fit it without knowing anything about the internals of it. All you know is what the connections are. That's exactly the same as a software package. The difference is that the telephone will have logos printed on it that assure you (subject to it not being a knock-off) that the relevant organisations have tested it and verified that it's safe and meets all the required standards.

      The problem is that because software is so easy to modify and inherently complicated and evolves so rapidly I don't see how we can develop any kind of 'kite mark' or international standard authority for it.

      1. inglenook

        Re: six million flies

        " I don't see how we can develop any kind of 'kite mark' or international standard authority for it."

        Funnily enough, there ARE international standards relating to software quality and software security. And some software even claims adherence to those standards. In some sensitive businesses, buyers demand adherence to such standards.

        However, this is very much a minority game. It would be good to change this situation, although the pressure to do so for open source software is pretty well non-existent.

        1. SCP

          Re: six million flies

          And typically the costs of meeting these standards is not cheap [even if you exclude buying the standard in the first place]. There was an OpenDO project (https://www.open-do.org/) that aimed to address some of the challenges - but that looks to have stalled/died around 2015 (still some useful pointers to work being done).

    3. AdamWill

      Re: six million flies

      "Whether security is a product or a process is not really relevant. The problem is that nobody takes responsibility for FOSS. Nobody has to fix problems when they arise, get written into new releases or are discovered. That lack of enforcability and the associated freedom from obligations, is the major factor behind such software being free (as in beer)."

      This just isn't true, though. Certainly not as a blanket statement. In general, the people who write F/OSS software take responsibility for it. It's not like the log4j bug wasn't fixed, is it? The Apache Foundation (under whose umbrella the project falls) and its maintainers took responsibility for it. It was fixed extremely quickly - much more quickly than many security bugs are fixed.

      Some F/OSS software is abandoned, or its nominal maintainers don't fix major problems. But this is also true of some proprietary software. If you bought a proprietary application, you likely have no practical way to "enforce" that the developer provides fixes for major problems promptly. If you bought a subscription or support contract, you might at least theoretically have that leverage - but then, you can buy subscriptions or support contracts for F/OSS too. Plenty of companies will be happy to sell them to you. I work for Red Hat, and we certainly had a "drop everything" level priority event to ship the log4j fixes to every product we provide that includes it.

      There's basically no justification for any blanket statement about F/OSS or proprietary software in this area at all. The standards of security testing and support vary widely across different F/OSS projects. They also vary widely across different proprietary projects. You should assess this individually for any project you want to use or depend on, whatever its license is. Generally speaking, a project being F/OSS-licensed provides a potential benefit in this area due to the "many eyes" effect, but that's a relatively small consideration overall. A well-maintained and supported proprietary codebase will still likely be superior to a F/OSS one which was abandoned years ago. But that doesn't have anything to do with the proprietary codebase's proprietariness or the F/OSS project's F/OSSiness.

      1. TheMeerkat

        Re: six million flies

        Red Hat drop everything to fix stuff because otherwise it won’t have an income stream. The same as any company selling proprietary software would do, no difference.

        Red Hat is parasitising on what many developers have done and are doing for free, so it is not a “knight in shining armour”.

        And log4j fiasco proved that idea of “many eyes effect” is just fiction. Nobody noticed the issue for very many years.

  8. Tilda Rice

    LOL. The myth that open source is more secure because anyone can look at the code.

    Stephen J Vaughan article though. So to be expected.

    1. doublelayer Silver badge

      Did you read the article? How about this bit:

      If, and it's a big if, those eyeballs are there and looking. If the code just sits there getting copied over and over again without a moment's thought, no bugs will be found. Simple, isn't it?

      or this bit:

      But would Goers get paid to go over old Java code with a fine-tooth comb looking for security vulnerabilities even if Oracle were to hire him just to work on Java? I doubt it. Coders are paid to make new code, not fix old code.

      The argument really does hold some water, but since you're incorrectly attributing that argument to the article author, you're already going the wrong way.

  9. EnviableOne

    Open source isnt the problem

    the apache licence, clause 7, last sentence:

    You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.

    the problem is no-one assess the risk in their situation

  10. Swarthy
    Facepalm

    Definitely how it's used...

    I just spent an evening patching log4J libraries on Production (late night, out-of-hours patch for Prod) One application had three different versions (2.11, 2.09, and 2.15) in five different locations.

    Because it's free, and it makes for a very handy debugging logger - every development team uses it. Then when it gets packaged they just roll it all up, because removing the redundant libraries counts as fixing "old" code.

  11. This post has been deleted by its author

  12. Anonymous Coward
    Holmes

    Bug bounty

    While giving support to or funding programmers is a positive step and SBOM is a great idea, one wonders if funding people to review old code, starting with the code in OpenSSF, through a bug bounty might be the most productive thing to do.

  13. Clausewitz 4.0
    Devil

    Security is also a Product

    I have code and it exploits some holes and/or zero-days, I have a product.

    The process being, to use encryption to keep my intellectual property safe from prying eyes, until payment and deals are done.

  14. Unbelievable!

    As charlie chan famously never said .. no person by themselves is a guru

    As charlie chan famously never said .. "no person by themselves is a guru"

    A guru picks up on lessons learned by others mistakes

    (probably)

  15. Ptothgriffiths

    If every cloud server is dark with all inbound ports close, do we care about Log4shell?

    If we embed private, zero trust connectivity embedded into applications and systems using open source components then we can close all inbound ports while ensuring transparency to users. This disrupts the Reconnaissance and Initial Access Tactics (as defined by MITRE ATT&CK) of malicious actors as well as restricting lateral movement - external network-level attacks (CVE or zero day exploit, DDoS, brute force etc) become all but impossible.

    1. doublelayer Silver badge

      Re: If every cloud server is dark with all inbound ports close, do we care about Log4shell?

      I think this is a bingo for anyone who chose a card for "meaningless technobabble buzzwords for security project".

      We have that. It's called IP. It has ports, and you put a firewall over them. That lets you connect some things to the internet and block others. It works great. It would have worked great for this vulnerability too, as long as people remembered to turn on the firewall part. All the components used to do this can be open source and often are. They're designed for a zero-trust environment, the internet, and the problem is when the software being contacted doesn't properly check what the user wants to do and whether they should be allowed to do it (or the software being contacted shouldn't be contactable).

      To the extent that your suggestion means anything, it appears to mean that we shut down the IP system and make another one, so that any bot scanning ports won't find anything. If that ever works, then bots will be rewritten to scan the new one again.

  16. SCP
    Joke

    Every dark cloud has a silver lining ... which is a backdoor that is ajar.

  17. fidodogbreath

    Coders are paid to make break new code, not fix old code.

    FTFY

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like