back to article When one open-source package riddled with vulns pulls in dozens of others, what's a dev to do?

Open-source security specialist Snyk has released a new survey combining data on vulnerabilities in available packages with responses from developers and DevOps teams about how they handle the challenge this poses. Snyk's Open source security report, 2020 Click to enlarge (via Snyk) The problem is easy to express. Software …

  1. Def Silver badge
    Facepalm

    Or we need to code defensively so that if a vulnerability exists, which doesn't have a fix, we do enough input validation or whatever is needed to avoid that code path from being attacked.

    Validating inputs isn't defensive coding. It should be standard practice. If you, as a software developer, are not rigorously validating 100% of your user inputs, your computer access rights should be revoked.

    1. DavCrav Silver badge

      "Validating inputs isn't defensive coding. It should be standard practice. If you, as a software developer, are not rigorously validating 100% of your user inputs, your computer access rights should be revoked."

      My guess, given the topic, is that this is talking about validating other inputs. You sanitize your user input, mess with it, then pass it into some third-party code. My guess is that the author then wants another set of revalidation and sanitization, in case the third-party code is compromised.

      1. Def Silver badge

        Yeah, I understand that.

        It's a bit of a circular argument though. Any third party library should be validating any inputs it is receiving.

        And yes, I realise the lack of validation there is the issue being discussed. It doesn't invalidate my initial statement though. If you're providing code for others to use you should be extra vigilant when it comes to input validation, and you should never rely on external systems to provide validation for you.

        1. DavCrav Silver badge

          "It's a bit of a circular argument though. Any third party library should be validating any inputs it is receiving."

          Yes, but it's where you put your trust. You don't trust the user as far as you can throw them but you do trust most modules you suck down off some repository. Perhaps you should be treating them as cack-handed or malicious as well.

          1. Def Silver badge

            Trust no-one. :)

          2. big_D Silver badge

            If you haven't written the code yourself, it isn't trustworthy.

            If you wrote it yourself, you only have yourself to blame when it isn't trustworthy.

            The first thing you should be doing with these imported modules is running your own security tests on it, before letting it anywhere near a public server. The problem is, most projects are now importing so much code this is not practical.

        2. SmartAlec

          I take the stance it's a bit like a calling convention, we agree on a convention and stick to it. Incurring the checking penalty every time is extreme so we don't.

          Constantly checking is really bad..... yet we sort of have to. Consider a stack that checks it has something on it before popping. Should we check every time? Return nullptr (or some smart ptr that == it?) exception... it's really nasty.

          It's temping to say "in any event (return null, exception, ect) /some/ caller if not the immediate is going to have to deal with it, so why not check it makes sense to call (as best as we can) first?"

          Having seen no way to unify these opposing views (or even decide if "interface to users demand checking" applies to libraries) in my spare time I've been working on something "transactional" instead.

          In this model every map key, list activity, state change ect ect either happens in full, or is rolled back, and let me tell you it's a fucking dream come true. In C++ an exception going past will free resources but not undo stuff, in this an exception going past completely undoes (or more correctly, never applies) whatever "happened".

          The only caveat is things like "drill hole" these can't be undone, or "send email" - these are queued to happen on success of the ENTIRE thing (you know like how SQL transactions nest, commit is right at the end but nested ones form checkpoints - it's just like that) - and the user has to know about failure and to retry there.

          I really love it, not to brag about how awesome I am but you literally don't have to pretend to care.

          What's the catch? Written in PHP, but it works hand-in-glove with MySQL transactions, you can still sort of subvert it, but it's very hard

          You'd need a managed environment for such a thing but honestly it's been a great experiment.

          Don't confuse this with transactional memory semantics from the TSX extensions and that, but it has worked. I've been debating releasing it because it lets me be so lazy and build stuff so quickly, if it doesn't work, bail, no harm done. It's also surprisingly quick.

          But also PHP...

    2. juice Silver badge

      True, but...

      > Validating inputs isn't defensive coding. It should be standard practice. If you, as a software developer, are not rigorously validating 100% of your user inputs, your computer access rights should be revoked.

      I'd agree with this, but the problem here isn't *your* code.

      It's the code in library A which you're pulling in. Which is then pulling in libraries B and C. Which are then pulling in libraries D, E, F, G, H. And so on.

      No matter how rigorously you're validating your own code, there's no way to verify that the pyramid of "external" code you've pulled in is safe, unless you're going to manually review it all.

      To use a recent personal example: I needed to compile some javascript using gulp. And after much failed faffing with nvm/npm, I gave up and grabbed a tarball of the required libraries from somewhere else.

      My code: a few hundred bytes, atop a few hundred kilobytes of existing in-house code.

      The code needed to get gulp working: 174 megabytes.

      To be fair, there's probably a lot of legacy cruft in the compilation system. And I'd guess less than 1% of that code actually gets fired up when compiling my code. But even so, it would take weeks or months to verify the pyramid of code which is being used, and I'd then have to repeat this exercise whenever a library is updated and/or a new dependency added.

      And therein lies the issue.

      In theory, that's the beauty of open-source software, in that you have access to the source and do have the ability to review it. And since everyone else can do the same, all code should be perfect and bug-free!

      In practice, many open-source packages only have a small number of contributers (assuming they haven't been abandoned/forked/etc), and the amount of oversight is limited.

  2. Smooth Newt Silver badge
    Pirate

    The problem is easy to express

    It's the lack of compartmentalization in conventional web software frameworks. These should be analogous to Android and IOS where each software package runs in its own sandbox with access only to the particular set of features that it needs.

    You are never going to be able to police and properly maintain all these software packages, so there will always be security (and other) bugs. And validating inputs only gets you so far - whilst a bit of simple syntax gets you past the stupid, simple problems like SQL injection, once you start thinking about context it can be enormously difficult. So a solution is to assume that these bugs are going to be present and design the framework in which they sit accordingly. Hence some security compartmentalization.

    1. Nick Ryan Silver badge

      Re: The problem is easy to express

      It also often feels like most developers are like a passing person that, using a pile of generic bricks and some ready made generic mortar, believes that this means that they are both an experienced builder and an experienced architect and able to use these to construct a tower block, a bridge or a watertight dam.

      1. DemeterLast
        Mushroom

        Re: The problem is easy to express

        It doesn't help that all of these frameworks, libraries, toolkits, etc. all put a gigantic Smiley Face all over their web site and documentation. "FooLib makes YOUR_PROBLEM_HERE so easy! And it's better than BarLib!"

        Then they demonstrate how easy it is to do something that's stupidly easy already.

        Unremarked upon is the 12.75GB of third-party libraries their framework, library, toolkit, etc. pull in as dependencies, which you barely even see scroll past when you run their recommended Yarn/npm/Composer/pip/etc. one liner to grab all that stuff. Or, they just tell you to download and use their stupid freaking Docker recipe.

        "It's so easy!" No it isn't. It never is. And your shiny happy people attitude is giving impressionable young developers hope and optimism, when what they should be feeling is despair and cynicism. This modern drive towards acceptance and unity and inclusiveness is killing development. Developers should be angry, bitter, hateful towards users, and above all deeply skeptical of anything new. Security becomes super easy when nobody is allowed to do anything.

        1. Nick Ryan Silver badge

          Re: The problem is easy to express

          @DemeterLast - I'm trying to work out how seriously I should be reading your post! :)

        2. Throatwarbler Mangrove Silver badge
          Thumb Up

          Re: The problem is easy to express

          "Developers should be angry, bitter, hateful towards users, and above all deeply skeptical of anything new."

          I see you've met El Reg's resident developer community.

  3. Phil O'Sophical Silver badge
    Flame

    if a developer is pulling in unchecked and untested third-party packages, and including them in a 'product' that is shipped to their customers, then that developer should face the full force of any legal penalties incurred by their customers due to the developer's incompetence.

    If an aircraft manufacturer bought components from a supplier, and just installed them without consistently checking they they met the safety specs, would they get away with saying "not my fault, blame my supplier"?

    It isn't 'agile', it's 'lazy'.

    1. Roo
      Windows

      First to market is king in (most) software development, first to go bang is rarely considered by the folks throwing money at it.

    2. DavCrav Silver badge

      "... and including them in a 'product' that is shipped to their customers ..."

      One glaring security hole, mentioned on The Register in occasional stories, is pulling in these packages on execution, rather than using a fixed, immutable version of the code. (Am I remembering right: BA had this problem?) Then if the package gets turned off or maliciously updated, you are screwed. Sometimes borkage can occur innocently, but I would worry more about the opportunity for malice.

    3. Charlie Clark Silver badge

      I don't think the law is on your side. Currently, software developers are not subject to strict liability, which is why we live in a world of updates. Then there are the licences: most open source licences explicitly exclude liability and I've not seen the clauses invalidated by any court yet. It's not as if commercial software is immune to such stuff either: both the flaws in own code but also in liberal use of open source libraries.

      While it is possible to check for known vulnerabilities in libraries, there's basically no way around extensive pen testing for modern web-based applications. But getting customers to pay for these is another matter. As is paying for updates of the software stack as new vulns get discovered.

      1. Phil O'Sophical Silver badge

        I don't think the law is on your side.

        I wouldn't be so sure. I work for a large international company, and the legal hoops I have to go through just to upgrade the version of some 3P code we choose to use are impressive. That even includes cases where we don't ship the code but just tell the end-user "you have to download xxx". The company lawyers obviously think there's a risk.

        1. Charlie Clark Silver badge

          Yes, I've seen that behaviour in some companies as well. It used to be the argument against open source, until it turned out that commercial software was just as shit but you wouldn't have a chance of finding out until it was exploited.

          Personally, I don't agree with either the current practice (all problems can be solved by an update) or the disclaimers in many licences. But the point is that I can't think of any court cases. Unlike, say, those that have upheld the GPL. No doubt there will be some case at some point but IIRC in the US there are some wide-ranging excemptions. Otherwise Microsoft would probably have been bankrupted multiple times in class action suits over Active X, which wasn't just an oversight but a design goal waiting to be exploited.

      2. Jim Mitchell

        "Then there are the licences: most open source licences explicitly exclude liability and I've not seen the clauses invalidated by any court yet."

        I'm not a lawyer, but in the physical world, I don't think liability waivers don't cover negligence, even if they say they do.

    4. Anonymous Coward
      Anonymous Coward

      Inspection+test at "goods inward"? How quaint.

      "If an aircraft manufacturer bought components from a supplier, and just installed them without consistently checking they they met the safety specs, would they get away with saying "not my fault, blame my supplier"?"

      Do I take it you're unfamiliar with the miracle of ISO9000 and related Quality Management Systems?

      As long as you get the bureaucracy right, nothing can possibly go wrong... go wrong... go wrong...

      1. Phil O'Sophical Silver badge

        Re: Inspection+test at "goods inward"? How quaint.

        Do I take it you're unfamiliar with the miracle of ISO9000 and related Quality Management Systems?

        Sadly not, I remember having to implement that back when it was still just BS5750. At the time it was so easy to confuse the auditors that "not worth the paper it was written on" was the kindest thing one could say.

        1. Anonymous Coward
          Anonymous Coward

          Re: Inspection+test at "goods inward"? How quaint.

          We implemented ISO9000 along with IIP and a few other hallmarks at basically the same time.

          At the time I was a more junior member of the team with a reputation for getting jobs done. I was passed a bunch of documents that needed all of the company procedures documenting in. I diligently filled in all of the ones I could, and put notes in ***ACTION REQUIRED; NO IDEA WHAT THE PROCEDURE IS FOR THIS*** where I couldn't fill it in so you could easily do ctrl-f to get to the appropriate bits of the documents and explained what i'd done when I sent the files back.

          I was thanked for my hard work, it was duly passed on and up through several layers of management getting bits added, and then ended up being given to the auditors and being passed.

          After the auditors had been, I was curious as to exactly what the procedure was on some of those really obscure things, so I flipped open the manual and went reading.

          Apparently the company official procedures in many areas signed off by about five levels of management and an auditor was ***ACTION REQUIRED; NO IDEA WHAT THE PROCEDURE IS FOR THIS***.

          I continue to assert that I am the only person to ever fully read the procedure manual.

          1. Anonymous Coward
            Anonymous Coward

            Re: Inspection+test at "goods inward"? How quaint.

            Not unusual. Just as one audit started I realised that we were missing an important document. To buy time I sent the auditors a similarly-named document from another project, assuming that by the time they noticed the mistake I'd have written the missing one and could just apologise for the 'error'. I needn't have bothered, they approved the audit as submitted.

        2. Kristian Walsh

          Re: Inspection+test at "goods inward"? How quaint.

          The flaw of ISO9000 is that it assumed that a business, on seeing the true horror of their process, would take action to simplify it. This was true in some businesses that introduced ISO9000 as a way to improve their operations, but not so in others, whose motivation was basically to do the minimum required to be able to print "ISO9000 certified" on their brochures.

          Later quality processes such as WCM move away from process and focus on the general concept of eliminating wasted efforts. That change of focus makes "...and now, go and fix these crazy things" part of the standard operating procedure.

          But you've still got to have a company that doesn't just talk about quality, while acting in every single way that's guaranteed to diminish it.

    5. amanfromMars 1 Silver badge
      Mushroom

      Shit happens ..... is no excuse for a remedy

      if a developer is pulling in unchecked and untested third-party packages, and including them in a 'product' that is shipped to their customers, then that developer should face the full force of any legal penalties incurred by their customers due to the developer's incompetence. ..... Phil O'Sophical

      That turns very interesting, Phil, whenever one considers the collective responsibility of the Conservative Party Cabinet Office and their Covid 19 developments pulling in unchecked and untested third-party packages, and including them in a 'product' that is shipped to their customers, incurring their customers legal penalties and crippling losses due to the developer's incompetence.

      And please, let's not be having any halfwit saying that's different and the liability does not apply .... just because it is an inconvenient and dangerous truth to be held responsible for.

    6. William Towle
      Happy

      > ...full force of any legal penalties...

      Nod, +1.

      I got my first security advisory email from GitHub this week due to a "vulnerable dependency in repositories[...]", which was nice to see because it's a reassuring upshot of automation being done on checking this sort of thing.

      Unfortunately due to "no server [...] currently available to service your request" I haven't yet read the page with the details, but at least I can see from the email's summary that it's not in any recent personal uploads or even in a repository I've got write access to (plus I can trust others to pay/have paid appropriate attention) and take comfort from that.

  4. Roo
    Windows

    Minimize dependencies

    Minimizing dependencies is the rational way to minimize the vulnerabilities that are pulled in. The vast majority of code I work with pulls in a vast amount of unrelated and unused code, the proportion of stuff actually used tends to be tiny - and worse still quite a few of the imports are redundant - they are replicating stuff that's already available in the stdlibs or other imports.

    YMMV, the stuff I work with on a daily basis has been worked over by a team with a high turnover of staff where the focus is very much on adding new stuff rather than fixing technical debt for over a decade, so I recognize that my viewpoint might not fit everyone's situation. :)

    It's interesting that the report identified the imports brought in by containers as a problem - it's an issue that has been glossed over by too many people for too long. Glad they brought that up because I am seeing containers broadening and deepening the pool of dependencies in practice because Devs now get to pick and choose what bits of the OS they want - and it's rare that two of them agree. :)

    1. Ben Tasker Silver badge
      Joke

      Re: Minimize dependencies

      > they are replicating stuff that's already available in the stdlibs or other imports.

      Ah, you're being subjected to jQuery

      1. Roo

        Re: Minimize dependencies

        ... and Spring

        ... and Boost

        ... and APR

        ... and ... etc... etc... etc...

  5. Anonymous Coward
    Anonymous Coward

    Speaking of bad code...

    ... what the hell is that graph? Stacked columns but stacked by years? Sorting by total over multiple years? Masking the trends over time. Should have been column per year stacked by category, top to bottom, best to worst.

    Then you'd notice the disturbing trend in malicious packages.

  6. Julz Silver badge
    Facepalm

    Other Peoples Code

    On other peoples computers; what could possibly go wrong.

  7. Nick Ryan Silver badge
    Facepalm

    The obligatory XKCD...

    https://xkcd.com/2054/

  8. amanfromMars 1 Silver badge

    In Praise of Future Scapegoats ......

    The result is a big chunk of code that gets deployed with the application, but was not written by the developer and may include security vulnerabilities.

    Yes, it can so include, however is there a problem whenever tempted to realise it definitely includes security enhancements with Almighty Overwhelming Powers?

    Would such incite to crazy battle or civilised engagement in current iterations of reality, virtual or otherwise?

    What would you reasonably expect to have always prevailed to always prevail even to this day in these strange days/surreal times/spaces/heavenly places/hellish hideaways?

    Jaw Jaw or War War Ware? One for the Sane and the Other Ideal for Insane Military Complexes ‽ .

  9. Smartypantz

    10000 instances

    So it apparently dawned on some people that if you download 10000 runtimes from "random internet dude" and deploy them in docker/kubernetes (because you know fuck-all about deploying servers) you have to maintain 10000 instances of security patches 1½ year later.

    About fucking time! Container-dazed idiots!!

    Better find that coffee and smell it!

  10. FlamingDeath

    In the real world

    Even house builders probably create their own cement, and understand its nature.

    Fudging crap together you do not fully understand just leads to ballsups, and while I’m of the view that no problem should be solved twice, if the solution is creating more problems, it isn’t a solution

    1. Glen 1 Silver badge

      Re: In the real world

      Yes, but how many levels of abstraction below the one we work on do any of us really understand?

      There has to be trust *somewhere*, else we would all be (bad) chip designers. Or possibly (bad) kernel developers.

      There comes a point where you call a function and "magic" happens.

      1. A.P. Veening Silver badge

        Re: In the real world

        There comes a point where you call a function and "magic" happens.

        If I have anything to say about it, that "magic" is supplied by the compiler manufacturer and there aren't that many of those I trust. For some reason I have a tendency to prefer IBM compilers.

        1. Glen 1 Silver badge

          Re: In the real world

          Sooooo you trust the operating system? the CPU manufacturer?

          While its not *quite* turtles all the way down, there are more layers than most people will see in their entire careers.

          1. A.P. Veening Silver badge

            Re: In the real world

            My preferred OS is also from IBM (OS/400), but yes, you are right, it is turtles all the way down. Having turtles of good pedigree does however help a lot.

  11. Anonymous Coward
    Anonymous Coward

    I feel regulation creeping in

    So like Electricians, Plumbers, Building engineers, Programmers/coders may require a licenses at some point, that can be revoked for sloppy/bad coding. Those not licensed yet (journeymen) will work under a company's license, which will pay large fines for bad code published.

    1. karlkarl Silver badge

      Re: I feel regulation creeping in

      Oh gosh you could be right. A dangerous world ahead.

      Microsoft has so far "bought" the professional portfolio market with LinkedIn, NPM and GitHub so they will "obviously" be in the best position to govern the regulations (gulp!).

      Good luck not using the latest Microsoft technology if you plan to try to remain in good standing XD

  12. ovation1357

    I still can't quite fathom why the hell anyone would use Node JS - whilst there is definitely plenty of dependency overload from many other languages and frameworks, node seems to take this to a whole new level.

    It's a couple of years since I last used it but I recently had to use npm to install the dependencies for a well known cloud storage provider's CLI utility... This led to the download of over ONE THOUSAND external libraries!

    Now: I will grant that it does at least have a built in vulnerability checker of some kind but this flagged up nearly 300 known vulnerabilities of which only about 200 could be automatically fixed by upgrading dependencies, the others were blocked by things like known API breakage in the newer (fixed) version.

    All this for the current stable version of a fairly basic command line utility! And someone chose to build it in a language with absolutely no type safety, that can't even get basic comparison operators right. (https://www.destroyallsoftware.com/talks/wat)

    I work with some air-gapped environments and one of the most galling things I have to deal with regularly is that just about every package manager is built with reliance upon an internet connection and has offline installation as an afterthought at the very best. In the case of npm it seems to be that the official answer is there is no offline install option: cue a multitude of projects to serve up local mirrors to fool npm into thinking it's connected :-(

    (Actually, I found a node module called 'pkg' which 'compiled' my CLI into a whopping 50mb standalone executable. It's really ugly and totally huge, but at least it it's smaller than the 300mb of JavaScript the npm pulled down and it seems to actually work too).

    I'm certainly getting the impression that many of the developers that I meet know very little about the underlying fundamentals involved in their applications (even simple concepts such as file descriptors and network sockets) and instead have become specialists in clipping together various Lego™ bricks to make something that vaguely resembles a program.

    There's a balance to be found in all this. Pulling in a huge library only to make use of one convenient function call just isn't cool. But on the other hand it's also not worth trying to reinvent the wheel every time. Common sense and careful planning both need to come back into the daily development lifecycle.

    1. Elledan

      I had the joyful experience of working on an embedded JS project (commercial) once, that involved a significant amount of NodeJS and NPM fluff.

      You don't understand how bad NPM is until you have glanced at the node_modules folder, noticed it's 1+ GB for a modestly sized project, and finding that deleting said folder in MacOS takes many, many seconds, SSD be darned.

      And don't get me started on NodeJS. Using NodeJS for 'embedded' means having a literal Hello World app sucking down 70 MB with just a basic comms framework in place that does some JSON parsing and writing.

      The effect of all that has been that I now refuse to do anything 'embedded' that doesn't at the very least involve a significant amount of C, C++ or Ada.Even TypeScript won't lure me back into the wonderful world of NodeJS.

    2. sabroni Silver badge

      re: that can't even get basic comparison operators right.

      JavaScript comparison operators come in two flavours, one that coerces types and one that doesn't. If you pick the one that coerces and don't understand the results then you're a bad workman blaming their tools.

      All languages require you to understand them before you use them.

  13. Paul Johnston
    Flame

    What can you expect?

    "The second step, however, isn't to triage across the org, but rather to start fixing issues. Teams who overspend energy triaging at the expense of fixing end up incurring greater risk.

    When a very large proportion of the team are "managers" whose sole job it to push bits of paper around and try to look busy where is the incentive to actually fix something?

  14. LucreLout Silver badge

    Easy and yet ...

    ...the author completely overlooks the problem of scale.

    The problem is easy to express. Software development today typically makes use of packages from online repositories. A developer sits down to create a web application and starts by installing libraries from npm.js (more than 1 million JavaScript packages to choose from), Maven (for Java), NuGet (for .NET) or PyPI (for Python).

    You can't objectively compare packages in NuGet with packages in npm. A typical NuGet dependency graph may go a few levels deep but it'll usually terminate there. A typical npm dependency graph can go between 10,000s and 100,000s of layers deep and pull in gigs of code. Its like comparing an itchy nose to being shot in the face with an elephant gun loaded for rhino. With proper frameworks like .NET or Java you can achieve an awful lot of work very quickly just using the vendors own packages, whereas with JavaScript you can't.

    Secure computing is anathema to JavaScript, so if its on your list of requirements, then you simply can't use node - it would require a total redesign of npm and the JavaScript language specification before you could ever hope to make it secure as its all insecure by design.

    How many times now has a node package gone stale because the maintainer has gone to jail? Died? Got bored? The damage is impossible to calculate and your ability to update a package for a known vuln depends on a maintainer often 100s or 1000s of packages deep doing some work first, and so on up the stack. JavaScript is a great language for teaching kids to code, but once you seek to be a professional you need to learn a professional language.

  15. EnviableOne Silver badge

    S2D2

    so basically

    OWASP 2010 = OWASP 2020

    coders can't program, but still convince themselves this scripting is proper programming, and libraries full of uselless code that sticks even more attack surface on your project are still a good idea. I'm all for not re-inventing th wheel, but use the wheel, not the whole cart

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020