back to article DevOps: Bloody hell, we've got to think about security too! Sigh. Who wants coffee?

Imagine you're an organisation that is looking to implement a DevOps approach to applications and services, or perhaps you’ve already started, but you’re worried about security. DevOps is all about rapid iteration and continuous delivery, but your security folks still want to be able to do checks to ensure systems are as …

  1. Jay 2

    From my experience (as a security-minded sys admin) security isn't anywhere on the radar/horizon of DevOps types. They think it's quite normal to drag in who-knows-what from the outside world to use every five mins. And then once it's in they'll have a long list of complaints that they can't do X, Y and Z because our (sane) security policies forbit it or that they've implemented something which doesn't conform to anything else we have.

    1. Bill M

      Good security is good and should always be implemented from day 0.

      But silly badly implemented security is bad.

      A while back I was setting up a production 2 node data engine cluster and somebody had decided to put a firewall between the nodes with no ports open. Obviously nothing worked. But they had direct non fire walled connections from both nodes to the Dev box so I had to route all the inter node traffic via the dev box until the firewall between the cluster nodes was removed.

      1. Andrew Commons

        @Bill M

        Presumably the firewall was relocated to where it was originally meant to be...blocking access from the Dev boxes to Production?

      2. Kabukiwookie Silver badge

        OMFG! You're exactly what's wrong with devops.

        You don't work around the problem, you fix it. If there's a firewall with no open ports, you create a change and ask for the ports that you need to be opened.

        What are the chances that you're workaround is still in place, even if the firewall gets opened up.

        I also presume you didn't report the insecure work-around to be fixed.

  2. Lysenko

    DevOps is all about rapid iteration and continuous delivery

    DevOps is all about cutting corners optimising processes and prioritising instability speed over all other considerations. Consequently, the iron triad of software development dictates that the result will be wrong, expensive or both.

    In the security field, your unit test robots are up against active attack by teams of highly skilled and motivated professionals commonly known as "hackers". The current balance of power between automation and Human Brain Mk 1 dictates that the robots frequently lose. Increase your attack surface feature release cadence and the equation always favours the attacker.

    Face facts. DevSecOps is an oxymoron. If you are determined to ship "Must have colours! Hot new look!!" then your quality will be crap or your costs will be astronomical. Having said that, TopShop are obviously more successful than Saville Row tailors. That's perfectly fine and fine for DevOps too - so long you face up to the fact that you're peddling cheap, low quality, high street tat rather than something hard wearing, well designed and reliable.

    1. Pascal Monett Silver badge

      Re: you're peddling cheap, low quality, high street tat

      And that is the direction programming has taken since Day 1. As the old saying goes, there's never enough money to do it right, but there's always money to do it over.

      Once upon a time programmers were engineers. With a degree. From a proper university. Nowadays, programmers are anyone with a keyboard. That might be good from a diversity point of view, but the downside is that those who can properly analyze a project and write good code are drowned in the masses of rent-a-suit shops and shipped-in-from-overseas keyboard mashers who may or may not have the chops but whose main quality is being cheap.

      Let me put this another way : my wife is fanatical about shoes. She has upwards of sixty pairs and, every time we stroll the streets of a new town or city we've never been to before, she can't help but be magnetically attracted to any store front that has pairs on display. After 15 years of marriage (that was already a good while back), she surprised me one day when, out of the blue, she declared that she was fed up with buying cheap shoes. She stated, and I quote : "I'd rather have one or two good pairs a year than buy a pair every month that won't last more than 8 months". I lit a candle that day.

      There is a market for cheap shoes, throwaway items that won't last, and that's fine. There is also a market for quality items that people need, items that will endure and give pride and pleasure to their owners for a long time.

      DevOps is the cheap throwaway market. Everything is described to make everyone believe that whatever issues exist will be solved by the next iteration, so they are not important.

      Sorry, but programming is not cheap. Programming is the very lifeblood of companies today, and there are some unavoidable medical practices and costs when it comes to dealing with lifeblood. The slew of hacking issues of last year demonstrate clearly that security is not something you just pay lip service to.

      I would like the industry to take a step back and realize that nothing that has ever been made in a rush has ever lasted or performed as expected.

      I would also like to win the lottery.

      I know which has a better chance of happening.

      1. David Roberts

        Re: you're peddling cheap, low quality, high street tat

        "Once upon a time programmers were engineers. With a degree. From a proper university."

        In your fixed view of the Universe, perhaps.

        Back in the day, most large organisations (and a lot of smaller ones) trained their programmers from scratch.

        When I came out of University in the '70s with a degree but no direct employment prospects in that discipline I searched around for any job available.

        One was for a computer salesman. I obviously wasn't a natural (or even unnatural) salesman but part of the interview including a programming ability test. I scored highly and was passed on to a software house who decided not to employ me because the boss was worried that I would take the training then head back to academia (which does show a distinct lack of commercial awareness).

        Anyway, that set me off looking for programming jobs and I was recruited by a major public body building a programming team for a new project.

        The recruitment favoured graduates or at least people who had spent at least a year at University. All sorts of disciplines including astronomy IIRC. Everyone was trained from scratch in COBOL and were generally productive within 6 months or less.

        So no "real engineers from real universities" involved at all.

        You may find that many of the commercial programmers out there today have no formal engineering qualifications, possibly not graduates from a "real" (or even Unseen) university. So what? That is what "introduction to programming" courses are for.

        Personally I found programming deadly boring after the first six months to a year and moved on to the more technical aspects of computing. It takes a special kind of mind to work long term at programming, especially maintenance programming. Not me.

        1. Naselus

          Re: you're peddling cheap, low quality, high street tat

          Either way, too many developers and programmers (including many with good engineering degrees from good universities) view security as something that's in their way, rather than something that should be integral, and so look to remove or circumvent security features rather than correctly writing to incorporate them.

          Meanwhile, the 'Ops' part of DevOps has tended to be marginalized (because Ops staff are 'unproductive' while still being about as expensive as code monkeys), so despite the relatively strong security culture that ops teams have built up in the last twenty years or so, it's just not translated into the developer-dominated DevOps scene. The result is a lot of decidedly shabbily-run devops infrastructures, even when the code is decent.

  3. Justicesays

    Good luck securing your app...

    Lets take a couple of examples from the article:

    Oracle JRE, 564 reported vulnerabilities over 8 years, ~6 per month

    PHP, 558 reported vulnerabilities over 17 years, ~ 3 per month.

    And that's just the main framework , never mind any libraries or other components you might be sticking together.

    By the time your release candidate app gets to the tests, it's probably already got at least one (known) security flaw, even if you built it for release weekly. (And obviously loads of undiscovered ones).

    This is why 99.97 % of apps are vulnerable when scanned, and are probably shipped with known security flaws.

  4. teebie

    [Making security integral is] "is about having the right tools, having the right processes, and about cultural transformation. "

    Is it maybe in some ways about having developers who know what the fuck they are doing?

    Automating regression testing is sensible, but anyone who suggests all testing should be automated should be, well, not fired because then they can do harm elsewhere. Maybe they can work in the canteen

    1. EarthDog

      Cultural transformation will never happen

      As long as management is in a high speed race for more profit. Security blocks time to market is cannot be sold. There is no "our application costs 10% more because it is 50% more secure" in the mix.

      1. Anonymous Coward
        Anonymous Coward

        Re: Cultural transformation will never happen

        Think our lot would be satisfied with "lose less", with profit, much like England winning the World Cup, a distant fantasy.

  5. Steve Aubrey


    "99.7 per cent of web applications tested by the firm had one or more vulnerabilities."

    And I bet the other three tenths of a percent had functional problems.

    There! One hundred percent - we nailed it!!

  6. Robert D Bank

    it's all about standards..

    Security is so much easier when you have really good well thought out standards that are properly enforced. If you start from a position of excellent standards then you should be able to have much more generic security that is applied by default rather than having a bespoke approach every time.

    Any function that is commonly used and that presents any security risk should be very closely scrutinised and managed by only very qualified and trusted people, and as much as possible isolated from bespoke code that calls the service. Sharp focus needs to be on identifying high risk areas and investing in that specifically so other users of that code know they're protected from creating risks and can concentrate on whatever they're creating.

  7. EarthDog

    Behave as adults

    I recently worked on a government project which required slightly higher security than normal. Security painted several stakes in the ground such as all libraries must be scanned and vetted, all tools must be scanned and vetted, patch levels must stay in synch etc. In other words, be having as adults.

    The resistance on the team was unbelievable. They were all refugees from the private sector ( *cough* CA *cough* HPE *cough* IBM *cough* Oracle *cough*) who were working fast and loose with OSS to meet marketing deadlines.

    I was working QA at the time and was backing up the security team. We was working to help ensure they played nice. Telling them things like they could not use the latest shiny shiny without proper vetting did not go over well. They kept screeching "waterfall! waterfall!" when it really wasn't. It was just another requirement to meet.

    Your basic private sector developer has no clue what a modicum of security means and DevOps "Engineers" had no real ideas in the beginning on how to implement it. It took quite a while to get it right. But there was no compromise on meeting the minimum standard.

    But basically it came down to vet the libraries, scan them, patch ASAP, vet and scan your own code, and good QA which included tests for various vulnerabilities. Basically, behave as responsible adults.

    One thing we did do which security was quite happy about was in code reviews in pull requests. It even allowed us to meet some requirements immediately. They accepted as good practice without much fuss.

    So just be an adult!

  8. Anonymous Coward
    Anonymous Coward

    Latest Fashion

    I was in at the start of an ambitious DevOps implementation at a major global financial services corp. All the stated aims sounded great, but at the coalface a lot of re-branding was going on.

    Sure, the IT dev and ops pillars were re-org'd so they integrated to form complete funcional teams, and the race was on to earn 'DevOps' badging ASAP. There were some definite benefits of this integration such as recognising operational (inc. security) issues much earlier in the dev cycle, rather than the traditional "here is my code that I know works on my under the desk server/single VM, please implement a production strength versiopn of it, and do it quickly as the biz are expecting this to be live in 2 days" approach.....but did we actually achieve the stated aims ?

    As per usual, a lot of hype and noise around DevOps, but pervaded by a distinct feeling of "plus ça change, plus c'est la même chose". i.e. another latest fashion, that pays lip service to quicker, cheaper and more reliable delivery. Actually, most of the DevOps concepts are just common sense, that seasoned IT professionals have been doing for decades, just without a title for the methodology. As per ususal in IT, very little process is really new.

    The DevOps programme provided some excellent sales opportunities for the many software firms, desperate to be on that bandwagon, and emboldened IT delivery teams to insist that following a DevOps methodology alone made their code bullet proof.

    No doubt DevOps will be replaced by some other new IT fashion in time, and not nessecarily an evolutionary one.....but hey, all this change keeps us gainfully employed. However, the very old maxim of Garbage In Garbage Out still very much applies :-).

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like