back to article Dev's last-day-of-contract code helped to crash app used by 350,000 people

Welcome to Monday morning and therefore to a new instalment of Who, Me? It's The Register's weekly column that shares your tales of workplace errors and absolution. This week, meet a reader we'll Regomize as "Ray" who reached the end of a temporary contract working at what he described as "a major corporation." Ray was ready …

  1. PRR Silver badge

    > Ray didn't work there any more

    I should hope not.

    1. Korev Silver badge
      Coat

      Yeah, it doesn't Rayn, but it pours

      1. The Oncoming Scorn Silver badge
        Coat

        The fact Rayn caused pain.

        Was mainly able to be explained.

        1. KittenHuffer Silver badge

          I wonder if he lived on the plain ..... in Spain ..... and if he ever had any falls?

    2. StewartWhite Silver badge
      Coat

      It's a shame about Ray

      See https://www.youtube.com/watch?v=NdU0fQzt0h8

    3. MachDiamond Silver badge

      "> Ray didn't work there any more"

      As a contractor, he wasn't ever an employ to begin with. How reckless of a "major corporation" to have outside IT services with no internal supervision.

    4. herman Silver badge

      Ray doesn’t work here anymore

      Ray had an office on the second floor

      Sorry that he left no forwarding address that is known to me…

      OK, OK, I’ll get my coat!

      1. PB90210 Silver badge

        You bar steward!

        That's an earworm that will be stuck in my head for days. I spent ages stuck trying to remember the name in the song (Car-Ray!)

        Funny, that didn't happen when it appeared on the Kelly Enema Video Show (damned spell chucker) on one of those obscure freeview channels a while back

  2. wolfetone Silver badge
    Pint

    "Fortunately, the major corporation had good backups. And of course Ray didn't work there any more, so this incident was not his problem!"

    I'd like to think that Ray got a few quid for giving Major Corp. Inc. a good real world disaster test of their back ups, and any issues found on the back of that were duly fixed.

    Have a pint Ray.

  3. Korev Silver badge
    Coat

    I hope he had a good Rayte there

  4. David Harper 1

    Reading between the lines ...

    It looks like Ray committed a file containing production database login credentials into the company's source code repository. That is shoddy practice on so many levels. If Ray had worked in my organisation, he'd be getting a very severe bollocking from our cybersecurity team *and* our DBA team.

    1. Anonymous Coward
      Anonymous Coward

      Re: Reading between the lines ...

      I'm still a bit fuzzy on that next bit of the story - someone finds that conn str in the repository and decides to use it to access access a test database ?

      The clue should be in the server name in the string .

      And thats not a random user , thats a developer with access to the repo

      1. Doctor Syntax Silver badge

        Re: Reading between the lines ...

        The clue is probably in "default base configuration" and "anyone else at the major corporation who could use that code" and if that code was now part of the applications used by the 350,000 users not just "a developer with access to the repo". OTOH if that code also allowed a user to issue a "DROP TABLE" there were more bugs than the one Ray fixed.

    2. richardcox13

      Re: Reading between the lines ...

      A contractor who did that should be told they're not working for the major company again.

      For a junior employee, a formal disciplinary would be appropriate, for anyone at all senior there is no excuse.

      This might seem harsh, but remember this would, amongst everything else, be a GDPR violation.

      1. John Brown (no body) Silver badge

        Re: Reading between the lines ...

        "This might seem harsh, but remember this would, amongst everything else, be a GDPR violation."

        "Could" not "should". There's no mention in the article regarding what data was contained in the database nor what country it occurred in and so may not be covered by GDPR anyway.

        1. KarMann Silver badge
          Headmaster

          Re: Reading between the lines ...

          "This might seem harsh, but remember this would, amongst everything else, be a GDPR violation."
          "Could" not "should". There's no mention in the article regarding what data was contained in the database nor what country it occurred in and so may not be covered by GDPR anyway. [emphasis added]
          "Would" not "should".

          1. Robin

            Re: Reading between the lines ...

            "Would" not "should".

            Coulda woulda shoulda

    3. Jou (Mxyzptlk) Silver badge

      Re: Reading between the lines ...

      The problem is "who has access to this repository" - should only responsible admins, hm?

    4. Just Enough

      Re: Reading between the lines ...

      Totally this. He committed to the repo details that allowed the script to be run by anyone, connecting with developer level permissions to a production database.

      The person who ran it was probably either expecting the script to use the connection they already had, or prompt them for authentication. But instead it immediately ran against the database defined in the script. And it would also be storing a password in plain text, for all to see.

      This is terrible code handling by Ray, who is far too smug about a situation he caused.

      1. Korev Silver badge

        Re: Reading between the lines ...

        It's also possible that his code pulled the secret from the secret manager and in this case pulled a prod account with too many privileges.

    5. tiggity Silver badge

      Re: Reading between the lines ...

      .. and with proper practices used that troublesome commit would never have occurred as would first have needed code review* with multiple people (of appropriate roles) approving and then it would finally get committed.

      * which *should* catch that sort of thing, though confused why any creds in code base as normally appropriate creds for that pipeline are injected in the test instance and (separate creds / system for this) then the deployment instance & never in the code base.

    6. MachDiamond Silver badge

      Re: Reading between the lines ...

      "That is shoddy practice on so many levels."

      But likely fast and easy, and Ray only has an additional day of pay approved to get everything done.

      1. veti Silver badge

        Re: Reading between the lines ...

        Yeah, I find it hard to blame Ray. A SQL query that authenticates itself - what kind of half-assed server configuration even lets you write such a thing? But it sounds like this was common practice at Major CorpTM.

  5. Anonymous Coward
    Anonymous Coward

    It's always a good idea to check the backups actually work, isn't it?

    1. Anonymous Coward
      Anonymous Coward

      Yes, but not this way.

  6. DS999 Silver badge

    As a long time consultant/contractor

    I knew well that permanent employees will always blame us for stuff after we're gone. Usually to cover their own ass, but sometimes it is even our fault. I always figured that's part of the reason we're getting paid a lot more than they are, we give them a convenient target to point the finger at. Both because we're no longer there to defend ourselves, and because they don't like higher ups bringing us in to "touch their stuff". Goes with the job, sort of like how a good manager will take the bullets for his employees.

    1. KarMann Silver badge

      Re: As a long time consultant/contractor

      Oh, PLEASE.

  7. trevorde Silver badge

    One tiny, wafer thin change

    Worked at a software company where one of the devs sneaked in a 'minor change' to a service pack. Cue 220k users having to uninstall said service pack and an emergency service pack release.

  8. Jou (Mxyzptlk) Silver badge

    Not Rays mistakte!

    Article says: "his configuration file". If his configuration file is readable by too many it is not his fault for having wrong access restrictions to his configuration file. (Actually, we would need more details, but still a high probability of "Not Rays mistakte!".

    1. breakfast Silver badge

      Re: Not Rays mistakte!

      At the very least not only Ray's mistake - competent devops would have checking for keys and connectionstrings being committed into the repo.

    2. doublelayer Silver badge

      Re: Not Rays mistakte!

      At the point you commit your configuration file into the source repository, it's no longer your configuration file. Now it's anyone with code access's configuration file. If there's a credential in there, now it's available to a lot more people which can be a lot more of a problem. You can't easily have a repository with a file that the people able to access it can't read, and trying to hack that, at which you technically can succeed, is evidence that something is going wrong somewhere.

  9. Pete 2 Silver badge

    Rolling heads

    > And of course Ray didn't work there any more

    Thus making him the ideal scapegoat. Not just for that issue, but for any other that needed blame attached.

  10. FirstTangoInParis Silver badge
    FAIL

    Code review before commit?

    Anyone? Though not.

  11. Anonymous Coward
    Anonymous Coward

    Why did source control permit him to commit code with embedded credentials?

    Why did he embed credentials in his source code in the first place?

    Also, please tell us that Ray is no longer employed in IT.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon