back to article We're not saying this is how SolarWinds was backdoored, but its FTP password 'leaked on GitHub in plaintext'

SolarWinds, the maker of the Orion network management software that was subverted to distribute backdoored updates that led to the compromise of multiple US government bodies, was apparently told last year that credentials for its software update server had been exposed in a public GitHub repo. Vinoth Kumar, a security …

  1. Doctor Syntax Silver badge

    At leas it wasn't Pasw0rd.

    1. Eclectic Man Silver badge

      Or 'AtLeast10Chars'.

      1. bombastic bob Silver badge
        Black Helicopters

        regardless, having ANY kind of singular or hard-coded password is *(ahem)* ALWAYS! A! BAD! IDEA! as illustrated with THIS example.

        Effectively the same if it had *(ahem)* A! MANDATED! GOVERNMENT! BACK! DOOR!!!

        (you know, like the kinds of encryption back doors our politicians often want)

    2. el kabong

      Pasw0rd?? Who would use that, that's foolish!

      Pa55w0rd is much safer, I use it everywhere. Only a fool would use Pasw0rd.

      1. bombastic bob Silver badge
        Trollface

        Re: Pasw0rd?? Who would use that, that's foolish!

        yeah, using l33t5p34k to encode your favorite word makes it SO much more secure... </snark>

        (although I admit doing that when certain password verifiers won't let me do something more secure and easier to remember like an equivalent of 'correcthorsebatterystaple' and then confuse you when you have to use 3 of 4 different things, and if you use all 4, it rejects it...)

  2. Anonymous Coward
    Anonymous Coward

    Hmmmm

    So using that weak password metric, Donald Trump’s password must be “iwon2020”

    1. Yet Another Anonymous coward Silver badge

      Re: Hmmmm

      That's just silly, it would be "Iwon2020'

      1. KittenHuffer Silver badge

        Re: Hmmmm

        So he included all of the evidence he's obtain about 'Massive Voter Fraud!' in his password. That's really neat.

      2. Anonymous Coward
        Anonymous Coward

        Re: Hmmmm

        >That's just silly, it would be "Iwon2020'

        Olredy nyet tovarisch, ve changed it back to "Ivan2020"

      3. jgarbo
        Facepalm

        Re: Hmmmm

        No. Trump's a notorious illiterate.

    2. Hubert Cumberdale Silver badge
    3. Anonymous Coward
      Anonymous Coward

      Re: Hmmmm

      I think the first person to go off topic mentioning an unrelated political figure in any comment thread should have their password changed to 'TwAt1' then swiftly pummelled repeatedly with a the ban hammer.

      Boring and annoying now.

      1. EVP

        Re: Hmmmm

        That password is already taken. Don’t tell anyone.

  3. Sparkus

    if true...

    criminal malfeasance on the part of SolarWinds......

    1. the spectacularly refined chap

      Re: if true...

      Under what law?

      Sure, it doesn't look good, but you are asserting a criminal offence has been committed. Not your opinion, something in statute law.

    2. Snake Silver badge

      Re: if true...

      It's more absolutely, gross-level stupidity.

      Why was a server of outbound software granting write access to FTP clients? Why??!!

      Basic security protocols would tell you that public access = read only. If World+dog needs write access then you do so in a separate directory, from which submissions can be vetted prior to integration to distribution. You don't go making a server read/write on the same resource.

      ...

      I seriously wonder about the "skill", "knowledge" and "intelligence" of IT people who thought of, and especially OK'ed, this setup. Really. I *thought* you went through higher education...but apparently I was very, very wrong.

      1. Jonathan Richards 1
        Thumb Up

        Re: if true...

        Violent agreement. Apparently nobody looks at these complex systems in the round. The fact that this is a penetration testing organization makes me think that they don't eat their own dog-food, as they say.

        I'd go further than a separate directory. In the situation where a design review has decided that there has to be weak authentication on uploads, the inbound server should be a physically separate machine on a physically separate network.

  4. cantankerous swineherd

    I didn't think it could get any worse than the OPM hack - https://www.bbc.co.uk/news/world-us-canada-33120405

    1. Pascal Monett Silver badge

      That article does not make clear how the data was accessed. Of course, obtaining personal, intimate data on up to 14 million government workers is very much a bad thing, but there is nothing that says that an FTP password was at fault.

      Solarwinds can explain all it wants, the fact that it has rubbish password security is now established and that is a stain that is not going to go away quickly for a company that is supposed to deal in Internet security and network monitoring.

  5. sanmigueelbeer

    Cue the violins performed by hungry lawyers.

    I smell blood in the water.

    1. Someone Else Silver badge

      Hmmmm. "Hungry lawyer"...

      Now there's a concept that doesn't exist in the real world. "Greedy fat-ass ambulance-chasing lawyer", now that's a thing. But my experience is that they all tend to eat well.

      1. jtaylor

        But my experience is that they all tend to eat well.

        Know any public defenders? Or someone who does pro bono work for charities? Or, heck, anyone who recently graduated from law school and is working an entry-level position at a partnership while paying off their student loans?

        1. Someone Else Silver badge

          Yes to both. And they also eat well (they just don't have the Porsche in garage).

  6. upsidedowncreature

    It's OK...

    They've secured it now, they've changed it to "SolarWinds123!"

    AFAICT this is the whole problem that code signing was designed to solve, I would love to know the precise details of how the attackers managed to sign the code.

    1. FILE_ID.DIZ

      Re: It's OK...

      You break into the build server and let it do all the hard work. Like what happened to CCleaner a few years ago.

      1. Michael Wojcik Silver badge

        Re: It's OK...

        Exactly. Typical software product supply chains have a lot of potential points of vulnerability. Developer machines, source-code repositories, CI, staging, build machines, artifact repositories, signing servers, private-key backups... Good separation of concerns can plug some of the holes (don't keep production signing keys on developer machines!), but the links between them can introduce new ones.

        Even if you follow the sorts of practices that the CA/BF's Code Signing Research Group was trying to push a few years back - requiring a FIPS 1040-2 Level 2 HSM to do the actual signing, for example - and follow reasonable practices like requiring mutual authentication and data integrity between production build machines and signing servers, it's really hard to lock attackers out of that process flow once they get access to the internal network. That's one reason why zero-trust corporate systems are a hot topic these days.

        Against that, though, you have to ensure developers can do their jobs and automated integration-build-test systems can do theirs. It's not an easy problem.

        1. teknopaul

          Re: It's OK...

          all you have to do is download and verify your own software to spot a hack like this. Stopping might be hard(ish) but detecting it is not.

          1. jtaylor

            Re: It's OK...

            all you have to do is download and verify your own software to spot a hack like this. Stopping might be hard(ish) but detecting it is not.

            Detect what? If you look for anomalous behavior, you can detect active malware. You won't detect a backdoor that's waiting passively to receive a trigger. You won't detect malware that deactivates if it detects it's running in a VM.

            Of course, you could just compare cryptographic hashes against those from your parallel build environment that pulls known-good source code and builds it in a trusted environment. But then, if you have that known good trusted toolchain, why didn't you use that to build the release in the first place?

        2. Jonathan Richards 1

          Re: It's OK...

          > you have to ensure developers can do their jobs and automated integration-build-test systems can do theirs. It's not an easy problem.

          Once again, Einstein's maxim has it. A solution should be as simple as possible, but no simpler. [1] If the enablement of development, integration, etc. to make a sleek supply chain results in a weakened chain, then the simplification has gone too far, and the IS architecture simply has to sacrifice some efficiency for increased effectiveness.

          [1] Apparently, Einstein never used these exact words, but wrote something very like it when talking about principles of theoretical physics.

      2. Anonymous Coward
        Anonymous Coward

        Re: It's OK...

        But...my assumption is that this isn't the build server and is part of the hack (i.e. the ability to write anything you want to a "trusted" software update server likely evades firewall rules and simple content inspection) but doesn't allow you to produce a signed executable with your embedded payload. That would take additional access.

        It's an update server with a questionable configuration (i.e. allowing a general purpose user to upload content into software dstribution folders) and it reinforces many of my concerns about Internet accessible FTP servers (they are largely abandonware from an operational perspective and any configuration errors are either ignored or not well understood).

        It's not clear the Github repository is a Solarwinds employee or from someone who had compromised Solarwinds and shared example code for other reasons.

    2. Eclectic Man Silver badge

      Re: It's OK...

      Obviously not a Ray Brown trio fan, or it would be 'SoularEnergy'.

      (Check out Live at the Loa, Summer Wind, track 2, 'The Real Blues'. Superb.)

  7. Yet Another Anonymous coward Silver badge

    That's the sort of password

    An idiot would have on their luggage

    1. Phil O'Sophical Silver badge

      Re: That's the sort of password

      I'd guess it was one of those situations where one developer puts a simple password into some internal test code, assuming that no-one would be daft enough to want to share it, and then some other developer decides to share the test code, assuming that no-one would be daft enough to hard-code a password in it.

      1. Anonymous Coward
        Anonymous Coward

        Re: That's the sort of password

        It's likely to have been an account that allowed existing Soalrwinds deployments to check for updates.

        Not great, but a common solution in the mid-00's to keep the world+dog out of your FTP servers but not any real security. Post-2010 it should have been replaced by a more robust system.

        Unless you're suggesting Solarwinds was having developers build and upload code directly to the FTP servers with code repositories/build processes/code signing left to individual developers to do as they want? I'm not saying its not possible BUT....

    2. This post has been deleted by its author

  8. Blackjack Silver badge

    So.. when they are gonna do something about all these G*ts?

    Something something, post illegal content and the police will be at your door while you are illegally printing a car, something.

  9. sanmigueelbeer

    GE puts default password in radiology devices

    GE puts default password in radiology devices

    The update and maintenance software authenticates connections by using credentials that are publicly exposed (can be found online) and does so periodically with GE’s online maintenance servers.

    The credentials can only be updated by the GE Healthcare Support team. If not updated through a customer request - credentials are left default.

    Makes me laugh sometimes how large corporations simply do not take this seriously.

    1. DevOpsTimothyC

      Re: GE puts default password in radiology devices

      Well this one has hit a number of gov agencies so hopefully it makes a country introduce a law that makes the CEO personally accountable (with jail time for computer hacking type offences).

      I imaging that many of these problems would go away very quickly with many CEO's taking a paranoid level of care around security.

      1. EnviableOne

        Re: GE puts default password in radiology devices

        in the same way as Health and saftey didnt improve much until CEOs were made criminally responsible

  10. chivo243 Silver badge
    Coat

    I had to assist a user who would be taking a computer home with the vpn connection. I used my phone for a temporary hotspot to test. I thought i made an easy password 12345678 I kept trying to connect the computer to my hotspot and it failed. I checked the hotspot password again, I fat fingered 12345679 SolarWinds maybe the password should have been SolarWinds124

  11. Muppet Boss

    I am sort of convinced they did not know that the password was insecure, like in "we did not know that FTP server was writeable at all because that guy who was administering the server and made redundant during a cost-saving exercise failed to tell us the password".

  12. simonb_london

    "The Washington Post reports that unnamed sources believe..."

    Hold the press! We've found some guy who "believes" it was the Russians!

  13. OGShakes
    Paris Hilton

    2020

    I wonder if that was the password they used for the FTP 2020 was on before we all downloaded it?

  14. ParasiteParty

    What a bunch of dicks....

    Absolutely amazed.

  15. Anonymous Coward
    Anonymous Coward

    But the Certificates!

    So they had a weak FTP password. What about the fact that their certificates had been compromised. That should be the big headline concern. How did that happen and what's been done to ensure it. doesn't happen again.

    1. sabroni Silver badge

      Re: But the Certificates!

      Doesn't this suggest that their certificates weren't compromised? The bad actors FTP'd the malware in and Solarwinds signed it and distributed it.

      1. mevets

        Re: But the Certificates!

        uhhh bad state sponsored actors, I think you meant. Who else has access to github.

  16. c1ue

    Yet another example of the utter bollocks of "sophisticated, patient nation-state spies" - as opposed to the reality of semi- and in-competent IT setups.

    What is abundantly clear is not that the "bad guys" are skilled, it is that their targets are not.

    This is pure "security by obscurity" gone bad...

    1. Anonymous Coward
      Anonymous Coward

      @c1ue - I wouldn't use the word security

      in this kind of situations. Idiocy seems much more appropriate to me.

    2. jtaylor

      What is abundantly clear is not that the "bad guys" are skilled, it is that their targets are not.

      They may be skilled at finding weak points to attack.

  17. DS999 Silver badge
    FAIL

    The US government should ban use of SolarWinds products

    That one example shows they are completely incompetent to claim they understand anything about security.

  18. tcmonkey
    Stop

    NaTiOn sTaTe AtAcKeRs!1!1!!one!!!

    I’m not sure where companies are getting the idea that their products are so secure that only a government assisted outfit can break into them, but as this post shows it’s clearly absolute horseshit in some/most cases. How about the next time they think about drumming up fear and loathing of an entire country they actually provide some evidence of that claim?

    ‘Allegedly’ is not worth the paper it’s written on.

  19. Anonymous Coward
    Anonymous Coward

    "...the trojanized updates were digitally signed with a SolarWinds certificate"

    Un <expletive deleted> believable.

    :(

  20. Anonymous Coward
    Anonymous Coward

    At least there was no evidence Solarwinds were compromised

    Unlike Huawei.

    Or was it the other way round...

  21. Anonymous Coward
    Anonymous Coward

    At 13 characters length, with a mix of alphabetic and numeric characters, the password was actually a fairly strong one. According to the "How Big Is Your Haysstack?" calculator at the Gibson Research site, it could take decades to centuries to guess a password of this length. Posting it as plaintext in a public place was, well — stupid.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like