GitHub publishes RSA SSH host keys by mistake, issues update

GitHub has updated its SSH keys after accidentally publishing the private part to the world. Whoops. A post on GitHub's security blog reveals that the biz has changed its RSA SSH host keys. This is going to cause connection errors, and some frightening warning messages, for a lot of developers, but it's all right: it's not …

[Eclectic Man]

Sex, Drugs, Money and ...

Prof Fred Piper of Royal Holloway College, University of London claimed that the three most popular ways of defeating encryption were 'sex, drugs and money'. I suppose we ought to add 'stupidity' to that list.

"Whoops!" indeed.

It is Friday, have a pint to drown your sorrows.

[John H Woods]

Re: Sex, Drugs, Money and ...

I would have thought "Violence" probably comes first on that list (Including the state telling you "Hand over your keys or else ...")

[John Riddoch]

Re: Sex, Drugs, Money and ...

Because there's an XKCD for everything: https://xkcd.com/538/

[Steve Button]

Re: Sex, Drugs, Money and ...

Oh yes, when a nerd's fantasy meets cold hard reality.

I've been a nerd long enough to have bumped into cold hard reality too many times, and have the scars to prove it.

A bit like the great philosopher Mike Tyson who said "Everyone has a plan, until they get punched in the mouth".

[steviesteveo]

Re: Sex, Drugs, Money and ...

I never got the sense that those enthusiasts online even needed interrogated. If you've already told me your security strategy when I was just passing by on the internet then that secret was not going to get kept very long in an investigation

[Steve Button]

If you're unclear how SSH encryption works, about public versus private keys

It's pretty simple really., the "private" part is the part you are supposed to keep private. Like really private.

That image (which I assume is only on the RSS feed?) sums it up really nicely. Someone is very much tearing their hair out.

This is going to cause a small amount of disruption for millions of people, and probably a large amount of disruption for an unfortunate few who have inherited a system that they only half understand.

Oopsie.

[Wzrd1]

Re: If you're unclear how SSH encryption works, about public versus private keys

I've saw no end of confusion as when one is trying to get keys issued to a server and an IIS admin gives the wrong name for the asset, such as the server name, rather than its alias (bob.mydoman.net vs www.mydomain.net).

Had me tearing hair out. Of course, this nerd bled plenty over the years for real, so obviously the hair was not my own.

I was a dick and made them all attend a class on encryption.

[John H Woods]

Encrypted?

Were the private keys published encrypted or not? Or does it not make much difference?

[Steve Button]

Re: Encrypted?

Yes it makes a huge difference. If the key was encrypted they could have sorted this out with a bit more leisure, or perhaps not even worried about much depending on how good the encryption is.

[John H Woods]

Re: Encrypted?

I thought so --- that's what puzzles me. I mean I can see fudging a .gitignore and accidentally sending up the private key files in .ssh, but I wouldn't have my private keys hanging around unencrypted. I just wondered whether it might be the case that a (perhaps shortish) passphrase on these files wouldn't be considered enough protection, given that an attacker can run as many decryption attempts as they like.

[Orv]

Re: Encrypted?

These were the host keys, which are usually not encrypted. They're used to identify the host, not to authenticate to it. If they were encrypted someone would have to enter the password every time the system was rebooted.

[ChoHag]

Re: Encrypted?

Someone or something. There's nothing stopping you from having encrypted host keys with a hardware module containing their decryption keys. Not practically any safer against an active attacker but it's another layer to shield against *ahem* human error, which would be useful for multi-national identity harvesting corporations full of human failures.

It's actually a valid option for an organisation which has put itself into such a critical infrastructural position and not the massive overkill it would be at most other run of the mill companies.

[Zippy´s Sausage Factory]

Wait, GitHub did what?

Oh yes, Microsoft subsidiary. Business as usual, then.

[chololennon]

I experience that...

I experienced that 12 hours ago when I was pushing several commits (the first group Ok, the second one, a few minutes later, with the scary message)... and of course I panic because it wasn't my rsa key, but the one from GitHub. So I checked my keys, I checked my GitHub account... I ended up updating my keys and the ones from GitHub. My worry lasted until I read this article, thanks Liam.

[Mike 16]

Sufficiently advanced stupidity

is indistinguishable from malice

[Wzrd1]

Re: Sufficiently advanced stupidity

I'll be stealing that remark.

[Anonymous Coward]

"Glitch diverts net traffic through Chinese ISP"

The Register, Sat 10 Apr 2010

[IGotOut]

Re: "Glitch diverts net traffic through Chinese ISP"

And that 13 year old info is related how?

[Wzrd1]

Re: "Glitch diverts net traffic through Chinese ISP"

The obvious need to give new SA's a good dose of laxatives.

The old SA's, a diet for their fingers, as we age, our fingers grow ever so fat...

[ChoHag]

> This time, the reason was – as usual – plain old human error. Someone published GitHub's private RSA keys in a repository on GitHub itself.

Not human error. This is a systemic error which should not be possible in system which considers security at all. Nobody that incompetent should have access to those keys.

Not publishing private keys is security 101. Fuck, it's security 1.

This is a major fuck up revealing shockingly* bad internal operations and it should be a cause of concern for anyone relying on github (or, surprise!, anything else made by its owner). It's not a minor whoopsie.

[*] OK so it's not a shock, but still...

[Eclectic Man]

Possible or not possible

Early on in my IT security career, I analysed the command set for a Host Security Module (HSM)* used by banking systems to encrypt, decrypt an do various things with DES and the related keys. It transpired that analysing the instructions set with Prolog** showed that it was possible to get the thing to divulge keys in plain. Of course this was not supposed to be possible, never mind allowed.

*No names, no pack drill, sorry, client confidentiality and all that, plus I don't want to go to prison.

**Prolog, a programming language designed to automate a restricted version of 'Horn clause logic', which was quite useful to analyse instruction sets and others things.

[Wzrd1]

Nobody should be publishing keys alone. That is why reviews are supposed to be SOP, to prevent an acute burst of intracranial flatulence from becoming a security incident.

Security 101: trust no one, not even oneself.

[Mike 16]

Reviews?

When would they have time to spend on stuff that would enhance security, after using up the time allotment in fights over the "coding standards" that care a lot about the niceties of camelCase vs PascalCase, snake_case, etc.

Not to mention tabs vs spaces, commenting layout, etc.

[cookieMonster]

Where’s the I just pissed myself

laughing icon ???

[petef]

The GitHub blog instructions said that there might be one RSA line in .ssh/known_hosts to delete. I actually found half a dozen because the name github.com resolves to several IP addresses.

[Pomgolian]

Blinked, nearly missed it

This happened to me, did a quick Google to see what the story was. No joy there so I tried again a minute later and it all worked.

All this in the same week I finally gave them my card details. Not massively inspiring. But then, Micros~1.

[Wzrd1]

Re: Blinked, nearly missed it

Micros~1 security, direct from 127.0.0.1.

[DM2012]

From the company that..

.. actually provides secrets push protection as a feature on all repos.

[Anonymous Coward]

Easy fix

I'm thinking that some ex-employers just fixed this by disabling host key validation everywhere. I put ALL-CAPS warnings in the config file to not turn it off followed by instructions for updating the key, but they were mad that I turned it on in the first place. "Too much maintenance," they said.

[Wzrd1]

Re: Easy fix

Too much maintenance? Imagine the maintenance required once you're in traction... ;)

[karlkarl]

Clowns.

They disabled SSH password authentication to stop amateurs from undermining their own security. And then MS GitHub goes and does something even more amateur.

[Will Godfrey]

0 out of 10

or maybe even -1.

Not only did they screw up catastrophically , but the warning message implied us punters were under attack - without any mention that they'd changed their keys, so I immediately shut down everything including the router to take stock. After a couple of hours of checking against backups etc. I went on line again, at which point I got an email from a co-conspirator who had also been bitten by this the previous day and told me exactly what happened.

We keep exact duplicates on sourceforge as well. Rather than letting either repository auto sync we generate the commits from the same source in house. If the worst comes to the worst and github gets trashed, our users can still get clean copies.

[PyLETS]

Humans not careful enough for crypto

It's why resilient systems need more than that one layer of defence, so that while breakage of one layer is repaired the

other security defence in depth holds it together.

Cryptocurrency just has that one layer of defence. If you lose the keys or have them stolen, then you've lost the money.

[Anonymous Coward]

Open for Man in the middle now?

Whoever got the private keys, can now create a man in the middle attack to unsuspecting github users, right?

I mean, changing the public key only makes sense if people KNOW that it has changed. A malicious user can create a proxy using the old keys.....