How a chunk of the web disappeared this week: GlobalSign's global HTTPS snafu explained

GlobalSign has performed a postmortem examination on how, as one of the world's root certificate authorities, it managed to break a chunk of the web. The New Hampshire, US-based biz has to date sold 2.5 million SSL/TLS certificates to websites around the world. This week, it inadvertently smashed its own chain of trust: it …

[WibbleMe]

Shame there is no Open Source SSL you could use free of change. Woops there is "Let's Encrypt" project.

[Marco Fontani]

Shame they don't offer wildcards certificates, yet.

I look forward to that day!

[Tom Chiverton 1]

What's the point of a wildcard, when you can have as many as you like for free as often as your want ?

[Lee D]

Because signing up for thousands of subdomains is a pain in the arse, especially if they all renew at different times.

Fine while the software works. Major headache when it can't renew one for some reason or you need to move machines.

[Anonymous Coward]

Let's Encrypt is not an alternative.

For the matter, it can't issue some certificate types you may need.

[Crazy Operations Guy]

Re: Let's Encrypt is not an alternative.

Identity certificates, Extended or Organization Verification certs, authentication certificates, code-signing, time-stamping, etc...

Let's encrypt really only does web server certificates that can, with some effort, be used to secure SMTP, FTP, and some other protocols, but there are a lot that you can't.

[Tomato42]

Re: Let's Encrypt is not an alternative.

OV certificates are useless, they don't provide any additional functionality over Domain Validated certificates.

[Crazy Operations Guy]

Re: Let's Encrypt is not an alternative.

"OV certificates are useless"

Functionally, no, but they are useful for knowing that the CA actually did at least some basic checking before issuing the cert. There are too many CAs that will just issue a standard certificate to anyone with an email address and a couple bucks to spend for a basic certificate.

[Leeroy]

Re: Let's Encrypt is not an alternative.

Let's encrypt is fine for Web servers but as noted above it will not work with RDS deployments. Tried for a few hours with no luck and ended up buying a £5 one, shame I still have to pay the racketeers :/

[ElReg!comments!Pierre]

Web-o-trust, smmeb'ol'thrust

Ages ago I chose to rely on self-signed certificates. Everyone was OK with it but then "web of trust" happened. The gormless mass rushed towards "verified", paid-for, "you get what you pay for" certifs.

Soon enough, the so-called "web-of-trust" revealed itself as the extortion scheme it always was, and ever since we hear stories about its hapless victims. And yet, they all gobble it up hook, line and sinker. again, and again, and again, pouring endless streams of money into a despicable extortion scheme despite ample evidence that it exposes them -and their customers- to pwnage rather than protecting them from it.

One born every minute, as the saying goes.

[Brian Miller]

Re: Web-o-trust, smmeb'ol'thrust

Here's the thing about a self-signed certificate: how do I know that you issued it? If I don't have a method of independent verification, there's a valid chance that what I think is your web site really isn't your web site.

[Drew 11]

Re: Web-o-trust, smmeb'ol'thrust

"Here's the thing about a self-signed certificate: how do I know that you issued it?"

See: DNSSEC+DANE

Bypasses all this CA rubbish - which is why the browser authors don't want to bake it into their browsers.

How about a "PPS:" in the actual article about that, to raise awareness?

[A Non e-mouse]

Re: Web-o-trust, smmeb'ol'thrust

Here's the thing about a self-signed certificate: how do I know that you issued it? If I don't have a method of independent verification...

Because the CA's have such a good track record for diligently checking every certificate request and only issuing them to the correct people....

[Anonymous Coward]

Re: Web-o-trust, smmeb'ol'thrust

They could at least revoke dodgy ones - how do you revoke a self-signed certificate? Moreover anything protected by a self-signed certificate can be MITM at will, and the user would have little chances to understand unless they personally known the issuer so they can check.

Should the CA become more reliable and liable? Of course. But it would mean higher prices for certificates (proper vetting has costs), and it looks many here are not willingly to pay of safer certificates.

[Jack Douglas]

How a chunk of the web disappeared — and some important stuff too

SSL isn't only for the web of course — some RDS client machines ended up unable to connect to their server. Clearing the caches was no help (I don't know why) but reissuing the cert and/or using the new intermediate did the job, after a reboot.

The most annoying thing was GlobalSign continuing to tweet marketing drivel in between updates about the crisis on their Twitter feed.

[diodesign]

Re: How a chunk of the web disappeared — and some important stuff too

Yeah - it's not HTTPS websites. Any client-server setup using SSL/TLS certs from GlobalSign were at risk - Kaspersky software was affected too, for example.

C.

[Ken Hagan]

Still slightly confused

The headlines (on this article and elsewhere) still seem to be describing it as GlobalSign's fault and yet the small print (in this article and elsewhere) still seems to be explaining that the certification revocation only broke stuff because one or more browsers interpreted it wrongly.

So are we saying that GlobalSign issued a duff revocation, or that they failed to test how their valid revocation might be invalidly acted upon by various browsers, or that GlobalSign basically did everything they could reasonably be expected to and the fault likes with some browser that remains anonymous because GS don't wish to incur their wrath, or are we saying something else entirely?

[Brian Miller]

Re: Still slightly confused

It wasn't the browsers, it was the "third party security accredited load balanced OCSP responder system" that brought everything down.

So while the responder system was "secure," it just happened to have this one wee hole in it, owning to a problem with comparing dates.

[Ken Hagan]

Re: Still slightly confused

Ah! Thanks for that pointer. It makes much more sense now and I see that this was explained in the article and it was just my eyes glazing over at the crucial paragraph.

[Anonymous Coward]

Is there still trust in the Cert system?

How many articles has El Reg published in the past 2 years alone confirming rogue issuance, Malware cert hijacking etc, Lenovo Superfish etc etc....

[Anonymous Coward]

Re: Is there still trust in the Cert system?

I'd be more interested to know how big the problem is compared to the system being affected. Is it a minor nuisance, or is the system teetering on a knife edge, or what?

[Crazy Operations Guy]

Re: Is there still trust in the Cert system?

I'd say that the certificate system is a lot like plane crashes: Works perfectly 99.99999999% of the time, but when a failure happens, the damage is widespread and makes headlines around the world.

[David Roberts]

Still struggling with the concept of

Two certificates with the same public key and name attributes.

It does not seem unreasonable for the software to assume that these are versions of the same certificate.

[Lee D]

Re: Still struggling with the concept of

Was about to comment the same thing.

Should cross-certs have the same public-key as any of the certs they are signing? That seems daft, to me.

It seems a not-unreasonable assumption to me that if you request revocation of a cert with a certain key, and a certain subject name, differing only in date, to then render that cert - and any that it signed - as invalid from the point of issuance of that cert.

Otherwise, what's the point of the revocation and cross-signing mechanism? If, say, the R1 cert WAS compromised and new intermediates signed against your will, isn't that EXACTLY what you would want to happen? Using the cross-signed R2 to revoke it AND its intermediates?

Seems to me to be blaming the software in use for a particular operation when that operation is quite within the scope of reasonable measures, even if it was unintentional or other software DOESN'T do that.

How about you be more careful when revoking certs, and issue cross-certs that aren't using the same public key? If R1 signs a cross-cert that signs R2, there's no need for any of those to use the same public key. If anything it defeats the point of the cross-cert if they share a public key, because they then also share a private key, which means that if that shared key is compromised SOMEONE ELSE can sign any cert they like.

Maybe I'm misinterpreting their PR, but it sounds like they skimped on the implementation, didn't test, and then tried to blame software that had a not-unreasonable, maybe even highly-desirable assumption in it.

[Nate Amsden]

living in a bomb shelter

Didn't notice any issues on any of the maybe dozen sites I regularly visit. Company I work for uses go daddy (where tf are the godaddy girls), so no issues there either over the 150 or so certs we have. (Same goes for CDN which uses another provider I forget who)

[Tim99]

Surprise?

The guardian.com HTTPS site died on me - Whoops, I have admitted that I look at the Guardian - Funnily enough it is one of the better sites for Australian news in Australia.

The one that surprised me was Microsoft's Outlook mail hosts falling over with my mail client.

[Hans 1]

I bet the subcontractor company will not even have to pay a dime in repairs for damage done ... because the contract most likely has a big disclaimer ... never subcontract your core business to anybody, ever ! Makes sense ? Not to everybody ...

Congrats on taking down that large a chunk of the web, as well as other corporate services, was quite a feat!

[Anonymous Coward]

I don't get it

All the prOn sites and the register still worked. What's left of the web after that worth visiting

[Ken Hagan]

Re: I don't get it

Secret cat video sites that can /only/ be reached through https over IPv6?

[Anonymous Coward]

Outsourcing fail

You've got one job - issue certificates. So you outsourced chunks of that one job and that failed.

I think we can all point at that and say - stupid move.

[Scaffa]

Let's Encrypt is Free?

So really there's no excuse for el-reg to not have HTTPS.

[Lee D]

Re: Let's Encrypt is Free?

Maybe the ACME client is incompatible with their IPv6 setup.....

(/sarcasm)

[sixit]

Can we please stop using SNAFU incorrectly?

I'm sure the author meant "snag" not SNAFU. There is no such thing as "a" snafu; it is an acronym for Situation Normal: All F*cked Up!