Simple numbers as keys
That's deliberately insecure. So who ordered that? The CA is toast.
Tragically simple private keys used by a certification authority based in India have been compromised, meaning the 60-million-plus certs it has issued can’t be trusted and necessitating a massive clean-up job. The CA of concern in this case is called eMudhra, which boasts of having issued over 60 million digital certificates to …
Or these days (the keys are from 2025 I believe), ChapGPT/Claude/AcmeSlopGenerator.
It's a brilliant example of "it works" being only 1 criteria (the keypairs are valid and do work) and how being able to do something with absolutely no understanding (crypto is hard folks) is dangerous ignorance.
I'd almost guess that someone inadvertently used a 8 bit random number instead of an 8 byte random number (though according to Scott Helme, its a 384bit keyspace not 256).
and somehow I've lost my ability to do basic maths (I need more coffee). 8 bytes is obviously 64bits. Trying to comprehend how the hell this happened has frazzled my brain apparently.
Following Hanlon's razor, I was wondering if some did something like (in pseudo code) private_key = SecPRNG.generate(8) believing generate() parameter is in bytes, when actually it's in bits, but this doesn't fit.
It's a brilliant example of "it works" being only 1 criteria
The frustrating thing is they didn't work. Almost immediately there was a discrepancy, but instead of stopping to understand why everyone blundered on:
| In light of the issues with the same intermediates in Bug 1970728 [2025-06-05], please describe to the best of your current knowledge how the wrong private keys came to be used to sign those CRLs and the steps eMudhra took in order to produce the correct signatures.The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys.
( https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11 )
I meant "worked" in the context that 4 is technically a valid private key value, and given that key successfully signed certificates (presumably thousands of them), the appropriate matching public key was correctly generated.
I've since discovered this:
According to eMudhra, a flaw in the key generation process caused a wrap/unwrap step to produce a second set of weak keys alongside the original strong keys. The weak keys were subsequently published to browser root stores. The company said it is working to establish the root cause of the issue.
Source: https://www.msn.com/en-in/news/other/emudhra-share-price-hits-20-lower-circuit-after-researcher-flags-weak-keys-can-the-stock-recover-what-lies-ahead/ar-AA2fJ9Fa
That explanation sounds difficult to believe...
"We generated strong keys, we wrapped the strong keys using our proven, certified wrap/unwrap function, but in doing so actually substituted weak keys, we then distributed the weak keys, deleting the strong keys from our system and replacing them with the weak keys"?
Probably more like:
"We asked AI to streamline the key generation process... and never checked the code it produced, or the keys it subsequently generated. But you have to admit, those single byte private keys really speed up certificate generation!"
You have confused a couple key types. These are ECC keys, which require only one number and do not require that it be prime. It must be in the field of the curve in use, but primeness is not required. RSA keys are where prime numbers are required*, and 4 could technically work there as well because you need two of them, so since 2 is prime, 2*2 makes a RSA key that does work.
* Technically, RSA keys work with any two numbers, prime or not. It's just that, if they're not prime, then it's broken. Or if they're prime but too small. Or if they're the same and therefore the square root of the public. Also if they're too close together because that makes it too easy to identify and factor. So for all of those reasons, 4 is not acceptable in RSA either, but it can do the calculations.
This has been going around in my head.. we know 4, and the others, are bad keys for all those reasons you mentioned. It's not just that they're low entropy or "technically as random as any other number but a bit lol". They fail fundamental checks. And yet 17 months passed and no one - on the white hat side at least - did those simple checks on the publicly available keys, until this Wayne guy. Everyone, worldwide, just trusted the process.
I don't think eMudhra is right to shift blame to everyone else, and I'm not doing either (I mean, I sure as hell didn't check them).
But it's a scathing indictment of any process that relies on public scrutiny. Open source software. Some of the subtler blockchain stuff. Corporate records. Court records. Political disclosures. Mathematical proofs. Wikipedia articles. Other obvious stuff I can't even think up right now. Obviously some of this is widely acknowledged to be imperfect (anyone spending 5 minutes on Wikipedia knows that) but I still strongly suspect there's in general way too much optimism that 'someone else' is keeping it honest when it in fact no one is.
Working with an Indian outsourcer some years ago, I noted a difference between how we in Britain define a person who knows what they are doing with how India defines this. We look for deep understanding of how a system works, they look for "Person has followed instructions to letter several times".
In other words, competent in some Indian outfits merely means "Follows a recipe accurately", it doesn't denote deep understanding. Obviously there are folks with deep understanding of things in India, but there are a lot of gormless instruction-followers. I think we are looking at a gormless instruction-follower and a flawed set of instructions here.
> "Obviously there are folks with deep understanding of things in India, but there are a lot of gormless instruction-followers."
As has been noted by others, there are likely plenty of talented Indians, but anyone half-decent will know they can do better working for almost anyone else- or themselves- than for the shitty consultancy middlemen who pay peanuts while charging customers through the nose.
I used to refer to this as the ‘press button A’ issue. The procedure is to press button A, then button B and finally C, and everyone follows it; but nobody actually understands what buttons A, B and C actually do, what happens behind the scenes?
In an earlier post, someone referred to possibly a blind following of a StackExchange post. I’m tempted to think this might well be it, a post of SE goes through the process using deliberately simple keys, because it made it easier to explain the process. Alas this has been followed ‘to the letter’, and hence......
When the procedure fails to include the "Before pressing A" step.
For example (and I'm sure I've mentioned this before) I used to work at a place that got a job experience worker to program EPROMs on their production line for a week. After a couple of days every unit was failing in test*. Investigation showed that the EPROMs were blank (EPROMs programmed on day N were used on day N+1). They tried to blame the work experience guy, until I pointed out that he was following the process he had been taught perfectly - they had forgotten to show the "Load the master into the programmer at the start of the day" step.
* Testing was done after sealing the units, but that's another story.
At number 1... it's the Bee Gees with "Massachusetts"!
Unless you meant this week, in which case it's apparently "Patient Zero" by Taylor Swift.
Yep, sounds like the whole situation is just one giant number 2.
I mean one or two screwed up private keys looks like a mistake but more than that? Methinks eMudhra are rather lacking in the technical competence, or indeed the general having some kind of clue, department.
@Empire of the Pussycat
At Number 1, "Mississippi" by the group "Pussycat"
The Spaceballs code is indeed almost 10 bits, which is indeed more secure than eMudhra's 8.
Maybe we need a new meme.
Dark Helmet: What's the secret key?
Colonel Sandurz: 4
Dark Helmet: 4? That's the stupidest key I've ever heard in my life! That's the kinda thing an idiot would use for a signing certificate.
eMudhra: 4? That's amazing! I've got the same key for my signing certificate!
Obviously, they were using the xkcd-approved random number generator for their private keys.
You say that in a manner that suggests you're suggesting that seriously, rather than as the outlandish joke it would normally be taken as.
Sad to say that, as ludicrous as that possibility *should* sound, I'm very much inclined to agree that it's quite a plausible explanation. And one which would be utterly damning were it to turn out to be true.
— Randomly choose an integer between 1 and a gazillion.
— I'll have 4.
Good isn't it.
It's not that 4 is any less random than a zillion + 1 but it is near an edge where all attacks commence and it will be used where it should be fairly large.
They have an impressive line up of logos on their home page, and boast of recommendations from the likes of Gartner and IDC. The red faces should not just be limited to their cert generation process, but to everyone who said "Looks good to me" and moved on.
I'm still wondering how you end up with "4" as a private key. You'd have to hand craft your certs as I don't think ssh-keygen would even let you do that. I can only assume these were certs generated for some kind of internal testing that ended up in production.
Stackoverflow obviously wasn't available so they cut and pasted the key generation code from the next best place.
@upsidedowncreature
SNAP!
"......there should be large financial penalties against them to ensure that they never, ever repeat this".
I would recommend nobody uses their outfit (or any other that employs anyone who has worked for them) for providing certificates ever again. Any other type of financial penalty is likely to amount to little more than something that can be written off as a cost of doing business.
How about false advertising, fraud, professional misrepresentation, failure of due diligence, just to start with.
This isn't "an innocent mistake", it's a company which was supposed to be trustworthy enough to function as a Certificate Authority exposing many people and companies to risk as a result of their incompetence and lack of safeguards.
I tried deleting the eMudhra CA certs from my browser and they reappeared - it's getting it from Linux.
I looked at deleting/blocking it in Linux, but distro updates may add the CA back if it issues new certs so I now have to script checks to ensure it's not snuck back in.
eMudhra certs can be added inside my browser when updated, not system-wide and regardless that they're blocked in the OS, so I have to check that periodically too.
Given the potential impact of a dodgy CA I'm astounded how difficult it is to block a CA who, in my opinion, are too stupid to be on the internet.
How bad does the breach of trust need to be for a CA to be distrusted and removed? Particularly when the CA operates under the authority of a foreign government.
You can use /etc/ca-certificates.conf on most distributions (check the manpage for ca-certificates or update-ca-certificates for your distribution).
Prefix certs you want to disallow with an exclamation mark (!), if this file is updated, you should be presented with the option to diff the files. (that may be skipped if you update with the -y argument, but that's not a great idea to begin with)
A very interesting related video: No one can force me to have a secure website!!! | suckerpinch | YouTube, it goes into great detail how you can do really stupid things with cryptography in order to make https wildly insecure.
It raises a serious question.
How do you determine whether a single allegedly random number is indeed randomly chosen? There are statistical tests for a large sample but not for a sample of one. Should we demand that, say quarter to three quarters of the bits be set and reject any that the random number generator produces that don't match those criteria? If we do that, then the result is less random, not more.
You use a certified random number generator to get you 48 random bytes (for a 384 bit key).
To span as much of the potential key space as possible, you ensure that the most significant byte is not zero - just set the most significant bit to 1. That's your private key - keep it safe!
You then multiply* the generator point by the private key to get the public key.
Job's a good 'un.
Gets more complicated if you want to support different key lengths - you need to generate 64 random bytes for a 512 bit key!
The security of elliptic curve cryptography is guaranteed by the difficulty of deriving the private key even if I tell you the start point, the properties of the curve, and the end point. If I want to brute force it, the search space is 2^384 possible keys and without a working quantum computer I won't be able to do it. However, if I have reason to believe that the private key is restricted to the range 1 .. 255 then I've already cracked it by the time you've read this sentence...
* Multiplication in an elliptic curve sense, essentially pushing a point around the elliptic curve each time you add to it.
Apparently the keys are easy to find when values are close to zero.
I don't understand the encryption, so I don't know if that's because computing the hash table is trivial when most of the bytes are zero, or if it's just that the decryption algorithm (which is O[sqrt(n)] instead of O[n]), is normally started at zero.
Apparently the keys are easy to find when values are close to zero.
It would certainly seem to make it more likely that someone could accidently stumble across them - for instance using the low value as a test case and noticing it produces a clash with a published public key.
I recall that some cryptographic functions have known weak keys and part of the normal key generation process is to check that the 'randomly generated key' is not a weak key*. I am surprised that the eMudhra key generation process did not automatically check for weak keys. (Though trom the snippets I have read so far it might be a case that some deliberately weak keys might have been introduced to the system [bad idea] and later stages of the verification and acceptance process did not/could not check whether the certificate being presented was derived from a weak key.)
Hopefully eMudhra will be transparent in the investigation of what went wrong and all CA's will carefully** look at their own processes to make sure a similar mistake can't occur again.
* It is a while since I studied the topic - I can't recall if it was related/similar to Carmichael numbers and the Fermat's Little Theorem prime test.** Not just put out a memo to the effect - "Security is our highest priority, lessons will be learned."
...could be potentially huge, given that a large number of Western governments (in particular UK) off-shore huge swathes of their systems integration, deployment and build to giant contractors sourcing a lot of their engineering in India.
What's the betting that certificates signed using eMudhra certs somewhere in the chain are scattered everywhere across critical infrastructure, for things like mTLS etc..?
In my experience, compliance teams will talk a good game and profess to provide assurance around the mitigation of risks such as this, but in reality when a large SI is pushing towards a deadline, everything is literally bent, tortured and shaped to fit some nonsense project or programme plan. The due diligence value effectively drops to zero.
This is a truly epic cl*sterf*ck, and eMudhra won't be the only ones.
any project where the deadline is considered more important than the quality
Every project in the world, probably. But a special mention goes to any old Soviet project, that had to be completed in time for some political event. Or, nowadays, Trump's America.
The two prime numbers approach is RSA keys. These are ECC (elliptic curve cryptography) keys. In principle, any integer that's in the field of the chosen curve is a valid private key, and then you compute the public key from that. In practice, 4 is only valid in the sense that you can do the mathematics on it, not that you should.
Think of that as an RSA (two primes) key where P (one of the primes) is 5. The key will work. It won't be good. In both cases, you can find out whether people did that by brute-forcing the easy numbers, limited only by when you feel like stopping.
Like, annually, by some organisation that audits the CA to ETSI or WebTrust... and isn't part of that audit a check on the way that signing processes and cert generation, etc. etc. are carried out (with sampling)? I'm a bit out of date as I am a happy retired greybeard, but I recall such audits on Belgian and Dutch CAs being a) thorough, and b) failure and dumping of CA was an option.I guess the Comodo hack gave them a warning. And yes, that was around 15 years or more ago.
plus ca change
Without telling me. That's exactly the sort of mistake that would happen if you gave an inadequate prompt to a crappy AI, and only checked that the code ran and that the certificates it generated were valid.
The only mystery is how you word your prompt so that instead of randomly generating a 256 bit AES key it randomly generates an AES key using 256 different numbers from 0 to 255.
That is almost always what happens since showing keys to humans doesn't do much (let's see, that's two really long strings of digits ending in an odd number*, guess that looks fine). The question is the quality of the code that generates the keys. I don't know what happened to make this very bad key get generated, but it was either bad code or someone deliberately trying to create bad keys for reasons unknown. Someone copying the key without permission would also be a problem, but that would be a problem even if the key was itself strong. In fact, if someone was deliberately trying to do that to sell a copy of the key, it would make sense for them to create a good key before copying it so that it wouldn't get revoked until someone knew it was being abused.
* Example of RSA keys, not ECC keys, but you get the idea. In fact, manual inspection would make it obvious that this private key is definitely not a valid one, so maybe that would have helped. The problem is that nobody expects to be able to identify a bad key by sight because they have to be ridiculously broken for that to be possible, and if you thought that would happen then the computer could also identify that with one if statement, so nobody tries it.
I would expect most "HSM"'s to be worse than writing a bash script that uses openssl (or preferably gnuTLS really) to randomly generate root certificates.
After all, hardware engineers are even worse at security than software engineers.
Some "HSM" collecting the key and then not printing anything about the key clearly worsens security - as we've found out, nobody notices that the private key is 0 prefixed 4, as they can't check it (a "HSM" should refuse to work and return an error if a key is too low entropy, but those hardware engineers…).
It would have marginally improved security in this case if someone knew the basics of how cryptography worked and viewed the private key - as they would have seen that the key is trivially bruteforcible.
Malice shouldn't always be assumed in cases that can be explained by incompetence.
Many governments aren't so dumb to require a key you can bruteforce by hand (although they often require things slightly less dumb).
The government would do something like demand the private keys be provided and then leak them.
In this case, the private keys of a trusted CA aren't useful alone - although for any MiTM'd TLS connection, for clients that trust the CA, the connection will successfully succeed with no error and the target won't notice, unless they check the certificate chain and notice that the CA is different from the one that the site uses (almost nobody does this).
I've read about governments making CA's sign malicious TLS certs for MiTM use.
If only the powers that be would impose a $1,000 fine... per cert signed with these keys. (From the article, that's 60 million +.)
To quote eMudhra's post you so helpfully linked to:
"There were no end entity certificates issued out of the affected CA’s. Its infrastructure was not compromised and no data was breached."
That you KNOW of. Given the private key was trivial, anyone could have created any certificates and signed them with that key, and they wouldn't know, as it didn't happen on their systems. Which would allow anyone to impersonate any site (Gmail, a bank, government websites...), using a certificate trusted by most browsers.