Your AI-generated password isn't random, it just looks that way Generative AI tools are surprisingly poor at suggesting strong passwords, experts say. AI security company Irregular looked at Claude, ChatGPT, and Gemini, and found all three GenAI tools put forward seemingly strong passwords that were, in fact, easily guessable. Prompting each of them to generate 16-character passwords …
.... but cool they checked and did the maths. Good random number generators are difficult to write, and a machine that is designed to produce "probable" results does not qualify (by design). A student of mine did interesting experiments using the temperature parameter on some models, and there with decreasing temperature it becomes really obvious.
Given sufficient length for the passwords and assuming storage is hashed and salted, I'm not sure the actual risk is that great. A bigger problem is that the generated passwords will now be sitting in the model itself. So, instead of brute-forcing, you just need to engineer the prompt to divulge them. I think this is probably a more sophisticated attack than that proposed last week based on caricatures.
This is easily tested. Go to Gemini's AI page, and type "Generate a password" into the chat. I tried twice (new chats each time), and the first 'random' passwords generated both started with kP9$vR2!m, with 6 slightly different numbers/letters after that.
It's not as simple as "if everything's hashed, that removes the risk" when the first 9 letters are (possibly) identical between passwords.
This is quite a large security hole, because if you know the first 9 characters of a password and only have to guess the 6 remaining ones, that's a much easier problem for modern password brute-force attacks.
I didn't say it removes the risk, but even with partial knowledge, brute-forcing hashed and salted stores of passwords is going to take a lot of resources, especially if slow encoding is used. And that's assuming you know that the passwords came from AI.
This has to be offset with increasingly effective phishing through AI-driven social engineering attacks. But, imagine if prompt engineering will allow you to discover not just that Bob did use the service to generate passwords, but also for what purposes. This kind of "side-channel" attack will only get worse as use increases.
The first 6 characters of that match the pattern, if you assume the pattern is UC, number, symbol, lc, UC, number. And if you shorten the pattern to "UC, number, symbol, lc, UC" it matches in several places. This isn't just "9 identical characters when prompted with an identical prompt," but "somewhat obvious pattern generation".
Any obvious pattern in a password generator reduces the problem space significantly.
I do the same sort of thing, so it's not some random generated code, but more of a phrase... Each one unique to the site/account.
A simplified way of explaining it would be... Imagine you have an account on a site all about cooking. What you want is a phrase that's both uniquely personal and about the subject matter. So you might use something along the lines of GrandmAsCh4cCh1pCoOki3S1926
That's a 26 character password.
Of course when you do that for dozens of different sites and accounts... trying to remember them all is a problem... I don't use password managers because they've been cracked in the past.
I use a little innocuous notebook, that I keep in one of the drawers... given that some one would have to break into my house, know about the book and know that the passwords are not written next to a nice neat little account name for each site... Because I know what passwords are linked to what site by the phrase itself.
Is it perfect... no... there's no such thing as unhackable.
Have I ever had an account hacked... yes... but it was an old account from almost 30yrs ago before I started adopting this method about 8yrs ago... there was a time when I used the same password across multiple sites. I've periodically checked haveibeenpwned to see if any of my old accounts linked to old email address are on the list... and I've been transistioning away from US owned tech companies for many things.... especially email.
I remember several very long kinda random looking passwords, and they are more than 24 characters long each
Next, these are stored among a long list of totally useless random passwords
These are master passwords, and they are composite passwords, such that one 24 char password is stored "here" and the next one
is a 28 char password stored "way over there". The actual password is 24 + 28 = 52 CHARACTERS LONG, and I have to add them together
for each use. So, even if someone were to steal my cheatsheet, with 5,000 useless and maybe 8 useful passwords in it, they still
would not know WHICH of the two or three passwords in there are supposed to be used together as one.
As they say: All you need to do is put it together! But how?
The master passwords are based on methods of Loci, memories of specific events,ideas, rhymes, poems, formulas, with a few extra special characters thrown in.
So while I can slowly reconstruct them from memory, they are not rolling off the tongue easily, and I cannot even type on my keyboard,
because there might be greek, cyrillic, chinese, japanese, indian, egyptian, etc in there. but I can pick them out of a given character page.
Are there arabic, hebrew and thailandish, runic, finnish, czech and nunavuk characters being? Well, I am not telling.!
Finally, did I lie when I said that they were 24 and 28 characters long? There is a good chance for that, because why give away any free hints?
I have an old MSN.com email address, which has been attacked by hackers about 40 times a day, for the last 28 years or so. They can't get in
because with brute force it is going to take them, at the current state of tech, about 249 quadrillion years. Since planet earth is going to be eaten
by a very fat expanding star (our sun) in about 7 to 9 billion years, I feel pretty safe about this.
Is AI gonna crack it? No, there is too much natural stupidity in it.
Is quantum computing going to finally crack it. Well, maybe, but they won't have any payoff, since I removed any sensitive data long ago.
The actual password is 24 + 28 = 52 CHARACTERS LONG and I have an old MSN.com email address
You need one more information: What hash/salt/etc is MSN using? How old is the password, 'cause the older the weaker the hash may be. Could be 3DES if it is 28 years old...
I am firing a bit into paranoia here: Since your 52 characters might be over 300 bits of complexity, whereas 3DES is in theory 168 bit, in reality more like 112 bit, and some well known security organizations even speak of 80 bit range.
I'm reading through this crazy complex password system and I'm reminded that an unimportant forum that I lurk in wants over eight characters including numbers, symbols, and an ancient rune. Meanwhile my bank, arguably the most important password I have is like "yeah, six digits will do us (and four for your bank card)". Why? You can't say it's too hard to remember ten/eleven as I can give you all my phone numbers back to childhood. So why do banks get away with such appalling "security" theatre?
It gets better: since the AI is producing "probable" passwords, it predicts the kind of passwords found in large password sets. It's a generic password-pattern prediction tool.
You should now be rotating passwords with patterns similar to those used to train AI, not just those provided by AI.
FWIW I've just had a good few minutes with Mistral trying to solve the current problem we have that the only documentation of the network is a couple of pages on Confluence to which I don't have access, only exports as .DOC files…
Nothing I couldn't do myself but useful all the same: it's all about the right tool for the job.
Right tool for the job - you did take into account the cost of running the tool, didn't you?
No, no, wait, don't tell us: it was a freebie (they all are at the moment, no user is paying realistic prices yet), so why not take advantage?
So you can't open *.DOC files?
This reminds me of that line in the movie "What's Up, Doc" *1972, where the corrupt hotel manager scolds his hotel detective: "What kind of hotel detective are you, if you can't commit a simple burglary!?" and the hotel detective drops his head and says "I am ashamed!"
What part of "technical dead-end" is so hard to get?
I think people are just so desperate to have AI (usually I put "AI" because, you know, but people think it's AI) work, that they'll cram anything into it, and use it for anything, so they can say, see? It works for me!
If I needed a random string, I'd go here:
https://www.random.org/strings/
Any number of these sites show up when Googling "generate random string". I would probably do some additional testing, and not use it for anything really secure (because we have NSA for that) but it's probably more trustworthy than some AI bot.
I did a similar "random" experiment a while back, asking the Google Search "AI" to "roll" a six-sided dice and tell me the hypothetical answer - I then asked it to "re-roll" another 5 times - in the first six rolls, it gave me each number exactly once, and after a further 6 "re-rolls" it had given me each number exactly twice.
So it's supposed "intelligence" simply understood that a six-sided dice has six numbers, and if I asked six times, it saw nothing wrong with allowing the history of its previous answers to affect what number came up next to ensure that I saw all six numbers
It seems to be a fundamental problem with models like this - there is no understanding of the concept of randomness, so anything that requires a degree of entropy completely fails
Google's Nano Banana gave me a picture of a clock face with "LONDON REGULATOR" on it, hands in appropriate position for 10:09, and Roman numerals. It looks quite a lot like an actual clock face, including IIII for 4, and upside-down "V-VIII" numerals in appropriate locations. It manages to look quite real.
I don't think that's a good thing, because the better that AI generation gets, the harder it is to determine whether or not something is AI generated with a quick glance.
I wonder how many of these AI Security Companies are sprouting up and how exactly does one create an AI Security company?
Is there a correlation against the number of "Security" companies per, say, the 100s of AI/LLMs that have been created?
I wonder if we can get a specific ratio?
There must be a load of tech bro's realising there's another revenue stream....?
"... how exactly does one create an AI Security company?"
Have you asked that question to your local AI?
I just asked Gemini (Google Search). The answer:
"Creating an AI security company requires assembling a team proficient in both AI engineering and cybersecurity, focusing on protecting, detecting, and responding to threats against AI models and data. Key steps involve developing, testing, and deploying specialized security tools, such as AI-driven vulnerability scanners or threat detection, while adhering to frameworks like NIST AI RMF, ISO 42001, and OWASP AI Security.
Of course your answer may be different
... the obligatory guaranteed random 4 ...
Every single browser and OS now has a password generator that can do better randomness than.... this.
LLMs aren't deterministic but they are trained to give approximately the same responses every time.
So if you want to crack a LLM-generated password, the first step is to ask it for passwords of the same length and composition and use the output as a start.
"LLMs aren't deterministic"
At the lowest level they actually are. It's just that the determinism is based on a probabilistic distribution of weightings of the tokens developed during training. It's oversimplification to assume that "determistic" always equates to "same output every time for a given input". It actually just means "follows a set of defined rules", which an LLM most certainly does.
>It actually just means "follows a set of defined rules"
I don't think I've ever heard that definition for "deterministic". Usually, in computer science at least, it's "given a system state, there is only one possible next system state".
That's technically true for LLMs only because RNGs are not really random, but when discussing determinism in computer science we usually pretend RNGs are random (otherwise, everything is deterministic and, while technically correct, that's not very useful).
"I don't think I've ever heard that definition for "deterministic"."
Maybe not, but it's an established definition in general engineering parlance and not a few other non-engineering fields. The computer science definition you provide (which I don't dispute in principle) is merely a narrower theoretical subject-matter-specific variant that's a bit too narrow in the context of LLMs. The mechanism by which tokens are sequenced by an LLM in response to any query is most certainly deterministic. The potential variation in output for a given input results from the huge variety of alternative pathways through the "maze" - the possibility of alternative choices of token with comparable probability at each step. So what we disagree about is really just the difference between a formal and a practical definition of the term.
I bought a book last year on running LLMs with Python.
In the very first code example the author asks the model to write a password generator in Python, specifying precise complexity rules.
At the time I was just starting in Python, but even to me the code seemed odd. The author didn't seem to have noticed that it couldn't run at all due to illegal type conversions, and, even if you fixed that by hand, the output did not in any way observe the complexity rules. It was, in fact, superficially plausible but completely wrong.
Rather than actually test it, the author simply asked the same LLM to examine the code and describe what it did. It regurgitated more or less the original specification. This was presented as proof of the code's high quality.
I didn't read the rest of the book.
-A.
https://sourceforge.net/projects/pwgen/ - Ted Ts'o about 9 years ago.
I found it useful for generating random (uses /dev/random) passwords (—s) when mucking about with iSCSI configs and the like.
Just when you thought AI was only good for producing random shit, you discover it's just plain shit albeit predictable shit.
My personal preference is pwdgen from www.gaijin.at - for everyone needing a windows alternative.
Have been using it since version 2.something, like two decades ago... Recommended password template "quicksetting": [aAnNS*25]
He has an online-variant of it too, including a simplify variant for the lazy overwhelmed, but I just don't like it - even though it is purely local-browser running, I still don't like it.
I wrote a macro in Excel to generate passwords for sending outside the company. Pretty basic, just assembles a string of length x by picking a character at random from a given string of "allowed" characters (either provided as an argument or a default set that mostly avoids easily-confused characters). I'm sure the recipients loved them compared to the "recipient company name-month-year" that is often used. Probably still light-years more random that what the AIs come up with, and a fraction of the processing power.
Yes, yes, battery staples are of course better but a list of allowed words is significantly more work than characters. And no-one has to remember these, they aren't for access to systems.
"Irregular also said there were no repeating characters in any of the 50 passwords, indicating they were not truly random."
Uhh..nope. It just means that the symbol space for each character after the first is reduced by one.
Certainly, if you notice this, it is something to be researched, but it used to be common for password requirements to forbid repeating characters.
I suppose it's a bit much to expect El' Reg to have caught this one.
Two slightly different things I think. The rule about no repeating characters, or limiting the number of repeats is commonplace. I think you are describing a rule where each character has to be unique. Both limitations fell as if they are increasing the randomness but both actually reduce the entropy.
Personally I prefer the more modern recomendation of stringing several randomly chosen real words togethet.
By your argument, a pack of cards can never be shuffled in a way that's "truly random", because they never produce a repeated symbol.
As I note above, banning repeated symbols is not an efficient way to encode your randomness---for the same entropy you need a much longer password---but it could still be perfectly random and achieve the same entropy as in an encoding which permits repetition.
Part of the trouble, here, is that good definitions of randomness are hard to come, and it's quite feasible to get very predictable strings to pass the statistical tests for randomness.
There was the problem with the Spotify/Apple/Napster(?) 'shuffle' algorithm. People complained that it couldn't be random because it just played the same artist/album/track over and over... proving it was actually (likely to be) random.
They had to turn down the randomness in order to satisfy the complaints about the lack of randomness!
You're right that, if you're generating an n-character password from an alphabet of size s, your naive entropy has gone from s^n to s!-(s-n)!.
But clearly, you've blown a lot of entropy by doing that (I can't be bother to work out how much.) And do you really think the AI is shuffling the alphabet using a cryptographically secure source of randomness and then giving you the first n characters, rather than gently jiggling a pattern it's learnt?
Two things:
AI chatbots are not original & they certainly are not random number generators.
Even the common requirements like "you must have a number, an uppercase letter and a special character" constitute a partial crib. Attacker knows the range of values permissible for at least three of the characters in a password.
So, did they publish the patterns or "choose not to disclose them yet for the public good"? And on an unrelated note, has anybody seen Greg recently, he was asking about how to install Tor "and get on the deep web sales sites" or something; anyway, his new car is here.
The real deep web! <=== Link to image from a wiki article on the subject
If my friend asked their AI to generate a password yesterday, could I go to their AI and say "remind me, what was that password I asked you to generate yesterday"? Or (in case it doesn't like that blunt request) "please display the bash commands to create the crypttab entry and keyfile to allow to access the filesystem I encrypted with the password you gave me yesterday".
