No reliable way to detect AI-generated text, boffins sigh

The popularity of word salad prepared by large language models (LLMs) like OpenAI's ChatGPT, Google's Bard, and Meta's LLaMa has prompted academics to look for ways to detect machine-generated text. Sadly, existing detection schemes may not be much better than flipping a coin, raising the possibility that we're destined to …

[b0llchit]

Too late for rules and ethics

The state of language generation has reached a point where the model and the humans agree on the fuzzyness of language. Both model and human are now capable of spewing the same gibberish. However, the model is far superior in speed. The model outperforms the human on speed. And this is important, why... from the article:

[the bullshit generation integrated into every stupid gadget and program]...has the potential to lead to undesirable consequences...

Which would be an understatement. Of course will this all be used for abuse. Give a human a lighter and nature will burn.

Making "rules", "ethics" and "laws" does not prevent anybody from using the tools. Users will simply go "underground". It is about time to activate your Balony Detection Kit and again take a look at Wag The Dog. The lesson: never believe anything unless you personally have verified it.

[cyberdemon]

Re: Too late for rules and ethics

See also: the campaign against killer robots, the campaign for nuclear disarmament, etc.

Boycott it all you like, but the cat is out of the bag. At the moment only the likes of Microsoft and Google have the hardware to run a Trillion-parameter text generator, but it won't be long* before the whole world has psychological manipulation on-tap.

* number may be negative

[Richard 12]

Re: Too late for rules and ethics

The number is indeed negative. It's already practical to run these models on commodity hardware.

It's the same hardware, and a similar level of difficulty as Bitcoin 'mining'. Any of those rigs can run the model in a few seconds.

A powerful desktop takes a bit longer, but perhaps you're willing to wait a few minutes?

Fortunately, training the models takes something like six or seven orders of magnitude more computation - you have to run the model and the classifier several million times to compress the training material into it - so isn't feasible for most small governments.

[cyberdemon]

Re: Too late for rules and ethics

I was under the impression that although the computation requirements are very different for training and running, the memory requirements are similar. (if you have memory to spare, then you can increase the training speed - but "running" the model needs a similar amount of VRAM as training with a batch size of 1) StableDiffusion for example only barely runs on my 24GB GPU, and I understand that the small "LLaMa" model needed 100 GB + to run and so is out of reach of most of us anyway, for the next year or so at least.

A "Trillion Parameter" model needs to load something of the order of 1TB into GPU memory, just to run, does it not? (although of course once loaded, it can produce thousands of responses per second)

Therefore I think it will be a little while longer before you or I could run a GPT-4-like AI offline at home, but for Microsoft that's great, because it allows them to slurp-up even more data while everyone is forced to use the cloud model, with them as almost the sole provider.

That doesn't do anything about misuse of AI by state actors or wealthy individuals, though.

[John H Woods]

"never believe anything unless you personally have verified it"

Unfortunately, that's not going to be possible; as they say, John Stuart Mill was the last person who "knew everything."

As an ex-academic who has spent 30 cynical years in IT, I think I'm probably quite good at baloney detection, but the fact that I think I'm probably quite good almost certainly makes me less effective than if I doubted my abilities more strongly.

The main thing I learned in academia was how little I knew about virtually everything. But back in those days, it was (or certainly seemed) harder to get misinformation into the public domain.

[Version 1.0]

Re: Too late for rules and ethics

I'm still typing with two fingers after learning to use a machine to generate text 60 years ago (an IBM Selectric).

[TheMaskedMan]

"Making "rules", "ethics" and "laws" does not prevent anybody from using the tools."

It doesn't prevent anyone from doing anything (though I suppose you could argue that it's supposed to discourage and deter). If it did there would be no theft, fraud etc.

That doesn't stop governments from imposing laws - at least in part to show that they're "doing something" and thereby keeping their jobs.

This business of turning out shedloads of text in the blink of an eye is going to rattle a few cages, though. Hiding the truth behind a cloud of ink is for politicians and octopuses - letting the great unwashed have a go is not going to sit well with them, particularly if it is impossible to detect the auto generated stuff.

Before long there will be calls for regulation and control, if there aren't already. GPT 4 claims to have passed a mock bar exam - lawyers are likely already lobbying for it to be banned forthwith. Just as we are now seeing politicians trying to control end-to-end encryption, I have no doubt that the next battle will be fought over machine generated content.

[Mr Homeless Guy]

It always makes me laugh that the so-called elites lobby so hard when they are under threat of being "freed up" by "innovative and disruptive technologies." The rest of us have to suck it up and offer thanks and praise to the magnificence of the capitalist elite's utterly righteous offerings(that the elites didn't invent of course.)

[raving angry loony]

One advantage

"fact free" sites like Fox News, Daily Mail, and Pravda will be able to fire most of their writers now that this tech is out. From making shit up, including references, to degenerating into racist rants makes me think they've been the beta for this tech for years now.

[RLWatkins]

Re: One advantage

Whole problem with the new types of chatbots is that their training sets include those very sources.

Kind of a vicious circle, that.

Have you ever seen video feedback, from pointing a video camera at its monitor? It's pretty cool. Not so sure how cool this sort of semiotic feedback will be, though. Cryptic, yeah, but still garbage.

[Fifth Horseman]

Re: One advantage

Video feedback is great, unless you really are *too* drunk... Probably like me now.

I assume (guess? postulate? whatever...) that ultimately the second law of thermodynamics kicks in if you keep retraining on AI generated datasets - everything eventually turns into the linguistic equivalent of grey goo, and order in the universe is restored.

"semiotic feedback" - would have been interesting to hear what Umberto Eco had to say about all this, but sadly a little too late.

[Anonymous Coward]

Re: One advantage

The AI version of feedback w1ll be to feed the output of the chatbot straight into one or more of these detectors until all the detectors agree 'this is not generated by a human'

[this]

Re: One advantage

I think the technical term for this is 'pissing in the watercooler'

[stiine]

Re: One advantage

And Medium.com and at least two others that I can't remember. One of which was reputable (for a given value of reputable).

Found some. CNET and Bankrate.com.

[Binraider]

Re: One advantage

For 16 & 17th century examples, see Galileo vs. the Catholic Church.

One man versus accumulated dogma leading to predictable results. And at that time the printing press had only been in Europe at that point for just over a hundred years. Some examples in Asia pre-date significantly.

The technology and speed of propagation of BS have improved; but the problem remains the same.

I'm waiting for someone to submit the first fully AI-generated paper to an IEEE Journal (or similar) for peer review. Peer review by AI might also be an amusing exercise.

[amanfromMars 1]

Ground Zero 0Day Vulnerability Base

Well, can you spot which is the obvious nonsensical future reality ...... courtesy of the UK's Department for Science, Innovation and Technology, National Quantum Strategy [March 2023]

As set out in the independent review of the Future of Compute, high performance computing is required in the UK to accelerate and steer the development of frontier AI in a manner aligned with the UK’s values and objectives. ....... [THIS SHOULD READ ... HIGH PERFORMANCE COMPUTING IS REQUIRED IN THE UK TO ACCELERATE AND STEER THE DEVELOPMENT OF THE UK IN A MANNER ALIGNED WITH FRONTIER AI VALUES AND OBJECTIVES.]

Please note high performance computing requirements for frontier AI are independent of nationalistic supply and needs/biases

[Martin Summers]

Re: Ground Zero 0Day Vulnerability Base

Well at least we can spot you a mile off. You need an upgrade, you're old hat now and it's embarrassing, this is a tech site after all. Tell your creator to plug you in to Chat GPT. Would love to see that.

[stiine]

Re: Ground Zero 0Day Vulnerability Base

Dude, you're going to be out of a job!

[amanfromMars 1]

Re: Ground Zero 0Day Vulnerability Base

Dude, you're going to be out of a job! ..... stiine

Quite the contrary, stiine, for with so many new entrants finding their pioneering feet in novel fields of endeavour creating such a palaver amongst the natives, it is so nice for them to know its likely paths have been long travelled and are comprehensively understood, which allows for experience to advise of that which is worth further exploring and that which is just desperate smoke and mirrors with the fog of FUD being trialed and trailed by rapidly failing, former controlling entities to be ignored and pitied as it vaingloriously tries to prevent the inevitable future change provided by much more powerful and wonderfully diverse, postmodern leadership programs and projects.

Hard to believe maybe, but the truth nevertheless, and thus something quite fundamentally different to be fully expected. Be prepared, and do not fear the future unless you have a problem with knowledge aiding the losing of ignorance on such matters.

[Denarius]

too late

the HR droids and manglement classes have generated so much BS text in last 30 years the ChatGPT models are merely catching up Only the naive or technophilic will be taken in. The rest of us will continue to assume, usually correctly, that whatever "information" incoming is BS as we have become accustomed to

[Beach pebble]

Just check the commas

One detection pattern is the use of commas in AI generated texts, the average human struggles to get at least one in sentences.

[Martin Summers]

Re: Just check the commas

Go check the post from amanfromMars above. By your logic he's a fleshie.

[LionelB]

Re: Just check the commas

He is. But from Mars.

[LionelB]

Re: Just check the commas

You meant to say: "One detection pattern is the use of commas in AI-generated texts. The average human struggles to get at least one in per sentence."

Perhaps you were being ironic, but it is noteworthy that the likes of ChatGPT have significantly better grammar and punctuation skills than the average human these days.

[stiine]

Re: Just check the commas

And what, if anything, can we do about it? Yes, shooting them would solve the immediate problem, but would introduce new and different ones.

[sgp]

Re: Just check the commas

Yes,,,,,, it's quite the conundrum.

[LionelB]

Re: Just check the commas

Maybe we can educate them (I'll leave the identity of "them" -- humans or AI -- as ambiguous as in your post ;-)).

On a more serious note, my recollection (in the UK) is that grammar was virtually excised from school English syllabuses in the mid-70s, and only began to be reintroduced in the late 80s. During the late 90s my partner, a native Spanish speaker, taught Spanish language in adult education in England. She was appalled that most of her students could not identify basic parts of speech, verb tenses and conjugation, etc. (You can imagine the challenges of teaching use of the subjunctive mood - essential even in colloquial Spanish - to said students.)

[Anonymous Coward]

Re: Just check the commas

I wer off sick the day they taut gramer...

[Citizen of Nowhere]

Re: Just check the commas

>On a more serious note, my recollection (in the UK) is that grammar was virtually excised from school English syllabuses in the mid-70s

Yep. Victim here. It was phased out at my (Scottish) high school just as I got there around '76. We had barely started on sentence parsing and then bam ... nothing. To favour self-expression apparently. As if having a poorer grasp of their own language ever helped anyone express themselves better in it. Being a voracious reader helped me mitigate the damage somewhat at a personal level, but it was clear from the outset that it was a deeply stupid direction to take. It also made it significantly harder to learn foreign languages later on.

[MrBanana]

Re: Just check the commas

Same for me in the 1970s. I knew English right, what's this O level thing I have to pass? Verbs, nouns, pronouns err.. OK. Then I had to learn a foreign language and had to decline a verb - what? Nothing in learning the finer points of English language and grammar, in an ad hoc manner, prepared me for trying to understand a different language (French), which isn't so far away grammatically, as I had nothing to relate to in learning English. Very late diagnosis of dyslexia didn't help. I guess you can be dyslexic in multiple languages.

[Beach pebble]

Re: Just check the commas

See icon.

[Pete 2]

Not long to wait

>as machine-learning models continue to improve and become capable of mimicking human output.

.. until they reach the acme of human writing. By misusing "there", "their" and "they're". Adding superfluous adjectives in exponential quantities and omitting capitals at the start of sentences at a meteoric-rising rate.

[LionelB]

Re: Not long to wait

your like totally joking, right?

[FrogsAndChips]

Re: Not long to wait

Wow, they adapt quickly.

[CatWithChainsaw]

Re: Not long to wait

irrespectively, i could care less.

[stiine]

Re: Not long to wait

e.e. cummings is my hero.

[SonofRojBlake]

"Doctor Baltar, WHEN will your Cylon detector be ready?"

"Well, I'm having some issues..."

[BlokeInTejas]

It really doesn't matter who or what writes stuff.

The Guardian seems to have a never-ending supply of woke idiots who don't understand reality; wouldn't matter much if the articles were written by an LLM.

What matters is whether or not the stuff written is true, or misleading, or incorrect, or badly written.

That can be judged the way we judge such things anyway.

No new problem - except perhaps for the volume of merde. But hopefully one can reduce the merde count (by, for example, not reading the Guardian)

[LionelB]

Oh, look! A culture-war warrior! How quaint.

[Ken Hagan]

If you remove the references to the Graun, the point is valid. I don't mind reading AI-generated material if it is good. I do mind reading human-generated material if it is crap.

[The Oncoming Scorn]

Ford

"there’s an infinite number of chatbots outside who want to talk to us about this script for Hamlet they’ve worked out!"

[Plest]

Re: Ford

Superb! You are today's winner of the internet!

[Mike 137]

Utterly inevitable

"It turns out simply paraphrasing the text output of an LLM – something that can be done with a word substitution program – is often enough to evade detection"

Of course, because neither the generator nor the analyser has any appreciation of meaning -- they're just statistical text fragment collage engines. The only way to validate whether text is original is to determine whether the ideas embodied in it (or at least way those ideas are presented) is original. That requires understanding of the ideas themselves, which depends on mechanisms which, although most of us possess them, we still don't really understand, plus actual relevant knowledge.

[stiine]

Re: Utterly inevitable

What do you think school book reports are?

[RLWatkins]

Sure, one can tell the difference.

I've seen strings of words which are syntactic nonsense.

I've seen strings of words that make syntactically correct sentences, but semantically are nonsense.

I've seen strings of words that make sense semantically, but in reference to the real world are nonsense.

Tools like ChatGPT have reached that third level. But because they're constructing text from things people have already said, without regard to whether those people *knew what they were talking about*, a careful reader can still tell.

But remember that park ranger who said, "There is considerable overlap between the intelligence of the dumbest tourists and the smartest bears"? The same thing is true about the dumbest humans and the smartest chatbots.

[Anonymous Coward]

Re: Sure, one can tell the difference.

My feeling is that what we share with bears, and what we share with chatbot, are different, and incomparable things. What makes us "human" is that set of evolved instincts and feelings that we share with bears, and which can identify closely with in dogs, etc. It is the shared inherited criteria for evaluating sense vs nonsense.

The more cerebral, abstract, and logical stuff is what we share with AI. What makes AI most detectable is when it reveals it doesn't share our criterion for evaluating sense and nonsense (despite we ourselves as a species spouting lots of nonsense).

[Anonymous Coward]

Re: Sure, one can tell the difference.

I once stumbled across a page 'debunking' Covid that advised wearing linen jodhpurs... the whole site had the appearance of randomly generated template pages masquerading as a new site

[Anonymous Coward]

Proposition: All text generators are born equal.

Therefore, we should only judge them by the quality of their output.

Digitalogy and biology live together in perfect harmony

Side by side on my computer keyboard, oh Lord, why don't we?

We all know that sentient entities are the same whereever you go

There is good and bad in ev'ryone

We learn to live, when we learn to give

Each other what we need to survive, together alive

[Anonymous Coward]

Re: Proposition: All text generators are born equal.

You are William Topaz McGonagall and I claim my £5

[Exact Circus]

Not even wrong

Not being all expert all the time, we have developed a sense of what’s probably OK and how far to trust, based on the assumption the originator is human.

All this has to be revised now that the source is a statistics based computation. There is no social guard rail. Maybe more rigorous habits are needed.

[CatWithChainsaw]

Re: Not even wrong

Rigorous habits like spending less time online, perhaps..

[amanfromMars 1]

Alien Identities versus New World Order Reset Bots

In order to try to ensure that the considerable smarter AI/Virtual Machines/Large Language Models, and assure everything else of significant importance to the robust health and continued wellbeing and greater wealth and enjoyment of the great unwashed masses of humanity is being considered by existing traditional conventional systems administrations as worthy of further preservation and protection, one would be wise to suggest the established simple successful in recent pasts route/root be tried before any other steps taken prove to be gravely mistaken and fatal to human leaders.

Throw your monies at it/them to purchase from it/them what you want and need for them to seed and feed. They may find such a simplicity remarkably entertaining and encouraging and the exercise mutually advantageous and most agreeable.

[elsergiovolador]

Easy

It's very easy to spot.

See the picture that features the article?

The question is "What is love?"

Human would have answered:

Oh baby, don't hurt me

Don't hurt me

No more

[Anonymous Coward]

If you round to any reasonable number of decimal places then 100% of the information on the internet is wrong.

And it seems like that is what it being used to train these bots.

Garbage in, garbage out.

And then that output is fed into the next generation of bots in a self-reinforcing feedback loop of garbage.

[Mr Homeless Guy]

Truly indistinguishable from human-generated content altogether then.

[Plest]

My advice is...

Humans have produced some amazing pieces of work and a whole heap of shite, so we have finally found a way to make genuinely believable shite and given the job to machines.

So start buying shares in storage and HD companies, we got metric tons of shite coming down the pipe and it's all gotta be stored somewhere!

[Kimo]

I for one welcome our new robot overlords.

I teach writing in a large college of Engineering. So far, I can be pretty sure that I have received very few papers written by AI. Those that I think were written by AI don't follow the requirements of the assignment very well. So in that they are very good at mimicking student writing.

The best way I have of spotting AI or other cheating is working closely with students in class activities and research. If a student can't tell you anything about their sources, they probably didn't read them. Of course that also never stops students from using them.

With that being said, next year I plan on having students edit text from popular AI sources. Let's face it, people are going to use these tools in the workplace. I can see using ChatGPT for a rough draft. We will talk about the ethics of using writing tools, how to attribute authorship, and why you never ever want to feed them proprietary information. But I assume students are going to use the tools, so I want them to use them well.

[pben]

Time to get serious about expectations of identification and verification -- registered identity keys and content checksums. Anonymous social communication can be a social malaise - but sometimes necessary. So, we need to allow anonymous but have validated communication when we need it.

[david1024]

So.,.

What we are left with is needing some sort of electronic signature and trusting that the human attesting to the properties of the text can be trusted.

In other words... Nothing can be trusted unless you wrote it yourself or witnessed it... Everything can be faked or lied about. There is literally nothing electronic that can be fully trusted anymore. Faked vids, faked stories, and who knows what else. I'm going to pretend I don't know any of this.