OpenAI is developing software to detect text generated by ChatGPT

OpenAI is building software capable of detecting whether text was generated by its ChatGPT model after New York City education officials announced it was blocking students from accessing the tool in public schools. Reports of students using AI to do their homework for them have prompted teachers to think about how they affect …

[Omnipresent]

The Robot Warz

So now we have competing AI entities with control over robot armies that can repair and reproduce themselves. Companies like Apple and RING are going to force you into an ecosystem nobody wants to be in. Consumers are ready to eat thoughts produced by machines and read to them by machines. WE ARE DOOMED. Too stupid to live. For some reason (call it greed) we cannot stop doing evil to each other. "Give a monkey a stick, and he will beat another monkey to death with it."

-The Expanse.

[Anonymous Coward]

Re: The Robot Warz

I can easily see, sooner rather than later, that you have to enter more than just a few words of text (say, applying for a job online), and when you click 'submit' or 'move to page 434, you will see the following:

'Unfortunately our systems have detected what appears to be AI-generated text. Please note that we do not allow AI-generated text. Please re-phrase in your own words and try again. Good luck!'

[Arthur the cat]

Nice way to make profits

Sell traps to the poachers and sell trap detectors to the gamekeepers.

[JDX]

Re: Nice way to make profits

Except they don't sell it.

They are producing a tool, that's all anyway. It's not cynical for the same company to sell scissors and tape/glue.

[b0llchit]

Pandora's Box

• OpenAI opens Pandora's Box
• Everybody panics
• OpenAI tries partly to close Pandora's box
• Everybody, including OpenAI fails to mitigate the fallout
• ...
• Profit or the world ends?

[Anonymous Coward]

Re: Pandora's Box

Probably both..

[Anonymous Coward]

Re: Pandora's Box

well, one way to 'solve' it is to build a LARGER pandora box, to fit a small box into it. So, you open a larger pandora box and...

[zuckzuckgo]

Re: Pandora's Box

If we train new AIs on mostly web scraped text, what happens when most web text was actually created AIs? A positive feedback loop that results in an AI controlled evolution of written language?

Not quite "the singularity" but one step closer...

[Pete 2]

Just ask it

If people are worried that a piece of text was written by ChatGPT, can they not just ask it?

Say "Hello, did you compose this text?" and if the answer is yes, then you know where it came from.

Whether the AI would be smart enough to detect subtle changes in wording, such as substituting another word with the same meaning, would be worth knowing, too.

[T. F. M. Reader]

Re: Just ask it

[I]f the answer is yes and, indeed, if the answer is no it is most likely not ChatGPT.

I played with it a little bit, and one thing I could never manage was to make it give a simple, straightforward answer. Certainly it never gave me a yes/no answer to a yes/no question. Of course, this is a direct consequence of how it works: the basic algorithm is something like "what is the most probable next token (where token = a word, a phrase, or punctuation) that would follow this text fragment (prompt) based on the statistics of the corpus of text that is your training set?" I find it rather amazing that the algorithm that can be stated in a single simple sentence would give such good results just because you get good statistics from a large enough sample. But once it comes up with a few sentences it never reduces the answer to something simple like yes or no or maybe.

Rather, ChatGPT sounds to me like a rather bad grammar school teacher who repeats memorized texts no matter what the question is. If you pardon what sounds like a pun but is actually quite serious, it sounds robotic.

So you can ask it if it wrote something, but then it will be up to you to decipher the answer that is likely to cover every possibility.

[Version 1.0]

Re: Just ask it

A student has already sorted this out by creating GPTzero - to help educators detect ChatGPT.

[Quentin Finknottle Again]

Re: Just ask it

I just tries GPTzero with 3 paragraphs (28 sentences) of computer generated lorem ipsum from lipsum.com. It got a score of 397 and the suggestion that "Your text is likely human generated!"

[Polhotpot]

Dey took 'er jerbs

"Consequently, GPT-3 is prone to making authoritative statements that are utterly and completely false."

It sounds like media pundits should be more worried than software developers about their jobs being replaced by ChatGPT.

[tiggity]

Re: Dey took 'er jerbs

And politicians - masters of false statements said authoritatively (in the UK, cannot comment on other areas)

[Anonymous Coward]

Re: Dey took 'er jerbs

I can comment on about 3 - 4 other 'areas' which I monitor, and for all of them, it's the same. Because, you see, honesty doesn't pay, while being a lying (...) does, and when they realize and force you out (maybe, just maybe), by that time you will have filled your pockets. And boots. And it's not stealing, nosir, stealing is for stupid people...

[sreynolds]

Re: Dey took 'er jerbs

Yeah those lackeys just regurgitate press releases these days. Most of the "journalism" is actually corporate propaganda, If the current model is the human centipede then AI can't possibly make it any worse.

[david1024]

This is not the way

While we are always going to want detectors to at least make folks use the latest version of the opposing AI.... Everytime a new bear trap comes along, the bear will just get smarter.

This is just going to be whack-a-mole -- AI edition.

[Occasional Comentard]

Sources

Just require students to add sources to their work. This should be standard practise for factual essays anyway. Chat GPT cannot list it's sources. Probably it has no record of where it's training data came from

[Killfalcon]

Re: Sources

And no way to link a given bit of training data to the output.

Even if you were to ask it to mimic the style of, say, Stephen King - it's model of "Stephen King style" is built on every King-authored work it has, *and* every work tagged up as imitating him (including the ones that are tagged that way in error). It's got no way to say "this bit of my training is from StephenKingFan1997, this bit is from that AO3 fad where people rewrote Lord of the Rings in other styles, and this bit is from The Dark Tower".

note - I have no idea if the works of Stephen King or AO3 was used in training ChatGPT - just an example of the function.

[JDX]

Re: Sources

I actually asked ChatGPT a SK question "Who is ted brautigan" the other day. It gave me a believable but incorrect answer, referencing him as a character in totally the wrong book.

I asked again today, and it gave another believable answer which is partly correct:

Ted Brautigan is a character in Stephen King's science fiction novel "Hearts in Atlantis". He is a telepath who has the ability to read minds and has fled to the United States to escape persecution for his abilities. He befriends a young man named Bobby Garfield and helps him to understand and develop his own telepathic abilities.

That last part is simply untrue. It sounds exactly the sort of thing you would read in a SK novel but Bobby has no special powers and Ted never teaches him anything of the sort, Ted's powers are mysterious and only explained in a separate novel where he returns.

It's a great example of the dangers of GPT. Even SK fans might not spot the error because it 'sounds right', and could easily miss detection if copied.

[Anonymous Coward]

Re: Sources

I asked it a fundamental question relating to beam strength (resistance to bending moments) which it got spectacularly wrong. Be careful which shiny new bridges you cross if you suspect not bothering with a real Civ/Struct Eng saves a few quid.

[katrinab]

Re: Sources

I'm pretty sure the "source" for most of my student submissions, certainly in the earlier years, was what I learned in the classroom.

[Killfalcon]

Re: Sources

That's not usually a problem - check the syllabus, and cite what it cites. (Or the textbook, which will likely cite it's own sources too).

It's essentially you saying where people should look to double-check what you've said, and/or to find further related detail. Just think of it as helping out your future readers.

[Wyrdness]

This could make ChatGPT better

I believe that one way to train AIs is to have another AI that's designed to spot whether the output from the first is real or not. These two AIs are then put into competition with each other. The first gets better at avoiding detection, whilst the second gets better at detecting fake content. So this is likely to help ChatGPT produce content that's harder for humans to spot.

[Anonymous Coward]

Apple is seeking to partner with indie writers and publishers to help them

ah, 'help', great word. Goes hand in hand with 'free'. Be afraid, be very afraid.

[Killfalcon]

Re: Apple is seeking to partner with indie writers and publishers to help them

In fairness, that 30% cut apple plans to take is half of Audible's cut - Audible takes 60% of the sale price of audiobooks if the book is exclusive to them. If an author wants to release on other platforms, Audible's cut goes up to 75%.

Right now, they have such a strong market share that few, if any, authors can afford to go elsewhere.

[DS999]

Re: Apple is seeking to partner with indie writers and publishers to help them

With the extra 15% if you release on other platforms it seems unlikely very many will be interested in adding Apple as a second publisher.

If you compare a 60% cut to Audible vs a 75% cut to Audible and 30% cut to Apple, they need to have 1/3 of their sales on Apple to break even. Any less then it doesn't make sense to accept the higher cut to Audible for a "non-exclusive". You'd need pretty significant non-Audible sales even if you had the option of a publisher will to PAY you to publish on their platform!

[Killfalcon]

Re: Apple is seeking to partner with indie writers and publishers to help them

It's basically an option for people who don't care about the revenue from audio books that much (eg, they make the vast majority of their money from text versions), or are writing books laser-targeted at users of the apple ecosystem.

[JDX]

So know the chatbot has to learn to evade detection

If we feed the results from the detection agent back into ChatGPT can it not train itself to produce output that will evade detection? Isn't that one of the main points of a ML chat-bot designed to appear human, that it is constantly learning how to be more believable?

Could be an interesting arms race, except these things learn in hours rather than months.

[vekkq]

indistinguishable

> Yes, generative language models can be good but they don't know what they're talking about

Problem is, I can say the same thing about many humans.

[ZenEagle]

First Impression.

When promoting a product or self the first impression is crucially important.

Here are my issues:

Opening an account requires entering an email address ... it is OK, but right underneath asks to enter the password. Is it to the email? Or a new password to the potential account to be opened.

After completing the account request a message said that a verification number has been dispatched. No verification in my Inbox has been found. After a while of searching through the junk and spam mail folders, a verification link has been found addressed From Register. I was expecting a confirmation name for the ChatGPT.

I just hope the actual software will be more accurate and refined.

[Mitoo Bobsworth]

Artificial Intelligence?

Coming from a species that still struggles with its' own biological intelligence - sounds like a really bad idea to me.