back to article OpenAI is developing software to detect text generated by ChatGPT

OpenAI is building software capable of detecting whether text was generated by its ChatGPT model after New York City education officials announced it was blocking students from accessing the tool in public schools. Reports of students using AI to do their homework for them have prompted teachers to think about how they affect …

  1. Omnipresent

    The Robot Warz

    So now we have competing AI entities with control over robot armies that can repair and reproduce themselves. Companies like Apple and RING are going to force you into an ecosystem nobody wants to be in. Consumers are ready to eat thoughts produced by machines and read to them by machines. WE ARE DOOMED. Too stupid to live. For some reason (call it greed) we cannot stop doing evil to each other. "Give a monkey a stick, and he will beat another monkey to death with it."

    -The Expanse.

    1. Anonymous Coward
      Anonymous Coward

      Re: The Robot Warz

      I can easily see, sooner rather than later, that you have to enter more than just a few words of text (say, applying for a job online), and when you click 'submit' or 'move to page 434, you will see the following:

      'Unfortunately our systems have detected what appears to be AI-generated text. Please note that we do not allow AI-generated text. Please re-phrase in your own words and try again. Good luck!'

  2. Arthur the cat Silver badge

    Nice way to make profits

    Sell traps to the poachers and sell trap detectors to the gamekeepers.

    1. JDX Gold badge

      Re: Nice way to make profits

      Except they don't sell it.

      They are producing a tool, that's all anyway. It's not cynical for the same company to sell scissors and tape/glue.

  3. b0llchit Silver badge

    Pandora's Box

    • OpenAI opens Pandora's Box
    • Everybody panics
    • OpenAI tries partly to close Pandora's box
    • Everybody, including OpenAI fails to mitigate the fallout
    • ...
    • Profit or the world ends?

    1. Anonymous Coward
      Anonymous Coward

      Re: Pandora's Box

      Probably both..

    2. Anonymous Coward
      Anonymous Coward

      Re: Pandora's Box

      well, one way to 'solve' it is to build a LARGER pandora box, to fit a small box into it. So, you open a larger pandora box and...

    3. zuckzuckgo Silver badge

      Re: Pandora's Box

      If we train new AIs on mostly web scraped text, what happens when most web text was actually created AIs? A positive feedback loop that results in an AI controlled evolution of written language?

      Not quite "the singularity" but one step closer...

  4. Pete 2 Silver badge

    Just ask it

    If people are worried that a piece of text was written by ChatGPT, can they not just ask it?

    Say "Hello, did you compose this text?" and if the answer is yes, then you know where it came from.

    Whether the AI would be smart enough to detect subtle changes in wording, such as substituting another word with the same meaning, would be worth knowing, too.

    1. T. F. M. Reader Silver badge

      Re: Just ask it

      [I]f the answer is yes and, indeed, if the answer is no it is most likely not ChatGPT.

      I played with it a little bit, and one thing I could never manage was to make it give a simple, straightforward answer. Certainly it never gave me a yes/no answer to a yes/no question. Of course, this is a direct consequence of how it works: the basic algorithm is something like "what is the most probable next token (where token = a word, a phrase, or punctuation) that would follow this text fragment (prompt) based on the statistics of the corpus of text that is your training set?" I find it rather amazing that the algorithm that can be stated in a single simple sentence would give such good results just because you get good statistics from a large enough sample. But once it comes up with a few sentences it never reduces the answer to something simple like yes or no or maybe.

      Rather, ChatGPT sounds to me like a rather bad grammar school teacher who repeats memorized texts no matter what the question is. If you pardon what sounds like a pun but is actually quite serious, it sounds robotic.

      So you can ask it if it wrote something, but then it will be up to you to decipher the answer that is likely to cover every possibility.

    2. Version 1.0 Silver badge

      Re: Just ask it

      A student has already sorted this out by creating GPTzero - to help educators detect ChatGPT.

      1. Quentin Finknottle Again

        Re: Just ask it

        I just tries GPTzero with 3 paragraphs (28 sentences) of computer generated lorem ipsum from It got a score of 397 and the suggestion that "Your text is likely human generated!"

  5. Polhotpot

    Dey took 'er jerbs

    "Consequently, GPT-3 is prone to making authoritative statements that are utterly and completely false."

    It sounds like media pundits should be more worried than software developers about their jobs being replaced by ChatGPT.

    1. tiggity Silver badge

      Re: Dey took 'er jerbs

      And politicians - masters of false statements said authoritatively (in the UK, cannot comment on other areas)

      1. Anonymous Coward
        Anonymous Coward

        Re: Dey took 'er jerbs

        I can comment on about 3 - 4 other 'areas' which I monitor, and for all of them, it's the same. Because, you see, honesty doesn't pay, while being a lying (...) does, and when they realize and force you out (maybe, just maybe), by that time you will have filled your pockets. And boots. And it's not stealing, nosir, stealing is for stupid people...

        1. sreynolds Silver badge

          Re: Dey took 'er jerbs

          Yeah those lackeys just regurgitate press releases these days. Most of the "journalism" is actually corporate propaganda, If the current model is the human centipede then AI can't possibly make it any worse.

  6. david1024

    This is not the way

    While we are always going to want detectors to at least make folks use the latest version of the opposing AI.... Everytime a new bear trap comes along, the bear will just get smarter.

    This is just going to be whack-a-mole -- AI edition.

  7. Occasional Comentard


    Just require students to add sources to their work. This should be standard practise for factual essays anyway. Chat GPT cannot list it's sources. Probably it has no record of where it's training data came from

    1. Killfalcon Silver badge

      Re: Sources

      And no way to link a given bit of training data to the output.

      Even if you were to ask it to mimic the style of, say, Stephen King - it's model of "Stephen King style" is built on every King-authored work it has, *and* every work tagged up as imitating him (including the ones that are tagged that way in error). It's got no way to say "this bit of my training is from StephenKingFan1997, this bit is from that AO3 fad where people rewrote Lord of the Rings in other styles, and this bit is from The Dark Tower".

      note - I have no idea if the works of Stephen King or AO3 was used in training ChatGPT - just an example of the function.

      1. JDX Gold badge

        Re: Sources

        I actually asked ChatGPT a SK question "Who is ted brautigan" the other day. It gave me a believable but incorrect answer, referencing him as a character in totally the wrong book.

        I asked again today, and it gave another believable answer which is partly correct:

        Ted Brautigan is a character in Stephen King's science fiction novel "Hearts in Atlantis". He is a telepath who has the ability to read minds and has fled to the United States to escape persecution for his abilities. He befriends a young man named Bobby Garfield and helps him to understand and develop his own telepathic abilities.

        That last part is simply untrue. It sounds exactly the sort of thing you would read in a SK novel but Bobby has no special powers and Ted never teaches him anything of the sort, Ted's powers are mysterious and only explained in a separate novel where he returns.

        It's a great example of the dangers of GPT. Even SK fans might not spot the error because it 'sounds right', and could easily miss detection if copied.

        1. gerdesj Silver badge

          Re: Sources

          I asked it a fundamental question relating to beam strength (resistance to bending moments) which it got spectacularly wrong. Be careful which shiny new bridges you cross if you suspect not bothering with a real Civ/Struct Eng saves a few quid.

    2. katrinab Silver badge

      Re: Sources

      I'm pretty sure the "source" for most of my student submissions, certainly in the earlier years, was what I learned in the classroom.

      1. Killfalcon Silver badge

        Re: Sources

        That's not usually a problem - check the syllabus, and cite what it cites. (Or the textbook, which will likely cite it's own sources too).

        It's essentially you saying where people should look to double-check what you've said, and/or to find further related detail. Just think of it as helping out your future readers.

  8. Wyrdness

    This could make ChatGPT better

    I believe that one way to train AIs is to have another AI that's designed to spot whether the output from the first is real or not. These two AIs are then put into competition with each other. The first gets better at avoiding detection, whilst the second gets better at detecting fake content. So this is likely to help ChatGPT produce content that's harder for humans to spot.

  9. Anonymous Coward
    Anonymous Coward

    Apple is seeking to partner with indie writers and publishers to help them

    ah, 'help', great word. Goes hand in hand with 'free'. Be afraid, be very afraid.

    1. Killfalcon Silver badge

      Re: Apple is seeking to partner with indie writers and publishers to help them

      In fairness, that 30% cut apple plans to take is half of Audible's cut - Audible takes 60% of the sale price of audiobooks if the book is exclusive to them. If an author wants to release on other platforms, Audible's cut goes up to 75%.

      Right now, they have such a strong market share that few, if any, authors can afford to go elsewhere.

      1. DS999 Silver badge

        Re: Apple is seeking to partner with indie writers and publishers to help them

        With the extra 15% if you release on other platforms it seems unlikely very many will be interested in adding Apple as a second publisher.

        If you compare a 60% cut to Audible vs a 75% cut to Audible and 30% cut to Apple, they need to have 1/3 of their sales on Apple to break even. Any less then it doesn't make sense to accept the higher cut to Audible for a "non-exclusive". You'd need pretty significant non-Audible sales even if you had the option of a publisher will to PAY you to publish on their platform!

        1. Killfalcon Silver badge

          Re: Apple is seeking to partner with indie writers and publishers to help them

          It's basically an option for people who don't care about the revenue from audio books that much (eg, they make the vast majority of their money from text versions), or are writing books laser-targeted at users of the apple ecosystem.

  10. JDX Gold badge

    So know the chatbot has to learn to evade detection

    If we feed the results from the detection agent back into ChatGPT can it not train itself to produce output that will evade detection? Isn't that one of the main points of a ML chat-bot designed to appear human, that it is constantly learning how to be more believable?

    Could be an interesting arms race, except these things learn in hours rather than months.

  11. vekkq


    > Yes, generative language models can be good but they don't know what they're talking about

    Problem is, I can say the same thing about many humans.

  12. ZenEagle

    First Impression.

    When promoting a product or self the first impression is crucially important.

    Here are my issues:

    Opening an account requires entering an email address ... it is OK, but right underneath asks to enter the password. Is it to the email? Or a new password to the potential account to be opened.

    After completing the account request a message said that a verification number has been dispatched. No verification in my Inbox has been found. After a while of searching through the junk and spam mail folders, a verification link has been found addressed From Register. I was expecting a confirmation name for the ChatGPT.

    I just hope the actual software will be more accurate and refined.

  13. Mitoo Bobsworth

    Artificial Intelligence?

    Coming from a species that still struggles with its' own biological intelligence - sounds like a really bad idea to me.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like