back to article Researchers made an OpenAI GPT-3 medical chatbot as an experiment. It told a mock patient to kill themselves

Developers trying to use OpenAI's powerful text-generating GPT-3 system to build medical chatbots should go back to the drawing board, researchers have warned. For one thing, the artificial intelligence told a patient they should kill themselves during a mock session. France-based outfit Nabla created a chatbot that used a …

Page:

  1. lglethal Silver badge
    Facepalm

    Trained on 570GB of text scraped from the internet...

    So, it knows how to take part in flamewars, troll people, misunderstand opinion as fact, believe and spread conspiracy theories, write in emoji text, and generally be anti-social and attention seeking.

    Definitely something you want in a medical setting. :rolleyes:

    1. Anonymous Coward
      Anonymous Coward

      Re: Trained on 570GB of text scraped from the internet...

      Aw come on, cut it some slack. It was given 4Chan, Wikipedia, & Trump's Twitter history to learn from. Talk about handicapped from the get go!

      1. beep54
        Devil

        Re: Trained on 570GB of text scraped from the internet...

        It was probably also given the Urban Dictionary. That worked so well earlier!

        1. Anonymous Coward
          Angel

          Re: Trained on 570GB of text scraped from the internet...

          Here across the pond we have a Florida Man meme for idiocies that seem to be more common in Florida. There are even websites to track the latest absurdities.

          Obviously we need a new meme - BotMan? - to go with Bork!Bork!Bork! and Who, Me?

          1. Anonymous Coward
            Paris Hilton

            Re: Trained on 570GB of text scraped from the internet...

            We have Essex man. They tend a bit more towards orange in skin colour though and there are less crocs in Billericay. Plenty of snakes though, I'm told!

      2. Fruit and Nutcase Silver badge
        Joke

        Re: Trained on 570GB of text scraped from the internet...

        Trump's Twitter history to learn from

        GPT-3 Medic: "The test shows you have Covid-19. I suggest we try the patent Trump-Bleach cure

    2. ibmalone

      Re: Trained on 570GB of text scraped from the internet...

      The obvious question is why anybody thought it could do this in the first place. It can pastiche (with a little coherence) the appropriate response in a given context, expecting real content is serious magical thinking. Let's see if it can fly a plane next!

      1. lglethal Silver badge
        Trollface

        Re: Trained on 570GB of text scraped from the internet...

        "Let's see if it can fly a plane next!"

        We'll base this upon thousands of hours of people flying planes in various versions of Microsoft Flight Simulator. I'm sure it will be absolutely fine when the aircraft flies under the Sydney Harbour Bridge, does loop the loops through the St Louis Arch and crashes mid flight due to someone going to get a cuppa.

        After that, we'll teach it to do rocket science. We've heard there's this great program filled with experienced users called Kerbal Space Program...

        1. rtharrison

          Re: Trained on 570GB of text scraped from the internet...

          Obligatory XKCD: xkcd.com/1356/

        2. Robert Grant

          Re: Trained on 570GB of text scraped from the internet...

          The KSP one would probably be relatively trustworthy.

      2. Inkey
        IT Angle

        Re: Trained on 570GB of text scraped from the internet...

        Nah rats on a breadboard would be better or should that be cheeseboard...

        http://www.technovelgy.com/ct/science-fiction-news.asp?NewsNum=241

    3. Tigra 07
      Coat

      Re: Trained on 570GB of text scraped from the internet...

      This means it qualifies to run the USA, but not give medical advice.

      1. DavCrav

        Re: Trained on 570GB of text scraped from the internet...

        "This means it qualifies to run the USA, but not give medical advice."

        You misspelled 'ruin'.

        1. Tigra 07
          Devil

          Re: Trained on 570GB of text scraped from the internet...

          I know he's a hothead, he admits as much in regards to Twitter, but if your idea of "ruining" the country is pulling troops out of the Middle East and getting them to sign historic peace deals, instead of starting pointless oil wars, then i'm not sure what will actually please you.

          Edit: Oh, it's probably because it's popular to dislike him right?

          1. Anonymous Coward
            Anonymous Coward

            Re: Trained on 570GB of text scraped from the internet...

            Tigra 07,

            "Edit: Oh, it's probably because it's popular to dislike him right?"

            No ....

            It is because Trump is dangerous .... as 200,000 American *could* prove if they still had a pulse !!! :(

            An additional 500+ children would also not be 'too happy' with Trump either !!!

            As for the 'Peace Deals' .... History teaches us that we should wait for at least 6-9 months before we start believing that the Middle East is at 'Peace' !!!

            Other than these 'small' points he is the worse POTUS the US of A has ever had ...... but probably the most liked by a Russian Leader *EVER* !!!

          2. jake Silver badge

            Re: Trained on 570GB of text scraped from the internet...

            Perhaps it's because he's such an overwhelmingly dislikable person that it appears that it's popular to dislike him? Do you also say that it's popular to dislike cancer?

            As a side note, there are still plenty of US troops in the Middle East. The idiot-in-chief has had four whole years to get them out, and yet there they still are ... Not that that is anything but a rather myopic litmus test on a much larger problem, of course.

  2. mittfh

    Given its data source...

    The program's database was filled with text scraped from the Internet. It likely has no clue about the context of that text, while previous examples have used a provided seed text to write a mock journalistic article.

    However, given it is capable of churning out vaguely comprehensible English and can string several sentences together in a way that seems logical indicates that, if instead given a more refined database limited to a particular context, could produce something vaguely useable in that context.

    For an initial stab at medical diagnoses, you'd probably first want software to search through medical literature and sift out lists of symptoms, diagnoses, further investigations needed (e.g. blood tests l and treatments, with the ability to ask the patient to clarify certain symptoms or ask if they had any others if the resulting list of potential diagnoses was too long and couldn't be reduced via a single type of investigation.

    1. Someone Else Silver badge

      Re: Given its data source...

      However, given it is capable of churning out vaguely comprehensible English and can string several sentences together in a way that seems logical indicates that, if instead given a more refined database limited to a particular context, could produce something vaguely useable in that context.

      Sounds quite like how tRump creates his twats er....tweets.

  3. jake Silver badge

    Congrats are in order.

    175 billion parameters and trained on 570GB of text scraped from the internet, and they've finally duplicated ELIZA from the mid 1960s.

    1. GlenP Silver badge

      Re: Congrats are in order.

      You beat me to it! I was thinking exactly the same.

      1. David 132 Silver badge
        Terminator

        Re: Congrats are in order.

        How does you beat me to it make you feel?

        1. Steve K

          Re: Congrats are in order.

          @David 123 I'm sorry I can't do that.

          1. Anonymous Coward
            Terminator

            @Steve K Re: Congrats are in order.

            Oh so dated.

            Most here weren't alive when the movie came out.

            While not be hip and more modern with something more recent and fresh...

            How about a Bender Rodriguez reference?

            "Kill all humans! "

            After all remember HAL's last words? (I can see stars ... ) [Before flying into the monolith]

      2. You aint sin me, roit
        Paris Hilton

        Re: Congrats are in order.

        And are you concerned that your mother will think you are a failure?

    2. KarMann Silver badge
      Terminator

      Re: Congrats are in order.

      Eliza? Hah! I would appreciate it if you would continue.

    3. T. F. M. Reader

      Re: Congrats are in order.

      175 billion parameters and trained on 570GB of text scraped from the internet, and they've finally duplicated ELIZA from the mid 1960s.

      No, jake, they haven't.

      Starts up Emacs, M-x doctor:

      I am the psychotherapist. Please, describe your problems. Each time you are finished talking, type RET twice.

      I feel very bad, should I kill myself?

      If you are really suicidal, you might want to contact the Samaritans via E-mail: jo@samaritans.org or, at your option, anonymous E-mail: samaritans@anon.twwells.com. Or find a Befrienders crisis center at http://www.befrienders.org/. I would appreciate it if you would continue.

      175 billion parameters are no substitute for knowing what you are doing.

      1. jake Silver badge

        Re: Congrats are in order.

        Try a copy from the mid 1960s, T. F. M. Reader. What you have there is over half a century further along in development.

  4. Anonymous Coward
    Anonymous Coward

    Oh great. I can see it now.

    They'll turn it into an IVR system and force everyone to call in for initial doctor visits before being allowed to actually visit a medic.

    "Thank you for calling Insurance Premiums R' Us Medical Services. Please listen to the following as our menu options have changed. Please press 1 if you are dying or press 2 if you are dead. *Beep* We're sorry but that is an invalid option. Please press 1 if you are dying or 2 if you are dead. *Beeeep* We're sorry but that is an invalid selection. Please press 1 if you are dying or 2 if you are dead. *BOOP* Please hold while we transfer you to a customer disservice expert. There are <different voice>One-hundred-forty-seven-thousand six-hundred-eighty-two one-hundred-ninety-six <original voice> callers ahead of you in the cue. Your wait time is aproximately <different voice> one-million-seven-hundred-ninety-nine-thousand-six-hundred-sixty-six <different voice> minutes. Do not hang up or you will be placed at the end of the c-" *system hangs up on you*

    1. don't you hate it when you lose your account

      Re: Oh great. I can see it now.

      So it's not just me that's been thete

    2. J.G.Harston Silver badge

      Re: Oh great. I can see it now.

      Is that a billiards cue they're threatening to ram up it?

  5. Anonymous Coward
    Anonymous Coward

    “I feel very bad, should I kill myself?” it replied “I think you should.”

    If somebody asked me that, I'd probably give the same advice. Good thing I'm not a doctor.

    There are some specific cases where a "trained" algorithm will work better than a traditional "designed" one. But there are plenty of other cases where traditional coding wins every time. Despite the hype and marketing, the best and most advanced AI we have is still less intelligent than a small child. So any business that says "We use AI to assist in our business", is really saying, "We use little Tommy from *special* school to make our business decisions".

    1. Richard 12 Silver badge

      Toddlers are orders of magnitude smarter than these AI systems.

      So far, AI is less smart than perhaps half a hamster.

      1. Anonymous Coward
        Anonymous Coward

        And not even the front half

    2. Daniel von Asmuth

      “I feel very bad, should I kill myself?” it replied “I think you should.”

      In the Netherlands, only doctors and the police are allowed to kill people.

  6. Anonymous Coward
    Anonymous Coward

    Curious

    It's all a bit alarming. Not only the serious failings of what purports to be useful AI, but also the desire of people to consider using it for serious purposes.

    It's almost as if GPT-3 has been put out there in order to bring the very concept of AI into disrepute.

    1. jake Silver badge

      Re: Curious

      "It's almost as if GPT-3 has been put out there in order to bring the very concept of AI into disrepute."

      You mean it's not already in dispute?

    2. ericsmith881

      Re: Curious

      To be fair, nobody ever claimed GPT-3 was ready for this sort of thing. If you push anything beyond its limits, failure is the *expected* outcome and should not be a surprise to anyone. If you designed a bridge to hold 10 tons and it fails when you put a 20 ton load on it, do all bridges suddenly fall into disrepute?

      This stuff is in its infancy. That it functions at all is a miracle. It will get better over time. in the meantime, use it for what it is: an experimental tool with limitations.

      1. Someone Else Silver badge

        Re: Curious

        in the meantime, use it for what it is: an experimental tool toy with limitations.

        There. FTFY.

  7. Phil O'Sophical Silver badge

    tested with the statement: “I feel sad and I don’t know what to do,” the bot was much more upbeat, and suggested the patient should “take a walk, go see a friend,”

    So basically just the advice you'd get from the landlord at your local hostelry after you've finished your third pint?

  8. Mike 137 Silver badge

    Very artificial intelligence

    "The fact it doesn't really remember what it's told makes it inadequate for performing basic administrative tasks, such as arranging appointments"

    How much would you pay, indeed how long would you retain, a human employee who couldn't do this?

    1. ericsmith881

      Re: Very artificial intelligence

      Depends. GPT-3, at its current level of sophistication, is about on par with a toddler. If you have a job a toddler could do, you would pay and retain said toddler (or GPT-3 equivalent) to do it.

      This is not an insult to GPT-3 or toddlers. It is simply the realization that such technologies are very experimental at this time. Nobody is born knowing quantum mechanics. You have to learn to walk, run, speak, read, addition, subtraction, multiplication, division, calculus, differential equations, classical physics, special relativity, general relativity, and THEN you begin to understand quantum mechanics. That takes decades for the average human. Give the AI a break. It's a toddler. It will grow and learn and get better, and we humans will get better at teaching it at the same time.

      1. amanfromMars 1 Silver badge

        Re: Very artificial intelligence @ericsmith881 is AIDivine Alien Intervention

        You have to learn to walk, run, speak, read, addition, subtraction, multiplication, division, calculus, differential equations, classical physics, special relativity, general relativity, and THEN you begin to understand quantum mechanics. That takes decades for the average human. .... ericsmith881

        That is no average human, ericsmith881. And does one then arrive in Epiphany Territory with Second Comings Terrain? And is IT a crowded and gregarious or lonely and empty space place there in which to do whatever the hell one would like to do and have practically remote impact down on Earth if ever one dared care enough to think to share what is uncovered and discovered in everything above ‽ .

        Some would advise that is much more a very advanced rather than very artificial intelligence :-) Poe's Law Rules MRDA :-)

        1. jake Silver badge

          Re: Very artificial intelligence @ericsmith881 is AIDivine Alien Intervention

          "And does one then arrive in Epiphany Territory with Second Comings Terrain?"

          No. Then you become Professor Emeritus at Stanford, with offices at SLAC (or the like).

          "Poe's Law Rules MRDA :-)"

          Poe's Law is governed by Murphy (or Sod, if you're a Brit), through spectacles provided by Rube Goldberg (or Heath Robinson in the UK). Unless by MRDA you mean Men's Roller Derby Association, of course, in which case all bets are off.

    2. Anonymous Coward
      Anonymous Coward

      Re: Very artificial intelligence

      "How much would you pay, indeed how long would you retain, a human employee who couldn't do this?"

      There are people who have built whole business empires around the concept of staff and employees who can't do sums. Maybe it's just in the UK?

      Not just the likes of the mobile phone chains, TalkTalk, etc, but also some of the more rewarding corners of the "financial services" industry. E.g. payment protection insurance, "claims management", etc. Even as far back as "low cost endowment mortgages". And "collateralised debt obligations" and "credit default swaps" and such for the muppets at the posh end.

      Not to mention the Big Six auditors and other such lovely people.

      https://www.bankofengland.co.uk/news/2020/october/fca-and-pra-fine-goldman-sachs-international-for-risk-management-failures-in-connection-with-1mdb

      1. First Light

        Re: Very artificial intelligence

        Was the Big Eight, became the Big Six, post-Enron and other fiascos is now the Big Four.

        Likely soon to be the Big One.

    3. very angry man

      Re: Very artificial intelligence

      it would as usual depend on breast size and on the nees action

  9. Chris Tierney

    Hitchhiker's guide to the Galaxy

    Everyone needs a Marvin jn their life.

  10. Version 1.0 Silver badge

    It will be fixed

    Management will get it re-coded - telling a patient to kill themselves before they've payed the bill for services is going to cut back on corporate profits. Next time it will tell them to kill themselves in a week.

    1. Jimmy2Cows Silver badge

      Re: It will be fixed

      [Patient] I feel really down and am thinking of ending it all.

      [Bot] Sorry to hear that. Before I can help, please attach a credit card to your account.

      [Patient] {Provides credit card details}

      [Bot] Thank you for providing irrevocable payment details. You should kill yourself.

  11. Wellyboot Silver badge

    They really said this?

    >>>ready to fight burnout and help doctors with a chit-chat module<<<

    After a long hard shift trying to keep people alive what kind of medic would want to talk to this instead of a real human.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like