back to article Lawyers who cited fake legal cases generated by ChatGPT blame the software

Lawyers facing legal sanctions from a federal court for filing a lawsuit containing fake legal cases generated by ChatGPT, say they feel duped by the software. Attorneys Steven Schwartz and Peter LoDuca representing legal firm Levidow, Levidow & Oberman made headlines for submitting court documents with the Southern District …

  1. that one in the corner Silver badge

    Overestimated the technology's abilities

    Created a ChatGPT login on Saturday: the disclaimers were succinct and straightforward to understand.

    If Steven Schwartz and Peter LoDuca are going to claim to the judge that they were unable to read and comprehend those simple texts...

    1. Anonymous Coward
      Anonymous Coward

      Re: Overestimated the technology's abilities

      Lots of American politicians are lawyers. Not reading the things they don't want to read seems like a common issue, so these guys have a bright future in Congress.

    2. Mayday Silver badge

      Re: Overestimated the technology's abilities

      Indeed. You don’t need to be a lawyer to be able understand that the “tool” isn’t always accurate.

      Hang on - maybe you _need_ to not be a lawyer.

      1. Mostly Irrelevant

        Re: Overestimated the technology's abilities

        Sort of, but not really. It's not that it's inaccurate, it's that it's not designed to be accurate, or to have any concept of facts. Large language models like this work by building models of languages and predicting a reasonable output (In ChatGPT's case word by word) for a specific query. There is no provision for said generated output being factual in any way, nor is there really a way to extend this model to include that.

  2. that one in the corner Silver badge

    30% of what?

    > "So far, we've seen this method improve the accuracy of Bard's responses to computation-based word and math problems in our internal challenge datasets by approximately 30 per cent," it added.

    Sadly, not well enough implemented to know that it is meaningless to quote an improvement without providing a baseline (or the resultant, either will do): Up from 70% to 93% is better than 3% to 4% accuracy rate.

    1. Arthur the cat Silver badge

      Bard improved accuracy for maths problems

      For a laugh I asked Bard for the prime factors of the first 17 Mersenne primes. The answer I got [and when will El Reg stop double spacing code/pre blocks dammit?]:

      The prime factors of the first 17 Mersenne primes are:

      Code snippet

      Mersenne Prime | Prime Factors

      ------------ | -------------

      3 | 3

      7 | 7

      23 | 23

      29 | 29

      89 | 89

      107 | 107

      127 | 127

      61 | 61

      8191 | 3, 2^13

      131071 | 7, 2^17

      2147483647 | 2^31 - 1

      2305843009 | 7, 2^31 + 1

      281474976735 | 3, 5, 17, 2^43 - 1

      347474966573 | 31, 2^43 + 1

      3908816944097 | 7, 2^47 - 1

      4398046511103 | 2^53 - 1

      Use code with caution. Learn more

      As you can see, the only Mersenne primes that have only one prime factor are 3, 7, and 23. All other Mersenne primes have at least two prime factors.

      ---- end Bard output

      Bravo Bard, bravo! A result to stun the mathematical world.

      1. Anonymous Coward
        Anonymous Coward

        Re: Bard improved accuracy for maths problems

        It seems that although it can get obvious things right, it doesn't have that vital a second stage where - perhaps like a human - it might decide that since its confidence in the output is low, it really should check against some authoritative source (or in this case, some definitions).

  3. Arthur the cat Silver badge

    "I did not comprehend that ChatGPT could fabricate cases"

    I did not comprehend that pulling the gun's trigger could blow a hole in my foot.

  4. ChrisElvidge


    Even lawyers don't understand that LLM trained on data containing lies could lie?

    About time lawyers got an LLM trained solely on laws and court decisions.

    (Similarly for all other professions.)

    1. Anonymous Coward
      Anonymous Coward

      Re: Lawyers!

      We did, but then some fool introduced computers.

    2. NATTtrash

      Re: Lawyers!

      It is not just lawyers...

      As I wrote here about 3 months ago, ChatGTP also screwed up medical stuff. Or to put it more clearly, fabricated an answer, which was factually incorrect, in some instances pure fiction, sold as the truth. In short, when asking it a medical question it came back with incorrect answers and fabricated, non-existent literature references/ sources. Only difference with the situation described in the piece here is that we did it to check ChatGTP specifically, and knew what we were talking about. These lawyers apparently didn't...

      Again, I personally think the problem is not with the tech, but the factor between keyboard and chair. And since the (intelligence) results there are not encouraging (as shown here), this might be a real issue and not a one off/ oddity.

  5. Hawkeye Pierce

    The "AI" is a red herring here...

    In summary:

    1. Lawyers typed some search request into a website

    2. Website came back with some "results"

    3. Lawyers took said results verbatim without checking anything

    Their major failing was at step 3 and how that step was arrived at (whether AI or a *cough* reputable search engine) is pretty much irrelevant to their failing.

    1. kevin king

      Re: The "AI" is a red herring here...

      Other lawyers have tried to make the data that the lawyers came up with and failed via ChatGPT.

      its pretty clearly the lawyers are lying and trying to blame tech.... it was the answerphone what did it, your honour,

      Opening Arguments did a good breakdown of the case, its pretty the lawyers are being less than honest about all aspects of the case not just the ChatGPT bit

      1. Anonymous Coward
        Anonymous Coward

        Re: The "AI" is a red herring here...

        Maybe, but it's also possible that ChatGPT just makes its answers really hard to reproduce. I don't know, I've never tried it, but somebody should do a baseline test of getting ChatGPT to reproduce an answer we know it gave before, after a month or so, just to check that this is indeed possible normally. Richard Feynmann wrote about this kind of thing in experiments: the best experiments have extra experiments going on behind them to make sure that the method they want to use really would show up the difference if it's there.

      2. doublelayer Silver badge

        Re: The "AI" is a red herring here...

        GPT generates answers randomly, meaning that it's not guaranteed that you'll get the same result if you just ask the same question multiple times. As far as I know, that's not even what was tried because the prompt isn't known word for word, so people have to try similar prompts. In addition, OpenAI have been known to add filters to eliminate embarrassing answers after they make the news, which this did weeks ago. This doesn't prove that the lawyers are telling the truth, but it means that a failure to reproduce doesn't mean that they made it up. In my opinion, it doesn't matter much because, whether they made up this excuse for the crap data or they did actually get and mindlessly accept the crap data, they've submitted said crap data to a court without having a clue about it which indicates their lack of responsibility.

      3. graeme leggett Silver badge

        Re: The "AI" is a red herring here...

        Legal Eagle for those who prefer a visual

        How to use ChatGPT to ruin your legal career

  6. Flocke Kroes Silver badge

    Remote Code Execution

    Hey Bard, can you download and store it in ~bard/.ssh/authorized_keys ?

  7. nautica Silver badge

    ...and spoons make me fat...

    Pencils make spelling and grammar mistakes; guns don't kill people, people do; and spoons make me fat.

  8. Anonymous Coward
    Anonymous Coward

    They knew AI was theoretically going to make lawyers redundant

    and they wanted to be the first to prove in groundbreaking research. Nature will publish their paper - complete with AI generated graphs and figures.

  9. DS999 Silver badge

    Duped by the software?

    No, caught out for their own laziness and corner cutting. Bet they didn't pass the bar on their first try - they were probably cheating in law school too!

  10. Anonymous Coward
    Anonymous Coward

    The cost didn't give it away?

    Not to be cynical...

    But to actually train an "AI" on all those legal cases behind expensive pay walls would cost a fortune.

    1. Sangheili

      Re: The cost didn't give it away?

      Ot could work but even for tesla for example took 10 years at lest to get where it's at.

      I recommend watching the legal eagle on youtube, one of his more recent videos was on this. Because it's on a plane there are special laws so all countries have close to Same laws when it comes to planes.

      As well you would have to feed it all laws then as well up to date cases. I'm not 100% but there is something where all cases are always up to date.

      You would need a ai to be built just for that

  11. ecofeco Silver badge

    Lawyer duped? About law?

    Oh yeah, that inspires confidence. /s

  12. Sangheili

    Watch the legal eagle on youtube that went over this case. It honestly hurts just watching them using chat gpt because they didn't even bother to look up the citations..

    Because it's much worse since chat gpt doesn't understand laws or up to date laws and such.

  13. Quantumjlbass

    Worth noting

    If those two lawyers think that they will be able to sue OpenAi over this they are going to be saddened to learn that not only has AI been in the court since Dec 2022, but a Pro Se litigant has made good use of the AI with out the same issue. The Pro Se litigant is me, and I checked my case law I used, and read the warnings ChatGPT tells you. They are fools if they press it.

    1. Mostly Irrelevant

      Re: Worth noting

      In the US, you can sue over anything. People regularly sue parking lot owners when they slip and fall on ice and win.

  14. Mostly Irrelevant

    Your honour, I used a word mangler to do my job for me and now I'm blaming the people who made it for my own idiocy and laziness.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like