Judge slaps down law firm using ChatGPT to justify six-figure trial fee You'd think lawyers – ostensibly a clever group of people – would have figured out by now that relying on ChatGPT to do anything related to their jobs could be a bad idea, but here we are, yet again, with a judge rebuking a law firm for doing just that. The legal eagles at New York-based Cuddy Law tried using OpenAI's chatbot …
I've known several lawyers who were lovely people and provided a critical service. A non-expert who tangles with the law pro se is a fool. And I've worked with plenty of lawyers who as far as I could tell were conscientious, ethical professionals.
The knee-jerk lawyer-bashing that's so popular here just demonstrates the immaturity and intellectual laziness of those posting it.
At the "jump"? Misbegotten from the start, surely, at the very point someone on the legal team even had an inkling to use[1] ChatGPT for this purpose, let alone the point where, having prepared everything, they finally made the jump and submitted the prompt.
[1] would say "misuse" here, but that would imply that there is a sensible way for these people to "use" ChatGPT in the normal day to day[2], and it is difficult to come up with a scenario where anyone in the legal profession should ever hand over responsibility for writing anything to ChatGPT, let alone something that is intended to be examined by the one person who has such power over the case!
[2] but what about simple, repetitive, things, like writing flyers to leave under windscreen wipers, I hear you cry? Well, if the interns aren't doing the simple assignments properly first, they'll not be getting the practice needed to get the meaty stuff right. Though if they are in a firm that advertises by interfering with parked cars...
Wow, there is some serious hate for ChatGPT in the room. While I also enjoyed some schadenfreude while reading the article, and also concur with the judge, I don't think that using ChatGPT is inherently foolish.
Since much has been made about jumping here, I'll go there too: In my opinion, ChatGPT is an excellent starting point for research and ideas, much as Wikipedia was before every single editor ran away. BUT, you need to know when to jump off, take what your learned, and then dig in with more factual sources and with your own scrutiny for the truth.
I do wonder if some of the hate here today is stemming from fear? If you are honest enough to admit it, does anyone care to explain what they are so afraid of?
It's not fear, it's disgust.
The Average Joe in the street has no technical skill and does not know that ChatPGT, Eliza, Perplexity, Bard, Jasper, Chatsonic, etc are all simply statistical models stringing words together and that THERE IS NO artificial intelligence behind "AI"
They see "Artificial Intelligence" and just like "Self Driving Cars" expect it to be what it says on the tin.
We're a lot more technically skilled and can see behind the curtains to understand it's all a pile of shit.
So this is the disgust for the snake oil salesmen that are selling this crap using technobabble and jargon to mislead and confuse people.
We can also see there's going to be a lot of pain and damage before people understand this and AI falls by the wayside as the fad it is.
Edit: As much as I hate Wikipedia, the steaming output from these LLMs are not even in the same category. At least Wikipedia usually starts with facts and doesn't just string words together.
As far as I can make out if you take any single word in the middle of the output the few words either side of it are words which are more likely to be found in those positions in the training material than any other words taken at random. The run of words is very likely to look like it was written by an intelligence because it has been so written many times. But as you get a word or two further away the next lot of adjacent words also emerge a very likely from the training material but not necessarily from the same contexts as your central word and even less likely will the words a little way either side of your central word have come from the same contexts. It might all appear to make sense as a well constructed sequence of English words but not as a coherent assemblage of facts or argument being expressed by those words.
This is an attitude that I've heard from generative "AI" fans over and over again, "Of course you're in denial, you're worried about your job".
It's not hate, it's not fear. It's disgust, but more than that, bone weariness over seeing the same thing happen over and over again in this industry, and having to listen to the same breathless horseshit about a different fad every two to three years because, for some reason, we let people with a shit-ton of money and no compunctions about spouting nonsense run things.
"If you are honest enough to admit it, does anyone care to explain what they are so afraid of?"
I am afraid of yet more *crap* plugging of ChatGPT and its ilk as if it really is 'Artificial Intelligence' and even more afraid of people who deliberately ignore the masses of evidence that shows the 'AI' we are being sold is yet more smoke & mirrors.
ChatGPT and all its copies/siblings etc are just yet another attempt to sell 'clever' pattern matching as intelligence.
The fact that lawyers have tried to use it, with minimal oversight, proves that they think that judges and clients are stupid and will pay for 'No work' at a premium rate.
This AI fad will blow over eventually, probably when some major damage has been done by some other over-confident chancer who thinks thay are cleverer than the rest of the world.
The bigest damage at the moment is the wholesale 'ripping off' of artists works by all the 'Give me a prompt and I will clone the style of someone elses work for free' sites.
[El Reg, I am looking at you, with all the .ai generated artwork you use !!! :( ]
I will try to explain, assuming that you really want to know our views and aren't just trying to troll. First, I need to clear up some problems with what you asked. We aren't saying what we do because we're afraid of GPT, at least I am not. I do not fear it, and I'm not worried about losing my job to it. Call that overconfidence if you like, but that is not a consideration for me.
My problem with GPT and models like it is that it is so frequently wrong that it is bad to use it for situations where accuracy is important. Most of the time, accuracy is very important. I have had similar problems with many things before. For example, I know some people who, when they're feeling a bit sick, will try to use Google to diagnose themselves. Their arm aches because they did more work with it than they usually do, but they're now worried about cancer. That is something I'd rather they not do. It's not that any particular page is lying to them, but that it's not the correct tool for what they want done. GPT is fine if you want something fictional generated, and it may work if you want a virtual conversation, but it is not a good tool for research.
I do not see it as similar to Wikipedia at all. Wikipedia gave you lots of details with the possibility that some of them were wrong or that some important information may be missing. GPT gives you summaries with the chance that everything it's saying is rubbish and no way to get more information. I have tried it numerous times and, whenever I ask for details that aren't commonly found, it either fails to find them when a simple search could turn them up or it makes them up. Someone who relies only on Wikipedia is likely missing something, whereas someone relying only on GPT is probably going to do something catastrophically wrong sooner or later.
I was not trolling and I am grateful for your thoughtful reply. Thank you.
I find ChatGPT helpful. I am frustrated sometimes by the results I get from it, but I am much more often astonished at the thorough and on-point replies I get, even to very short or poorly worded prompts. I often prompt with my own prejudices and cynicism and I am offered a more well rounded perspective. I like it. Just saying.
Mostly people seem pissed about the salesmanship of AI, but that's just salesmanship. It could be a car or vitamins or any number of other items which, in themselves, are useful and desirable. You are welcome to pissed at how people spin lies about technology in order to sell something that isn't real.
But ChatGPT is real. Perfect, no way. But very, very real. It's a game changer and I think being pissy about it and pretending it's all horribly broken and fake is just a foolish kind of sour grapes.
I am much more often astonished at the thorough and on-point replies I get
OK, but given you claim this:
ChatGPT is an excellent starting point for research and ideas
...then you have no idea if the replies are on-point. You cannot possibly. By your own statement, you're using it to start research, so you don't know what's right or wrong, and ChatGPT can lie to you and have absolutely no idea it's doing it. That's because it doesn't know what true or false or right or wrong or anything else means. It's just a clever, very convincing pattern matcher which does a remarkable job at convincing our monkey-brains that it's intelligent just because of the truly mind-boggling amount of data it's using for that stats matching. But there's no intelligence there.
We know this isn't how human intelligence works for many reasons - first and foremost, because humans can't possibly ever read, assimilate and remember the literal billions of documents that ChatGPT uses for training.
But ChatGPT is real. Perfect, no way. But very, very real.
"Real"? Well, I guess, the product does exist, so it is real, yes?! But if you mean useful for starting research - clearly, no. It's just very confidently incorrect and if you don't know the subject yourself, there is no possible way you can have any idea without cross-checking each and every thing it says manually which would take longer than a Google search or Wikipedia from the get-go.
Ironically, that's probably where most people go to cross-check ChatGPT's output, if they bother to do so. Why? Because they know it's more accurate. But don't take my word for it! Here's what OpenAI, the makers of ChatGPT, say about their own product:
https://help.openai.com/en/articles/8313428-does-chatgpt-tell-the-truth
"Does ChatGPT tell the truth?"...It might sound right but be wrong... ...It doesn’t know everything... ...No access to tools like calculators or the internet (mostly)...
When the vendor itself has to cover its ass with disclaimers about the inaccuracy of its product, then - given how reluctant large corporations are to ever admit any kind of flaw whatsoever in their products - you know without a doubt that things must be really bad.
LLMs would be problematic for starting research even if they were far more accurate.
LLMs, in most of the popular use cases (search, "improving" prose style, and so on) are competitive cognitive technologies. They inculcate intellectual laziness, learned helplessness, and shallow learning. In most cases they rob the user of serendipitous discovery of tangential and distantly-related information, reduce recall by reducing the effort and connectivity of the discovery process, and interfere with critical thinking by presenting a fictitiously consistent result with unwarranted rhetorical force.
Intellectual labor isn't valuable only for its direct output. It also serves as a mental paideia, improving the cognitive capabilities of the worker. Delegating the most important aspects of that labor to a machine is inevitably detrimental to the user.
Nick Carr made this argument about web search years ago, but LLMs are orders of magnitude worse because they greatly aggregate all the negative factors. They remove friction, reduce the need to consider the form and content of queries, provide falsely-coherent responses rather than a set of alternatives, and are (partly by accident) optimized to exercise human psychological weaknesses and avoid stimulating critical responses.
Code For Broke,
ChatGPT isn't a search engine. It's a language engine, fed loads of training data that contains information as well as language. Some of that training data was curated, but large chunks of it were scraped from the internet, books and other text sources. That might be encylopeadias, customer service chat logs, history books, novels or even YouTube comments threads... And while the pre-training data was categorised, terrabytes of the later data weren't.
Also, this was apparently done in 2021. Which means bits of it are increasingly going out of date. Science, technology, even history get regularly updated as new information becomes available.
That means that sometimes it will give you summaries of real facts. Sometimes bits from multiple sources. And sometimes it will give you stuff from novels or online discussions. And sometimes it will simply invent stuff. And you don't know which is which.
As a start for research that's terrible. Having a quick look at a Wiki article on a subject you know nothing about, plus a couple of other brief general pieces will give you a starting point. These will give you some dates, some names, and some references. All of which can be followed up to find out more. Then you can start to find the inconsistencies and areas of debate - and this can lead you to know what and where to research.
ChatGPT might also do that. Or it might insert a few made up bits in there. Or just invent the lot. As with the lawyers who asked it to give them case law to support and argument (and save them research time) and it just made the cases up - including the text summaries of what they said. How do you know which is which? At which point you then have to start your research by checking if all or part of the data you started with is garbage. Wheras with Wikipedia you know that controversial stuff may be missed out, or slanted to make a particular argument - but it's less likely that whole events are completely made up. Though that's still possible. This is where finding textbooks can be useful, as they're curated data, and hopefully will discuss the disagreements on a topic and explain why their sides hold those views, so you can see the bear-traps for future research.
ChatGPT is a statistical system designed to group words in a plausible seeming order, so they make sense. This can be used to summarise data sources, which I guess is what rewards good prompt choices - assuming the thing you want summarised is in the dataset. But it doesn't understand what it's summarising and it's designed to make words that often appear together in the dataset, also appear together in the outputs. Even if that means inventing the titles of nobel prize winning entries for economics - including making up the names of the authors.
Because people keep calling ChatGPT 'AI', and the credulous - i.e. most of the non-tech public and the grossly stupid VC firms - just accept it as AI.
LLM's are one of the foundations needed for AI, but they are AI in the same way the concrete slab foundation for a house is to a house. It's not, it's one of the cornerpeiecs, one of the building blocks needed to get to AI, but everyone who wants a slice of the pie is declaring it as AI as that's an easier sell. Thye are looking at some foundations and half-built walls and declaring "we have a house ready to move into today!" despite the lack of compete walls, glazing, roof, ceiling, plumbing, lighting or any electrical wiring at all.
Grifters, the fucking lot of them.
Some human will submit ChatGPT generated code as his own into something I am responsible for that includes a fragment of Oracle copyright code like rangeCheck. The legal bill would be devastating. I know Microsoft has some (probably non-binding) promise to defend people brave enough to use Copilot. Perhaps Microsoft really would step up and provide a defence... generated by their LLM. I am sure it would not be worth the hassle to make them honour a promise.
For every AI horror story we hear about I assume there are ten more where the shit is still falling towards the fan and another hundred that will not get reported.
Any tool can be misused if put in the hands of someone who doesn't know what they're doing with it. If you're an aspiring artisan carpenter, and you fail to understand why you don't want to hit every nail or joint with said hammer as hard as you possibly can, every time... I don't anticipate your career being long-lived or fruitful.
I'm afraid that simpletons will believe that the results that come out of LLMs are accurate, and that it will negatively affect my life and the lives of others.
I'm afraid that simpletons will believe claims about how other AI products work, and that it will negatively affect my lives of others.
I'm afraid that simpletons will believe claims about how other AI products work, and that money and time will be channeled toward the snake oil peddlers, instead of being used for something useful.
It's not hard to use ChatGPT professionally, you treat it's output like a search engine result, you ask for the citations and sources used to construct the argument and then cross-check the information yourself against both the cited sources, and similar.
AI generated content should be the *start* of something that you produce, not the entire product.
"It makes up the citations too, and be honest, you don't bother checking them."
So check them. It's not ChatGPT's fault you're a lazy fucker who can't be bothered to fact-check the output - in that respect it's no different to asking your know-it-all mate down the pub if he knows how xyz works, he gives you a bullshit answer and you roll that into your university thesis. And then flunk your finals because the professor actually does know what he's talking about.
A bad workman always blames his tools; it's your fault for not fact-checking, and you deserve everything you get.
A bad workman blames his tools because he uses crap tools. A good workman does not need to blame his tools because he chose the right ones. GPT is not the right tool, and using it indicates that something is going wrong in the researcher, and if they're not exceedingly careful, the research. Similarly, a researcher who asks someone who has no reason to be trusted as an authority is also doing research wrong, even if they put little chunks into a search engine and find a page that says something that looks similar. They actually have to understand what they're reporting on to do research correctly.
Indeed. A good workman blames his tools when the tools are to blame.
It may not be hard to use ChatGPT professionally, as OP suggested. That doesn't mean it's a good idea.
When searching for legal precedents, for example: A proper Lexis/Nexis or WestLaw search, done with appropriate, thoughtful search terms and refined after considering the results, is likely to have both high precision (few false positives) and high recall (few false negatives, i.e. few instances where a relevant document wasn't found). A ChatGPT search might or might not hallucinate cases, which means yes, citations have to be confirmed — so not that much labor has been avoided anyway — but it might also miss many relevant cases. It'll fail to find ones tangentially related which might on closer examination yield important relevant results.
Starting or not, it's a poor way to do any research.
Certainly there are ways to improve searching large corpora of technical documents. Some years back I consulted a team working on a grant to improve the search process for Cochrane medical meta-studies. Those are typically conducted by a team of academic doctors, and sometimes clinicians, using straightforward keyword searches with Boolean connectives in medical-study databases. We proposed a number of enhancements, but a key aspect was they all featured attributes like transparency and reversibility: users could see the effects of every modification to their query, reverse to any previous point in the query timeline (and advance again), and so on. Proper research requires understanding what functions are optimizing the information entropy. Hiding search in a black box is a terrible, terrible idea.
I've posted to note that ChatGPT can make shit up
Not just refer to last year model, or a particular sub/superset of knowledge. But just make shit up.
My favourite areas are parameters and switches to command line programs, or option/check boxes in GUIs.
Really as someone in IT I should be grateful when a non-techie has to ask why they can't seem to run "C:\FixWindows --recover-lost-data" which is one output I have seen. (When challenged it went "I apologised, you are correct that does not seem to be the correct approach. Here are some more suggestions you might try")
I guess you didn't pay much attention to the news the past few days, but ChatGPT had a bad week. Here's an excerpt from a response to "What is a computer?"
"It does this as the good work of a web of art for the country, a mouse of science, an easy draw of a sad few, and finally, the global house of art, just in one job in the total rest. The development of such an entire real than land of time is the depth of the computer as a complex character. The education of these designed “important to them” computer talents is a great, average, or one-room spot for the word of a single part. The elite of its machine or talk is the book of life, and the shared essence of the self of a family is the white coat of the dove. Stand the computer as the most important culture of success, and it is just an understatement."
And I grant you that has a certain poetry to it, not without its charms, but also doesn't help explain what a computer is.
What sort of prompt massaging came up with that?
ChatGPT v3.5 just said A computer is an electronic device that can receive, store, process, and output data. It typically operates under the control of instructions stored in its memory, and it can perform a variety of tasks depending on its programming and hardware capabilities. which is about as dry and boring as it gets. The poetic gibberish was a much more interesting read.
"Please explain what a computer is in the style of Lovecraft" was better. ;)
Not proven wrong so much as simply unaware of the story until it was pointed out. I don't have time in my life to read everything, you know. There's this work shit, then since I'm an introvert (or maybe some sort of autistic) that's usually followed by some quiet time at home cuddling a mug of tea with my eyes and the shutters closed (and the lights off) to unwind after having to interact with people. Toss in sleep time (well, time allocated to sleeping that sometimes I actually manage to sleep in), there's not that much time left over that everything else is supposed to fit into.
So I missed a story, went to ChatGPT, asked it the question, got a boring answer.
It happens.
Bullshit.
ChatGPT 3.5
User
What is a computer?
ChatGPT
A computer is an electronic device that can receive, store, process, and output data. It operates under the control of instructions stored in its memory, and it can perform a variety of tasks, ranging from simple calculations to complex computations and data manipulation.
> a fairly new technology
New? In comparison to other computer tech?
Neural Nets predate WYSIWYG wordprocessors. By decades.
The only "new" things here is the scope of the hype, the way that has reached pretty much everyone who uses computing devices on a daily, nay, minute-by-minute basis.
Actually, even that is barely true. As noted, NNs are old and people have been building them and offering them to compulsive computer users pretty much the whole time. Without much success, beyond making money before the bubble bursts (to the point of just-inside-the-letter-of-the-law fraud[1].
There have been - still are - places where NNs have done good - have saved lives[2] - so there is certainly no reason for thinking that there is some irrational anti-Net bias: but those genuine successes are not the subject of the massive hype machine.
The change now is that so many, many more of us are now in the minute-by-minute computer users category and the economy of scale means that spending one dollar per user has allowed a (very small number) of players to build some very big 'nets.
But being big doesn't mean the maths and logic behind these things has in any way suddenly changed[3]. All the flaws are still there[4].
[1] e.g. 1980s/90s, as automated trading took hold: take a stock exchange feed, train up as many 'nets as you can get PCs to run them - don't worry, the data rates are low enough and you don't need to waste 'net nodes on pseudo-parsing natural parsing language - then switch them to output mode before trading opens the next day: ta-da, predictions for the market that day. Repeat for, say, a month. Most models' predictions made losses - but this handful won Big Time! With a totally straight face, sell copies of that handful of 'nets (with witnessed guarantees that they did make those predictions!) to anyone with a big old pot of cash.
[2] 'nets used in medical image screening - *but* we just happen(!) to give very high value to the true positives, enough that we are willing to forgive the false positives - and quietly shrug our shoulders over the false negatives, because we are still catching more than we did previously.
[3] there are good reasons why AI research spent time on more than just NNs - and it wasn't just because the hardware wasn't available: Moore coined his law a while ago (and the current LLMs are appearing in a time where we are concerned that said law is losing steam)
[4] big bug bear: inability to explain their reasoning *and* to have that path tweaked to improve the results; instead, next run (irrespective of whether you change the prompt) you get a totally new output and need to go over inch of it again to look for flaws.
The only "new" things here is the scope of the hype, the way that has reached pretty much everyone who uses computing devices on a daily, nay, minute-by-minute basis.
Overstated, I think. Deep Learning is arguably just Moar Scale, but the transformer architecture is a novel approach. No one had published any ANN technique quite like that prior to "Attention is All You Need", AFAIK. Diffusion models are also fairly novel.
I may not be a fan of the results, but the research has been quite interesting. That said, using DL for NLP goes back at least to AlexNet, which is more than 13 years ago, and then the various BERT-related models; and Vaswani et al published "Attention" in 2017. Diffusion models were introduced in 2015, according to Wikipedia (I haven't paid much attention to them, so I'm not really familiar with the research). So not all that new, particularly in the frenetic world of IT.
There are many, many papers demonstrating invalid output from various ChatGPT generations. Clearly you're not following the research at all.
I'd dig up a dozen or so citations just from the ones I've saved, but I'm starting to feel that you're not capable of learning. Frankly, it's hard to take any comment from an LLM proponent seriously if they don't know about, oh, SolidGoldMagikarp, let's say.
What you appear not to be taking into account is that GPT changes a lot. It's not just because the algorithm has a lot of random elements, but that there is a lot of premade prompt material which is added to your prompt before you start, OpenAI changes it frequently, and you can't see what it is. What one prompt produces today may not be generated, even similarly, by the same prompt a day from now.
This "what is a computer" example is a stark demonstration of the effect. It did actually happen. OpenAI verified that it happened. This is not supposition. It's working now because people posted that and other examples where it was clear that someone had flubbed something very important, making a lot of prompts produce gibberish. This works for other cases as well. If I post a prompt which returns completely made up information and you run the same prompt some time later, you may get the same inaccurate information, an I don't know response, completely different inaccurate information, or even correct information. There's really no way to tell. Asking people to post prompts then pointing out that you can get something different suggests that you might not know what is happening when you do so.
Since we were discussing it in a thread above, this inconsistency is one of the reasons why I have a problem with the use of GPT when accuracy is important and one of the reasons I thought you might be trolling. Even getting a useful answer once doesn't guarantee that you can get a useful answer later with the same prompt.
> That is, generally, a good thing.
But not a thing that is related to the point doublelayer was making.
Reacting to the context of the conversation would indeed be a useful feature, if it were based on a solid foundation.
Although, what is the conceptual difference between "reacting to the context of the conversation" and simply re-starting from scratch and feeding in the whole conversation again as the prompt?
In other words, "reacting to the context" is really an amazing feature only insofar as they are spending the resources just to keep your session alive whilst waiting patiently for you to type something, rather than dropping it and letting another process use the expensive hardware.
what is the conceptual difference between "reacting to the context of the conversation" and simply re-starting from scratch and feeding in the whole conversation again as the prompt?
Depends on what concepts you're interrogating, but generally you're correct that there isn't any. This recent piece on LessWrong considers this question.
"there is a lot of premade prompt material which is added to your prompt before you start, OpenAI changes it frequently, and you can't see what it is. "
So it's not your prompt at all, is it? If you aren't even allowed to ask your question in your own terms, how can you rely on the answer, whatever the actual capabilities of the machine?
Because it happened as a temporary malfunction is not the same as what was implied: that the system is routinely fantastically incorrect.
I have already openly admitted that I have experienced ChatGPT be very wrong about very cut and dry factual matters. It isn't perfect.
But it does seem as those most of the hate here is based on fear and 1 in 1m type examples of a bad outcome.
> But it does seem as those most of the hate here is based on fear
Hmm, but you said
> I have already openly admitted that I have experienced ChatGPT be very wrong about very cut and dry factual matters.
So it must be fear, not a desire to have reliability, especially when there are apologists around (hint: mirrors exist) to wave away the unreliability as unimportant.
So, unreliability is unimportant in something that is being literally thrust upon the masses, who have no way of even judging whether what they see came from that source? Let alone, if it did come ChatGPT, in which direction it has decided to be unreliable today.
Relying on the unreliable is a common trait, but at least you *know* that you've just read an Info Wars article.
People can take some levels and forms of unreliability. For example, when they turn on their navigation systems, they know there's a chance it might send them the wrong way, but they also know that most of the time, it is correct, and often if it is not, they can use their eyes to detect it and their physical control of their vehicle to prevent it from causing problems to them, the odd story about a person deciding to drive directly into a lake excepted. If the navigation system made errors like that on a significantly larger set of trips, people would be more cautious about using it, as they are in areas where map data has not been updated. If those errors were not immediately detectable and frequently led to wasted time or risked lives, it would be dangerous for users to rely on it.
GPT's reliability and accuracy is along those lines, since, as you know, it is frequently wrong and it doesn't make that obvious unless you already know it. People who trust it are making a mistake, and people who use it in such a way that others are depending on GPT-derived unreliable data are harming others. I have been clear about this, and others have stated their opinions. Your repeated assertion that we must be saying these things out of fear is beginning to confirm my original assumption that you know what we think, cannot respond to it except by trying to understate the inaccuracy of GPT, and are resorting to making up a different reason and putting it in our mouths. I can only tell you that, if you sincerely think I and many others here are actually fearful of GPT's effect on our jobs, you are profoundly mistaken.
what size samples do *Redacted* send
Triggers the response:
*Redacted* typically send samples with a normal size ranging between 5 and 10 milligrams (mg). However, modern instruments allow for reliable measurements even with smaller sample sizes, such as 1 or 2 mg. While this sample size might be too large for fiber and most paint casework, it can be useful in cases involving materials like drug packaging or adhesive tapes1. So, whether you’re exploring new materials or evaluating existing ones, *Redacted* ensures that their samples provide valuable insights for various applications.
Given that the company ships product by the truck load, are not involved in fiber, paint, drug packaging or adhesive tape, and samples are between 20kg and 200kg this is total bullshit.
"You'd think lawyers – ostensibly a clever group of people"
Having mixed professionally with a good few barristers I'd say their cleverness followed a normal distribution with a fairly large variance. Some were sharp and picked up on what they were being told PDQ but there were one or two whose abilities didn't seem to get much further than putting the gown and wig on the right way round.
Fortunately the judges did seem to get picked from the upper end of the distribution.
I typically hear it as:
Q: What do you call a doctor who graduated lowest in their class?
A: "Doctor"
(To be fair, an awful lot of medical students, and presumably law students, can't make the grade and never graduate. Which is the system doing its job, trying to ensure that those who do get their degree can do the job.)
Even vanilla LLMs have their uses. I watched a talk the other day by Terry Tao, who said that he'd got useful assistance with a proof from ChatGPT - it makes up nonsense, but it suggested a line of attack he hadn't thought of, which worked. When one of the smartest people alive thinks LLMs will be useful, it's probably time to consider more carefully the kneejerk reaction. And when they're combined with things that *can* reason, they could be seriously useful. A combination of LLM and deductive reasoning solves International Maths Olympiad geometry problems better than most competitors.
> it makes up nonsense, but it suggested a line of attack he hadn't thought of, which worked.
These anecdotes, where someone - anyone - has used ChatGPT and "it didn't solve the problem, but I came way with a new idea" make me wonder if they could get the same result by using "Rubber Duck Debugging" and a random choice of new approach to the problem by, say, drawing a card from Brian Eno's Oblique Strategies deck[1]. Cheaper, far fewer resources needed.
Now, if you have a citation, including the transcript, so we can see that it was, indeed, ChatGPT that came up with the idea out of the blue, without either being led by the nose[2] or it being one out of dozens of otherwise useless ideas[3] then we can reevaluate the level of incredulity.
[1] hmm, to make it more Modern and Computery, we could drop a copy of those cards into ELIZA and save on buying rubber ducks; I mean, have you seen how much they charge nowadays for a decent model, like a Shakesduck or an Isambard Kingduck Brunel?
[2] I've had some useful info out of ChatGPT, because, unlike Google, after it has come up with the latest, most repeated/hyped, suggestion, you can tell GPT "no" and it'll dig out something a bit more obscure; did that a few times and it finally spat out a reference to some 1970s tech that turned out to be what I wanted to find.
[3] carry on saying "no, that isn't what I want" and it'll happily apologise and then spit out yet another semi-random selection, throwing stuff on the wall and seeing what will stick. Then a simple bit of confirmation bias[4] will get you to remember - or just retell - the sticky bit and forget to mention the mess piled up on the floor or the bits that were so off-target they flew out the window.
[4] even the smartest people are subject to this; I'd even wager the smart ones are more used to having lots and lots of thoughts about a problem that they are really used to just forgetting about all the ones that never lead anywhere that they may not even notice how many such fly past.
"These anecdotes, where someone - anyone - has used ChatGPT and "it didn't solve the problem, but I came way with a new idea" make me wonder if they could get the same result by using "Rubber Duck Debugging" and a random choice of new approach to the problem by, say, drawing a card from Brian Eno's Oblique Strategies deck[1]. Cheaper, far fewer resources needed."
They quite possibly could have achieved the same result in a different way. This in no way negates the point though; ChatGPT is one of many tools that can provide useful information IF prompted correctly AND if the responses are fact-checked and verified, and/or used as a 'feeder idea' to generate other lines of thought.
Related note: remember the harrumphing from the academic classes when Wikipedia was launched? News sites were replete with examples where 'experts' had reputedly quoted verbatim from the "encyclopedia anyone can edit", and ended up embarrassing themselves by quoting or promoting obviously erroneous information. Point being, Wikipedia is also a tool, useful in it's way but fallible and requiring thought and care when interpreting the 'conclusions' it brings; AI, and ChatGPT, is in this respect the new Wikipedia.
The difference is that Wikipedia is right most of the time, and often when something incorrect is added it gets fixed fairly quickly. (Not always, but often.) Compare to any LLM - sometimes it's right, an awful lot of the time it's talking out of its nonexistent posterior, and you get different results every time. How is this useful?
(Come to think of it, it's like talking to a know-it-all 7-year-old.)
My experience with ChatGPT (and some others) is that it's very 'early Wikipedia'-like. You can get some good answers IF you phrase your prompts correctly, and IF you apply thought and care to interpreting the output. Remember that early Wikipedia wasn't like it is now, it was the Wild Wild West - people creating pages willy nilly, rogue editors... you name it. It really wasn't reliable; and they put a lot of work into making it reliable. Now, it's quite good.
To me, the biggest problem with LLMs and generative AI is that it SOUNDS authoritative, even when it's spouting absolute shite. Which means people get into an authority bias mode, and switch off their logical reasoning centers.
In the same way they do when somebody puts on a white coat and a clipboard and tells you that RadioActivin-A will cure all your ills, despite having zero medical training or accountability.
Or the Finance intern puts together an Excel spreadsheet with an amazing looking frontend that gets senior management all gooey, but makes a rookie error in the backend that makes the conclusions useless.
So can walking across a bridge, and that's far less likely to mislead, exercises the mind and body, and is much less resource-intensive.
ChatGPT's occasional usefulness for people who already use their brains far more than the average sod does not establish that it's a good tool.
So ChatGPT is a tool and we're currently experiencing the (sometimes painful) process of working out exactly what it's good for.
Your interesting Terence Tao instance is a great example of it being used well.
The original article is a great example of it being used poorly, predictably, by someone who, in their professional capacity, clearly ought to have known better. That (to paraphrase) is what the judge said, anyway. Cue adverse consequences, and rightly so.
It seems not enough people who do use it think critically before deciding if it's the right tool for them and their task. Too many seem to swallow the kool-aid that it's AI, when it's nothing of the sort.
"This is Cuddles Pony Fluffy & Partners, how may I help you today?"
"Yes, I'll put you through to Mr Rumpole straight away. Oh, yes, I'm sure you didn't kill all those people. Oh yes, Mr Rumpole is an expert on blood stains. He won the Penge Bungalow murders case alone, and without a leader. Don't you worry about it. He'll soon have you free."
"Mr Rumpole, I've got a murder case on line 1 for you."
Micro$oft has at least a $10 billion investment in ChatGPT:
https://www.forbes.com/sites/qai/2023/01/27/microsoft-confirms-its-10-billion-investment-into-chatgpt-changing-how-microsoft-competes-with-google-apple-and-other-tech-giants/
Copilot is being forced down the throats of Windows 11 users
From:
https://www.microsoft.com/en-us/windows/copilot-ai-features
"Kickstart your Windows 11 experience. Along with other AI-powered features, Copilot in Windows can enhance your creativity and productivity"
It may enhance "creativity" but more than likely "productivity" will go the opposite way...
On the one hand I would have expected the use of tools like this to bring down the bill .. after all you're using a "cheap" computer compared to an "expensive" lawyer to do the leg work ...
But on the other, I see that it acts exactly as you would expect a lawyer to do so and jacks up the bills ...
"As intelligent as a lawyer" is possibly progress, but maybe not what we were hoping for.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
