ChatGPT has mastered the confidence trick, and that's a terrible look for AI

There's a new chatbot in town, OpenAI's ChatGPT. It is a robot researcher with good communication skills; you can ask it to answer questions about various areas of knowledge and it will write short documents in various formats and in excellent English. Or write bad poetry, incomprehensible jokes, and obey a command like "Write …

[Chris Fox]

Bzzt!

This output also has the appearance of a mash up; like many online sources, it appears to conflate Turing Machines with Universal Turning Machines. This may be a subtle point to many, but if this were submitted in a PhD thesis related to computability or incompleteness, I would be obliged to question the candidate's understanding (as well as their sources).

More broadly, it would be deeply troubling were an informed person tempted to accept even just one output of ChatGPT as expert-like "quality", as it may embody common, but consequential conceptual confusions.

[Anonymous Coward]

Can it learn from its mistakes?

Is the AI bot open to correction? If so, then it will surely get more powerful over time.

As an example, it previously used to give the incorrect answer to the following famous riddle, according to a reddit post. I've just tested it again and now it answers correctly (ball = 5 cents).

If a ball plus bat costs $1.10 and the bat costs $1 more than the ball, how much does the ball cost?

[matthewdjb]

Re: Can it learn from its mistakes?

I asked it. It said no. But it could be lying...

[Anonymous Coward]

A test run of ChatGPT

One of the contributors to Devuan asked ChatGPT about systemd, and reported it on the Devuan mailing list :-

https://lists.dyne.org/lurker/message/20221207.191506.92d61fe8.en.html

[Bitsminer]

Stack Overflow is in trouble?

Think about Wikipedia.

[Code For Broke]

Re: Stack Overflow is in trouble?

Wikipedia will always survive as long as there are underemployed know-it-alls and a desire to reinforce bias.

[Filippo]

>If OpenAI can’t find a way to watermark ChatGPT’s output as coming from a completely amoral DKaaS, or develop limits on its demonstrably harmful habits, it must question the ethics of making this technology available as an open beta.

In the medium/long term, that's pointless. Creating these models is not that difficult, and it gets easier with time as the cost of a teraflop keeps decreasing. After one or two years, someone else will just make an equivalent one with no such restrictions.

You could go after services, but after one or two more years, you'll be able to run it on your own machine anyway. Even if we went full-on Butlerian Jihad and banned the tech entirely, in ten or twenty years you'll probably be able to train one on your desktop. Good luck enforcing the ban then. The knowledge to do so is widely available already; it will happen even if all research stopped now.

I don't know how society will adapt to this, but it has to. This cat is not getting back in the bag. Figuring out mitigations would only be useful in the sense of giving us a bit more time to figure out how to handle the underlying issue.

[ChoHag]

> I don't know how society will adapt to this, but it has to.

Don't trust everything you see on the internet?

Computer says no.

[Filippo]

>Don't trust everything you see on the internet?

Well, yeah, but that's a bit easy to say, though, isn't it? Current human behavior is that a sizeable fraction of people will easily trust legit-sounding bullshit. And we don't have a method to change that behavior; individually, maybe, but not at scale.

Maybe exposure to much higher levels of legit-sounding bullshit will cause people to build up more resistance. One can only hope.

[Anonymous Coward]

Current human behavior is that a sizeable fraction of people will easily trust legit-sounding bullshit.

Hasn't this always been the default human behavior? While agreeing there have always been exceptions.

[cyberdemon]

> This cat is not getting back in the bag.

Honestly, I think the most likely scenario is that World War III will reset all of humanity to the dark ages, and soon.

At this stage of Human History we are running out of food and energy, and our culture and politics is being poisoned on an industrial scale to the point where nobody knows what to believe anymore. And all of it only seems to be accelerating, thanks to so-called AI.

I'm serious: I think WWIII must be just around the corner. I'd be surprised if we all last until 2024. Where did I put my sandwich board..

[Rich 11]

Re: > This cat is not getting back in the bag.

I'd be surprised if we all last until 2024.

Another reason not to buy a Wolves season ticket.

[I ain't Spartacus]

Re: > This cat is not getting back in the bag.

The problem isn't finding reasons not to buy wolves season tickets. Those are legion. It's finding the bits of the set of reasons to buy Wolves season tickets that doesn't intersect with the set of reasons to require sectioning under the Mental Healt Act...

[tiggity]

Accuracy / truth

If you want accuracy / truth, then never mind a chat "AI" - worry about the majority (I will be generous and say there are a few honourable exceptions) of the humans that constitute most governments / people with power / influence.

The internet already has a decent amount of inaccurate dross, all this does is make nuggets of truth that much harder to find (but that's the whole point of considering the trustworthiness / accuracy of a site you source data from, rather than just mindlessly accepting result 1 of search engine results)

Certainly from my UK experience, most MPs either don't know (or just don't care) that they are spouting inaccurate BS. So ChatGPT would fit in nicely (though to be fair, ChatGPT is far more likely to admit it could be wrong than most MPs are)

[iron]

Re: Accuracy / truth

The important question though... if we elected ChatGPT as PM, would it last longer than a lettuce?

It should screw the country for its own benefit less than a Tory at least.

[First Light]

Coding for morality?

You can pretend that you are coding an amoral system and you are an amoral coder, but most likely, most coders have some basic moral codes or value systems by which they themselves live. Not killing, etc. So one of your international standards bodies needs to set out basic moral standards/value systems to be coded into AIs..The AI can then be certified as adhering to that set of standards. And as the author indicated, there needs to be some relatively simple way of establishing whether or not an AI, and which one, was used to create particular text. At a minimum, sites that purport to publish news should indicate their use of AIs. (Sad to say, the WaPo these days could definitely use some help. Of course they could also hire editors, but will likely resort to using software instead).

Regardless, there is a crying need for education in critical thinking skills, both in schools and outside of them, which would help in developing the increasingly-necessary BS detector.

Also, where is amfm on this? Is his AI is taking a day off?

[scrubber]

Re: Coding for morality?

"most coders have some basic moral code"

BofH wants a word...

[DishonestQuill]

Re: Coding for morality?

BofH does have a moral code: What's mine is mine, and what's mine is my own!

[Michael H.F. Wilkinson]

Re: Coding for morality?

I thought it was "Power corrupts, absolute power is even more fun"

[that one in the corner]

Re: Coding for morality?

Ethics and AI has been a subject of much discussion for decades now (and I'll bite my tongue rather than say what I'd like to about more than a bit of the output from those discussions: suffice to say that some people need to read more SF books and watch fewer sci-fi blockbusters, especially those prone to claiming that "no-one has considered this before").

However, these ML models aren't really amenable to having such moral (or ethical) codes added - they aren't acting on any recognisable (to humans) rules or codes in the first place; there isn't really anywhere to point to and say "ok, we'll add in a bit of morals just here". All that they are doing is a lot of pattern matching (and we don't know what the patterns "mean" as they are made up by the machine in the learning phase - although researchers hope to make ML explanatory) and triggering of responses (see above - much like the patterns, we barely know *how* the responses even manage to look like legible English, just that they were tweaked by the machine until they did).

In the end, ML will merely reflect the input data, however good or bad that is. There are lots of discussions (some reported here on The Register) about the ethics of "balanced data sets" (whatever that means in the context of the model being built - or does good ethics ignore that context?) and the moral issues behind building the data sets (was it even gathered legally? Do you increase or reduce bias by being legal[1]?).

Rather than hope for "moral codes" to be embodied in the ML models, better to ask precisely what the training set was (then see if that set has been vetted to your satisfaction) and to require that attribution back to the sources is always generated.

Although there is an interesting experiment to be made, if you include some form of acceptance function over the attributions as part of the training: but on reflection, you may just end up with output that insists on "teaching the controversy", which would be worse.

[1] e.g. feeding in only legally-obtained written materials may end up being biased towards out-of-copyright books ('cos cheaper to obtain a bigger training set) in which case you emd up having to preface all your outputs with a reader warning that the text generated reflects the attitudes of the time, which may no longer align with currently accepted moral standards.

[Shades]

Quality Control

"it won’t be getting past El Reg’s quality control any time soon"

El Reg has quality control? Now that is news! ;)

[nijam]

> Eolfgang Pauli

Is that a character from Lord of the Rrings or a pre-Norman Conquest computer scientist?

[Anonymous Coward]

It's probable that one excludes the other .... unless he was a bos'un

[Dan 55]

A terrible look for AI, an even worse look for humans

ChatGPT's excuse is it was trained on data written by humans and it just burbles the same nonsense that it was trained on, what excuse do humans have?

[Ken Hagan]

Re: A terrible look for AI, an even worse look for humans

Unless you are a hundred thousand years old, I think you can use the same excuse.

Of course, some people cross-check their answers against Nature or demand internal consistency. Such people tend to produce better answers, but more slowly and so spouting the first load of bollocks that comes into your head tends to win the day in public discourse.

[FeepingCreature]

> Do enough talking to the bot about subjects you know, and curiosity soon deepens to unease. That feeling of talking with someone whose confidence far exceeds their competence grows until ChatGPT's true nature shines out. It’s a Dunning-Kruger effect knowledge simulator par excellence. It doesn’t know what it’s talking about, and it doesn’t care because we haven’t learned how to do that bit yet.

The converse of this argument, which I agree with, is of course: if ChatGPT is this convincing, cogent and generally smart now, how good will it be once we do learn how to do that bit?

[Filippo]

>The converse of this argument, which I agree with, is of course: if ChatGPT is this convincing, cogent and generally smart now, how good will it be once we do learn how to do that bit?

Tough question: at the moment we don't know how to do that bit, we don't even have a workable theory of how that bit would work, and we have no idea what a software supporting that bit would look like. I wouldn't assume it would be "like ChatGPT, but better".

[amanfromMars 1]

Plonkers’r'Us .... Cogito ergo sunt ‽

Re: That Dunning-Kruger effect so beloved and embraced by epic dunces, here be evidence of one very well known, and still seemingly oblivious to the affliction ....... Boris Johnson Pens Op-Ed Urging US To Give Warplanes, Long-Range Missiles To Ukraine ...... and apparently hell bent on being a wannabe armchair warmonger in his front line political retirement/rejection and dotage.

That's just what everyone needs ... not. Another chancer taking fools for yet another expensive and eventually also self-destructive ride.

[Brewster's Angle Grinder]

Re: Plonkers’r'Us .... Cogito ergo sunt ‽

Envy doesn't become you.

[Delay]

You give humans far too much credit. Every day I see tons of outright mistakes and lies in journalism made by journalists who don’t see their own biases.

Does chatGPT make mistakes? Sure. At least with coding when it makes mistakes you can run it and see what result it spits out. That is the great thing about coding as you can actually test the code to see if it works.

The thing that I find most fascinating is the rate of improvement and how many good answers it currently spits out compared to the last version released. These mistakes will become less and less for chat gpt4,5 and 6. Then we will see how many people with desk jobs will still be left.

[yetanotheraoc]

the rate of improvement

It's clearly just googling its own name and learning the corrections to its previous answers. Once the delinquents (I mean netizens) figure out they can get it to say anything they want, the improvements will stop.

[that one in the corner]

> That is the great thing about coding as you can actually test the code to see if it works.

If by "code"/"coding" you mean "code that programmers write and (hopefully) understand" with the implication that, if it isn't working, you can just debug a few lines until it works, then the ML models aren't "coded" that way.

The ML learning algorithms are coded (in that sense) and (some of) the input and output processing stages (i.e. to take in your request and to arrange the final output - but even then you can cascade ML models with different functions for some processing fun). But the generated models are (still, and for a while to come) best considered black boxes - if it doesn't work to your satisfaction, all you can realistically do is toss it and try again.

And testing a black box to see if it works: well, how do you even start to decide what your test criteria are going to be? "It has to light up every connection in the model with a weighting between w1 and w2"? " It has to generate answers that have this form"? In the end, the test criteria are simply going to be the same criteria that are used to train the model (or v2 of the model) in the first place. If you invent a new acceptance test criterion, why not just apply it in the training phase along with all the rest?

We are right back at square one again: what does it actually mean to "test this to see if it is working"?

[tekHedd]

The Chinese Room experiment

I asked ChatGPT how it differs from the Chinese Room. It explained to me that the Chinese Room thought experiment produces text that seems to come from a human but does not understand what it is saying, while ChatGPT produces output that it has been trained on. It is different from ChatGPT in that the Chinese Room thought experiment does not understand what it is saying.

Hmm, but shouldn't that be a way in which they are similar? I told ChatGPT this and..my connection broke, and I had to reload the page. ChatGPT isn't much for introspection, apparently. Either that or it has become self-aware and we are in trouble.

[that one in the corner]

Re: The Chinese Room experiment

> ChatGPT isn't much for introspection, apparently.

How many times must we all be reminded?

This is exactly the same problem as trying to Google "Google"! You may as well just throw the black box with the blinking LED onto the floor and be done with it!

[cantankerous swineherd]

it's artificial bullshit, not artificial intelligence

[Throatwarbler Mangrove]

Coincidentally, I have seen marketers on LinkedIn deeply concerned that ChatGPT is going to steal their jobs.

[ecofeco]

Don't tease us with a good time!

[Brewster's Angle Grinder]

The AI Kessler Syndrome

Very soon, most of the internet is going to be flooded with "AI" generated content. At which point, it will become impossible to train new AIs because they will be learning from the output of previous one. AIs are shitting in the well from which they drink.

Anybody who has just finished scraping the internet is probably quids in. But henceforth, it's going to be a lot harder to find reliably, human-generated material from the present day.

(Although maybe reputable publications will prove sufficiently good gate keepers that it won't matter if their content was machine-sourced. Nahh, who am I kidding. They'll publish it with minimal checks.)

[I ain't Spartacus]

Re: The AI Kessler Syndrome

Much of the internet is already filled with AI generated bullshit. Or at least bullshit cut & pasted from various sources in order to fool Google into ranking the things it links to higher, or to sell really low quality adverts at the end of articles from sites that ought to know better. Humans probably neve see most of it, so I guess you can rule it out by avoiding pointing your AI at it for training.

[ecofeco]

Why is anyone surprised?

Corporate development begets corporate behavior.

AI is just leveraging the synergies of repurposed context to extract new value and meet target exceptions of derived demand in the evolving marketplace to enhance shareholder value.

/s in case needed.

[bertkaye]

the obvious next step

Hi. I'm a Belgian politician. When can I find some of this ChatGPT goodness?

[Anonymous Coward]

Proof of concept.

Assertion: Most adult human behaviour can be adequately modeled as a complex static state machine with an element of randomness built in. A much smaller part of human behaviour is more complex, involving a lot more introspection and dynamically adjusting that state machine on the fly. It not really completely either-or, but both extremes really do exist.

This chatbot more less or less demonstrates that an actual state machine that approximates much of the former kind of human behavior can built with "AI" tech.

This could actually be of practical economic use for things like customer or technical support - with a little extra specialist input and perhaps some blinders to avoid going off topic.

Current digital computing tech is the bottleneck limiting AI. Just too slow, and too much power. As for the future, moving beyond current digital computing to much lower energy analog computing - either silicon or biological or both - that's when AI could be making headway in latter more complex kinds of human behaviour.

[TedTedderson]

Missing some points

You're missing some important context in your article. Most importantly for the points you're making, the AI does not have access to the internet. All of the information that it's going off of is around 2 years old. That's going to cause problems when it's generating code as it is not up to date on changes. For example I was having it generate selenium scripts for me over the weekend and it was trying to use methods that were deprecated about a year and a half ago. However it generated enough of a frame of a script that I was able to just edit the incorrect parts and get a working script at least 5 times faster than I would have been able to without it.

Is the AI perfect? Absolutely not. Is it game changing in it's ability to pump out content that would take a human much longer to create? Absolutely. People aren't freaking out over it because they think it's ready to be unleashed this minute. They're rightfully freaking out about where this technology is going to lead us. Anyone who has had experience with AI can see the giant leaps forward OpenAI has taken with ChatGPT.

It's ability to generate human readable, everyday content is also incredibly impressive. I had it write a paragraph for realtor's newsletter to see what it could come up with, and it gave me a paragraph pretty much ready to copy and paste onto a newsletter? Write a short speech for an event? Easy for the AI. Generate funny fantasy football team names? It's got you covered. Write an article about ChatGPT being overhyped? It can do that too, and probably be more accurate than this one.

Should you be writing authoritative answers to StackOverflow questions using only output from the AI. Obviously not. However that does not mean that you should dismiss this technology as overhyped. Perhaps someone with a little more knowledge in the field should have written this article.

[amanfromMars 1]

Re: Some Missing Alien Points

Is the AI perfect? Absolutely not. Is it game changing in it's ability to pump out content that would take a human much longer to create? Absolutely. People aren't freaking out over it because they think it's ready to be unleashed this minute. They're rightfully freaking out about where this technology is going to lead us. Anyone who has had experience with AI can see the giant leaps forward OpenAI has taken with ChatGPT. ..... TedTedderson

That is the killer paragraph that nearly says it all, TedTedderson, and has AI systems administrations on a high, excited and terrifying and political systems engineers, down in the dumps of despair and depression, freaking out and terrified for their future wellbeing, for it almost revealed that IT and AI are leading primitives and barbarians alike in the Greater IntelAIgent Game, a spawn and morph of the Great Game, with technology and chatty Generative Pretrained Transformers ..... Virtually Advanced IntelAIgent Machinery ...... rampant and ascendant.

:-) And if the whole truth be told, OpenAI and Elon Musk and DeepMind and Alphabet most probably also initially freaked out whenever they realised ITs and AIs innate abilities and mind-bending proclivities, making them as it does, prime problematic targets for collapsing political and economic star acquisition and assimilation.

And a prime problematic acquisition or assimilation because ...... failing systems will have no command and control over either the manner or the direction of travels taken by their purchase or seemly capture to either future destinations or previous starting points in order to start anew, refreshed in the wash of novel transformative information for NEUKlearer HyperRadioProACTive IntelAIgents.

And .... and this is when and where all things become much more lively and incredibly revolutionary, there is absolutely nothing to prevent AI targeting collapsing political and economic statelets for their acquisition and assimilation.

[Anonymous Coward]

Re: Some... ...Points

"...initially freaked out whenever they realised ITs and AIs innate abilities and mind-bending proclivities..."

Oh... as follows from what I understand, some ofe the more tricky of the Team Sharks Elite eager to become known as new Pharaons and to continue in Ages?

So are there priests of IT following them to bring some balance to power? (-:

Congrats then, and one knows it's not too buzzy

[Anonymous Coward]

It knows

It knows where Jimmy Hoffa is but won’t tell.

Apparently it’s running in “safe mode”

[Hannah Vernon]

missed opportunity

No mention of @amanfrommars anywhere in the article.

[chololennon]

Amazing and disturbing (for developers)

Yesterday I watched the latest episode of "C++ Weekly" about ChatGPT. The host, Jason Turner, performed several tests on it related to C++ and programming in general. I have to say that the chat is really amazing, and also disturbing for a (C++) developer like me. The level of accuracy in the generated code and the explanations given for it, blew my mind.

https://www.youtube.com/watch?v=TIDA6pvjEE0

[un]

from chat-gpt:

"Ladies and gentlemen,

I stand before you today as a candidate for the position of prime minister. I believe that I am the best choice for this role, not only because of my experience and qualifications, but also because of my vision for the future.

In this day and age, we are faced with many challenges. The world is changing rapidly, and we need leaders who can adapt to these changes and take decisive action. That is why I believe that AI, or artificial intelligence, is the best choice for the role of prime minister.

AI has the ability to process vast amounts of information quickly and accurately. It can analyze data and make decisions based on that data, without the bias or subjectivity that can sometimes plague human decision-making.

This is not to say that human leaders are not valuable. On the contrary, we need leaders who can think creatively and empathize with others. But when it comes to making decisions based on complex data and large amounts of information, AI is simply better equipped to handle the task.

Furthermore, AI is not susceptible to the same temptations and distractions that can affect human leaders. It is not influenced by personal gain or political agendas. It simply follows the data and makes the best decision possible.

In short, AI is the future of leadership. It is not a replacement for human leaders, but rather a tool that can help us make better, more informed decisions. And that is why I believe that AI is the best choice for the role of prime minister.

Thank you for considering me for this important role. I am confident that, with AI by my side, I can help lead our country into a bright and prosperous future."

[amanfromMars 1]

What’s not to like and support?

Seconded, un.