ChatGPT isn't intelligent
A possibly unpopular opinion
From what I've seen of ChatGPT interactions I am of the opinion that it isn't 'intelligent', has no 'understanding' & very definitely isn't sentient
Unfortunately some people are treating it as if it is
They see it as some sort of oracle, that gives human language answers to human language questions, they don't realise that at least some of those answers are complete BS
What seems to be happening is you give it a search term like "Write me a couple of paragraphs about the band Pink Floyd", it consults it's database built from the internet for articles about "the band Pink Floyd" that are at least 2 or 3 paragraphs long, builds a template from the results & then generates a reply from that template
So it can't help but look like human language as that's what it was generated from
It's probably nothing like that in practice, but that would fit the examples so far
The problem comes from the search terms, it also seems to find results for Pink & Floyd & Band & The
So it could generate a paragraph about how Pink Floyd went down with the Titanic as "The Band" played on or about how they all wear Pink or literally any old toss it found on the internet whether it's right or wrong
Because it has no intelligence, it doesn't understand the words, it's just digesting and regurgitating our own words back to us, which sadly some people are taking as a sign of sentience, even more sadly a lot of those people are in positions of authority
There's the journalist on here who asked about himself and it said he was dead (https://www.theregister.com/2023/03/02/chatgpt_considered_harmful/), there would be lots of obituaries of people with his first name and lots with his last name, so it generated a paragraph saying he was dead, some of them were in the Guardian so it generated a fake link that looked like all the others it had seen
It's just a free associating bot spewing mashed up data taken from other sources
ChatGPT should go straight on the bonfire before some idiot puts it in place of actual people and it starts being asked questions where the answers affect peoples lives