AI chatbots waffle on GOV.UK queries, then get facts wrong when told to zip it Artificial intelligence chatbots can be too chatty when answering questions on government services, swamping accurate information and making mistakes if told to be more concise, according to research. The Open Data Institute (ODI) tested 11 large language models (LLMs) on more than 22,000 questions, comparing their responses …
It can really turbocharge dealing with benefit applications.
I am guessing this is almost exactly what was not intended.
Is there a sweepstakes yet on the year (or month at this rate) when there will have to be "state approved AI". as the rush to recreate religious factions I can see holy wars between the various sects of AI
There was a recent article in the Guardian about the results of the cartoonist Martin Rowson (who has quite a large on line presence) asking an AI who his wife is (who does not have much of an on line presence). The suggested options are quite amusing (and all incorrect).
I asked AI to name my wife. To the hopelessly incorrect people it cited, my deepest apologies
One of the most disturbing recent interactions I had with an LLM was with the Gemini (full model) system last month. I passed it a web page and started to ask questions about its content. The first warning flag was that Gemini spent no detectable time downloading and processing the page (it was quite a long page that can load slowly). After that it proceeded to make up quotes from the page and criticise the content. But checking the quotes revealed none of them were actually in the page, when asked about this Gemini apologized and proceeded to make up more quotes. When further pressed on this Gemini started to blame the authors of the page (that it hadn't read) for being wishy washy and informed me that in coming to Gemini I was going to get the truth whether I liked it or not. Completely clearing Gemini's discussion history returned behavior to normal so I now run with history disabled. I wonder if these questions were asked serially in a single session or over multiple sessions with inter-session history disabled?
If I asked a DWP droid about UC I would both expect that person to give accurate information and be legally liable for the accuracy of that information. Where does the responsibility for data accuracy lie with this new "AI" system?
In the fantasy novel "The Court of the Air" one of the functions of the Court appeared to be retrieving individuals caught on the chunks of the planet which have been projected into space by some geological process.
One reviewer wrote "The Land of Jackals is a warped echo of Britain during the early 19th century"
We might write: "Britain in the early 21st century is a warped echo of its past."
Surprising what you read picking up discarded books from the kerbside. Karst Geomorphology anyone? Actually not too dry. ;)
"Smaller, cheaper-to-run LLMs can deliver comparable results to large closed source ones such as OpenAI's ChatGPT 4.1, the ODI said."
To my mind, the quoted statement ranks almost as self-evident.
Pursuit of 'all singing, and all dancing' LLMs may, in part, be attributed to an assumption that the more an 'AI' knows the closer it becomes to a 'general intelligence'. As of now, that assumption appears ill-founded.
Apparently, general purpose commercial 'AIs' are trained using whatever digital 'content' is to hand. 'Discrimination' appears anathema to 'AI' technicians. Hence, the phrase 'slop-in gives slop-out' is relevant when 'AI' training uses the content of Twitter (and similar) alongside the best texts Anna's Archive can offer.
Thus, it would be sensible for organisations requiring databases for interrogation by employees and/or clients to commission bespoke 'AIs'. Learned professions should consider commissioning and updating specialised 'AIs'; that doesn't preclude tapping into general purpose models. Also, it shouldn't be assumed that highly specific models require immense electricity-hungry computers for their training: hardware and software technologies advance apace. At present, relatively small resources are needed for fine-tuning and for 'distilling' enormous models to make them both adept at particular tasks and containable on modest equipment.
I don't think that is even particularly true of human intelligence. Part of intelligence is being able to quickly discard irrelevant information/knowledge and to focus its analytical processes on the remainder.
Sherlock Holmes was famously unaware of the Heliocentric arrangement of the Solar system but as he pointed out to Watson that it was irrelevant to his work whether the Earth went round the Sun, or the Sun the Earth.
"General Intelligence" even in the perverse sense the slop merchants intend must ultimately be those focused analytical processes that characterises human intelligence and not the purely representational manipulations of encoded information that amounts to current AI.
"Sherlock Holmes was famously unaware of the Heliocentric arrangement of the Solar system"
I strongly suspect that, given the character of Holmes as portrayed, he did know but didn't care, and this was actually heavy sarcasm aimed at Watson's predilection for concentrating on irrelevancies.
"They found that models often waffled, burying the facts or going beyond authoritative government information"
Sounds just like a civil servant then. Or indeed any worker in the public sector.
So we could replace them all with AI and no-one would notice the difference?
"Verbosity is known behavior of LLMs – they are prone to 'word salad' responses that make them harder to use and decrease their reliability,"
Of course! They're language models, not concept models. The LLM has absolutely no understanding of anything - it's merely a statistically driven blind token stream generator. I've given up trying to fathom why this really basic fact has not sunk in. Maybe it's down to the hype, as in the case of the tulip mania, the South Sea bubble and the convulsionnaires of Saint-Médard? Or maybe we've just given up using our own brains because it's so much easier to query an LLM than actually think.
Hey! There's something AI is good for - second rate haiku for lazy English speakers! After a few tries, Perplexity offered me this haiku, to cover how crap AI is (well, I didn't use those exact words):
Whispering code lies
Empty gods of silicon
We kneel, jobs undone
That's not half bad really? I do wonder how many billion have been thrown on the bonfire for this low achievement.
Well they would presumably expect to find 'authoritative information' from a government website or app - where these chatbots are.
They are happy enough to pay for dodgy US planes that are not compatible with our missiles, but can't be arsed to train and pay people to help UK citizens use their own government services.
I continue to be fascinated at the ongoing LLM tradeoff of scale vs. accuracy.
Yes, LLMs can enable far greater access and manipulation of data but at a direct cost of double digit percent inaccuracy.
GIGO is replaced by GOAATAS : Garbage Out, At Any Time, At Scale
Is this corporate enshittification run rampant? A modern societal weakness for quantity over quality, or ease of use over reliability?
Compute has been taken over by grading on a curve.
