I see no way
That this will end well.
NHS England is handing Microsoft Copilot to more than half a million staff after a pilot claimed the AI assistant could claw back 43 minutes a day from administrative work. On Monday, NHS England announced plans to roll out Copilot to 505,000 clinicians and support staff. Its confidence comes from a pilot involving 30,000 staff …
I recall a thread on here about Google Summariser and an example - besides the summary being longer than the original, a comment from a someone in the medical world indicated that the summarised version was dangerous in various area's due to what it had done.
Worried that coPilot will replicate that on someones health record and the consequences will be a little worse than a comment
NHS England has announced that it will hand Microsoft 365 Copilot to 505,000 staff, on the strength of a pilot that "found" each user saved 43 minutes a day. That figure is self-reported, gathered under trial conditions, and will not survive contact with half a million live workflows. It is the kind of number that exists to be quoted, not tested.
Set it aside, because the more revealing admission is structural. The tasks Copilot is being bought to absorb are discharge paperwork, bed management, rota planning, board papers, briefings, FOI handling, complaints, procurement and finance. None of that is frontier work. It is the basic operational competence of a back office. When an organisation outsources its own paperwork to a vendor's language model, it is conceding that the people employed to design and run those processes have not managed to. Administrative dysfunction is being procured around rather than fixed.
Now consider where the data goes. Microsoft is a US company, and US companies fall under the CLOUD Act, which compels them to produce data on demand from American authorities regardless of which country it physically sits in. Microsoft has already admitted, in writing to Scottish policing bodies, that it cannot guarantee data sovereignty for M365. So some of the most sensitive personal information that exists, the medical records of the population, is being routed through infrastructure a foreign government can reach by court order, with no obligation to tell the patient or the UK.
It gets worse when you look at who is being paid. Microsoft books returns of three to four per cent in the UK while channelling the overwhelming majority of its foreign income through Ireland, Puerto Rico and Bermuda, structures analysts have described as starving the public sector of revenue. The British taxpayer is therefore funding, at a scale that runs well into nine figures a year, a contractor that goes to considerable lengths to avoid paying tax into the very services it is billing. Public money out, very little back.
And once again a public body has reached for a US giant rather than build or buy domestic capability. Every contract of this size is a quiet vote of no confidence in British industry, and in the government's own sovereign-AI rhetoric, which evaporates the moment a procurement form lands in front of someone.
Then there is the product itself. Copilot is not best in class. It hallucinates, it stalls mid-task, and Microsoft has been publicly scaling back its consumer ambitions for it. Drop that into a clinical environment, where a confident fabrication in a discharge summary, a handover note or a medication record becomes a vector for harm. Avoidable injury and death in healthcare are overwhelmingly driven by failures of communication and information. Introducing a system that invents plausible text into exactly those documents is a remarkable thing to call a safety improvement.
Microsoft's own terms of use, until they went viral, described Copilot as being "for entertainment purposes only" and told users not to rely on it for important advice. The company rebranded that as "legacy language" only after it was noticed. The vendor's own lawyers would not stand behind the product. NHS England is staking patient care on it.
"So some of the most sensitive personal information that exists, the medical records of the population, is being routed through infrastructure a foreign government can reach by court order, with no obligation to tell the patient or the UK."
EMIS is one of the main providers (2nd largest?) of electronic medical records systems for English GP Practices. EMIS originally used to host that data in its own UK datacentres but several years ago moved to AWS instead. So already for some time the (GP) medical records of a large percentage of the English population havebeen "routed through infrastructure a foreign government can reach by court order".
"it is conceding that the people employed to design and run those processes have not managed to. Administrative dysfunction is being procured around rather than fixed."
For me that's the stand out key point.
Some £, time and decent people invested in some competant systems that (a) eliminate admin (b) do some of the admin or (c) help make the admin much quicker will produce a much greater ROI in time and £ than is being talked about here
But that's boring, not at all sexy, and those in charge often are unable to see the wood for the trees.
Time for some extremely strong leadership in the real of boring admin!
Bed management.
That's an optimisation problem, right? Which is a numerically hard problem to solve well never mind perfectly.
With LLM's being staggeringly bad (even relative to their usual level of performance) at anything numerate how could you have any expectation of an improved outcome?
So far as the rest, ML models need good patterns as input in order to produce good output. If the input is nothing but dysfunction then what do you suppose you get out?
Bed usage optimisation is now critical because of underfunding over decades
A properly funded and managed system would have a capacity cushion built in to cope with peaks and troughs. Instead they deem it a failure if bed occupancy isn't at least 99.93% 24/7/365 - which is why we have waiting lists.
When I was in hospital a couple of years back, there was an elderly gent on the ward waiting for a transfer to another hospital for a replacement heart valve. He was in generally good nick, spent the days reading and walking the corridors. They kept giving him a date, then cancelling due to an emergency coming in to the other hospital - or lack of transport for the 100 mile journey. He was taking up a bed in our cardiac unit for nearly three weeks before he finally went.
> after a pilot claimed the AI assistant could claw back 43 minutes a day from administrative work.
Well yes, if you don't care if the output is correct at all you can save a lot of time. Thank heavens the NHS does not!
Which is further made obvious by their picking worst in class AI Copilot* for this.
* Okay, it shares that throne with Grok
IIRC copilot interactions are used as training data. If you can guess a piece of someones records copilot may well tell you what comes next / before.
Agentic AI handling complaints: "You have the wrong bank details for my salary", "You can settle out of court for £5,000", ...
NHSE's 'technology' (read: bullshit) spending is totally out of control.
They are rapidly approaching a point that there will be a 14-day wait to get an AI response from the NHS App, and no staff will be working in the NHS because they'll all have been made redundant, or left for the private sector in disgust.
NHSE's absolute contempt for its staff, coupled with its total capture by Big Tech, has never been more apparent.