NHS major 'cyber incident' forces hospitals to use pen and paper The ongoing cyber security incident affecting a North West England NHS group has forced sites to fall back on pen-and-paper operations. We have reverted to our business continuity processes and are using paper rather than digital in the areas affected The Wirral University Teaching Hospital NHS Trust updated its official …
Healthcare worldwide and the NHS has resolved all the problems we saw worldwide a few years ago - healthcare is so good, it's kept me super healthy and never been infected (icon)!!!
But all the internet communications problems have never been more than occasionally investigated and solved - we need to accept that it's still Cybersecurity Organized Viral Internet Defections risking everyone and everything on the Internet. This is just an opinion about all the risks these days, not a joke.
With respect, no they wouldn't. The issue is simply that, when the IT system(s) stop working, Business Continuity (going back to pen and paper in many cases) slows down the process back to those Matronic levels which means not as many patients can be seen, tested and treated in any given timeframe so the backlog, usually in Critical Care settings, increases and can get to a point where the A&E doors are closed. Cancelling non-urgent care, as bad as it is, protects the Critical Care services from getting so clogged up they need to shut.
No to mention Radiology reporting would be majorly impacted with reporters having to view imaging on the actual medical imaging modality rather than a viewing workstation with no prior imaging available for comparison. This impacts flow throughout a hospital with Ambulances stacking up outside unable to get patients in and bed log jam at the other end.
Not sure how "Matron" is supposed to fix that.
Yes that's the story I hear from an overworked A&E doc.
It was quieter during lockdown - that's the answer! In order to improve the nations health, people must be forcibly locked away and be ill/die at home. Make every home a prison, sorry I mean personal healthcare pod, is the answer. The the stats will look good, it will cost less but we can still gather the same taxes, save the NHS and pay Palantir to give us amazing dashboards and send the data to China (or CIA).
As a Radiologist who would be in this situation, I can assure you this is untenable.
There is the problem of producing a written report and then having a mechanism to retrospectively enter it on hospital systems.
Worse still is some installations have a single screen/instance for the scanner so you are either scanning, or reporting. both in parallel may not be achievable.
At my NHS trust, we have 6 scanners across 3 sites and it is not possible to report at the rate scans are acquired during the day time and impossible "out of hours" with 1 radiologist available.
I am assuming that remote reporting is not possible - as where would the e-reports land?
The only mitigation is massively reducing scans to only those that are life-and-death.
The rules are changing despite management expectations.
http://thecodelesscode.com/case/154 seems apposite
I'd think if the systems are isolated then no , there would be no offsite / remote reporting as the scanners wouldn't be able to send to PACS and the reporting workstations wouldn't have a connection to PACS to view any imaging either - so it's not just scans being done that would be a problem. Anything done the past few weeks wouldn't be available to report.
I'm actually in the middle of re-writting a business continuity plan for Acute Radiology for a hospital board and yes... untenable is one word for it. Requestings coming in from the PMS and getting vetted would be a massive problem as well.
I'm going to point the medical-organisation-vulnerability finger at medical facility execs, managers and IT people who chose to accept equipment with Internet connectivity requirements. In other words, those who chose systems which cannot work properly without Internet connectivity. Whether that failure is because systems which cannot "phone home" throw error-message pop-ups which operators cannot inhibit, or because an essential service required to run the equipment runs on a non-local-to-the-building "cloud", they still are knowable-in-advance failures.
Anyone in medical IT or medical management who believes Internet service is somehow "uninterruptable" and always-safe-to-be-connected-to needs a boot to the head, and their power to influence equipment and service acquisitions removed.
Of course, medical facilities can't buy it if the manufacturers won't sell it, which is why The Finger(TM) also points at medical-device-regulatory-agencies' techs, managers, and execs which did not require medical equipment to fully-operate in non-Internet mode before approving said equipment.
Supplementary features requiring Internet use are okay, but not features whose lack will affect speed and accuracy of patient treatment.
As one of those "IT people" , we choose to ignore the "cloud first" directive from above. Got several systems being replaced at the moment and current have ruled out any options that require an internet conenction to work. "Cloud where appropriate" is how we view it.
>>we choose to ignore the "cloud first" directive from above.
Lucky you, being able to ignore the cloud first directive.
"Cloud fist" (do you see what I did there?) is being mandated here in the sunny principality (OK for education rather than healthcare - though it may apply to that context as well) - with the care, tact, planning and attention to detail usually associated with these projects, that we all know and love.
Ignore is probably the wrong phrasing - but I'm lucky enough that our higher ups in Digital and Security understand that the internet is a single point of failure that is too risky for some systems that need to be available 24/7. So suppliers are free to submit proposals in the cloud, and I'm free to mark these down as unnessesary risk.
Ah yes. Tell me you've worked with public sector procurement without telling me you've never worked with public sector procurement.
When I was doing IT at an NHS org, we had 3 senior people dealing with 13000 users. I personally was replaced with 10 ATOS offshore "experts".
Working in other public sector organisations, I as the SME can spend MONTHS investigating a solution. Talking to vendors, finding out what's best, most value etc etc. THEN when the business case and procurement goes public, everything that I've done can be over ruled by a bunch of people who can't turn a laptop on because some vendor hS lied about the abilities of their product. All in the name of "taxpayer value"
I was at a public sector cybersecurity event & every department representative the said the same thing, they'll choose something to buy & then get over ruled by procurement or finance & end up with something that doesn't do the job.
Add to that...the sheer bullshit from the consultancies who will maximise their profit over getting the correct solution using the cheapest offshore staff they can find.
> The NHS maintains a central list of patients.
There is no single "NHS" entity - there are NHS England, NHS Wales, NHS Scotland, and HSC NI and therefore there is no central list of all patients in the UK.
There may or may not be "central" patients lists for each of the "NHS" organisations in the 4 countries.
I wonder if that's true. If the pen and paper processes had received as much money and attention as digital would they be less efficient? They are more resilient. The main reason every thing is being digitised in healthcare is so Government and Pharma can gather data and we can grab a slice of taxes for IT. It has the helpful side effect of keeping us IT folk occupied.
They could do it the old way and gather up the paper for input a day later. I haven't noticed during my life more efficiency when using the NHS, the reverse actually, certainly less competence in many areas. Bring back the matrons - one saved me a great deal of discomfort and a second unecessary op.
Whenever I see the "Bring back the matrons" call it does make me wonder if people realise that hospitals are much much more that just a bunch of wards.
They are a complex series of interconnected organisations trying to exchange information being supported by an army of sparkies, porters, cooks, cleaners, IT staff, admin staff, radiographers, chemists, radiologists and much much more. The complexity of a Hospital estate is mind boggling at times and I don't know how our IT Security officer stays sane to be honest. (or maybe he isn't....)
A cyber attack has very little direct impact on "Matrons" domain. It does however cripple every other part of the hospital and make their work more difficult, but a Matron wouldn't be able to do anything about it.
Those that do this, to hospitals, schools, and other organisations that are a soft target but provide critical services, are a bunch of handjob artists who deserve to have their tackle removed with a pair of pruning shears, fried in butter and served to them on toast. Defenders must block every hole, the attacker needs to find just one. At the same time, with systems as numerous and complex as those in healthcare and with no money available, it's not possible to establish meaningful contingency options (other than paper an pen).
Now think of all the state actors who've planted their digital "sleepers" in the systems of every one of our critical services, just waiting to press the big red botton ... like the Israelis did with the pagers (albeit they added a gruesome and unnecessary physical payload).
The disabling of pagers could have been done without the explosive part, but the operators could them buy new ones and continue. When active in military operations, it is necessary to disable equipment and operators, or re-supply of equipment enables the foe to continue. You need to remove a) the knowledge how to act; b) the actor; c) the equipment; and d) sometimes the facilities to that place (e.g. remove electrical supply).
The explosions of the pagers required deliberately-installed explosive materials. You cannot cause that kind of damage or injury with a battery. At most, you can create a nasty fire and hope the holder of the device can't throw it away immediately. Most of the time, you can't even do that as the battery safety mechanisms will disable things when they get sparky and can't be deactivated via normal hacking. Normal hacking could possibly have broken some pagers, but they needed to build their own custom pagers and get them into the hands of people they wanted to hurt in order for the explosions to happen.
1) Do all the NHS hospitals/primary care groups/etc. run such disparate systems that lessons cannot be learnt from the first incursion and spread across the whole organisation?
2) Is money or manglement so tight, or staff so overworked, that precautions cannot be put in place?
3) Is the whole NHS computing so far out of date that most of it is not supported?
The first question is a genuine question about learning best practice.
For the second two, I think most people know the answer already :-(
To answer your first question: Yes.
There's a lot of work across local Integrated Care Boards (the new CCGs it seems) to try to do more joined up thinking. However, each organisation (GP, Local Hospital Trust, wider Hospitals Group) are individually funded and run with different levels of Technical Maturity/Technical Debt and different clinical priorities which directs or diverts funding/attention. I can say that it's getting a lot better with, for example, NHS England, paying for Windows E5 licences so everyone can at least get MDE on the desktop and server environment and provide assistance from the NCSOC. Everything else is the wild west.
1) Workflow throughout a hospital is dependant on information from various computerised systems, you unlink those systems and departments are running at 50% speed IF you are lucky.
2) Precautions are in place and plans are made for these situations - this is the mentioned contingency plans.
3) No - it's a cyber attack - you have to isolate systems,
What if time were spent on backup pen and paper processes. I don't think that is ever considered, the answer is more digitisation, what is the question.
I suspect the thought process and balance is wrong. There should always be a tried and tested way to run without computers. "If". WW3 does happen it may be the winner will be the remnants of the society that has least dependence on technology. All the drive from government seems to be towards less resilience in that scenario (or any other cataclysmic scenario) - why? Resilience comes from the ability to operate in a distributed way in small units.
If you have and know a tested manual process I would bet it is not 50% less efficient, probably more like 20%. The fact that the human network has an hour latency may not be an issue. Is everyone able to type faster than write. How much of the typed input needs to arrive at destination sub second? The real problem is everything has been built to expect to use a network. If the x-ray machine came with the ability to print would we panic if the network was down? How often would carrying the image across the hospital change the outcome?
We can run without computers but it would be a disaster, using digital imaging we are able to have reporters in other locations view and remote on scans within minutes.
There are plans for if everything is down and how we would operate but if you think it would be ever close to what gets done currently in any hospital you are just wrong.
The figure of 50% was quoted to me by a department head when I asked them how it would affect them if the booking system was offline, I don't think their figure is accurate though. It would be a lot worse. 20% is correct though, but you'd need to flip your estimate on it's head, they'd be running at 20% efficency.
>> Is everyone able to type faster than write. How much of the typed input needs to arrive at destination sub second?
LOL , that's NOT how it works. Digital dictation is how most reporting gets done with template insertion, we've a few reporters that rattle through chest x-rays at a rate that needs to be seen to be believed - now if they had to WRITE those..... ouch!
Throw in AI triage for cancer pathways and moving to a paper/pen system would result in a lot of people dying
"LOL , that's NOT how it works. Digital dictation is how most reporting gets done with template insertion, we've a few reporters that rattle through chest x-rays at a rate that needs to be seen to be believed - now if they had to WRITE those..... ouch!"
Not to mention that you'd also need an army of "clerks" to enter the data later or face stopping everything for days or weeks while the clinical staff transfer all their paper notes. Actually, double whatever army of clerks is needed since this is mostly critical information that must be accurate. Anyone remember the days o "coding clerks" punching tape from coding sheets than passing the tape and sheet along for the "checker" to retype it all while reading the tape, the keyboard then locking if the checker types a different character to the punch operator? if the data is important, then accuracy is even more important.
This post has been deleted by its author
There have been at least two concerted attempts to bring consistency and compatibility to NHS IT systems. They are among the more-talked-about government IT disasters of this century.
Yes, money and manglement are notoriously tight, and staff are indeed overworked. The NHS has still not recovered from the pandemic, horrifically mismanaged as it was in the UK.
As to your third question - I don't know, but ransomware can hit anyone, even the most up to date systems. Of course it's easier on older ones, but the level of security you would need to eliminate the threat entirely is simply not practical for the NHS environment - it would probably slow down operations almost as much as reverting to pen and paper.
Scotland had a pragmatic plan in the 1990s.
They tested a bunch of systems and their interoperability. They then said these are your choices for Lab/diagnostics/EPR/pacs etc with 2or 3 preferred providers in each class.
Buy these systems - big support/discount. Roll your own, good luck.
Strangely lots came into line while providing the choice for best local fit.
I was there at the time.
<........."The NHS has still not recovered from the pandemic, horrifically mismanaged as it was in the UK.".....
Another misleading sweeping statement. The NHS managed very well indeed compared with other countries. That they had to virtually bring to a halt all other treatments apart from emergencies was not due to mismanagement, but rather due to not having sufficient staff or resources to do everything - in part because at the onset of Covid it was a new and unknown/unpredictable disease, which required over provisioning in healthcare until the extent of its effects and coverage became apparent, and partly (or mainly!) because of politicians of all colours who have failed to provide the resources for the NHS to cope with the ever widening range of ever more expensive treatments of an increasing number of health problems for which new treatments become available. This has been going on since the time the NHS was created, and current politicians show every sign of continuing to 'kick-the-can-down-the-road' :(
Clatterbridge hospital was still using token ring at least a decade after everyone else had moved on to ethernet (I quite like TR but it was obsolete a very long time ago), their IT department was a fucking shambles and utterly chaotic, they didn't know their arse from their elbow.
Twenty five years ago I was the network admin for a secondary school which just got its first Internet connection. Before releasing it to all an' sundry, we implemented things like strict proxy servers and firewalling, to prevent computers just arbitrarily connecting out to the Internet.
We allowed port 80 and 443 access to all domains, unless they were on a content filtering blacklist (initially, and then category-based filtering as time went on). But overall, we prevented access to the net for most things, unless there was a good reason to not. We did MITM SSL inspection too, and pushed our CA certificate to domain-joined workstations (and there was no guest wifi).
Back then (he says, bring his blanket a little closer and the ash from his pipe flaking in his beard) there were less threats and more control. Now it seems like we have more threats and less control: pesky CDNs and AWS/Azure VMs means you can't just block a range, or even a domain name sometimes, as it's shared by something else. Everything is dynamic, and we seem to have reduced our ability to respond appropriately: "just let the machine do whatever it wants" seems to be the default firewall setting.
Why aren't we segmenting our LANs from our WANs properly anymore? Why is it the default to just let the computer, the phones, the IoTs do whatever they want on our networks and down our pipes?
We do that where I am employed, we've moved from a properly segmented LAN to Azure and no local control, we can't lock machines out, there are "ghost"achiness which don't appear in any of ouranagent tools but still have access to our resources, our network is actually just one great big WiFi hotspot and the two gaping big holes in our local admin rights policy I've spotted and demonstrated to our CISO are still wide open.
I'm attending an interview tomorrow for a new job because I'm sick of screaming my years of experience into the void trying to explain to business grads in management how IT works and how users think.
And not just because it has been a mess for years due to Covid, Brexit staff shortages and strikes.
Most hospitals function a little like trench warfare. For long periods of time, nothing much happens. Doctors wait for test results or scans. Patients wait for doctors. It can get full in A&E, but in general, hospitals worked fine with paper (or at least whiteboards) for decades and still will. They should isolate an internal network and airgap it with staff. If a companion net-connected network goes down, they can still operate the internal systems. Hospitals always used to use fax, and having that still working would be helpful. As it is, they will have to use their smartphones for visual stuff and communication.
Each time this happens it is a lesson in the need for better basic security and a viable backup. How many places actually act on that is questionable.
If the NCSC are involved then it is most likely malware and won't be fixed for some time.
Fax has gone. Thankfully.
We are (nearly) paperless - revert to paper is a disaster.
Medical Royal Colleges are clear that internet access is a prerequisite to be able to work in some (most, and definitely my) speciality safely.
The days of departmental libraries of up to date references are long gone.
I research scientific papers daily in my day to day job in a district general hospital - to the benefit of the patients and referrers to diagnostics.
That is all very well, but if the systems are not properly and securely set up, and there is no back-up system to take over if it the front line system fails (for whatever reason) you are going to end up with dead patients as a result of these breaches, which I think rather takes the edge off all the convenience and advantage of having all this interconnectivity.
Having an appoitment for an operation or treatment cancelled because of situations like this one is not likely to viewed as beneficial by a patient with a potentially terminal health problem!!
Ok - so cancellation of non critical treatment is step 1 and is the only realistic option as Hospitals cannot function at the same level if you turn off digital systems.
Someone needing a hip replacement goes back on the waiting list so that critical cases can be dealt with
Plans are in place in every Hospital across the country for what you do for almost every conceivable scenario, these are the boards business continuity plans and they take account for capacity drop off.
Each department has incident cards which detail what staff need to do in the event of A,B or C.
Thinking they can just carry on as normal is just wrong.
It isn't clear to me that the library needs to be on the same network as the patient records.
Even if it is being used on the same desk.
(Indeed, working on two screens is probably more effective than one, particularly if the clinical record is designed to use all the screen.)
I had an outpatient appointment at Manchester Royal Infirmary today. Its IT systems have been off since the early hours of Wednesday (27th), but the failure mode seems to have resulted in a read-only system. My consultant could still access all my previous records and could even request blood tests. But the test request couldn't get through to phlebotomy and they couldn't print labels for the samples.
The Trust runs on Epic.
As a former patient of the hospital concerned, I am concerned about which data have leaked in this case.
APH/WUTH went through a big "modernisation" project, moving from green screens to a new Cerner Millennium installation. Cerner have since been taken over by Oracle, of course.
It used to be that hospital and healthcare systems ran on mid-range systems - OpenVMS ruled the roost for Cerner, then they moved to HP-UX when they didn't want to jump to the Itanic. Now, it's all clicky-draggy client-server with a PeeCee on every desktop and email to the masses.
It's perhaps hardly surprising that Jurassic Park or other health providers have been targeted. They do still send faxes to GPs after clinic visits though!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
