back to article Appointments on hold as (computer) virus wreaks havoc with NHS trust systems

An NHS trust in England shut down all of its IT systems today and has all but ground to a halt in general after a virus compromised them on Sunday. In a bright-red warning labelled "Major incident" on the website for Northern Lincolnshire & Goole NHS Foundation Trust, patients are warned that their appointments have been …

  1. lansalot

    A free hip, says it's ransomware...

  2. boltar Silver badge

    Let me guess...

    They're still running XP and allow staff web access.

    In something as critical as a medical network it should be semi air gapped from the internet with no external access for users except with express permission and with any inter site data being transmitted via secure VPN. Oh, and no USB sticks and no bloody XP!

    1. Anonymous Coward
      Anonymous Coward

      Re: Let me guess...

      Maybe not even as complex as that. As I understand it, there are PCs around hospitals that are generally unlocked so that they can be accessed as quickly as possible in the event of an emergency.

      1. Anonymous Coward
        Anonymous Coward

        Re: Let me guess...

        No there aren't, most require a security card AND password to use

        Credentials are usually trust wide, but you'll never get internet access unless you are required to have it

      2. Korev Silver badge

        Re: Let me guess...

        In the hospital (not in the UK) physiotherapists where I'm currently being treated there's a PC which is never locked and they only use one account between them. On the upside it's Windows 7.

        I work in a big pharma and "clinical data" is incredibly tightly controlled and audited - when I think of how slack this hospital is I almost wonder why we bother!

    2. Ashley_Pomeroy

      Re: Let me guess...

      My experience of NHS Trusts in the south is that they tend to use some kind of virtualisation system - the machines don't have direct access to the outside, and USB ports are blocked. Everything goes in and out via the virtualisation client.

      But they tend to have a mass of different systems - one for clinic management, another for results, another for theatre management etc. And physical security is difficult in a hospital because there are always people walking in and out.

    3. Anonymous Coward
      Anonymous Coward

      Re: Let me guess...

      No bloody XP....or Vista, 7, 8 or 10...

    4. Pat Harkin

      Re: Let me guess...

      "They're still running XP and allow staff web access."

      Many - admittedly not all - staff need web access. Patients walk in with a single screen shot from a website and we need to be able to assess their source and show them what credible sources look like instead.

      And I'd never get my Xmas shopping done if I couldn't get to Amazon.

  3. Mr_Pitiful

    Not all systems are affected

    We have monitoring equipment there, and that's running fine...

    I can see a time coming in the near future where all external comms are prohibited by trusts

    Their email is air gapped, but not the clinical systems i.e. Medical Imaging

  4. Anonymous South African Coward Silver badge

    Ne'er-do-wells *rolls eyes*

    The possibility of a cryptolocker (or similar) is possible, not nice to have.

  5. Anonymous Coward
    Anonymous Coward

    I bet it'll be another old dear of a health visitor who's opened an attachment called invoice.pdf.exe sent to her from "Bank of America".

    1. m0rt

      I really would like to see some data that supports this. It is easy to assume that older people are not as aware...the industry is rife with this. And we all know of a person who fits this type and action. But I reckon we all know of people who would do this who are not 'old dears'

      1. John Brown (no body) Silver badge
        Thumb Up

        "I really would like to see some data that supports this. It is easy to assume that older people are not as aware...the industry is rife with this. "

        El Reg recently ran a story showing that the young'uns are far more likely to fall for online scams than older people.

    2. Doctor Syntax Silver badge

      "I bet it'll be another old dear of a health visitor"

      More likely a dumbo millennial.

      1. Anonymous Coward
        Anonymous Coward

        In my considerable experience the standard profile for the person most likely to infect themselves is:

        1. Female (sadly).

        2. In her 30s

        3. Works too many hours and is usually too dumb to understand that your productivity is higher when you arent tired.

        4. Lives in fear of losing her job.

        5. Works in the accounts department as an administrator.

        I know ill get flack for this, but of all the ransomware dramas ive been dragged into the vast majority of the time this is the person that got infected.

        Taken from a sample of approx two dozen cases.

        Other close contenders include the CEO snooping on an employees mailbox thinking they've found dirt.

        1. Anonymous Coward
          Black Helicopters

          May I also add that they are also usually the ones that accidentally allowed Win10 to install across the company network by clicking on a MicroShit pop-up.

          (Happened at my sons place of work, by some miracle, none of their bespoke machine control software crashed).

          Still, this might give other trusts an idea.

          "Look people, we need to save money until the end of the financial month, any ideas??"

          "Well, we could claim a computer virus ate all our paperwork, and cancel all operations, that would save a bundle".

          Oh look, the Air amb........

        2. Steve Button

          Don't be such a dick. Seriously we need more diversity in IT, and it's attitudes like this that turn good people away. Please take your "considerable experience" and stop profiling individuals. It's not helpful at all.

          1. boltar Silver badge

            "Don't be such a dick. Seriously we need more diversity in IT, and it's attitudes like this that turn good people away. Please take your "considerable experience" and stop profiling individuals. It's not helpful at all."

            Yes, god forbid someone should use real world experience. Far better to pretend reality is the same as the liberal left narrative morons like you obviously subscribe to.

        3. N2 Silver badge

          And

          Still has to click on every attachment even though they promised not to after the last fisasco, because you as administrator have all that anti virus stuff that sorts this out.

          Yes, I know a few, thanks for nothing.

  6. Tezfair
    Unhappy

    I suspect it was an image file as xrays are digital and are emailed between depts (when required). I had a recent xray in cornwall and the specialist is in Devon. As the xrays hadn't been uploaded to the server, the Mr called the xray dept and had them emailed over.

    Bet something similar happened here

    1. Anonymous Coward
      Anonymous Coward

      A clinician's guide to digital X-ray systems

      A clinician's guide to digital X-ray systems

      1. pxd

        Re: A clinician's guide to digital X-ray systems

        I feel obliged to warn you about introducing facts into what looks like becoming a useful little flame war. Don't let this happen again. That is all. pxd (now have an upvote)

  7. Anonymous Coward
    Anonymous Coward

    Time to roll back the years and give them all dumb terminals.

    1. Archtech Silver badge

      Definitely!

      "Time to roll back the years and give them all dumb terminals".

      The PC is grossly over-engineered for most purposes. Thing is, that's why it has achieved such great technical progress: by addressing almost all markets and applications, it sucks in vast amounts of profit, some of which goes to R&D.

      The time has come to perform triage on operating systems, and ruthlessly get rid of everything that isn't strictly necessary for the given task. For someone to be able to do their Christmas shopping at Amazon through a hospital computer (as someone, perhaps facetiously, suggested) is not just unnecessary - it's criminally irresponsible.

  8. Howard Hanek
    Childcatcher

    A Suggestion

    NHS and trust system definitely IS an oxymoron and should never be used to describe the attributes of the NHS

  9. Doctor Syntax Silver badge

    "All adult patients (over 18) should presume their appointment/procedure has been cancelled unless they are contacted. Those who turn up will be turned away."

    It's the computer system that has a virus not the doctor.

    The patient knows what time their appointment's for and may already have arranged to take time off work.

    The doctor knows what time they have a clinic.

    In some cases the examination may be impeded by lack of access to the system but that's not necessarily the case for all examinations. Take written notes. They'll need to be put online later and if there were notes from a previous visit they'll need to be compared to those earlier notes and maybe an extra visit to take action that follows the comparison. But a hospital should not be totally dependent on functioning IT systems. It sounds like the decision of an administrator totally divorced from any perception of patients' circumstances. If they can manage without the system for younger patients why not for adults?

    1. Adam 52 Silver badge

      "Sir Ian Whitchurch: [to Sir Humphrey] First of all you have to sort out the smooth running of the hospital. Having patients around would be no help at all."

    2. Commswonk Silver badge

      @ Doctor Syntax...

      I really would like to agree with you but I suspect that you are being over optimistic. Every consultation would start with the words "Who are you, and why are you here?" There would be no notes from previous consultations available; no letters from GPs referring patients for initial consulations or anything. No means of issuing / recording prescriptions for action by the hospital pharmacy - nothing.

      As if that was not bad enough, post consultation notes written down for uploading later would result in a backlog that would have to be dealt with along with the "new" material from the next clinic, possibly resulting in one patient's notes being confused with another and so on. I could easily envisage both administrators and clinicians being in a complete funk in case there was a serious lapse that resulted in misdiagnosis, misreporting, inappropriate treatment and so on. The endpoint of that is claims for medical negligence; in the worst case it could even lead to increased patient mortality.

      As long as A & E could keep going I can see that cancelling everything else would be seen as the least worst option.

    3. Anonymous Coward
      Anonymous Coward

      Sounds like the usual lack of business continuity plans, in the NHS where I work we all have a fall back plan if we lose power etc so we can continue to treat patients where it's safe to do so with the assumption that access to clinical systems isn't possible for 1 day, 1 week, 1 month - with each scenario having detailed plans on which staff should do which task, so the longer systems are down, the more admin support we'll get to ensure notes etc are filed for later transcription to clinical systems.

      There are instances where without systems we'd be unable to treat but these are fairly rare - the biggest problem will be medicines, with most hospitals moving to an electronic prescribing system, it'll be a shock to be back to filling in forms, I wouldn't want to work in pharmacy when that happens especially as the medicines are currently dispensed by a couple of robots in their own storage facility.

  10. cantankerous swineherd Silver badge

    about time all the digital bollocks was stopped.

  11. Anonymous Coward
    Terminator

    Computer virus wreaks havoc with NHS trust systems

    If only they had stuck to the industry standard Microsoft software written in C# under Visual Studio.

  12. x 7

    A lot of rumour, speculation and bollox being spoken here by the unknowing. I don't know the details of this specific case, but at the moment I'm involved in a multi-site clearup of another trust which was recently hit by ransomware on its servers.

    Over the various sites there are something like 600 XP machines remaining which we are urgently upgrading to Win7, the antivirus licence has expired on the XP machines (its being replaced as we upgrade) and many of the staff - both clinical and non-clinical - log on with a generic user name with an obvious password - and admin rights. And people wonder how they got infected.........

    On top of that the PCs themselves are all around ten years old and fit only for scrap.

    Simple truth is theres no money available to properly upgrade to new secure kit and we're having to life-extend geriatric crap. Not even enough cash to properly secure the network and user accounts.

    Needless to say, I'm not going to say which trust - but my understanding is that many are in similar straits

    Upgrading the machines themselves is in many cases proving to be a nightmare. For instance, stock control programs written using Borland for which the licence expired ten years ago - but is still in use because the replacement software doesn't work. Several examples of lab instrumentation software which is 16-bit, and so old we don't have installation media, and is no longer available from the original suppliers. Several examples of bespoke databases which have been in use for years - totally undocumented and the programmers have long since moved on. The more we dig the more crap like this we find - and so far attempts at virtualisation have failed as the VHD files created from the existing PCs won't boot (remember no installation media so building new VMs insn't an option).

    Based on what I've seen elsewhere, whats been reported from Lincoln comes as no surprise. There's a lot more Trusts out there living on a prayer that something similar doesn't hit them

    1. Displacement Activity
      Meh

      A lot of rumour, speculation and bollox being spoken here by the unknowing.

      You need to publish. The reason that we have these problems is that the people who know keep their mouths shut.

    2. Anonymous Coward
      Anonymous Coward

      This...

      @x7 is absolutely correct.

      We haven't had any major disasters here (touches wood and thanks Ghod that eHealth scrapped their own email service and moved to NHSMail.), but the problem is no cash for equipment or IT staff. Many areas (like mine) rely on staff-with-an-interest to provide first line support (eHealth are good, but they can't know everything about the hundreds of systems that make up a hospital). Yes, it's not sustainable, but patient-facing services get the publicity and the kudos and the money, and infrastructure and support just have to thole it.

    3. Anonymous Coward
      Anonymous Coward

      guuurr

      I worked NHS contracts for about 5 years and (almost) all of this is true. The NHS IT systems are a shambles. Completely and utterly mismanaged from top to bottom. When you throw in all of the bespoke systems you mention, a lack of coordination between trusts (why do we use so many different PAS's!?!?) bobbins security.....the list could go on and on

      I did my fair share of XP upgrades (over 4 different trusts, probably 12-15 hundred machines) and was never able to use a mass deployment tool once. WHY are tech's manually installing OS's???? wasting time and money due to a lack of skill or vision by managers and 3rd line. 2nd and 1st have to just deal with the shit left behind.

      The hardware out there is often a joke. I worked on a project a couple of years back where 2 million pounds worth of hardware (Dell 10's running Windows 8!!!) was bought without any proper testing by technicians. The result is that they now sit in DN's desks because there genuinely rubbish. Madness.

      So, the only part I slightly disagree on is the 'no money available'. There is money, and boat loads of it. It just gets massively wasted.

      Its no wonder virus outbreaks like this happen. The techs can only do so much because the systems as whole are so convoluted. Everyday is spent firefighting so it was only a matter of time before something actually got burnt.

      Someone really needs to take the NHS IT by the balls and get all trusts singing from the same page. Systems need to be unified, in many cases infrastructure needs overhauling. But most of all, managers and directors who know bugger all about IT need to be shown the door as they are simply making things worse.

      Jesus, I've not been in the NHS for a year or so now and it still makes me very angry....

      chill..... relax.....I have a different job now.........

      Just seen a headline on google news

      "Chancellor pledges £1.9bn to protect UK from cybercrime"

      that's right, just throw some money at it.

      1. Anonymous Coward
        Anonymous Coward

        "Someone really needs to take the NHS IT by the balls and get all trusts singing from the same page. Systems need to be unified, in many cases infrastructure needs overhauling. But most of all, managers and directors who know bugger all about IT need to be shown the door as they are simply making things worse."

        That was called Connecting for Health wasn't it?

    4. Archtech Silver badge

      The real solution

      Ironically, if everyone were to eat and drink healthily, get adequate sleep and exercise, and refrain from harming themselves with tobacco, excessive alcohol and other drugs, the total NHS bill could probably be halved or better. But that would be bad for profits! First corporations make a fortune selling crap food and drink, cigarettes, booze and (the illegal corporations) illegal drugs. Then the pharmaceutical, medical and insurance industries make a bundle selling drugs and treatment to alleviate (but not, if possible, cure) the conditions that were caused by the crap food and drink and drugs.

      Also, of course, managers persist in making their employees work all the hours that God sends and more, treating them in highly stressful ways, and denying them adequate time and facilities to rest and exercise.

      If people ate, drank, slept and exercised healthily and were treated decently at work (and were able to get work) far, far fewer of them would be chronically ill.

      1. Anonymous Coward
        Anonymous Coward

        Re: The real solution - FAIL

        Tax from tobacco £12 billion, tax from alcohol £14 billion.

  13. Nifty Silver badge

    Bitcoin & tax

    Rumours abound that local authorities do pay ransoms occasionally. That requires accounting for if it's public money.

    Now, I wonder how many ransoms paid in Bitcoin I can put on my small businesses tax return per year. My Bitcoin pension is growing nicely.

    1. Anonymous Coward
      Anonymous Coward

      Re: Bitcoin & tax

      Except the ransomware payments would not be tax deductible. You'd just have to take them out of your profit. Check with an accountant of course.

  14. Anonymous Coward
    Anonymous Coward

    The NHS is still...

    Being dragged kicking and screaming into the 21st century, and it's mainly down to individual trusts just straight refusing to update anything because what they have "works just fine", I'm genuinely amazed this didn't happen sooner.

    Posting anon as they are a client

  15. Anonymous Coward
    Anonymous Coward

    Utter, utter mismanagement

    And, speaking as someone who's run very large (25k+ user) desktop estates but didn't even get past a sift for a role in a trust as I have no NHS experience, completely maddening.

    This is not a hard management problem for anyone outside the NHS as, by a process of ruthless Darwinism, anyone who can do the job has already ensured that (a) the risk it never happens is properly managed and (b) has an effective, tried and tested, process for ensuring it only ever, ever, happens if at all!

    1. sunrise

      Re: Utter, utter mismanagement

      To be honest after reading the second paragraph of your comment I wouldn't employ you either - it makes no sense whatsoever.

  16. Not previously required
    Devil

    Did you miss something

    ... Goole NHS Trust on 31st October? Not 1st April??

    I work in a hospital. With a fantastic IT department (I'm not in it). But hospitals are chaotic, complex places unlike most IT companies. How many of your colleagues would you trust to make stuff happen quickly if you were ill ....

    And if you think the NHS is poor at data security, my bank makes it damn near impossible to log on, but they sent me important financial papers in plain email. The twit on the phone promised it would be encrypted. The figures were wrong and the next twit offered to email again, but at least the 2nd twit knew it was not encrypted and agreed to send by snail mail

    And a little red devil for season's cheer.

    1. Anonymous Coward
      Anonymous Coward

      Re: Did you miss something

      " But hospitals are chaotic, complex places unlike most IT companies"

      I think quite a few of the comments here have come from techs who have worked Hospitals\GP sites etc. They are chaotic and complex, but so is IT.(even though it shouldn't be) That's why IT needs to be left to IT (who would keep it clean and simple) and not bureaucrats who actually run it on a band 9 pay bracket.

      My local Hospital where I first worked for the NHS has over 600 different applications to run! That's a lot of knowledge for any tech to keep hold of. Who knows how to navigate the old green screen patient booking system they STILL run via remote XP session? Not the new guy on a 2 month contract!

      Trusts make it complex by the "it works" attitude someone else mentioned a few posts up.

      "If you think the NHS is poor at data security,....."

      o good god it is

      I could walk into ANY hospital in the country with a fake badge, say I'm from IT, sit down at a computer run my linux build from a pen drive (all trusts that i have worked at do allow usb's from boot), change the local admin and wait for a real admin to enter a password into my key logger.

      Your network is only as strong as your weakest point. That's not necessarily an IT issue. Although we can help\train if allowed to do so.

      Do you know how often a Doctor demanded local admin rights just because they were a self-important Doctor? And they got it. Total Insanity!

      Your network is only as strong as your weakest point!

      Clinical, admin and everyone in between do tremendous work but are seriously let down by weak IT management who do not have a clue. All of the things need to change to make the NHS IT systems great.

      Something that will sadly never ever happen

      1. x 7

        Re: Did you miss something

        "I could walk into ANY hospital in the country with a fake badge, say I'm from IT, sit down at a computer ......"

        A couple of contracts back the management at one Trust decided it wasn't worth getting me an NHS ID badge because I would only be there a few weeks. They expected me to wander around two major hospitals carrying out software audits. No ID, no warning to the wards / departments, nothing. You can guess the results when I tried to get onto geriatric or kids wards..............I was quite rightly told where to go. After three days I told them to stuff their job and walked.

        In the main, the management of IT within Hospital Trusts is carried out by two types of incompetent: mainly those who've only ever worked at the one hospital, have "grown with the department" and as a result haven't a clue about modern practices, have no management skills, and no real-world experience. The other type are top level "Professional Managers" who flit from contract to contract, leaving just before they get pushed, collect massive pay packets and are generally totally incompetent.

        The few managers who know what they're doing, who actually have the required skills become frustrated and leave.

        There really is a case for taking IT support totally away from the hospital trusts and setting up a new national support organisation with a management sourced from industry. With any luck it could also replace the useless CSUs who provide support in GP-land

    2. x 7

      Re: Did you miss something

      "I work in a hospital. With a fantastic IT department (I'm not in it)."

      Glad you feel that way about the IT team. However I can assure you that the support you get will be due to the dedication of the front line support team slogging their guts out in the face of management adversity, and lack of financial support. I bet you think they're fantastic because the fix problems quickly. However if the department was working properly - and not chasing its own tail - you'd never notice them because there would be no problems to fix.

  17. jrdld
    Unhappy

    Silver Bullet

    I work in NHS IT security. It's a nightmare, and it will continue like this until managers are held PERSONALLY accountable, like they are under HIPAA in the US. Fining public sector bodies is just fining innocent tax-payers. Only when the bosses themselves are facing HIPAA-style five-to-seven-figure fines will they take it seriously.

  18. David Pollard

    Coincidence or delivered in person?

    Earlier this year, in late January, Lincolnshire County Council's computer network was shut down after being hit by ransomware. Is this attack on LIncolnshire NHS in any way connected?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020