Devonitely not great: Torbay and South Devon NHS declares 'major IT incident'

Torbay & South Devon NHS has declared a state of emergency of the tech kind, warning that a “critical IT systems failure” has forced it to cancel all scheduled routine surgery and outpatients appointments. The crash first happened yesterday, the Foundation Trust confirmed on social media, and its team of techies has been …

[Anonymous Coward]

Failover?

Surely they know about that?

[Jemma]

Re: "Failover"?

Yup, but they got it confused with "Fallover".

[John G Imrie]

Re: Failover?

Failover requires explaining to some bean counter why that equipment in the corner isn't being used at 100%. But if you do win that battle you then get the Daily Mail or other tabloid newspaper screaming about wasted NHS resources.

[LewisRage]

Re: Failover?

Of course they've heard of failover, they've probably been told by the consultant that it's in and working but they've never bothered to test it and/or massively under specced the 2nd site. Something has gone awry, the failover happened and it's still awry, just differently awry now.

Or the guy that preceeded me at a place worked there. Had a SQL failover cluster which did work but it was all running on the same storage so when the SAN had a wobble both nodes fell over at the same time.

[Pascal Monett]

What ?

"There is £6bn backlog of repairs across the NHS & the underinvestment in physical & IT infrastructure has led to repeated safety failures & delays"

A £6bn lack of funding is not a mistake. This is not somebody who forgot to sign an expense. This is willful negligence and someone should pay.

More likely an entire committee. The last five years of top-level decision-makers are all collectively responsible for this complete and utter failure of management.

[Anonymous Coward]

Re: What ?

More likely an entire committee.

Much more likely an entire political party.

[Alister]

Re: What ?

Much more likely an entire political party.

No, just the UK government - of any political shade - over the last 20 years.

[Cederic]

Re: What ?

Yet somehow NHS funding continue to rise, and rise, and rise, and rise.

Something tells me cash isn't the issue here.

[BebopWeBop]

Re: What ?

Or the cash really is not there?

[Anonymous Coward]

Re: Record Levels of Investment

Yet somehow NHS funding continue to rise, and rise, and rise, and rise.

1. Cut social care with NHS taking the slack.

2. Forced tendering with no advantage onto NHS.

3. 61 Tory MP including then PM and Health Sec, bank a fat profit from private healthcare investments.

https://www.bmj.com/content/349/bmj.g6982

Previously the massive centralised buying power of the NHS drove efficiencies in consumable purchases, e.g. syringes and swabs at cost price.

2014 Tories introduced "a market in health care", neatly cutting the purchasing power of the NHS, while cynically being able to claim "Record Levels of Investment".

This is the table for fixed fines, so it might be cheaper to cover up evidence and pay the "failing to explain" charge of £300.

https://www.legislation.gov.uk/ukdsi/2014/9780111117613/schedule/5

Every pound given to the NHS has a percentage of internal market profit available to well placed people with healthcare investments, like PM and Health Sec. Resulting in NHS paying more, getting less, despite "record levels of investment".

[TheVogon]

Re: Record Levels of Investment

So those MPs have private healthcare investments that presumably help reduce the load on and costs of the NHS?

[EnviableOne]

Re: Record Levels of Investment

The companies supported by those private investments are paid for from the NHS budget, they do not have their hands tied by the NHS rules and cost base, they do not have the public scrutiny, as they pay less they get less qualified and competent staff, they have worse outcomes, and they are robbing the country blind.

[Anonymous Coward]

Re: Record Levels of Investment

Sounds you are talking about all those Labour PFI deals?

[Anonymous Coward]

Re: What ?

yes cash not the issue here there kit is pretty good. The issue seems to be everything is in a flat layer2 network including the SAN, the outage has been caused by a spantree event that took down the network so the SAN went south too

[TVU]

Re: What ?

"Much more likely an entire political party"

...and one that is obsessed with grand ego projects like the £86 billion HS2 (independent estimates put it at £100 billion+) which needs to be cancelled asap so that the money can be put into projects of real worth.

[Richard Jones 1]

Re: What ?

Well perhaps we should cancel HS2 after the 31st October in favour of something better suited to our UK needs? It started as an EU project, to impose high sped rail across the EU.

Of course a previous government spending did include such as the failed NHS IT project under Blair's lot; or on an unpopular ID card scheme.

Scotland produced their Named Person plan. Then their courts ruled it conflicted with other legislation.

Every government tries and falls foul of objectors. Sometimes it spends all the cash as the Blair-Brown team, remember the famous note from Darling?

[BebopWeBop]

Re: What ?

An EU project to impose high-speed rail across Europe - you need to get the tin foil hat back on. When did the EU propose imposing it?

[EBG]

Re: What ?

1. There were EU documents showing an EU wide high speed rail network back in the 1990's, including the UK. I can assure you of that as they passed across my desk in a UK .gov role.

2. There is the great "modal shift" policy, locked in at EU level that says 50% of intercity travel will be by rail by ( forgotten the exact date, but it's always ending in a round number, let's say 2030 ). The industry will tell you that HS2 was " always about capacity, not speed" and needed for the UK to be seen to be complying with the "shift to rail" policy.

See - not everything that you don't know about, or don't agree with, is "tinfoil hat"

[Anonymous Coward]

Re: What ?

The biggest problem with the NHS IT is that it's not centrally managed and has no central guidance. NHS Digital are widely ignored, and in most cases the local trusts have very limited IT and just go away and do what they feel, without any standardisation across different trusts. NHS IT also don't pay well compared with other private companies or even contractors, and they don't have the money to get in the decent contractors to deliver the service.

Many of the NHS IT Directors I know have no clue about IT and are usually also the HR Director or Finance director.

[Doctor Syntax]

Re: What ?

You may be right but I have the uncomfortable feeling that more centralised management will simply increase the scale of failures.

[JulieM]

Re: What ?

The NHS is big enough to have its own IT department and even create its own custom software, based on Open Source to avoid further enriching billionaires, for nationwide deployment. If things are ever running smoothly, NHSIT could keep busy doing contract work for third party organisations.

[old_iron]

Re: What ?

Ahh, Opensource, that'll be free then !

Why didn't anybody think of that?

[JulieM]

Re: What ?

Because proprietary vendors engage in dysangelism about Open Source software.

Open Source requires work, and the people who do that work will expect to get paid; but at least that money stays in the local economy. Local programmers eat in local restaurants, drink in local pubs, shop in local stores and visit local tourist attractions with their family.

[Anonymous Coward]

Re: What ?

Lol. It maybe, but it doesn't run like that.

In the real world, most of the software they use on a day to day basis is Windows based and clunky at that.

As I said before, there is no single NHS IT department. The NHS is a single entity in name only. The trusts across the UK run IT very separately from each other. There are a couple of shared services - such as the networking provision - and NHS Digital with relation to what are referred to as 'Spine' digital services for sharing between different parts of the NHS, and the NHS main website. But that's about it.

[Anonymous Coward]

Re: What ?

Munich tried that and it ended up costing them tens of millions of Euros more than sticking with non OSS software, after a decade over 20% of stuff still hadn't been migrate, the users hated OSS, compatibility issues were endemic, and they eventually had to migrate back to the latest Microsoft everything to provide an environment that actually worked.

[Boris the Cockroach]

Re: What ?

Quote:

"A £6bn lack of funding is not a mistake. This is not somebody who forgot to sign an expense. This is willful negligence and someone should pay.

More likely an entire committee. The last five years of top-level decision-makers are all collectively responsible for this complete and utter failure of management."

You are forgetting this is NHS/government area.... so the decision to cut back and not invest in IT resources was made by the committee not by a single named person

However since all the committee members who made the decision have moved onto other positions within the NHS, then it would be unfair to penalise the current members of the committee for the previous decisions of the committee.

Or to penalise former members of the committee since they are hard at work in other committees making the decisions about the running of the NHS

The top level management have nothing to do with the staffing of the committees as this is decided by the members of the committees working with human resources to identify the best staff to work on said committees and also the top level management of the NHS merely implement the committees decisions.

Apologies to Sir Humprey

[John Brown (no body)]

Re: What ?

"However since all the committee members who made the decision have moved onto other positions within the NHS, then it would be unfair to penalise the current members of the committee for the previous decisions of the committee."

Why not? Entire countries are still being blamed for things which happened generations ago and have paid or are being asked to pay compensation, despite none of the current population/government having been involved in the past problems or, in some case, none of the victims/survivors still being around to benefit from compensation.

[RichardB]

Re: What ?

Bloody Romans.

[EnviableOne]

Re: What ?

that was a nice cup of tea too....

[Pascal Monett]

Re: it would be unfair to penalise the current members of the committee

I do not see the current members of the committee putting in £6bn to correct the situation, so they are guilty.

Unless they can demonstrate that they are scraping every farthing they can find to correct the situation, they are guilty.

[TheVogon]

Re: What ?

If only we were not paying for all those overpriced Labour attempts to privatise the NHS via PFI contracts we might have had the money to fix this...

[Anonymous Coward]

Re: What ?

Shush! How will be able to sell privatisation to the plebs if they knew we were deliberately underfunding the NHS in order to suggest what we need was a private cash boost.

E.g National Rail in the 70s and 80s.

[MiguelC]

OPEL 4

As a bootnote, the Opel 4 (no similar Vauxhall model in the UK) was known as the doctor's car

[Jemma]

Re: OPEL 4

Norfolk would still be using them as ambulances if you could get the parts...

[The Oncoming Scorn]

Re: OPEL 4

I thought that his car was called Bessie.

Joking aside - Mate of mine has one from the original molds (Which he acquired in 2017, shipped over to Canada & then repaired for use), built onto a scrap Triumph Spitfire chassis (Rather than a Ford Polar\Anglia). Just for good measure, he now owns a distillery.

http://tardisbuilders.com/index.php?topic=8150.0

[EnviableOne]

Re: OPEL 4

its only Opel 4 as they ran out of colors on the old scale, and they thought with the state of funding there's a good chance they would need to add an OPEL 5

[TheVogon]

Crypto virus again?

[Locky]

Odds on

[Anonymous Coward]

Smells like ransomeware to me

But it often does these days.

[Brewster's Angle Grinder]

Ahhh, more "efficiency savings", I see.

Yes, we have cut back on needless bureaucracy so we can spend more money on doctors and nurses and front-line staff. What do you mean those back-end staff weren't all diversity officers?

[Chris G]

When I lived in Devon, if we was sick or ailin' we used ter go an' see the Hedge Wife, bit like Nanny Ogg but scarier.

[Anonymous Coward]

What about the 50% + of the population that is not on Twitter etc?

I guess we don't count in todays world.

Proud to be a (anti)Social Media refusenik.

I see no reason to share my life with Evil American Conglomerates that ONLY exist to sell that data on to Advertisers and anyone else who will stump up the lolly.

[Anonymous Coward]

Re: What about the 50% + of the population that is not on Twitter etc?

You’re counted when you post on forums such as this one.

[John Brown (no body)]

Re: What about the 50% + of the population that is not on Twitter etc?

...and even if you were a Twitter user, why would you "follow" your local NHS trust? Or is there some way for NHS accounts to broadcast to all Twitter users in a certain area?

Actually, something like that might actually be useful if it could be managed down to a reliable geographic area with safeguards to make sure it's not abused.

[Jemma]

Re: What about the 50% + of the population that is not on Twitter etc?

I'd follow it all right, at speed with a couple of Panzer Corps, lots of lovely Pervitin, and a mate of mine by the name of Erich..*

The NHS is very much like Soviet Russia - in that there's lots of it, it's all on the verge of collapse, mostly empty and/or clapped out, and its organisational motto is "and then it got worse" and it'll do its level best to kill you as general policy.

*with NHS commissioning managers playing the part of soviet Commissars.

They could rebrand it of course - OPEC maybe - Obligatory Patient Eradication Corporation...

Well, at least it'd be honest.

[Gordon 11]

Are we sure this isn't just part of the Government's "Get ready for Brexit" campaign?

[don't you hate it when you lose your account]

No

That was Thomas Cook

[Anonymous Coward]

Could be issues with a N3 to HSCN migration. Not as simple as you would think, particularly as many organisational changes have meant changes in which NHS organisation are partnering us for networks (COiNs).

Mind you, I would expect any Acute Trust to have a backup phone system for resilience.

Re Open Source Software - 3 main issues

1) NHS digital mandates the software for Smartcard Authenication and hence Spine access. This only runs on Windows and only IE supported. This is why the NHSUbuntu project in Leeds died. Without Spine Access it was doomed to failure.

2) Open Source does not equal free. Support contracts are still required for systems. Would you really want a critical system at your hospital having posting on a forum as the means of support. Same thing tends to limit smaller companies as they struggle providing support.

3) Not all staff are computer literate. At least providing them with Office & Windows at least gives them something they may have seen before.

Open Source Software is used where appropriate. Many software packages include it to some extent, even if it is just Apache or Tomcat.

A/C for obvious reasons but i don't work for the NHS organisation in question.

[EnviableOne]

1) no NHSBuntu got sued by the NHS for copywrite infringement, and before they announced the W10 purchase, the NHS cosied up to them synically to get microsoft to drop the price, they renamed OS for NHS and although dormant, it is infact compatible with the NHS smartcard authentication system.

2) technically true, but put these together with the zero cost of ownership and the ability to customise to your specific needs, and the TCO is a lot lower.

3) if your changing from w7/office 2010 to w10/TBC(Still), this is just as big a change as going to say OS for NHS and LibreOffice

[TheVogon]

Zero license cost != zero cost of ownership. Supported OSS software rarely has a lower TCO when you include migration and integration costs.

[Anonymous Coward]

"if your changing from w7/office 2010 to w10/TBC(Still), this is just as big a change as going to say OS for NHS and LibreOffice"

No it isn't. Having done several such migrations users rarely require any retraining from W7 to W10 and it just works with all their existing settings, files, addins and macros. Unlike when migrating to Libre Office.