Patch now? Why enterprise exploits are still partying like it's 1999

Some vulnerabilities remain unreported for the longest time. The 12-year-old Dell SupportAssist remote code execution (RCE) flaw – which was finally unearthed earlier this year – would be one example. Others, however, have not only been long since reported and had patches released, but continue to pose a threat to enterprises …

[Paul Crawford]

That Microsoft Equation Editor memory corruption vulnerability (CVE-2017-11882) mentioned earlier that has been patched since 2017, is "used especially in attacks on the healthcare industry"

I wonder if that is due to the "patch" simply breaking the ability to edit older documents using equations?

It really is piss-poor that MS could not fix it themselves as apparently it is a 3rd party code. And they cannot afford to do so? Really?

[big_D]

It seems to be Microsoft's mantra at the moment, "break it and move on!"

Our boss suddenly had problems printing Word documents. Printing the document works, printing a page work, but printing a range doesn't! Turns out it was a bug introduced recently and no fix released, but using a version out of the beta channel might solve the problem.

Sorry, not putting beta software on the boss' computer!

What am I talking about? Almost all software these days is beta status, or worse!

[Anonymous Coward]

Well the problem is your afore mentioned bosses getting us devs to rush shit out the door or making our testing teams redundant. If I've said once i've said it a million times automated testing is just to make sure it behaves as expected under expected conditions, but you can garantee once its in the hands of a user it will break almost instantly.

[big_D]

Not my boss, he is over 80 & runs a chemical company. Slow, patient and precise is his mantra.

[David 132]

Not my boss, he is over 80 & runs a chemical company. Slow, patient and precise is his mantra.

I would imagine that anyone fast, impatient and slapdash working in the chemical industry doesn’t usually get to finish their 3-month probationary period, let alone achieve company ownership.

(Obligatory link to FOOF, Things I Won’t Work With)

[Clausewitz 4.0]

Why exploits are still partying like it's 1999

Reminds me of Happy99 by Spanska.

I believe exploits are still partying like it's 1999, mainly because its not only a technical situation, but also a creativity one.

Where an average security professional flag a bug as low-risk, a creative professional will, in a few days, chain it with others innocent-looking bugs - and make it high-risk.

[big_D]

Big problem...

Is legacy hardware.

We have a shield printer (metal and plastic), the control software only runs on DOS and the thing is attached over a serial cable. We found a "spare" on eBay for a couple of thousand Euros. A new one is around 6 figures. We collect old PCs and keep them in storage, in case the old one fails. Is it really worth spending a hundred thousand for a new printer that does exactly the same thing as the one you already have? And we don't print enough to get an ROI inside about 15 years, so we would be back at square one.

Likewise lab kit. A lot of it still works, does exactly what we need, is reliable. But the software won't run on anything newer than Windows XP. If we want a version of the software that runs on Windows 10, it will cost 6 figures and involves throwing out a perfectly working piece of precision equipment, just because the OS on the PC that collects the data has changed.

We just isolate the damned stuff from the network - either stand-alone or a separate segment just for such devices, with no access to the "office" network or the Internet.

An IT director I know works for a metalwork production company. They have an old CNC machine that works fine. Same problem. It is isolated. Every time they call up the manufacturer for support, they want to connect to the controlling PC (Windows XP) using TeamViewer. She refuses and tells them the the viewing Team is them and the console operator, who they can remote control. If they want to use TeamViewer, they need to supply an update that is Windows 10 compatible. And, no, a replacement CNC machine (250K+) is not a software update!

That is one of the biggest problems we have. We have taken expensive kit, with multi-decade lifespans and support and connected it up to a cheap PC that is obsolete after a couple of years and no upgrade path to keep it safe!

The manufacturers hope to jump on the software & services gravy train and milk companies every 5 years for a new piece of kit, when the kit is so expensive that it has a 20 year write-down. They don't/can't make the plant equipment cheaper and they won't support it with current software, that is restricted to the newer devices, to entice people to upgrade the "expensive bit".

Nobody in their right minds throws out millions of Euros worth of working equipment, just because a 300 Euro PC can't have its operating system upgraded, because the controlling software isn't compatible.

The sensible one isolate the stuff from the network and carry on as usual. The idiots leave it connected to the Internet, so a manager can check it from the beach in real time!

[DS999]

Re: Big problem...

Yes that's it right there. It is easy for IT to patch all the servers in the datacenter and all the PC/laptops on people's desks (or now in their homes) I say "easy" even though it is not always done, but I've seen places that do a very good job of staying within a few months of the latest patches across the entire environment IT manages.

It is that hidden hardware, either embedded within a larger product (EMC VNX arrays have an embedded Xeon server running Windows (yes, Windows!) as a "controller") or sitting on a factory floor, in the corner of someone's office controlling some weird hardware like a plotter, or in a basement/closet somewhere controlling building security or HVAC.

Not only does IT not patch those systems, in most cases they don't even have access to them - either an Administrator password or physical access to where they are installed. They may not even know they exist, but if they find out about them and suggest "these should be patched" they're told "the vendor is responsible for them". For example if you have a Daikon HVAC for the building with a PC controlling it, do you really want to patch it and risk having it no longer work if there is a problem with the patch or some weird OS issue that makes the patches a problem? Does Daikon even want people patching "their" hardware or are their service people supposed to be doing that?

Once someone breaks into one of those systems with an old exploit they can report back to the ransomware crook "hey I'm inside someone's network!" and provide a communications path to the crook who can then manually try some newer exploits on the more well patched devices to install the ransomware. Though even if you only encrypt the computer running the building HVAC depending on what time of year it is you might get a quick ransomware payment if the executives on the top floor are sweating in their suits!

[Anonymous Coward]

Re: Big problem...

That's nothing. What if that HVAC is to the server room? You want to REALLY make the suits sweat? Threaten to kill the heart of their business by heatstroke...

[Gerhard Mack]

Re: Big problem...

This is why IT needs to segregate devices they don't control from the rest of the network. Devices like that are why we have VLANs and firewalls.

[DS999]

Re: Big problem...

Yes and how is that segmentation implemented? Typically via network devices, many of which have known exploits of their own and may not be up to date with patching. The ransomware guy will surely use his way into the network to probe that device and see if he has any exploits in his bag of tricks to p0wn it and move into the part of the network where the goodies are stored.

Though as stated even if that's secure, or you are smart and such devices are air gapped from the rest of the network, it might be almost as bad for a company to have a server responsible for building security or HVAC taken over as it is to have critical corporate servers on the datacenter floor taken over. Or well it was, back when people worked in the office instead of from home.

[Dan 55]

Re: Big problem...

Is legacy hardware.

We have a shield printer (metal and plastic), the control software only runs on DOS and the thing is attached over a serial cable. We found a "spare" on eBay for a couple of thousand Euros. A new one is around 6 figures. We collect old PCs and keep them in storage, in case the old one fails.

How about a NuXT - modern hardware functionally the same as mid-80s hardware?

Maybe you could also try FreeDOS, which should work on more modern hardware and if you're lucky you can get USB-Serial working on it.

[Charles 9]

Re: Big problem...

Or perhaps something less esoteric, like a modern machine with a USB serial adapter and a Virtual DOS machine running on it? Now, granted this isn't possible if there is custom hardware involved (like that lathe with the custom ISA card), but it can't be all thorns, can it?

[Dan 55]

Re: Big problem...

Unfortunately thanks to Sillicon Valley's ADHD you can't be sure that an update to your VM software or W10 won't screw up your VM or W10.

[Paul Crawford]

Re: Big problem...

You can have timing issues with huge chains of virtualisation like that, and often DOS serial code would use the UART registers/interrupts to work and not the approved BIOS calls (as they sucked and were s..l..o..w..) so any emulation has to be pretty darn close to the actual hardware in operations. More so for odd things that might use non-standard Baud rates that are not emulated by the modern system.

[Charles 9]

Re: Big problem...

Yet things like high-baud USB modems (especially those with internal UARTs) still work in the same circumstances? Would love to see the specifics.

[Pascal Monett]

Patching is essential for a business

Not long ago I read an El Reg forum post from someone saying that, from 1985 to 1992, he only had to patch once.

Sure, in those days, to hack a computer you had to be sitting in front of it.

Since then, this thing called The Internet has grown from a university project to the basic global international communications network that permeates our very lives. There isn't a single electronic device on sale today that doesn't want to connect to it. Hell, they're even making "connected cars", as if a driver needed more distractions from actually driving.

In this kind of environment, patching should be a regular part of a business' procedures. And the sysadmin's job is to determine which patches should apply, not whether or not something should be patched.

[Anonymous Coward]

No ones wants to be responsible for a patch breaking something...

... they prefer to be responsible for a ransomware hitting their whole network, it looks. Than executive becomes " badly affected by the emotional pressures".

My company is "heavily" protecting my company phone using Knox and some kind of "highly secure" apps. This way IT decided applying OS upgrades is too risky. Thereby my personal phone is far more up to date than that. I don't use the company phone for personal stuff, but others do....

[Cuddles]

A situation 20 years in the making

Pretty sure it's been a lot longer than that.

[John Brown (no body)]

When there are big data breaches...

...do the prosecuting authorities take into account the (lack of) patch levels, especially if the breach can be shown to have happened because of a flaw where a patch has been made available and there could be a reasonable expectation that it should have been patched eg within the past year at most , thus stopping the breach from happening?

[General Purpose]

Re: When there are big data breaches...

>do the prosecuting authorities take into account the (lack of) patch levels, especially if the breach can be shown to have happened because...

Bad locks don't cause burglaries. People do burglaries and flimsy locks are no excuse.

Likewise, prosecutors don't say "the breach happened". They say you broke in.

[Terry 6]

Re: When there are big data breaches...

But insurance companies do.

[J27]

No one wants to pay for the effort necessary to properly secure software, so we're all just "doing the best we can". Hard to make money on security features your users never see and don't understand.

[ecofeco]

What a mess

A mess we'll be cleaning up for generations.

[Will Godfrey]

Ancient Kit

There are small companies dotted round the whole world who's entire existence centres on one or two bespoke machines. If the computer fails irreparably, so does the company. They simply can't afford to replace the kit. Fortunately a lot of this has no internet capability. That is until someone actually manages to shoehorn the code into a VM, and tack on some hardware adapters.

These companies are now extremely exposed. You can absolutely predict that someone will find a way to 'just monitor production' to keep the uninformed boss happy. The kit now is living on a knife-edge timing wise. A delay at a critical moment (when said boss tries to do a data dump), means a pulse is missed that should have moved product guides aside before a press comes down.

The cast iron frame of the machine is now shattered, and 50 people have no job. All this is without adding Internet bad actors to the mix. My old boss has seen exactly this situation occur.

The (slightly) more modern establishments are in even a worse position. They rely on the Internet, because the maker of their kit does remote diagnostics only. They have no in-house software guys, and have no way to protect themselves even if they knew how to (or that there could be a problem).

I became aware of this, when fitting a replacement drive to a fully automated timber cutting and forming machine. I had to phone the maker once the drive was fitted. They then sent various data over the 'net to set up the drive while the motor was disconnected, using information stored in the machine itself but not locally accessible (for security reasons HA!), after which I connected the motor, and they performed final static configuration.

I didn't say a word to the woodyard, there was no point. I did have a good old rant when I got back to the workshop.

[Sparkus]

"Technical Debt"

working a project right now whose sole reason for being is to clear a few years of accumulated tech debt (doc'd via Jira) including a massive backlog of security and reliability patches.

As you might understand, we are pushing this project as rapidly as possible to stop the REMFs from diverting us into things like GUI look-pretty cruft (ooh, Dark Mode!), double-byte language support for the lower-slobovian market and so on.

It's not an easy slog.

[Terry 6]

Predates computers

When my late father was working ( and I'm retired) he worked for several companies that wouldn't ever update anything. Old, creaking sewing machines were constantly repaired and used until they broke down again. Even though the cost of lost production and staff time or brought in repairmen to get the things running again massively exceeded the cost of replacement. There was, and from what I've seen, still is, a weird disconnect in some bosses between proactive investment in the business and sweating the kit. Every drop of use has to be squeezed from the system, whatever the cost. From what I've gathered, from what I've heard, in more modern times and larger businesses (small ones just get automatic Windows updates, which is why they aren't maybe as bad a thing as we usually think- at least until they go out of support) this is echoed in a kind of mentality that won't allow the computers to be touched. A view that says "If it ain't broke don't fix it" - that doesn't accept that a vulnerable system is already broken.

It's not about software, or even hardware, it's about management.

[Anonymous Coward]

Re: Predates computers

Some of the problems are 'accounting departments' and 'outstanding shares', and CFOs. If you walked into GE (medical division) and asked whether their software for communicating with a 10yo piece of $1M hardware would run on WIndows 10, and they said no, it will only run on WIndows 7, would shooting them on the spot fix the problem? Maybe, but not quickly.