back to article CISA and friends raise alarm on critical flaws in industrial equipment, infrastructure

Fifty-six vulnerabilities – some deemed critical – have been found in industrial operational technology (OT) systems from ten global manufacturers including Honeywell, Ericsson, Motorola, and Siemens, putting more than 30,000 devices worldwide at risk, according to private security researchers.  Some of these vulnerabilities …

  1. Pirate Dave Silver badge
    Pirate

    Hmm

    Funny thing - we were just told a few weeks ago by our Cyber-insurance carrier that we need to protect our Industrial network from our "office" network. The Industrial net needs to be completely and securely isolated from the rest of the network, and preferably from the Internet as well. Isolation from the local net isn't easy, since the PLCs need to send a sizeable stream of data to the SQL servers over in the office network.

    Judging from this story, that may be holding the tiger by the wrong end. That, or it's a subtle push to move our SQL servers (or maybe ALL of our servers) out to "the cloud".

    1. DS999 Silver badge

      Re: Hmm

      Wait, the requirement is that it "must" be isolated from your office network but it is only "preferable" it be isolated from the internet? Surely if it must be fully isolated from the office network it must also be equally isolated from the internet!

      I didn't see it in person, but I recall an architecture diagram for a enterprise network that had an isolated network that was connected via a fiber with the receive line physically cut. It could get data out via UDP, but nothing could get in. I believe they used some sort of verbose LPDC coding like satellite transmissions use for error tolerance since there wasn't any way for the receiver to report missing data (though if your network is solid this shouldn't be much of a concern)

      If the PLCs won't send their data via UDP, you'd just need a collection server inside the isolated PLC network that would handle the TCP/IP connections to the PLCs, and be the UDP source to the SQL servers in the office network.

      1. Headley_Grange Silver badge

        Re: Hmm

        I assume it’s because the bigger threat is employees either by phishing or other nefarious activities.

      2. Pirate Dave Silver badge

        Re: Hmm

        No, isolation from the Internet is not required, only isolation from our office network, and no in-bound connections allowed from the Internet. Seems strange to me, too, but that's how they want it to roll. We have until August to get that part straightened out.

        The bigger PITA is that they told us in mid-May that we have until the end of June to fully implement MFA for Office365 and anything on our office network that's accessible from the Internet (primarily our VPN and our AS/4000). Other than the Admin account I use to twiddle Office365, we have no MFA enabled on anything anywhere else. So, errr, 6 weeks to spin all that up is easy, right? No, it's not. We won't he hitting the target date. Hell, I doubt we'll even have it setup and running by then, much less have 500+ employees' phones enrolled. Gonna be a nightmare, but I guess we'll be "safer" from phishing afterwards.

        It's definitely a sea-change with the cyber-insurance. I don't think they're playing the odds that we won't be hit anymore. I think they're assuming we will be hit, so they're just trying to mitigate and minimize the damage.

        1. Claptrap314 Silver badge

          Re: Hmm

          In order to ensure 100% isolation from your corp net, you either need 100% isolation from the internet or 100% isolation between your corp net & the internet.

          You might want to bring that up to your boss...

        2. martinusher Silver badge

          Re: Hmm

          Seems reasonable to me. Industrial systems not only use a lot of Windows but by necessity they can't be updated every five minutes like the office workers' PCs. They often have specialized tweaks to the OS and,anyway, their applications need to be tested, retested, certified and generally run by the rule "if it works don't mess with it".

          The business network is more of a free for all even if it is managed by a competent IT department. Therefore machines on the business network may be able to look at data but that's it -- its read only or nothing, and preferably "read only through a carefully managed connection".

          So the problem these people face is not bleating about how thousands of systems have level whatever vulnerabilities -- everyone knows that. The trick would be coming up with guidelines as to how they might be used safely in a modern connected environment.

          1. Claptrap314 Silver badge

            Re: Hmm

            "safely" "connected network"--I'm not certain that you fully understand each of these terms...

    2. Doctor Syntax Silver badge

      Re: Hmm

      That, or it's a subtle push to move our SQL servers (or maybe ALL of our servers) out to "the cloud".

      Certainly not to the cloud. Put the servers for the plant onto the plant's network, separated from the office network. The tricky bit comes when you need to push reports back from the plant to the office.

      1. Pirate Dave Silver badge

        Re: Hmm

        Yeah, somehow that data has to get into our dataflow so it winds up in reports that Manglement sees. We're still not sure how this is going to work out. Cloud is one way to be "compliant" with their requirements, sort of like an inside-out DMZ - the PLCs send their data out to a SQL server in Azure (as an example), and our office stuff pulls that data out when needed and shoves it into the dataflow. That's my (very rough) idea as a non-industrial network admin. There are also services like Ewon that are more specifically aimed at industrial control networks. Or, well, sneaker-net with a CD burner as a last resort. Or, hmm, is Laplink still a thing? A serial connection isn't technically a network connection...

        1. jake Silver badge

          Re: Hmm

          "A serial connection isn't technically a network connection..."

          Excuse me? A serial connection is very definitely a network connection. So is everything than can be loosely grouped together under the mantle of "sneaker net".

    3. thames

      Re: Hmm

      You need to find out what they mean by "isolate", whether it's an actual air gap (probably not) or just separate networks (more likely).

      When you say "the PLCs need to send a sizeable stream of data to the SQL servers over in the office network", is it the PLCs which are talking directly to the database servers, or is there one or more PCs in between which runs software which polls the PLCs and then writes to the database servers? I suspect it's the latter.

      Alternatively, it may be the PCs which are the thing which needs to be isolated from the office network, especially if they are running some sort of SCADA or HMI software which is collecting the data as well as doing it's main job.

      In either case you may need some sort of firewall and proxy server between the industrial network and the office network. Think of it as being a simplified version of what is used to isolate the office network from the Internet (I'm assuming you are doing this). That way an attacker who gets into the office network doesn't have direct access to the industrial network (the thing which is connected to the part of the company that actually makes money).

      If you google for "scada firewall proxy server" (or something like that) you should be able to find plenty of examples.

      1. Pirate Dave Silver badge

        Re: Hmm

        They don't require air gapping, just no routing or NATting directly between the two networks. As it stands now, we can use the same firewall for both networks, we aren't (yet) being required to have physically separate firewalls for each (I guess they need to save something to suggest/complain about next year...)

        Some of the PLCs send data directly, we also have a few RedLions that aggregate and send data from other PLCs. I think the engineers have also put in something even newer for some machines in the past 3 or 4 months, but I haven't laid eyes on that equipment yet.

        1. jake Silver badge

          Re: Hmm

          "They don't require air gapping, just no routing or NATting directly between the two networks. "

          Out of curiosity, who are the networking rookies making these hair-brained decisions? I'd like to avoid them, if at all possible.

          1. Pirate Dave Silver badge

            Re: Hmm

            I don't know, the boss just refers to them as "the cyber-insurance carrier". I haven't had to deal with them directly, although I did help him fill out the annual audit form in years past, before all of this became so serious and they started taking a more active, dictatorial role.

      2. Bitsminer Silver badge

        Re: Hmm

        you may need some sort of firewall and proxy server between the industrial network and the office network

        This.

        It will amount to renumbering your PLCs and related equipment onto a new subnetwork. Perhaps you can add a second NIC to your Windows SQL server (assuming just one) to be on this new network (assuming it is not also an AD DC -- that isn't allowed.)

        Your Windows machines have firewall features, so you should use that to control access between the two networks -- such as "none from here can go there"....but "that one is allowed to talk here...."

        Then you can think about a third item called a firewall which also implements most of the same firewall rules but which has a big alarm bell that rings whenever any are violated. The notion here is that the Windows firewall implements Policy and the physical firewall implements Policing.

        You mentioned your factory engineers who work on adding equipment. They need to be educated and directed by management to implement "network security" on adds/moves/changes to the network. Otherwise you'll be constantly chasing after them and fixing things after the fact. Perhaps after your next intrusion....

        (Scare quotes around network security because your organization will need to identify a policy that defines "network security", e.g. PLCs can be on the same LAN as other PLCs. Or not. Which in turn depends on a review of assets at risk and mitigations....all that good stuff.)

  2. Paul Crawford Silver badge
    Facepalm

    Additionally, the security shop recommends isolating OT and industrial control systems' networks from corporate networks and the internet when possible.

    Should that not have been the case from day #0?

    1. Doctor Syntax Silver badge

      Ideally, yes. However the controllers may well have been designed before anyone started thinking about such things. For the last decade or more, however, the default assumption should have been that such equipment was inherently vulnerable and should be isolated.

      1. jake Silver badge

        Many decades ago, actually. Try connecting to the gear that monitors The Beam at SLAC, for example. Or the controls for the Stanford Dish. Or San Francisco's Hetch Hetchy water supply. Or rather, don't bother. You can't. Grad students wanted to hook 'em up to the 'net back in the late '70s or early '80s; the sane among us put the kibosh on their plans.

        Commercial interests of today, however, are truly insane. We tugged on their capes, and were shrugged off. We tapped 'em on the shoulder & were elbowed away. We tugged on their shirts, and were thrust aside. Some even kissed their boots, and were trodden upon. Our message was always the same: "Please, PLEASE, **PLEASE!!** don't connect SCADA kit to publicly available networking systems!"

        But did they listen? No. They did not. The idiots.

        On the bright side, those of us with a clue are making a pretty penny in our retirement, cleaning up the resulting mess :-)

        Yes, I know, I've posted this or similar before. It's still accurate.

    2. Kevin McMurtrie Silver badge

      Yes, but then real-time monitoring, integration with change control processes, access for manufacturer diagnostics, ... Securing a bridge can take more technical skill than is available in the office.

  3. Pascal Monett Silver badge
    Coat

    "Fifty-six vulnerabilities [+4] – some deemed critical – have been found"

    I think the hackers already know . . .

  4. Claptrap314 Silver badge

    This is the Sonos problem

    I interviewed with Sonos a few years ago. Spoke with their head security guy. Their original security model was, "don't plug in to other computers". Yeah, customers didn't do that. So, they updated to "don't plug into the internet". I think you know how that went.

    The difference of course, is that Sonos is providing entertainment.

    There is no way in 4377 that these systems should be anything but air-gapped to the public internet. Any any connections between systems must be carefully analyzed for absolute need to connect by security experts. The hw for these installations tends to start around $1B, and goes up (way up) from there. Security is a core requirement. Pay up.

  5. Ribfeast

    I remember working for a water/sewer provider, and their SCADA network was fully separated from the main corporate network. With only a few ports open between the networks for status traffic to get through from the collector probe in the SCADA network across to the central NOC in the main network. Everything was locked down tight.

  6. Daedalus

    PLC hell

    A PLC project and its priorities.

    "The XML output isn't formatted for readability"

    "It's XML. Use a reader, or just dump it into any web browser."

    "We use Notepad++"

    "Which has an XML Reader add-on"

    Marketeers and sales droids. I wish they'd stick to what they're good at. Whatever that is.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like