back to article When the bits hit the fan: What to do when ransomware strikes

When I first became a company chief techie, the finance director patronisingly explained the basic asymmetry of prevention vs cure. Spending money on assets to stop an attack come out of capex, but spending after the disaster would be up to the insurer, with premiums deducted out of opex. Also, prevention costs reduced current …

  1. alain williams Silver badge

    Did Mr Connor ask the finance director

    if his patronising explanation could be quoted to the press if/when something went wrong ?

    I suspect not as it would be career limiting as most of the board would want to maintain their current bonuses. However the chief techie will take the rap even if he was not allowed to do the right thing.

    But prevention is the best way out. It might cost a bit but how much will shares & bonuses be hit when an attack happens ?

    1. Dominic, Writer of this aritcle

      Re: Did Mr Connor ask the finance director

      Good point, but FDs are lamentably poor at grasping cyber risk.

    2. Spamolot

      Re: Did Mr Connor ask the finance director

      > Did Mr Connor ask the finance director if his patronising explanation could be quoted to the press if/when something went wrong?

      Get it in writing. First rule of Cover Your Ass...

      1. Dominic, Writer of this aritcle

        Re: Did Mr Connor ask the finance director

        Absolutely.

      2. Naselus

        Re: Did Mr Connor ask the finance director

        Yes, it's remarkable how effective the phase 'And can I get that in writing?' is at making senior managers suddenly take something seriously.

        1. Anonymous Coward
          Anonymous Coward

          Re: Did Mr Connor ask the finance director

          An old boss of mine who was a grand master of weasel wrangling, did his critical emailing at night and dawn to increase the chances of spontaneous and ill considered replies.

          When the shit hit the fan, the weasels were skinned by their own paper trail.

    3. Anonymous Coward
      Anonymous Coward

      Re: Did Mr Connor ask the finance director

      I'm sure there is a lot of cringing going amongst cybersecurity folk over the terminology being used here.

      There is no such thing as prevention, only mitigation.

      You can't prevent 0days. Period.

      You can mitigate them by not exposing too much of a surface or oy allowing exposure if it is completely unavoidable.

      For example, if I wanted to prevent VPN attacks. I would have to completely disable VPN services. You can't attack what isnt there.

      However, if I can't avoid providing VPN services to my users, I can only calculate risk and offset that risk or mitigate the risk by protecting against vulnerabilities that are already known. That doesn't prevent attacks though, because some unpatched vulnerability could become apparent that isn't possible to protect against. This is why we have security advisories and vulnerability disclosures. So that as soon as a bug is known mitigations can be rolled out (if at all possible).

      Once again, switching something off does not prevent attacks, it is just a DoS on yourself.

  2. Mike 137 Silver badge

    Excellent argument, but only part of the picture

    "Spending money on assets to stop an attack come out of capex, but spending after the disaster would be up to the insurer, with premiums deducted out of opex."

    Very true if you take the technocentric approach to security. However, in some 20 years of security consulting I've found the most prevalent problem that leads to or facilitates data breaches is not lack of appropriate tech kit but abysmal security management. Just for example, Equifax [1] having acquired numerous third party services via mergers and buy-outs, didn't have an applications inventory. So when the critical vulnerability was announced (with sufficient notice) they couldn't find the vulnerable system. [2] The attackers found on the network a list of access credentials in clear for critical servers on the same network. The list goes on...

    Improving security management doesn't cost capex, but it's hard because lousy management results from cultural flaws intrinsic to the enterprise. However it's almost always the most important starting point for improvement of security.

    1. Dominic, Writer of this aritcle

      Re: Excellent argument, but only part of the picture

      Security and resilience costs Capex and running costs, though the mix can be varied to optimise share price and bonuses.

  3. elsergiovolador Silver badge

    Opportunity

    When ransomware wreaks havoc, it's a great opportunity to rise from that like a phoenix - independent and more resilient.

    By adopting Linux across the company.

    The only reasons, in my opinion, why Windows is widely installed across corporations are kickbacks, minesweeper and solitaire.

    Now that more people play games under their desk using their phones, Microsoft wants to lure them in back to desktop with ability to run Android games on Windows 11.

    It's going to be another golden era for procurement. All that hardware that needs to go, so it can be replaced to meet W11 specs...

    1. the spectacularly refined chap

      Re: Opportunity

      When ransomware wreaks havoc, it's a great opportunity to rise from that like a phoenix - independent and more resilient.

      Really? The business is probably paralysed and in the midst of that chaos you want to do an unplanned company wide migration at the same time? The key is restoring as much capability as possible as quickly as possible. Instead of perhaps two or three weeks of noticeable disruption (probably followed a long tail period for the less critical stuff) you want to put everything on hold for what? 18 months? Two years?

      I spent three months this summer upgrading a dozen servers from NetBSD 7 to ... NetBSD 9. Not full time of course but as migrations go that's quite straightforward. Pouncing on something like an attack to push a personal agenda in fast way of getting yourself ignored or even maneouevred somewhere you can't do any harm.

      1. Dominic, Writer of this aritcle

        Re: Opportunity

        You have had a better life than me. You seem to think it insane to do a migration in the midst of a crisis, well

        a) as a CTO in the midst of them it has been demanded of me

        b) one *VERY* important database got ported from Oracle to SQL Server during the crisis because it was the only way I could fix it quickly. It was a bit of a SciFi moment for me.

        I "knew" like Spock or McCoy or House MD that a certain "brave" tech decision would make things better, but knowing and doing are quite different. So I pressed OK and went and had a coffee whilst my laptop which was about to become the server had a good hard think. This was because I both needed caffeine and also to project absolute confidence that the ugly fix would work.

        Which it did. First time.

        You and I both know that whereas this happens in the last scene of SciFi quite a lot, but the reality is a lot more messy and success more equivocal.

        There was two consequences. A polite but difficult conversation since it was my personal laptop, hence it had extra tooling not to be found on most corporate PCs and also a shed load of money for a permanent fix.

    2. Dominic, Writer of this aritcle

      Re: Opportunity

      Windows is more vulnerable than Linux, but it is naive to believe Linux makes you safe.

      1. deadlockvictim

        Re: Opportunity

        Surely the quality & experience of the sysadmins & DBAs running the hardware is more important than the software choice.

        That is, I regard a system set up and run by experienced & capable Windows Server sysadmins to be safer than a Linux system run by less experienced or less capable Linux sysadmins to be more secure and vice versa. Windows Server can be made secure if you know what you are doing.

        Now, you may very well be right that a well-run Linux network is more secure than a well-run Windows network, if all variables are equal bar the systems software.

        It could simply be snobbery on my part but I suspect that Linux sysadmins tend to be more devoted to the cause of systems administration and deal with larger, more valuable networks than those of Windows Servers sysadmins and I am open to correction on this point.

        1. Dominic, Writer of this aritcle

          Re: Opportunity

          Yes, but more important still is how they are managed.

          I comment to you the report on the Space Shuttle Challenger Disaster and in particular the dissident appendix by Richard Feynman.

          The short version is that the "physical" engineers were given the job of showing that their parts worked.

          It never even occurred to the computer engineers that their parts worked.

          So one engineer who pointed out the flaw that caused the Shuttle to exploded was told to "stop thinking like an engineer and start thinking like a manager". That's a quote and the rest is horrible history. People who found problems were trouble makers.

          The IBM guys took the opposite approach, someone who found a bug or bizarre improbable state was seen as doing what Gus Grissom would have called "good work"

          The difference is stark.

          The computers of the Challenger kept on working after the explosion, then falling from near orbit, then for a while after it hit the water . Some of what they reported is apparently still classified.

          Earlier incidents include a massive over voltage melting one of the circuit boards with the result of small lumps of solder floating in microgravity causing a constant stream of random short circuits.

          Yeah, that didn't bring the system down either.

          ITpros who bring up problems aren't rewarded, often denigrated which is why in my survery 22% of them have been affect by a ransomware attack.

      2. bombastic bob Silver badge
        Linux

        Re: Opportunity

        it is naive to believe Linux makes you safe.

        When you apply the correct security-related thinking to setting up a Linux server, it's pretty frickin' solid as far as security goes.

        SO yes, and no. YES it is naive, so you need to hire a Linux admin (or consultant) who knows what he's doing, and then you should be as prepared as you can be for any kind of malware storm

        Uber-security might involve putting the server and data into a VM, and then have the host machine buttoned up tighter than a bullfrog's behind (and host the recent backups, with offsite storage for the older ones).

        There are a LOT of things you can do with Linux (security-wise) that aren't so easy in the windows world, and they are VERY effective. But yeah, it cannot be set up by an IDIOT or you'll be pretty bad off when the storm hits.

    3. vtcodger Silver badge

      Re: Opportunity

      Also keep in mind a good deal of the software businesses depend on is Windows only. Yes Open Office (or whatever we're calling it this month) works fine and even (I'm told) runs some Excel macros nowadays. And yes MS support for it's products is at times a bit wobbly. And their QA is rather ...ahem... problematic. But a lot of stuff -- likely including mission critical software isn't available for Unix and probably won't run under Wine without a daunting amount of tinkering. Unix is probably a non-starter for most businesses.

      BTW, the finance folks who would probably need to approve the funding for the switchover often understand Excel macros and use a lot of them. All the time. They will surely be less than enthusiastic about a world without MS Office. And their managers won't be wild about a world without Power Point.

      Now a new operation with no dependence on some sort of special software that everyone in their sector uses? THEY probably ought to seriously consider Linux for a lot of reasons -- including security.

      1. Doctor Syntax Silver badge

        Re: Opportunity

        "But a lot of stuff -- likely including mission critical software isn't available for Unix"

        Let's look at that one. Storing the day-to-day data is fairly mission critical. The sort of stuff for which you use an RDBMS. There have always been Unix packages for that. Microsoft started development of SQL Servier from one of those, Sybase.

        Perhaps a good start would be to use one of those for the main business database(es) and have an isolated system to test the backups, just in case.

        1. Dominic, Writer of this aritcle

          Re: Opportunity

          Yes, that is the gold standard for resilience, different applications written in different languages running on different platforms on different hardware.

          Not cheap.

          1. Doctor Syntax Silver badge

            Re: Opportunity

            Not cheap?

            Back in the day small businesses quite frequently ran on Informix/SCO/Intel tower server combinations. It was cost effective enough for them then.

            I've no idea what the cost comparisons would be with Windows stuff today but in any case you could take the SCO out of that and replace by Linux or BSD You could take Informix out and replace by PostGres or MySQL. Or your could replace them by Microsoft products, only the server remains constant. Which is cheapest?

          2. FIA Silver badge

            Re: Opportunity

            Not cheap

            Not initially no. However, disparate systems force you to consider the boundaries much more carefully, which results in much easier systems to upgrade and modernise as needed.

            If that also improves security to then that’s just an added bonus.

            Unfortunately the financial argument for fully integrated systems was often too great and the downsides only visible retrospectively. (Oracle forms has a lot to answer for…)

      2. bombastic bob Silver badge
        Linux

        Re: Opportunity

        accountants probably edit (or at least copy to) files on a share. Then, malware ANYWHERE on the network that has write access to that share can pooch it.

        A Linux server could make and store backups of the share using a cron job and store them where windows machines cannot (easily, or even remotely easily) access it. THEN, you restore the latest backup after you clean the malware off of the various computers, and go from there.

        And proper directory-level and file-level security would help to keep EVERY windows computer from having write access to those shares.

    4. MJI Silver badge

      Re: Opportunity

      Corrected for you

      The only reason, in my opinion, why Windows is widely installed across corporations is the WIN32 API

    5. bombastic bob Silver badge
      Linux

      Re: Opportunity

      By adopting Linux across the company.

      At least some of us agree with you in principle, and enough evidence exists that the changeover WOULD save you money. Convincing every employee in a large company to switch to Linux desktops might be difficult.

      HOWEVER, on the server end you'll have better luck. You could, for example, do automatic daily (compressed tarball) backups of a data share, with weeks' of history and frequent-enough offsite mirroring (so you can go back > a year if you have to) and at least protect important data assets THAT way. And unless the Linux machines were set up by an IDIOT, the likelihood that a windows malware application COULD affect the servers (aside from pooching data on shares) is SMALL.

      (important note, non-SMB-shared directories will effectively be shielded against malware attacks from windows systems, assuming an IDIOT did not set up the servers)

      Unfortunately someone may some day figure out how to crack Linux security, and the "inside job" is ALSO not protected against, at least not completely.

      But with a good BOFH and properly configured LINUX or BSD servers, you'd have a MUCH better chance of weathering a malware storm. In My Bombastic Opinion of course.

  4. MJI Silver badge

    To stop it

    Almost all are Russian gangs.

    Need to snipe Putin in the leg then release a statement.

    Stop the ransomware or the next one will be a little higher.

    It would stop.

    1. hammarbtyp
      Mushroom

      Re: To stop it

      Well I guess that's one solution, although I can a few issues....

  5. js6898

    backups

    not sure I completely understand - if you do a backup to eg tape every so often, and don't re-use that tape then this provides a backup surely? Just as long as you flip the read-only switch when you restore.

  6. Adelio

    Can someone shed some light on a question i have.

    Ignoring them copying data how do they go about encrypting the data?

    Specifically, a SQL server database is permanently locked by the server so how would thieves encrypt it without bringing the server down immediately.

    Also many "files" are is use daily so as soon as they are encrypted they would be unreadable, that would raise some red flags.

    One way to protect again this i think would be to "log" details of all the files in the system and then anytime they change do a comparison to see if they are still ok. (a little bit of magic here, but i presume that any file that is encrypted looks different to a normal file)

    Might be slow but i assume that in a mature system there are probably not many files that are "changed" on a regular basis compared to the total number of files.

    1. bombastic bob Silver badge
      Pirate

      Specifically, a SQL server database is permanently locked by the server so how would thieves encrypt it without bringing the server down immediately.

      It could be done by first dumping all of the data into an encrypted file, THEN issuing "DROP TABLE" and other commands on the actual database. Restoring would do "CREATE TABLE" and "INSERT" commands (hopefully along with indexes and stored procedures and so forth)

  7. SammyB

    Re: Did Mr Connor ask the finance directorl

    When communicating with higher ups regarding a situation where you feel it is ethically, professionally or morally wrong, always follow up with an email repeating the conversation stating your position and followi with a question asking how the other would react if/when the you know what hits the fan and the results is publicly published. Had to do that once, I was never asked to to implement a specifically requested process ever again.

  8. MJI Silver badge

    I have been reading up on malware

    It seems it works by making itself admin then terminating a lot of processes including most major SQL server processes.

    Then it gets to work on the data.

    Some of this ransomware is rather scary in what it can do.

    What a waste of programming talent.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like