back to article Why a top US cyber spy urges: Get religious about backups

Not all defense tech is bleeding-edge cyber — or kinetic — warfare tools. Sometimes the best defense is as boring as ... backups. Yes, backups.  The rest of the world should take this lesson learned from the Russia-Ukraine war to heart, said Rob Joyce, director of the US National Security Agency's cyber security arm, speaking …

  1. Will Godfrey Silver badge
    Facepalm

    But, but...

    Profits!

    Need I say more?

  2. b0llchit Silver badge
    Holmes

    Good backup is expensive

    Wasn't it an IBM redbook that specified: "When planning backup, start by adding and hiring a backup team to be backup for your backup team."?

    With the current climate of reduced spending, what are the chances of structural backups?

    And, besides, the beancounters will look at the monthly bills and complain all the way up to the board that huge amounts of money are used on that backup project that has not had any ROI during its existence. The C-suite will surely act accordingly to improve money flow back to investors.

    1. Doctor Syntax Silver badge

      Re: Good backup is expensive

      It's the board that needs to get the message first. Then they can kick the complaints all the way back down the ladder. There does seem to be an inkling in govts that critical infrastructure is - well - critical. They might even be getting insistent about it. We can only hope that they work out PDQ what they need to insist on.

    2. DS999 Silver badge

      Re: Good backup is expensive

      You don't need to hire people to be the backup to the backup team. You just need to cross train a couple people from other teams - and make them actually perform the processes often enough they retain their knowledge.

      This doesn't matter for a really big organization that might have a half dozen people on their backup team, you won't have enough illness and attrition that they disappear. But most places have a backup "team" that's one person, or maybe two. Or one or two people where it their part time job. The chances that one is on vacation and the other gets sick or quits should worry management enough that a couple more people get cross trained in their skills.

      The biggest risk is probably the smallest organizations - where you have only one or two people as the entire IT "team".

      1. Orv Silver badge

        Re: Good backup is expensive

        Yup. I have trouble just going on vacation because I'm the only IT person and there's no one to back me up.

        1. DS999 Silver badge

          Re: Good backup is expensive

          I hope when you find another job you give them the minimum notice possible. They need to learn their lesson the hard way, so whoever has your job next doesn't have the same problem.

          1. Orv Silver badge

            Re: Good backup is expensive

            Yeah, well. This is academia, where having a full-time IT person at all in a department is seen as dangerous overspending.

    3. Anonymous Coward
      Anonymous Coward

      Re: Good backup is expensive

      Good backup isn't expensive, there are PLENTY of vendors out there now who can do an excellent job to backup your environment.

      I've been amazed at how many places I've worked where backup, even by the IT teams is seen as boring, not important, "we don't care about that because we've got failover" etc.

      The "we don't need to spend money on this" mob are quick enough to come screaming for their files when they lose them though....

      and you DO need to back the cloud up! Trusting MS or Amazon or Google is NOT a backup strategy

  3. Anonymous Coward
    Anonymous Coward

    Hot Off The Interweb......Stuff You Really Need To Know!!!

    Quote from "expert": "You don't want to find out in a crisis that your backup process didn't work"

    He'll be saying next "....and you need to ensure that you keep offsite backups too....."

    And then he will add even more value by saying ".....and of course, you need to do a test restore immediately after taking a backup, so you know the backup is OK...."

    Yup......you can always rely on "experts" to tell you things that you and your colleagues have known for years...........

    .......and you can rely on the media to transmit the "expert advice".......breathlessly......because it's "hot news"................."new stuff"........"stuff that you REALLY need to know"..........

    Please.......give us all a break!!!!

    P.S. Next time round we'll get a puff piece for "cloud" backups......Please, again, give me a break!

    1. Orv Silver badge

      Re: Hot Off The Interweb......Stuff You Really Need To Know!!!

      Sure, it's info "everyone knows." But how many actually do it?

      1. ThisIsTrue

        Re: Hot Off The Interweb......Stuff You Really Need To Know!!!

        Exactly. We've seen HOW many reports about hospitals, municipalities, etc. etc. etc. having to either pay a ransom or rebuild their systems from scratch? This article is simply another clue-by-four between the eyes. So you're damn right I'm circulating this URL: it's short and understandable even to the obliviots who don't have IT backgrounds.

  4. Potemkine! Silver badge

    Backups are the first step, testing them regularly is the second one.

  5. Anonymous Coward
    Joke

    Free secure data backups! Now!

    And that is why we at the NSA will be offering free back-up services, for the Enterprise and Consumers alike. Completely free of charge!

    Actually, the service has been retroactively running for the past 20 years! No matter what, a copy of your data is safe with US!

    1. b0llchit Silver badge
      Coat

      Re: Free secure data backups! Now!

      But restores are such a legal hassle.

    2. HereIAmJH Silver badge

      Re: Free secure data backups! Now!

      I know you're joking, but if the government gave everyone a TB of storage and encouraged Microsoft to add it as a free 1-click feature in Windows, I think you'd be surprised how many people would use it. Free Storage!! Some people would encrypt, but most wouldn't bother because it would be too much work. Probably the only people to complain would be cloud storage providers.

      Of course there would be 'legal protections', just like Google would never do anything inappropriate with your GMail account.

      1. Ken Moorhouse Silver badge

        Re: ...encouraged Microsoft to add it as a free 1-click feature in Windows

        Fine until there is a need to restore it.

        MS has form when it comes to Backup.

        ISTR that the versions of msbackup between v6.21 and v6.22 were incompatible, which meant having to copy the appropriate msbackup.exe to the system you wanted to restore to. And because you couldn't run the v6.21 version on a v6.22 system or vice versa, setver had to be invoked to fool it into thinking it was the correct version.

        (Not the sort of minutiae that you need to be thinking about when your data has gone awol).

        1. Roland6 Silver badge

          Re: ...encouraged Microsoft to add it as a free 1-click feature in Windows

          MS aren’t the only ones with form…

          I would avoid the backup tools OEMs bundle, they are great at creating backups, but try and restore in the event of a system failure and you will hit the problem, you need a working system from the same OEM with a compatible version (ie.one that can read the archives) of the bundled utility. Lesson learnt from a Thinkpad running XP, who’s HDD had failed, my replacement (W7) laptop to hand was from Dell…

      2. Doctor Syntax Silver badge

        Re: Free secure data backups! Now!

        I take it you're only suggesting it be free with Windows because Windows users are most in need of backups?

        1. Roland6 Silver badge

          Re: Free secure data backups! Now!

          Don’t give MS idea’s: new feature in W12, windows backup automatically on and backs up to OneDrive…

    3. pip25
      Devil

      Re: Free secure data backups! Now!

      It's scary, but I can actually see a market for this.

      "If you lost some precious company or personal data, give us a call. There's a possibility that we may have ACCIDENTALLY backup up your stuff in the recent past, and are willing to return in to you for a small fee."

  6. Andy Non Silver badge
    Facepalm

    Why do so many people overlook backcups?

    I know they are second nature to those of us who have worked in IT for many decades, but the message doesn't seem to filter down to individuals. There are regular desperate moans from folks saying they've lost their entire PhD thesis or twenty years of photos or messages because their phone or computer malfunctioned or there was a glitch in their cloud service. They always blame someone else for their lack of foresight and lack of effort in backing up their personal data. External USB storage is so cheap. Keeping several backup copies of your data is easy. Ideally off-site (maybe with a family member) or at the least a copy in an outdoor garden shed or garage, suitably encrypted in case of burglary.

    1. Yet Another Anonymous coward Silver badge

      Re: Why do so many people overlook backcups?

      >Keeping several backup copies of your data is easy.

      But having a spare mirror system to restore them to isn't. Especially in a corporate environment, especially when you want to back-up router and firewall configs and domain controllers and other stuff.

      Running verify to re-read the tape is easy, practising a bare metal restore and recover - isn't

      1. Anonymous Coward
        Anonymous Coward

        Re: Why do so many people overlook backcups?

        Virtual machines.

    2. Ken Moorhouse Silver badge

      Re: Why do so many people overlook backcups?

      Getting your data back involves some sourcery ;-)

  7. VoiceOfTruth

    The USA, religion, and backups

    I thought the whole point of religion is that no backup is necessary, you just need to believe. Rather like those "good God-fearing Christians" who won't get covid because they believe in the words of some people whose names are unknown writing centuries after a supposed man/son of God/Holy Spirit which supposedly happened.

    How many takers for my new company: 666beelzebubbackups.com.

    1. matthewdjb

      Re: The USA, religion, and backups

      I must be a bad God fearing Christian. I've had all the jabs and COVID. But then, I'm not American.

    2. Orv Silver badge

      Re: The USA, religion, and backups

      Yes, but have they considered post-rapture business continuity? You need at least one person on call who has sinned against the Holy Spirit and can keep the business running during the time of tribulation.

  8. ChoHag Silver badge

    > And in addition to having backups in the first place, "think about the practical step of checking your backups,"

    NO!

    This is not an addition! This is THE critical part of taking backups. If you don't do this you're just masturbating tapes.

    1. VoiceOfTruth

      I've been in this boat before with somebody telling me how good their backups are. They had the reports that all the jobs were complete, etc. Tried restoring? Oh, it doesn't actually work properly. It mostly works, but then there are hours left to fix up what is broken. I've seen $boneheads making tar balls of mysql data directories when the db was live. Yeah.

      1. Roland6 Silver badge

        Also need to be making the correct backups.

        BackupAssist for example supports two types of backup: one that is good for individual file recovery and one which should be used for backing up VMs so that they can be restored as work8ng machines on another platform.

    2. This post has been deleted by its author

    3. PK

      Schrodinger's Backup: The condition of any backup is unknown until a restore is attempted.

  9. Paul Crawford Silver badge

    The companies that are best equipped to deal with a breach already have implemented security basics including two-factor authentication and vulnerability scanning

    Which is only tangentially related. You would rewrite that as "folks who care don't do dumb shit" or similar, the key point is to have a safe verified copy of your data, ideally at another site. And said data is not modifiable by anyone on the first site, so no shared admin passwords (or AD entries) for file system snapshots, tape machine control, etc.

  10. Anonymous Coward
    Anonymous Coward

    .. and drop the most unsafe platform on Earth

    Restoring crap means you still have crap..

    1. matthewdjb

      Re: .. and drop the most unsafe platform on Earth

      You mean Mac OS?

  11. Mike Lewis

    Backups are important

    I returned from a holiday to find that the system administrator had wiped my hard drive and the backups. He said he thought I had left the company. Fortunately, I had backups on computers in other cities that he didn't know about.

  12. amacater

    VMs, containers and "it'll just work"

    VMs and containers are the new frontier of things to bite you, I suspect. Chucking stuff into a VM, then regarding it as vital - now you've got another 10GB or whatever to snapshot and backup regularly.

    VMs as cattle not pets - now you've got the problem of knowing *which* VM. Chucking it into the cloud and adding S3 storage or equivalent - did you remember to specify and pay for backup of that???

    Creating an ephemeral VM - easy - I can do it 20 times a day. Maintaining a VM ...

    And that's before you decide to base your ecosystem on Docker or similar: "I can't be bothered to build locally, I'll just pull down the latest from Dockerhub, sight unseen ... it'll be fine."

    Likewise depending on GitHub repositories to be there ...

    if you *are* relying on VMs - document the process for rebuilding them, go through and do that once in a while: archive the metadata in version control. For the stuff you care about, archive *all* the software that you need and build in SPDX or similar so that you know which bits depend on which, not just for your VM but also for software bill of materials ...

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like