back to article SSD missing from SAP datacenter turns up on eBay, sparking security investigation

An SSD disk missing from an SAP datacenter in Walldorf has turned up on eBay, leading to a security investigation by the German software vendor. According to sources close to the incident, four SSD disks went missing from SAP's Walldorf datacenters in Baden-Württemberg, southwest Germany, in November last year. One of the …

  1. Danny 14

    Thats ok because they use self encrypting drives right? So a person with the drive cant use the data on it correct?

    1. Lee D Silver badge

      Quite.

      I'm intrigued how they were able to identify the drives as belonging to SAP otherwise... they must have been unencrypted or else they'd just look like a bunch of random data at best, or not even let you see the drive at all (if you have ATA-level encryption on).

      I suppose in theory they could have connected them to a system and supplied SAP drive encryption credentials and then seen data that nobody without those credentials could have, but it doesn't sound that way. It sounds like plain-text to me.

      1. Anonymous Coward
        Anonymous Coward

        "I'm intrigued how they were able to identify the drives as belonging to SAP otherwise... they must have been unencrypted or else they'd just look like a bunch of random data at best, or not even let you see the drive at all (if you have ATA-level encryption on)."

        Hard to know exactly.

        I'm assuming these were Enterprise high-capacity SSDs (so not the "common") made by a vendor that SAP used. It is possible the particular model/models of SSD were only sold to a limited number of enterprise customers and so them being sold on eBay would point to only a small number of enterprise orgs they could have come from.

        Many enterprise/cloud providers shred HDDs/SSDs on-site when they come to the end of their operational life and so any such "unusual" models of drive would be unlikely to be resold anywhere.

        Also OPAL drive encryption has in the past been broken/bypassed by security researchers, usually not due to flaws in OPAL in general but rather due to vendor implemention-specific flaws.

        It is possible that the drives were not encrypted at all (on the assumption that they would be securely wiped when finished with and shredded if end-of-life).

        1. AustinTX
          Trollface

          Possibly, they scan the "serial numbers" of all of their drives into some sort of "inventory database" so they can also track their movements and confirm delivery to a secure shredder I hear data centers do stuff like that.

          1. Anonymous Coward
            Anonymous Coward

            Not all SEDs provide on access encryption

            Some provide transparent "clear" access but internally encrypt storage with a stored key. The idea being that you can decommission the drive by telling it to delete the key, which then eliminates the sticky problem of getting a modern SSD to actually "Delete" or "overwrite" all of the data stored on it. Of course if the controller board craps out you may not be able to decommission it successfully, and if someone forgets to decom it anyone who plugs it in can access it.

            Or you suck up owning your own key management and go OTT. Just don't lock yourself out, as nobody will be able to help you.

          2. Telecide

            Or maybe each SSD had a company asset tag assigned to it which, where the SSD evidentially hadn't been wiped, would have shown up and confirmed it to be part of SAP inventory?

          3. Marty McFly Silver badge
            FAIL

            Seems like a paradox to me.

            We have accurate & reliable inventory controls. That allows us to identify these eBay drives as stolen.

            Therefore our inventory controls are not accurate & reliable.

            Therefore we cannot trust our inventory control data.

            Therefore we cannot definitively conclude the drives were stolen.

        2. An_Old_Dog Silver badge

          (Previous) Drive Owner Determination

          Perhaps they had a property tag reading "Property of SAP" / "ZZL8530CF", or, "ZZL8530CF" "looks like" a SAP property tag number to a SAP employee who's seen tons of 'em.

      2. Yet Another Anonymous coward Silver badge

        >I'm intrigued how they were able to identify the drives as belonging to SAP

        The top level folder contained "SAP secret business stuff.pptx", "Copy of - SAP secret business stuff.pptx" "Copy of - SAP secret business stuff - final.pptx" "Copy of - SAP secret business stuff - final copy .pptx" "Copy of - SAP secret business stuff - final, - review.pptx" Copy of - SAP secret business stuff - final - review - please use this one.pptx" and "SAP secret business stuff - Print me.pptx"

        1. $till$kint
          Facepalm

          Ok, so you hacked my OneDrive....

        2. Joe W Silver badge

          I am not a violent person, but I contemplate using a LART (preferably the flat, Commonwealth version) against any cow-orker who uses that sort of naming convention...

          LART: (L)user Attitude Readjustment Tool. See also Clue-by-Four.

    2. big_D Silver badge
      Coat

      If it was in a data center, I'm surprised it wasn't part of a RAID array and totally unintelligable... :-S

      On the other hand SAP HANA SaaS backups sorted, if you lose data, just look for it on eBay.

      1. Bitbeisser

        Yeah, that single disks, SSD or not, out of a datacenter, would be readable to a single file level just doesn't smell right. In case of any RAID or object level storage setup, this should be hard to impossible, regardless of encryption applied or not...

        1. LateAgain

          With RAID 5 or 6 perhaps. But using mirror pairs has become the default "fast" option.

    3. Anonymous Coward
      Anonymous Coward

      And this is the company that is trying to get customers moving their data onto the SAP cloud with some urgency? This is hardly encouraging!

  2. Doctor Syntax Silver badge

    "a SAP spokesperson said the disks contained no personally identifiable information"

    Given that the drive bought on eBay contained data on 100 SAP employees either SAP employees don't count as persons or the spokesperson is pushing things even further than the standard script for this situation.

    1. Anonymous Coward
      Anonymous Coward

      "no personally identifiable information (PII)"

      Wondering whether this is a literal quote from the SAP spokesperson (unlikely as it was not inside quotation marks) or written by TheReg's Lindsay.

      As the SSDs were stolen in Germany (i.e. EU) then what matters is whether they contained any "Personal Data" which is what EU Data Protection Law (GDPR) is about. PII tends to be a USA term which has a narrower definition than "Personal Data" and is not relevant in the EU.

      I'd expect Lindsay to know the difference between Personal Data and PII...

      1. Korev Silver badge
        Thumb Down

        This is one of the big problems with the Americanisation of The Register, using an American term here makes things very unclear...

        1. rcxb Silver badge

          Should El Reg refrain from interviewing Americans as well?

          1. Joe W Silver badge

            Reading comprehension?

            "Using an American term here makes things unclear." (emphasis mine).

            Not everything is a personal foul by our offence...

        2. Yet Another Anonymous coward Silver badge

          >This is one of the big problems with the Americanisation of The Register,

          Americans are persons too

          (well some of them)

      2. rcxb Silver badge

        Wondering whether this is a literal quote

        The article isn't that hard to comprehend. That line was obviously a summary of the EXACT QUOTE WHICH IMMEDIATELY FOLLOWED IT... in quotation marks... followed by "they said"...

        PII tends to be a USA term which has a narrower definition than "Personal Data" and is not relevant in the EU.

        And yet that is the term the SAP rep used:

        "we can confirm we currently have no evidence suggesting that confidential customer data or PII has been taken from the company"

    2. rcxb Silver badge

      Except that's not *quite* what they said...

      "we currently have no evidence suggesting that confidential customer data or PII has been taken from the company via these disks or otherwise"

      This may well fall into the burying ones head in the sand category, or the proper business accountability rule: "Just stick your finger in your ear and go ting-a-ling-a-loo."

    3. CowHorseFrog Silver badge

      There shuld be a law that allows the public or the government to sue and jail spokemen at any level that makes obvious lies to the public.

  3. Anonymous Coward
    Anonymous Coward

    SAP takes data security very seriously.

    Demonstrably they don't, Because this doesn't happen to people who do.

    1. Stumpy

      Re: SAP takes data security very seriously.

      ... at least, not five times in two years.

      1. Korev Silver badge

        Re: SAP takes data security very seriously.

        And SAP wonder why no one wants to "Rise" to their cloud...

  4. Vincent van Gopher
    Coat

    Poor saps.

    1. Korev Silver badge
      Coat

      At least they didn't get root...

  5. Ball boy Silver badge

    'SAP takes data security very seriously'

    When we get caught, we take it seriously. Normally, we just assume everything is tickey-boo. Very relaxed about our data warehouse and its appalling physical security until it was discovered to be a weak point. Now it looks like we have a PR disaster on our hands so we need to show that we really do care, honest.

    There, FIFY.

  6. Missing Semicolon Silver badge
    Facepalm

    WTF

    No encryption at rest?

    The GDPR fine should be large.

  7. WolfFan

    Where I work

    Is a lot smaller than SAP, but apparently we are far more secure.

    1. All servers, NASes, etc., are inside locked racks. Yes, the racks aren’t anything special, and the locks are pretty flimsy, but it would be obvious if someone liberated a drive.

    2. All racks are in locked rooms, using card and touchpad access. And with security cameras inside the rooms and outside, pointing at the doors. Only authorized people are allowed inside. And the cameras would spot some taking a drive.

    3. Entrance, and exit, to the building are via security doors in a mantrap system, with lots of cameras.

    4. The sysadmins would probably notice missing drives, not least when reviewing the backup logs.

    5. The Morgue where we play with dead equipment is right next to the server rooms. And is locked and the door has security cameras.

    6. Anyone who pulls any equipment from the server rooms has to sign for it. And sign it back in when done. We definitely do not allow drives to leave the building. Unless they are headed to Iron Mountain for storage or destruction.

    7. As getting into the Morgue or the server rooms requires the key card, which is your ID card, you can't even access anything without it being clear whodunit.

    8. Backup tapes are stored in fire-resistant cabinets in the Morgue until the guys from Iron Mountain show up to take them to secure storage. The cabinets are, of course, locked. (Yes, we still use tape. It works.)

    There is no reason for a drive to be anywhere except the Morgue or a server room, or in transit between them. (Except for brand new drives arriving from outside, of course.) Any drive going out of service is reformatted seven times, if it still works, and destroyed by physically drilling holes through it. Really good recovery people might get some data from the drive, but it won't be easy. Destroyed drives are sent to Iron Mountain and run through their shredder. Now not even a really good recovery service is going to find much.

    So how, exactly, did a major corporation manage to let drives get out into the wild, and not once but multiple times?

    1. John Robson Silver badge

      Re: Where I work

      "2. All racks are in locked rooms, using card and touchpad access. And with security cameras inside the rooms and outside, pointing at the doors. Only authorized people are allowed inside. And the cameras would spot some taking a drive."

      Really? If the cameras are pointing at the doors then presumably I can remove a drive from a server and have it in a pocket as I leave.

      Now it should still be possible to say "this was when the drive disappeared, and there were only three people who accessed the room then", but drives are no longer large enough to need any skill to hide them.

      1. The Oncoming Scorn Silver badge
        FAIL

        Re: Where I work - Cistern Failure

        Research Machines assembly plant had a flooded male lavatory, the cause was a jammed open valve.

        One of the contractors had signed out a replacement HDD (Still in its shrink wrapped factory bag), then gone to the lavatory & hidden it in the cistern for later retrieval at the shift end but jammed the valve open.

        He was traced from serial number on the HDD, the timestamp of signing for it & the security footage from that point & then going for his data dump.

        1. Zarno
          Coat

          Re: Where I work - Cistern Failure

          Upvoted for the data dump joke.

          One may say he "flushed his cache attempting to liquidate assets".

          Mine's the one with no storage at all except pockets.

          1. David 132 Silver badge
            Coffee/keyboard

            Re: Where I work - Cistern Failure

            There's a "they had to examine the logs" joke in there somewhere too but I'm loathe to make it.

    2. AustinTX
      Holmes

      Re: Where I work

      Well, we can certainly never worry about Iron Mountain and their army of underpaid temps!

      And this is true, because I have been an underpaid temp working for IM.

  8. AMBxx Silver badge
    Boffin

    Sorted!

    It's the code for multi-currency they need in Singapore.

  9. Mage Silver badge

    And in the Cloud

    Cloud Computing:

    Just someone's servers in someone else's building with unknown computer and physical security.

    But this wasn't even Cloud. It was their own datacentre! Oops!

    1. Joe W Silver badge

      Re: And in the Cloud

      Well... it is the "cloud" for other companies. Too bad many customers (non-technical) don't get that...

  10. Mike 137 Silver badge

    Don't be a sap

    "SAP takes data security very seriously..."

    So seriously that folks can leave a secure area with kit unchallenged. This reminds me of the famous wikileaks incident where the perp transported secrets out of a SIPRNET environment on writable CDs that he pretended were pop music he'd brought in to play on his phones. The investigation found that the stringent SIPRNET security protocols were not applied to guys they knew: "Personal use was common, widespread and casually accepted – “Defense (Coombs): How was it enforced? Lim: No. You trusted people

  11. Richard 31
    Mushroom

    I used to know a guy that would run a scam thusly:

    1) pop out a drive on a raid array

    2) reseat it

    3) wait for the compaq (yes this was ages ago) engineer to turn up, replace the drive and leave the old one in the rack because of the disk retention policy.

    4) swipe the 'faulty' drive

    No idea if they got flogged on fleabay though

    1. John Robson Silver badge

      Blimey - that's a risk.

      I *think* it was a compaq array (raid 1) that a colleague of mine replaced a disk on and the machine promptly copied all the data from one disk to the other - take a wild guess which way round. oops

      1. John Geek

        yeah, thats the fun with traditional raid mirroring, you can't tell which instance is the 'good' one. another reason I'm a big fan of ZFS, every block on every device has a timestamp and checksum. conventional RAID assumes disks have only two states, working perfectly, or not working at all, they don't allow for anything in between.

  12. Anonymous Coward
    Anonymous Coward

    Surely everyone uses BitLocker on their servers for years now? They must have really poor security standards.

    1. OhForF' Silver badge

      A majority of servers probably do not use an OS compatible with BitLocker.

      While solutions to encrypt data at rest will be available it is very optimistic to assume that all major corporations use that - just look through recent articles on el Reg and you'll find examples where they struggle when they have to restore data from backups.

      With good phyiscal security in the server rooms and data that is not very sensitive it might even be a good decision not to encrypt the disks removing an extra complication layer for backups (and saving you some cpu cycles for a software solution or saving some money if using hardware based encryption).

  13. Auror

    This is why we wipe drives, preferably a couple of times before they go off to sit in the recycling pile.

  14. sketharaman

    Incidents like this will add fuel to the fire of SAP user group's concerns about whether SAP is really better at running SAP than its customers. https://www.theregister.com/2023/05/23/sap_americas_user_review/

  15. CaptainFarms
    Trollface

    Move everything to S4HANA on the cloud, they say

    Your data will be secure on our Waldorf data centres and so will your business processes...

    even if now and then some hardware pieces go missing.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like