back to article I've got a super free multi-petabyte storage box for you: /dev/null

As data volumes increase in all industries and the challenges of information management continue to grow, we look for places to store our hoarded bytes. Inevitably the subject of archiving and tape comes up. It is the cheapest place to archive data by some way; my calculations give tape a four-year cost of something in the …

COMMENTS

This topic is closed for new posts.
  1. Evil Auditor Silver badge

    But, but...

    can't we just store all the data into The Cloud?

    1. Graham Dawson Silver badge

      Re: But, but...

      For all practical purposes assume /dev/cloud is mapped to /dev/null and act accordingly.

      1. TeeCee Gold badge
        Coat

        Re: But, but...

        "and act accordingly"

        Use /dev/null as it's much faster?

    2. Wensleydale Cheese

      Re: But, but...

      can't we just store all the data into The Cloud?

      If it takes 88 days to transfer a petabyte from one tape medium to another, how long will it take to put it in the cloud?

  2. Lusty

    "my calculations give tape a four-year cost of something in the region of five-six times cheaper than the cheapest commercial disk alternative."

    Can we see your calculations please? This article doesn't really say much without them but seeing what you included would be very useful.

    1. Martin Glassborow

      Apologies

      But no you can't see them...it's commercially sensitive information. :-(

      You'll have to trust me; I have no vendor affiliation; suffice to say, it is not quite as simple as taking the cost of an LTO tape, you have to factor in SAN, Robot, Tape library costs; both capital and operational expenditure.

      1. Anonymous Coward
        Anonymous Coward

        Re: Apologies

        You also have to factor in the software to handle the migration.

        Sure you can hack together some scripts*, but they don't give calendar based scheduling, use of drives in only downtime, node priority, etc. etc.

        *You could also hack together some scripts to do your backups or archives, but you don't for very sound reasons.

      2. Lusty

        Re: Apologies

        But you could at least list what is and is not included. For instance did you include the cost of man in a van/WAN replication? The cost of media, drives and software a rarely even a consideration for these things in the grand scheme :)

  3. Alister

    Archive to disc

    If you archive to disc, which is then spun down and stored, is there more or less data degradation over time in comparison to tape?

    1. Mage Silver badge

      Re: Archive to disc

      Depends on type of disk and type of tape and humidity and temperature and density of each.

      Finding an interface to read the disk vs a working tape drive and interface in 30 years is easier maybe for Disk. I can still read any IDE/PATA drives easily. Ancient SCSI via PCI or PCMCIA adaptors. But MFM drives need me to boot a machine with DOS or Win 3.0 or Win 3.1 and full length ISA slot. But new Laptop doesn't take the PCMCIA SCSI adaptor, and some of the newer PCs have no PATA/IDE interfaces, nor PCI slots for older ones.

      I do have some drives that are a "kind" of IDE (on board CPU, same connection) but used a custom "PIO" ISA like adaptor. Not readable on anything except original Wang Mobo with ISA riser card and Pseudo IDE connection.

    2. Anonymous Coward
      Anonymous Coward

      Re: Archive to disc

      The problem with disk is that it's more fragile than tape, takes up more space per GB than tape and has the motors which drive it internally. Sure it may work after 20 years, but properly stored tape WILL work after 20 years.

  4. Anonymous Coward
    Joke

    Help!

    SO having complete faith in El Reg to know what's best for me I first tried to move my stuff info /dev/null, only to find out that this doesn't work:

    [me@smtp ~]$ mv test.jpg /dev/null

    mv: inter-device move failed: `test.jpg' to `/dev/null'; unable to remove target: Permission denied

    SO then I asked our admin for the root password, but that didn't go too well either.... After he asked me why I wanted it I told him that I needed to move my data into /dev/null, then he only gave me a rude comment about lame jokes and hung up on me!

    But I figured it out:

    [me@smtp ~]$ cp test.jpg /dev/null

    [me@smtp ~]$ rm test.jpg

    However, there's one problem when I try to get my picture back:

    [me@smtp ~]$ cp /dev/null test.jpg

    [me@smtp ~]$ file test.jpg

    test.jpg: empty

    So I hope anyone can help, I need this picture back yesterday ;-)

    1. Stephen Rodda

      Re: Help!

      I'm so sorry to hear you're missing the picture.

      My advice to you is to locate the drive /dev/null, remove it from the computer and send it off to a professional data recovery company. I'm sure they will be most helpful.

      Do remember to replace the drive once removed, (you will need to format it:

      sudo dd if=/dev/random of=/dev/null

      -- this may take some time) as otherwise you will not have any backup space to use.

      Good luck!

    2. Michael Habel
      Trollface

      Re: Help!

      mv: inter-device move failed: `test.jpg' to `/dev/null'; unable to remove target: Permission denied

      LOL no wonder you forgot to say sudo!

      ya should've used:

      [me@smtp ~]$ sudo mv test.jpg /dev/null

  5. Stephen Rodda
    Alert

    WOM?

    Most backups are WOM, anyway. People don't check 'em until it's too late.

    1. AnonymousCoward

      Re: WOM?

      A wise, bearded and be-sandled system administrator once told me

      "A backup is not a backup until its been read"

      Wise words.

      1. Anonymous Coward
        Anonymous Coward

        Re: WOM?

        And a backup process doesn't exist until a recovery has been successful.

        That's one lesson you don't want to learn the hard way..

      2. Anonymous Coward
        Anonymous Coward

        Re: WOM?

        'A wise, bearded and be-sandled system administrator once told me

        "A backup is not a backup until its been read"

        Wise words.'

        Actually, a backup is not a backup until it has been read and you are able to use that data to replicate the system from which the backup was taken.

        I had to learn that one the hard way. :(

  6. Michael H.F. Wilkinson Silver badge
    Joke

    I think many of us share the view that we should archive all plans from middle management (and indeed many government plans as well) in /dev/null. It can be done without any loss of real information

  7. b0llchit Silver badge
    Coat

    And nothing of value was lost

    No value got lost. Actually, a lot is saved with moving to /dev/null since most files are already copies from /dev/zero and gzip only allows for so much compression.

    When we talk about management documents, then we can safely assume that they were retrieved from /dev/urandom (no, not /dev/random, management types are so predictable 50% of the time), so /dev/null makes a lot of sense. At least more sense than the management types do, have done and ever will do.

    (sarcasm may be a part of this post, ymmv)

  8. mark 63 Silver badge
    Facepalm

    "Firms no longer seem to be able to categorise data into separate "keep" and "delete" piles."

    Aint that the truth, it seems in the future a biger and bigger percentage of stored data will be

    1) "stuff we dont know if we need"

    2) "stuff we couldnt be bothered going through"

    3) duplicates of 1) & 2)

    1. TeeCee Gold badge
      Meh

      When it comes to tape storage, there is a bit of a snag here. You may well know that, of that backup, only a few files and databases are actually needed long-term, but short of unravelling the tape and snipping out the unwanted bits, there's not a lot you can do.

      Usually cheaper to store the whole lot than it is to pull 'em all in and then tie up a load of lads and hardware in restoring 'em somewhere and resaving the "required long-term" bits.

      1. Anonymous Coward
        Anonymous Coward

        @TeeCee

        That's why people need to understand the difference between backup and archive.

        You backup for DR, or accidental deletion, you archive for long term legally required retention.

        1. Captain Scarlet
          Pint

          Re: @TeeCee

          But people won't, they think servers have millions of 4TB drives in which they can store what they want and IT is just being annoying by not increasing disc space.

          1. Anonymous Coward
            Anonymous Coward

            Re: @TeeCee

            IT is just being annoying by not increasing disc space.

            I found being tight on quota is a wonderful tool to promote tidiness :).

          2. keithpeter Silver badge
            Paris Hilton

            Re: @TeeCee

            If each department had to pay the full cost of indefinite storage at some sensible figure per Gb, would it make any difference?

            Paris: I haven't a clue about this stuff at scale. I just keep everything then chuck it out if I have not looked at it for a year.

  9. Anonymous Coward
    Anonymous Coward

    blinf leading the blind

    In the Government department I subcontract for fellow well known IT contracting company has a system where when a user pulls a customers record and makes an alteration several pages are printed out and then amended. The pages are then posted across the country to a scanning facility, where they re scanned and returned to the database - in addition to their previous scans, SO each time a record is altered its size doubles! and we are talking tiff files here not txt.

    Glad I'm not in charge of the arcghive strategy for that.

  10. Jon Green
    Coffee/keyboard

    The funny thing is...

    ...how often do these tape archives ever get read back onto disk?

    Exactly.

    So you might as well use /dev/null, because your short-term backup covers the critical stuff for disaster recovery, and the longer-term tape archive is just a waste of space and money.

    1. Anonymous Coward
      Anonymous Coward

      Re: The funny thing is...

      Try defending that when the regulator comes knocking on the door...

  11. Sandpit
    Thumb Up

    I offer a /dev/null storage archive at extremely competative rates (4 or 6 times lower than the tape storage guys)

    I also offer a low cost option for destruction of data should you ever decide you don't need it any longer.

  12. S Watts
    FAIL

    Rigorous backup processes...

    Late last millennium, the establishment I was working for was transitioning to ISO9001; and outsourced the generation of their operating procedures to a third-party, as is the way with such places. Clearly, the third-party was paid by the kilo, as we received and had to dually implement and adhere to bookcases of glossy manuals.

    Of course, there were reams of procedures ensuring that all our data was rigorously backed up to removable media; which could be archived safely off-site.

    However, when the need arose, we discovered that there were no procedures outlined to ever get any of this data back...

    1. Anonymous Coward
      Anonymous Coward

      Re: Rigorous backup processes...

      Best thing about ISO9001 is that if you document your procedure as archiving everything to /dev/null then so long as you do that then you are ISO9001 compliant .... always amuses me when companies make a big deal of have ISO9001 accreditation when in theory it could just mean "we have documented that we are utterly useless but we rigourously adhere to that level of (in)competence"

      1. Anonymous Coward
        Anonymous Coward

        Re: Rigorous backup processes...

        cf Dilbert.

        1. john bertelsen

          Re: Rigorous backup processes...

          Or Dilbert

          http://dilbert.com/strips/comic/1995-09-02/

  13. Ragequit

    Long term storage...

    While I generally agree with the idea that humans (and especially businesses run by humans) tend to be major pack rats when it comes to data.. This really isn't such a new problem if you consider the warehouses of file cabinets that were common not that long ago (and in some cases still are).

    As our ability to store information digitally increases so does our desire to save everything. Nobody dares try to create an algorithm for sorting out what data is truly valuable for fear of making a mistake. Irregardless, we really could use a storage technology that is reliable and whose means of access are adaptive to changing technology.

    How about bioengineering a storage medium that uses already established data redundancy and integrity techniques to recover lost data or reject mutated storage. :P You'd just have to engineer new interfaces for the storage over time. Or maybe you could induce a forced mutation to 'upgrade' the entire system? The cost would be in feeding the thing I imagine. And it could scale to any size as needed. lol. Kidding. Kinda.

  14. Anonymous Coward
    Anonymous Coward

    I've often wondered...

    I've often wondered what you'd see if you hung something on /dev/null that actually captured what got written there (along with timestamp, process name, and user name).

  15. sisk

    I've got a manager over one site who's never deleted an email in his life. The man literally saves it all, even the spam. Do you have any idea how much space 20 years worth of email takes up in backups? Thankfully he retires in a couple months. I think I'll do the final backup of his inbox to /dev/null.

    1. Ian 55
      Big Brother

      When I find out who you are, you're fired :)

      You've seen this - http://blog.stephenwolfram.com/2012/03/the-personal-analytics-of-my-life/ - I presume?

      1. Pascal Monett Silver badge
        Pint

        So I've seen it now. Impressive collection of personal data, to be sure, what with the number of keystrokes, mails and steps taken. It's also an impressive show of analytical power that Mathematica demonstrates.

        And ?

        Apart from giving this guy a chart saying that he's worked a lot, what does one get out of it ?

        I don't see much use in a chart telling me that I send more mail now than I did ten years ago. Does that mean I'm wasting time sending mail, or does it mean that I'm more occupied and doing more efficient things with email ? I know the answer already, and I didn't need to saddle my life with a bunch of stat collectors to tell me.

        I salute the performance in monitoring, but frankly I don't see what it can tell me about what I might need to do in the future, and forecasting future needs is the only reason to collect all that data in the first place.

        So no, I don't see that "everybody" will be doing that in the future. Actually, I don't see that anyone will be doing that in the future. Having a good beer is so much more fun.

  16. Christoph
    Facepalm

    What is the point?

    The value of big data is mining it for information by subjecting it to whatever crafty analysis you can think up.

    But if it takes months to simply read the data in for analysis, the eventual results will be hopelessly out of date.

    The data that's valuable in and of itself without all that analysis is only a miniscule proportion of that data heap.

  17. Anonymous Coward
    Anonymous Coward

    All together now

    (ALL:)

    Every byte is sacred

    Every byte is great

    If a byte is wasted

    God gets quite irate

    (BOFH:)

    Let the heathen store theirs

    Up in the wispy cloud

    God shall make them pay for

    Each byte that cant be found

    (ALL:)

    Every byte is sacred

    Every byte is great

    If a byte is wasted

    God gets quite irate

    (PFY:)

    LTO, DDR, VXA

    Record theirs just anywhere

    But God loves those who treat their

    Data with more care

    (ALL:)

    Every byte is sacred

    Every byte is great

    If a byte is wasted

    God gets quite irate

  18. Peter Brooks 1
    FAIL

    not that silly...

    Actually /dev/null is quite sensible. I've pointed out to people quite often that there's not much point in keeping stuff on tape that's going to take so long to get back that it'll be no use in a disaster. At least with /dev/null you find out quickly that there's no archive and do something else about the problem - instead of waiting three days to come to the same conclusion.

  19. OzBob
    Meh

    So, April 1st falls on a Bank Holiday this year,....

    and someone couldn't wait until 2014 to use this one.

  20. The Nameless Mist
    Facepalm

    /dev/nul .. oh no please no ..

    I had this once. Long long ago, some joker had pointed a SunOS system do backup to /dev/nul

    Getting a callout to restore a clients backup, only to find this in place was a facepalm moment.

  21. JimTopbloke

    Data Abhors a Vacuum

    The more space we create the more shit we find to fill it..

    I used to be careful what I stored and where I stored it with 40Mb of drive space, not so now I have a couple of Tb to play with..

    It's amazing what you don't actually need, and never ever use more than once (if that!)

  22. Tyou

    Typical disk bias

    This is not the silliest stuff I've read but comes close. Anyone running a big data center is migrating data all the time as tech goes obsolete. The bigger the data center the more you migrate off old disk to new. You do the same with a tape archive. Start with T10kA drives run those a few years and skip the T10kB drives and start writing new data to T10kC. Then wait for it; you also start migrating those T10kA tapes to T10kC tapes. Please think about using a HSM which has built in migration functions and if you have more than a PB you probably have more than 2 drives to do it with.. I know I do. I also, like my disk purchases, plan for obsolescence and build migration plans into my costs. This fiction that tape is write once read never is just tha I've also done the numbers and the cost it's 4 times cheaper. Go read David Rosenthals article on the cost of long term storage and the cost of cloud storage

This topic is closed for new posts.