Sputtering ...
is what my kids did while eating jars of mushy brown baby food. Never thought I'd see this achieving 880Gb/in
IBM has claimed its fifth-in-succession world tape density record with a 330TB raw capacity technology using Sony tape media tech. Back in April, 2015 IBM and Fujitsu demonstrated a 123 billion bits/in2 220TB tape using so-called Nanocubic technology and barium ferrite tape media. IBM_Tape_records IBM tape density records …
How things have progressed from the older 600BPI tape.
Even though tape is not so fashionable these days, tape still has a place as the bad guys have yet to find a way of compromising the off-line backups and archives and the power required to maintain the data after its written is hard to match - just some environmental and physical security !.
Tape is on the way out.
The more sensitive it gets, i.e. the more data it holds, the harder it is to maintain properly. Because it's a mechanical, moving-part, friction-based item, it can't hold up against constant use, and the more you cram on it, the worse it gets.
Off-line backups and archives are also not exclusively the domain of tape. You could easily disconnect and power off other backup devices too, and they will retain data without power for decades. Hell, nowadays, they are talking about persistent RAM that can do the same while still working at RAM speeds.
Tape, though, will always suffer from problems with regards replacement equipment. Though you might have the tape in ten years time, the cost of the drive to read it becomes prohibitive quickly. Given that it's supposed to be your insurance against disaster, having to source another compatible tape drive rapidly, installing the software, restoring from the tape, etc. can take DAYS. Other storage devices are inherently easier to read from, almost being entirely self-contained units, so the need for additional, specialist, mechanical equipment isn't there and you can restore from them in HOURS.
I don't see a future for tape at all. We are already at multi-terabyte SSD. There's no reason that some electronic memory (be it memristors, SSD, Flash or a variety of other technologies, even traditional hard drives) won't take over from tape entirely. Same amount of data, contained in the same space as a tape, no additional "reader" required, data retention into the decades without specialised handling, built-in error correction if you were making archive-quality stuff, if you go solid-state it's impervious to shock, temperature variation, etc. and you could even make it waterproof (handy in a fire scenario, for instance).
Calibrating little slivers of tape in a dusty atmosphere, across a specialised mechanical head and then spinning the tape at high speed, in a little plastic box that you expect people to carry with them without exposing it to the outside air, that's a fragile technology.
As it is, I've worked in several places over the last few decades where tape is at best a second-class citizen, playing only a minor role compared to networked storage, offline traditional hard drives, replication, etc. Though some of that is vulnerable to your theoretical network attack, there's no reason that it can't be disconnected, snapshotted, restricted access (i.e. push backups rather than pull), etc. to remove that threat.
I've been in IT for nearly 20 years. In that time - excluding test restores - I have never once been required to restore from tape, mainly because I specify lots of "cheap, fast, let's not put all our eggs in one basket, if this doesn't work we DO have other backups, but this is quick" backup storage. Sure, a USB drive stuck on a server on its own is not "a backup system" (BTW: I'm not claiming it's a good idea!). But you'll be glad of it when you only want the one file that was deleted last week and not have to go to your tape catalog to retrieve it. I've recovered data from short-term backups like NAS and connected drives, DFS mirrors, remote replicas etc. infinitely more than I ever have from tape.
I'm sure in large datacentres, etc. the regime is different but whereas every small business used to have a server with a tape drive in it (often costing almost as much as the server itself), nowadays everything from cloud-backup to network storage to VM replication to a redundant site to just plain "Lots of copies of everything everywhere on all kinds of devices" has taken over. I can easily carry a small NAS home with several copies of my entire workplace's VM's and data, for example. Encrypted, full-disk-speed, historical snapshots, network-accessible in an emergency, hell, I could even turn on the iSCSI option and run all the VM's direct from the device in a pinch.
Tape doesn't really have much of a future except in specialist scenarios. Those places are already into the hundreds of thousands with library robots and so on. But even they won't stay there forever.
I know that if I was suddenly made a billionaire, and could set up a company the way I wanted, tape wouldn't figure heavily. Lots of copies. Lots of snapshots. Lots of devices. Lots of locations. Lots of technologies. Redundancy in EVERYTHING. Tape would only figure as one of those as an equal partner to the others at best, for such technological redundancy, not because it offers any particular advantages. As far as advantages goes, it would actually be the bottom of the pile.
(P.S. I have never lost a byte. Not one. I specialising in recovering schools that have experienced previous disasters and sanitising their systems).
"I don't see a future for tape at all. We are already at multi-terabyte SSD."
4TB SSD. £1300 ex VAT & delivery.
6TB LTO7 tape. £89.95 ex VAT & delivery. (50% bigger capacity, 14 times cheaper)
5 day tape rotation with 6TB LTO7's. £449.75 ex vat.
5 day tape rotation with 4TB SSD's. £6,500 ex vat.
Just because it's old technology at base doesn't mean that it's either bad or irrelevant.
I'm still using a much earlier tape format here because it's installed and still task adequate, so I just googled the price of the LTO7 tape and cited the price Misco offered & the 4TB SSD.
I'm sure I could get both the SSD & tapes cheaper through suppliers if I was trying hard, but it's a reasonable guide as to the retail price difference between both.
"Though you might have the tape in ten years time, the cost of the drive to read it becomes prohibitive quickly. Given that it's supposed to be your insurance against disaster, having to source another compatible tape drive rapidly, installing the software, restoring from the tape, etc. can take DAYS."
it sounds like you're envisioning a restore of 10 year old data to recover from a disaster. There isn't an organization out there that can survive rolling their business back by 10 years. The exact timing of the change is debatable, but 10 year old data is an archive (for reference), not a backup (for restore/recovery/resumption of business).
Not to mention that Backup/Archive Management Systems have provisions for migrating data to new media. So you wouldn't have data you need quick access to on media that you can't access quickly.
"I'm sure in large datacentres, etc. the regime is different..."
Yes, yes it is.
@Lee D
I really don't understand all of the down-votes. They must all be from Millennials that have never tried to recover a server from a tape!
I've been working in IT for about 25 years now. I can count the number of times I have SUCCESSFULLY restored data from a tape on two hands with fingers left over! The number of times I've tried to recover data from a tape probably numbers in the hundreds (I used to work for an IT consulting company).
On one occasion, we had three backups of a server. Each one ran a successful verify pass. Not one of the three would run a complete restore. Not one! The WORN concept is so very true.
People that are relying on tape are fooling themselves. At my current employer, we don't use tape at all. We have other solutions for offline storage (removable hard drives in a fireproof safe, along with offsite backups that are also offline).
I've been burned way too many times by tape failures to ever rely on it again. It seems like as the density increases, an already unreliable medium is only getting worse.
You may have been working in IT for 25 years,but you obviously have no clue what you're doing if you can't even manage a proper backup and restore with a tape! The bare minimum for an IT person who is in charge of backups is to actually verify that they can do a restore, if they cannot then they start troubleshooting, calling support, whatever they need to do to find out if the problem is hardware, software or IdioT related.
What do you tell people who want a restore, that it isn't possible because you have never figured out how to successfully restore anything from tape? Maybe you are using disk based backup like a Data Domain, but if you still have tape drives and use tapes you need to figure out how to make them work (or have them fixed if the hardware is faulty) and not just assume that because you can't make it work that "tape can't be read after it is written".
If tape wasn't actually able to be read, don't you think people would have caught on by now and quit buying them? Do you really believe that no one ever needs a file restored, so they just haven't caught on to something you are "smart" enough to have figured out? If someone said what you did during an interview I'd stop him right there and thank him for his time, and figure he was a poser with a fake resume who had no clue what he's doing.
As for the comment above that SSDs will replace tape, because technology changes over a decade means you won't have equipment to read those tapes any longer. Have you ever tried to read an SSD that hasn't been powered on for a couple years? I think you'll find it has lost all its data. It is not useful as an archive medium unless it is kept powered on. If you can afford to keep your "archive" SSDs powered on, more power to you, but for a tiny fraction of the cost you could afford to pay a tape monkey to insert those tapes into the library and copy them to newer formats every few years to insure you don't have the problem. Or simply keep a few older technology drives in your library (if you have one of the huge ones) or keep an old library with the old drives (if you one of the small ones)
DougS already covered it, but you must be doing something very wrong to have that low of a restore success rate.
I've been on the Backup/Recovery side of IT for 20 years and can count on one finger the number of times I had a tape cause a restore failure, and that's for probably tens of thousands of restores (most of them tests).
SSD's? Cant trust them. They leak and wipe themselves. 10 years on and your SSD will be very easy to read as it will be blank. Extremely limited capacity and highly expensive. Might fail to "chip rot " if made from cheap chinese crap.
USB flash drive? Cant trust them. See SSD, apart for the expensive part. They are cheaper for a reason...
HDD, Generally affordable and miles cheaper than SSD however cant trust them as they leak (well the magnetic fields fade) and seize up. The on board DEDICATED and CLOSED electronics allow you to easily plug them into any compatible host device. But those electronics, oh, I hope they dont have bad caps that fail and kill the on board computer that reads the platers.... I hope the chips dont have a manufacturing defect that causes "chip rot" over several years and the drive when needed in the future wont talk to the motherboard. Is the firmware even in PROM? Ah its on a f*cking SSD chip? See SSD. Even facebook is too scared to leave theirs off for very long and no drive gets to be more than a few years old as they are constantly spun up checked and replaced.
Memristors? Cant trust them, they appear to not exist yet.Hmm in 10 years maybe I can find one for sale. Oh but my data is here now. Hmm may have an issue using memristors at the moment.
RDX? Cant trust them. Marketing spin designed to empty tape haters pockets of money while buying a slightly higher grade HDD (see HDD) or SDD (see SDD) in a magical plastic case that makes them better somehow. Yet the Achilles heel of HDD and SDD tech was not removed from RDX. THE CONTROLLING ELECTRONICS ARE EMBEDDED I THE F*CKING CART!!!!!!!!
Basically I will not trust anything that is dependant on embedded electronics to let me access data decades down the line. Reading and writing should be handled by a device that uses REMOVABLE media. HDD's and SDD's are not media, they are small computers that contain media. Its the same as archiving your data to a fileserver and locking it away. Still think it will power up on 20 years? It probably will and it equally can emit the magic smoke. Is it using HDD's? (see HDD's)
Removable media today: Tape, Optical disc.
Tape is dense and reliable. Its just a spool of tape. Treat it like shit and you will get shit off of it. Accidents can happen (see conclusion) however but I trust it miles further than anything listed above. Downsides include mechanical handling of the tape may go awry and damage it. Tape reading/writing relies on contact with tape heads.
Optical, specifically media that does not rely on phase changes. This includes M-Disc DVD's and any HTL spec blu-ray media. If you really want, go further and have a proper pressed disc made. Treat the disc like shit and read shit off of it. Downside: as it is a multi layer media you are relying on the manufacturing process to seal it up properly. YMMV unfortunately but a little bit rot will not prevent the media from talking to a motherboard (see HDD and SDD). More expensive than tape but upsodes over tape include no physical contact other than what is needed to spin the disc.
Conclusion. Nothing can really be trusted but tape and optical (non phase change media) are the best we have at the moment. The universe HATES your data and DESIRES to turn it into random mush. Use a multi level, varied media system with regular scheduled tests. Replace bad media with direc replacements or the new version/better replacement. Maintain 3 copies on different media with all that checking and you should be fine.
Oh and if you cant be bothered, upload it to the "cloud" (cough someone elses servers) and hope they care enough to do this for you, correctly. Make sure you read their small print to be sure you can give them a legal slap should they let your data leak off their SSD RDX "media".
Anyone here remember the Iomega Jazz drives that were supposed to be our small business archive salvation circa 1995? Or zip disks for that matter?
I happened upon a box of Jazz cartridges a month ago and had a nearly irresistible urge to destroy them. Not just destroy ... but go completely retro. Annihilation with extraordinary prejudice!
Death to perfidious media! (looking at you, RDX)
Tape density won't save IBM. They're floundering, losing money, sacking staff and contractors while keeping managers (who aren't going to do the work that staff and contractors do) and Watson is billed as "AI", when it's really just automation, at best. IBM is the Dinosaur joke of the industry, who's paying out an annual 33 million bonus to their CEO, and it's not tied to her performance or their stock's performance.
While I agree with the sentiment I'm not entirely sure that it's relevant to the story. At least in this case IBM are doing something genuinely innovative.
I'm impressed that the spokesman managed to mention "cloud" but disappointed that he forgot to get "cognitive" in there somewhere, or mutter something about "transformation".
Effin' A !
Here I was thinking that "Nanocubic technology" was about as good as it was going to get. Once again, real, actually intelligent people prove that boundaries are made to be pushed back.
Such a shame that such progress doesn't exist in the battery market, otherwise we'd have 12KW/H button batteries for a dime.
Storing more energy per volume only becomes a "bomb" if a fast uncontrolled discharge is possible. The reason some LiON batteries go pop and catch on fire isn't because they have reached some sort of bomb limit, but because of the chemistry. Some explosives that hold a half dozen orders of magnitude more energy per volume than the battery in your phone are remarkably stable against unintended explosion. If only there were some way to cause them to discharge that energy on demand at a desired rate they'd be great for grid storage (in secured areas or buried under a few meters of concrete to prevent their use as explosives by evildoers)
The guy who invented LiON batteries originally has come up with a new version that is denser, lighter, charges and discharges faster and can't go boom or catch on fire. Not only will they not catch on fire if pierced, but they will keep working even if you stab them multiple times with a knife while they're operating! Hopefully they'll figure out how to mass produce them affordably, because that will be a welcome advance.
Do IBM tapes only work when launched through the air? If they're going to release official graphics boasting about world records, you'd have hoped they at least have someone involved who knows how to spell "areal" (the "area" part is the clue when you're dealing with something measured per unit area).
Backup != archival storage.
What you use for backup is a function of how much churn your data sets have and how quickly you need the data back. What you use for archival is a function of how many years in the future you are going to need it.
If you are in the backup business, tape, disk, etc are all viable, depending on your options.
If you are in archives, I believe that a mandatory part of your strategy needs to be rolling from one technology to another over the interval between recording the archive and needing it, and without losing or gaining any information. This idea that you should have bought two 1/2" tape drives in 1975 so that in 2040 you can still read the 1976 tape is kinda loopy. Budget for one transition every five years from recording to retrieval and have meticulous comparisons when you switch from one to the other.
I agree and from what I've read it appears datacentres with tape libraries with LTO-X wait until LTO-(X+2) comes out about 4-5 years later and then they spend a couple of months moving thousands of tapes over to the new ones needing on average about a fifth in number as the LTO-(X+2) drives can still read the LTO-X tapes. They can then finally get rid of the old perfectly good drives which are still reasonably good value to other people and the one main advantage of the new hardware is if they do actually need to recover data then they have much faster drive speeds with which to do so. I expect to see a lot of this happen again when LTO-8 comes out presumably somewhere in the expected October-January 2018 timeframe. I'm not sure what they do with the old tapes but either they sell them off or they just keep them as they are because they are a perfectly good backup of the original data for another 4-5 years until LTO-(X+4) comes out and the process repeats again.
I've been to lessee now, 22 DRP exercises.
Other than two particularly outrageous DB backups (VeritasNB/Rman) that used I think 6 or 8 drives on BU and only 2 on restore -- (if you've been there you'll get it) I can't think of a case where we had *everything* coming back from tape fail. I've seen Day1 runs involve nuking and rebuilding the NB master more than once, but this was inevitably a case of the Docs needing updating.. (guess why we did the exercises?)
I've just dealt with executing a move from HSM to permanent dirt cheap disk for an archive solution. It wasn't what I suggested at the beginning but it will suffice. Some of our HSM tapes had been around for 15 years. Of ummm between 2200 and 2500 odd tapes, some 38 of them did not contain the data that was originally written to them. Of those 38, 24 were overwritten in a physical server migration about 8 years ago (eye-dee Ten T errors). The remaining 14 had bitwise errors in block headers -- sometimes right at the beginning of the tape, sometimes way the heck into the tape, of *those* 14 I was able to recover 3 of them with manually correcting the one bitwise error, and on 4 more I was able to skip the dead file by changing the pointer to the next block. On the balance of the tapes I would have had to spend more than 7 days walking the data to attempt a recovery.
Tape's 3 biggest issues are
1) environmental affect. One of the reasons we likely had the bitwise error issue in so many tapes was that in the time frame when these were written we had raised floor tiles that were shown to be shedding microscopic EM active particles to the air when they were pulled and placed. These particles were found in all areas of the DC, including on tape surfaces. We had a *LOT* of tape read errors for a year or two until all work was completed. We just made more copies.
2) Hardware lifecycle. In the 13 years I was in OS/Platform support we went from LTO to LTO5. Most of the drives claimed at least one generation of backward support, but to be honest we've often found it advantageous to keep the drives == tapes, mostly for reliability. And rolled through *countless* updates to the backup software itself.
3) One of the projects that was wrapping up when I started in Platform/OS support was a 4 year exercise of migrating from WORM disks to tapes. I know, precisely, where in the primary hardware warehouse our two remaining WORM drives are stored. I *seriously* doubt that anyone else does. Same can and will apply to our LTOx drives some time in the future.
If you've got a tape solution that has never restored data for you, your *BACKUPS* are broken and you need *serious* assistance with your hardware and software. Someone with experience is recommended. Not someone off the street with a CS degree. I can admit that I've seen horrifying reasons why a backup/restore will not work, from misconfigured NICs (MTU settings at source/switch/path/switch/dest) to assumptions being the mother of all fuckups (overnight DB tables complete drop and reload during an RMAN full!!!), to just plain not understanding (backup over the application network causing app to 'Go Down' in the middle of the busy period) basic best practices, to only grabbing two tapes out of an 8 tape backup and expecting the restore to work.....
Personally? I *DESPISE* tape. Hate it with a passion. But I know my tools and can get it working when needed. It will *always* be the cheapest, most effective way to hold data for long term archival storage. It *WILL* have problems beyond X years in every case, physics being physics, and Drives will be an issue Y years out. Migrating the data with the physical tape lifecycle HAS TO BE WRITTEN INTO THE PROJECT. LTO4/5 tapes when we first saw them had 10 year lifecycles. My life could have been much easier lately....
/rant
Sorry - - just finished writing up the audit responses.