And now for our annual ‘Tape is still not dead’ update Shipments of tape storage media increased again in 2024, according to HPE, IBM, and Quantum – the three companies that back the Linear Tape-Open (LTO) Format. The three companies on Tuesday claimed they shipped 176.5 Exabytes worth of tape during 2024, a 15.4 percent increase on 2023’s 152.9 Exabytes. For those of you who …
"The three companies on Tuesday claimed they shipped 176.5 Exabytes worth of tape during 2024..."
What they didn't admit is that this was all... to one old guy in Omaha, Nebraska, who hasn't quite figured out this whole internet shopping thing, still uses AOL via dial-up, and is mystified that no matter how many "tapes" he buys, and from where, none of them on arrival is actually the copy of the Doobie Brothers' Toulouse Street on 8-Track that he thinks he's ordered. But keeps plugging away anyway.
... is to be as bulletproof as possible.
LTO is still around because no-one's come up with a provably better alternative in decades.
A company I worked for had a full-height rack system with a robotic mechanism to switch tapes between multiple LTO's, a slot to insert/retrieve tapes and an inbuilt bar-code printer/scanner to label the tapes.
Sometimes in tech it's good to be a luddite and say I don't *need* anything allegedly "better".
If the nightly backups are running OK then I'm a happy bunny.
Yes, especially with ransomware on the increase and everyone using Windows having a backup that is NOT 100% online is almost mandatory.
You can abandon Microsoft products or abandon offline backups, but not both, and the former is for most companies impossible.
I think it could reasonably be argued that online "backups" are only fit for convenience, not for protection. And even then, you need a well-rehearsed plan for what you're going to do with them when the time comes - which might well require access to a device capable of reading your chosen media somewhere other than the smoking ruins of your former HQ.
If your on site tape library is destroyed in some type of disaster, your servers and storage will also be destroyed meaning your first problem isn't "what do we use to read our [moved off site] backup tapes?" since you don't have a place to restore to.
Companies with enough money will have their own separate site, but smaller ones can't afford that so the best they can do is some sort of arrangement with third party to make servers and storage available. You'd have to insure they have what's needed to read your tapes as part of that contract. Though if it is a disaster that affects more than just you good luck getting them to follow through on that contract because that recovery hardware is going to be significantly oversubscribed.
That is fine if you are protecting against a catastrophic failure, and of course you do need to protect against those.
You also need to protect against individual files being corrupted or lost, usually due to user error, and that's where snapshots and online backups become useful, because you can revert to a known-good version of that file much quicker. This scenario, while less damaging, happens a lot more frequently.
I was at one one place where ZFS data snap-shots meant they were back up as soon as they confirmed the shut down the ransomware attack vector.
They set the data to RO when they found out they were being hit and lost a few hours of data due to the lag in detecting the issue.
If they had found it right away and locked down it would have been a minimum of about 10 minutes lost.
Offsite tapes are always a good, ahem, backup plan though in case you lose your facility.
"I think personally the threat of ransomware is what is keeping tape back ups going, for now."
Maybe in the enterprise computing world, but in HPC/research computing, it's a LOT cheaper to keep petabytes of infrequently accessed archival data on tape (even with multiple copies to protect against tape corruption) than on spinning rust or SSDs. (Especially if you have a file system that supports tiering/HSM like GPFS/Storage Scale/whatever IBM calls it this week.)
"You can abandon Microsoft products or abandon offline backups, but not both, and the former is for most companies impossible."
And, if you do choose between those and opt for abandoning Microsoft, you made the worse of the choices, because plenty of ransomware targets Linux servers because that's where lots of companies store the valuable data. Only taking out the Windows desktops is annoying and they'll lose some things, but if you can take out that central infrastructure which is often Linux-based, that's the valuable option. Offline, or at least immutable, backups makes recovery much easier, although it's still worth putting more time into prevention.
It's called a tape library, and they have been around for decades, for many different tape formats.
The stunning thing is that as the capacity increases, if you have fairly static sized data, the libraries get smaller.
Last year, we replaced some full rack LTO5 libraries with new ones with LTO9 tapes and drives. 5TB compressed to 45TB compressed per tape, a 9 times increase in effective capacity. Went from a whole rack to about 15U of space, while increasing capacity many-fold, but keeping the same number of drives.
The only real problem is that data transfer speeds need a boot up the bracket. At the same time as replacing the tapes and drives, we doubled speed of the Fibre Channel fabric from 8Gbs to 16Gbs adapters (we could have gone further if IBM had homologated the faster FC cards for older Power system models) and faster switches, as well as migrating the primary storage from spinning rust SAN servers to all-flash storage arrays, but the time to back up the data to tape did not show a significant reduction, either for the LAN-Free backups or the disk to tape storage pool copies.
This could be because the systems doing the backups, which run IBM Storage Protect (aka TSM as a previous name) were still the same processor/memory config. but measuring the performance of the systems, the backup's do not max out either processor or memory.
I did some raw I/O speed testing of the drives, and actually found we could get something like 500MB/s (with drive compression turned on) to the tape drives when taking raw data from the flash SAN server straight to tape. This is actually faster than the LTO5 speed, so we were expecting some improvement, but what we got was mostly negligible.
These are low-profile LTO9 drives, so beside being single ported, the quoted write speed is a little slower than the full-height drives (probably because the motors that move the tape are smaller and weaker). Raw speed was much in line with what we were expecting. But when doing LAN-free backups from the database, we only achieved about 40% of the raw speed. Could be that we're shoe-shining because we can't get the data out of the database fast enough, but I believe that these drives can do speed matching.
One other thing I found strange was that the brand new LTO9 drives only had 8Gb/s FC interfaces, but increasing the speed of the adapters and switches will have reduced contention. In fact, the SAS connected LTO9 drives at 10Gb/s, according to the documentation, are faster than the FC ones!
I once scuttled off a big shed outside Heathrow to do a disaster recovery exercise. I had AIX up and running in an hour or so and SAP and Informix were installed and ready. All I needed was the tape sever to play the tapes back into it. Sadly someone had chosen HP OmniBack running on Windows for the tape solution which meant that I was reliant on the Windows team to bring their box online for the next step. It turns out it was impossible to bring the Windows system up in an isolated way without knowing the magic domain keys which we didn't have access to, and then when pressured, no one could actually provide them...
The exercise was deemed an utter failure, but it was useful in that it proved we'd be stuffed. Later on we moved to a pair of data centres, and could do hot failover from A to B and back again and that worked without the business even knowing. We also had a more robust backup solution and that was still tape based, but not on Windows servers. I don't know if that worked in a disaster scenario, as I wasn't involved in that anymore, but I know the data centre switch worked perfectly.
Something like TSM works best on AIX, another UNIX, or maybe Linux. I would never build a backup solution that relied on Windows ever, but then I'm an old fart who's been doing this for over 30 years, so what do I know!
I would say that for fast recovery, it is best to keep your backup server, plus any essential infrastructure servers (say, a network installation server) on discrete, non-virtualised systems that you don't need to have any virtual storage or VM infradtructure set up. This allows you to plough ahead with getting your backup server operational while the other stuff is being done by other people. Otherwise it becomes a serial operation with many dependencies.
But this is not the way that the modern world operates!
At my last job I kept bugging my boss until I got the budget to buy a tape library.
Of course, my budget wasn't that big, so instead of a full rack system, it was only 2U, but it still managed to hold 16 (iirc) tapes, and an LTO5 drive in the centre. It too had a barcode reader, which made knowing which tape was which much simpler. I think it was SAS connected.
Great bit of kit, and massively reduced the amount of time I had to spend swapping tapes manually.
Constant vigilance?
No technology lasts forever, and in a number of years it a case of scrounging the bone-yards for bits to keep going, Or you have a plan that has all tape with some redundancy (duplicated/RAID style) and also transferred to the newer standard after something like 5 years. Trouble is that is really hard to do manually, and gets expensive for a nice tape robot to do it, but then how valuable is your data?
>>> I reckon a lot of data won't need to be kept 5+ years.
For _most_ (certainly not all) businesses even that is overkill, provided accounting requirements are met.
I work for a pharmaceutical company and the regulations ask for _up_to_ 30 years of data retention when production ends. This includes environment / reagent / QC measurement data. And more.
There's a whole warehouse of old manufacturing machines and sealed servers, backups and whole backup systems with dates when they can be destroyed at the earliest. The data on servers and media is not refreshed or transferred to newer tapes or other storage, since the pharma company itself has no need for it. Regulators do not require refreshing the data.
Really? In my industry, US Construction, it is routine for data retention periods of 7, 10, 15, & 20 years to be written into contracts. Several states have data retention regulations for funded projects to retain data for 7-15 years depending on the state! We are currently in an Audit with a state that is pulling data from almost 20 years ago.
So yes, many businesses need to retain data for longer than 5 years.
I did not make a recommendation as I am not happy with the current choices.
What I have been doing for myself is tarring off my ZFS snapshots to pseudo-tapes, those pseudo-tapes being entire hard drives, mounted in hot-swappable enclosures. These drives have no filesystems on them; they are treated as merely a series of storage blocks.
"Using tape for near-line storage and short-term backups is fine, but not for archival storage, due to oxide delamination. "
I would have imagined the opposite based on 1960s ½" open real tapes still readable.
I remember there was (probably still is) a WORM LTO technology intended for archival storage - whether it was based on a magnetizing metal oxides or something optical I don't recall.
I imagine some type of linear sequential storage will persist for archival purposes preferably WORM.
Provided you keep the tapes within reasonable temperature and humidity bounds, tape will remain readable for decades, and the manufactures offer guarantees of this, not that any guarantee would make up for significant data loss (it's probably capped at the cost of the tape!) These are not your 8 track or cassettes from the 1970's, they're enterprise grade tapes, which if bought from reputable manufacturers, were much better quality than audio or video tape.
The biggest problem is keeping drives around that will read your old media. As pointed out, LTO, with some exceptions in the roadmap, will write the previous generation of tapes, and read the one before that, but that's it.
The LTO5 tapes I talked about in a previous thread cannot be read in the LTO9 drives we installed. So if we had opted to keep some of the LTO5 tapes for long-term archive, we would have to have kept one or more LTO5, 6 or 7 drive around to read them.
On the subject of the LTO5 drives and tapes, although the tapes have a documented expected lifetime, measured as the number of passes of the tape over the heads (but remember, LTO is a serpentine recording model, so an end-to-end pass moves the tape over the heads multiple times), we were using basically the same tapes for in excess of 10 years (with each tape cycling around used at least once every couple of weeks, and often more frequently), which by my calculations is many times the expected lifetime. We were getting signs that the tapes were wearing out (the drives perform read-after-write tests as the data is written even when streaming), and we had replaced individual tapes that went bad throughout the lifetime of these libraries, but I can be pretty certain that a significant number of the tapes in the library were from the original installation.
I still know of a working backup system using LTO4 tapes and drives....
I'm somewhat surprised that no manufacturer has built an LTO drive specifically to be fully backward compatible with all the LTO generations.
Probably technically tricky, but when a company has data on an (eg) LTO5 tape and no compatible drives, they'll pay any price you name if you can read it.
Clearly not giving the bloke on the corpse collection cart a big enough backhander. ;)
Some passive linear long term storage medium is going to be with us pretty much forever, I suspect.
Backups might be redundant/replicated active storage of snapshots but archives will always be "tape~ish" in character.
Would have been curious what Curtis Preston's (Mr Backup) views on tape and archival storage in general.
it remains relevant despite its slow data transfer speeds
That's only a recent phenomenon. A few years ago discs couldn't keep up with LTO tape. You frequently had a DDT (Disc to disc to tape) backup system whereby the slower first tier discs dripped their data to the second tier discs which were much faster. Once the second tier filled up, they could offload their data to the LTO drives and saturate the LTO drives bandwidth.
Getting backups to work well and fast was an art.
Staging data through disk pools before writing to tape is a technique that can help, but you eventually get to the point where copying the data multiple times before it ends up on tape means you run out of day to carry out the staging.
Many storage systems, like TSM, have a backup server that has the tape libraries attached to them. The multiple systems in an environment send the data across the network to the backup server that can either put the data into a disk pool for later de-staging to tape, or write directly to tape.
The problem is that, unless you have a really fast network, that can often be a bottleneck.
One solution to this is to use what is called LAN-free backup, where the tape drives are zoned to the systems holding the data, and those systems write straight to tape, while the backup system just manages the tape use and the metadata storage for these backups. Many backup systems have this technique. It's normal in large DB environments to do it this way, with normal backups of OS files being written over the network to a disk pool, while the volume backups go LAN-free. An additional advantage is that backup straight to tape ties up at least one tape drive per backup (more if you are doing a multi-stream backup for speed,) which limits the number of backups you can do concurrently by the number of tape drives. But you can have multiple network backups writing into a disk pool concurrently.
So the big data goes LAN-free direct to tape, and the small data gets written to disk, and then moved to tape at a more convenient time.
But there are exceptions. At an HPC site running GPFS over the IBM Power 775 PERCS infrastructure that I worked at, which had a very, very fast multi-plane optical network and significant numbers of storages servers running GPFS, it turned out to be best to allow the TSM server direct access to the GPFS filesystems, and for it to directly manage the backup of all of the large data from GPFS to tape.
While working for a small subsidiary of a major media firm that married the "You've got mail!" company (they really needed a prenup) early this millennia, an edict came out from Manhattan to the suburbs: "No unencrypted tapes shall leave the data centers."
Hmmm. We had created a partially home-made back up system (disk-2-disk via some scripting, then NetBackup to the tape libraries).
My Sun contacts found me a card they made that was intended for (en||de)crypting traffic over its included 1-Gbit NIC. They also obtained the header files that one needed to access the card via C.
OK so I added a pre-step to encrypt the data being fed off the Sun/EMC Sym based servers to our D2D JBOD trays...Yeah "Mission accomplished!"
Or so I thought. The guy that handled swapping the LTO tapes said that suddenly he had a swap 4x as many tapes as in the past...
Hmmm. Hmmmm. Hmmmmmmm...Ah! Said LTO tape drives did their own compression and I was sending them basically I guess white noise (versus our (mostly) highly-compressable Oracle DB data).
So a new pre-pre-step: Pass the data through "compress" first...&& All was well once again! // This system was still in use past my tenure within the worst US corporate marriage early this century
"Drives built to work with the current ninth-gen spec, which offers 18 terabytes of uncompressed capacity on each tape, can therefore write the 12TB eighth-gen tapes and read seventh-gen tapes to access the measly 6TB each can contain."
Two-generation read compatibility has historically been true for LTO, but sadly it isn't for LTO9. I am in this middle of an LTO7->9 migration that has been significantly complicated by the fact that LTO9 drives won't read LTO7 tapes. (To the point of having to replace and repack a tape library frame at our DR site because otherwise we don't have enough drive slots to accommodate both sets of drives at the same time...)
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
