I know PRML mean "Probably Right, Maybe Lucky" but NPML? "Nearly Positively, Maybe Lucky" maybe.
IBM Research has devised technology with FujiFilm to create a 35TB capacity tape, but it will take 3 days to write the data at LTO5 speeds. The new hyper-capacity half inch tape technology has been successfully read and written at a 29.5bn bits/sq in areal density, which means a tape capacity of 35TB according to the …
Chris Mellor (no relation) is normally very good on storage issues, so I may be missing something, but - if you increase the areal data density of the tape, you should get correspondingly increased data transfer rates without any need to increase physical tape speeds. You have to upgrade the electronics to handle the higher data rates, but that shouldn't be too big a challenge.
Now that HDD's can be had for a about 5p per gig, who would still choose slow, unreliable, low tapes?
The drives themselves cost a fortune, and I can't believe these new tapes will cost the £1.50 i'd say they were worth. And who wants the job to swap them over all the time?
This medium should have died a death years ago with the C-90
...I take it you keep your disk back ups for several years, like many, many companies are legally required to do so?
I take it you have Weekly / Monthly back ups as well as your dailys?
Are your backups stored in a fireproof safe?
Just a thought, after all, HMRC may not be happy if your backup site catches fire and you loose all your financal records for the last 5 years.
Us? we do it properly.
We have offsite D2D back ups, that are then archived off to tape, to a 3rd site.
Almost all large businesses use tape. It's the only practical way to back up TBs and PBs of data onto media that is easily duplicated, stored in fire safes and taken off site. They're physically lighter and cheaper per GB than HDDs which is why the attraction.
For eg, you can get an LTO4 tape (800GB or 1.6TB compressed) for £35 ish, whereas a 1.5TB hard drive will cost you upwards of £50/60. Multiply that out by the number of tapes you need and you'll soon work out that it's cheaper than HDDs, even when you include the cost of the drives and robotic libraries. You can't get robotic libraries for HDDs, so swapping them over and taking them off site is impractical.
we recently looked into LTO4 and came to the conclusion that the capacity advantage over HDD was reasonable but the random access disadvantage was horrible. So 800GB tape (+ pricey tape drive) at £35 or 500GB SATA II HDD at £35 (and ever falling). Discs are certainly heavier, but they're self-contained.
Still haven't decided which way to jump.
I trust you're not an IT prof. Duplicating disks (preferably at a remote site) is straightforward and great for disaster recovery (providing you think a disaster can't affect both sites simultaneously), but it can't handle the "Oh dear, I accidentally deleted my directory. I'm not sure when, but it was definitely there a month ago" problem (unless you're prepared to increase your total disk space by a factor of 20x or so). And off-siting hard disks is a pain compared to tape.
I confidently predict tape will be around for a long time yet.
Last I checked, I couldn't pick up a 35TB hard drive... A 2TB HDD runs near $150-200, and I'm sure these tapes will be at a premium of near $200 each, but will probably reduce to $100 in a few years. Even with a $4000 investment in the drive alone, large data warehouses (think IBM, M$, Apple, US DoJ, etc) will eat these for breakfast and ask for more. Cheap(er) removable media is why tapes have been so popular, even now. Up until now though, they've only had 800GB (compressed) per cartridge, costing near $60-100 each. So, $170 for a 2TB HDD (very reliable if you make it past that 1st month vetting period), or 2 tapes at $140, yielding 1.6TB (2:1 compression, if you are so lucky. I'd say 2:1.5 for real-world [think of what Pixar would get...]). Personally, I'd take the HDDs myself, but I haven't (yet) had to back up more than 1TB of data in a single go.
$Work still use tape for archival and off-site backups; It's solid, proven technology that stands up well to archival conditions. (IIRC, hard drives are generally not rated for holding data safely while in storage.) And hard drives tend to be fragile, even when powered down. Tape cartridges will safely withstand some pretty serious G loads that a hard drive's mech may not.
Some of us who work in the real world have to do things like send backup data offsite (it's a good idea even if it's not a requirement, but I guess you wouldn't know that). Tape is actually quite fast for backup (less so for recovery), especially with compression, and it retains the data for longer than an unpowered hard drive.
Other than those objections, your point is valid.
Slow, unreliable tapes are faster than disks (when streaming), cheaper than disks, more portable than disks, the drives have built in encryption (at least IBM's do) and they don't require anything like the same amount of power or cooling as disks - a very big concern when you're backing up large amounts of data. Oh, and they're probably as reliable if not more so than crappy SATA disks, especially after they've been left on a shelf for 20 years. Other than that, bang on.
(OK it was a few years ago) had both. I suspect quite a few places use this type of system.
The data was backed up over the network to a NAS unit, this happenned overnight when there was no-one using hte systems and meant the servers were ready for use the next morning without any inconveniences. Once the data was on the NAS it was then written to tape during the day when there was someone there to monitor it in case another tape was required etc.
The other benefit of this is that if a file was required to be recovered the next day it could be accessed very quickly and easily since it was still on the NAS. Larger drives in the NAS would allow several days worth of data to be stored - in most cases saving the need to resort to the tapes (but definitely not in ALL cases, we had to go back 5 months for a deleted directory once).
This shows that disks are good for transient data storage, but not so good in the long term.
I dunno, high-capacity tape drives are ridiculously expensive, and the tapes themselves aren't cheap either. When my LTO2 drive blew out two years ago, I switched to removable SATA drives with el-cheapo mobile racks. Works like a charm, way cheaper, more reliable, fast as shit on a shovel, and all the rest.
Goodbye tape, I'l not miss your sequential-access slowness nor your over-priced components.
LTO4 typically already writes 16 concurrent tracks using as many heads... I can't find specs on LTO5. I'd assume more tracks per inch means more heads, otherwise we'd have to dramatically increase how many times the tape has to be reversed in order to fill it, meaning more passes through wheels and across scanning/writing surfaces, dramatically increasing tape wear and increasing the likelihood of bit failures. I can only assume that 23x density and smaller heads would mean MANY more heads. I can also assume, as with each previous leap in technology, improved bit write performance from each head. I'd be willing to bet full 35GB write completion in less than 8 hours.
The problem is slow bandwidth nowadays. For instance, to repair a raid-5 which uses large discs, 4TB or so, will almost take one week. During that time the other drives will be stressed, increasing the chance of another fail. Sysadmins testifies this happens quite often. Therefore you need raid-6 which allows two drives to fail. But, soon that is not enough either. You need three drives to fail, raid-6 does not offer this functionality. But this can be found in ZFS, called raidz3.
What sort of disk array are you using? I have never come across bandwidth as a problem, or re-creation time for a raid set as a problem.
Need more bandwidth? Slap in another fibrechannel or two.
Need your RAID sets to re-create faster? Make multiple smaller RAID sets rather than single monolithic sests.
Anyone smart enough to know that *system* costs are what matter in a backup environment.
Backup to a disk drive. *lots* of parts to fail -> unreadable data.
Backup to a tape catride. Mostly *passive* device. Drive fails. Replace it. Put old tape in it. Read tape.
They have been saying *hard* drives are obsolete as flash drives have gotten cheaper.
They said it when magentic bubble memories were the new tech on the block.
Wrong then. Wrong now. The price per unit storage (bits, Kb, Mb, Gb whatever) is still *orders* of magnitude above what a tape (albeit a very advanced tape formulation) costs.
*perhaps* someone like Plastic Logic can make a storage tech with the price pwer unit storage comparable to tape. I am quite sure that nothing like a conventional chip fab can.
This is a first rate development. Its nice to see the tape biz push the envelope and not just a x2 improvement but a x35 increase.
until the tape drive eats the one tape that has the one version of the file that the Comptroller corrupted an hour ago (because he's an incompetent goon who shouldn't be allowed to even have a calculator).
I'm not saying backup to HD is the end-all, be-all for backup. Just that, for now, it's proven to be far better than tape for my relatively small (2 TB) backup set.
<puts on fireproof suit>
@Adam Wheeler - LTO4 stores 800GB, natively, before compression. Don't believe me? - Go and look at http://lto.org/technology/uformat.php?section=0&subsec=uformat for information on LTO3 and 4. If your data can be compressed at 2:1, you will be able to put 1600GB of data on a single LTO4 tape. The native capacities for all the LTO Ultrium generations so far are 100,200,400,800 GB, and you add on compression to those native sizes if your data can compress.
As for the discussion about the number of heads required, versus the speed etc - it may be useful to know that current tape tech routinely writes multiple streams of data to a single tape head anyway, on separate tracks. LTO4 and various other drives write data to the tape in a longitudinal serpentine interleave pattern; What this means is that the drive writes a track from start to end of tape, then moves the head very slightly to one side, and writes a new track from end to start, moves the head a little again, and so on. From memory, LTO4 actually writes 16 tracks at a time, and the tape will go from start to end quite a few times (56 passes, so 28 times start-end-start?). By the time the tape is full there are hundreds of tracks (896 tracks in total for LTO4). As a handy side-effect, the end point of the tape is the same physical wind point as the start, so there's not much in the way of rewind time - just the couple of metres of leader to unspool (takes a couple of seconds at most).
In order to be able to write the tracks close together with the accuracy required, there are special hard-coded tracks put on the tapes at the factory (these are called servo tracks - because the tape head servo mechanism follows them) - degaussing an LTO tape may/would destroy these special tracks and render the cartridge useless.
In terms of speed, when we double capacity of either HDD's or tape, we don't necessarily double speed - in fact, usually only a fraction of that is the case. A 2TB hard drive isn't 4x the speed of a 500GB one, is it?.
I'd be shocked if the 35TB prototype got anywhere near 44x the speed of a current-gen drive... in fact I'd expect it's only a few times quicker. IBM did a prototype of a 1TB drive in 2002, so this sort of thing isn't unprecedented.
Q:Who uses tape any more?
A:People who want to offsite data to a vault, and people with lots of data. One of the largest generators of data I know of is CERN - they generate about 15PB of data a year at the present time, and yes, the majority of their data is stored on tape (~45PB at the moment?); it's not historical either as they've recently been adding new kit. Tape doesn't use power when you're not actually accessing it, and doesn't usually require the same level of raised flooring or expensive cooling systems to run. Over a lifecycle, it can be cheaper - which is why CERN are proponents of it. They also use disk, because tape isn't good at everything.
I hope this has been helpful for anyone who can be bothered to look past the rubbish people spout (and if I got anytihng wrong.. let me know and I'll correct it if I can)
I've learnt something about the way LTO works, which meets my target of one new fact per day. But if the areal density increases 44x, that means the linear density increases sqrt(44)=6.6x. So (in principal) you get 6.6x as many tracks each holding 6.6x more data. Therefore your data transfer rates for each track (if physical tape speed remains unchanged) go up by 6.6x and the number of tracks could be increased 6.6x (from 16 to 100+ in your example). Now the latter is unlikely to be achieved, 2x or (pushing it) 4x is more likely because of the physics of the read/write heads. So I would expect that the time to write a full tape would increase 3.3x or maybe even half that.
BTW to get your factor of 23.33x at the end of the article, aren't you comparing the raw data capacity (35TB) of the new tech with the compressed data capacity (1.5TB or 800GB raw) of existing tapes?
Prof posts, "who would still choose slow, unreliable, low tapes?"
Tape (i.e. streaming rust) can survive a lot more G-Force and shock than a hard drive (i.e. rotating rust) can... especially when the rust coated media is operational!
If an application needs reliability of massive quantities of storage under extreme conditions, tape is still superior.
This does not mean that flash might not catch up - but at 35TB, it might take flash awhile...
Good for IBM, keeping a very important technology viable!