backup error condition./run out of tape...
Good grief... It just goes to show each new generation fails to lean from its parent's mistakes... That's a very old chestnut and I'm quite suprised modern software isn't throwing better reporting out...
Plane or train? We asked four Reg-readers with storage smarts to say where and when we should use disk-based data protection and where we should cross the line and use tape. Three did just that. The fourth identified a fourth use-case for tape and added a salutary reminder that it has to be managed; it is absolutely not a start- …
From the article: "On a side note here, VMS had/has a versioning file system, which would inherently keep older copies of files available for easy retrieval, and just remove the oldest ones as the disk filled up. But this is a very unusual feature. I have not heard of another system that has this."
The author clearly hasn't looked very hard or he would have found CopyFS - a fuse based copy-on-write versioning file system. Stack this with a deduplicating file system like ZFS, and you have a very efficient continuously versioning file system. Combine that with lsyncd to acquire on-close remote copy capability and snapshotting on the remote copy, and you have a solution that alleviates the requirement for any tape solution, except in cases where external bandwidth is insufficient.
Once had a customer who loaded the tape and kicked off their backup every Friday night, when they came in on Monday morning the backup had 'failed' (their words) and uloaded the tape, so they reloaded it religiously and the backup completed in no time.
This was only discovered when they had a disk crash and couldn't restore to the new disk so I was sent onsite by DEC to help. Every tape they had only contained about the last %25 of their data.
We had to send the dead disk off to the factory to be stripped down to recover their data.
Open Enterprise Server, the Linux version of the Novell services, also supports the NSS file system and will keep copies of deleted files around, only cleaning them up when the disk fills up, just like VMS does. You can safely undelete multiple versions. So if you go to OES when you migrate to Linux you will keep this functionality. From the users point of view there is no difference to a Netware server or an OES Linux server.
weirdly i have been looking at backup solutions today for a friends start up company of around 15 users 1mail 1 file and 1SQL server. they would like to remove the backup and take off site and also recover deleted files quickly.
They can achieve their requirements for around £1500 New system total but it will actualy only cost them around £600 as they only need for symantec backup exec they have the rest lying around as spares.
Server or PC
Windows Server OS
2x icybox removable storage drives
2x 1TB HDD or 2TB HDD as they grow
Backup with remote agents from the other servers to removable HDD's
1 of them for offsite backup that they can take out everyday. the other for incremental archiving.
This offers fast and reliable backup solution
as they arnt a big company they wont use more than 1TB of data for the foreseeable future and no staff are waiting to long for someone to restore data for them.
yes its not as cheap as a tape drives but time is money right? if you can save time you earn more money! i can see bigger company's will still use tape drives for archiving huge amounts of data but small business's... really?
First interview person Henry Wertz: "The big one? Price; tapes cost about 1/10th the cost per byte of hard disks."
1.5TB Maxell LTO5 tape from Amazon: $67.95. 1.5TB Western Digital Elements external HDD from Amazon: $78.62. If you want to get really technical, you can get one of those HDD docking stations (similar to requiring a tape drive for tapes [which run about $2,600 for LTO5 btw]) and buy raw disk drives: Western Digital Caviar Green 1.5TB from NewEgg for $59.99. If you want to get really picky, you can assume no compression on the hard disk and an optimal 2:1 compress for the tape to achieve the 1.5/3.0TB capacity, then you have to get a HITACHI Deskstar 3TB (NewEgg $139.99), but mind you, compression on disk is quite easy and 2:1 is by no means difficult to achieve using even low on-the-fly streaming compression. Back up a video or JPEG library and you'll only see 1.5TB out of the tape. Therefore, even worst case (no compression for disk and optimal 2:1 for tape) lands at 2.06x the cost of tape. Best case is only 88% the cost of the LTO5 tape for like capacity. So no, not 1/10th the cost. Sorry. Especially when you factor in the $2600 tape drive vs a moderately priced Cavalry EN-CAHDD2BU3-ZB disk dock (for instance) at $64.99 @NewEgg.
Second interviewee Evan Unrue: "but also, disks keep spinning, so doing this comes with a larger physical footprint in the datacenter and a larger power bill. Tape scales by adding cartridges which don’t spin when not being use and don’t take up space in the IT room as they scale"
Why is it that everyone assumes that a disk-based solution mandates the drives are always on? Sure, the first target in the D2D2T or D2D2D will be required to spin, but not the last stage. Disks would work as removable medium just as effectively as tapes in this regard. I would suggest that disks are less vulnerable to environmentally-caused "bit rot" as well, due to the platters not prone to going brittle as tape has a tendency to do (at the very least it can withstand being in a less-than-ideal storage location better [think attic of IT Director's house or the like] if necessary).
I applaud the third interviewee Chris Evans for pointing out some of the shortcomings of tape solutions. Granted, disk has disadvantages too, and as Chris said, it comes down to finding a balance between the two based on your RTO/RPO requirements. The key is finding the best spot to use the appropriate medium. For enterprise environments with hundreds (or even tens) of TBs to backup, you can't beat a tape library for convenience. For anyone with 3-6TB or less for a full backup set, anything more than tape drive or external HDD is likely overkill, especially for the sub-1TB market.
As always, check your logs on your backup jobs frequently. If that's too much of a pain, find a way to have the results emailed (same as paged nowadays) to you upon completion/failure. For those willing to roll up the sleeves (such as the ZFS/CopyFS commenter above), there's plenty of methods you could employ to produce a better setup for your organization than BackupExec or the like could provide, and using HDDs just makes that solution even easier and more feature-full.
There seems to be a touching, albeit naive, view that reliability is a static concept: you can compare a disk with a tape and come up with a reliability metric.
Unfortunately, this is just not so. As things age, the relative reliability shifts. So a tape drive may be considered (for some bizarre reason) to be less reliable than a disk today, but how do the two stack up when you try to access the data on each in 5 years? 10 years? 40 years? Tape is inherently more stable.
Do you (for some value of "you") care? Maybe. It probably depends whether you are thinking of backups vs archives, since those are very different things.
P.s. those cautionary tale about tape backup errors? You know they apply equally to any other type of backup, right?
I've successfully mounted MFM and RLL drives from the 1980s repeatedly on very old VMS and Altos servers. Power them on, wait for the spin up, and you are in business. Worst case scenario, you pull the platters and rebuild the drives. Although expensive and bulky, old hard drives handle tortuous storage conditions quite well.
Tape...not so much. What can go wrong? The acetate substrate becomes brittle or melts. The top coat/binder rots, gets sticky, or simply flakes off. Oxidation of magnetic particles lower the signal-to-noise ratio. The lubricant and release agent breaks down or transfers off. High speed access leave the spools with an uneven wind. Tension & transfer rollers in the cartridge crack. Pinch rollers in the tape drive become brittle or sticky.
Restoring data from 10+ year old 7-track and 9-track was sketchy at best, but broken segments could be patched and low densities limited the amount of lost data. Newer helical scan tapes are much more difficult to patch. One inch of LTO-5 tape records over 47MB of data before compression. One inch of StorageTek T10000 T2 records over 116MB of data before compression.
Although tape manufacturers quote archival life in the range of 30 years, these metrics assume pristine storage conditions with no dust at constant temperature and humidity. Ask any technician who has "baked" a tape and quickly transferred the source with fingers crossed...
Personally I would like to see optical or holographic media developed with an eye towards archival needs, specifically:
- Long life
- Low cost
- High density
- Random access
- High speed transfer
- Write-Once / Read-Many option
RE: Version Control. Windows Server has a feature called Volume Shadow Copy (VSS, or just Shadow Copy), which does point-in-time copies of files. NetApp's version of this is SnapCopy (i.e., it takes a snapshot of the data at configurable points in time for your version control and accidental deletion needs. I understand that the other big SAN players have similar functionality with their storage devices.
RE: Cost: Yes, Disk has largely caught up with tape in price.
RE: Reliability: I've had disks that were DOA, I have on my desk at this moment an LTO4 tape from our under-one-year-old tape library that went bad far before it's time. As with anything, Your Mileage May Vary. This is also why you test your backups in either a DR simulation, or in a test lab on a regular basis, to make sure everything is working properly. I would like to point out, though, that an LTO tape can survive a drop of 3 feet (1 meter) to the raise floor of a data center; a disk drive, even when parked? not quite.
Our configuration at $work is a Disk to Disk with a tape auxiliary copy, largely for audit and compliance; the disk copy is essentially a two week spool, and backups older then that get shuffled off site via tape auxiliary copies. It works put pretty decently- between our SAN's VSS functionality and the disk spool, I rarely have to pull tapes for restores, and even then it's usually someone who is insistant that they need their 3517786135 GB mail file online at all times...
Good article and well informed points of view.
Regrettably, tape and hard disk are both susceptible to bit rot in a way that some other optical media derived products are not. It all comes down to how long the content or data is to be preserved for.
If the average spinning life of three to seven years for an HDD or six to twelve years for a minimally maintained tape are suitable for the project, then you're on a winner and well catered for with magnetic-based storage media. At the datacentre level, there is no alternative owing to individual data carrier size limits.
Want to store content or data for longer than that, or under ruggedised or non-optimal conditions? Perhaps there is a need to look a little further afield as valid permanent digital storage solutions exist.