Do you think spinning rust eats flash's dust? Join the hard drive daddies club Disk drive makers have joined forces in a desperate bid to keep disk sales up and fend off the flash threat. Western Digital, HGST, Seagate and Toshiba have formed the Storage Products Association to provide "a forum to educate and advocate about the role of hard disk drives (HDDs) and Solid State Hybrid Drives (SSHDs) in …
Laugh all you like. When a platter HD starts going bad you typically have a while to get the data off before it becomes inaccessible. The difference between a working flash drive and a dead flash drive can be the blink of an eye. A combo of flash and platters to make a hybrid drive is a best concept, combining speed with reliability. It would also keep the costs down as flash does not seem to be getting reasonable in cost.
When I've had a drive go bad, it's always been gone with no warning and I never see my data again. Like the thing doesn't spin up, or if it does, it refuses to read, or it's not seen on the bus. Once a drive started making horrible noises and was never visible to the computer again.
Once I was reading from one of those WD MyPassport external USB drives, and my glasses slid off the keyboard onto the drive (a drop of about an inch) and the drive immediately started clicking and giving USB errors. I didn't get any data off that one either.
Yes, I do once-a-week backups, why do you ask?
I've only had 2 SSDs (one's in this computer as I type) and one failed once with SATA errors about 6am before I got up for work. Linux immediately blacklisted it as a device. I powered off and back on, and have not had trouble in the 3 months since.
And yes, flash is getting reasonable in cost considering it's a new technology. The first hard drive I worked with was a Fujitsu Pragmatic PD-40M which was 40 MEGAbytes for about $5K at the time, so I may have a weird idea of reasonable cost.
Really? Did you run software to check the SMART statistics that turned into bricks? Did you try ddrescue after the failure? You must be a very unlucky chap. Based on a sample size of several hundred over a decade, the majority of hard drives (probably 2/3) do show signs of going bad before they fail completely (at which point I pre-emptively replace them), and ddrescue can retrieve a large part of the data from maybe half of the other third.
Anyway, we have backups, don't we?
I agree with Gene Cash, HDs regularly die without warning. I've many boat anchors around me--boxes of dead HDs, mainly Seagate but with a sprinkling of WD and Samsung thrown for good measure as I've already said in an earlier post a week or so ago. These masses of dead drives have caused us considerable angst not to mention hundreds of hours rescuing data despite having a reasonable backup regime. (We've even had new HDs fail when in the process of storing rescued data from failed drives.)
Simply, the facts are that these hard disk manufacturers are making product that is reliably unreliable and they deserve to go broke unless HD reliability is urgently and dramatically improved! They're making drives at the limit of technology and they're doing so whilst cost-cutting in the extreme. It's pointless having $0.50/GB cost if the data frequently vanishes off the platter into the aether without warning.
Anyone who knows the vaguest about the operation of HDs knows that these manufacturers are already pushing the HD design envelope into unreliability. Raw data coming off HDs is already below the noise and has been so for quite some time, and if it weren't for very efficient data separators and error correction these HDs wouldn't have a hope in Hades of ever working let alone make it out of the factory. Essentially, the data density on large terabyte HDs is so high that what is recorded is hardly recorded at all.
One of the consequences is that even when HDs are used as non-operating backups they'll often fail in storage. It's a risky business keeping a drive containing a million or so files stored for a year or two, it's far from a certainty that you'll get the data back after this time. This means that users have to employ more redundancy by using more B/U drives--the drive price war is really a false economy, users end up eventually paying the equivalent of much better manufactured drives but with a lot less convenience.
If drive manufacturers are to wean many of us off SSDs back to HDs then they'd better stop the bullshit about MTBFs of millions of hours--which most know is utter crap--and genuinely make their drives more reliable.
Here's a few suggestions:
1. Make the essentially useless S.M.A.R.T. reporting system actually work by being effective and meaningful.
2. Build beepers and indicators into HDs that scream out audible warnings and flashing lights at the first sign of trouble, these failure indicators would work independently of any computer--just power-on would be enough.
3. Build in redundancy--a complete and independent second head assembly with its own separate head amp electronics and such (in normal operation this could be used to increase drive throughput).
4. Provide a rescue/data recovery mode--drive goes into slow speed mode where heads are lowered very close to or onto the surface for a once-only recovery pass.
5. Provide standardised outputs between the HDs PWA/electronics and the drive chamber electronics--when the drive electronics fails then it's easily replaced or the chamber can be easily coupled to external recovery electronics.
6. Stop the ridiculous secrecy surrounding the operation of HDs which stops people rescuing their data from failed drives. Tell users about the internal drive protocols and how the manufacturing test jig connections work etc.
Of course, such notions are both obvious and revolutionary--so revolutionary in fact that we'd be back to the way electronic maintenance was done prior to say 1980 when maintenance handbooks, revision sheets and circuit diagrams were commonplace and openly available.
However, don't hold your breath. In this age of secrecy, hidden IP info and such, manufacturers would, it seems, prefer to go broke than provide the customers with what they actually want: i.e. reasonable protection that their valuable data won't vanish off into the never-never.
A well written and well thought out response, Rob, and you make some very good points. I have thrown away more hard disc drives than I care to try and remember, and have another two on my trash pile right now - subjectively, it certainly feels like the failure rate has been accelerating in recent years.
Of course, manufacturers are under increasing pressure - from a number of directions - to make their kit ever cheaper, so naturally reliability is the first expected casualty. You would think that they'd try what mfrs in other ore related industries have done: add value to make their products more attractive. And any of the features you describe could provide that "edge" - but I suppose only time will tell which direction they choose to go.
"...to make their kit ever cheaper...reliability is the first expected casualty."
Thanks. I strongly resisted using SSDs until recently because years ago I attended lectures by an Intel engineer on how charge was stored in such devices and it's quite frightening really, that they work at all is a sort of engineering miracle. Briefly, he produced engineering info to show how the charge deteriorates when say a "1" is put into the matrix and a "0" put next to it. His overly simplified but good analogy was to consider a naughts & crosses matrix with the "1" in the middle and a "0" on an outer square and that it is made of marshmallow. As you press your finger down into the outer square to represent the "0" you will see the "1" square also deforming downward. Now consider substrate fatigue with repeated operations and it's easy to imagine why SSDs have a finite and definable life.
Nevertheless, modern SSDs, are turning out to be more reliable than HDs, thus the only things saving HDs are their large capacity and price and those margins are narrowing. Here, we put SSDs in all places where their capacity is not a limitation--HDs are only used for large archives.
Clearly, manufacturers have to make drives cheaper but they are already cheap compared with some years ago. I recall having paid nearly $2000 for a 100MB drive not that long ago--that's about 20 times as much for a piddling capacity by today's standards. It seems to me that we've gone too far when the average HD cannot be relied on to operate for more than a couple of years.
Still, SSDs aren't a long term solution either. The startling facts are that we've no really long term high capacity storage. Paper records and photography can last hundreds of years but DVDs, HDs etc. only have a tiny fraction of that life. Only a month ago I played--or tried to play--a DVD that I purchased some 5 years ago but it would not. Upon inspection, it was deteriorating: in places the 'silvered' surface had become translucent.
Long term data storage is now a very significant problem awaiting a solid technological fix. In the meantime, HD manufacturers aren't helping one iota and they'll only fix the problem when we customers start walking elsewhere. SSDs are a convenient temporary measure--not to mention a good weapon to threaten drive manufactures with, and they're well aware of that threat.
Very astute.
Archiving is an issue that is not being addressed.
Currently, digital libraries must survive by copying from one media to another before they deteriorate.
But when you think about it, archiving has always been full of risk. Even paper archives are subject to fire( ref. Alexandria). Even from the earliest days, replication was how knowledge was spread and survived.
So, no matter what media is being used for storage, we will never rely on a single media.
Replication,copying,distribution is how it will have to survive, and we also need to ensure that we still have the methods to read it out while it still is fresh.
I wasn't aware that dedupe and compression were flash only technologies.
If they aren't then a side by side price comparison to platters is a more reasonable comparison since the efficacy of dedupe and compression both depend on the types and quantities of data being stored.
...and I'll say it again:
If spinny drive manufacturers made a 2.5" Hybrid that had say 64Gb of Flash, and say a 500Gb spinny drive, that appeared to the system as two separate drives (like any harddisk thats been partitioned, Flash for the OS, HDD for data) then I'd buy it in an instant. However, if the flash portion of the drive could be reduced in size (of the users choice) and the remainder gets used as a cache for the spinny bit, giving the best of all worlds, then I'd rip their bloody arms off!!
Are you listening HDD manufacturers??
Two key facts are missing from the article – the reliability of SSD versus platters, and the current capacity.
I look after 90 PCs, two of which are SSD. One of those SSDs failed after 9 months. I’ve not had a regular HDD fail at under 4 years or so for a long time now.
The capacity is also not high enough without the price going mad.
I think SMEs will be slow to adopt SSDs as these issues are resolved.
Just as servers are either pets or cattle, PCs are pets to an IT Manager such as myself due to the cost of replacement…
SSDs are certainly more rugged. For laptops that get banged about, they might be the better choice. SSDs aren't more reliable on the desktop. They don't give advanced warning of failure. One moment they're AOK, the next monent they're done for. They wear out and die - the more that is written to them, the faster they wear.
Nevertheless my employer is going over to SSD for the system drive of all future desktops. The speed-up (especially in boot and log-in times) is extremely significant. They also save quite a few Watts per PC (and again on the air-con). No irreplaceable data lives on the SSDs. Everything that matters is on "enterprise" big RAID-6 arrays of spinning rust in the server room (and incrementally backed up to another site every night). If the desktop's SSD fails, just throw it away, connect a new one, reinstall the image across the network, and carry on as if nothing happened.
"One moment they're AOK, the next monent they're done for. They wear out and die - the more that is written to them, the faster they wear."
Well, bad luck DOES exist. Apart that, You will find that SSDs inform You about the wear of the cells. Consider this SMART output (from an OCZ Agility 3, 120 GB). Only the relevant part is showed.
177 Wear_Range_Delta 0x0000 000 000 000 Old_age Offline - 3
230 Life_Curve_Status 0x0013 100 100 000 Pre-fail Always - 100
231 SSD_Life_Left 0x0013 075 075 010 Pre-fail Always - 0
241 Lifetime_Writes_GiB 0x0032 000 000 000 Old_age Always - 100712
242 Lifetime_Reads_GiB 0x0032 000 000 000 Old_age Always - 571
As You can see, it reports the life left on the devide (231 SSD_Life_Left).
Yes, it does not help with defects, but is good enough so we can replace a drive BEFORE it goes "read only".
Quote: "I’ve not had a regular HDD fail at under 4 years or so for a long time now."
...I have. Many. Far *too* many to call a hard drive reliable. And I'm only responsible for something under 80 computers.
In some cases the failure has been covered by warranty (drives with 2-, 3-, and 5-year guarantees) but the "free" replacement drive (a) often ends up costing almost as much as a new one, if you have to mail the damn thing to an overseas destination as I do in from South Africa, and (b) in any event is w-a-a-a-y less than the cost of your time to recover / rebuild and reinstall the machine.
The current "problem" with SSD isn't the technology, IMHO, it is the price/performance ratio. Everyone knows you get higher performance with SSD, but then again everyone knows you will pay a substantial premium for that performance at any reasonable capacity level (read: above 256MB, which in today's society of 12 to 24MP digital cameras and digital media rips goes very, very quickly). We are all waiting for prices to come down but, as the HDD mouthpieces note, there simply isn't enough fab capacity - so, we keep waiting.
Therefore, SSHD's seem to be the near-term logical alternative. If only the manufacturers would give us what we want: decent SSD capacity attached to good sized (over 500MB) 7200RPM drives. But they don't want to do that, they want to cut corners: 8GB SSD's attached to 5400RPM drives with 32MB caches.
No thanks.
Those specs simply aren't enough attraction to move from what we already have. When we upgrade, we want an upgrade, read: we want a substantial enough difference that we really notice the price/performance ratio jump, not just reading it on the page and only get to tell ourselves "It performs better!" because the difference is negligible.
HDD manufacturers, you want to stay in the game for the near-long term? Give us what we want!! 32GB (low end) to 64GB (high end) SSD attached to 7200RPM 500MB-1TB drives with either 32MB (low end) to 64MB (high end) caches. That give you 2 levels of performance to sell: 32GB SSD / 32MB cache and 64GB SSD / 64MB cache. We're willing to pay for it so fire the moron who is deciding to give us mediocre products with minimal specs and start spinning up production abilities on devices with real storage matched to real performance.
Current best consumer prices are in the range of $0.05/MB for large hard drives ($150 for 3 TB), and $0.80/MB for large SSDs ($200 for 250GB). Those numbers are not quite apples to apples comparisons, but I'd say that if a MacBook Pro cost 16 times what a Dell or Lenovo laptop cost, Apple would be out of business. SSDs are niche today (mostly for tablets and ultra high dollar PCs and servers). And of those, only the tablets can be considered normal consumer products.
In fact, I have only one SSD in my collection (256GB) and more than 200TB of spinning disk space (counting cloud space). I'm not likely to change over soon, and I wouldn't even have that SSD if there were any alternative. I admit it is 2-3 times faster, and easily 1/10 the weight. But with between 3 and 5 times the price of a comparable size 7mm drive and (my gut feel) half the expected lifetime, SSDs aren't there yet.
To take advantage of the metaphor, I would still invest in horse drawn vehicles in 1870 rather than automobiles. Henry Ford won't come along for another 40 years....
On the other hand, there are a lot of web servers where that 2x increase in speed is invaluable, and the cost of the drives is trivial compared to the box they go in. And the same applies to a top of the line gaming machine. So SSDs are certainly not going away either (like the 1870-1910 automobiles).
When products like the Hyper-Memory Cube hit the market next year with on-board flash, we will see the Bulk-Disk-Drive (BDD) tier under enormous pressure. Most of the drive products will fade rapidly, leaving just very high capacity 3.5 in drives.
Disk isn't going away, but like tape, no-one is going to talk much about it a couple of years from now.
I'd like to see some REALLY large drives for archiving. This is an area where spinning rust can hold a price based monopoly for quite a while yet. I'd be happy with slower spin and increased seek time, etc. Just put together a 5.25 inch full height jobby packed with platters, should be able to get 40 terabytes or more easily even using quite old technology. I don't care if it takes a couple of months to fill, as long as I can get to any file I want in a few 10s of milliseconds and read it at HD video streaming speed, that's all I need. I've never really trusted tapes and they're don't provide the random access I need. How's about it?
Yup, my thought too. Make them like the old IBM dustbin-size platters from the '70s, but with modern storage densities, and spin them real slow, with one travelling head to read the whole lot. This should satisfy FaceBook's storage requirements too. It's world energy tomorrow, but that's me done for the day - off for a beer now.
I'm guessing it won't work. At the very best you'd have tracks twice the length, but you'd have to slow down the disk rotation speed to half both to maintain the aerodynamics that floats the head and to reduce stress on the bigger disk to a manageable level. Data bandwidth would decrease as data size increased. At worst vibrational instabilities would kill the idea. (General engineering rule: the bigger it is, the less stiff it can be).
If the manufacturers can get the price of a 4Tb 3.5 inch disk down to the £35 which has been the price of every previous size of disk drive at the time it was superceded, they'll be in business for a while yet. Get HAMR and/or BPM to market and sell 40Tb disks for £35, and I doubt Flash will ever compete.
The dark horse is ReRAM. I think it may kill flash first, followed by disks, both within 15 years of today.
I know it's not the point of the article... but was anyone else stunned when they read the bit about disk arrays costing over $2/gb?!
With bulk HDD capacity at around $0.10/gb, the disk arrays have plenty of room to cut costs to be competitive with flash.
No wonder the network drives at the office are so small, new capacity costs a fortune.
My boot disk is two 128GB SSD, in a RAID 1 array, because when flash dies you will never get your data back; this size was picked because 64GB is far too small, but 128GB is plenty! The RAID 1 rebuild time is so much faster for SSDs than rotary disks. If you need more space, say for data and big games, just add extra disks, I'd suggest these be spinning disks, because SSDs are not cost effective for bulk storage.
Most of my other storage is in a FreeNAS PC box with six 3TB WD Red spinning disks, in a RAIDZ2 array, with a 16GB DDR3 (expandable) for ZFS caching, for about £1100. Just try doing that with SSDs, say twenty 0.9TB ones, and wide SATA/SAS PCIe cards, for about £12K and much more for server grade parts!
Basically SATA SSD is still completely useless for bulk storage, unless you can afford this massive price multiple; so for most people, it is still only really useful for fast modest sized storage.
As for PCIe flash ROFL! It's far too expensive, and seriously capacity limited by the number of spare wide enough PCIe slots! The more compact DDR SSD memory stick route is only viable for quite large and expensive rack mounted motherboards.
>FreeNAS PC box with six 3TB WD Red spinning disks
Yes, it's cheaper but it's also slower than the equivalent flash which is often the point. Flash disks can entirely flood a SATA channel now - which is the point in PCIe drives.
We need fast storage rather than lots of it, so flash works for us.
The big drawback against HDD is performance.
Drives are now 3 TB, and I am sure they will reach 10 TB or more.
But while density has increased 1000-fold over the years, the performance per drive has remained almost constant. 180 IOPS/s is pretty common for a modern HDD. That is about 1 IOPS per 20 GB, for a drive that is around $100-$1000 (consumer-enterprise). To get 5000 IOPS we would need 30 such drives, striped,
a cost of $3,000-30,000.
Flash can do 5k IOPS on a single drive. That can be had for $200-4000. The performance is so superior that enterprise flash outperforms(price performance) consumer HDD.
HDDs are not going go get any faster, and ultimately they will be used for backup only.
I am not very concerned about reliability.
Flash as well as HDD will be deployed with RAID, and DR replication.
And one thing that is always increasing is bandwidth, so there is no reason not to a DR solution and off-site backups.
Flash has very poor write performnce though.
I predict a return of DRAM as SSD to address this issue.
So we will have many layers of "Memory".
On chip cache, in incteasingly large microprocessors.
External L2-L3 cache.
DRAM
---
External DRAM with battery backup for write-intensive workloads with de-staging to flash in case of power falure.
Flash for the persistence layer.
HDD for backups, if very large storage is required, otherwise also backup will be on flash.
Tape libraries for long term backups (1Y+)
HDD will eventually be squeezed further and further down this list. That is quite inevitable.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
