Flash is dead ... but where are the tiers?

Flash is dead: it's an interim technology with no future. But yet it continues to be a hot topic and technology. I suppose I ought to qualify that. Flash will be dead in the next five to 10 years and I’m talking about the use of flash in the data centre. Flash is the most significant improvement in storage performance since …

[Anonymous Coward]

The simple, elegant solution...

... seems to be something along the lines of flash-backed memory* backed up to elsewhere, first to disk then to tape for long-term archival.

* You know, those sticks with RAM to do the work and flash for the power outages, with a supercap to do the writing on lights-out.

[James 100]

Re: The simple, elegant solution...

That makes a lot of sense in fact ... you could use the cheapest (i.e. lowest write-frequency) TLC Flash, since it's only getting written once each power failure; save a fair bit over the "average" write time, both by pre-erasing the whole device ready for the next dump and then by writing entire device sectors at a time. No need for fancy wear-levelling - the wear will be completely uniform anyway, with each sector getting a single write each power failure...

I'm sure I've seen devices going in this direction in the past - pure RAM disks on PCI(e) cards with an external PSU to keep the contents alive when the machine is off. A tiny niche market due to cost, of course, but that will change in time.

ZFS has had a Flash cache 'tier' of sorts (L2ARC) for a while, and even Microsoft's ReadyBoost could be argued as being a move in that direction. Given the price differential, do you really need to *move* data between tiers rather than accept the slight overhead of duplication for simplicity's sake? If you're paying through the nose for a few Tb of Flash cache, do you really mind having a few extra Tb of cheap spinning rust behind it? I'd prefer the robustness of "if the Flash caching device dies, the data's still safe, it just gets a bit slower without the caching".

[Storage_Person]

Re: The simple, elegant solution...

If you have enough RAM to store all of your data then you don't even need flash at the backend, just use disk. Streaming all of your memory to hard drives is not significantly slower than flash, and is certainly possible to do with a few supercaps.

In my opinion the biggest change in storage in the near-term is the one that moves high-end storage from being centralised to localised again. When there is a software layer that can handle dispersing data intelligently around the available infrastructure, ensuring availability and protection as well as good enough performance, *that* is when the storage landscape changes. Right now there really isn't a lot of difference between your low-end HDDs and high-end SSDs beyond $/GB and $/IOPS.

[Anonymous Coward]

Re: The simple, elegant solution...

How is this scheme different from any array sold in the last 15 years or more, from the cheap SATA cards that may include 64MB of NVRAM to VMAX with 256GB of mirrored cache?

Given that a VMAX that's SSD for 5% of it's storage and SATA for the other 95%, plus FAST 2.0 to handle keeping hot data on the SSDs provides around 10x greater IOPS in my experience (obviously depends on load, YMMV) your elegant solution may be simple, but it's far from optimal.

[Michael Habel]

Perhaps not the place to get into this but,

Given that you can pick up an old school External 2TB HDD with connections for USB3.0 for less then 90.00€.

BTW: Why are External HDDs almost always cheaper then there Internal cousins?

Where is the incentive to get a 250GB SDD for no less then TWICE this amount?

Call me greedy, but I for One demand more BANG [e.g. Available Space] for my Buck.

Why would some Bean-counter see this any differently?

If Flash can be saved. It needs to step up to the Platter Capacities 1TB+ and match damned well on price. A slight premium would and should be expected. But when a given 250GB SATA II Drive costs 'round 40.00€ you're talking about a 225% premium, and we're talking about the difference between filling up a Multi-bay NAS RAID Array or just the One low capacity Drive.

[Anonymous Coward]

Yeah !

Apples are cheaper than Oranges. That means you will only eat Apples from now on ??

[Michael Habel]

Re: Yeah !

As long as I don't starve to death. Then yeah I guess I would. But chalk it up to not enough Dollars and to much Sense. These Devices do essentially the same thing. I like to equate it as buying a VW Golf instead of an Audi R8. But hay we're not all in the Filet Mignon class like you seem to be.

[Anonymous Coward]

Re: Yeah !

I eat many more apples than oranges, but sometimes I can afford an orange. You can't ?

[Michael Habel]

Re: Yeah !

Not when I can have Three perfectly fat round Apples for the price of One measly disease ridden Orange with offers only a 1/4 of the of the Space. like I said, some People like the gas guzzling Veyron's, and R8s'.

No One is stopping you. But how can you possibly draw any value in this venture is beyond my scope.

[foxyshadis]

Re: Yeah !

The value is that it can be 10-20 times as fast even in the worst case scenario than your 2TB spinning 7.2K disk. You sound like you have no idea why anyone every bought 10K or 15K disks, either. Despite your proclamation of pointlessness for storing your torrent collection, flash drive sales are growing by an order of magnitude every year, to the point that external storage drives will soon be the only use for spinning disks.

[Mike Bell]

Damn! You got me going!

I thought the title was about Adobe Flash.

No such luck, that's still going to be clinging on for dear life in applications like BBC iPlayer.

[Anonymous Coward]

Electrical Engineers To The Rescue !

"Committing" a transaction to volatile memory (ie. using DRAM as an tier in the storage hierarchy) of course is quite fishy. Can you really depend on the backup power supply of the DRAM to work with high reliability ? Can you assure the same level of reliability as "written on magnetic material" or "written into flash memory" ?

If you don't do handwaving, you need three redundant DRAM copies which can each burn down without affecting the other. That means full galvanic isolation from external power/ground lines. It's doable, but it will require good engineers and some money.

But of course, DRAM mass storage would allow for very high transaction throughput. For some applications (e.g. OLAP) you don't need transactional guarantees, so you could skip the three-copy redundancy requirement.

[Disgruntled of TW]

Re: Electrical Engineers To The Rescue !

Deary me ... I trust we're not all assuming the transaction that matters is at the storage interface. It's one piece of a much bigger puzzle.

It's a sad fact that the majority of organisations don't have a complete view of where their "business" transactions actually take place, and by inference what must be persisted, and how, given various types of failures.

Don't we have to fix that, before we trumpet to the business "replicate the array and we're all good, buddy"?

Try bringing up 46 Windows VMs from a corrupt VMFS v5 partition on a 1TB LUN. Ain't happening. Even if they booted, and NTFS found MOST of the orphaned clusters on the vdisks, the apps are unlikely to be consistent especially if they use a binary DB of some description. But blindly we continue to trust ... "she's up - must be good".

Putting too much money into a single link in the technology chain supporting a transaction isn't good value, and doesn't solve the bigger problem. Spreading infrastructure investment appropriately across the entire transaction footprint, driven by business requirements and risk, is the way of the enlightened grasshopper.

I would argue that storage alone is not the answer here, nor is HSM (which mainframes had ohhhhh ... a while back), nor is the RDBMS, nor is transactional JMS - it's a little bit of everything, in just the right amounts. Just add beer.

Fetching coat, again.

[Anonymous Coward]

Re: Electrical Engineers To The Rescue !

My post was meant to make sense in the context of "database server comitting data to mass storage" and of course "reading from mass storage".

It's bleeding obvious that you should not "throw everything and the kitchen sink" on an expensive HSM system. Dumb users and IT depts will create massive costs. Why can't you store VMs on pure-disk storage ?

More generally, identify what is really valuable (and needs backups&redundancy) and what can safely be done on less expensive storage. Look at what drones store on network drives. Remove their private MP3 and JPG collections. Set Quotas and charge departments for "large" quotas. Make the cost of high-fancy and less fancy storage transparent to users. Only store important stuff in the RDBMS. Think about storage cost when developing apps.

[foxyshadis]

Re: Electrical Engineers To The Rescue !

Or just screw all of that and keep anything used regularly on the expensive fast disks, while everything, old or new, is on the stodgy cheap disks. It would cost more in manpower for one person to enforce that for even a mid-size organization than to just add another disk. Why is it so hard to extend the flash-cache concept that hybrid drives use to entire storage arrays? Supposedly the NetApp I work with has 15K vs 10K vs 7.2K tiered arrays, but it never works without a ton of manual intervention, to the point that we just say screw it and put anything large on the cheap disks, important or not.

[Anonymous Coward]

Re: Electrical Engineers To The Rescue !

@foxyshadis

for some time now I've been wondering how things would pan out if an array were built from the laptop hybrid drives - Momentus XT or similar. Fast access to the hot data, not much heat, no complicated controller, quite reasonable cost/GB - what's not to like?

My only reservation with the present Momentus XTs is that the amount of flash is relatively small.

[Anonymous Coward]

Re: Electrical Engineers To The Rescue !

The IBM XIV Gen3 already does that, as does some NetApp arrays.

Gen3 XIV can have up to 6TB of Flash based read cache.

[Arachnoid]

Perhaps we need to take another page from the sci fi manuals and look at some form of liquid data storage

[Ant Evans]

liquid storage

I'll drink to that.

[Arachnoid]

I'll drink to that.

But how much data could you absorb before unpreventable anomalies set in?

[Jim.Darragh]

Feedback from our customers suggests that storage will be a growth area for them next year and the ability to offer multiple tiers aimed at different requirements and different budgets is a key part of their strategy. Our customers are building award-winning IAAS services and offering multiple storage tiers using the Abiquo platform.

[JAA]

...

It's interesting how small shops like the one I am with have figured this out but massive manufacturers, huge VARS, and substantially sized IT ops can't figure it out. We sustain several hundred thousand transactions per second at peak (close to a million IOs), find a way to bifurcate data from sources avoiding costly ETL processes and do this all on whitebox servers and HP P2000 (MSA) arrays.

The key here is app development has never needed to contemplate things at the scale we are at today. Newer and younger app devs deal with this from the get go and we build to accommodate the loads with the budgets we have. Looking at expensive arrays and paying to have small amounts of flash act as cache blows my mind. We don't host a huge amount of data (70is TiB) but could easily scale this infinitely with no concern to equipment builds, disk types, memory types, etc...

Have fun out there!

[jcrb]

Its fascinating watching people make pronouncements about what technology will look like in 10 years. Saying flash *will* be dead in the next 5 to 10 years but that spinning rust will hang around for longer is not really a statement that holds up.

Sure, 2D floating gate NAND flash will be dead in a few years, replaced by charge trap NAND, and/or 3D NAND, but it will still look like flash as far as you can tell. At some point 3D NAND gets replaced but ReRAM or some other technology, but it will still look like flash as far as you can tell. At some point ReRAM or whatnot gets replaced by yet another technology that hasn't even been chosen yet, because its so far in the future, but it will probably still look like flash as far as you can tell.

Now as far as spinning rust goes, sure it will be around but its not going to look like the disk drive you are used to. HDDs are running into their own scaling issues, and once they go to shingled magnetic recording (SMR) they turn into something like a cross between flash and a file system. This is not the disk drive you are used to. If anything is going to be *dead* in the data center its the hard drive, too slow, too unreliable, too much power consumption.

As far as flash pricing goes when buying tier-1 storage you are buying performance not GB, if $/GB was all that mattered no one would buy anything but tape, or maybe slow SATA drives. The reason people buy 15K SAS drives is because they understand they are buying performance not bulk storage. And that is why people flash. Even for what some might call tier-2 storage used for VDI and other apps, people buy flash because disks are just too slow.

Enterprise storage is so much more than $/GB, disk storage could be free, but that doesn't matter if it takes more space than you have, more power than you have, more cooling than you have, can't provide performance for the applications that you have, or more importantly the applications you wish you had if only your storage was fast enough to run them.

As far as tiering goes (no I wasn't going to forget about tiering), while I could dispute the cost differentials of storage *systems* as opposed to just looking at component prices, I will just observe that I think saying it will never happen that applications will manage their data is pretty clearly not going to be the case, toss in app aware/guided file systems and/or hypervisors and I think that would definitely not be a safe prediction.

And I'm sure such tiering systems will come in handy for moving your data between your performance SLC storage, your bulk MLC storage, between your bulk MLC storage and your TLC archive/backup storage. Standard mechanisms will probably suffice to move your data to your long term SATA archive storage.

Don't believe that tiering will wind up being between memory types rather than between memory and disk. That's ok, people scoffed at the idea of disk to disk instead of disk to tape backups too.

[Chris Mellor 1]

Sent to me and posted anonymously

What about what IBM does with their object based file system on IBM i? It seems to work pretty effectively at putting the data that will benefit the system most on flash and leaving the rest on the spinners.

Do their patents ( or the radically different nature of the storage management ) mean that no one else can do this? I seem to recall the BeOS file system having some of the same features, too.

--------------------

Chris