What benchmarks can tell you about your solid-state drives

When it comes to storage, what benchmarks to use, how to configure them and how to interpret the results has been the subject of many a heated debate. Benchmarks are supposed to provide empirical data that can be used as evidence for drawing rational conclusions. Of course, if you torture data long enough it will confess to …

[Nick Triantos]

Inline <name your efficiency> and Benhcmarks

As always a well written article. One important point when it comes to benchmarking, Most of these benchmarks have preceded arrays using inline efficiencies (compress, dedupe, zero detect).

Furthermore, a lot of these popular benchmarks tend to write either repeating patterns or zeros which constitute a highly compressible workload. Needless to say the perf values and latencies produced are highly unrealistic with very little work actually happening on the back-end.

Personally, I like vdbench because it allows testing with certain compression and deduplication ratios that can be more realistic vs a 1000:1 ratio. fio has that some of that capability as well but it's not as granular as vdbench.

Cheers

Nick

Disclosure: NetApp employee

[jaxboe]

Re: Inline <name your efficiency> and Benhcmarks

Fio has support for both specifying the level of "dedupability", and the compression ratio of the individual blocks. Additionally, even the default settings for write buffers ensure that it defeats basic dedupe attempts.

[John Tserkezis]

As an average Joe (at least nowadays) I use the ever-reliable "suck it and see" test method. Ignore the marketing data, take the tested reviews with a bucket of salt, and only trust how fast it's going to go in your actual rig. Sometimes I've shuffled things around using spares I have lying around, just to see if and what difference it's going to make.

Or sometimes I don't bother testing speed at all in cost-sensetive applications. Heck, if they want the thing to go fast, they should have opened their wallets more.

[Vaughn Stewart]

== Disclaimer: Pure Storage Employee ==

Trevor

I think you hit the nail not head with, "...the holy grail of benchmarking is sticking as close as possible to the exact applications that will be deployed on the storage in question and as close as possible to the exact access patterns." - in other words, take a copy of your applications and run them on the new platform.

IO benchmarks miss the mark because applications 1). process data not just transfer IO 2). applications have multiple IO streams 3). application IO streams vary in IO size per process.

it as just as critical to test performance under load as it is with faults (failures) and data services like replication, snapshots, clones and data reduction technolgoies.

Nice insight - keep up the good work!

- cheers,

v

[storman]

What about Load Dynamix?

I’m surprised that you didn’t mention Load Dynamix in your article. Load Dynamix offers comprehensive, easy to use workload modeling and load generation appliances for storage engineers/architects that are serious about understanding, comparing and predicting networked storage performance. It is not a benchmark, but a highly accurate, large scale simulation product designed for flash and hybrid array testing. It enables pre-conditioning, dedupe & compression testing, hot spot simulation and allows detailed “what-if” analysis on dozens of parameters such as block/file size distributions, file/directory structures, queue depths, etc in addition to the usual R/W, random/sequential, and data/metadata command mixes.

It is heavily used by nearly every storage vendor (EMC, NetApp, HP, IBM, HDS, Pure, Cisco, Oracle, Dell, Tegile, Nimbus, Skyera, etc) and an increasing number of Global 2000 enterprise IT and service provider organizations. It’s not inexpensive as it is designed for organizations spending $1M or more per year on storage who want to ensure I/O performance and cut their storage costs by 30% or more. Our customers, who are mostly former Iometer/Vdbench/Fio users, tell us repeatedly that it is really the only viable solution to cost-effectively analyze how existing installed workloads will perform on different vendors, products, technologies and configurations. A single 2U appliance scales to many millions of IOPS and doesn’t require the complex scripting and set-up of the open source tools.

[Trevor_Pott]

Re: What about Load Dynamix?

Until you commented here, I'd never even heard of it, to be perfectly honest. I guess I'll add it to my list of things to investigate in 2015...

[Anonymous Coward]

fio + blktrace can duplicate just about any workload

If you have a particular app whose I/O pattern you want to duplicate, you can use the Linux utility blktrace (btt) to store the I/O, and then replay it with fio. Quite handy.