Posts by dikrek

NAND chips are going to stay too pricey for flash to slit disk's throat...

Re: Some of us have been saying that for years

Hi all, Dimitris from HPE Nimble here (recoverymonkey.org).

Disk is indeed here to stay, but this doesn't mean ignoring AFAs is wise. What Infinidat is doing is akin to the saying "to the man with a hammer, everything looks like a nail". We get it - they have a decent hybrid.

Nimble (and others) have incredibly efficient and scalable hybrid systems, but also we have AFAs for the people that can't tolerate any latency variations due to the data not being in cache. Because 90% cache hit rate means those 10% of I/Os have to come from 7,200 RPM disks, with the latency that implies.

We can replicate from AFAs to hybrid to save costs, users aren't required to have all the same type of storage this way. Or have mixed hybrid + AFA clusters, with non-disruptive data movement between cluster members.

And with the Nimble ability to inline dedupe and compress for hybrid and AFAs (dedupe is now available for hybrid with Nimble OS 5), the price delta between hybrid and AFA becomes wider, not narrower. Yes, disk is here to stay.

And future very large capacity disks (both spinning and SSD) is one of the reasons Nimble RAID is Triple+ (5x more resilient than plain triple parity, and many orders of magnitude more resilient than other RAID types).

Still, I admit Nimble made a mistake to delay coming to market with an AFA. Our rationale was that the hybrid was SO GOOD that people didn't need an AFA. Sound familiar? :)

Yes, when one can cluster together 4 Nimble hybrid engines (each with 2 controllers) and enjoy huge capacity and performance in a single pool, it's easy to think an AFA is not needed. After all, that configuration could be set up with 400TB of cache, surely that's enough? :)

Ultimately, it was a mistake and the stock market punished us heavily for it.

I've presented in the same room as Brian from Infinidat, his entire thesis was that AFAs are unecessary.

If we are having these arguments we are not discussing things customers actually care about.

Thx

D

Enterprise storage sitrep: The external array party is over

Re: Centralized all flash is just an impressively bad idea.

Hi all, Dimitris from HPE Nimble (http://recoverymonkey.org).

Cheesy often responds like that in multiple posts, it's interesting to see his past posting history.

Based on the commentary I suspect he either works for or resells Microsoft. Which is perfectly fine, just be up front with it.

Anyway, is it possible to do grid-based storage go fast? Sure. Fast servers, fast drives in those servers, huge RDMA links between those servers, and it will be fast. Maybe not space-efficient but definitely fast.

Speed isn't difficult to achieve.

Reliable storage, with useful enterprise features plus speed, now that's the hard part.

Then there's troubleshooting. How easy is to troubleshoot the cobbled together (yet fast) grid system?

Especially if something really weird is going on.

How easy is it to get automated firmware updates for all the components the entire grid solution?

Do those updates take into account the rest of the environment that it's connected to and how the system is configured and used?

I could go on.

Doing storage right is very hard.

I've seen weird (and dangerous) bugs in drive firmware that, unless one is deeply immersed in storage and have all the right tooling and automation, would be next to impossible to figure out for someone selling SDS.

D out.

HPE's Nimble Secondary Flash Array uses... disk?

Re: Doesn't matter what you call them, doesn't change what they are

Hi all, Dimitris from HPE-Nimble here (http://recoverymonkey.org)

Nimble global inline deduplication in the SFA is extremely optimized so that what the user ends up with is a hybrid system that's really cost-effective and space-efficient, capped at about 40K r/w IOPS.

It's fully compatible with existing Nimble estates so it can be a replication target, etc. But it also has extremely deep integration with products such as Veeam.

The typical use case is things like backups that you can actually use to do stuff with (testing, development, whatever) without needing to restore first.

But I can also see people buying this to be even a general-purpose, very efficient system, as long as their performance needs fit what it can do.

If one needs to go very fast and have great data reduction, then a full-blown AF is of course the preferred vehicle. Less expensive yet fast and with decent data reduction, then a CS hybrid is still the answer.

Hybrid systems from other vendors typically can't sustain large-scale inline deduplication (notice how most vendors that offer AFAs and hybrids only offer the cooler inline data reduction tech for AFAs only).

Thx

D

Infinidat puts array to the test, says it 'wrecks' Pure and EMC systems

Re: block sizes matter

Hi all, Dimitris from Nimble, an HPE company.

@Vaughn - hey bud, transactional applications do transactional I/O in the 4/8K range, not 32K. http://bit.ly/2oUDNFI

Are you saying Pure arrays massively slow down when doing transactional 4/8K I/O, and are only fast for 32K? What happens at 256K or above, which is a typical I/O size for apps that aren't doing transactional things?

I can't comment on the validity of these specific results, but I'll say one thing: unless Infinidat supplies the exact testing parameters, it's hard to be sure the testing was performed correctly.

In general, when seeing any test, you need to know things like:

- number of servers used

- number of LUNs used

- number of fabric ports (and their speed)

- server models & exact specs, switch models, HBA models, host OS version, all host tunings that differ from the defaults

- benchmarking app used

- detailed I/O blend

- amount of "hot" data - even with a 200TB data set, if the amount of hot data is 1TB and that fits in RAM cache, the array will likely be fast. If you want to stress-test caching systems, my rule of thumb is: The amount of hot data should exceed the amount of the largest RAM cache by 5x.

A couple of vendor-agnostic articles that may help:

http://recoverymonkey.org/2012/07/26/an-explanation-of-iops-and-latency/

http://recoverymonkey.org/2015/10/01/proper-testing-vs-real-world-testing/

Thx

D

Why did Nimble sell and why did HPE buy: We drill into $1.2bn biz deal

Re: Why It Makes Sense

Hi John, Dimitris from Nimble here (John and I have known each other for a long time BTW).

Actually Nimble's InfoSight and NetApp AutoSupport are only very vaguely similar. Sure, both rely on some data being sent from the system back to the mothership.

This might help everyone understand - if it's any consolation, Nimble in general hasn't done a great job explaining the true differentiation of InfoSight.

http://recoverymonkey.org/2017/03/14/how-nimble-customers-benefit-from-big-data-predictive-analytics/

Thx

D

Nimble: Just as well our cloud storage runs in our own cloud, eh, eh?

Re: What about Zadara Storage, Chris?

For the Nimble competitors popping up all of a sudden (lots of comments from AC and people that registered just now and have 1 post):

Unless your offering truly is as simple as "I need X capacity attached to Y VM with Z SLA", then it's not really comparable to the NCV service from Nimble.

NCV is a "enterprise-grade volumes as a service" offering. It's NOT a "virtual or physical array as a service" offering.

When people want to consume cloud storage, they typically don't want to have to manage yet another array, even if it's virtual and living in the cloud.

If with your "cloud" offering a customer has to worry about things like:

- RAID (whether physical or made up of AWS or Azure disks)

- disk replacement (whether physical or virtual disks)

- drive counts, speeds and capacities (to go with the RAID above)

- firmware updates

- controller CPU and RAM

- software licenses

- cache

- pools

- tuning

Then you aren't truly offering what most cloud consumers are asking for. You're offering yet another array, that just happens to be living in the cloud. And missing the point.

D

Re: Are NetApp Offering This?

We didn't enter the market to validate your message, @Orsebox... ;)

The big differentiator Nimble is offering with NCV is analytics beyond what anybody else is doing (think troubleshooting things well outside the storage, for instance), and extreme reliability. The flexibility, fast backups/recoveries/clones, performance and ease of use are welcome bonus items and we are happy we are also providing those.

It was more about providing the amazing benefits of InfoSight to cloud customers. The InfoSight benefits are something no other company is even remotely close to having. And very very hard to replicate - we've been gathering and analyzing gigantic amounts of infrastructure telemetry for years, and have unique insights due to both the sheer amount of data, and also the time spent, elbow grease and sheer engineering brilliance. It's the cornerstone of what we do.

This is really why I joined this company. The big data analytics capability is a game changer.

It's also humorous what other vendors think when they hear the "analytics" word. Most think of basic trending and performance data or maybe a bit of what if analysis.

Can you figure out whether an unrelated piece of the infrastructure is responsible for any weirdness in behavior, and why? And what to do to fix it?

"Analytics" doesn't quite cover it since there are AI and machine learning components in addition.

Thx

D

Re: Nothing at all like NetApp does - and that's good

NetApp does not provide a cloud Storage as a Service offering. Some partners buy NetApp gear and offer various services (and they ALSO do that for Nimble, 3Par, EMC etc etc - nothing unique there, the classic case of service providers buying storage gear and reselling portions as a service).

NCV is different: it's a straight Nimble offering, with Nimble doing all the procurement, management, support, provisioning, orchestration and accounting.

It's as simple as that: cloud STaaS offered directly by Nimble.

Thx

D

Nothing at all like NetApp does - and that's good

@AC, this has NOTHING to do with what NetApp does (and I know - having worked there for many years, an experience for which I'm still thankful).

What NetApp does is simply put customer systems at Equinix data centers connected to AWS & Azure. But then customers have to own and manage those arrays. It's just collocation. Every other vendor does something similar. Nimble's equivalent program is called Direct Connect, and it's exactly the same concept. It's not rocket science. Anyone can easily do this and it's not exactly groundbreaking stuff. But it's very convenient if you want to both own an array of your choice and do cloud bursting.

However, Nimble's NCV offering is NOT the same thing. NCV customers never own, see or manage an array at all. All they see is a portal to a cloud service and they can select what they want, and keep it for as little or as long as they want, even if it's just 1 volume. No requirement to own and manage an ENTIRE array. Nothing to patch, either. It's a true cloud service.

And for those wondering about the other kind of ONTAP: NetApp's ONTAP Cloud is ONTAP VMs running in AWS or Azure (with all that entails). Again, customers have to still own and manage ONTAP, plus troubleshoot it. In that case it's still like owning an array, it just happens to be virtualized, and it's faster to acquire than a physical system. ONTAP Cloud write performance is ultra slow BTW, unless they fixed that since I left almost a year ago.

Why not join the NCV beta program and try it out? ;)

Thx

D (the Dimitris mentioned in the article, in case there was any confusion)

Nimble offers AWS and Azure cloud-wrapper

Re: Nimble are only imitating what Zadara Storage has delivered for the last 6 years

I think nobody will raise their hand... :)

For the Zadara employee:

Don't confuse features with risk mitigation. No company can be perfect at doing block, NAS and object.

With NCV we are focusing on enterprise block as a cloud offering. Lower risk cloud block storage.

In addition, NCV gives customers the choice of easily repatriating the data to on-premises if needed.

Most customers want, as a risk mitigation strategy, the ability to have compatibility between on-premises and cloud. We are offering exactly that - a multicloud service that is also compatible with physical on-premises storage if need be.

Not sure on Zadara's size but Nimble has over 10000 customers now and most of them have more than 1 system, so we aren't exactly small. We use well proven technologies in NCV.

For more articles on ways to provide risk mitigation for block storage, visit recoverymonkey.org.

Thx

D

Re: Soo let me get this right

Correct. Use AWS for what it's best: stupendous amounts of easy to deploy compute and object storage. Even block storage if the workloads aren't critical.

But for more serious workloads, there are great benefits to be found in a hybrid public cloud (maybe a new term? Time will tell).

It's like everything else: you don't need to source everything from the same place unless there are compelling reasons to do so and little risk.

Already there are companies that successfully use products from 4-5 public cloud services.

This is another one of those cases.

Once more, with feeling, visit these 2 links:

https://aws.amazon.com/ebs/details/ (search for AFR)

http://recoverymonkey.org/2017/02/27/nimble-cloud-volumes-removing-risk-from-cloud-storage/

Thx

D

Re: AWS EBS has a 0.2% AFR - according to Amazon!

For performance- and integrity-critical workloads that still want to be in the cloud, NCV is a great solution. You want to take instant snaps and clones? Recover instantly without any performance impact? NCV is a great solution.

But by your comment it's clear you're not getting how NCV is connected. It is NOT on-premises. It is a purely cloud service.

Re: AWS EBS has a 0.2% AFR - according to Amazon!

Because S3 simply isn't block storage. You can't (yet) run, say, Oracle, SQL etc. on S3.

The more successful businesses that use AWS storage use S3 but most of them were also designed from the beginning with that protocol in mind.

Re: AWS EBS has a 0.2% AFR - according to Amazon!

@Adam 52 - Thanks for the comment. EBS is 5 nines for what? It's "designed" for 5 nines uptime, "designed for" and "achieving" are two different things.

In any case, the data integrity is the bigger issue, at 0.2% AFR.

The "millions of times more durable" statement simply takes into account data integrity. In actuality it's a conservative number.

Let's not confuse site failure with data integrity.

In addition, the EBS clones and snaps do happen in the background but there's I/O variability while the blocks are copied to/from S3.

Furthermore, what happens when someone wants to recover a volume that was snapped? How long does that take? How is the performance SLA impacted?

The AWS service that has truly good integrity is S3.

Thx

D

AWS EBS has a 0.2% AFR - according to Amazon!

Hi all, Dimitris from Nimble here.

Amazon readily admits EBS isn't very durable. To wit from here: https://aws.amazon.com/ebs/details/

"Amazon EBS volumes are designed for an annual failure rate (AFR) of between 0.1% - 0.2%, where failure refers to a complete or partial loss of the volume, depending on the size and performance of the volume. This makes EBS volumes 20 times more reliable than typical commodity disk drives, which fail with an AFR of around 4%. For example, if you have 1,000 EBS volumes running for 1 year, you should expect 1 to 2 will have a failure. EBS also supports a snapshot feature, which is a good way to take point-in-time backups of your data."

So that includes the replication etc.

Some info on the Nimble Cloud Volumes offering:

http://recoverymonkey.org/2017/02/27/nimble-cloud-volumes-removing-risk-from-cloud-storage/

The "1 million times more durable" marketing number we quote is actually way too conservative.

Thx

D

That is pretty, er, Nimble. Storage firm claims 'six nines' availability

Re: Nimble fluff

Interesting since we formally announced 10000 customers (https://www.nimblestorage.com/blog/10k-customer-milestone-blog/). And the number keeps growing.

Re: With downtime it's more like 2 99s

You're referring to an obsolete doc from 2014 where we hit the 5 nines. Since then we did a bunch of work to go to 6 nines.

As someone else stated, this stuff isn't at all easy, and it takes years of hard work to get the extra "nines".

This is the most aboveboard company I've worked at, when I first joined last March I saw internal stats from the 2.3.9.2 codeline and newer. The uptime was already comfortably over 6 nines for those releases.

I asked, "why not tell the world", and I was told that's not how Nimble measures - there's no cherrypicking, NO counting of only specific releases (unlike a competitor that recently also announced 6 nines).

All releases are counted, including older ones that are at 5 nines.

We waited until the true average uptime across ALL releases was over 6 nines before making this public.

All this means is that anyone with a release that's 2.3.9 and up will have 100% uptime most likely (most of our customers are at 100%).

And for whoever asked how many customers we have: the number keeps growing. Was about 9500 last I checked but might be over 10K now. Haven't looked, but the actual number is not the important point.

The important point is we don't cherrypick the reliable ones. If we did that we would just claim 100% and be done with it ;)

But, ultimately, our happy customers are testament to the reliability of the gear. Ask your local account team to get you in touch with various customers and hear the stories.

And for those of you that work at competitors and posting anonymously - grow a pair and state your affiliation. If nothing else, it's good manners.

Thx

D

There are no maintenance windows for firmware and hardware upgrades, these happen non-disruptively (a few seconds for a controller failover) so they are not counted.

In addition, if we detect an actual outage, we investigate. If it turns out the outage was because of something like a customer shutting down the gear in order to move it or other similar major site-wide maintenance activity, we will not count that as downtime. Otherwise it’s all counted.

The key thing is that this is measured

Hi all, Dimitris from Nimble here (http://recoverymonkey.org).

This is true measured data across over 9000 customers running all versions of the code, new and old. Unlike some vendors that say "designed for 7 nines" (and across a handful of customers and very specific code levels).

I don't so much care what it's theoretically DESIGNED for, I care more about what it can DO ;)

You can see the evolution of code uptime here:

https://www.nimblestorage.com/blog/six-nines-availability-across-more-than-9000-customers/

Thx

D

Kaminario pumps up K2 all-flash array processor speed and SSD capacity

Re: The Great Block Size Scandal

Oh - and if you are a Nimble customer, go to InfoSight and look for the new "Labs" tool.

This will show you I/O histograms per app, per volume or per whatever you want.

So you can see the true I/O distribution for each application.

Highly educational.

Re: The Great Block Size Scandal

Actually, there IS research that links block size to specific apps. More like ranges of block size per app.

For example, certain DBs will do random I/O in the 8K I/O size range, and sequential in huge blocks, while doing redo log writes in 512 byte sequential appends. All very predictable. Especially if one has the tooling to do this kind of I/O research per application...

This is why the "average I/O size" makes no sense whatsoever.

https://www.theregister.co.uk/2016/09/26/getting_nimble_about_storage_array_io_profiling/

https://www.nimblestorage.com/blog/busting-the-myth-of-storage-block-size/

Thx

D

(Disclaimer: I work at Nimble Storage but "average I/O size" is something that's always irked me).

You better layer up, Micron's working on next-generation XPoint

Re: What about NVMe?

NVMe is a protocol. 3D Xpoint is a way to build solid state storage.

For the fastest speeds, 3D Xpoint needs a memory interface anyway, not NVMe. But there are only so many DIMM sockets you can have, plus it's not like you can use a memory interface easily in order to connect hosts to arrays.

http://recoverymonkey.org/2016/12/09/practical-considerations-for-implementing-nvme-storage/

Disclaimer: I work at Nimble but this is a general media discussion.

Thx

D

Pure Storage is betting its FlashArray farm on NVMe

So many Captain Obvious points...

Hi all, Dimitris from Nimble here.

My viewpoint was posted here:

http://www.theregister.co.uk/2016/11/25/nvme_storage_sea_change/

I did state that even without NVMe drives in the array, speeds would overall be improved if the client side adopts NVMe over Fabrics.

In general, this whole business with NVMe is all about vendors trying to get mindshare about a technology that everyone will have anyway. Trying to build differentiation where there is none.

It's pretty simple:

1. The NVMe protocol itself helps increase headroom a bit, which means arrays will get a bit more efficient

2. It shaves 15-20 microseconds of latency at the protocol stack (which isn't significant for NVMe SSDs - 100 microseconds vs 115 microseconds won't set the world on fire)

3. AFA controllers are already maxed out with current-gen SSDs.

Nimble's architecture is NVMe-ready, and other vendors' too. It's not rocket science, it's more about waiting for the right bits to drop in price and customers to be willing to adopt NVMe all the way to the client.

The more exciting tech is next-gen byte-addressable storage like 3D Xpoint and the like, sitting in a DIMM. Not everyone is ready for THAT tech... ;)

FYI, NVMe is positively glacial in every way compared to non-volatile memory in a DIMM. Nimble has been using byte-addressable NVDIMM-N instead of NVRAM for a while now...

Thx

D

Storage newbie: You need COTS to really rock that NVMe baby

NVMe over Fabrics is of course needed to fully realize NVMe benefits

Hi all, Dimitris from Nimble here (http://recoverymonkey.org). Clarification:

1. Putting NVMe drives and/or something like a 3D Xpoint DIMM in existing (modern) arrays can improve speeds, up to a point.

2. Implementing NVMe over Fabrics is necessary to unleash the total end-to-end performance all the way to the client.

3. Beware that speed doesn't get in the way of enterprise features, especially things that mitigate risk like multi-level checksums, snaps, clones, replication, encryption etc. Many devices out in the market are focusing on speed so much that they are ignoring even basic creature comforts.

The challenge really is that most customers move cautiously and aren't always ready to adopt things that have barely been standardized, especially in low risk tolerance environments.

Thx

D

Software-defined traditional arrays could be left stranded by HCI

It's not about HCI per se

It's interesting to examine why some people like the HCI model.

Part of the allure is having an entire stack supported by as few vendors as possible. Very few HCI vendors fit that description.

The other part is significantly lower OPEX. Again, not all HCI vendors shine there.

And for CAPEX - the better HCI vendors that fit both aforementioned criteria tend to be on the expensive side. So it's not about CAPEX savings.

It's also interesting that the complexity of certain old-school vendors has quite a bit to do with certain solutions becoming more popular (not just HCI). Compared to certain modern storage systems, you may find that the difference in ease of consumption is minimal.

Be careful that just because you like the HCI dream you don't give up things that have kept you safe for decades.

Case in point: several HCI and SDS vendors don't do checksums! (Even VSAN and ScaleIO only recently started doing optional checksums).

That's like saying I like electric cars, and in order to achieve the dream of having such a car I need to give up on ABS brakes.

Or things like proper firmware updates for all the components of the stack. Again, many solutions completely ignore that. And that inability can significantly increase business risk.

More here:

http://recoverymonkey.org/2016/08/03/the-importance-of-ssd-firmware-updates/

Disclaimer: I work at Nimble but if I didn't there's only one HCI vendor I'd go to work for. One. Out of how many?

Thx

D

IO, IO, it's profiling we do: Nimble architect talks flash storage tests

Re: But Nimble's data is mostly SMB customers

Not at all true. Nimble's install base ranges all the way from huge to SMB. We have detailed data for all kinds of enterprise apps - Oracle, SQL, SAP, plus more exotic ones like HANA, SAS, Hadoop.

Plus the more pedestrian VDI, Sharepoint, Exchange, file services...

The interesting thing is that Nimble's and Pure's stats actually agree

What differs is each company's reaction to the math and their ability to drill down in even more detail so that one knows how to automatically treat different kinds of I/O even within the same app.

For instance, DB log files always expect extremely fast response times. That exact same DB doing a huge table scan expects high throughput but the table scan is not nearly as latency-sensitive as being able to write immediately to the log.

Being able to differentiate between the two kinds of I/O is important. Being able to act upon this differentiation and actually prioritize each kind of I/O properly is even more important.

It's far more than just letting random I/O through like Nate's drawing of what 3Par does. It's about knowing, per application, how to properly deal with different I/O classes intelligently, and accurately alert with very little noise if there is a problem.

Thx

D

Allow me to clarify

Hi all, Dimitris here (recoverymonkey.org).

In case it wasn't clear enough from the article:

1. Pure is variable block, their very own results show the bimodal nature of I/O (clearly split between small block and large block I/O), yet they keep talking about 32K I/O sizes which is the (correct) mathematical average of overall throughput. Nimble's research shows that using the throughput average isn't the optimal way to deal with I/O since the latency sensitivity hugely varies between I/O sizes and apps.

2. Nimble arrays are of course variable block and in addition application-aware. Even the latency alerting takes into account whether I/O is latency-sensitive or not, which increases alerting accuracy by 10x.

3. Nimble's research shows that you need to treat different size and type I/O accordingly. For instance, large block sequential doesn't need to be given preferential latency treatment. Small block random I/O on the other hand is more latency-sensitive depending on the app. Nimble will automatically take into account app type, I/O size, latency sensitivity and randomness in order to intelligently prioritize latency-critical I/O in the event of contention to protect the user experience where it counts. This is an extremely valuable ability to have and helps automatically prevent latency spikes in the portion of I/O that actually cares about latency. See here: http://bit.ly/2cFg3AK

4. The ability to do research that shows detailed I/O characteristics by app instead of the whole array is what allows Nimble to be so prescriptive. The analysis we do is automated and cross-correlated across all our customers instead of having to ask someone in order to find out what apps an array has on it.

Final but important point: This is a disagreement about math, not about the actual I/O prowess of the two vendors. Both have excellent performing systems. But my problem with the math is when it starts getting misleading for customers.

The other disagreement is talking about hero numbers at what looks like 100% reads. Sure they look good but if the performance drops to half when doing heavy writes then it's not that exciting, is it?

Thx

D

Mangstor tells IT managers: Hey SANshine, c'mon in, the fabric is fine

There's performance, then there's RAS and functionality...

Front-ending other arrays in and of itself and adding a cache layer isn't horrendously difficult.

There are a few ways to do it.

Often this involves making the hosts use the virtualizing array as their target.

If this is done, some things to consider are:

1. How is multipathing handled by the virtualizing system? As well as the underlying array?

2. How are checksums handled? How is the virtualizing array able to ensure the underlying array has proper integrity?

3. How does the virtualization affect underlying functionality like replication, QoS, clones, snaps? Existing scripts? Existing orchestration software?

4. If the front-end array will provide only cache, how will flushing be handled? And how does that affect overall reliability?

5. What's the process for front-ending? Migration? Or just path switchover? (both techniques are used by various vendors, migration is obviously more painful).

6. If I want to remove the front-ending device, is that as easy as path switchover or more convoluted?

Thx

D (disclosure: Nimble Storage employee, http://recoverymonkey.org, and overall nerd).

SPC says up yours to DataCore

Re: Why use and array of any type anyway?

Ok Cheesy I'll bite.

In almost every storage-related post (I've been keeping track) you mention these behemoth server-based systems and how fast they are. We get it. You don't like centralized storage and believe server-based is faster. To get this out of the way: properly designed server-based storage can be extremely fast - but not necessarily less expensive.

However, enterprises care about more than just speed. Far more. Otherwise we'd all just use RAM disks.

Out of curiosity, does Windows-based storage do things like live, automated SSD firmware updates regardless of server vendor?

What about things like figuring out how to automatically stage firmware updates for different components including the server BIOS, HBAs, the OS itself, and other components?

Or analytics figuring out what patches are safe to apply vs what bugs other customers with similar setups to yours have hit in the field? (maybe a certain patch will break your specific config).

How about analytics figuring out what exact part of your infrastructure is broken? Based on advanced techniques like pattern matching of certain errors, machine learning and predictive analytics?

Does server-based storage provide comprehensive protection from things like misplaced writes and torn pages? (Hint: checksums alone don't do the job).

In addition, are you running NVMe over fabrics? Because simply a fast Ethernet switch isn't enough to maintain low latencies. Or are you doing SMB3 RDMA? And what if my application won't work on SMB3? Maybe it's an AIX server needing crazy fast speeds?

Are the servers perchance mirrored? (Since you need to be able to lose an entire server without issue). If mirrored, doesn't it follow that 50GB/s writes to the cluster will result in an extra 50GB/s of intra-cluster traffic? Isn't that wasteful? (server-based should really be triple mirrored BTW).

And if erasure coding is employed, how is latency kept in check? Not the best mechanism to protect NVMe drives at scale.

Honestly curious to know the answers, maybe server-based really is that cool these days.

Thx

D (disclaimer: Nimble Storage employee)

DataCore drops SPC-1 bombshell

Re: ByteMe, you're forgetting something Is SPC-1 still relevent with massive cache systems?

<sigh>

Huawei had a 68TB data set and 4TB cache, or a 17:1 dataset:cache ratio.

Datacore about 11TB and 2.5TB cache, or a 4.4:1 dataset:cache ratio.

Not even close.

You probably work for datacore and are understandably excited. It's a good number and does show that the engine can provide a lot of IOPS when unencumbered by back-end disk.

BTW: Here's why not all data in SPC-1 is hot.

In the spec here: http://www.storageperformance.org/specs/SPC-1_SPC-1E_v1.14.pdf

Look at page 32. Notice there's an "intensity multiplier" AND a "transfer address".

ASUs 1 and 2 do reads and writes, and are 90% of the data.

If you look at that page you'll see that there are 3 hotspot zones with clearly identified multipliers and limited transfer address ranges:

- ASU1 stream 2

- ASU1 stream 4

- ASU2 stream 2

For example, ASU1 stream 2 has a multiplier of .281 (28.1%) and an address range of .15-.2 (merely 5% of the total data in the ASU).

If you do the math you'll see that of the total capacity, 6.75% across those 3 streams is significantly hotter than the rest of the data. So if that 6.75% comfortably fits in cache in any of the systems in the SPC-1 results, there will be a performance boost.

Not all the systems measured in SPC fit that percentage in cache, but many do.

Datacore went a step further and made cache so large that the entire test dataset is only 4.4x the size of the cache, which is unheard of in any enterprise deployment. How many of you will have just 11TB of data and 2.5TB RAM? Really? :)

And, last but not least, something anecdotal - all I will say is we had Datacore fronting some of our systems and the overall performance to the client was overall less than native.

The servers used weren't the behemoths used in the SPC-1 test of course. But this may mean something to some of you.

Thx

D

DataCore scores fastest ever SPC-1 response times. Yep, a benchmark

Re: NetApp comments -- nothin but FUD, dispelled here...

<sigh>

Go to the spec here: http://www.storageperformance.org/specs/SPC-1_SPC-1E_v1.14.pdf

Page 32.

Do a bit of math regarding relative capacities vs the intensity multiplier.

See, it's not uniform nor 100% hot at all. Different parts of the dataset are accessed different ways and certain parts will get more I/O than others.

i've been dealing with SPC-1 for years.

Not with NetApp any more but facts are facts.

Thx

D

Can NetApp's 4KB block writes really hold more data?

Re: What's so special?

Indeed, there are architectures that don't even have the concept of a block as such, let alone variable. They don't have this issue of partially filled blocks.

Still doesn't address the fact dedupe is within a volume domain and not global.

Nimbus Data CEO shoots from the hip at NetApp's SolidFire buy

Re: Means little

Hi Jeremiah,

As someone who, in a previous life, spent time in SPs (Acxiom) and large enterprises (Halliburton, United Airlines) I can tell you that balance is absolutely required in large shops.

Maybe that's the disconnect. I'm thinking of the potential in multi-PB or even EB deployments.

Most large SPs want capacity and performance tier differentiation, not just performance QoS and the ability to provision via an API.

For example, they will charge a lot less for NL-SAS hybrid capacity than SSD-based capacity, partly because it costs them less in the first place, partly because customers expect it to cost less.

They also have needs for different kinds of density for different kinds of data.

The $/GB to store something on 10TB low cost spinning disk are a tiny fraction of the costs of storing it in mirrored SSD.

This means that Service Providers naturally tend to have a variety of tiers instead of just one. Unless they're very very small, in which case a single box is enough. But then again those very small SPs also tend to be very cost-conscious.

SolidFire has had some success in that industry but the large Service Providers typically have much more than SolidFire. If anything, the percentage of capacity in the typical SP will be less SSD and much more of other stuff, as much as we'd all like that to not be the case :)

And finally, the overall number of SolidFire customers is very small.

Percentage-wise many of those customers may be Service Providers, but that doesn't mean significant penetration in that space.

D out.

Re: Means little

I'm the furthest thing from a hater. I just find it interesting that every time someone explains about inefficiencies in the architecture, people complain that it's not important, and "look at all the business benefits".

It IS true that the people SolidFire is aimed at ALSO care about things like power/space/cooling/cost.

Ignoring that sort of requirement for that type of business is just silly.

Attacking people for calling this out just because they work at competitors is probably not very productive.

Go through every post of mine on The Register. You won't find a single mention of SolidFire when I was at NetApp. Actually there is one, where I thought the competitive material they had vs ONTAP was silly.

Check my blog too. No mention of it there either. I never pull any articles. It's all there. No mention of the purchase, no glowing article about what kind of problems it solves.

Now, start thinking about possible correlation. And it's not that I'm an ONTAP bigot. I was involved in the EF and FlashRay projects too, and it's funny how wrong people get the FlashRay story. It was a kickass product. Never given a chance. Out of respect I won't say more.

BTW - I can always post anonymously. Like most vendor peeps seem to do.

Or just stop commenting altogether, like my friends advise me to. "Who reads the comments anyway" :)

Thx

D

Re: Means little

Instead of focusing on the size of nimbus, focus on what the man is actually saying.

Working for a competitor doesn't automatically render an argument meaningless. Logic is logic.

It is true that service providers value multiple things:

- automation

- ease of use

- predictability

But they also value:

- price/performance

- rack density

- low power and cooling requirements.

Hand-waving away the second set of requirements doesn't make sense.

A solution that is, best case, 40% usable:raw, and, best case, 4 TIMES less dense than some competitive products (some from the same parent company), grossly violates the second set of SP requirements.

It is what it is - customers need to carefully consider what's most important to them and act accordingly.

Thx

D (Nimble employee, ex NetApp, scribblings at recoverymonkey.org)

SolidFire's thin flash density eats more racks than HPE. What gives?

Re: A seriously first world datacenter/business problem

Correct, it's not so much about the size of the SSDs as what the overall efficiency of the solution is. SolidFire is still not very dense due to the architectural decisions I mentioned in my first comment. A lot of expensive SSD capacity gets wasted in any server-based grid storage approach. ScaleIO, VSAN are similar. Erasure coding is the enemy of low latency so server-based storage typically goes for 2-3 total copies of the data (ideally 3 copies to avoid exposure in the case of node failure but SolidFire doesn't do that kind of protection at all).

There's no perfect platform, it's all about choosing what's best for your business and it's always down to some sort of compromise:

Price, performance, density, capabilities needed, RAS.

Thx

D

An inherently inefficient architecture

SolidFire, due to the requirement for mirroring, leaving a full node's worth of capacity free (to withstand a node failure) and not wanting to be over 90% full on the remaining space left, means usable:raw for a 4-way config is closer to 30%.

Assuming the industry standard 5:1 on the 30% means a very low final multiplier (Effective Capacity Ratio).

See here:

http://recoverymonkey.org/2016/05/19/the-importance-of-the-effective-capacity-ratio-in-modern-storage-systems/

This is all fairly basic math. Solidfire even has a sliding calculator on their website and you can verify the ratios easily using that.

See here:

http://www.solidfire.com/platform/hardware

There's the other wrinkle of metadata handling that makes it harder for SolidFire to use large SSDs since there's a RAM ratio that needs to be enforced, similar to XtremIO.

The value of SolidFire isn't dense flash storage. If you want that, look elsewhere.

Thx

D (Nimble employee & ex-NetApp)

HPE bolts hi-capacity SSD support onto StoreServ

Re: Purpose?

Ok Cheesy I'll bite.

In almost every storage-related post (I've been keeping track) you mention these behemoth server-based systems and how fast they are. We get it. You don't like centralized storage and believe server-based is faster. To get this out of the way: properly designed server-based storage can be extremely fast - but not necessarily less expensive.

However, enterprises care about more than speed. Far more.

Out of curiosity, does Windows-based storage do things like live, automated SSD firmware updates?

What about things like figuring out how to automatically stage firmware updates for different components including the server BIOS, HBAs, the OS itself, and other components?

Or analytics figuring out what patches are safe to apply vs what bugs other customers with similar setups to yours have hit in the field?

How about analytics figuring out what exact part of your infrastructure is broken? Based on advanced techniques like pattern matching of certain errors?

Does server-based storage provide comprehensive protection from things like misplaced writes and torn pages? (Hint: checksums alone don't do the job).

In addition, are you running NVMe over fabrics? Because simply a fast Ethernet switch isn't enough to maintain low latencies. Or are you doing SMB3 RDMA? And what if my application won't work on SMB3? Maybe it's an AIX server needing crazy fast speeds?

Are the servers perchance mirrored? (Since you need to be able to lose an entire server without issue). If mirrored, doesn't it follow that 50GB/s writes to the cluster will result in an extra 50GB/s of intra-cluster traffic? Isn't that wasteful?

And if erasure coding is employed, how is latency kept in check? Not the best mechanism to protect at scale NVMe drives.

Honestly curious to know the answers, maybe server-based really is that cool these days.

Thx

D (disclaimer: Nimble Storage employee)

NetApp shrinky-dinks ONTAP 9: Will support 4:1 data reduction

Re: Beware of what is being included in the data reduction ratio

Oh, I resigned all right. Far too many bad decisions one after another. Far too much management fluff. Out of respect for my former colleagues I will not elaborate further. The market will show whether cancelling an extremely promising product like FlashRay and buying SolidFire instead was the correct decision, for example.

I had my pick of vendors to join and I joined Nimble. And it wasn't about the highest bidder or anything crass like that. I like having purpose and making a difference. And the products are amazing.

InfoSight is far, far more than Nimble markets it as, and the Nimble AFA systems have more innovation in them than several competitors combined, again under marketed.

Exciting times ahead!

Thx

D

Beware of what is being included in the data reduction ratio

Hi all, Dimitris from Nimble (ex-NetApp).

Several vendors show a single efficiency number that is nebulous and includes all kinds of stuff, including thin provisioning and snapshots. Diving into the true savings often reveals a much less compelling story.

It is more appropriate to be able to show what efficiency is coming from where. For instance, this much saving from dedupe, this much from compression, this much from clones, stuff like that.

A guarantee needs to be clear regarding what precisely is counted in the data reduction.

I've written a vendor-agnostic article on this subject:

http://recoverymonkey.org/2016/05/19/the-importance-of-the-effective-capacity-ratio-in-modern-storage-systems/

Thx

D

Sick of storage vendors? Me too. Let's build the darn stuff ourselves

Re: Anyone can build something small

Folks, it's all doable, just remember that something as seemingly simple as automatically managed online drive firmware updates can be of paramount importance.

Especially in this age of SSDs, drive firmware is updated rapidly - lots of corruption issues are resolved.

Not being aware of the issues is one problem. Not being able to update the firmware live is a different problem.

Find the release notes for firmware updates for some popular SSDs out there and you'll quickly see what I mean.

Thx

D

The kid is not VSAN: EMC buffs up ScaleIO for high-end types

Re: Ah the good 'ol quick jab

Actually my point is checksums are kinda assumed in storage, taken for granted.

Most people won't even think to ask since it's unthinkable to not do it.

Yet EMC wasn't doing this until recently (for those 2 products only, on their other storage platforms they do plenty of checksumming).

It's not trashing them, it's more about reminding people not to take this stuff for granted.

Ask!

Only now getting checksums? After all this time?

Hi all, Dimitris from Nimble here.

Is it just me or is the addition of checksums in both VSAN 6.2 and the latest ScaleIO a bit glossed over?

Checksums are immensely important to storage - so important that nobody in their right mind should buy storage that doesn't do quite comprehensive detection and correction of insidious errors.

My sense of wonder stems from the fact that EMC and VMware have been happily selling ScaleIO and VSAN to customers without mentioning this gigantic omission up until now. And now it's quietly mentioned among many other things.

Interesting...

Thx

D

Back-to-the-future Nexsan resurrects its SATABeast

Nobody thinks of torque and vibration?

Hi All, Dimitris from NetApp here.

I'm shocked anyone thinks a top-loading shelf full of heavy SATA drives is a good idea. You pull the ENTIRE THING out in order to replace a SINGLE drive??

How safe is that?

Both for drive vibration (you're shaking 60 rotating drives at once) and torque (such a system is HEAVY!)

There is a better way. Front-loading trays (imagine that). On our E-Series platform, the 60-drive shelf is divided into 5 slices, each with 12 drives.

Each slice is shock mounted, much lighter than all 60 drives, and slides out butter-smooth in order to replace a drive.

Thx

D

After all the sound and fury, when will VVOL start to rock?

Very few arrays can do VVOL at scale

Hi all, Dimitris from NetApp here (recoverymonkey.org).

This is why we allow 96000 LUNs in an 8-node ONTAP cluster, as of ONTAP 8.3.0 and up.

Thx

D

HPE beefs up entry MSA with a bit of flash

Hi all, Dimitris from NetApp here.

Not understanding the comparison to the FAS2500.

The FAS runs ONTAP and has an utterly ridiculous amount of features the MSA lacks.

A better comparison would be to the NetApp E2700, a much more similar in scope device to the MSA.

Of course, whoever provided the math (I doubt it was Chris that did it) might not have a very good case then ;)

Thx

D

NetApp hits back at Wikibon in cluster fluster bunfight

Re: Some extra detail

Hi Trevor, my response was too long to just paste, it merited a whole new blog entry:

http://recoverymonkey.org/2016/02/05/7-mode-to-clustered-ontap-transition/

Regarding your performance questions: Not sure if you are aware but posting bakeoff numbers between competitors is actually illegal and in violation of EULAs.

We have to rely on audited benchmarks like SPC-1, hopefully more of the startups will participate in the future.

I suggest you read up on that benchmark, it's extremely intensive, and we show really good latency stability.

Though gaming SPC-1 is harder than with other benchmarks, it too can be gamed. In my blog I explain how to interpret the results. Typically, if the used data:RAM ratio is too low, something is up.

Which explains a certain insane number from a vendor running the benchmark on a single Windows server... :)

Take care folks

D

Re: queue the Ontap Fan boys

<sigh>

Flash makes everything CPU and pipe bound. If anything, CPU speed is becoming commoditized very rapidly... :)

And yes, cDOT 8.3.0 and up IS seriously fast. We have audited benchmarks on this, not just anonymous opinion. I understand that competitors don't like/can't come to terms with this. Such is the way the cookie crumbles. Look up the term "confirmation bias".

Regarding SolidFire: It's not about speed vs cDOT. As a platform, SolidFire is very differentiated not just vs cDOT but also vs the rest of the AFA competition.

The SolidFire value prop is very different than ONTAP and performance isn't one of the differentiators.

What are some of the SolidFire differentiators:

- very nicely implemented QoS system

- very easy to scale granularly

- each node can be a different speed/size, no need to keep the system homogeneous

- ridiculously easy to use at scale

- little performance impact regardless of what fails (even an entire node with 10x SSD failing at once)

- the ability to run the SolidFire code on certified customer-provided servers

- great OpenStack integration

All in all, a very nice addition to the portfolio. For most customers it will be very clear which of the three NetApp AFA platforms they want to settle on.

Thx

D

Some extra detail

Hi All, Dimitris from NetApp here.

It is important to note that Mr. Floyer’s entire analysis is based on certain very flawed assumptions. Here is some more detail on just a couple of the assumptions:

1. Time/cost of migrations:

a. The migration effort is far smaller than is stated in the article. NetApp has a tool (7MTT) that dramatically helps with automation, migration speed and complexity.

b. It is important to note that moving from 7-mode to a competitor would not have the luxury of using the 7MTT tool and would, indeed, result in an expensive, laborious move (to a less functional and/or stable product).

c. With ONTAP 8.3.2, we are bringing to the market Copy Free Transition (CFT). Which does what the name suggests: It converts disk pools from 7-mode to cDOT without any data movement. This dramatically cuts the cost and time of conversions even more (we are talking about only a few hours to convert a massively large system).

d. NetApp competitors typically expect a complete forklift migration every 3-4 years, which would increase the TCO! Mr. Floyer should factor an extra refresh cycle in his calculations…

2. Low Latency Performance:

a. AFF (All-Flash FAS) with ONTAP 8.3.0 and up is massively faster than 7-mode or even cDOT with older ONTAP releases. To the order of up to 3-4x lower latency. cDOT running 8.3.0+ has been extensively optimized for flash.

b. As a result, sub-ms response times can be achieved with AFF. Yet Mr. Floyer’s article states ONTAP is not a proper vehicle for low latency applications and instead recommends competing platforms that in real life don’t perform consistently at sub-ms response times (in fact we beat those competitors in bakeoffs regularly).

c. AFF has an audited, published SPC-1 result using 8.3.0 code, showing extremely impressive, consistent, low latency performance for a tough workload that’s over 60% writes! See here for a comparative analysis: http://bit.ly/1EhAivY (and with 8.3.2, performance is significantly better than 8.3.0).

So what happens to Mr. Floyer's analysis once the cost and performance arguments are defeated?

Thx

D

Don’t get in a cluster fluster, Wikibon tells NetApp users

Some extra detail

Hi All, Dimitris from NetApp here.

It is important to note that Mr. Floyer’s entire analysis is based on certain very flawed assumptions. Here is some more detail on just a couple of the assumptions:

1. Time/cost of migrations:

a. The migration effort is far smaller than is stated in the article. NetApp has a tool (7MTT) that dramatically helps with automation, migration speed and complexity.

b. It is important to note that moving from 7-mode to a competitor would not have the luxury of using the 7MTT tool and would, indeed, result in an expensive, laborious move (to a less functional and/or stable product).

c. With ONTAP 8.3.2, we are bringing to the market Copy Free Transition (CFT). Which does what the name suggests: It converts disk pools from 7-mode to cDOT without any data movement. This dramatically cuts the cost and time of conversions even more (we are talking about only a few hours to convert a massively large system).

d. NetApp competitors typically expect a complete forklift migration every 3-4 years, which would increase the TCO! Mr. Floyer should factor an extra refresh cycle in his calculations…

2. Low Latency Performance:

a. AFF (All-Flash FAS) with ONTAP 8.3.0 and up is massively faster than 7-mode or even cDOT with older ONTAP releases. To the order of up to 3-4x lower latency. cDOT running 8.3.0+ has been extensively optimized for flash.

b. As a result, sub-ms response times can be achieved with AFF. Yet Mr. Floyer’s article states ONTAP is not a proper vehicle for low latency applications and instead recommends competing platforms that in real life don’t perform consistently at sub-ms response times (in fact we beat those competitors in bakeoffs regularly).

c. AFF has an audited, published SPC-1 result using 8.3.0 code, showing extremely impressive, consistent, low latency performance for a tough workload that’s over 60% writes! See here for a comparative analysis: http://bit.ly/1EhAivY (and with 8.3.2, performance is significantly better than 8.3.0).

So what happens to Mr. Floyer's analysis once the cost and performance arguments are defeated?

Thx

D