back to article Dell EMC's Pure-crushing benchmarks are flawed, says, er, Pure

We've seen selected benchmarks that show Dell EMC Unity arrays giving Pure a hard time – and, unsurprisingly, Pure says the tests are "obviously flawed." The Unity arrays are reengineered VNX arrays that were introduced in May. All-flash versions were used in the two benchmarks below. An obvious caveat is that these are …

  1. Anonymous Coward
    Anonymous Coward

    I heard speeds and feeds were all the rage

    Back in 2009. If you are selling only on performance, you have probably already lost, or selling to legacy storage admins who will be out of a job in 5 years anyway.

    Good luck growing that market.

    1. Anonymous Coward
      Anonymous Coward

      Re: I heard speeds and feeds were all the rage

      On a related note, it shows how seriously EMC are taking Pure - EMC seldom (if ever) published any kind of performance benchmarks for their kit before......

    2. Anonymous Coward
      Anonymous Coward

      Re: I heard speeds and feeds were all the rage

      So is your logic that "speeds & feeds", as you put it, are not important? If so, then I think the guy who finds himself out of a job is going to be the one who agrees with you. Obviously speeds & feeds matter or there wouldn't be an //m10 & an //m70. Pure would have just released one system and said "Hey world this is all you need!"

      Take it a step further, why keep chasing bigger, faster, stronger CPU's? Why go after NVMe? These things aren't "application specific" they're very much generic. What about NVDIMMs or Storage Class Memory? There are multiple hardware advancements that exist for one reason only: Speeds & feeds matter. Your argument is that we've reached a point where most vendors are "fast enough" so we don't have to dive too far deep into it. Your point should also be that in no way shape or form should anyone buy an array where performance has to be managed. I agree on that point.

      That said, Dell EMC's test is obviously bogus--and this isn't the first time they've done that so it's no big deal. BUT there's some truth to where they're going here. Pure is definitely not the fastest array. It might even be the slowest AFA out there--which is why they recommend testing a specific application workload to measure its performance. AND that recommendation is flawed. Workloads change over time. Environments grow over time. If your array can't handle the changes or the growth then you're screwed.

      EMC is a bit of a joke, they always have been when it comes to this stuff, but let's not all of a sudden act like these things don't matter.

  2. Anonymous Coward
    Anonymous Coward

    Seriously, that's ridiculous

    If the results had a 10-30% differential then you could think that there might be an element of truth to it, but when one vendor is claiming to have 4-11x performance over the other one (and they're both AFAs) then the testing must be massively flawed. This is like an Audi RS4 vs BMW M4 and saying the Audi did 0-60 in 5 secs but the BMW did it in 9 secs.

    If you're going to lie, at least make it semi-believable.

    1. bitpushr

      Re: Seriously, that's ridiculous

      I work for NetApp, so neither EMC nor Pure, and even I don't believe the results. :)

  3. Marc 25

    Oh no, not THIS again!

    /waits for Howard to weigh in.

  4. Anonymous Coward
    Anonymous Coward

    Did someone mix the tables up these results are all over the map ? talk about inconsistent. It really wouldn't have been such a bad hatchet job had EMC's own results (using their own benchmark) not been so poor also.

  5. Anonymous Coward
    Anonymous Coward

    LOL

    Future pure customer here. We benchmarked the m50 and for this profile "16K IOPS (80% Read)" we measured ~ 330k IOPS.

    Nice try EMC ...

  6. Anonymous Coward
    Anonymous Coward

    AFA Presales

    Speaking as somebody who works in presales in the AFA industry... All vendors lie and all synthetic benchmark results are pointless.

    Also, water is wet and Turkeys don't like Thanksgiving.

    Next story please.

  7. Nate Amsden

    how can you not know

    how many SSDs are in the system? for Pure it says 40 (estimated) for each test. I tried looking for a link to the results in the article but did not see any.

    1. Anonymous Coward
      Anonymous Coward

      Re: how can you not know

      I assume that the numbers being posted are leaked from a POC. Pure's End User License Agreement prohibits any customer from publishing results. I have heard of some being threatened with lawsuits after they shared results. #WhatIsPureHiding

  8. Anonymous Coward
    Anonymous Coward

    I work for a Service Provider running a combination of IBM, EMC, NetApp and Pure storage and we've done cross-platform benchmarking for a number of hosted workloads. One customer recently moved their SAP batch processing from our EMC VMAX 100k All-Flash to Pure M//70 and saw their main batch go from 8 hours duration to just over 2 hours.

    As they say, there are 'lies, damn lies and benchmarks' and I'd take anything EMC say these days with regards to storage performance with a heavy grain of salt (and do your own real-world testing wherever possible).

    1. ptbbot

      "do your own real-world testing" - Quite.

  9. Anonymous Coward
    Anonymous Coward

    How to make Pure look bad (and other vendor tricks)

    The trick to making Pure's performance drop through the floor is to run your performance testing soon after the "pre-conditioning" phase. Filling the array first causes the garbage collection process to kick in, and if you try to do I/O while GC is active on a Pure array the results look ugly. That's how EMC do their testing .. yes we should all be shocked .. likewise we should be shocked about the way Nimble and Pure used to use IOmeter to generate workloads full of zero filled blocks in their testing which compressed / deduped themselves out of existence so everything came out of RAM.

    Few people will ever see the GC behaviour hit this badly on a shiny new array, but if you sweat the asset so you're pushing 80%+ capacity utilisation and start doing lots of write intensive stuff its likely you'll see it eventually.

    Outside of those situations, the amount of CPU and RAM Pure throws into an M70 (32 Core 2.6 Ghz Xeon E5-2600) makes things like the VMAX100K (6 core 2.1 GHz Intel Xeon E5-2620-v2) look pale and weak, but the VMAX will probably be a lot more predictable no matter what you throw at it.

    By the way, is it just me, but wouldnt you expect EMC to show an XtremeIO benchmark in that comparison ?

    1. Anonymous Coward
      Anonymous Coward

      Re: How to make Pure look bad (and other vendor tricks)

      To answer your last question. From what I understand Unity and Pure have the same dual controller architecture and are the most natural point of comparison. XtremIO is a scale out architecture that can scale to 16 active controllers. Pure and XtremIO were compared for a long time because Dell EMC did not have a dual controller architecture to compare apples to apples with Pure.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon