back to article Solidfire is 'for people who **CKING HATE storage' says NetApp Founder Dave Hitz

NetApp founder Dave Hitz has apologised to a VMworld 2016 audience for being slow to produce proper flash arrays. In a session on the future of storage he shared with SolidFire founder Dave Wright, Hitz said that work on Clustered ONTAP had consumed more resources and time than the company anticipated, leaving it short of …

  1. CheesyTheClown

    Scale up vs. scale out

    Scale out exists not because you want to have more storage. It's because storage array controllers and SANs are too slow to meet the needs of high density servers. Storage has become some a major bottleneck that it's no longer possible to populate modern blades and actually expect to have even mediocre performance of VMs because it's like running a spinning disk on a laptop. It's just horrible.

    Local storage scaled out is far better. So internal tiered storage works pretty good. You get capacity and performance in a single package. It doesn't scale up as well as a storage array... unless you buy more blades. Instead, it's pretty damn good for trying to make sure that your brand new 96 core blade isn't sitting at 25% CPU usage because all machines are waiting on external storage to catch up.

    Scale-out in a SAN environment is just plain stupid. Even with the fancy attempts by some companies to centralize NVMe which is performed using PCIe bridge ASICs, the problem is that you'd need to have dedicated storage centralized for each blade to make use of that bandwidth. Additionally, NVMe is quite slow. NVMe generally only uses 4 PCIe lanes. Using local storage, I can use 32 PCIe lanes which is a small but noticeable improvement.

    Scale up is still quite useful. Slow and steady wins the race. Cabinets that specialize in storing a few petabytes are always welcome. You really wouldn't want to use them for anything you might need to read, but an array that can provide data security would be nice. So, maybe Netapp should be focusing on scaling up instead of out. Cluster mode was kinda of a bust, it's just too slow. 8 controllers with a few gigs of bandwidth each don't even scratch the surface of what's needed in a modern DC.

    1. Anonymous Coward
      Anonymous Coward

      Re: Scale up vs. scale out

      NVMe is a protocol, your imagined 4 lane limit is a hardware package implementation. The two are not related.

      You can't scale up storage in blades - there's nowhere to add storage without significant costs. If you add a disk blade you lose a slot in the chassis, but still have to pay for your interconnects.

      Local storage is a management and availability problem, both of which are solved with centralisation.

      SAN is not a bottleneck when designed properly for the requirement. It also brings other benefits such as DR, integrated backup, copy offload, instant cloning, shared disk for clustering.

      How are you using 32 PCIe lanes for local storage in a blade? I've never seen one that even surfaces that many lanes, let alone in a way that allows connection to local storage.

      Cluster mode isn't slow, and isn't limited to "a few gigs of bandwidth". The large controllers scale to enormous bandwidth as well as allowing scale out to meet the needs of any high density modern data centre. Your inability to design said solution is not a failing of the platform. In actual fact, Cluster Mode drastically reduces SAN to server bandwidth through the use of copy offload and integrated backups which have allowed many properly designed data centres to reduce overall network capacity while increasing density.

      You clearly have a single use-case in mind which you probably know quite well. If I had to guess based on the above I'd probably say desktop workloads such as VDI. Don't mistake this for understanding the wider subject. When you look outside that use-case there are many different solutions to the problem of storage, each with their own benefits and problems.

      1. CheesyTheClown

        Re: Scale up vs. scale out

        I'all grant you have many good points. I work with quite a few different workloads. Agreed that NVMe is simply a method of remote procedure calling over the PCIe bus as well as a great method of command queuing to solid state controllers. It is designed to be optimal for single device access and has severe limitations in the queue structure itself for use in a RAID like environment. In fact, like SCSI, it has mutated from a single device interconnection protocol to something which it really sucks at. If creating virtualized devices in ASIC, there are extreme issues regarding upgradability. If implemented in FPGA, there are major issues with performance as even extremely large scale FPGAs have finite bandwidth resources. In addition, even using the latest processes for die fabrication, power consumption and heat issues are considerable. A hybrid design combining a high performance/low power crossbar along with FPGA for implementing localized storage logic could be an option, though even with the best PCIe ASICs currently available, there will be severe bandwidth bottlenecks as expandability is considered. PCIe simply does not scale well in these conditions. Ask HPC veterans why technologies like Infiniband still do well in high density environments for RDMA when PCIe interconnects have been around for years. SGI and Cray have consistently been strong performers by licensing technologies like QPI and custom designing better interconnects because PCIe simply isn't good enough for scalability.

        So NVMe is great for some things. For centralized storage... nope.

        As for storage clustering, I'm not aware of any vendors that cluster past 8 controllers currently. That's a major problem. Let's assume that somehow a vendor has implemented all their storage and communication logic in FPGA or dreadfully within ASIC. They could in theory build a semi-efficient crossbar fabric to support a few dozen hosts with decent performance. It is more likely, they have implemented their ... shall we say business logic in software which means that even if they had the biggest baddest CPUs from Intel, overall their bandwidth on scale will be dismal. There are only so many bits per second you can transfer over a PCIe connection and there are only so many PCIe lanes in a CPU/chipset. Because of this limitation, high performance centralized storage with only 8 nodes will never be a reality. Consider as well that due to fabric constraints in PCIe, there will be considerable host limits regarding scalability without inplementing something like NAT. This can be alieviated a bit by running PCIe lanes independently and performing mostly software switching and mostly eliminating the benefits of such a bus.

        Centralized storage has some benefits such as easier maintainance, but to be fair, if this is an issue, you have much bigger problems. When using a scale out file server environment configured with tiers, for DR, backup, snapshots, etc... this makes use of centralized clusters of servers. You may choose to use a SAN for this, but that just strikes me as inefficient and very hard to manage. When configuring local storage properly, there is never a single copy of data and it is accessible from all nodes at all times with background sharding that copes well with scaling and outages. If there is a SSD failure, that means the blade failed and should be offlined for maintainance. This is no different that a failed DIMM, CPU or NIC. These aren't spinning disks, we generally know when something is going to die.

        You're absolutely right about blades and PCIe lanes. Currently, so far as I know, no vendor is shipping blades like this which is why I have been forced to use rack servers. Thankfully, my current project is small and shouldn't need more than 100 per data center.

        I am actually doing a lot of VDI right now. But that's just 35% of the project. The rest is big storage with a database containing a little over 12 billion high resolution images with about 50,000 queries an hour requiring indexing of unstructured data (image recognition) with the intent of scaling to 200,000 queries an hour. I am designing the algorithms for the queries from request through processing with every single bit over every single wire in mind.

        I have worked with things as simple as server virtualization in the past on small and gigantic scale. With almost no exception, I have never achieved better ROI with centralized storage than with localized, tiered and sharded storage.

        The only thing that centralized storage ever really accomplished is simplicity. It makes it easier for people to just plug it in and play. This is of great value to many. But I see centralized NVMe being an even biggest disaster than centralized SCSI over time.

  2. Anonymous Coward
    Anonymous Coward

    Neither Solid Nor on Fire

    They can prop this pig all they want but anyone who has used it, tested it and laid hands on it knows it's one of the worst and most wasteful architectures in the all flash market.

    1. RollTide14

      Re: Neither Solid Nor on Fire

      Can you elaborate here? I'm assuming by "wasteful" you mean the usable capacity ratios right? Completely agree with you there, but I think they've done a great job with their licensing model to combat that.

      The customers that I have running it absolutely love it.

      1. Anonymous Coward
        Anonymous Coward

        Re: Neither Solid Nor on Fire

        it is wasteful in more than one ways. The platform targets Cloud Providers who care about

        power, cooling, rack units, low cost, efficient scaling.

        The existing SF architecture blatantly defeats all of them and forces unnecessary opex costs and capex investments (network).


  3. Anonymous Coward
    Anonymous Coward

    Solidfire is 'for people who **CKING HATE storage' says NetApp Founder Dave Hitz

    ... That's after they bought Solid Fire ? Very True.

  4. ptbbot

    "work on Clustered ONTAP had consumed more resources and time than the company anticipated" - Uh huh, shows how long it takes to polish a turd.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like