Benefits and Tradeoffs to tightly coupled scale-out vs Distributed archs
Without a doubt there are benefits and tradeoffs between these types of architectures. Lets examine a few from some who is NOT a vendor although i am biased.
1) I don't know about you, but i'd prefer my failure domain to absolutely be "non-linear" as is the case for Netapp's cluste ontap (tightly coupled) vs Isilon's (distributed). In former case a dramatic failure will be isolated to the specific domain, in the later case it will affect the entire cluster. Conceptually, this is similar to creating FC zones. Did you dump everything into a single zone or did you isolate and logically partitioned the fabric? Why? Because we wanted smaller logical failure domains.
2) Performance - Aggregating performance across multiple nodes with file stripping and load balancing is certainly important, however, I would claim that this was an essential component 10-12 years ago, with older nodes with limited CPU power, limited amounts of memory and slower buses. However that's not the case any longer as it is evident from the VNX 2 benchmark. With the advent of multi-core CPUs, memories into the hundreds of GBs per node, multi PCIe buses and Flash, a single node will rarely be the bottleneck for a LUN or a file.
3) Is stripping the answer to good performance? - Well, VMware taught us that it's not and certainly i don't see anyone complaining that a single VM is not stripped across multiple vsphere nodes. What VMware has taught us is that intelligence, data mobility, fine grained control, QoS and workload isolation provide more benefits and good performance than just cross-node stripping. How different is clustered ONTAP from this concept? I'd say it's not.
4) HW Upgrade path - What does it take to do a HW upgrade on a distributed scale-out architecture? Well, cash. You need to upgrade all the nodes that comprise it, at one shot! Do tightly coupled architectures have this requirement? They don't.
5) SW Upgrade path - What does it take to a SW upgrade? See #4
6) Node failure - Can a node failure impact performance on the entire distributed architecture? It sure can. How about a SW failure? It can as well.
7) Isolation - Do distributed architectures provide complete workload isolation (SW and HW)? Unfortunately they can't.
Does this mean distributed architectures such as Isilon are not good? Absolutely not. What it means, is that they need to be positioned for the environments for which they were initially developed.
VNX-VMAX - The Symm came into the picture in the early '90s, a wonderful, rock solid array with thousands of satisfied customers, well ahead of its time, and the company's cash cow for years. If I were EMC would I consider replacing the current version with a VNX? Would you do it? I don't think so. Instead, I'd milk that cow until she was bone dry. Customers are happy it, EMC's happy, and with the company being publicly traded, and a for profit organization, so are its shareholders.
VNX 2 - Active-Active - I think this option is great, and i'd have liked to see more vendors go down that route (do you hear Netapp?) but what does this mean if I'm a VNX customer? Does it mean I have to bypass my pools and use Flare LUNs only? And what about the space efficiencies tied to pools? Do I lose them?
EMC can't on one hand beat the drum of scale-out with Isilon, XTremeIO, VMAX and Scale-IO and at the same time extoll the benefits of the VNX2 scale-up architecture. Why not have them both? Why not provide customers with the option of a dual controller system that can scale out? You can still sell them 2 controllers if that's what they need.
Does anyone have any reservations, that if EMC were to truly re-write the VNX codeline and not just update 10%, the VNX wouldn't make the transition to a scale-out? I don't.
As far as ViPR is concerned, what active need does it serve? Well, If you're an EMC customer there's a good chance you will need ViPR. EMC can positioning it any way they want to, but at the end of the day, the primary need it serves is data services enablement and management across several different platforms. If i'm an EMC customer, in this position, I will, without a doubt, consider ViPR to centralize data services and management functions and that's good thing.