Running applications in the storage array

This topic was created by Chris Mellor 1 .

[Chris Mellor 1]

Running applications in the storage array

Violin Memory is going to introduce a Violin Memory array that can run application software, like SAP's HANA database. In fact you can already get HANA to run on Violin via a special request to SAP apparently.

EMC is going to run apps on X86 servers inside its VMAX and Isilon arrays.

DataDirect Networks says it already runs applications, like filesystems, inside its arrays.

The idea, it seems, is to combine storage array capacity with servers in pine place and so get rid of network latency in delivering data to servers. Violin is going one step further and getting rid of disk I/O latency as well by using flash memory.

It says this is a massive, a secular, change in the storage industry, and something that no modern mainstream storage vendor has had to face before. It will be a huge challenge for them.

Is Violin right? Is SAN-size storage running at flash speed and with servers inside the array a secular change for the storage industry, or is it just another linear development?

[Erik (TMS)]

Re: Running applications in the storage array

I believe it is relatively trivial for storage appliances that already include a heavy x86-based software layer to run more software on it, up to and including actual business applications. I don't believe that actually gives significant advantages, because network latency is usually trivial compared to the latency in the aforementioned software layers or in the array hardware designs, except in truly extreme performance cases: think some high frequency traders that have skipped Flash and store everything in DRAM, or some HPC environments.

In my opinion, converging compute and storage resources is a cyclical change, not a secular change in the industry. I don't think there is a right answer or a consensus among enterprises as to whether the converged approach is better. The pendulum has swung in both directions if you look at the history of computing.

Most storage manufacturers would LOVE to have servers integrated into their storage. Most server manufacturers would LOVE to have storage integrated into their servers. Manufacturers want to own the whole stack, and complete integration can offer some benefits. The magnitude of those benefits varies widely. It is nearly always possible to build non-vendor-integrated systems, for the same price or less, that work just as well or better than integrated stacks.

So let's look at the angle from IT shops' perspectives. There are basically two camps here. One says that vendor-integrated solutions are great because they reduce complexity. The other says that vendor-integrated solutions are lousy due to lock-in and interoperability concerns. There is no right answer for 100% of workloads.

(Disclaimer: I work for TMS.)

[@jgilmart]

This is definitely an interesting question, but it's hard to predict how this evolves. As the previous poster points out, there is an historical pendulum in system designs that swings between integrated and modular. For some applications, such as Hadoop, this could be the answer, but there are many applications that don't have the same requirements profile for performance, capacity, and cost. A big challenge in all this will be in orchestrating applications and infrastructure to ensure that compute and data are at the right place at the right time in the right ratios. Applications change and grow constantly, and we need systems that can dynamically adapt.