
A Proprietary Transport
is not going to fly.
Apeiron Data Systems' external ADS1000 array uses NVMe media to deliver block storage access using NVMe over Ethernet (NOE) but not NVMe over fabric technology, (NVMeF) which can use Ethernet. How does this subtle distinction work and what is the difference between NOE and NVMeF? NVMeF is a way of using the NVMe protocol, …
Storage Array and SAN makers keep peddling an architecture designed for disk that is simply obsolete in an era where latency is everything.
The greatest benefit of NVMe isn't throughput, it's low latency, largely by simplifying the protocol compared to ATA or SCSI. Putting NVMe devices inside an array with multiple levels of protocol conversion guarantees latency will be at least an order of magnitude slower, probably closer to two, and completely defeats the purpose of fast NVMe storage.
I don't think that I can agree with the statement ".. in an era where latency is everything". I also think that you are contradicting the title of your post "I've got this hammer, everything...", but I may have misunderstood you there!
Surely latency is not the only thing that is going to make a new technology succeed in the data centre? Simplicity and scale-out are also very important.
However, I do think it will be difficult for Apeiron to succeed. We are inundated with new storage technology, so it might be difficult for the company to convince customers that this is the "silver"/"magic" bullet.
With that said, I do recall many years ago hearing about a small disk system that used commodity drives to emulate IBM mainframe DASD. At the time I said "I find it hard to believe that that thing will fly"! A few years later I began my; 20 year career with EMC (which involved quite a bit of flying for myself). You might like to check out Apeiron's careers page :-)
I checked out their website.
The interesting thing is that they talk about a YARN plugin. (Read Hadoop)
However this kind of replaces the HDFS to a point.
Would love to see a more detailed discussion on a cluster configuration, and how it fits in.
For spark, I'd probably go MESOS however again. YMMV in terms of build out.