Future is bright for NVMe-over-Fabrics with TCP and Ethernet, say Solarflare, Lightbits

Lightbits Labs and Solarflare are promoting the idea of NVMe-over-Fabrics (NVMe-oF) using TCP-over-bog-standard Ethernet, instead of RDMA over Converged Ethernet (RoCE) via data centre-class Ethernet, iWARP, InfiniBand or Fibre Channel. Solarflare claims NVMe-over-TCP using Ethernet provides IT departments with the same high …

[baspax]

Right. I am totally going to install a KERNEL EXTENSION from some small outfit. Into all my tier 0 and tier 1 app servers, change management will wave this trough no problemo.

LOL

And then, instead of a standards based architecture (RoCE) which is actively supported and maintened by all my vendors Oracle, Red Hat, Msft, Vmware, Dell, Pure, and Cisco I would have to rely on these guys to maintain code and drivers in a stable and timely fashion for all mentioned above. Yeah right.

[muliby]

NVMe/TCP is on the fast track for standardization

NVMe/TCP is on the fast track to become just as standard as NVMe/RoCE -- we (Lightbits Labs) are working hard to standardize it as part of NVMe.org in collaboration with Facebook, Intel and others. There will also be upstream Linux drivers as soon as the standard will be ratified. Pre-standard versions already exist. In fact, if you, like me, believe that the best thing about standards is that there are so many to choose from, NVMe/TCP is going to be *more* standard than NVMe/RoCE. Whereas RoCE is a niche technology in the data center, TCP is everywhere. NVMe/TCP will be everywhere as well.

[baspax]

Re: NVMe/TCP is on the fast track for standardization

Fair enough. The issue is with who carries the burden of support and code maintenance. Is it *you* writing and maintaining drivers for Linux, Windows, Solaris, AIX, Vmware, KVM, etc. (in all their glorious versions and patch levels) permutated with all possible array models and versions, or is it those vendors writing and maintaining their code to adhere to this standard. The difference is monumental.

And yes, it’s true, TCP is everywhere. But NVMe’s biggest advantage is getting rid of the SAS stack. Why would I introduce the even crappier TCP stack unless it’s just for tier 2 and 3 but why use NVMe at all then?

[muliby]

Re: NVMe/TCP is on the fast track for standardization

Just like with iSCSI, we expect every operating system and hypervisor to include NVMe/TCP client drivers. For Linux, NVMe.org will submit drivers upstream as soon as the standard is ratified. For other operating systems and hypervisors that are not Linux-based, either the OS vendor (for proprietary OS's) or the community (open source OS's) will contribute drivers. If someone needs help creating drivers that follow the spec, Lightbits will be able to assist.

As to "replacing SAS with TCP", the Linux TCP/IP stack is pretty awesome. I think that you will be surprised by what NVMe/TCP can do. You can find more information on our website, http://www.lightbitslabs.com.

[flashdude]

Pavilion Data Systems also supports NVMe Over TCP

I just wanted to add that Pavilion Data Systems supports NVMe Over TCP as well. You can actually leverage both protocols simultaneously from the same array if required for different use cases. Some customers will use RoCE within a rack, but TCP to access the storage array from other legacy servers that don't have a RoCE ethernet NIC. The NVMe-Over-TCP host protocol driver that can work with any Ethernet NIC will be standard inbox in future versions of Linux, just like the NVMe-Over-Fabrics for ROCE driver is today.