Re: Scale up vs. scale out
NVMe is a protocol, your imagined 4 lane limit is a hardware package implementation. The two are not related.
You can't scale up storage in blades - there's nowhere to add storage without significant costs. If you add a disk blade you lose a slot in the chassis, but still have to pay for your interconnects.
Local storage is a management and availability problem, both of which are solved with centralisation.
SAN is not a bottleneck when designed properly for the requirement. It also brings other benefits such as DR, integrated backup, copy offload, instant cloning, shared disk for clustering.
How are you using 32 PCIe lanes for local storage in a blade? I've never seen one that even surfaces that many lanes, let alone in a way that allows connection to local storage.
Cluster mode isn't slow, and isn't limited to "a few gigs of bandwidth". The large controllers scale to enormous bandwidth as well as allowing scale out to meet the needs of any high density modern data centre. Your inability to design said solution is not a failing of the platform. In actual fact, Cluster Mode drastically reduces SAN to server bandwidth through the use of copy offload and integrated backups which have allowed many properly designed data centres to reduce overall network capacity while increasing density.
You clearly have a single use-case in mind which you probably know quite well. If I had to guess based on the above I'd probably say desktop workloads such as VDI. Don't mistake this for understanding the wider subject. When you look outside that use-case there are many different solutions to the problem of storage, each with their own benefits and problems.
Biting the hand that feeds IT
