Nexenta seems cool - but seems a little lacking
We have a very small vmware cluster in one location that isn't production or all that important in any sense. Its running on a decade old FC array that takes up more room than a car, and we only need 5TB of storage.
Wanted to buy a filer, but mgmt said no. We bought a commodity server and figured we would install freenas, nextena, rockstor - check them all out and see what worked best. For VMware I a big fan of NFS, and not a fan at all of iscsi. We will use it, but only if we don't have much of a choice. We also wanted CIFS so we could get off the virtualized Windows File Server.
Rockstor - Has tons of promise. I really like brtfs, but doesn't do LACP, so its out unless everything else sucks more.
Freenas - NFS performance is terrible for VMware - apparently this is a common problem. ISCSI is excellent, but I don't want to use iscsi. Spent several hours trying to get it to join our AD, and couldn't do it. Also apparently a common issue. At this point I said screw it, on to nexenta.
Nexenta - Took a while to sort out the inanities of using LACP out of the box. The initial wizard forces you to use a standard access connected link, which I didn't want to do. After swearing at it and figuring out how to get into the shell instead and create an LACP group I was able to get it working. Took a lot of rebooting after creating LACP connections to work - which is not a good thing in my book - but it worked.
AD join was a piece of cake, and CIFS performance is nothing short of incredible. I was very much looking forward to...
NFS - great as long as you run it over a primary interface. Tried setting this up to use a private non-routeable subnet between the VMware hosts and the nexenta box to fence off traffic, and no matter what I do - its not useable. I can mount the NFS shares over the private vlan, vmware seems happy with them, but any writes do not occur. Run the same mount over the primary nexenta interface and it runs great.
I started a discussion on the nexenta forums, and the responses were pretty poor. Everyone kept asking me about the ZFS config and all sorts of other questions that are not the problem.
Haven't tried iscsi yet.
Nexenta seems promising, but it also seems to have its quirks. I am sure if I had support and training it could be very usable, but I am not 100% sold on this being a one for one replacement for a netapp filer.