Re: Im surprised they are bothering with this...
In simple core terms we span from 16-80 cores per node. We have 39 "servers" in total with 8 of them being hot standby (one for each type of deployed hardware). Our oldest hardware is dual socket, Haswell era Xeons (E5-2675 v3 iirc), 2 x 16 cores. We have a couple of slightly newer gen, single socket, 20 core Xeon 6138Ps. Our latest and greatest are single socket EPYC 7713P's, 64 core, very good value and performance. We also have some prior gen 32 core EPYC's which were our first introduction into AMD servers. Our primary build farm is made up from 12 x Threadripper 3990X 64 core, self built boxes. Not server CPU's of course but they provide better performance and lower cost than the equivalent EPYC's thanks to them being commodity hardware and having higher clock speeds (though its a product line that seems to have been cancelled now). Our ARM build servers are Solidrun LX2 Honeycomb boxes, 2 x separate 16 core servers in a single 1U chassis. We are currently evaluating a Gigabyte Ampere Altra Q80-30 80 core ARM server which is a huge step up in ARM performance but Im not convinced about value yet
As you can see, its a very mixed non enterprise topology hardware wise and we are far from enterprise level budget wise! We try to reuse wherever possible. Our infrastructure is split into 5 main regions; internal admin, build farm (X86+ARM), client build services, CI testing and storage. Each of these have different performance characteristic requirements
-For example our entire internal admin systems (sales, support, finance and admin) and its 9 VM's can run comfortably on a single EPYC 7713P 64 core box. We spread them over 2 with automatic fail over but they can all run on a single box should there be a hardware failure
-Our internal build systems need as many cores as possible at as high speed as possible but dont use a lot of RAM comparatively speaking so Threadripper 3990X's fit the bill perfectly here. On the ARM side the LX2 was the only commercially available option at the time
-Client build services are a mixture of 64 and 32 core EPYC's split over 2 generations. CPU performance isnt as critical as our internal build systems (I pay my own staff, not clients!) and the RAM requirements are much higher than you can pack into 3XXX Threadripper. We basically over provision the CPU cores because most of these jobs are submitted for overnight building, ready for the next day
-CI is a hodgepodge of repurposed, old build servers from Intel and AMD of various core sizes. Performance isnt critical here and in a push we can steal some cycles from the build farm
-Storage is done on the old Haswell era Xeons
I have no issue with VMWare from a technical point of view. Its definitely the most noob friendly ecosystem and its solid and reliable in the most. Its the constant redefining of the product into multiple, chargeable SKU's and the corresponding invoices that I had to pay that pissed me off. If Im the IT director at a bank then Im playing it safe and I would go VMWare, its off the shelf with an abundance of certified bods but has a price tag to match it. I dont care about the price in that situation. As a small to medium sized dev house I need better value, much better value. I also need stability, I dont have the resources to constantly throw at the upgrade cycle