Mine's bigger than yours
And you have to trust me on this.
Microsoft has spun up a custom AI supercomputer in its Azure cloud for OpenAI that ranks within the top five publicly known most powerful supers on Earth, it is claimed. The behemoth is said to contain “more than 285,000 CPU cores and 10,000 GPUs,” with up to 400Gbps of network connectivity for each GPU server. If this beast …
An NVIDIA Tesla supercomputing-specific GPU is the most LIKELY hardware for the GPU portion and is about 50 TeraFLOPS for 32 bit Single Precision Floating Point number crunching (or 100 TeraFLOPS for 16-bit Half-Precision Floating Point performance) and I suspect Microsoft went with AMD Rome 7742 CPUs for the 64-bit number crunching which maxes out at 2.3 TeraFLOPS for 64-bit Double Precision Floating Point operations or 4.6 TeraFLOPS for 32 bit Single Precision number crunching.
Since MUCH supercomputing work is done at Single Precision 32-bit number crunching, the total performance with 10,000 NVIDIA Teslas GPUs and 4,454 (i.e. 64-core CPUs) of the AMD EPYC Rome 7742 CPUs calculates out to 500,000 TeraFLOPS total for the GPU portion and 20,488 TeraFLOPS total for CPU portion or a Grand Total of 520,488 32-bits wide TeraFLOPS or about 520 PetaFLOPS at 32-bits wide. If you compare it against SUMMIT which 64-bits throughout, that means the Microsoft Supercomputer could do about 210 PetaFLOPS at 64-bit Double Precision Floating Point Number Crunching.
I should note however in MULTIPLE locations within the sprawling US military industrial complex, I know of at least FOUR of ExaFLOP+ scale 64-bit Double Precision supercomputers now operating!
Sooooooo, that’s actually not too bad for a civilian supercomputer!
P.S. Ours is the MOST POWERFUL SUPERCOMPUTER OF ALL !!! 119 ExaFLOPS at 128-bits wide Floating Point Operations SUSTAINED on 60 GHz GaAs substrates using combined-CPU/GPU/DSP super-chips !!!
WE STILL WIN BY A FAAAAAAAAAAAAAR MARGIN !!!!!!
Biting the hand that feeds IT © 1998–2020