Posts by Barbara Murphy

WekaIO almost, but not quite, summits Summit supercomputer on storage performance

Re: IBM Summit - 2.4x more storage nodes, 3.2x more NVMe Drives, 12.5x more memory, 5% faster

Hi,

Re 1) Flash based storage will help. My understanding though from reading the IBM document is that flash is used for small file as well. Given that IBM scored very high on IOPS, (MDtest) I have go to believe that NVMe and the 1TB of memory was instrumental in this result.

re 2) Both IBM Spectrum Scale and WekaIO have a POSIX client that runs on the compute nodes which then communicates with the storage nodes. This is standard across all the parallel file systems including Lustre. It is not using something like NFS, which would never be able to deliver the kind of performance numbers you see on the IO-500.

re 3) No question that the Summit storage system is possibly one of the biggest in the world. The benchmark is all about performance per client, which is a different metric. Our software does not preclude putting disk behind the flash tier to get the great scalability. We leverage object storage and have customers with WDC ActiveScale behind us getting great performance and scalability.

Regarding the motor bike vs, truck analogy, it sounds good but that is not what is happening in the case of the IO-500 test. If you look at the results, you can see that WekaIO did over 2x the bandwidth of Spectrum scale (27GIB/sec vs. 9.8GIB/Sec)

IBM Summit - 2.4x more storage nodes, 3.2x more NVMe Drives, 12.5x more memory, 5% faster

I agree with Anonymous coward #1, we need to have more transparency in the systems used so you can get a side by side cost estimate. Anonymous coward #2, you are partially correct, each node has both HDD and NVMe drives. According to IBM's literature the Summit Storage System Configuration is utilizing 2 NVMe drives in every enclosure in the storage configuration. It was a bit of a mystery hunt to find the system specification but here is a link to the IBM sponsored paper that outlines its build. I have taken that data and put is side by side with WekaIO and I encourage you to check it out because the paper talks extensively about how they are using NVMe to improve small file I/O performance. Page 6 outlines the storage node build. They have 77 storage nodes total compared to WekaIO's 32 node system.

https://public.dhe.ibm.com/common/ssi/ecm/75/en/75017375usen/sci-ibm1804-spectrum-scale-perf-v020_75017375USEN.pdf

Summary:

IBM Summit number of nodes 77, WekaIO 32 (2.4x more nodes used by IBM)

IBM Summit memory per node 1,000GB, WekaIO 192 (5.2x more memory per node use by IBM)

****** Total IBM cluster memory used 77TB, WekaIO 6.14TB (12.5x more memory overall used by IBM)

IBM Summit NVMe drives per node 8 (2 per enclosure x 4 enclosures per node), WekaIO 6 (1.3x more NVMe used by IBM)

****** Total IBM cluster NVMe drives used 616, WekaIO 192 (3.2x more NVMe used by IBM)