Re: FileSystem for AI
Disclaimer: IBMer here.
Interesting discussion... but I don't agree with this statement:
> ...the only distributed file system that was architected from day 1 for NVMEs and flash devices, therefore It is able to handle small files ...
The abililty to handle small writes does not solely (and should not) depend on the availability of flash devices that can do the job for you. Technologies like distributed log-structured small writes buffering give decent performance gains on "old" flash technology and HDDs alike, and the same is true for NVMe devices - which basically gain faster access and less queueing, but they are still EEPROMs at the core.
So if "architected for NVMe" - i.e. not using the system block io driver - means "we didn't care implementing media accelerators because the media is quick enough", then this is not the correct way forward IMHO, because others will be able to copy it easily once NVMe is widespread. Intelligent distributed metadata management is a better investment.
Btw. "IOs were terminated by clients unprotected RAM" ... huh? I don't think this exists in *any* of the IBM Spectrum storage products. The secret of the exceptionally low latency in Spectrum Scale is non-blocking metadata management and parallelism, and very shallow software stacking. This worked for Terabytes, works for Petabytes, and will work for Yottabytes. Page 15 ('client - server - device roundtrip)' in the CORAL presentation discloses what to expect: 0.074 ms Avg Latency. http://files.gpfsug.org/presentations/2017/SC17/SC17-UG-CORAL_V3.pdf
Thank you for the discussion!
Axel
Biting the hand that feeds IT
