Computational storage specs hit v1.0 after 4 years of work

The Storage Networking Industry Association (SNIA) has at last published version 1.0 of its Computational Storage Architecture and Programming Model, the specs meant to help develop the new performance-boosting tech by providing interoperability between different vendors. A new direction in computing Computational storage …

[I Am Spartacus]

Very Interesting

I can imagine a rack of storage devices, not unlike the racks of SCSI drives we used to install in the 90's, with 8 racks, front and back, each with 8 disks in them, giving 128 spinning platters.

Replace these with computational storage devices, each with a high number of GB (if not TB) of storage and their own processor. Now imaging a parallel database that can send a query off to run in all of these simultaneously. Hadoop on steroids with go-faster stripes. Of DB2 (who remembers that) which parallelises easily.

Or more exotic stuff, such as facial recognition, or AI/NLP/Pattern Matching.

Am I alone in thinking that this could be the next game changer in the big iron arena?

[Anonymous Coward]

Re: Very Interesting

It really all depends on the nature of your query and the design of your data architecture.

If I want to do say, image recognition, how many storage devices do I have to search through before finding it? Can I send the query to all devices and have them all respond simultaneously?

Engineer hat in me says definitely possible.

Corporate inertia surrounding me says investing in necessary architecture and transitioning existing systems to such a platform will be nothing short of a nightmare.

Even recognising that capability does exist is awkward for people will often spec systems to reflect the current state, not to where the future state might be. Low and behold old problems carry over from version number to version number while never actually improving.

Corporate nonsense aside, I think this is a really useful capability and would be a game changer for asset condition monitoring IF properly executed.

[SloppyJesse]

Re: Very Interesting

You just described Teradata's architecture.

[Anonymous Coward]

Re: Very Interesting

See also: F1, Impala, Presto, Spark, Redshift, Snowflake, BigQuery, (parts of) Synapse, etc. etc. etc.

MPP SQL engines are ten a penny. I don't think they'll benefit from this class of architecture directly, as the market as a whole has landed on architectures near-exclusively leveraging cloud object stores as primary storage, and there's no appetite (or motivation) to revisit that.

If Amazon or Microsoft or Google plug some of this into S3/abfs/gcs then we might end up consuming it indirectly, but there's no desire to re-couple our processing engines to local storage again any time soon unless someone demonstrates massive concrete price or performance advantages *and* delivers those advantages in an elastic form factor.

[fidodogbreath]

Obligatory XKCD reference

XKCD, 15 competing standards, etc. No link needed, we all know it.

[David McIntyre]

Appreciation to Scott Shadley / Jason Molgaard

SNIA Computational Storage TWG leaders Scott Shadley and Jason Molgaard were instrumental in the development of this SNIA Computational Storage Architecture Specification 1.0, along with the SNIA Computational Storage TWG industry members. Please see the following SNIA blog to learn more.

https://sniacmsiblog.org/2022/09/reaching-a-computational-storage-milestone/

[Soreuser]

Congrats! No surprise that spec this took four years. Can't wait to see what the industry does with it.