'Limitless enterprise storage'. Really? Digging deeper into Symbolic IO

Remember the company we wrote about yesterday? The startup with the startling technology which claimed its storage and compute technology could run database queries 60 times or more faster than other systems, and offers ”limitless enterprise storage". Well here's a Q&A that sheds a little more light on the tech. The main …

[SirWired 1]

This smells very strongly of B.S.

"Data can be ingested from a local memory channel or any wire such as USB, Direct Wire, TCP/IP Packets, Fibre Channel, InfiniBand, iSCSI, etc."

Huh? I was not aware that "TCP/IP Packets" were a way to ingest anything. Somebody needs to go back and fill in the blanks on his OSI chart. And what's "Direct Wire"?

" The conversion process consists of primary data (D’ [prime]) that is dismantled into substrate components called “fractals” and processed into SbM (Symbolic Bit Markers). Unlike other technologies an advanced algorithm allows for substrate fluctuation."

This reeks of something a Star Trek scriptwriter would toss together. It appears to be word-salad that hints at data compression.

"One of the most compelling elements of Bit Marker technology is that it is lossless and does not require any additional overhead, unlike traditional compression schemes. The output of the conversion process is to store, transmit or both depending on use case. This entire conversion process does not require any delayed post-processing and happens in real-time for the end-user.

Symbolic IO refers to the ingestion conversion process as the “constitution” of data; whereas the data being stored or transported has been converted into this proprietary and unique language format and will remain in that state until a recall or read-back/read-request is received from the system. "

Errr... explain to me again, and please use smaller words, how this is different from Real-Time Compression?

Ahh... here it is (well, not really): "Symbolic IO’s patent is based on being a non-compressive algorithm. Compression is a non-deterministic algorithm that requires many CPU cycles and no guaranteed results. By reformatting binary we see consistent results that de-duplication and compression cannot achieve. There is nothing stopping either us or a customer from applying additional compression techniques to further reduce data, if that is the sole primary focus of the end-user, and they were willing to accept the performance penalties associated with standard compression."

BZZZZTTTT!!!! Wrong answer. If you claim you can reduce data size (with "guaranteed" results), for starters, that is, in fact, "compression" in any sense of the word. When you claim you can then further compress the data afterwards? That's the classic compression scam, with a lot of fancy words around it. There IS NO SUCH THING as "guaranteed" compression (this is not a difficult concept to understand or prove); you cannot ever compress random data. Claiming you can further compress it significantly afterwards using another algorithm is just icing on the B.S. cake (all but the lousiest compression algorithms produce data that cannot significantly be further compressed); again, this is classic compression-scam material.

[jzl]

Ouch

The meaningless buzzwords nearly took my head off at that speed. Good allotment fodder.

[myxiplx2]

Full system de-dupe?

There's a lot of bullshit in the patent, but isn't that always the case? Waffle about a whole bunch of magical things that it might be able to do?

It reads to me as though it's de-dupe, but applied all the way through the system, from CPU cache, RAM, nvram, all the way down. And it seems they're implying that this can give enormous improvements in data storage and throughput, but with the caveat that this will depend on the particular data in question (which is par for the course with de-dupe).

In memory de-dupe would of course be interesting to suppliers of expensive nvram hardware, so that could explain the links there.

[Michael Wojcik]

Re: Full system de-dupe?

The patent is just a broad description of lossless compression by various symbol-substitution techniques, with variable-length input symbols and codewords. It's undergraduate-level CS bullshit. If that's they're key patent, then they have nothing (aside from a patent that can easily cover pretty much any lossless compression algorithm, so they can go ahead and start suing people, until it gets overturned).

The Q&A is even worse. The claims about compression ("a non-deterministic process") are simply wrong, and the implication that they can compress any data is an obvious violation of the Pigeonhole Principle.

I'm not sure how much of this is confusion and how much mendacity, but this article does even more than the previous one to cast grave doubt on Symbolic IO technology.

(And de-duplication is a form of compression.)

[Hurn]

RTC and Dedupe and Encryption, Oh My!

" By reformatting binary we see consistent results that de-duplication and compression cannot achieve. "

That's because, for this sentence, he intentionally forgot to add encryption. The description of the device clearly employs all three technologies.

Any metrics of the claimed consistent results would no doubt be distilled snake oil, probably based on processing easily compressed text files.

Am guessing with highly compressed video files, the main "storage saving" mechanism is the fact that many people (cable subscribers) would be watching the same show. With spinning rust, only so many processes can access the exact same data at the same time. NV-DIMM would allow greater parallel access.

Might "Symbolic Bit Marker" be a fancy term for inode?

[Anonymous Coward]

"Brian Ignomirello’s entire theory when founding the company was based on changing the format of data and the concept that all digital data could be co-generated by implementing an advanced algorithmic compute method to materialise and dematerialise data in real-time utilising a proprietary “conversion engine”.

You have to admire their bravado for talking bullshit like this, but sadly some investors will be conned and burned, and this is not good for the technology sector overall.

Compression algorithms are not new - but neither is Shannon's theorem. Dependent on the type of data you present, there's a fundamental limit beyond which you can't compress; otherwise you'd just keep compressing it until you got to a single bit (0 or 1).

File under "perpetual motion machines".

[Michael Wojcik]

You needn't even involve Shannon and information entropy. They're claiming they can compress anything, and that obviously falls foul of the Pigeonhole Principle. Their claims aren't sensible enough to warrant further analysis.

Frankly, it's disappointing commentators have to point this out in a case like this. The descriptions of the technology in these two articles are so clearly bogus that they should be obvious to competent tech journalists.

Yes, there's some smoke and mirrors (ooh, we can use linear combinations of bit vectors!), but the outline ought to be apparent to anyone with even a basic understanding of computer science, a modicum of capacity for critical thinking, and the ability to read through a patent without losing consciousness.

[Mage]

Odd

A lot of marketing speak there.

1) Simple deterministic compression will compress certain kinds of uncompressed data. It won't compress anything already compressed, especially JPEG, MPEGx, MP3 etc.

2) Conventional disks do waste space for smaller files, if they use a fixed block size (which is commonly the case). Perhaps it's got a clever file system than FAT, which wouldn't be hard!

No doubt they are doing "something" but not likely saving anything on large files of already compressed file formats.

[SteIMG]

I can't believe The Register's famously cynical hacks are just reposting a bunch of marketing crap.

Why don't you tell us what it's _really_ doing? Where the cynicism? You can't just expect us to take marketing crap as news.

Storing data not-exactly-how-you-gave-it is what hard disks have been doing for years...

[Anonymous Coward]

Middle-out compression?

I'm surprised no one has mentioned the fictitious "Middle-out Compression" at the heart of the storyline in the excellent Silicon Valley sitcom.

If you've not seen the episode where they invent it, check it out. Hugely funny.

https://youtu.be/P-hUV9yhqgY

I'll cite that as prior art :-)

[ChiefDS]

Do your homework.

If you had bothered to chat with some of their actual real world production users, you might realize it is not BS or magic or from the middle kingdom. Those users have been blown away by the results. Up to 3000% better performance for SQL Server. Up to 84 HD streams running from a single IRIS concurrently. Amazing stuff. Do some research on Chaos science fractal math. It is brain warping math. Hard to get one's head around it and yet that is what they are using. No, it is not compression. No, it is not dedupe. Just because it is new does not make it BS.

[SirWired 1]

Re: Do your homework.

Nobody's expressing rank incredulity at the idea of a performance improvement; it's the hard-to-believe data reduction claims that are "not compression or deduplication, and are guaranteed" that are either impossible, or suffering from a bad game of Telephone from a clueless marketing hack.

If it isn't compression or dedupe, then what are we to make of the "data reduction" claims? You know, the ones where they both claim they can get "deterministic" results AND run the output through a compression scheme afterwards to get further reduction? (Given that they allege that the output is encrpyted, color me a deep shade of skeptical at the idea it can still be compressed afterwards; unless their "encryption" is ROT-13, you cannot effectively compression encrypted data. Certainly the patent reads like a generic description of compression/dedupe to me.

If anybody's going to be convinced, it's going to take a lot more than

Q: "How is this not compression/dedupe"?

A: "Well, yeah, it sounds like compression/dedupe but isn't."

Q: "But what is it then?"

A: "[Unexplained phrases that look a lot like technobabble] It's so amazing, you just wouldn't understand."

Compression scams that look a lot like this one are literally decades old in computing, and have likewise featured credulous testimonials by investors/beta testers that had been hoodwinked.