Don't want unified file, block and object storage

This topic was created by Chris Mellor 1 .

  1. Chris Mellor 1

    Don't want unified file, block and object storage

    HDS has announced HUS which combines block, file and object storage. Well, almost, but is it a good thing? First HUS uses a block storage controller allied to NAS heads to provide file storage layered on block storage. Both files and blocks are basically blocks of data on disk in a storage array.

    I'm going to argue though, that, although objects are basically blocks of data on disk, standard block and file drive arrays are not good for storing objects, IF the object storage vendors are right in their marketing.

    They say object storage is needed because filesystems are slow and inefficient when locating one file in a population of billions of files - literally big data. Because object storage technologies use scale-out designs, typically, and have an object's address in a flat storage address space computed in some way from the data content , an object data block is located amongst a set of object storage nodes in a particular way that means it's not best stored in a standard disk array via a block controller with an object storage head.

    On that basis we should not mix - unify - object storage with file and block storage.

    An irony is that many existing object storage customers are not storing billions of objects at all and a file-folder system would be adequate for them. But the purveyors of object storage have to start small and grow big, and sell other benefits such as self-healing via data integrity checks.

    If some object storage applications are never going to get to the billions of objects stored level then, for them, having an object storage head layered onto a filer or a block storage array would be fine, and unified file/block/object storage a way to have one physical silo (with three virtual silos) with less percentage wasted space, one management interface and tool set, and simpler data protection.

    In anther object storage case, the one where a file interface has been layered on top, then it may be assumed that having a unified storage array underneath the objects and files, is a good idea too.

    So, for real big data object storage I wouldn't think unified storage is a good thing at all. For "little data" object storage unified storage is probably a good thing. Where the dividing line is between big and little object data is debatable and I couldn't guess where it is, other than thinking a billion objects and up is probably in the real object storage area, whereas a million or less is probably in the unified storage area.

  2. StorageBuddhist

    IBM's GPFS doesn't seem to have a problem with billions of files

    IBM's GPFS seems to be fast at processing billions of files, for example...

    http://www.almaden.ibm.com/storagesystems/resources/GPFS-Violin-white-paper.pdf and

    http://afs2010.civ.zcu.cz/desc.php?name=ibm

    Which leaves me thinking that perhaps Object storage still hasn't found its real purpose in life.

    1. Chris Mellor 1

      Re: IBM's GPFS doesn't seem to have a problem with billions of files

      "Perhaps Object storage still hasn't found its real purpose in life." Exactly; that;s what I think too. There has too be more to object storage than storing Xray images in hospitals and that kind of stuff. HDS says it's selling lots of HCP object storage arrays in Europe so, perhaps, it's use for large unstructured data vaults will grow but, in that case, I still think a separate object silo is needed.

  3. TomLeyden

    Correct, GPFS has good performance (throughput) and was designed to scale out. But object storage has many more benefits: REST (no file system involved) has become very popular with developers in a short term. But it's not only about the interface; the better object storage platforms will provide much better storage efficiency than systems that rely on data copies, etc. I haven't seen an IBM TCO for a 50PB system that comes close to the cost of tape. Systems like Amplidata's Amplistor will provide ten 9's with less than 60% overhead. And S3, one trillion objects can't be wrong? Right?

  4. TomLeyden

    1 trillion objects in S3, guys

    We will indeed have to motivate application providers to use object interfaces and get rid of legacy protocols. File systems will never really disappear, but applications will hopefully evolve and use more direct interfaces.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021