Re: Is Object storage really appropriate for 100+ PB stores?
I think the fact that Amazon S3, an object store, has scaled to over a trillion objects (http://aws.typepad.com/aws/2012/06/amazon-s3-the-first-trillion-objects.html) is evidence that the technology can and has scaled to those sizes. Other service providers, such as Rackspace and HP, offer object storage as "cloud storage" and enables them to sell space on their cluster and scale horizontally to meet demand from not only many objects but many users as well.
Also, the claim that the LHC data is being recorded to tape may be true, but that doesn't mean CERN has given up on object storage all together. In fact, the CERN folks are taking a hard look at object storage and other cloud computing technologies because they want to process more data AND let more researchers get access to said data for processing (http://www.techrepublic.com/blog/european-technology/cern-where-the-big-bang-meets-big-data/636), and with everything stored on tape, there is an obvious bottleneck when many people want data from many different experiments.
Data generation is not going down, so while 100PB seems like overkill, it will be used. Remember, 1GB was at one time considered "too much space."
Biting the hand that feeds IT
