Snowmobile, Amazon's truck-powered migration service, reaches the end of the road Amazon Web Services is abandoning its fleet of Snowmobile data haulers, the trucks packed with petabytes of spinning disks designed to get large enterprises into the cloud. Introduced in 2016, Snowmobile offered a way to transfer very large amounts of information into the public cloud, recognition that it’s hard to shift …
I used a Snowball device about three years ago, and at that time it was 10's of TB in a suitcase (it was much larger than a briefcase IMHO, and weighed about 22KG - I know, they chose to deliver it on a day the lifts to the 1st floor of the data centre were down!)
Since then, I've seen the IBM Flashcore Storage Modules, each the with 38TB of capacity in a 2.5" form factor which IBM say will do 3:1 compression in the module. It does not take too many of these to scale up to petabytes of storage, even in a quite small foorptint. Amazon say that the current Snowwball devices will hold up to 1PB.
The real problem is actually loading and unloading the data. The situation we had was a relatively small amount of extracted data (around 10TB), and the time issues were extracting and encrypting the data (Snowballs are encrypted anyway, but the end client insisted on it being encrypted before even loading it on to the device), and transferring it. This time makes it very difficult to move a service into AWS without having some relatively sophisticated staging mechanism involving a bulk upload and then deltas of the data, or taking a several day outage.
"which IBM say will do 3:1 compression in the module"
Sorry, but claims about compression are BS, because you can do the exact same compression when writing to any device.
Also, if you are dealing with PB of data, it is likely to be the sort of data (images, videos, audio files) that you can't really compress, or has already been stored in a compressed format.
"That's a shame, I'm in the middle of studying for my AWS Cloud Foundations cert and just learned about Snowmobile this morning. How else are you supposed to "FedEx a shoebox full of hard drives" on petabyte scale now? :'("
Snowball still exists. It's just the snowmobile service Amazon are dropping. "Migrate petabyte-scale data to AWS with Snowball" according to https://aws.amazon.com/snowball/
When we moved from locally hosted Exchange to Office 365, our IT manager copied everyone's mailbox to a couple of 10tb hard drives and I walked them down the road to our network support office and they uploaded everything to Microsoft's servers for us. I think they used to call that kind of thing Sneaker Net...
When you think about it, the DNA - the information required to recreate a whole human - is 3x10^9 base pairs*. 4 bases so 2bits for each one. So less than a GB per person. Multiply that up by the population of the world and ...
WTF is all this data? Maybe this is true https://xkcd.com/908/
*Yes, mitochondria, I know
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
