Need to move 1.2 exabytes across the world every day? Just Effingo Google has revealed technical details of its in-house data transfer tool, called Effingo, and bragged that it uses the project to move an average of 1.2 exabytes every day. As explained in a paper [PDF] and video to be presented on Thursday at the SIGCOMM 2024 conference in Sydney, bandwidth constraints and the stubbornly …
Who says computers can't be fun.
Some decades ago (college course) we were introduced to various project/programming/data analysis methodologies e.g top down, bottom up, JSP (Jackson Structured Programming) etc etc. Our lecturer was real old school so he also told us about the one method we really needed to learn and would always find a use for:- JFDI. Still use it today.
I will repeatedly agree with this.
The only good UI to ever come out of Google is a plain page with a text box and a search button and even that was essentially a cut-down evolution of AltaVista at the time. Everything else is an absolute pie and often, bafflingly, goes deliberately against long-established UI intuition.
I used to manage a pair of Netapp clusters at opposite sides of the country, and trying to keep them in sync so that they were DR sites for each other was... tough. We had a T3 at the time, which was huge for our small company. But 85+ ms of latency just killed Netapp SnapMirror copies, you could see the spikes in the classic sawtooth shape as the TCP Bandwidth Delay Product kept hitting us.
We tried using WAN accelerators like Riverbed, SilverPeak and others. We tried using 'bbcp' (great tool btw!) for single large files to fill the pipe. It all just sucked big time. It's just hard to move that much data when you had (at the time) 1tb of change in the data per-day. It just didn't work. It must be fun having google's resources to play with and to test things and just to learn how to do stuff better.
I really like how the data moving is under 10% of the code, it's always exception handling that bites you.
"LTOs are about 10T, so you'd need 120,000 of them."
I didn't say you'd only need one 'plane for that much data ... However, the small pile of them[0] on my desk weigh 200g each. That's 24 ton ... Doable with a couple large business jets with the comfy chairs, movie theater, kitchen, sauna, bed(s) and bar stripped out.
IBM Ultrium 7, p/n 38L7302, weight according to my OXO kitchen scale.
Way back in time at a company that no longer exists (thanks Hanson Trust) we had a vehicle management system. It ran on three sites and was kept in sync by posting tapes around.
The initial setup consisted of me driving about 500 miles to the three sites in one day to load the data.
microSD cards would be the way to go, rather than tapes.
Conveniently, Randall Munroe has done the sums in a What If?
This post has been deleted by its author
It's like car model names. Whatever name you choose, there's a country somewhere where people will titter.
I vaguely recall Camry sounded similar to Thai slang term which initially caused a flutter of titillation.
Prius is perhaps close enough to Priapus to excite a quiver along a prurient stiff upper lip.
Effingo is a compound I think of (e, ex) and fingo (fingo,fingere,finxi,fictum) the principle parts of which appear to lend themselves to any amount of innuendo without even considering the entire conjugation.
Was "twitter ye not" one of Lurcio's (Frankie Howerd) lines from Up Pompeii?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
