Poor Meta. Technical debt and user training made its exabyte-scale data migration tricky Here’s one from the “welcome to the real world, kids, we have no sympathy for your plight” files: social media giant Meta’s engineering team has bemoaned the complexity of migrating from legacy technology. In a Thursday post detailing migration of exabyte-scale data stores to new schemas, a quartet of Meta software engineers …
In the past days, libero.it, a mail system still used by millions of Italians (it was one of the commom mail providers before the gmail era) went down for days because a data migration to a new storage system went bad. Rumors say the issue was due to a bug in a new NetApp system. People could not access, send or receive email from 23 january until yesterday - and it looks the service is not available again for everyone.
As one who's main job (when I had one) was data centre migrations, I can sympahise - but they should have know this before they started.
Data migration is HARD! It's amazing how many of my customers thought you could just pick the data up, move it (sometimes to another country) and just expect it to work. Until I showed them a mind map I'd developed of all the different bits that needed to be sorted out. [I was going to write a book on it when I retited, but I haven't had time].
It sounds like making a database system that can handle all the data you decided to record even though I didn't give you permission to have it works a lot better than tying a bunch of disconnected systems together with APIs that don't know about the system as a whole. Imagine that.
I'm dealing with a similar set of codebases at the moment, and it's as if people don't understand that building something with no design for expansion then patching each new feature on in the way that takes the least amount of time has any downsides. They don't seem to think that there's anything bad about the fact that there are three systems for doing the same thing, all of which work in different ways, have different interfaces, and are missing a few features that another one has. In fact, one suggestion that was recently made was to build a shim connecting two of these redundant systems together, but only to use about 10% of the functionality of one of them so let's just leave out all the other functions of that system. This is why I have come to dislike old systems; the original design was probably fine back in the day (some exceptions apply), but if people have been sticking on new editions for the past twenty years, it's less likely that it in its entirety is of acceptable quality and trying to clean it up is such a difficult task that it appears completely futile.
> it's as if people don't understand that building something with no design for expansion then patching each new feature on in the way that takes the least amount of time has any downsides. They don't seem to think that there's anything bad about the fact that there are three systems for doing the same thing, all of which work in different ways, have different interfaces, and are missing a few features that another one has.
Odd, I thought the discussion about Windows Registry was going on in a different thread.
"Meta’s four engineers probably have offered useful insights for those who face similar data-wrangling challenges."
I have to interact with Meta/Facebook regularly.
So far as I can tell, there's just the one engineer. And he knows nothing.
Oddly enough they (Meta) consider these interactions sufficiently productive to shovel really quite large numbers of dollars to my employers, an advertising agency
-A.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2023
