Google reports halving code migration time with AI help Google, which peddles AI software with as much giddy enthusiasm as Microsoft, reports dogfooding its own AI concoction and leaving the lab with a pleasant taste in its mouth. In a pre-print paper, Google computer scientists Stoyan Nikolov, Daniele Codecasa, Anna Sjovall, Maxim Tabachnyk, Satish Chandra, Siddharth Taneja, and …
I wonder how much time they had to spend teaching the LLM how to do the job. That time certainly wasn't figured in to the calculation. In the end, LLM-written code is about as good as would be produced by some third world outsourcing shop, and you probably paid more for the LLM than you would have paid for the outsource.
Geez ... the next AI Winter can't possibly come soon enough.
In the end, LLM-written code is about as good as would be produced by some third world outsourcing shop
That may be true, which is why they're not using the LLM to write code. They're using it to modify existing legacy code. Two of the examples they gave are migrating Java 8 applications to Java 17, and migrating JUnit 3 tests to JUnit 4.
Unlike writing all-new code, these are tasks that are much better defined, much easier to train a model to perform... and much, much more tedious. They are high-effort, low-perceived-reward tasks. (Especially when success is defined as there being no discernible effect on the software itself. A successful migration is one where nobody can tell you've done it, because the post-migration code works exactly the same as before... except, you've sunk thousands of person-hours into making internal updates.)
Scouring hundreds of thousands of lines of code looking for deprecated or changed APIs, then updating the code to newer interfaces, is such a tedious job that many shops end up deciding to never do it at all. Legacy code is just kept limping along in legacy systems as long as possible, until it's eventually replaced with all-new code written to newer standards.
(That all-new code is very frequently a bare-bones affair. It eliminates deprecated APIs and other outdated internals, yes, but also strips away many of the features of the older codebase that — due to time, money, or design constraints — don't get reimplemented in the first versions of the new, replacement system. That's how you end up with software releases that hemorrhage functionality compared to their previous versions, because the "totally rewritten!" new codebase threw away everything that came before — including a ton of the features.)
If Google can use AI-assisted processes to avoid some of that write-bandage-discard loop, and break the ground-up rewrite cycle, I'm all for it.
Effectively, kind of yeah! But it's clever how they did it.
We validate the resulting changes automatically. The validations are configurable and often depend on the migration. The two most common validations are building the changed files and running their unit tests. Each of the failed validation steps can optionally run an ML-powered “repair”. The model has also been trained on a large set of failed builds and tests paired with the diffs that then fixed them. For each of the build/test failures that we encounter, we prompt the model with the changed files, the build/test error and a prompt that requests a fix. With this approach, we observe that in a significant number of cases the model is able to fix the code.
So they have a unit test process with an AI component that can react to failed tests by suggesting fixes. The two AI processes are totally independent of each other.
There's a bit of hand-wavy assumption to this, but..
They provided some tens-of-thousands of lines of hand-created examples with which to train an LLM. (Presumably they trained it from scratch on the codebase, not just passed it into an LLM as context.) At the least, then, they needed a few thousand lines of manual conversions.
They most likely had to cover every *common* case that they wanted to be handled by the LLM, with some degree of repetition. Then, the hand-handled lines, or other odd differences, required human intervention -- and presumably the LLM did not tell them that something was awry.
The code follows some sense of standards -- Google has coding style documents, variable-naming documents, how-to-reference X documents. Things are somewhat "regular" -- i.e. this is for *their* code, and not _your_ code unless you follow the same peer-reviewed style guides.
In the end, they hit the 80/20 rule: they were able to migrate 80% of data, with that last 20% taking 50% of the total time -- and requiring humans to perform the work, at least somewhat. (Probably all of the not-so-well represented training cases, styleguide exceptions, ...)
Google is a failed company falling in slow motion. Whatever they proclaim in any sort of announcement is probably just being said to try and get you to buy ads or click on ads, or accidentally activate the AI popup when you were trying to dial emergency services on your phone. They had to try AI code generation because they laid off all the other employees. Doesn't matter, they were canceling those products anyways.
My only fear is that all the ex-Googlers there are now will carry the Google-style dysfunction into their new endeavors.
int main(enter the void)... The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
