Hybrid LLM-Deterministic Pipelines for Enterprise-Scale Code Migration at Bloomberg
A · Strong evidence Bloomberg engineers and UCL researchers report two production migration campaigns with full PR outcome tables, an explicit architecture and frank threats to validity; a well-documented industrial case, though self-evaluated.
Summary based on the full text·Original: about 30 minutes to read
In short
An arXiv industrial experience report (3 November 2026) by Bloomberg engineers with UCL co-authors on "Automated Uplift": LLM-assisted pipelines that migrate low-priority infrastructure across thousands of repositories. Deterministic code does most of the work, and the LLM handles one bounded file transformation with pre-computed context, including production runtime data. Generating changes was the easy part. The bottleneck was owning teams reviewing, testing and deploying them, which went faster when the pipeline could verify a change itself.
Key numbers
- 2,333migration pull requests generated across about 8,600 candidate repositories
- 729PRs merged two months or more after opening (34.2% for C++ build, 10.5% for Python packaging)
- 67.7% and 57.7%Python and C++ PRs with no developer decision at all by the end of observation
- 34 of 51Python repositories that rejected the bot's PR and later completed the migration themselves
- well over a thousandperson-days of manual work the merged changes represent, by the team's estimate
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Two campaigns over about six months: Python packaging (about 1,900 candidate repositories, 294 PRs) and C++ build system to CMake (about 6,700 candidates, 2,039 PRs).
- Outcomes at 30 June 2026 for PRs opened by 30 April: C++ 698 merged (34.2%), 165 rejected (8.1%), 1,176 no decision (57.7%); Python 31 merged (10.5%), 64 rejected (21.8%), 199 no decision (67.7%).
- C++ PRs arrived compiled and tested, so reviewers checked a build that already worked; Python PRs had only packaging and smoke checks, so owners carried the deployment risk.
- C++ projects fell into a few archetypes, so fixes carried over; Python candidates spread over 18 archetypes with many one-off cases.
- In 34 of 51 rejected Python repositories, teams later landed the migration themselves, and 14 of those PRs reused most of the bot's changes. A common reason for re-creating a PR: developers could not push to the bot's branch.
- LLM inference cost was negligible next to the engineering effort of building and running the pipelines. Manual migration estimates: about 2 person-days per C++ task and 1-2 per Python task.
- Models in use as of April 2026: Sonnet 4.5 and Opus 4.6.
How they got it
An experience report on live campaigns, with PR outcomes counted from the repositories and timelines of opened and merged PRs. Architecture, archetype classification and lessons are described from the building team's records.
Limits
- No controlled experiment; pipeline code, prompts, validation rules and models changed during the period, so improvements cannot be attributed.
- The claim that runtime data is essential rests on engineering judgment, not an ablation.
- Cost and person-day estimates come from the team that built the system; one organization, with proprietary tools and a many-repository setup.
Why it matters for companies
For large estates, the question is no longer whether an LLM can write the migration but whether owning teams will accept it. Investment should go into validation inside the pipeline, evidence attached to each PR, and letting owners edit the generated PR directly. Uniform, compile-checked migrations are the best first targets; heterogeneous ones need more human effort per change. The economics also open up backlog work that never cleared the priority bar when each repository cost days of engineering time.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org