Catching Developers in the Flow: Low-Latency Agentic Program Repair at Google Scale
A · Strong evidence Google engineering paper accepted at ASE 2026 with production telemetry from 2,567,829 changes and 36,479 developers plus an expert evaluation of 195 failures, one of the largest industrial agent deployments reported.
Summary based on the full text·Original: about 40 minutes to read
In short
A preprint (arXiv 2610.07289, 5 October 2026) by Celal Ziftci and four Google colleagues, accepted at the ASE 2026 software engineering conference. It describes FlowAgent, an agent that repairs failing tests in Google's continuous integration before code is submitted, aiming to deliver a fix while the developer is still working on the change. In expert review it fixed 67.18% of failures correctly; in production since October 2025 it posted fixes on 295,508 changes, and developers applied 28,554 of them.
Key numbers
- 67.18%correct fixes in expert review of 195 real test failures
- 28,554changes where developers applied an agent fix, out of 295,508 that received one
- 9.85 minmedian time for the agent to produce a successful fix
- 27.48 minmedian time before a developer starts editing after a test-failure notice
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- In a 30-day September 2025 sample, the time from a test-failure notice to the developer's next edit was 2.37 minutes at p10, 6.72 at p25 and 27.48 at p50; this is the window the agent targets.
- Three developers with at least five years' experience judged 131 of 195 fixes correct (67.18%), across failures from 37 teams.
- Production funnel: 2,567,829 changes with failing tests; 1,785,955 dropped by pre-filters; 781,874 attempted; 421,818 fixed (53.95%); 126,310 dropped by safety filters; 295,508 posted.
- Developers previewed fixes on 65,069 changes (22.02% of posted) and applied them on 28,554 (43.88% of previewed).
- Apply-to-preview rate was 37.84% in the web code review tool and 59.69% in the IDE, where developers can edit the fix further.
- Attempts covered changes by 36,479 developers, with a median of 6 files and 2 failing tests per change.
- Median agent runtime for a successful fix was 9.85 minutes, p90 25.83 minutes, under a 30-minute cap and 100 tool calls.
How they got it
FlowAgent uses an internal Gemini 2.5 Pro fine-tuned on Google code, in a ReAct-style loop with tools to read, edit, build and test code in a sandbox. Filters skip flaky, automated, stale or oversized changes (over 100 files or 1,000 lines) and re-run tests before posting. The authors combine the manual evaluation, production telemetry from October 2025 onward, and 45-minute interviews with 9 volunteer developers.
Limits
- Results come from Google's monorepo, internal tools and a fine-tuned model; transfer to other environments is untested.
- Applying a fix is a usage signal, not a correctness measure, and the production data do not control for developers' attitudes to AI.
- Interviewees self-selected, and the 67.18% accuracy rests on 195 cases.
Why it matters for companies
The paper shows what a production agent funnel looks like: most triggers are filtered out, about half of attempts succeed, and fewer than one in ten posted fixes is applied. Companies piloting agents should expect similar drop-offs and measure each stage of the funnel alongside model accuracy. The design choice that matters is timing and placement: fixes are delivered inside existing tools before the user switches context, with a deterministic test check before anything is shown. That pattern applies to agent support in finance, HR or procurement workflows as well as code.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org