---
id: "lib-2026-10-10-do-modifications-made-by-coding-agents-make"
type: "library"
title: "Do Modifications Made by Coding Agents Make Subsequent Software Maintenance Harder?"
lang: "EN"
org: "arXiv"
source_type: "Paper"
date: "2026-10-10"
week: "2026-W42"
url: "https://nosov.ai/library/lib-2026-10-10-do-modifications-made-by-coding-agents-make/"
original_url: "https://arxiv.org/abs/2610.14768"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "Peking University, UNSW and partner researchers use a paired design (same task, agent and tests, two starting histories) on 1,996 events with confidence intervals and a separate threats-to-validity section."
summary_basis: "full-text"
original_minutes: "40"
tags: ["ai technology", "coding agents", "maintenance", "benchmarks"]
frames: ["verification-capacity"]
---

# Do Modifications Made by Coding Agents Make Subsequent Software Maintenance Harder?

Tier A (Strong evidence): Peking University, UNSW and partner researchers use a paired design (same task, agent and tests, two starting histories) on 1,996 events with confidence intervals and a separate threats-to-validity section.

Summary based on the full text.

## In short

An arXiv paper (10 October 2026) by Ruwei Pan, John Zhang and colleagues. When a coding agent keeps maintaining a repository, its own earlier changes become the starting point for the next task. The authors run each later task twice, once on top of the developers' real patches and once on top of the agent's own changes. Building on its own history lowers success in all ten system-model combinations. The cost comes almost entirely from earlier tests the agent left failing; when the inherited code passes those tests, the penalty disappears.

## Key numbers

- 5.6 percentage points: average drop in later-task resolution when an agent builds on its own earlier changes
- 14.0 points: gap when all earlier tests must also keep passing
- 1.9%: events built on repositories already failing earlier tests that end with all accumulated tests passing
- +2.7 points: difference once inherited code passes all earlier tests (95% CI −0.5 to +6.1), i.e. no penalty
- 1,996: paired later events across five coding systems and two backbone models

## What it reports

- SWE-Maintain: 100 three-event maintenance episodes from 50 real Python repositories, with hidden tests certified to hold across each sequence.
- Agent-History resolves 5.6 points fewer later tasks than Developer-History (95% CI −7.2 to −4.1); at the first event, where both start identical, the difference is +1.8.
- Requiring earlier tests to keep passing widens the gap to 14.0 points.
- Two-thirds of Agent-History starting repositories already fail earlier tests; only 1.9% of events built on them end fully green, and even the new task alone is solved 7.6 points less often.
- With inherited code passing all earlier tests and the next task not already solved, the agent does as well as from developer history (+2.7 points, CI −0.5 to +6.1).
- The cost holds for assertion-only failures (−4.4 points), so it is not just interface mismatches; the gap grows from the second to the third event.

## How they got it

Paired experiment: for each later event, same instruction, agent, backbone and hidden tests, varying only the starting code. Five systems (including Mini-SWE-Agent, OpenHands, Claude Code and the non-agentic Agentless pipeline) with two backbones, among them DeepSeek-V4-Flash. Intervals are clustered by repository. Agents never see the hidden or future tests.

## Limits

- One run per setting and event; conclusions rest on aggregates.
- Python only, three-event sequences; longer chains may compound.
- 137 of 300 instructions are restated from short commit titles and may underspecify tasks (main results hold on both subsets).
- Developer patches may appear in pretraining data, which could favour the developer-history setting.
- Mixed human-agent histories are not studied.

## Why it matters for companies

The finding narrows where agent "technical debt" comes from: broken earlier behaviour, more than messy code. That gives a simple control for teams letting agents maintain code over many changes: run the full accumulated regression suite on every agent change and block merges that leave earlier tests failing. Benchmarks that start every task from clean human code overstate how well agents sustain a codebase. Investment in regression coverage pays twice when agents do the maintenance.

Original: https://arxiv.org/abs/2610.14768