---
id: "lib-2026-10-19-documentation-drift-in-ai-generated-pull-requests"
type: "library"
title: "Documentation Drift in AI-Generated Pull Requests"
lang: "EN"
org: "arXiv (Monash)"
source_type: "Paper"
date: "2026-10-19"
week: "2026-W43"
url: "https://nosov.ai/library/lib-2026-10-19-documentation-drift-in-ai-generated-pull-requests/"
original_url: "https://arxiv.org/abs/2610.27381"
tier: "B"
tier_name: "Useful, limited"
tier_reason: "Monash University mining study on a large public dataset with a replication package, but a short paper built on a file-path proxy, with no human baseline and associations that are not causal."
summary_basis: "full-text"
original_minutes: "15"
tags: ["ai technology", "coding agents", "documentation", "technical debt"]
---

# Documentation Drift in AI-Generated Pull Requests

Tier B (Useful, limited): Monash University mining study on a large public dataset with a replication package, but a short paper built on a file-path proxy, with no human baseline and associations that are not causal.

Summary based on the full text.

## In short

A short preprint (arXiv, 19 October 2026) by Will De Michele and Kla Tantithamthavorn of Monash University. It asks whether AI coding agents update documentation when they change code. In most agent pull requests they do not, files changed without documentation attract more later fixes once change size is controlled, and the gap is rarely closed later. Documentation debt from agents accumulates.

## Key numbers

- 29,669: AI-generated pull requests from 2,585 repositories analysed (AIDev dataset)
- 63.2%: AI pull requests that change code without updating any documentation file
- 77.5%: drift rate for refactor PRs, versus 55.2% for feature PRs
- 8.6-11.7%: how much more often undocumented code files get a same-file fix within 30-90 days
- 24% vs. 48%: share of files later re-documented within 90 days, changed without versus with documentation

## What it reports

- 63.2% of AI PRs change code without touching any documentation file. Every agent drifts in 48-73% of its PRs.
- Task type matters: 55.2% for features, 77.5% for refactors.
- Undocumented code files are followed by a same-file fix 8.6-11.7% (relative) more often across 30-90 day windows.
- A raw PR-level comparison shows the opposite, because documented PRs are 3.9x larger (24.83 vs 6.43 files) and so attract more fixes; controlling for size reverses it.
- Documented changes merge faster despite their size (median 40s vs 184s), though merging in this data is largely automated.
- A file changed without documentation is re-documented within 90 days at about half the rate (24% vs 48%).

## How they got it

The authors used the public AIDev dataset (MSR 2026 Mining Challenge), PRs from Claude Code, Cursor, Devin, GitHub Copilot and OpenAI Codex between 24 December 2024 and 4 August 2025. Each changed file was classified as code, documentation or other from its path; a PR is "drifted" if it changes code and no documentation file. Fix events come from 19,881 merged PRs, of which 4,526 are bug fixes, attributed to earlier PRs that touched the same file within the window. Scripts are published on Zenodo.

## Limits

- Drift is a proxy: an unrelated CHANGELOG edit counts as documented, and inline comments, docstrings and external wikis are invisible.
- Fix attribution is file-level and temporal, not causal; RQ2 does not control for task type, agent or repository workflow.
- No human baseline, so 63.2% cannot be judged high or low against human practice.
- The dataset is dominated by Codex (65.8%, against 1.4% for Claude Code) and predates current agents.

## Why it matters for companies

Teams that let agents open pull requests should treat documentation as a review item, since agents mostly skip it and nobody returns to it. A simple gate (code change without a documentation touch on public interfaces) is cheap to add. Internal knowledge bases and retrieval systems that index repository documentation will drift from the code faster as agent share grows, which degrades any assistant built on that documentation.

Original: https://arxiv.org/abs/2610.27381