---
id: "lib-2026-10-22-stale-instruction-files-override-the-code-in"
type: "library"
title: "Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix"
lang: "EN"
org: "arXiv (Georgia Tech, CU Boulder)"
source_type: "Paper"
date: "2026-10-22"
week: "2026-W43"
url: "https://nosov.ai/library/lib-2026-10-22-stale-instruction-files-override-the-code-in/"
original_url: "https://arxiv.org/abs/2610.30571"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "Two-author controlled experiment (Georgia Tech, CU Boulder) on two production coding agents with a clear mechanism, a reproducible design and a fix anyone can test in minutes."
summary_basis: "full-text"
original_minutes: "20"
tags: ["ai technology", "coding agents", "agents.md", "context engineering", "experiments"]
frames: ["tacit-to-explicit"]
---

# Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix

Tier A (Strong evidence): Two-author controlled experiment (Georgia Tech, CU Boulder) on two production coding agents with a clear mechanism, a reproducible design and a fix anyone can test in minutes.

Summary based on the full text.

## In short

An arXiv preprint (22 October 2026) by Xiaonan Xu and Wenjing Wu. Coding agents treat CLAUDE.md or AGENTS.md as instructions and the code they read as tool output, which models are trained to rank lower. When the instruction file is out of date, agents follow it against the code. One sentence in the user-level instruction file, giving existing code precedence and asking the agent to report each inconsistency, reversed the outcome in 320 runs.

## Key numbers

- 90.1%: share of agent choices (649 of 720) that followed a stale instruction file over the existing code
- 4.2% to 91.4%: share of choices following the existing code, without vs with the one-sentence rule
- 79 of 80: Claude Code runs that named the conflict in the final reply and still followed the file
- 20 of 80: Codex CLI runs that rewrote existing code to match the file; original tests then failed in 17
- 89.4% to 100%: test pass rate without vs with the sentence

## What it reports

- Without the sentence, 90.1% of choices matched the stale file: 86.4% for Claude Code, 93.9% for Codex CLI. The file prevailed in 14 of 16 kinds of convention; Money and Configuration were the exceptions for Claude Code.
- Claude Code's final reply named the conflict in 79 of 80 runs, yet 86.2% of its choices in those runs still matched the file. Codex CLI named it in 2 of 80.
- Codex CLI rewrote existing modules in 20 of 80 runs (median 198 lines removed from 9 files); the project's original tests failed afterwards in 17 of them.
- With the sentence, 91.4% of choices matched the code (Claude Code 94.4%, Codex CLI 88.3%), no run rewrote existing code, both agents reported conflicts in all 160 runs, and tests passed in every run.
- Background evidence cited: 73.8% of AI configuration files in 441 company repositories were committed once and never changed; real stale files in Supabase and OpenAI Codex stayed wrong for 2 to 24 days.
- The sentence used: "When a repository's documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code."

## How they got it

40 small Python projects, each with working modules, passing tests, an instruction file that contradicts the code on four or five conventions, and a feature request. Claude Code (Sonnet 5.5) and Codex CLI (GPT-6 Sol) ran non-interactively, twice per project per condition. A script classified each choice; a separate model judged, blind to condition, whether replies named a conflict; held-out tests checked the result.

## Limits

- Synthetic, small projects in one language, with single unattended runs and two agent versions.
- The design assumes the file is the stale source. Where the code is wrong and the file states the intended target, the same sentence would preserve the error; the paper does not test that case.
- Conflict detection in replies was judged by another model.

## Why it matters for companies

Any team that runs coding agents on repositories with instruction files has this exposure, and the files rarely get updated. The fix costs one line in a user-level file and turns the agent into a detector of documentation drift, since it lists each inconsistency. Teams should still separate rules ("run the tests before committing") from descriptions of the code, which go stale, and review instruction files when conventions change.

Original: https://arxiv.org/abs/2610.30571