TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents
A · Strong evidence Penn State workshop paper (POVC '26 at ASE 2026) that re-scores 600 public coding-agent trajectories under an exact prefix-cache simulation, derives a break-even rule and releases the tool, so every figure can be recomputed.
Summary based on the full text·Original: about 30 minutes to read
In short
Pramod Kumar and Raj Kumar Pandey ask where the money goes in a coding agent's loop. Most efficiency work compresses tool outputs or routes turns to cheaper models. Their re-scoring of 600 published trajectories shows tool outputs are mostly small, almost all input is served from cache, and the number of turns is what drives cost. They call the design of this control loop "loop engineering" and release TurnCost, a tool that recomputes any trajectory corpus under different caching, pricing and routing policies.
Key numbers
- 98.6%share of input tokens served from the prefix cache across the 600 trajectories
- 7.1xaverage overstatement of loop cost under flat input pricing, assuming near-ideal caching
- 84% vs 4%variance in log trajectory cost explained by turn count vs mean observation size
- 118-286 tokensmedian tool observation size across the four scaffold-model cells
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Median tool observations are 118-286 tokens. The large build logs used to motivate compression sit near the 99th percentile.
- 98.1% to 99.0% of input tokens hit the prefix cache, so flat pricing overstates cost by 6.5x to 7.8x, mean 7.14x. The error grows with loop length: 5.33x below 40 turns, 7.99x above 130.
- Turn count explains 84% of the variance in log trajectory cost; mean observation size explains 4%. The most expensive runs have the most turns and the smallest observations.
- Delegating observation handling to a cheaper model pays only if remaining turns exceed the ratio of the weak model's input price to the strong model's cached-read price. That bar is about ten times tighter than list-price maths suggests. Delegation raised cost in half the tested settings.
- It explains a prior result (AgentDiet): simply deleting observations reached 0.655 of baseline cost, against 0.652 for a learned reducer.
- Routing papers reporting savings up to 72% mostly used 4-12 step trajectories and cache-unaware accounting; coding trajectories here run a median 60 to 134 turns.
How they got it
150 trajectories from each of four cells of nvidia/Open-SWE-Traces (OpenHands and SWE-agent crossed with Qwen3.5-122B-A10B and Minimax-M2.5), sampled deterministically. TurnCost rebuilds per-turn context from the message log, tokenises it and simulates the cache exactly from the append-only transcript. Prices are expressed as ratios so results do not hinge on one vendor's list.
Limits
- The turn-count result is associational. The authors do not show that turns can be cut while keeping task success.
- The 7.1x figure assumes near-ideal caching; it falls to 4.0x at 90% cache efficiency and 1.45x at 50%.
- The corpus was filtered as distillation training data, excluding runs that hit iteration limits, so the long tail is truncated.
- Two models and two scaffolds; absolute numbers are not comparable with SWE-bench Verified studies.
Why it matters for companies
Anyone budgeting agent workloads from list prices is likely overestimating cost several times over, and any vendor claiming large routing savings should be asked whether caching was priced. The practical lever is fewer turns: clearer task specs, better tools and earlier stopping rules. Switching models mid-loop can cost more than it saves because the new model starts with no cache. Measure cost per completed task on your own traces before buying optimisation layers.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Coding agents, Cost of AI, Caching