Agentic Coding in the Wild
A · Strong evidence Microsoft Azure Research systems paper built on production telemetry from GitHub Copilot's coding agent covering 3.2M users and 761M LLM calls, the first characterisation of agentic coding at this scale.
Summary based on the full text·Original: about 60 minutes to read
In short
Banruo Liu (UIUC, Microsoft intern) and colleagues at Microsoft Azure Research, including Ricardo Bianchini and Esha Choukse, analysed sampled telemetry from GitHub Copilot's coding agent in Visual Studio and VS Code for the first week of June 2026. The dataset covers 13.5M sessions, 3.2M users, 760.5M LLM calls and about 95T tokens. They show that agentic coding behaves very differently from chat: a few user prompts each set off long autonomous loops of model calls and tool runs, which changes how compute, caching and cost behave.
Key numbers
- 87%LLM calls initiated by the agent rather than the user
- 9% of turnshit tool failures that trigger retry loops amplifying compute up to 4x
- 50xrange in tokens per turn across five user archetypes (23K to 1.1M)
- 86-90%share of idle time captured by the authors' idle-time predictor
As reported by the source.
English·PDF·opens microsoft.com
Figure from the source
Exhibit 1
The paper page and abstract
What it reports
- 87% of LLM calls are agent-initiated; model calls and tool calls run at a near 1:1 ratio (760.5M LLM calls vs 774.7M tool calls).
- The median session has 3 user turns, 15 LLM calls and lasts 4.2 minutes; the mean is 6.1 turns, 40.6 calls and 62.6 minutes. P90 sessions exceed three hours.
- Prompts dwarf outputs: a typical call carries 68K prompt tokens and 247 output tokens; in total 44.9T prompt tokens against 39.3B completion tokens.
- Cached prefixes cover 90% of tokens within a turn, 55% across turn boundaries and only 8% after a model switch.
- Context compaction occurs in 7.8% of sessions but affects 44% of total tokens, dropping over 70% of prompt tokens.
- Tool failures occur in 9% of turns and trigger retry loops; "deep-loop with failures" turns (9.1% of turns) run 36 LLM calls, four times the median, each processing 80K+ tokens.
- Turn types: read-heavy exploration 30.5%, pure reasoning with no tools 20.2%, read-edit-build cycles 19.0%, terminal-heavy 8.1%.
- User types: Readers 41.7%, Coders 30.4%, Terminal users 11.0%, Deep-loop users 9.2% (1.1M tokens per turn), Chat-only 7.6% (23K tokens per turn).
How they got it
Anonymised client-side telemetry with timestamps, token counts, model and tool names and success status, but no prompt text, code or outputs. Traces come from US regions, more than 27 models and 45 tools. Turns and users were clustered into archetypes, and a LightGBM model was trained to predict idle time at turn boundaries.
Limits
- No quality signal: the data cannot say whether tasks succeeded.
- Client-side only, US-only, one week; no server GPU metrics.
- Written by the platform operator, focused on serving efficiency rather than developer productivity.
Why it matters for companies
Agent usage cost is driven by long autonomous loops and repeated context, not by the number of prompts a person types, so per-seat or per-prompt budgeting understates real consumption. A small group of heavy users drives most tokens, which matters for licence and chargeback models. Failed tool calls multiply cost, so reliable build and test environments reduce AI spend as well as developer frustration.
Summary based on the full text. Read the original for the full argument and data.Open PDF on microsoft.com
- Tags
- Coding agents