Did Microsoft's code get worse with AI? Benchmarks of .NET libraries from 6.0 to 11 RC (Стал ли код Microsoft хуже с приходом ИИ: замеры .NET-библиотек от 6.0 до 11 RC)
A · Strong evidence Single-author Habr study with a pre-set statistical method (hierarchical bootstrap, Mann-Whitney with Benjamini-Hochberg), Git attribution of agent commits, a bisect to individual commits and published code and data.
Summary based on the full text·Original: about 12 minutes to read
In short
A Habr post (6 November 2026) by a .NET developer tests whether Microsoft's libraries got slower once AI arrived, using .NET 7 (released three weeks before ChatGPT) as the baseline. It benchmarks 46 operations across DI, configuration, logging, caching and EF Core over six major versions, then checks in Git who touched the hot methods. Warm paths got faster and first calls got slower. The biggest 11 RC regression came from one line written by a person; a smaller one came from a Copilot-agent PR that fixed a deadlock. The author finds no sign AI made the code worse, and no proof of the opposite.
Key numbers
- 1,734 / 1,207PRs opened by Copilot coding agent in dotnet/runtime by end of October 2026 / of them merged
- 0 of 113agent commits among DI and memory-cache commits from v7.0.20 to v10.0.14
- 9 faster, 13 slowerof 46 operations, 10.0 vs 7.0 (10 within noise, 14 without a verdict)
- 61 to 145 µsPOCO binding of 50 fields, 7.0 vs 11 RC; traced to one human-written line checking an attribute per property
- +16.5%first DI resolve cost added by one Copilot-agent PR that fixed a deadlock, about 0.9 µs
As reported by the source.
Russian·Web page·opens habr.com
What it reports
- 10.0 vs 7.0: repeated operations 14-35% faster (for example LoggerMessage.Define 377 to 244 ns); first calls and object creation 39-64% slower; the first EF query in a process rose from 996 to 1,162 ms.
- Most change happened at the step to 8.0: 19 of 46 confirmed shifts, versus 5, 5 and 6 of 38 at the next steps.
- Git attribution: no agent commits among 113 DI and cache commits from 7.0 to 10.0; in 10.0 to 11 RC the agent has 11 of 21 DI commits and 73 of 256 EF Core commits.
- 11 RC: POCO binding 61 to 145 µs and IOptionsSnapshot 52 to 136 µs, both about +10 KB per call. Replacing one IsIgnoredProperty check (added for a new [ConfigurationIgnore] attribute in a human PR) restored 10.0 levels. The author filed issue #136759; the maintainer accepted it as a regression, and a draft fix PR exists.
- The agent PR #124331, fixing a deadlock with circular factory dependencies, adds 16.5% (95% CI 14.3-18.8) to the first DI resolve; reverting it restores 10.0 speed. The author calls this a fair trade.
- Turning off turbo boost, as benchmarking guides advise, made all operations 1.6-2.1x slower and changed relative results by up to 9-14 percentage points.
- Allocations were the most reliable signal; DI code grew 25% and DI.Abstractions 129% from 7.0 to 11 RC.
How they got it
Runtime fixed at .NET 10.0.11 while NuGet packages vary; each scenario in its own pinned process, 20 warm-up and 200 measured iterations, 30 rounds with shuffled version order. A verdict needs the whole 95% interval beyond 5%, no contrary round and adjusted p below 0.05. Agent commits were identified by the copilot-swe-agent app id and co-author trailers. Code, data and method are published.
Limits
- One laptop with a hybrid CPU, no core isolation; the verdict rules were tightened after the first rounds.
- Git shows only PRs opened by the agent; IDE assistants leave no trace, so the "assistant era" cannot be measured.
- EF Core, where agent commits are most numerous, was only counted, not tested for cause; release candidate, not final 11.0.
Why it matters for companies
The debate on whether AI-written code degrades software usually runs on anecdote. This study shows how to answer it for a specific codebase: fix the method, measure, attribute commits and bisect. Here the agent's change was a reviewed, measurable trade-off, and the worst regression was human. For teams adopting coding agents, the practical lesson is to keep performance tests realistic (the official binding benchmark used a one-property class) and to record agent authorship so it can be audited.
Key passages in translation
Russian original
Самую крупную внёс человек одной строкой. Вторая, поменьше, пришла с PR от Copilot coding agent, который чинил дедлок.
The largest was introduced by a person in one line. The second, smaller one came with a PR from Copilot coding agent that was fixing a deadlock.
Russian original
Коммит агента значит «PR открыт агентом», а не «написано без человека»: такие PR проходят ревью и вливаются людьми.
An agent commit means "PR opened by the agent", not "written without a human": such PRs go through review and are merged by people.
Russian original
в том, что я проверил, нет признаков того, что код стал хуже из-за ИИ, но и доказательства обратного нет.
In what I checked there are no signs that the code got worse because of AI, but there is no proof of the opposite either.
Summary based on the full text. Read the original for the full argument and data.Read the original on habr.com