Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests
A · Strong evidence An independent researcher and a UNSW academic mine 26,999 pull requests from two public datasets with exact strata matching against a human baseline; method and threats to validity are documented, though the causal framing rests on observables only.
Summary based on the full text·Original: about 40 minutes to read
In short
An arXiv paper (9 October 2026) by Mohammad Wathiq Soualhi and Basem Suleiman. It looks at pull requests from coding agents that change both application code and deployment infrastructure (CI pipelines, containers, provisioning). These are rare, under 5% of agent PRs, but they split into two outcomes: when reviewed they take over three times longer to merge, and roughly half are never reviewed and merge within half an hour. Maintainers review agent PRs much less than equivalent human ones.
Key numbers
- 26,999pull requests analysed across AIDev v5 and MOSAIC-3M, six agents plus a human baseline
- 63.7%lower odds of human review for agent-authored PRs vs human PRs, even among contributors without write access
- 43.1% to 56.2%agent PRs touching both code and deployment infrastructure that merge with no external review
- 3.24×multiplier on time-to-merge when such PRs are reviewed (+13.13 hours median)
- 4.48% to 4.55%share of agent PRs that modify both application code and deployment infrastructure
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Agents mostly write application code or maintenance chores; pure maintenance is 17.70% to 25.41% of agent PRs. Dual-scope PRs are 4.48% to 4.55%.
- Reviewed dual-scope PRs add a median of +13.13 hours and multiply time-to-merge by 3.24× (AIDev v5). CI manifests and provisioning files drive this, at 3.8× and 2.7×.
- Human review rates for human dual-scope PRs stay at 72.2% to 73.4%. For agents they move erratically: 47.5% to 53.1% in one dataset, 52.5% to 36.2% in the other.
- Among external contributors with no write access, agent identity cuts the odds of human review by 63.7%.
- 43.1% to 56.2% of agent dual-scope PRs merge with no external review, in a median under 31 minutes; reviewed ones wait 15.59 to 49.80 hours.
- The authors argue unreviewed changes to CI weaken the build check itself, since the change can alter the test that validates it.
How they got it
Two public PR datasets (AIDev v5, 21,855 PRs; MOSAIC-3M, 5,144 PRs split agent and human), restricted to repositories with over 100 stars. A deterministic four-tier taxonomy classifies files by path. Greedy 1:1 nearest-neighbour matching pairs dual-scope and app-only PRs within exact strata (agent, architecture, test maturity, language) and close application file volume. Log-normal models with cluster-robust errors estimate latency effects.
Limits
- Only merged PRs are observed, so abandoned ones are missing.
- Matching controls observables; task intent (bug fix vs feature) and offline communication remain unobserved, so "causal" is stronger than the design supports.
- Open-source repositories with over 100 stars; enterprise monorepos with SRE teams may differ.
- Bot-comment filtering by regex adds some noise; permission metadata exists only in one dataset.
Why it matters for companies
Agent-authored changes to pipelines and infrastructure are exactly the ones that should get more scrutiny, and in this data they get less. Firms letting agents open PRs should enforce path-based rules: changes to CI, container and provisioning files require a named human approver regardless of who or what authored them. Otherwise the speed of agent PRs is either eaten by review queues or bought with unreviewed supply-chain risk. Benchmarks that test agents only in sandboxes miss this integration cost.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Coding agents, Governance