Developers as Managers of AI Agents: A Workplace Shadowing Study
A · Strong evidence UC Irvine field study with Thomas Zimmermann and an SAP Labs co-author that video-codes real workplace sessions second by second and checks interpretations in interviews; rare direct observation with a transparent method, though 11 developers in about 90-minute sessions.
Summary based on the full text·Original: about 50 minutes to read
In short
A preprint (arXiv, 24 October 2026) by Yi-Hung Chou, June Hyung Kim, Vasisht Kartik, Victoria Jackson, Tobias Schimmer (SAP Labs), Thomas Zimmermann and James A. Jones. Researchers shadowed 11 professional developers using coding agents on their own work at a multinational software company and an open-source foundation. A third of the time went on directing and checking agents, and working styles split into three patterns. Agents supplied technical answers; colleagues still supplied organisational context, authority and agreement.
Key numbers
- 33.4%average share of observed time spent writing prompts (11.3%) or checking AI responses (22.1%)
- 0 to 101.6task switches per hour across participants
- 10 of 11developers who spent more time on agent-related activity than on inspecting code directly
- 255submitted prompts analysed with their surrounding activity
- about 15 hourstotal observation across 11 developers, plus 4.6 hours of follow-up interviews
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Time: 11.3% writing prompts (range 6.0%-18.6%), 22.1% checking AI responses (6.6%-59.4%), 8.9% inspecting code (0.5%-36.2%). One developer spent 77.9% of the session on prompts and checks.
- Participants handled 1 to 17 distinct tasks per session; switching ranged from 0 to 101.6 per hour.
- Prompts served to produce artifacts, understand agent behaviour, clarify requirements, configure agents and coordinate work.
- Supervisors delegated work but inspected the resulting code themselves.
- Delegators averaged 11.7 tasks and 79.7 switches per session; 69% of their prompts were followed only by reading the AI response, so checking was largely handed back to the agent.
- Learners used agents to understand unfamiliar code, tools and data; 76% of their prompts were inquiries.
- Window management took a measurable share of session time.
How they got it
Screen-recorded sessions of about 90 minutes with 13 developers, 11 analysed (one interrupted by a meeting, one hit its token limit). Recordings were segmented into 4,481 activity segments, prompts were linked to preceding activity and subsequent checks, and follow-up interviews used visualisations of each participant's session to test interpretations. Approved by an institutional review board.
Limits
- 11 developers, US-based, from two organisations; two authors had worked as contractors at one of them.
- One session per person; no measure of output quality or speed.
- "Checking" is coded from screen activity, without eye tracking or think-aloud data.
- Message content with colleagues was not analysed, for privacy.
Why it matters for companies
Agent-assisted development shifts the job toward directing, deciding what to verify and juggling parallel work, and those management skills vary widely between people. Delegators who let agents check their own work are a quality risk worth spotting in code review practice. Planning and training should treat prompting and verification as core engineering skills, and leave room for the human channels (colleagues, leads) that agents do not replace for priorities and decisions.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org