Looks Fine to Me: Understanding Oversight in Agentic Interactions
A · Strong evidence NUS Augmented Human Lab and MIT Media Lab (Pattie Maes) run a controlled within-subject study combining task outcomes, logs, gaze and physiology, with per-condition numbers and tests reported; small sample, but the method is shown and the result is specific.
Summary based on the full text·Original: about 45 minutes to read
In short
A preprint (arXiv, 20 October 2026) by Dinithi Dissanayake and colleagues at the National University of Singapore and MIT Media Lab, including Pattie Maes and Suranga Nanayakkara. As agents do more of the work, people's job shifts to checking it. In a lab study, programmers working with a coding agent missed defects that produced no visible failure while staying highly confident in their result. A visible failure sharpened attention, but only for the very next task.
Key numbers
- N = 36programmers in the controlled study, each doing six tasks with a coding agent
- 79%failed silent-defect tasks where participants still rated confidence 4 or 5 out of 5 (34 of 43)
- 27.8% to 51.4%silent-defect resolution rate, round 1 to round 2
- 50% vs 9.1%silent defects caught right after a visible failure versus after one intervening correct task
- 10.8 vs. 5.8 minutestime on task when a silent defect was caught versus missed
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Each participant did two tasks per condition: correct code, a silent defect that passed visible pre-merge checks, and a blatant defect that failed them (214 usable task instances).
- Silent-defect resolution rose from 27.8% (10 of 36) in round 1 to 51.4% (18 of 35) in round 2, varying by task from 8% to 75%.
- Confidence did not track success: 34 of 43 failed silent-defect tasks (79%) were rated 4 or 5 out of 5. Mean confidence stayed near 4.1 in both rounds.
- Behaviour did track success: caught defects took 10.8 minutes on average versus 5.8 for missed ones.
- Self-rated thoroughness relative to a person's own norm was informative; absolute ratings across people were not.
- Python proficiency predicted catching silent defects (16.7% beginners, 39.1% intermediate, 57.1% advanced). Experience with agentic AI tools did not.
- A visible failure helped only when the silent defect came next: 50% caught (12 of 24) versus 9.1% (1 of 11) after one correct task in between.
- Failure modes: generic checks, not generating diagnostic evidence, and misreading evidence of the defect.
How they got it
A formative interview study (N=12) on everyday AI use, then a controlled within-subject study (N=36) on Python tasks with a coding agent whose outputs were seeded by condition. Data: task outcomes against hidden checks, interface and agent interaction logs, first-person video and gaze, heart-rate signals, and self-reports after each task. Permutation and Fisher tests handle the small cells.
Limits
- 36 participants in a lab, on short Python tasks; real codebases and stakes differ.
- Task difficulty varied widely and positions were not fully balanced, which the authors address only partly.
- Some sequence effects rest on small cells (11 tasks in one condition).
Why it matters for companies
"Human in the loop" is a common control for agent output, and this study shows its weak point: people sign off confidently on work that looks complete. Self-reported confidence is not a usable signal of review quality. Better levers: make failures visible (tests that would catch the defect class), require diagnostic evidence rather than a click, and do not count general AI-tool experience as review skill. Domain expertise was what predicted catching problems.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Work and AI, Coding agents