Impact of Expertise and Trust on AI-assisted Work
B · Useful, limited Carnegie Mellon and Microsoft researchers ran a careful think-aloud study on a real hiring task, but with 20 participants, one task and self-reported expertise the authors themselves call the results hypothesis-generating.
Summary based on the full text·Original: about 45 minutes to read
In short
An arXiv paper (10 October 2026) by Qianou Ma, Kenneth Koedinger, Tongshuang Wu and Microsoft colleagues. Twenty professionals used GitHub Copilot's agent mode on a data-science take-home test once used for hiring at a large US software company: build an employee-attrition model from data with no attrition label. Those who delegated more finished more of the notebook and were more likely to submit invalid work. Experts more often slowed down to decompose and check, trading completion for understanding. Trust made people feel better, not perform better.
Key numbers
- 20professionals (designers, PMs, engineers, data scientists) observed on the same task with the same agent
- 11 of the 15submitted notebooks with trained models that contained label leakage, none noticed by their authors
- 65%final-artifact validity when participants explored and decomposed the task at planning
- 16.8%workplace ChatGPT use involving work outside the user's own occupation (cited)
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Experts more often decomposed the problem, investigated assumptions and steered the agent; novices more often copy-pasted, delegated decisions and reviewed only on the surface.
- Greater involvement was linked to better understanding but less complete notebooks within the 30-minute limit.
- One-shot agent output produces label leakage (the proxy label is built from the same features used to predict it). 11 of 15 notebooks with a trained model had it; none of those authors (4 experts, 7 novices) noticed. Only two participants, both data scientists, caught or avoided it.
- Validity-preserving moves: explore and decompose at planning (65% validity), manual steering during implementation (62%), critical guidance at review (62%). Reading output line by line without interrogating it left validity low (38% for two participants).
- Trust in AI was associated with confidence, engagement, satisfaction and lower cognitive load, but not with artifact quality.
- Stated willingness to delegate barely related to observed delegation.
How they got it
In-person think-aloud sessions with surveys on expertise and trust, full Copilot interaction logs, screen and audio recordings, submitted notebooks and post-task interviews. Notebooks were scored for completeness and validity; Spearman correlations and qualitative coding link behaviours to outcomes. Human-only subtasks tested defect spotting without an agent.
Limits
- N = 20, one organisation, one task, one agent and a 30-minute limit.
- Expertise and trust are self-reported.
- Correlations with wide intervals; patterns are directional.
- The time limit favours completion over checking, which may exaggerate the trade-off.
Why it matters for companies
Finished-looking AI work can be wrong in ways only domain experts catch, and in this study even most experts missed the core error. Firms that measure agent value by output volume or completion will reward the behaviour that produces invalid work. Where results feed decisions, require a reviewer with domain knowledge, check the specific failure modes (here, leakage), and allow time for it. Training should teach decomposition and critical review, since confidence and satisfaction are poor proxies for quality.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Work and AI, Verification