Reflections and New Directions for Human-Centered LLMs
B · Useful, limited A broad Stanford survey led by Diyi Yang with James Landay that grew out of a CS 329X course draft; useful as a map of human-centred LLM research, but it is a non-peer-reviewed arXiv literature review with no new data beyond cited studies.
Summary based on the full text·Original: about 3 hours to read
In short
An arXiv survey (2605.06901, 7 May 2026) from Stanford by Caleb Ziems and Dora Zhao with Rose E. Wang, Diyi Yang, James Landay and about 55 co-authors, mostly students from Stanford's CS 329X course. It sets out a framework for "Human-Centered Large Language Models", arguing that user needs, values and goals must be built into every stage of model development (design, data, training, evaluation, deployment) instead of being added during post-training. It closes with a case study on LLMs and the future of work. Its main claim is that the most important outcomes often come from design choices rather than model capability.
Key numbers
- 5xincrease in app usage time with the LLM coach versus a no-LLM control, while physical activity stayed comparable (N=54, four weeks)
- over 93%share of GPTCoach conversational strategies coded by experts as consistent with motivational interviewing or neutral
- 1,500US workers surveyed in a cited study showing the tasks they want agents for differ from where tools and funding go
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- A physical-activity coaching case (GPTCoach and the Bloom app): expert coding found over 93% of the coach's conversational strategies were consistent with motivational interviewing or neutral, though gaps against skilled human coaches remained.
- In a four-week randomised field study (N=54), the LLM condition produced a 5x increase in app usage time; mean physical activity stayed comparable to the control. The gains were in mindsets: belief that activity helps, enjoyment, self-compassion.
- Prompt chaining was enough for the coaching behaviour; the non-prescriptive style and a "coach" avatar mattered more to users than analysis of their health data.
- On work: a cited survey of 1,500 US workers (Shao et al. 2025) found the tasks workers want agents to handle differ substantially from current deployments and funding; the authors read this as a risk of favouring displacement over augmentation.
- A cited Harvard Business Review study of 200 employees at a US tech firm found "work slippage": LLM use expanded tasks and cognitive load rather than reducing hours. Teachers spent extra time validating generated outputs.
- Deskilling risk is greatest when tasks are automated wholesale and for novices, who rely on tools less critically; iterative use where the worker checks and corrects output may preserve skill.
- ChatGPT adoption correlates with higher education, higher socioeconomic status and urban zip codes, pointing to a "productivity divide".
- Benchmarks rarely reflect real work; GDPval and domain benchmarks in finance, law and medicine are named as better steps.
How they got it
A narrative literature survey across HCI, NLP and responsible-AI research, structured by pipeline stage. Students drafted sections in autumn 2024 under Diyi Yang; Ziems and Zhao rewrote and restructured the text through 2025-2026. The quantitative findings come from cited studies, chiefly the authors' own coaching work.
Limits
- No systematic search or quality grading; selection of studies reflects the authors' field.
- The future-of-work chapter relies on a handful of secondary studies, some of them practitioner articles.
- The coaching evidence comes from a single small trial of 54 people over four weeks.
Why it matters for companies
For a company deploying LLM tools, the useful message is to measure what changes in the work rather than task speed alone: quality of output, extra checking time and cognitive load. The "work slippage" and validation-time findings suggest that time savings measured in isolated task tests may not show up in a working week. The paper also argues that how a tool is framed and scoped (a defined role, clear refusals, non-prescriptive style) shapes adoption as much as model choice. Leaving room for staff to check and correct outputs, rather than full automation, is offered as a guard against eroding junior expertise.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Future of work