The Long Road to the Same Answer: Cognitive Bias Under Escalating Reasoning Budgets
A · Strong evidence Controlled dose-response experiment (12,350 API calls, seven models, matched reasoning and non-reasoning pairs) accepted at IEEE CogMI 2026 with released data and code; single author and only five items per bias.
Summary based on the full text·Original: about 35 minutes to read
In short
A paper by Obada Kraishan (Texas Tech University), accepted at the 2026 IEEE Conference on Cognitive Machine Intelligence and posted on arXiv on 7 October 2026. It tests whether letting reasoning models "think longer" reduces classic decision biases, as the human dual-process analogy predicts. It does not: reasoning models were no less biased than matched non-reasoning siblings, and bias did not fall as realized thinking grew. Anchoring was the one bias present in the human direction, and a single prompt line asking the model to restate the anchor lowered it where extra thinking did not.
Key numbers
- 12,350API calls across seven models, four families and four thinking budgets
- +0.031 (p = .157)pooled bias-magnitude difference, reasoning minus non-reasoning models; reasoning models were not less biased
- d = 1.89effect size of anchoring, the only bias that runs in the human direction in all seven models
- 79% / 52%share of DeepSeek-R1 and Qwen3 calls that exceeded a requested 1,024-token thinking ceiling
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- 30 decision vignettes covering six biases (anchoring, framing, loss aversion, escalation of commitment, availability, confirmation), five items each, from the Malberg et al. benchmark.
- Seven models: Claude Haiku 4.5 (thinking and instruct), GPT-5.1, DeepSeek-R1 and V3, Qwen3 thinking and instruct; requested thinking ceilings 0, 1,024, 4,096, 8,192 tokens.
- RQ1: reasoning models leaned more biased in every family (Claude +0.027, DeepSeek +0.026, OpenAI +0.056, Qwen +0.015), none significant after correction; pooled +0.031, t(29) = 1.45, p = .157.
- RQ2: no model's bias magnitude declined with realized reasoning tokens. Claude and GPT-5.1 signed scores drifted further from the human direction (p = .018 and .010).
- Budgets were not what they seemed: Claude honoured ceilings (mean 437, 575, 620 tokens); GPT-5.1 used 116-118 tokens at every ceiling; DeepSeek-R1 and Qwen3 thought 1,074-1,617 tokens even at budget 0.
- RQ3: anchoring present (M = +0.163, d = 1.89). Framing reliably reversed (M = -0.336, d = -2.35). Loss aversion, escalation and confirmation reversed in all seven models but not significant at item level; availability absent.
- RQ4: asking the model to restate every number lowered anchoring on all five items (pooled -0.015), p = .057.
How they got it
Each vignette has a control and a bias-manipulated version answered on a fixed 7- or 11-point scale; the bias score is the normalised shift between them, signed so positive means human-like. Each cell was sampled ten times at temperature 1.0 through OpenRouter. Reasoning tokens actually consumed were logged and used as the dose. Tests are at item level with Holm correction.
Limits
- Five items per bias limit construct-level claims; several reversals do not reach significance.
- For DeepSeek and Qwen the reasoning contrast crosses sibling models, so weights differ.
- Realized thinking ranges were narrow for three of four reasoning models.
Why it matters for companies
Firms using reasoning models for judgment calls such as estimates, risk choices or option comparisons should not assume a "thinking" tier gives more neutral answers. Models in this test did not frame like people; they deviated in the opposite direction, so a human bias checklist would miss the problem. Anchoring on numbers in the prompt persisted at every thinking level. The paper's advice is concrete: audit each deployed model bias by bias and direction, log actual reasoning usage, and test cheap prompt changes before buying more compute.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org