Monocultural Biases in LLM Hiring
A · Strong evidence Peer-reviewed COLM 2026 paper from Oxford, UCL and the Burning Glass Institute with a controlled correspondence-style experiment of 121,600 calls per model across ten base and post-trained model pairs.
Summary based on the full text·Original: about 35 minutes to read
In short
Matthew Bone (Oxford, Burning Glass Institute), Fabian Stephany (Oxford) and Maria del Rio-Chanona (UCL) test whether widely deployed LLMs share the same hiring biases, so that the same candidates are rejected everywhere. Comparing ten open models in base and post-trained form, they find post-training makes models better at picking candidates on skills and qualifications, and also makes them agree with each other far more. The result is a tripling of the share of candidates rejected by nearly all models, with older applicants hit hardest.
Key numbers
- 5.6% to 17.3%global systemic exclusion rate, base models versus post-trained models
- -3.6%callback likelihood for older applicants in post-trained models versus their base versions
- 20.1% vs 14.5%systemic exclusion rate for over-40 versus under-40 applicants
- 8 of 10models whose age bias moved against older applicants after post-training
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Post-trained models are 3.6% less likely than their base versions to call back older applicants; eight of ten models shifted against age, median shift -4.7%, largest Granite 4 (0.0% to -10.0%).
- Post-trained age bias ranges from -12.7% to -1.2% in callback rates across models.
- Race and gender shifts (+1.1% to +1.5% towards black and female applicants) are driven by outliers: Gemma 3 and Granite 4 for race, Nemotron Nano 3 for gender (+3.3% to +9.8%).
- Human capital controls explain 9.3% of callback variance in base models and 16.5% in post-trained ones; demographics explain 1.5% and 1.6%.
- Average pairwise correlation between models rises by 0.377 (residual, quality-driven) and 0.092 (bias-driven) after post-training.
- Global systemic exclusion (rejected by at least 9 of 10 models) rises from 5.6% to 17.3%.
- Exclusion across intersectional groups: 2.9% to 8.3% for base models, 12.2% to 21.7% for post-trained; over-40 average 20.1% versus 14.5% under 40.
- Olmo checkpoints point to supervised fine-tuning as the stage that adds the age bias.
How they got it
Synthetic vacancy and applicant pairs drawn from Burning Glass Institute data (over 600 million vacancies, nearly 100 million profiles) across 76 bachelor-level occupations. For each base prompt, one applicant's name and graduation year were varied across eight race, gender and age combinations, with 20 trials each in both orders: 121,600 calls per model. Models: Nemotron Nano 3, Granite 4, Gemma 3, Olmo 3, Qwen 3.5, GLM 4, Ernie 4.5, Moonlight, Ministral 3 and Falcon H1, run on one H100 at temperature 1.
Limits
- Only open models of 14-35B parameters; the frontier commercial models most employers use were not tested.
- Binary demographics (black/white, male/female, under 35/over 45) signalled by names and graduation years; other groups and signals are not covered.
- A two-candidate forced choice is a simplification of real screening pipelines.
Why it matters for companies
A bias audit of the one model a company uses misses the larger risk: if most employers screen with similar models, the same people get rejected everywhere. Age is the clearest pattern, and post-training, the step that makes models useful, is where it appears. Companies using LLMs to screen candidates should test for age effects specifically, keep human review at the screening stage, and ask vendors which base model and tuning data they use. The authors propose regulator-run audits of correlation across firms, which no single employer can do alone.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Hiring