Learning about Markets from Models: Evidence from an AI Hiring Platform
A · Strong evidence Harvard Business School economists (Alidaee, Khanna, Koning) analyse 172,242 applications and firm-specific retention models on a Mexican hiring platform with stated estimators, confidence intervals and recalibration checks.
Summary based on the full text·Original: about 70 minutes to read
In short
A paper (arXiv, 31 October 2026) by Hossein Alidaee, Tarun Khanna and Rembrand Koning of HBS. A hiring platform for large employers of low-skill workers in Mexico trains one model per firm to predict 90-day retention from a common psychometric test. Each model predicts well at its own firm and no better than chance at others. Comparing the models across firms still yields information: which applicants would stay longer elsewhere, and how differently firms value the same traits.
Key numbers
- 172,242job applications from three of the platform's largest clients, 2020-2025
- 0.855 to 0.914AUC of each firm's own retention model at its own firm
- 26.9 vs 16.4 percentage pointspredicted retention gain from redirecting an application to its best-fit firm, top vs bottom trait quintile
- 21.8 to 15.1 percentage pointsmean gain after equalising baseline retention across firms
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Firm models rank a stayer above a leaver at least 85% of the time at their own firm (random-forest AUCs 0.855, 0.891, 0.914), but are essentially useless at other firms.
- Scoring every application with every firm's model gives a firm-application matrix. Top-quintile applications gain 26.9 points of predicted retention from redirection, bottom-quintile 16.4.
- Recalibrating models to the same average retention removes gains due only to higher baseline retention: the top quintile loses 7 points of gain, the bottom 2.2.
- Mean redirection gap falls from 21.8 to 15.1 points after baseline equalisation.
- Firms disagree on whether important traits raise or lower retention at the margin, even in this low-skill market.
- The authors call this a "model comparison externality": value from a platform holding many firm models even when pooling data does not improve any one prediction.
How they got it
Platform data on assessments and tenure, firm-specific random forests chosen by nested cross-validation, cross-firm scoring, and a share-matching inversion borrowed from discrete-choice demand estimation to separate worker-firm complementarity from sorting on baseline retention.
Limits
- Three firms, one country, one psychometric instrument, one outcome (90-day tenure).
- Redirection gains are model predictions, not observed outcomes of reassigned workers.
- Applicants are people who passed each firm's existing hiring process, which limits what the models see.
Why it matters for companies
A retention or screening model trained on your own hires does not transfer to another employer, and a vendor's pooled model may not transfer to you. The comparison across firms is where the market insight sits: which traits competitors reward and where your hiring differs. Firms using such platforms should ask what cross-client comparisons the vendor runs and who gets the results.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org
- Tags
- Future of work, Hiring, HR