Where Do LLMs Learn to Recommend AI?
B · Useful, limited Bar-Ilan University researchers including Sarit Kraus run controlled fine-tuning ablations across many models, a sound design, but the outcome is a narrow keyword measure of AI mentions and each training branch has a single run.
Summary based on the full text·Original: about 30 minutes to read
In short
A short paper (arXiv, 21 October 2026) by Benaya Trabelsi, Jonathan Shaki and Sarit Kraus of Bar-Ilan University. Asked what to study, which career to pursue or what product to build, language models often recommend AI-related options. The authors trace where this comes from. The tilt starts in pretraining, roughly doubles in released instruction-tuned models, and depends on what post-training data contains and how the question is worded. It is not an inevitable result of instruction tuning.
Key numbers
- 27%responses from pretrained base models that include explicit AI/ML terms in their recommendations
- 59%same measure for the released instruction-tuned versions of those models
- 14 percentage pointsdrop in AI mentions after removing the 7% of instruction examples that concern AI
- 8 and 18 pointsrise in AI mentions when the question asks for the 'best' option or what to do 'right now'
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- Pretrained models include AI/ML terms in 27% of recommendations; their released instruction-tuned versions in 59% (ten Base-Instruct pairs).
- In the authors' own fine-tuning, general instruction tuning leaves the rate unchanged. Removing the 7% of instruction examples about AI cuts it by 14 points, below the pretrained starting level.
- Preference data rewarding helpfulness raises the rate by 9 points; data ranked by diverse simulated users lowers it by 9.
- With models fixed, asking for the "best" option adds 8 points and "right now" adds 18. Telling the model it is an AI advisor adds 4, with no significant extra effect from calling the assistant itself AI.
How they got it
Twenty open-ended advice questions in four domains, ten paraphrases each. Responses were searched for explicit AI/ML terms, with two LLM filters keeping only mentions inside the actual recommendation. Ten released Base-Instruct pairs were compared; eleven pretrained models were trained with six post-training recipes and compared with their parent checkpoints. System messages and question wording were varied for ten instruction-tuned models.
Limits
- The measure counts explicit AI terms, not whether the advice is justified; it has 86.6% recall against a human expert, so rates are conservative.
- Training recipes are compared whole, with one run per branch per model.
- Results may differ for other models, languages and domains; inference resamples questions, not models.
Why it matters for companies
Career-guidance, learning and internal-mobility tools built on general models will nudge people toward AI-related roles and courses more than the underlying facts may justify, and urgency or superlative wording in prompts strengthens the effect. HR and L&D teams deploying such tools should test recommendations for this tilt across role families, word their own prompts neutrally, and avoid presenting model advice as an objective view of the labour market.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org