Recursive Language Models
A · Strong evidence MIT CSAIL preprint by Zhang, Kraska and Khattab (the DSPy author) proposing a general long-context inference method, with benchmark tables, costs and ablations reported against several baselines.
Summary based on the full text·Original: about 40 minutes to read
In short
A December 2025 arXiv preprint from MIT CSAIL. Instead of feeding a long prompt into the model, a Recursive Language Model stores it as a variable in a Python REPL. The model writes code to inspect and split the prompt, and calls itself (or a smaller model) on the pieces. This handles inputs far beyond the context window and, on information-dense tasks, beats the base model even when the input fits, at similar median cost.
Key numbers
- two orders of magnitudehow far beyond the model's context window RLMs handled inputs, reaching the 10M+ token scale
- 91.33RLM(GPT-5) score on BrowseComp-Plus with 6M-11M token inputs, where base GPT-5 cannot fit the input
- $0.99average RLM(GPT-5) cost per query on that task, vs $1.50-$2.75 for GPT-5-mini to ingest the full input
- <0.1% to 58.00%F1 on the information-dense OOLONG-Pairs task, base GPT-5 vs RLM(GPT-5)
- 28.4%RLM(GPT-5) gain over base GPT-5 on OOLONG, a task that fits within the context window
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- RLMs handle inputs up to two orders of magnitude beyond context windows, into the 10M+ token range.
- BrowseComp-Plus (6M-11M tokens): RLM(GPT-5) scores 91.33 at an average $0.99 per query, beating summarisation and retrieval baselines by over 29%.
- OOLONG (131K tokens, within the window): RLM gains of 28.4% (GPT-5) and 33.3% (Qwen3-Coder) over the base model.
- OOLONG-Pairs: base models score F1 under 0.1%; RLMs reach 58.00% (GPT-5) and 23.11% (Qwen3-Coder).
- Ablation: the REPL alone lets the model exceed its window; recursive sub-calls add 10%-59% on information-dense tasks.
- Median costs are comparable to or below baselines, but tail costs rise sharply on long trajectories.
How they got it
Tested with GPT-5 (GPT-5-mini for sub-calls) and Qwen3-Coder-480B across four tasks of rising information density (S-NIAH, BrowseComp-Plus, OOLONG, OOLONG-Pairs, plus CodeQA), against direct calls, context compaction, a retrieval agent (CodeAct with BM25) and a summary agent. API costs are reported with standard deviations.
Limits
- Recursion depth of one only; synchronous sub-calls make runs slow.
- Some benchmarks are small (for example 50 S-NIAH tasks), and OOLONG-Pairs is the authors' own modification.
- Current models make inefficient decisions about their context; results may change with models trained for this use.
- Preprint, not peer reviewed.
Why it matters for companies
Many enterprise tasks involve reading a whole corpus rather than finding one fact: contract sets, codebases, policy libraries, audit trails. The paper shows that how a system feeds context to a model can matter more than the size of its context window, and that retrieval-only designs fail on dense aggregation tasks. Teams building document or code agents should test REPL-style decomposition against their current RAG setup and watch tail cost.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org