Quant Arena: half of frontier models lose money on A-shares
大模型被A股上了一课！EinsiaAI发布Quant Arena，半数大模型亏本
B · Useful, limited Summary of an EinsiaAI (Navers Lab) arXiv benchmark with a sound out-of-sample design on 3,042 A-shares; the method is clear, but each model gets one frozen run on a single year, and the vendor places its own strategy at the top.
Summary based on the full text·Original: about 6 minutes to read
In short
A Jiqizhixin article (22 October 2026) on Quant Arena, a benchmark from EinsiaAI's Navers Lab (arXiv 2610.30113). Nine models from six vendors each ran a full quantitative research loop on Chinese A-shares: write Python strategies, backtest on 2022-2023, revise with at most five candidate evaluations, then freeze one strategy for a single test on sealed 2024 data. Five lost money and none beat the equal-weight benchmark. Better stock-ranking signals did not reliably turn into returns.
Key numbers
- 5 of 9frontier models whose frozen strategy lost money on sealed 2024 A-share data
- 10.7% vs 17.1%annual return of the best model (GPT-6 Astra) versus the equal-weight market benchmark
- 3,042A-share stocks in the dataset, with daily and Level-2 data
- 6.1%average daily overlap of top-75 picks across the nine strategies, despite 0.47 rank correlation
- 36.47 to 43.55average research-period score without and with feedback; out-of-sample returns improved for only 4 of 9 models
As reported by the source.
Chinese·Web page·opens jiqizhixin.com
What it reports
- Annual 2024 returns after costs: GPT-6 Astra 10.7%, GPT-6 Sol 8.6%, GPT-5.6 Terra 8.4%, Claude Opus 5.5 4.4%; Claude Sonnet 5.5 -12.2%, Gemini 3.8 Flash -2.9%, DeepSeek V4.1 Flash -5.7%, Kimi K3 -8.5%, GLM-5.3 -7.8%. Benchmark 17.1%.
- Risk: GPT-6 Astra max drawdown -6.8%, GPT-6 Sol -2.8% (Sharpe 1.89); Opus 5.5 -32.7%, Kimi K3 -36.4%.
- Signal quality and returns diverge: Gemini 3.8 Flash RankIC 0.045 and -2.9% return; Opus 5.5 RankIC -0.002 and +4.4%. In 7 of 9 strategies the top decile underperformed the sample mean by about 10 bp a day.
- GPT-6 Astra wrote its own market-timing switch, stayed in cash through the February 2024 crash (drawdown -1.7% versus -29.6% for the market) but captured only 16% of the rebound.
- Feedback helps in-sample (average score 36.47 to 43.55) but improved out-of-sample returns for only 4 of 9 models. Gains came mostly from fixing obvious early errors.
- Strategies looked alike in broad ranking (0.47 average correlation) but shared only 6.1% of daily top-75 picks. Most models built small-cap, value, low-volatility portfolios.
- EinsiaAI's own strategy Kepler-1 scores 77.56 with a 2024 end value of 1.93, versus 51.29 and 1.11 for GPT-6 Astra. The arena is open to outside submissions.
How they got it
Models received the same data access and feedback rules; 2024 data and results never flowed back. The evaluator builds a 75-stock equal-weight long portfolio in round lots, with transaction costs and market impact charged to a cash account. The leaderboard combines return, risk, signal quality, turnover and feature diversity. An ablation compared single-shot generation with feedback-driven research.
Limits
- One frozen strategy per model, tested on one year of one market: results are a single draw, not a distribution.
- Long-only, equal-weight design limits what strategies can express.
- The benchmark is built by a firm that sells quant strategies and ranks its own model far ahead; that model's method is private.
- The digest is based on the media summary, not the full paper.
Why it matters for companies
The finding generalises beyond trading: agents improve fast on in-sample feedback, and much of that gain does not survive contact with sealed data. Any firm letting agents iterate on its own metrics should hold back a test set the agent never sees. The gap between ranking accuracy and returns is also a reminder that model outputs pass through position sizing, execution and costs before they create value.
Key passages in translation
Chinese original
得分最高的 GPT-6 Astra，虽然年化收益为 10.7%，但仍低于 17.1% 的市场基准（券池等权收益）。
The top scorer, GPT-6 Astra, had an annual return of 10.7%, still below the 17.1% market benchmark (equal-weight return of the stock pool).
Chinese original
即使模型尽力研究市场，能把行情预测得更准，却不一定能把这份准转成收益。
Even when a model researches the market hard and predicts it more accurately, it cannot necessarily turn that accuracy into returns.
Chinese original
反馈能让模型改掉明显错误，却很难让策略跑过市场基准。
Feedback lets models fix obvious mistakes, but rarely lets a strategy beat the market benchmark.
Summary based on the full text. Read the original for the full argument and data.Read the original on jiqizhixin.com