Zhipu's GLM-Legal tops legal benchmarks
智谱新模型，懂法方面全球第一了
C · Weak signal QbitAI launch coverage of Zhipu's open-weight legal model, relaying the vendor's demos and Vals AI leaderboard results; it adds context on the benchmark but no independent testing.
Summary based on the full text·Original: about 6 minutes to read
In short
A QbitAI (量子位) article (17 October 2026) on GLM-Legal, an open-weight legal model from Zhipu built on GLM-5.3-Flash. On Vals AI's latest legal evaluations it ranked first among open-weight models on Harvey LAB, with 19.2% of tasks passing every criterion, and on LegalBench, with 86.23% accuracy. The article itself notes that a 19.2% full-pass rate means most tasks still fail under strict scoring.
Key numbers
- 19.2%Harvey LAB tasks where GLM-Legal met every scoring criterion (top open-weight model on Vals AI)
- 86.23%GLM-Legal accuracy on LegalBench (top open-weight model on Vals AI)
- ~70xGemini 4 Argon's cost per test relative to GLM-Legal, at similar scores
- 1,200+simulated law-firm tasks in the first version of Harvey LAB, across 24 practice areas
As reported by the source.
Chinese·Web page·opens qbitai.com
What it reports
- Training: mid-training on legal corpora on top of GLM-5.3-Flash, then reinforcement learning across tens of thousands of legal task environments, with the agent framework iterated alongside.
- Covers Chinese legal practice, English legal tasks and South-East Asian cross-border compliance.
- Vendor demos against the base model: contract review (shorten termination notice from 60 to 30 days, add up to 90 days of transition support), evidence analysis (a project reported as 60% complete externally against about 41% in internal records), and a privacy-compliance checklist for launching in Singapore, Malaysia and Indonesia.
- Harvey LAB uses all-criteria pass scoring; its first version has more than 1,200 simulated law-firm tasks across 24 practice areas.
- At similar scores, GLM-Legal was nearly 30 minutes slower than Gemini 4 Argon, but Gemini cost about 70 times more per test.
- Zhipu is inviting law firms, legal-tech companies and in-house teams to a beta and plans to integrate the model into AutoClaw.
- Context: Harvey and Fireworks post-trained Tenet on Kimi K3 in August; Harvey says it completed close to twice as many LAB held-out tasks as its base model.
How they got it
Leaderboard results from Vals AI as reported by the article, plus Zhipu's official demo comparisons. No independent tests by the publication.
Limits
- Launch coverage in an enthusiastic register; demos are vendor-selected.
- The Vals AI leaderboard itself was not reviewed for this digest.
- The cost and time comparison with Gemini 4 Argon is given without units or setup.
Why it matters for companies
For in-house legal and compliance teams the number to remember is the low one: under strict scoring, even the best open-weight legal model fully completes about one task in five. The cost gap against a closed model with similar scores makes self-hosting a specialised model a realistic option for confidential legal work, provided outputs are reviewed by lawyers. The training recipe (domain mid-training, then RL in simulated work environments) is spreading across vertical models.
Key passages in translation
Chinese original
GLM-Legal分别以19.2%的任务全通过率和86.23%的准确率，摘得Harvey LAB与LegalBench双榜开放权重模型榜榜首。
With a 19.2% full task pass rate and 86.23% accuracy, GLM-Legal took first place among open-weight models on both the Harvey LAB and LegalBench leaderboards.
Chinese original
19.2%的全通过率也意味着它在严格评分条件下，多数任务仍未实现全部达标
A 19.2% full-pass rate also means that under strict scoring, most tasks still do not meet every criterion.
Chinese original
在成绩相近的情况下，GLM-Legal比Gemini 4 Argon慢了近30分钟、但Gemini的每项测试成本约为GLM-Legal的70倍
At similar scores, GLM-Legal was nearly 30 minutes slower than Gemini 4 Argon, but Gemini's cost per test was about 70 times GLM-Legal's.
Summary based on the full text. Read the original for the full argument and data.Read the original on qbitai.com