Open-weight models trail closed models in cyber capabilities by 6-7 months
A · Strong evidence Epoch AI data insight with a documented index method, downloadable scores and benchmark CSV, confidence intervals, and an explicit list of limits.
Summary based on the full text·Original: about 6 minutes to read
In short
An Epoch AI data insight (Jaeho Lee and Alexander Barry, 16 October 2026) using its Cyber ECI, a cybersecurity version of the Epoch Capabilities Index. The best open-weight model, Z.ai's GLM-5.3, scores 160.1: above GPT-5.4 (March 2026) and below Claude Mythos Preview (April 2026). The open cyber frontier therefore sits where the closed frontier was 6-7 months ago.
Key numbers
- 160.10Cyber ECI of GLM-5.3, the top open-weight model
- 6-7 monthslag of the open-weight cyber frontier behind closed models
- 171Cyber ECI of the closed frontier, Claude Opus 5.5 and GPT-6 Astra
- 76 models, 35 benchmarkscoverage after the October 2026 refit (from 56 and 20)
As reported by the source.
English·Web page·opens epoch.ai
What it reports
- GLM-5.3 160.10 (90% interval 155.83 to 166.97); GPT-5.4 157.04; Mythos Preview (April) 164.82.
- Closed frontier: Claude Opus 5.5 171.14 and GPT-6 Astra 171.03.
- Data as of 8 October 2026: 76 models, 35 cyber benchmarks, 324 model-benchmark results from UK AISI, CAISI, Irregular, developer reports and other sources; 57 models with at least two cyber results are shown.
- The refit lowered several top scores: Claude Mythos 5 by 3.9 points and GPT-5.6 Sol by 4.7. Earlier published values are not comparable.
How they got it
Benchmark parameters are taken from a joint fit of general and cyber benchmarks; each model's Cyber ECI is then estimated from its cyber results alone, on the general ECI scale. The lag is read from the release dates of the closed models whose scores bracket GLM-5.3.
Limits
- Few results per model: GLM-5.3 rests on three CAISI benchmarks, Opus 5.5 on four reported by Anthropic. Opus 5.5 and Astra share no benchmarks.
- CAISI did not name US models; Epoch attributed results to the models it judged most likely.
- Comparisons use central estimates; uncertainty in differences is not yet modelled.
Why it matters for companies
Cyber capabilities that only closed labs had in spring 2026 are now available in open weights that anyone can run without usage controls. Security teams should plan on attackers having that level of automated capability within months of the closed frontier. The figures are thin, so treat the lag as an order of magnitude, not a precise date.
Summary based on the full text. Read the original for the full argument and data.Read the original on epoch.ai