Pretraining data filtering in open-weight biological models
A · Strong evidence UK AI Security Institute experiment that pretrains ESM-2 protein models from scratch at several scales with defined filtering tiers and recovery tests; clear method, though one model family and imprecise cost measurement.
Summary based on the full text·Original: about 7 minutes to read
In short
A UK AI Security Institute blog post (26 October 2026) testing whether removing viral proteins from pretraining data is a durable safeguard for open-weight protein models. Filtering worked as intended: viral-protein performance dropped while performance on other proteins held. But continued training on the withheld viral data, which is routinely public, restored the capability fully at low cost. Filtering raises the price of recovery; it does not remove the capability.
Key numbers
- 8M to 3Bparameter range of the ESM-2 protein models pretrained from scratch with viral proteins filtered out
- about $80H100 compute needed to fully restore viral-protein performance in the largest 3B model by continued training
- 3tasks used to measure the targeted capability: masked language modelling, mutation-effect and contact prediction
- 150M / 650Mmodel sizes above which contact prediction and mutation-effect prediction were degraded by filtering
As reported by the source.
English·Web page·opens aisi.gov.uk
What it reports
- Filtering reduced performance on viral proteins at every exclusion tier, from removing a single influenza strain to removing all viral proteins and close homologues, with non-viral performance largely preserved.
- Masked language modelling degraded at every model size; contact prediction only above 150M parameters; mutation-effect prediction only above 650M.
- The effect grew with model size, because filtering withholds gains that unfiltered models make as they scale.
- Fine-tuning on the excluded viral sequences completely restored performance at every size. Cost rose with scale but stayed small: about $80 of H100 compute for the 3B model.
How they got it
ESM-2 models from 8M to 3B parameters were pretrained from scratch on datasets with nested exclusion tiers of viral proteins. Each filtered model was compared with an unfiltered one of the same size on held-out viral ("forget") and non-viral ("retain") proteins across three tasks, then fine-tuned on the withheld data while recovery compute was recorded.
Limits
- One model family (ESM-2); generalisation to other protein models is uncertain.
- Models that retrieve data at inference time were not tested and may be less affected by filtering.
- Recovery-cost measurements are described by the authors as imprecise.
- Results are summarised in charts; exact performance deltas are not given in the text.
Why it matters for companies
Developers that cite pretraining filtering to justify open release now have evidence that, for small specialised models, the safeguard is cheap to reverse when the removed data is public. The general lesson for anyone releasing open weights in a sensitive domain: a safeguard is only as strong as the cost and availability of the data needed to undo it, so access to that data matters as much as the model itself.
Summary based on the full text. Read the original for the full argument and data.Read the original on aisi.gov.uk