An update on independent evaluators
B · Useful, limited Anthropic's own announcement of embedded third-party evaluators; a concrete governance arrangement with stated terms, but self-described and with no evaluator findings yet.
Summary based on the full text·Original: about 6 minutes to read
In short
An Anthropic announcement (15 October 2026) that it has given a team from METR, a nonprofit model evaluator, "employee-like access" to internal tools and systems. METR will publish its own risk assessments; Anthropic cannot veto publication but can require redaction of predefined categories. SecureBio, Accenture and Faculty work under narrower terms, and talks with others, including Hugging Face, are early.
Key numbers
- sixMETR staff onboarded with company hardware, office space and internal tool access
- threetiers of evaluator access Anthropic is developing
As reported by the source.
English·Web page·opens anthropic.com
What it reports
- Six METR staff onboarded with hardware, office space and access roughly on par with Anthropic's own risk teams.
- Excluded categories: customer data, attorney-client privileged material, model architecture details. Anthropic decides what falls into them; METR may comment on the access it received.
- Other METR staff are siloed from the embedded team under enforceable agreements.
- Three access tiers: tier 1 narrow investigations published with Anthropic approval; tier 2 publication without approval, with redaction disputes made public; tier 3 the METR arrangement.
- SecureBio is reviewing the chemical and biological weapons chapter of the August 2026 Risk Report after a published pilot review.
- The post compares the model to resident bank examiners. It follows a signed White House accord in which companies pledged to work with external evaluators.
How they got it
A policy announcement describing contract terms. No evaluation results are reported.
Limits
- Self-reported by the company being evaluated; the agreements themselves are not published.
- Anthropic keeps final say on what counts as a redactable category.
- No findings yet, so the value of the access cannot be judged.
- The post was updated twice in two days to clarify METR and SecureBio terms.
Why it matters for companies
Buyers of frontier models increasingly need evidence of safety practice for their own risk and procurement reviews. Published reports from embedded evaluators, once they arrive, could serve as third-party assurance in vendor due diligence. The tiered model is also a template for any firm that wants outside review of sensitive systems without giving up all control over disclosure.
Summary based on the full text. Read the original for the full argument and data.Read the original on anthropic.com
- Tags
- Governance