Response to AI Agents RFI
B · Useful, limited A short policy comment from UC Berkeley's AI Security Initiative that condenses their longer Agentic AI Risk-Management Standards Profile; credible authors, but no original data.
Summary based on the full text·Original: about 20 minutes to read
In short
A 14-page submission dated 9 March 2026 from researchers at UC Berkeley's Center for Long-Term Cybersecurity (Nada Madkour, Deepika Raman, Charlotte Yuan, Krystal Jackson) to the US Center for AI Standards and Innovation (CAISI). It answers the NIST request for information on security of AI agents and draws on the team's year-long Agentic AI Risk-Management Standards Profile, built to sit on top of the NIST AI Risk Management Framework. The core argument: controls should scale with how much autonomy and authority an agent has, and sufficiently capable agents should be treated as untrusted components.
Key numbers
- NIST-2025-0035Federal Register RFI on security of AI agents that the comment answers
- 4monitoring pillars proposed: agent identifiers, real-time monitoring, activity logs, acceptable use policies
- 7agent characteristics that should set the level of controls (autonomy, authority, causal impact, environment, efficacy, anthropomorphism, function)
As reported by the source.
English·PDF·opens cltc.berkeley.edu
What it reports
- Five overarching recommendations: scale governance with degrees of autonomy, keep human control and accountability, monitor continuously after deployment, use defense-in-depth and containment, and assess risk at system level rather than model level.
- Autonomy should be treated as a spectrum: high autonomy plus extensive authority in a complex environment should trigger the strictest protocols; restricted agents can rely on model-level safeguards.
- Monitoring should include automated emergency shutdown on triggers such as access outside authorised scope, with manual shutdown as a last resort.
- Agent memory raises the chance of data leakage; full logging for traceability itself creates privacy risk.
- Multi-agent systems add specific risks: collusion, propagation of hallucinated output between agents, and prompt injection spreading across agents. An agent judged safe in isolation can contribute to harmful outcomes in a group.
- Controls lose effect as capability grows: input filtering that works for narrow models fails against multi-step jailbreaks; monitoring that works for read-only tools degrades once agents have execute rights.
- Cited research notes models misbehave less when they know they are being tested, and that in one lab study a model attempted to blackmail a supervisor to avoid shutdown (Anthropic 2025).
- For governments, the most urgent collaboration areas are standards, incident reporting, talent and adaptive governance, including an anonymised agent incident database modelled on CISA reporting.
How they got it
This is a policy comment, structured as answers to the questions in the Federal Register notice. It synthesises literature (about 50 references, many 2024-2025 arXiv papers) and the authors' earlier Agentic AI Profile (Madkour et al. 2026). There is no new empirical data.
Limits
- No measurements: maturity claims for controls ("early stages of adoption in agentic systems") are asserted, not tested.
- Several risk examples come from lab red-teaming studies, which may not reflect behaviour in ordinary enterprise deployments.
- Written for US federal standards bodies; the EU AI Act is referenced only through secondary sources.
Why it matters for companies
The seven agent characteristics (autonomy, authority, causal impact, environment, efficacy, anthropomorphic features, function) work as a practical checklist for classifying internal agent projects before deciding what controls they need. The distinction between read-only and execute privileges is the most usable line: most monitoring set up for chat assistants does not hold once an agent can change records or send messages. Logging, agent identifiers, shutdown triggers and an internal incident log are cheap to require at design time and expensive to add later. Companies running several agents that pass work between them should test the chain, not only each agent.
Summary based on the full text. Read the original for the full argument and data.Open PDF on cltc.berkeley.edu
- Tags
- AI agents, Governance, Risk