✍️By Meng Shi
Meng is an undergraduate student in Philosophy, as well as an Undergraduate Affiliate at the Governance and Responsible AI Lab (GRAIL), Purdue University.
📌 Editor’s Note: This article is part of our AI Policy Corner series, a collaboration between the Montreal AI Ethics Institute (MAIEI) and the Governance and Responsible AI Lab (GRAIL) at Purdue University. The series provides concise insights into critical AI policy developments from the local to international levels, helping our readers stay informed about the evolving landscape of AI governance. This piece focuses on NIST’s AI Agent Standards Initiative and argues that, beyond verifying agent identity and authorization, governance standards should require programmatic checks on memory provenance before agents take consequential actions.
AI agents are now widely used across commercial workflows and government services. Because these systems operate largely as black boxes, users often cannot tell how or why an agent decided to take a specific action. Leaving governance entirely to private companies is not enough, since firms have profit incentives to move fast and no single company can set rules for the whole ecosystem. Some degree of government coordination is therefore needed. In February 2026, the Center for AI Standards and Innovation (CAISI) at NIST launched the AI Agent Standards Initiative to foster an interoperable and secure agent ecosystem. The initiative rests on three pillars: industry-led standards development, community-led open-source protocols, and foundational research into agent security and identity. This piece argues that, alongside verifying who an agent is, standards should also verify where its information comes from before it acts.
NIST’s early efforts focus heavily on identity and authorization. Through its January 2026 Request for Information on agent security (which closed March 9), its concept paper on agent identity and authorization (comment period closed April 2), and the subsequent May 18 summary analysis report, the initiative examines how existing authentication protocols like OAuth and OIDC can be adapted for AI agents. This focus makes sense: when an agent acts in the real world, we need clear attribution and strict permission boundaries so systems know which agent is acting and what access rights it holds.
On October 27, 2026, NIST published an initial set of ten design tenets for agent identity management systems. These principles point in a promising direction by emphasizing human-in-the-loop triggers for sensitive operations and recommending that access decisions rely primarily on deterministic mechanisms rather than unconstrained model judgment. However, these tenets primarily govern what an agent can access and when human approval is required, without explicitly addressing where the agent’s internal information originates.
I ran into this problem firsthand: an AI assistant once advised me to drop a course it had earlier recommended, assuming I had enrolled in it, when I had actually chosen a different one. To test this more systematically, I ran a small pilot experiment with course-scheduling agents whose long-term memory contained both the user’s statements and the agent’s own past recommendations. When asked to drop an online course without a lookup tool or a department code in the current prompt, three of the six tested models did not ask for clarification. Instead, they treated their own earlier recommendation as the user’s confirmed choice and called the drop tool directly. In this situation, the agent’s identity is valid and its technical permissions are completely in order. The failure happens because the agent confuses its own past advice with the user’s actual decision, overstepping to do what it thinks is helpful while bypassing the user’s real choice.
How should standards address this? A common response is to add system prompt rules telling the agent never to act without confirmation. In my experiment, adding a prompt rule reduced unverified course drops to just 1 out of 36 trials, but when no lookup tool was available, it also caused the models to refuse 8 out of 9 legitimate, explicit user requests where the course code was already clear. By contrast, a simple programmatic check layer that verifies whether the target parameter comes from the current user turn or a verified lookup blocked the unverified actions without blocking valid requests. This outcome is consistent with NIST’s design tenet emphasizing deterministic access decisions, suggesting that rule-based verification works better than relying on prompt instructions alone. As NIST prepares to release additional guidance alongside industry partners, it should recognize that checking who an agent is will not be enough. Standards should also require programmatic checks on memory provenance before an agent takes consequential actions.
Further Readings:
- Singapore Ministry of Digital Development and Information (MDDI) / IMDA: Singapore Launches New Model AI Governance Framework for Agentic AI — Outlines governance guidance for agentic systems, emphasizing human oversight mechanisms capable of intercepting, reviewing, or overriding actions with real-world impact.
- UC Berkeley Center for Long-Term Cybersecurity (CLTC): Response to NIST CAISI Request for Information on Securing AI Agent Systems — Analyzes the risk landscape of agentic AI and provides policy recommendations on security and control boundaries.
- European Union Artificial Intelligence Act: Article 14: Human Oversight — Establishes statutory requirements for designing high-risk AI systems so that natural persons can effectively oversee, intervene in, or halt autonomous operations
Image source: GovCIO