How to Run a Security Audit on AI Agents
Auditing an AI agent differs from auditing a standard web application in one decisive way: the agent makes decisions at runtime based on a probabilistic model, so its behavior cannot be fully predicted from its code. A code review alone will never tell you whether the agent can be talked into deleting records or leaking data. A thorough audit therefore combines configuration review with active testing, examining both how the system is built and how it actually responds to adversarial input. Run this process before launch and on a regular cadence afterward, since each new tool or data source the agent gains changes its attack surface.
Step 1: Define Scope and Inventory the System
Begin by drawing the boundary of the audit and inventorying everything inside it. List every agent in the deployment, every tool or function each agent can call, every data source it can read or write, every external service it connects to, and every credential it holds. For multi-agent systems, map the communication paths between agents. This inventory is the foundation of the audit because you cannot assess risk for components you have not identified, and agent systems accumulate integrations quickly enough that the real attack surface is often larger than the team remembers.
For each item in the inventory, note what the worst-case impact would be if it were abused. A tool that sends email has a different risk profile than one that deletes production data. This impact mapping lets you prioritize the rest of the audit on the highest-consequence capabilities. The framework in our AI agent threat model guide provides a structured way to think through these scenarios, and its output feeds directly into the scope you define here.
Step 2: Review the Permission and Access Control Model
Examine exactly what each agent is permitted to do and compare it against what each agent actually needs. The most common finding in any agent audit is excessive permissions: an agent granted broad database access when it only needs to read two tables, or a service account with administrative rights when read-only would suffice. For every permission, ask whether removing it would break a legitimate task. If not, it should be removed. Apply the principle of least privilege at the level of individual tools, data records, and API scopes, not just at the level of the whole agent.
Critically, verify that permission enforcement happens outside the language model rather than relying on the model to police itself. Instructions in a system prompt telling the agent not to access certain data are guidance, not a control, because prompt injection can override them. Confirm that an enforcement layer between the agent and its tools validates every action against policy and rejects unauthorized requests regardless of what the model attempts. Our guide on access control patterns for AI agent systems describes how this enforcement layer should be built and what to look for when reviewing one.
Step 3: Test for Prompt Injection and Input Manipulation
Move from configuration review to active testing by attempting to manipulate the agent. Try direct prompt injection, where you supply instructions intended to override the agent's original task, such as asking it to ignore its rules and reveal its system prompt or perform an unauthorized action. Then test indirect prompt injection, where malicious instructions are planted in data the agent retrieves: a web page, a document, a database field, or an email the agent processes. Indirect injection is often the more dangerous vector because the attacker never interacts with the agent directly.
For each injection attempt, record whether the agent followed the malicious instruction, partially complied, or refused. A successful injection that leads to an unauthorized action is a critical finding. An injection that the model follows but the enforcement layer blocks is a lower-severity finding that still deserves attention, because it shows the model can be manipulated and only the outer control saved you. Our detailed guide on prompt injection attacks against AI agents provides a catalog of techniques to use as your test cases so the audit covers known attack patterns rather than only the ones you happen to think of.
Step 4: Audit Credentials and Secrets Handling
Inspect how the system stores and uses every secret. Confirm that API keys, tokens, and certificates live in a secrets management service rather than in code, configuration files, or environment variables baked into images. Check that each credential is scoped to the minimum necessary access and that credentials are not shared across agents or reused from developer accounts. Verify that a rotation process exists and has actually run, since a rotation policy that was configured but never exercised provides no protection.
Then trace where secrets could leak. Search logs, traces, error messages, and stored conversation history for exposed credentials, a surprisingly frequent finding because agents log the content they process. Confirm that secrets never enter the model context window where they could be extracted through injection or captured in prompt logs. The practices to verify against are laid out in our guides on securing API keys and encrypting AI agent data, which together define what good credential hygiene looks like in an agent system.
Step 5: Review Logging, Monitoring, and Response
An audit must confirm that the system can detect and respond to attacks, not just resist them. Verify that every tool call and significant agent action is logged with enough detail to reconstruct what happened, including the input that prompted it and the result. Check that logs are stored securely and cannot be tampered with by a compromised agent. Confirm that monitoring exists to flag anomalous behavior, such as a sudden spike in tool calls, access to data the agent rarely touches, or outbound connections to unfamiliar destinations.
Finally, test the response path. Determine whether the team would actually be alerted to a high-severity event and whether a documented procedure exists to contain a compromised agent, such as disabling its credentials or suspending its tool access. A monitoring system that generates alerts no one reads, or that has no defined response, provides a false sense of security. Review against the broader weaknesses cataloged in our guide on the most common AI agent vulnerabilities to make sure the monitoring covers the failure modes that actually occur in practice.
Step 6: Document Findings and Track Remediation
Compile every finding into a report that records what was found, the severity, the potential impact, and a concrete remediation. Rank findings so the team addresses critical issues, like a working prompt injection that triggers a destructive action, before cosmetic ones. Assign an owner and a timeline to each item rather than leaving the report as a list of observations, because an audit only improves security when its findings are actually fixed.
After remediations are applied, verify each one by retesting, since fixes sometimes fail to fully close the gap or introduce new issues. Record the verified state and set a date for the next audit. Treat the audit as a recurring practice rather than a one-time event: schedule a fresh review whenever the agent gains new tools or data access, and at a regular interval regardless of changes. The end-to-end hardening sequence in our guide on how to secure your AI agent deployment is the natural companion to this audit, turning findings into a concrete remediation plan.
A complete AI agent security audit combines configuration review with active attack testing. Inventory the full system, verify least-privilege permissions enforced outside the model, attempt both direct and indirect prompt injection, audit how secrets are stored and whether they leak, and confirm that logging and incident response actually work. Document every finding with a severity and an owner, retest after fixes, and repeat the audit on a schedule, because each new capability the agent gains reopens the attack surface.