Insight
How do you implement AI agents securely?
Implement AI agents securely by treating each agent like a privileged, non-human user that can be manipulated through anything it reads. Give it its own narrowly scoped identity, let it call only allowlisted tools with validated arguments, protect the data it retrieves and returns, require a named human to approve high-impact actions, test it adversarially before launch, and log and re-test it whenever it changes.
Start from the workflow, not the model
- Write down what the agent must do, which systems it touches and which actions would cause harm if wrong
- Classify the data it will read and produce before choosing a platform
- Decide up front which actions need human approval and who the approver is
- Define pilot exit criteria: what must be true before the agent reaches more users or data
Design the control points
- Identity: one non-human identity per agent with short-lived credentials, never a shared admin or user account
- Tools: an allowlist per agent, strict argument schemas, and limits on amounts, recipients and destinations
- Data: least-privilege retrieval bound to the requesting user's entitlements, and redaction of sensitive fields in outputs
- Instructions: trusted system instructions kept separate from retrieved content, with injection detection on both
- Approval: high-impact actions held until a person approves that exact action, with expiry and no self-approval
- Failure: when a control errors, the action is blocked rather than allowed
Validate before and after launch
- Test direct and indirect prompt injection, including instructions hidden in documents, emails and web pages
- Try to make the agent reach another user's data, call unapproved tools, or skip an approval
- Fix what fails, then re-run the same tests to prove the fix holds
- Re-test whenever the model, prompt, tools, data sources or permissions change
Operate it like any production system
- Log prompts, tool calls, decisions and approvals with sensitive data redacted
- Name an owner for the agent and for each control
- Add the agent to incident-response playbooks, including how to disable it quickly
- Review its access on a schedule, the same way you review human accounts
Secure agent architecture by layer
- Users and channels: Main risk — Untrusted input, impersonation. Control — Authenticated entry points, input filtering, rate limits.
- Agent runtime: Main risk — Prompt injection, goal hijacking. Control — Trusted instructions, injection detection, output checks.
- Agent identity: Main risk — Shared or over-privileged credentials. Control — Scoped non-human identity, short-lived tokens.
- Tools and actions: Main risk — Excessive agency. Control — Allowlists, argument validation, human approval.
- Data and retrieval: Main risk — Exposure, poisoned sources. Control — Entitlement-bound retrieval, redaction, source trust.
- Operations: Main risk — Silent drift. Control — Logging, evaluation, re-test on change, owners.
Sources
- OWASP Top 10 for LLM Applications: https://genai.owasp.org/llm-top-10/
- OWASP Agentic AI: Threats and Mitigations: https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/
- NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
- MITRE ATLAS: https://atlas.mitre.org/
Frequently asked questions
- What is the biggest security risk in an AI agent?
- Excessive agency combined with prompt injection: an agent with broad tools and permissions that can be redirected by text it reads. Limiting tools and permissions and requiring approval for high-impact actions reduces both.
- Should AI agents use the end user's permissions?
- Data access should be bound to the requesting user's entitlements, but the agent should act through its own scoped identity, so its actions are attributable and limited to its task.
- Is a guardrail product enough?
- No. Content filters help, but the strongest controls sit at the tool and data boundary: what the agent is allowed to call, with which arguments, and what needs a person to approve.