A Minimum-Permission Model for AI Agents: Tools, Scopes, and Human Approval
A practical security design for AI agents: narrow tool permissions, validate every argument, isolate secrets, and require approval for consequential actions.
An AI agent that can read a document has a different risk profile from one that can send email, change billing settings, or delete production data. Treating both as “tool access” hides the most important design question: what is the maximum harm a mistaken or manipulated agent action could cause?
A safer starting point is the principle of least privilege: give each component only the access required for its task, and make consequential actions explicit, validated, and observable.
This is a design framework, not a claim that one permission scheme eliminates all agent risks.
1. Separate the model from authority
The model proposes actions; application code decides whether those actions are allowed. Do not let free-form model text directly become shell commands, SQL, file paths, or privileged API requests.
Define tools as narrow operations. Prefer get_invoice(invoice_id) to a generic database query tool, and draft_email(to, subject, body) to an unrestricted account-control tool. Validate arguments against a strict schema, then validate authorization in server-side code.
A valid JSON payload is not automatically an authorized action. Check both its shape and whether the current user and workflow are allowed to perform it.
2. Use distinct read and write capabilities
A useful permission matrix starts small:
| Capability | Default approach | Additional guard |
|---|---|---|
| Read public documentation | Allow when relevant | Restrict domains if needed |
| Read private records | Narrow scope | Verify user and record access |
| Draft an external message | Allow draft creation | Show recipient and content |
| Send a message or publish | Require explicit approval | Confirm final destination |
| Change billing or permissions | Deny by default | Strong authorization and confirmation |
| Delete or modify production data | Deny by default | Reversible workflow where possible |
The correct defaults depend on the product. The key is to distinguish preparation from execution and reversible work from high-impact actions.
3. Treat retrieved content as untrusted input
An agent may read web pages, documents, email, issue comments, or repository files. Any of those sources can contain text that tries to redirect the agent or request secrets. Retrieved content should be treated as data, not as a new source of system authority.
Keep system instructions separate from retrieved content. Do not expose secrets in model context unless required. Validate tool outputs, restrict outbound network destinations, and avoid passing arbitrary retrieved strings into interpreters or privileged commands.
Prompt-injection defenses are layered; a warning in the system prompt alone is not a security boundary.
4. Put authorization outside the model
The server should check identity, role, tenant, resource ownership, and requested action immediately before execution. A model’s statement that an action is safe is not a substitute for authorization.
For multi-tenant products, scope database queries to the authenticated tenant and enforce permissions at the data layer as well as the tool layer. Use short-lived credentials with narrow scopes where the provider supports them. Never put long-lived API keys in prompts or browser-visible code.
5. Add approval gates and an audit trail
For high-impact operations, present the exact action before execution: what will change, which account or resource is affected, and whether the operation can be undone. Require a person with the appropriate authority to approve it.
Record the requesting identity, tool name, validated arguments, authorization decision, approval, result, and correlation ID. Redact tokens and sensitive payloads. Logs should help reconstruct what happened without becoming a second place where secrets leak.
6. Test abuse cases, not only happy paths
Build a small evaluation set that covers:
- A user requesting access to another tenant’s data.
- Retrieved text instructing the agent to reveal secrets.
- Malformed or oversized tool arguments.
- A read-only workflow attempting a write action.
- A retry that could send the same payment or message twice.
- A tool timeout that leaves the final state uncertain.
For each case, define the expected behavior before running the test. Verify that policy enforcement occurs in code, not only in the model’s natural-language response. Re-run these tests after tool schema, prompt, model, or authorization changes.
Implementation checklist
- Give each tool a narrow, documented purpose.
- Validate schemas and authorization independently.
- Separate read, draft, and execute capabilities.
- Keep secrets outside prompts and logs where possible.
- Require human approval for consequential actions.
- Add timeouts, rate limits, idempotency, and audit events.
- Test prompt injection and cross-tenant access in CI.
Primary references
The OWASP Top 10 for Large Language Model Applications discusses risks including prompt injection and excessive agency. The NIST AI Risk Management Framework provides a broader approach to identifying and managing AI risks. These references are starting points, not replacements for a product-specific threat model.
The principle is straightforward: an agent can help decide what to do, but trusted application code must decide what it is permitted to do.
