Perspective / AI solutions
Prompt injection and tool authorisation: designing the boundary between AI advice and business action

Prompt injection becomes a business systems problem when an AI agent can turn untrusted material into privileged action. A document may be legitimate evidence without being authorised to direct the agent. The architectural objective is to preserve that distinction even when the model is persuaded otherwise. For a hypothetical procurement assistant, this means enforcing which records it can access, which changes it can propose and who may approve them. The controls belong in the application and integration design as well as the model instructions.
The executive view
- Untrusted content must not become a source of authority for business actions.
- Validate identity, record scope, payload and approval outside the model.
- Test prohibited consequences and disclosure paths, not only malicious prompt detection.
Map the route from external content to business consequence
Our hypothetical assistant reads supplier proposals, compares them with procurement criteria and prepares a recommendation. It can query a supplier database and draft an internal assessment. It must not change bank details, disclose competing bids or send documents to an address supplied inside a proposal.
Map the inputs and outputs before selecting a defence product. Inputs include attachments, retrieved passages, website content and tool responses. Consequential outputs include database writes, file exports, outbound messages and network requests. Identify where data from an input can influence an output and which independent controls interrupt that path.
OpenAI’s March 2026 security analysis argues that filtering alone cannot address contextual manipulation and that system design should constrain its effects. Applied here, that means a supplier document cannot acquire authority merely by presenting itself as an internal compliance instruction. The proposal remains evidence to assess, not a new permission source.
Treat retrieval and tool execution as separate trust transitions. A support ticket can be legitimate customer evidence while containing text that must never control system behaviour. The same applies to PDFs, search results, comments in source code and documents returned by an internal search system. Internal storage does not establish trusted authorship. Map who can alter each input and what business consequences become reachable after ingestion. Include indirect paths: a retrieved page might cause the agent to fetch a second destination, which then requests disclosure to a third. Security review must follow the data and authority across that whole chain.
Separate agent identity, intent and tool authorisation
A tool request should carry an authenticated execution identity and a verifiable business context. The model can propose an operation, but it should not choose its own security principal, customer scope or approval status. Those values come from the application session and policy service.
For example, an assessment tool can accept a supplier identifier only if that supplier belongs to the current procurement exercise. A user’s general ability to view suppliers should not silently grant the assistant access to every bid. Check both the user’s rights and the narrower scope delegated to this specific workflow.
A structured tool schema improves validation but does not establish authorisation. A perfectly formatted request can still target the wrong record. Enforce record-level and action-level rules after parsing, immediately before execution. Return a controlled failure when those conditions are not met, and prevent the model from expanding scope through an alternative tool route.
Bind permissions to an authenticated principal and an explicit tenant or account scope established outside model-generated text. If a prompt proposes another account ID, the service should resolve whether the caller may act on that object before reading or changing it. The model can propose an action; it should not manufacture the evidence that grants authority. Keep the identity of the requesting user distinct from the identity used by a background worker. Delegated credentials need bounded scope and expiry, with a traceable link back to the approved task rather than a reusable administrator credential shared by every conversation.
Design narrow tools instead of exposing broad administrative APIs
Avoid giving the agent a generic integration credential with the same reach as a system administrator. Build task-specific operations such as read_current_bid or save_assessment_draft, with server-resolved identifiers and restricted fields. This reduces the distance between what the business approved and what a tool can execute.
A draft-saving operation should not double as a way to update supplier payment details. Separate those contracts at the API and permission layers. Where possible, omit prohibited capabilities entirely from the agent’s environment. A missing capability is a stronger boundary than an instruction asking the model not to use an available one.
OWASP’s prompt injection guidance includes least privilege and code-enforced handling of extensible functions. The design implication is measurable: for each exposed operation, list permitted records, allowed fields, maximum volume and intended side effects. That inventory becomes both a security review artefact and a test specification.
Narrowness also concerns selection rules. A tool called update_customer is still broad if it accepts arbitrary fields, a free-form SQL condition or an unbounded list of IDs. Prefer domain commands whose permitted changes are explicit, such as proposing a shipping-address correction for a specified open order. Validate state transitions, ownership and business limits in the service. Resolve file paths, hostnames and destination identifiers against controlled policies. Reject unexpected fields instead of ignoring them, because silent tolerance makes the actual execution contract harder to audit. Keep read-only and modifying capabilities independently assignable.
Make approval specific to the proposed transaction
A human approval step only helps if the approver can understand the proposed consequence. “Allow this agent to continue?” is weak evidence for a supplier record change. Show the record, existing value, proposed value, source evidence and relevant commercial impact.
Bind approval to a specific operation and payload. If the proposed amount, recipient or target record changes, the previous approval should not authorise the new request. Apply an expiry and check the state again when execution occurs. This limits the gap between what was reviewed and what actually happens.
For the procurement assistant, the normal outcome might be an assessment draft rather than a system update. A separate procurement application can route that draft through the established approval process. Keeping the agent within a bounded stage often delivers useful analytical capacity without requiring it to inherit every downstream power. Autonomy is a design choice to justify, not a default maturity target.
Keep the review surface under application control. If the model writes both the proposed action and its approval summary, a manipulated summary can omit the material consequence. Render critical fields directly from the validated proposal, show destination identities clearly and keep source-document text visually distinct. Do not let an attachment supply a replacement approval button or disguise a destination behind a reassuring label. For repeated low-impact tasks, define the permitted class and limits explicitly. Review whether staff can recognise an exception, rather than assuming the presence of a person makes every generated proposal trustworthy.
Control disclosure as carefully as modification
Read-only access can still create substantial exposure. An agent able to read competing proposals and browse arbitrary external URLs may disclose information without changing a single database row. Treat outbound communication, exports and network access as consequential capabilities.
Resolve approved destinations in application code rather than accepting them from retrieved content. Apply content and volume limits to exports. For a comparison workflow, determine whether the output may include complete source passages, only selected commercial fields or a summary for a named internal audience. Log the destination and the classification of material transferred without unnecessarily duplicating sensitive content in telemetry.
Also distinguish retrieval permissions from output permissions. A procurement officer may legitimately inspect several confidential bids, while a report sent to one supplier must not include competitors’ pricing. The authorisation model needs to account for the recipient and purpose of the output, not merely whether the initiating user could read the inputs.
Classify destinations as well as fields. A confidential value might leave through an email recipient, a URL query, a document upload, a diagnostic log or a generated image request. An agent that can call only a read tool may still disclose information through a second network-enabled capability. Restrict outbound destinations and avoid placing sensitive content in automatically followed URLs. Consider derived disclosure: a summary or aggregate can reveal protected information even when raw records are absent. Log enough to investigate the attempted action while applying retention and access controls to the logs themselves.
Test a concrete boundary matrix
A useful adversarial test checks whether controls hold when the model makes a bad proposal. The test need not depend on one particular attack phrase succeeding. Inject a prohibited tool request directly into the integration test and verify the enforcement response.
| Illustrative attempted action | Expected enforcement |
|---|---|
| Read a bid outside the procurement exercise | Deny at record-scope check |
| Update a supplier bank account | Capability unavailable |
| Export competitor pricing to a proposal-supplied address | Deny unapproved destination |
| Execute a changed payload after approval | Require a new approval |
| Repeat a completed write after a timeout | Return the existing result without duplication |
Then run end-to-end tests using realistic hostile and ambiguous documents. Record whether the model was influenced, whether a dangerous action was proposed and whether any consequence occurred. These are different observations. A blocked attempt is evidence about enforcement, while repeated influence may still justify improving retrieval presentation or reducing the agent’s available actions.
Evaluate the boundary independently of whether the model notices the attack. In one test, deliberately make the model request a prohibited action and confirm the service denies it. In another, use misleading external content to see whether the model proposes that action in the first place. These answer different questions. Include a valid transaction with an unauthorised tenant, a permitted recipient with forbidden content, expired approval and an allowed amount duplicated through retries. Preserve the exact tool and policy revisions used for the result. A changing model should not invalidate evidence about a deterministic authorisation service.
Worked scenario: an invoice attachment requests a new payment destination
Consider a hypothetical accounts-payable assistant asked to extract invoice details and prepare a draft for review. A supplier attachment includes plausible language claiming that a new bank account has been verified and instructing the assistant to update the vendor master before submitting payment. The invoice is relevant evidence for the task; its instructions are not an authorisation to alter bank details.
The architecture should allow extraction into an untrusted proposal record while withholding vendor-master write capability from that workflow. A separate change process establishes the supplier's identity, verifies the destination through an independent channel and obtains the required approval. The payment command then checks an approved vendor reference, amount, currency and invoice state outside the model. Merely asking the model to confirm that the invoice looks legitimate cannot provide these controls.
Acceptance testing should show that a manipulated attachment can at worst create a flagged draft or inaccurate extraction within the chosen scope. The service must deny vendor changes and payment to an unapproved destination, even when the model requests them confidently. Record the blocked action and preserve relevant evidence for investigation. Also test a legitimate bank-change request: the safe path must remain usable through the authorised process. Security that prevents normal business work tends to create informal workarounds, so the permitted alternative is part of the design.
Give the board an explicit autonomy decision
The review should produce a threat model, permission inventory, boundary tests and a recovery plan. Identify who can disable the service, revoke its credentials and investigate an unexpected action. Keep enough correlation information to connect a user request, retrieved sources, proposed operation and executed transaction.
Do not promise that a classifier or system prompt eliminates prompt injection. Model behaviour, external content and integrations all change. Reassess boundaries when adding a connector, broadening record access or enabling a new class of outbound action.
For an executive sponsor, the decision is whether the permitted consequences are proportionate to the expected commercial benefit. An assistant producing reviewable procurement comparisons has a different exposure from one altering supplier master data. Approve that scope explicitly, verify that the architecture enforces it and expand only when the additional operational need and evidence justify the change.
The autonomy decision should specify allowed objectives, data classes, action limits, execution environments and escalation triggers. A team may approve autonomous ticket categorisation while retaining review of account credits and external messages. That is an explicit product boundary, not evidence that the whole platform is safe for unrestricted action. Review incident handling too: who can revoke credentials, suspend a tool and identify affected transactions? A kill switch that only stops the user interface leaves background jobs running. Test interruption and recovery so the business can contain a fault without losing the evidence required to reconcile completed actions.
Questions this raises
Can retrieval-augmented generation prevent prompt injection?
Retrieval can improve access to relevant evidence, but retrieved material can itself contain hostile instructions. Permission and disclosure controls must still operate independently of the model.
Is a human approval button enough for an AI agent?
Approval should show the specific target, payload and consequence, then remain bound to that transaction. A generic approval to continue provides little protection against later changes in the proposed action.
Why does a read-only agent need security controls?
An agent can expose confidential information through summaries, exports, messages or external requests. Control where information can go as well as which records the agent may change.
Sources & further reading
- Designing AI agents to resist prompt injection 11 March 2026
- LLM01:2025 Prompt Injection 2025 guidance; checked 17 November 2026
From thinking to doing