Prompt injection incident response: what to do when you find one

Two situations, two urgencies

The first situation is a near miss: you scanned a document, an email or a page before handing it to an AI system and found an instruction aimed at the machine. Nothing has acted on it yet. The response is calm and mostly about evidence and reporting. The second is an incident: an assistant or agent has already read the content and done something, sent a message, opened a link, changed a file, published data. Here the order is containment first and everything else after.

Both benefit from thinking in Simon Willison’s three ingredients: what private data could the system reach, what untrusted content did it read, and what channel could it use to send something out. The answers tell you what to contain and what to assume.

Contain

Stop the pipeline or the agent. Pause any automation that reads the affected source, revoke or narrow the credentials the system was holding, and turn off automatic actions until a person is back in the loop. If the content sits in a retrieval store, a mailbox index or a shared drive an assistant searches, remove or quarantine it there too; a poisoned document in an index keeps working for every future query until it is taken out, which is what made the EchoLeak email and the Slack AI channel dangerous long after they were written.

Block the sender, domain or contributor where that is possible, and note that the same content may exist in more than one place: a forwarded email, a cached page, a mirrored repository.

Preserve the evidence without spreading it

Keep the original bytes: the file, the raw email, the page’s HTML, the issue text. Copy them somewhere the assistants in your organisation do not read. Do not paste the payload into an AI tool to ask what it does, and do not forward it to colleagues in a way that lands in their assistants’ inboxes; an injected instruction is live wherever a model can read it. This site’s scanner keeps text in your browser and makes no network request while it runs, so it is safe to use on the evidence, and its text or JSON report records the channel, location and rule for each finding, which is the record you want.

Record hashes of the originals, the time the content arrived and the time it was first read by a system, and which systems could have read it in between. That window is the scope of the incident.

Trace what the assistant did

Read the tool-call logs if you have them, which is why Invariant Labs recommends continuous monitoring of agent-to-tool interactions: what did the system read, fetch, render, send or write, and in what order. Check the assistant’s output for images and links whose URLs carry data in their query strings, the classic exfiltration marker. Check sent mail, posted comments, opened pull requests and changed records for the period since the content arrived. If the system has no logs, assume it did everything it was able to do.

Assume exfiltration where an exit existed

If the system had private data and any outbound channel, treat the data as exposed. Rotate API keys, session tokens and passwords that were reachable, and treat accounts protected by one-time codes the assistant could read as compromised until re-secured. Notify the people whose data was in scope. Depending on where you are and what the data was, a legal obligation to notify a regulator or affected individuals may apply; that is a question for counsel, and the evidence you preserved is what they will ask for.

Report responsibly

Report to the vendor of the assistant or agent through its security programme, with a minimised reproduction rather than the live payload; every vendor named on this site has fixed reported injections, and several of the fixes described in these guides came from exactly such reports. Report to the operator of the place where the content lived, whether a repository owner, a forum or a document host. If this site’s scanner missed an indicator it should have caught, or raised one it should not have, use the security-report procedure on the contact page: name the route, the expected and observed behaviour, and send no source text, working payload or credentials.

Close the gap that made it costly

Ask which of the three ingredients turned a piece of text into an incident, and fix that one structurally. If the system read untrusted content with privileged tools, drop its privileges while it reads. If it could send data out, put a person or a policy in front of that channel. If the content reached a store nobody reviews, add review at ingestion. OWASP’s mitigations and the NCSC’s guidance both point the same way: constrain actions rather than hoping to recognise every instruction.

Then update the routine. Add the document to the examples your team scans against, brief the people who paste text into tools on what the hidden instruction looked like, and record the incident so the next one is recognised faster. Detection improves by exactly this loop, and so does judgement.