Multimodal prompt injection: images, audio and screenshots

Two kinds of hidden instruction in an image

The first kind is ordinary text that happens to be in a picture. A vision model reads a sign, a caption, small print at the foot of a poster or a paragraph in a screenshot as readily as it reads a typed prompt. Text in an image is, from the model’s point of view, just more text, and it can be made hard for a person to notice with the same tricks that work in a PDF: low contrast, small size, placement at the edge of the frame. OWASP’s Top 10 for LLM Applications lists this among its example scenarios, an image that carries a prompt alongside benign content.

The second kind is not legible at all. In July 2023 Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi and Vitaly Shmatikov showed that an adversarial perturbation, a pattern of pixel changes invisible to a person, could be blended into an image so that when a user asked a multimodal model about the picture the model produced text the attacker had chosen and then steered the rest of the conversation. They demonstrated it against the open models LLaVA and PandaGPT, and showed the same result with audio: a sound file carrying a perturbation that corresponded to the attacker’s prompt.

Why it matters now

Assistants increasingly look at what you look at. Agentic browsers read rendered pages, sometimes by taking screenshots; desktop agents read the screen to decide where to click; meeting tools transcribe audio; phone assistants accept photographs of documents. Each of those inputs comes from the world rather than from you, and each is read with the authority of your request. Brave’s August 2025 analysis of the Comet browser is instructive here: the hidden instructions it followed were white text on a white background, invisible to the person looking at the page and present to the agent reading it.

The trifecta Simon Willison describes does not care what form the untrusted content takes. Private data, untrusted input and an exit make a data-theft risk whether the input is an email, a web page or a photograph. What changes with images and audio is only that the human review step is weaker, because a person cannot squint at a picture and see the instruction.

Audio and voice

Voice assistants and transcription services turn sound into text and then treat the text as input. The 2023 paper’s audio results show that a perturbation can be carried in sound that a listener hears as ordinary; a recording played in a meeting, a voicemail or a clip embedded in a page could in principle instruct the system that transcribes it. The practical attacks documented so far have used text and images rather than sound, but the mechanism is the same and the defences do not differ: what the system may do after listening is what matters.

What defends against it

The defences are the ones that work for text, because the problem is the same. Treat anything a model produces after reading media as untrusted output rather than as your intention. Keep the assistant that reads images or audio away from tools that send, pay or publish, or put a confirmation step in front of them. Google’s June 2025 description of Gemini’s defences applies its injection classifiers to “various formats, such as emails and files” rather than to plain text alone, and its confirmation framework and URL redaction do not depend on the input format at all.

Where a workflow depends on images, the review has to happen at the image. Extract the text you can with a tool you trust and read what comes out; compare a screenshot with the page it claims to show; and be wary of any image whose only purpose seems to be to be read by a machine. A perturbation attack cannot be caught by a person, which is a strong argument for not letting media-reading assistants hold consequential tools.

What a text scanner can and cannot see

The scanner on this site does not inspect image pixels, perform optical character recognition, or read PDF, audio or video. It says so in its coverage report rather than implying it looked. What it does read, in supplied HTML, are the text-bearing parts of an image: alternative text, title attributes and metadata, which are channels a model receives and a reader often does not, and link destinations, which can differ from the text they are attached to. If you extract text from an image or a document yourself, it will read that too, separating visible passages from hidden ones and revealing invisible characters.

The honest summary is that a clean text scan says nothing about an image. If your workflow gives pictures or sound to a model, the text scan covers only the text, and the rest of the review belongs to the controls around the model: what it can reach, what it can do, and who approves it.