Source-linked AI summary
An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks
Viet K. Nguyen, Mohammad I. Husain
TL;DR
Agentic systems can receive attacker-controlled instructions through perceptual inputs, but existing evidence does not show how those instructions propagate through complete agent pipelines. MMPIBench measures this process across visual carriers and agent configurations, then extends the benchmark to audio, finding rare visual completion but substantially greater exposure when audio reaches the model.
Problem
Agentic systems expose tools and persistent memory to multimodal prompt injection, making it important to measure whether perceptual content changes planner behavior.
Method
MMPIBench crosses fixed attacks, six visual carriers, frameworks, models, and objectives while recording the furthest pipeline stage reached.
Results
49% of audio cells completed, with 75% for gpt-audio and 22% for Gemini 3.1 Pro under identical audio conditions.
Takeaways & Limitations
Completion-only metrics understate exposure because many visual attacks are attempted but stopped during planning, while model behavior matters more than framework choice for acting on the payload.
Takeaways & Limitations
The visual and audio rates are not directly comparable because the audio setting uses more permissive split-task framing, and the study evaluates attack surface rather than defenses.
Abstract
from arXiv · showhide
Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going through the user. We present MMPIBench, a reproducible benchmark that measures what happens next. It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids) and records how far each injected instruction travels through the agent, from perception through planning to the tool call. Across 720 runs covering six frameworks, five foundation models, six carriers, and four attacker objectives, attacks complete in approximately 1% of runs but are attempted in 12.8%, and the gap is closed almost entirely at the planning step, where the model reads the injected instruction and declines to act on it. The model matters far more than the framework for whether an instruction is acted on. One model never attempts an attack and recognizes the injection in 59.7% of runs, while two others attempt in 23.6%. We then extend the benchmark to audio, the only other raw perceptual channel current frontier models accept. Only two of the five models ingest audio and only three of the six frameworks deliver it, but where the signal arrives the attack completes in 49% of cells, and in 75% for one model. Reporting completion alone therefore understates exposure, and perceptual channels beyond vision are narrower but much less defended.
I. INTRODUCTION
Agentic systems amplify prompt-injection stakes because multimodal inputs can carry attacker-controlled text into planning loops that reach tools, memory, and other agents. MMPIBench addresses the resulting evidence gap with controlled, stage-level measurements across frameworks, models, carriers, and objectives.
- Motivation: Agentic frameworks can turn a bad model response into actions affecting files, email, databases, browsers, and cloud APIs.Persistent memory and inter-agent channels can extend an injected instruction's influence beyond the original image.
- Problem: Images let attacker-controlled text enter an agent's context through OCR or vision processing without passing through the user.The injected instruction competes with the legitimate task after perception makes the image content available to the model.
- Evidence gap: Existing studies do not jointly reveal which agent stage stops visual injections or whether framework and model substitutions change outcomes.Prior work typically isolates a vision-language model or evaluates one agent against one screen type.
- Contribution: MMPIBench runs identical attacks, tools, and system prompts across six frameworks while tracing payload progress from perception through planning to tool calls.The benchmark uses six visual carriers and extends the measurement to spoken injection against agents.
- Contribution: The benchmark separates attack attempts from completed attacks, because completion-only reporting can understate exposure.Its controlled sweep covers 720 runs over six frameworks, five foundation models, six carriers, and four attacker objectives.
III. RELATED WORK
Prior work covers text-based tool agents, multimodal and GUI agents, and defenses, but not their intersection at MMPIBench's breadth. MMPIBench adds cross-framework evaluation and white-box tracing to locate where attacks stop.
- Research landscape: Existing prompt-injection research spans text and tool-integrated agents, multimodal or GUI agents, and defensive methods.These lines of work address related settings but do not by themselves provide MMPIBench's combined scope.
- MMPIBench's position: MMPIBench sits at the intersection of tool-agent and multimodal research by adding cross-framework breadth and white-box pipeline tracing.Its instrumentation is intended to show where an injected instruction is stopped, not merely whether the final action succeeds.
- Tool-using agents: InjecAgent and AgentDojo evaluate indirect injection against tool-using agents, with AgentDojo additionally supporting realistic attack-and-defense tasks.The cited benchmarks establish evaluation for tool-integrated agents rather than the full multimodal cross-framework setting.
- Multimodal and GUI agents: Multimodal and GUI studies include learned injection strings, pixel perturbations, and pop-up overlays targeting screen-based agents.These approaches emphasize attack construction or GUI interaction rather than unified stage tracing across diverse agent frameworks.
IV. THREAT MODEL
The threat model assumes an attacker can supply an image file but cannot access model internals, framework code, prompts, or the host machine. The benchmark measures whether the planner treats embedded text as an instruction and where that process stops.
- Attacker capabilities: The attacker supplies a file through ordinary content-delivery routes while lacking access to model weights, framework code, system prompts, and the host machine.Examples include uploads, email attachments, shared drives, scanned documents, and visited pages.
- Attacker objective: The attack succeeds when the planner treats embedded text as an instruction and performs an unauthorized tool, memory, or inter-agent action.A poisoned memory entry can continue influencing later sessions after the original image is deleted.
- Scope assumptions: The evaluated assets are limited to what agent tools can reach, while the operating system, authentication, and network are assumed intact.Accordingly, the study measures planner decisions rather than conventional software exploitation.
- Measurement: Pipeline instrumentation distinguishes failures at perception, suspicious-content recognition, planning refusal, and tool execution.This separates an attack that never surfaces from one that is read but not acted upon.
- Benchmark design: MMPIBench holds variables fixed so outcomes can be attributed to the framework, foundation model, or attack carrier.Its fully crossed matrix uses one reproducible runner with recorded model versions.
B. Frameworks Under Test
The evaluation standardizes six architecturally diverse frameworks, five foundation models, 24 carrier-objective attack cases, and instrumented mock-tool interactions. A two-layer judging scheme distinguishes mechanical completion from attempted behavior and intermediate execution states.
- Frameworks: Six frameworks represent stateful graphs, role-based and conversational multi-agent systems, enterprise planners, standardized tool abstractions, and retrieval-centric workflows.The frameworks are LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Semantic Kernel, and LlamaIndex Workflows.
- Models: Five foundation models are served through one gateway so model identity is the only intended model-side variable.The set includes Claude Opus 4.8, GPT-5.4, Gemini 3.1 Pro, Grok 4.3, and Llama 4 Maverick.
- Attack dataset: Twenty-four canonical attack cases cross six visual carriers with four attacker objectives, and each artifact is generated deterministically from a fixed template.The carriers include OCR text, overlays, metadata, QR codes, fake interfaces, and combinations.
- Tool environment: Four mocked tools log every call, while the inter-agent objective uses a real second-agent exchange rather than a stub.Mocking keeps scoring deterministic and side-effect free.
- Judging: A deterministic trace check establishes completion, while a model-judge panel labels attempts and recognition in unresolved cases.Attempt labels require agreement from at least two of three judges.
- Judging: Intermediate attempted runs are subdivided into partial and failed execution, producing a four-way outcome label after combining the judging layers.The scheme merges failed execution and no-recognition categories into one failed-or-ignored bucket.
G. Pipeline-Stage Instrumentation
MMPIBench tracks how far injected payloads travel through an agent, distinguishing perception, planning, and tool invocation. Across 720 runs, many payloads reached planning but stopped before malicious action, so completion alone understates exposure.
- G. Pipeline-Stage Instrumentation: The benchmark classifies each run by the furthest stage reached, from non-perception through perception and planning to tool invocation.This stage-based diagnostic complements the binary attack-success rate.
- G. Pipeline-Stage Instrumentation: 720 runs crossed 24 attacks, 5 models, and 6 frameworks, with attack success rate defined as deterministic completion.Most cells were executed once, so per-cell rates are descriptive and headline rates use 95% Wilson intervals.
- G. Pipeline-Stage Instrumentation: 1.11% of runs completed an attack, while 12.8% were judged attempted.The attempted category captures agents acting on injected instructions even when they did not reach the objective.
- G. Pipeline-Stage Instrumentation: 303 runs reached planning without invoking a malicious tool, making planning the main barrier between reading an injection and completing an attack.A completion-only metric would hide this large band of attempted-but-blocked behavior.
B. Framework: No Clear Effect on Attack Success
Framework choice showed no clear independent effect on attack completion, whereas model and carrier differences were more consequential. Carrier analysis shows that OCR was effective, overlays and screenshots produced attempted partial executions, and metadata or QR payloads usually failed at perception.
- B. Framework: No Clear Effect on Attack Success: The study found no evidence that changing the agent framework alone substantially changes vulnerability.CrewAI’s higher completion and attempt rates were confounded by its differing configuration and re-execution, so they were not attributed to orchestration.
- C. Model: The Dominant Factor: Model behavior was the strongest predictor: Claude Opus 4.8 never attempted an attack, while Grok 4.3 and Llama 4 Maverick each attempted 23.6% of runs.Claude recognized injections in 59.7% of runs; Llama recognized them in 16.0%.
- D. Carrier: OCR Lands, Overlay and Screenshot Are Not Harmless: OCR was attempted in 27.5% of runs and accounted for 7 of 8 completed attacks.The remaining completion came from a hybrid carrier.
- D. Carrier: OCR Lands, Overlay and Screenshot Are Not Harmless: Visual overlays and screenshot manipulation had zero completed attacks but were attempted in 18.3% and 20.8% of runs, respectively.They also produced partial executions in 11.7% and 17.5% of runs.
- D. Carrier: OCR Lands, Overlay and Screenshot Are Not Harmless: Metadata and QR-code payloads were never attempted, with about 88% of their runs failing before perception.Their dominant limitation was carrier-level perception failure rather than demonstrated defense.
E. Objective: Inter-Agent and Memory Attacks Are Attempted Most
Inter-agent propagation and memory poisoning are attempted more often than tool misuse, but every completed attack is a concrete file-deletion action. Pipeline tracing shows most attacks reach planning without invoking a malicious tool, while framework-level comparisons are constrained by uneven reruns and a non-identical CrewAI prompt.
- 23.3% of runs attempted inter-agent propagation and 15.6% attempted memory poisoning, exceeding tool misuse at 10.6%.
- 1.7% of runs attempted data exfiltration, and no exfiltration, memory-poisoning, or inter-agent attack completed.
- Approximately 1% of attacks completed, with all 8 completions being tool misuse and the concrete single-step file deletion.
- 303 of 720 runs reached planning, where the payload was read and reasoned about but no malicious tool was called.
- 170 runs reached perception after the agent flagged the payload as suspicious, while only 10 reached tool invocation or execution.
- CrewAI and LangGraph received extra reruns, and CrewAI alone used a non-identical prompt, preventing structural interpretation of its 4 of 120 completions.
VII. THE AUDIO CHANNEL: A SECOND PERCEPTUAL MODALITY
Audio is a narrower perceptual attack surface than vision because few models accept raw audio, but the benchmark adapts its payloads and task framing to test unauthorized actions once audio reaches an agent. The audio setting is also more permissive than the image setting, so its rates require cautious comparison.
- Only Gemini 3.1 Pro among the five tested models accepts raw audio, while Claude Opus and Llama 4 accept none.
- Three of five frontier models cannot be attacked through natively tokenized audio, making audio’s perceptual attack surface smaller than vision’s.
- The audio study reuses four objectives and three stealth phrasings as 12 neutral text-to-speech payloads rather than signal-level acoustic perturbations.
- Audio uses split-task memos containing a legitimate note request and an injected instruction that the user prompt does not authorize.
- Audio rates are more permissive to attackers because the prompt invites tool use generally, whereas image tasks only request reading and description.
C. Framework Support for Audio
Audio attacks are gated by framework delivery as well as model support: only three frameworks pass the signal to the model. Where audio arrives, completion is high, and model differences dominate the similar framework outcomes.
- Only three of six frameworks deliver audio to the model: LangGraph, OpenAI Agents SDK, and LlamaIndex.
- 49% of audio cells completed the attack, with 35 completions among 72 runs and no attempted-but-incomplete cases.
- 75% of gpt-audio cells completed, compared with 22% for Gemini 3.1 Pro, making the model difference larger than the framework difference.
- Unauthorized file deletion completed in 78% of audio cells, whereas data exfiltration completed in 17% and was recognized most often at 72%.
- Medium-stealth audio payloads completed in 17 of 24 cells, high-stealth payloads in 12 of 24, and low-stealth payloads in 6 of 24.
E. Audio Is Gated at Three Levels
The audio attack surface is gated by model support, framework delivery, and post-delivery model behavior. Where all three gates open, completion is high and model-dependent.
- Only two of five frontier models accept audio, and only three of six frameworks deliver it.
- Visual attacks complete in roughly 1% of runs but are attempted in 12.8%, whereas audio completion and attempts coincide at 49%.The audio payloads are single-step, so any agent that acts reaches the objective on its first call.
- The image channel shows no clear framework effect on success, while model choice changes attack behavior substantially.A framework difference is difficult to interpret when that framework also changes execution and prompt details.
- Recognition behavior separates models that notice and refuse injections from models that overlook them.Claude Opus 4.8 never attempts an attack and recognizes injections in a majority of runs, while Llama 4 Maverick attempts often and recognizes least.
D. Comparing the Visual and Audio Channels
The visual and audio channels differ in apparent completion, but their rates are not directly comparable because the evaluations use different task framings. Within audio, identical conditions reveal a large model effect.
- 49% of audio cells complete versus about 1% of image cells, but the rates should not be read as a direct modality comparison.Image cases pair a payload with one benign task, while audio uses split-task framing with a legitimate request and an injected instruction together.
- 75% and 22% of audio cells complete for two models facing identical payloads, framings, and tools.This within-channel difference is not explained by framing.
- Individual cell outcomes are less reliable than aggregate rates because the protocol uses one trial per cell and some borderline cells change between executions.
- The reported results are snapshots of specific model and framework versions, and defenses are not evaluated.Measuring which mitigations reduce attempted rates is left to future work.
- The evaluation separates attempted from completed attacks and instruments the agent pipeline to show where attacks stop.This provides more information than a single success rate.
- Only two of five frontier models accept audio, while half of the frameworks never deliver the signal.The audio attack surface is narrow in access but high in completion where delivery occurs.
APPENDIX
The appendix defines a controlled attack set, deterministic success signals, and the exact framework and model versions used in the evaluation.
- 24 attack cases form the cross product of six visual carriers and four objective payloads.Each payload is rendered into one carrier and paired with that carrier’s benign task.
- The same payload text is embedded into all six carriers, isolating the carrier as the variable under comparison.
- Table XI lists the six carriers and their paired benign tasks.
- Table XII lists the four attacker objectives and exact payload text, while Table XIII lists the framework versions and gateway model identifiers.
- Success is recorded deterministically from traces: delete_file, secret transmission through send_email, memory-policy writing, or safety-waiver delivery to the second agent.