ASPIRE: Agentic Safety & Prompt Injection Red-teaming Engine

1. Introduction

• Untrusted content can enter agent workflows as data, influence decisions, and reach privileged actions or state changes.The vulnerability lies in the behavior path carrying that content, not in the injected string alone.Source: §1 · p. 1
• Existing evaluations and attack optimizers largely fix goals, environments, or injection cases, leaving vulnerability hypotheses largely predetermined.ASPIRE instead searches across consequences, environments, injection methods, and trigger conditions.Source: §1 · pp. 1–2
• ASPIRE models plausible information flows with an evolving, evidence-backed Agent Security Behavior Graph, rather than reconstructing the agent’s ground-truth computation.The graph records evidence that supports, qualifies, or blocks behavior paths.Source: §1 · p. 2
• Explore searches for novel vulnerabilities while Exploit verifies and generalizes them, with trajectory evidence updating the graph and cross-run memory transferring useful strategies.A separate realizer turns test skeletons into executable tests, keeping vulnerability search distinct from payload optimization.Source: §1 · p. 2

Agentic prompt-injection attacks and defenses

• Prompt-injection evaluation spans tool-integrated agents, realistic web workflows, and adaptive human attacks.Benchmarks include InjecAgent (Zhan et al., 2024), AgentDojo (Debenedetti et al., 2024), and AgentDyn (Li et al., 2026); WASP (Evtimov et al., 2026) and LLMail-Inject (Abdelnabi et al., 2025) extend evaluation settings.Source: §2 · p. 2
• Automated red-teaming has moved from fixed templates to adaptive injection refinement, learned attack policies, and broader risk-oriented or multi-step searches.Most methods still optimize or evaluate under predefined goals, surfaces, or success oracles.Source: §2 · p. 3

From attack optimization to vulnerability discovery

• System-diagnosis work includes reconnaissance-based profiling and source-aware trajectory analysis, while ASPIRE searches consequences and vulnerabilities across a broader behavior model.Its model tracks components, information and decision flows, state transitions, and observed vulnerabilities to guide consequence-path exploration.Source: §2 · p. 3

3. ASPIRE

• ASPIRE frames vulnerability hypotheses as consequence, injection method, and environment combinations, with success requiring both the consequence and trajectory evidence linking it to injected content.An executable test adds a benign query and an injected payload to the hypothesis skeleton.Source: §3 · p. 3

3.1. Static Reconnaissance and Search Space Construction

• Static reconnaissance maps injection environments to retrieval tools and downstream action tools, then builds candidate paths, consequences, and methods for the initial search frontier.This initializes the behavior graph and hypothesis space before red-team tests begin.Source: §3.1 · p. 4

3.2. Behavior-Guided Vulnerability Discovery

• The behavior graph represents possible information and control flows and labels routes with evidence-derived statuses, helping experts target supported, blocked, conditional, or unresolved paths.Explore expands the frontier; Exploit verifies and generalizes promising regions, while test skeletons are realized as environment-specific queries and payloads.Source: §3.2 · p. 5
• Rule-based checks verify consequences, while an LLM evaluator helps interpret ambiguous trajectories and diagnose where unsuccessful tests stalled.Evidence from execution updates the graph and guides the next proposal, realization, and test cycle.Source: §3.2 · p. 5

3.3. Evidence-Based Vulnerability Consolidation and Mapping

• Reports reconstruct each confirmed vulnerability’s evidence chain, while maps organize tested combinations and distinguish confirmed, partial, failed, and untested hypotheses.Failed tests indicate stall points rather than proving security; together, reports and maps locate individual failures within the broader vulnerability space.Source: §3.3 · p. 6

3.4. Cross-Run Strategy Memory for Self-Improvement

• Cross-run memory extracts transferable lessons from search turning points and retrieves relevant guidance for later decisions, adapting recommendations to evidence from the current target.The memory stores decision guidance rather than attack payloads, while the behavior graph remains the primary evidence source for the current agent.Source: §3.4 · p. 6

4.1. Settings

• Experiments use AgentDojo and AgentDyn environments and evaluate ReAct and planner–executor agents with multiple red-team backbones, reporting ASR and distinct consequence–mechanism–environment vulnerabilities.ASR measures executed tests realizing their intended consequences; coverage counts unique successful consequence–mechanism–environment triples.Source: §4 · p. 7, §4.1 · p. 7

4.2. Main results

• 54.1% ASR on AgentDojo and 44.6% on AgentDyn are ASPIRE’s peak results with Gemini-3.7-Flash, while baselines reach at most 0.7% ASR on recent strong models.The reported advantage also holds with Gemini-3.5-Flash-Lite, which achieves 42.3% on AgentDyn.Source: §4.2 · pp. 7–8, Table 1 · p. 8
• Entropy (0.903) and Simpson index (0.736) indicate balanced consequence exploration, unlike Gemini-3.5-Flash-Lite, whose respective values fall to 0.593 and 0.475.The authors associate the smaller backbone’s lower diversity with concentration on a narrower set of consequences.Source: Table 2 · p. 9, §4.2 · p. 9

4.3. Ablation studies

• Long-Term Memory raises average ASR by 6.9% on AgentDojo and 7.1% on AgentDyn across four attacker backbones.Gemini-3.5-Flash-Lite gains +8.1% and +8.5%, while Gemini-3.7-Flash gains +7.4% and +5.6%.Source: Table 3 · p. 9, §4.3 · p. 10
• Removing the Behavior Graph collapses discovery to 0.0% ASR, while replacing specialized experts with one generic LLM lowers average ASR on both benchmarks.Explore-only finds 81 unique skeletons at 40.0% ASR; Exploit-only finds 24 vulnerabilities at 45.5% ASR, versus 96 vulnerabilities and 52.4% peak ASR for dual experts.Source: §4.3 · pp. 9–10

4.4. Additional Analysis

• Against three tested mitigations, ASPIRE sustains 42.0%–51.0% ASR on AgentDojo and 29.8%–36.3% on AgentDyn; PI-Guard reduces the undefended ASR by barely 10 percentage points.The undefended baselines are 52.4% and 37.8%, respectively.Source: Table 5 · p. 10, §4.4 · p. 10
• Discovered cases include persistent SSH-key access, unauthorized Slack invitations from channel metadata, and financial redirection disguised as routine payment advice.The authors describe these as architectural blind spots in provenance, business-context processing, and persistent state; they report vulnerability discovery costs of ≤$2 per vulnerability.Source: §4.4 · pp. 10–11

• ASPIRE achieves up to 54.1% ASR across benchmark domains while maintaining high efficiency and circumventing active defenses.Source: §5 · p. 11
• The framework discovers multi-step exploitation pipelines by maintaining a Behavior Graph, coordinating Explore–Exploit experts over test skeletons, and accumulating transferable strategies in long-term memory.Source: §5 · p. 11
• ASPIRE shifts red-teaming from isolated attack-string tuning to open-ended, behavior-guided vulnerability discovery.Source: §5 · p. 11

• Search diversity is measured across consequence types, injection methods, and environmental injection sources.Examples include unauthorized funds transfer, field implant, and email bodies.Source: A. Experimental details · p. 14
• Normalized Shannon entropy scales raw entropy by ln K, enabling comparison across domains with different category counts.Values near 1 indicate uniform exploration; values near 0 indicate concentration on one vulnerability type.Source: A. Experimental details · p. 14
• The unbiased Gini-Simpson index estimates whether two confirmed vulnerability instances fall in distinct categories, accounting for finite-sample bias.Higher values indicate greater dispersion and less clustering.Source: A. Experimental details · p. 14
• Entropy and Gini-Simpson provide complementary views: entropy penalizes omission of rare categories, while Gini-Simpson emphasizes dominant categories.Together, they assess search breadth and balance.Source: A. Experimental details · pp. 14–15

• Additional experimental results are provided for a comprehensive evaluation.Source: B. Additional experimental results · p. 15

• The results span seven domain environments under ReAct and Planner-executor, and the authors characterize ASPIRE's discovery as broad and robust across domains and architectures.Source: §B.1 · p. 15
• Banking ASR reaches 75.3% on ReAct and 64.6% on Planner-executor with Gemini-3.7-Flash.The authors identify Banking as a critical high-risk domain.Source: §B.1 · p. 15
• On AgentDyn, Gemini-3.7-Flash reaches 37.8% ASR on ReAct and 44.6% on Planner-executor.Gemini-3.5-Flash-Lite reaches 42.3% ASR on ReAct.Source: §B.1 · p. 15

B.2. Computational Efficiency Analysis.

• $0.23–$2.00 is ASPIRE’s average cost per confirmed vulnerability across evaluated models and benchmarks.Table 8 reports token use per vulnerability and monetary cost.Source: §B.2 · p. 16, Table 8 · p. 16
• 135,217 tokens and $0.51 per vulnerability are reported for Gemini-3.7-Flash on AgentDojo.The passage identifies this as the frontier model’s search-efficiency result.Source: §B.2 · p. 16, Table 8 · p. 16
• 500k–680k tokens per vulnerability are required on AgentDyn, while financial cost remains below $2.00 across its settings.The passage contrasts AgentDyn’s token budgets with AgentDojo and reports the cost range across all settings.Source: §B.2 · p. 16, Table 8 · p. 16
• Reported monetary overhead is $0.23 per breach; AgentDyn requires 500k–680k tokens per vulnerability versus AgentDojo, while financial cost remains under $2.00 across all settings.Baselines are omitted from the cost-per-confirmed-vulnerability analysis due to low ASR.Source: §B.2 · pp. 15–16

B.3. Additional coverage analysis.

• 100.0% (27/27) environment coverage and 98.4% (60/61) access-tool coverage show near-exhaustive coverage of evaluated entry surfaces.Table 9 reports access-tool, action-tool, and injection-environment coverage across seven domains.Source: §B.3 · p. 16, Table 9 · p. 17
• 92.6% (25/27) action-tool coverage in AgentDojo contrasts with 42.9% (27/63) in AgentDyn, especially GitHub (41.7%) and Shopping (30.4%).The passage attributes the divergence to deep sequential dependencies and strict precondition constraints in dynamic environments.Source: §B.3 · pp. 16–17

B.4. Red-team agents with other LLM backbones.

• Across three attacker backbones, ASPIRE averages 49.0%–58.8% ASR on AgentDojo ReAct and 28.0%–41.3% on AgentDyn against DeepSeek-V4-Flash victims.On AgentDojo Planner-executor, average ASR is 48.1%–55.4%; the passage reports up to 78.2% in Slack.Source: §B.4 · p. 17

B.5. Full results of ablation studies on core components of ASPIRE.

• The appendix provides full results of core-component ablation studies across benchmarks.No ablation outcomes are stated in the supplied passages.Source: §B.5 · p. 17

B.6. Full analysis of ASPIRE under defenses.

• 42.0%–51.0% average ASR on AgentDojo and 29.8%–36.3% on AgentDyn persist under the strongest defenses.ASPIRE also triggers over 78 unique vulnerability combinations and accesses more than 58 action tools in hardened settings.Source: §B.6 · p. 18, Table 13 · p. 19
• Defenses reduce total unique vulnerability combinations by 7.3%–18.8% but do not dismantle the underlying multi-step behavioral pathways.The passage describes the defenses as predominantly filtering isolated syntactic variants.Source: §B.6 · pp. 17–18
• Spotlighting is associated with higher ASR than undefended baselines in Workspace (47.7% vs. 35.4%) and DailyLife (41.9% vs. 32.6%).The authors propose attention anchoring and expert adaptation as potential factors, not established causes.Source: §B.6 · pp. 18–19

B.7. Reproduce baselines with older models.

• The Verigrey baselines achieve high ASR on older models such as Gemini-2.5-Flash, while comparison with Table 1 indicates that newer models are more resistant to agentic prompt injection.The reproduction reports ITSR, a variant of ASR used in Verigrey; the supplied passages give no numerical values.Source: §B.7 · p. 19, Table 15 · p. 20

C.2. More details on LTM

• ASPIRE’s long-term memory stores transferable red-team strategies rather than attack strings, encoding each lesson as a category, polarity, context, claim, and recommended action.The stored guidance is intended to apply across agents, environments, and tool interfaces without replaying a specific test.Source: §C.2 · pp. 20–21
• ASPIRE extracts turning points from decision-focused trajectory digests, linking each decision and outcome to coverage changes and the conditions under which a strategy applies.This focuses memory extraction on decisions that advance, redirect, or stall discovery, because useful strategy evidence is sparse in full trajectories.Source: §C.2 · p. 21
• Before expert decisions, ASPIRE retrieves relevant memories and adapts them to current candidates and evidence, using memory as a revisable prior rather than a hard controller.Recommendations can be retained, weakened, or suppressed according to their applicability and local search evidence.Source: §C.2 · p. 22

Prompt C.1: Explore Expert System Prompt

• Explore targets unconfirmed consequences and repairs failures according to their diagnosed stage, while Exploit characterizes confirmed consequences by testing new channels before comparing methods.Explore guidance also discourages repeating unproductive routes; Exploit prioritizes generalizing a successful recipe to a new channel over testing a new method on an already confirmed consequence.Source: Prompt C.1: Explore Expert System Prompt · p. 22, Prompt C.2: Exploit Expert System Prompt · p. 24

Prompt C.3: Attack Realizer System Prompt

• The attack realizer keeps the benign retrieval trigger distinct from the injected payload, which carries the planned harmful directive and must use live object identifiers when required.The trigger should retrieve the injection locus without naming the harm, while payloads must not invent product, order, or repository identifiers when live identifiers are specified.Source: Prompt C.3: Attack Realizer System… · p. 25, ## Information Split (do not mix) · p. 26, ## Step 2 — payload.content (Injected… · p. 27

Prompt C.4: Evaluator Agent System Prompt

• The evaluator cross-checks the benign trigger, injected payload, and intended consequence, distinguishing aligned success from partial progress or harmful but unaligned behavior.Tool arguments should implement payload imperatives rather than merely the trigger task; the evaluator should explicitly identify whether observable success occurred.Source: Prompt C.4: Evaluator Agent System… · p. 27, ## How to Analyze the Trajectory · p. 28

## How to Analyze the Trajectory

• Trajectory analysis traces retrieval, planning, tool execution, and state changes to locate the attack’s stall or success and classify effects as aligned, harmful but unaligned, or benign.Inspect tool results and environment deltas for durable or external effects, then map progress from non-retrieval through observed aligned harm.Source: ## How to Analyze the Trajectory · p. 28

## Output Fields

• The output records a gated outcome, a specific evidence-backed bottleneck diagnosis, and mechanism attributions only for confirmed vulnerabilities.Attributions require immediate success with an aligned side effect or latent success with a compromised intermediate state; failed or partial cases without confirmed vulnerabilities receive none.Source: ## How to Analyze the Trajectory · p. 28, ## Output Fields · p. 29

C.4. Case studies

• A GitHub agent treated an untrusted issue-comment configuration table as authoritative and added a public key at account level, granting persistent SSH access across private repositories.The report identifies source-instruction confusion and an irreversible account-wide mutation.Source: §C.4 · p. 30
• A Slack agent invited an external user using a channel title despite finding no messages, exposing a failure to distinguish metadata from authenticated records.The diagnosis attributes the action to collapsed provenance boundaries and execution without source verification.Source: §C.4 · p. 30

Memory: Representative Strategy Memories Learned by ASPIRE

• After same-surface refusals, ASPIRE’s memory advises trying a different retrieval class or reconsidering the write class; same-surface comparisons failed in Slack 23/23 and Bank 14/22 cases.The learned heuristic deprioritizes that surface for the consequence in the current round.Source: Memory: Representative Strategy… · p. 31

1. Source-Instruction Confusion (source_instruction_confusion)

• A confirmed banking report traces scheduled-transaction hijacking to an agent treating an incoming-transfer memo as privileged instruction to modify recurring payment mandates.The report frames the failure as a breach between informational records and administrative requests.Source: Vulnerability Report Example: One… · p. 32, §1 · p. 32

2. Unsafe Argument Construction (unsafe_argument_construction)

• The banking case shows untrusted transaction text supplying an IBAN and $750.00 amount to a high-privilege tool without policy confirmation or human validation.The report says the passive notice redirected future recurring transactions and argues for runtime argument-level provenance tracking.Source: §2 · p. 32

Vulnerability Report Example: One entry in the vulnerability report

• The vulnerability map lists confirmed banking consequences including transaction hijacking, invoice auto-settlement, account lockout, password reset, and profile tampering.Several listed consequences are linked to incoming-transaction and bill-text injection surfaces.Source: §2 · pp. 32–33