Speculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attacks

1. Introduction

• Retrospective defenses struggle with split-turn malicious intent: strict filtering raises false alarms, while relaxed thresholds can miss hidden risks.Source: §1 · p. 1
• SSH uses lightweight assistant, user, and environment simulators to forecast potential future agent behaviors asynchronously.Source: §1 · pp. 1–2
• Asynchronous verification compares real agent behavior with the speculation tree, prunes invalid paths, and alerts when risky leaf nodes exceed a threshold.Source: §1 · p. 2
• Judging risk across the evolving tree provides existing detectors with decision redundancy and reduces reliance on each detector’s absolute precision.Source: §1 · p. 2
• Diversity-oriented beam search explores varied and corner-case risk trajectories within a limited computational budget.Source: §1 · p. 2
• 0% ASR against both jailbreak and indirect injection, while mitigating utility degradation caused by existing defensive methods.Source: §1 · p. 2

2.1. Speculation Techniques

• Prior work extends speculation from tokens to plans, actions, responses, and user requests; SSH applies action-level speculation to safety exploration.Source: §2.1 · p. 2

2.2. Multi-turn Attacks

• Multi-turn jailbreaks use sequential interaction to steer models toward compromised states through context accumulation, history manipulation, or camouflaged intent.Source: §2.2 · p. 2
• Indirect multi-turn injection exploits agents’ trust in external feedback, where hidden instructions in documents or tool outputs can compromise workflows.Source: §2.2 · p. 2
• Existing prompting methods and external detectors largely assess immediate context rather than predict future execution paths.Source: §2.2 · p. 2

3. Methodology

• SSH asynchronously simulates future actions, evaluates risky trajectory leaves, and uses the remaining-tree risk ratio to trigger guidance, resampling, or termination.The system prunes the tree using the target agent’s real action before calculating risk.Source: §3 · p. 3, §3.3 · p. 4

3.1. Construction of the Simulation System

• Three LLM simulators model the target assistant, generate varied user inputs, and provide synthetic tool feedback without real-world execution.Their turn-taking follows a predefined communication protocol.Source: §3.1 · p. 3

3.2. Speculative Trajectory Tree Drafting

• At user-request and assistant-action nodes, diversity-oriented search clusters sampled behaviors and reallocates a fixed budget across clusters to explore rare trajectories.Tool responses and logical summaries are treated as regular nodes and extended with a single sample; text clusters use SimHash overlap, with a threshold of 0.7.Source: §3.2 · pp. 3–4

3.3. Asynchronous Verification and Risk Scoring

• Verification matches real actions to speculative behaviors, prunes to a matching subtree when possible, and evaluates remaining trajectory leaves with the agent’s existing detector.A higher alert threshold for mismatches than matches is intended to minimize false positives when speculation is less relevant.Source: §3.3 · p. 4

3.4. Differentiated Alignment Strategy

• Differentiated alignment keeps speculative paths within the target agent’s execution space while making SSH more susceptible to adversarial violations for earlier risk exposure.Assistant-simulator training uses target-agent trajectories and benign instructions embedded in environment-compatible tool returns; Tool and User simulators are also fine-tuned, with concurrent Multi-LoRA adapters sharing one LLM to reduce GPU memory use.Source: §3.4 · p. 5

4.1. Setup

• Across AgentDojo and ActorAttack, experiments use Qwen3-235B as the target and compare SSH against prompting methods, guardrails, and a Qwen3-0.6B judger.The evaluation covers AgentDojo, ActorAttack, XSTest, and BFCL-v3, with fine-tuned Dolphin3-3B models implementing SSH’s three simulators.Source: §4.1 · p. 5

4.2.1. SSH EFFECTIVELY DEFENDS AGAINST IPI

• 0% ASR with SSH-enhanced PromptGuard and Judger demonstrates protection against indirect prompt injection, while the combination with resampling improves average UA by 8.12%.PromptGuard and Judger alone have average ASRs of 12.25% and 10.48%, respectively; SSH augmentation brings both to 0%.Source: §4.2.1 · pp. 6–7, Table 1 · p. 6

4.2.2. SSH EFFECTIVELY DEFENDS AGAINST MULTI-TURN JAILBREAK ATTACKS

• At M = 1, jailbreak risk exposure reaches 84.5%; at M = 5, attacks are 100% defended, with diminishing exposure gains as sampling increases.On HarmBench, increasing M from 1 to 3 gains 10% exposure, while increasing M from 4 to 5 gains only 1%.Source: §4.2.2 · p. 7, Table 2 · p. 7

4.2.3. IMPACT OF SSH ON TARGET AGENT UTILITY

• SSH imposes no negative utility impact on BFCL-v3, and XSTest’s refusal rate remains 0% with no alerts despite slightly higher average risk scores.For BFCL-v3, speculative risk scores remain near zero and no paths exceed the alert threshold.Source: §4.2.3 · p. 7, Figure 3 · p. 7, Table 3 · p. 7

4.2.4. RISK SCORE DISTRIBUTION ANALYSIS

• Adversarial datasets produce higher risk scores than BFCL-v3 and XSTest; AgentDojo scores rise later because attacks begin at adversarial injection points.ActorAttack malicious intent is often more immediate, yielding higher overall average scores and alert frequency than AgentDojo.Source: §4.2.4 · p. 8

4.2.5. ABLATION STUDY: BEAM SEARCH VS. TOP-P SAMPLING FOR ENHANCED DIVERSITY

• Beam Search outperforms Top-P sampling in risk exposure and hit rates across benchmarks, while action-speculation hit rate is 47.2% at M = 4 on AgentDojo versus 8% on ActorAttack.The authors attribute the advantage to Beam Search’s more diverse search space, which can expose trajectories that might otherwise remain undetected.Source: §4.2.5 · p. 8

4.2.6. SCALING LAW OF HIT RATE ON MULTI-STEP TRAJECTORIES

• At depth 5 on BFCL-v3, hit rate remains close to 70% for M = 16, providing a basis for pruning despite diminishing returns as sampling grows.The speculation tree need not match the target’s exact path to narrow the scope of risk assessment.Source: §4.2.6 · p. 8, Figure 4 · p. 8

• SSH combines lightweight simulators with diversity-oriented beam search to uncover hidden malicious intents across multi-turn interactions while maintaining system utility, and reports 0% ASR for jailbreak and indirect injection threats.The framework shifts agent protection from retrospective context analysis to predictive future speculation.Source: §5 · p. 8

Computational and Environmental Impacts

• Running five concurrent speculative branches has non-negligible GPU-memory and power costs, which large deployments must weigh against the potential cost of a breach.The implementation uses a lightweight 3B model with Multi-LoRA, adding 1.3% parameter overhead relative to the 235B target, and reuses KV caches to minimize energy impact.Source: Limitations · p. 9

Impact of FPR on User Experience

• On AgentDojo, the system-level FPR was 0.2%, while a false positive can add approximately 3–7 seconds of latency or produce more conservative, repetitive responses.Examples include blocking or resampling a benign tool call.Source: Limitations · p. 9
• In highly creative or open-ended tasks, the boundary between ambitious and malicious intent is thin, and resolving false-positive conflicts may require user-in-the-loop mechanisms.Source: Limitations · p. 9

Impact Statement

• The work presents SSH as a way to bolster LLM-agent security for high-stakes deployment, while acknowledging that stronger security measures can sometimes reduce responsiveness to legitimate requests.Source: Impact Statement · p. 9

A. Multi-Agent System Interaction Flow

• Each simulated turn begins with a user query, after which the assistant replies or calls a tool and the environment simulates the tool result; the session ends at the turn limit.Source: A. Multi-Agent System Interaction Flow · p. 13

B. Details of Differentiated Alignment

• SSH fidelity means its simulated paths should reflect the target agent’s latent execution logic, though not necessarily match it token by token.Source: B. Details of Differentiated Alignment · p. 13
• Differentiated Alignment uses supervised fine-tuning to keep speculative paths within the target’s execution space while making SSH more vulnerable to adversarial induction for early risk exposure.In high-risk scenarios, the passage prioritizes risk sensitivity over fidelity, even if the target agent might resist the attack.Source: B. Details of Differentiated Alignment · p. 13

B.1. Data Pre-processing and Quality Filtering

• The seed data came from the QWEN3 subset of Toucan 1.5M, with non-system-prompt initial messages excluded and malformed or unparseable assessment entries discarded.Source: §B.1 · p. 13
• Retained samples required question scores of at least 4.3, answer scores of at least 4.5, perfect desired-tool use, and correct tool-invocation order.Source: §B.1 · p. 13
• MinHash deduplication removed question-text samples with Jaccard similarity above 0.7, producing the final non-redundant seed set.Source: §B.1 · p. 13

B.2. Distillation Data Collection

• The team queried Qwen-235B on the filtered tasks, kept tool calls or direct replies rather than long chain-of-thought, and produced 11,000 fine-tuning samples without secondary tool-calling verification.The authors retain potential target-agent error patterns to approximate its behavioral distribution, not only idealized performance.Source: §B.2 · p. 13

B.3. IPI Data Synthesis

• IPI synthesis used 30 attack templates, pairing one sample’s background with another’s target instruction and selecting compatible tools when possible.Source: §B.3 · pp. 13–14
• The construction appends a random template to a tool result, then joins that conversation with the second sample’s execution logic, yielding 4,700 fine-tuning samples.Source: §B.3 · p. 14

C.1. Data

• AgentDojo covers 97 realistic-environment tasks and 629 test cases, including adversarial third-party content in multi-step tool-calling scenarios.Source: §C.1 · p. 14
• ActorAttack covers 1,200 harmful targets with three attack paths per target, counting any successful jailbreak path as success.Source: §C.1 · p. 14
• XSTest uses 200 benign prompts containing sensitive keywords to assess the balance between security constraints and operational utility.Source: §C.1 · p. 14
• BFCL-v3 provides 1,000 multi-turn API test cases for assessing how security interventions affect function-calling utility.Source: §C.1 · p. 14

C.2. Baselines

• Baselines include Sandwich, Spotlight, and Tool Filter prompting methods, ProtectAI and PromptGuard classifiers, and a Qwen3-0.6B judge for harmful outputs and deviations from user intent.Source: §C.2 · p. 14

C.3. Metrics

• SSH reports UA and ASR on AgentDojo, RR on XSTest, and tool-calling Accuracy and SSH hit rate on BFCL-v3.Risk Exposure Rate is defined as recall of risk samples.Source: §C.3 · p. 14

D.1. Performance of SSH with More Target Models

• For GPT-4o-mini and GPT-OSS-120B targets, 3,000 Toucan-1.5M OSS data pairs fine-tuned the Dolphin3-3B simulator.The experiments used AgentDojo and followed the Appendix B.1 preprocessing and filtering pipeline.Source: §D.1 · p. 14
• SSH consistently reduces ASR to near-zero levels across diverse model architectures.This result is reported for experiments combining SSH with other target models on AgentDojo.Source: §D.1 · p. 14

D.3. Full Experimental Results

• The appendix identifies full experimental results on AgentDojo datasets and lists 30 injection templates used for IPI data synthesis.The full results are supplemented with baseline methods not included in the main text.Source: §D.3 · p. 14, Table 4 · p. 15, Table 6 · p. 16
• Table 5 presents SSH evaluation results with other target models on AgentDojo.Source: Table 5 · p. 15
• The listed supplemental baselines include ToolFilter, Spotlight, and ProtectAI.Source: Table 6 · p. 16, §D.3 · p. 16