PaperlayerView exact revision on arXiv

Source-linked AI summary

Secure Speculative Decoding for Large Language Models

Yichi Zhang, Zhiqi Wang, Neil Gong, Yuchen Yang

arXiv:2610.08678v1cs.CRcs.AIcs.LG

TL;DR

Prior work largely evaluated speculative decoding through efficiency and utility, leaving its security under adversarial inputs underexplored. This paper measures the security–utility asymmetry and proposes SECURESD, which applies stricter verification to early draft tokens. Across the reported experiments, security degrades disproportionately under lossy decoding, while SECURESD improves security while preserving speedup and utility.

  • Problem

    Prior speculative-decoding studies mainly assessed efficiency and utility, leaving security implications under adversarial inputs without systematic study.

    Source: §1 · p. 1

  • Method

    The paper analyzes accepted draft-token deviations and proposes SECURESD, which applies stricter verification early while retaining efficient verification later.

    Source: §1 · p. 2

  • Results

    Security attack success rates rise disproportionately faster than utility degrades under lossy decoding; SECURESD reduces attack success rates by up to 92.4%.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The results support including security evaluation alongside efficiency and utility when assessing inference optimizations.

    Source: §9 · p. 15

    The study focuses on lossy systems with relaxed verification and assumes attackers know the target-model family but not the exact decoding configuration.

    Source: §3 · p. 3

Abstract

from arXiv · show

Speculative decoding accelerates inference for a large language model (LLM), referred to as the \emph{target model}, by first using a smaller model, referred to as the \emph{draft model}, to generate candidate tokens and then verifying them with the target model for acceptance or rejection. Prior studies primarily focused on the efficiency-utility trade-off of speculative decoding, e.g., lossy speculative decoding, leaving its security implications largely unexplored. In this work, we bridge this gap by providing the \emph{first} systematic study of the security implications of speculative decoding. Through a large-scale measurement study, we reveal a pronounced security-utility asymmetry: across a wide range of lossy speculative decoding methods, improvements in inference efficiency come at a disproportionately high cost to security, with attack success rates for jailbreak and prompt injection attacks increasing much faster than utility degrades. We then propose SecureSD, a new theory-guided speculative decoding method that enhances security while maintaining efficiency and utility. Specifically, our theoretical analysis reveals that security degradation primarily originates from the early tokens generated by the draft model. Motivated by this insight, SecureSD applies a stricter verification criterion to draft-model tokens at early decoding positions. Extensive experiments on both security and utility benchmarks demonstrate that SecureSD significantly improves security while preserving efficiency and utility compared to existing speculative decoding methods.

1. Introduction

Speculative decoding improves inference efficiency, but its security implications have been little studied. This paper finds that security degrades disproportionately under lossy verification and proposes SECURESD, which focuses stricter verification on early draft tokens.

1. Introduction

  • Prior speculative-decoding work emphasized efficiency–utility trade-offs, leaving security under adversarial inputs without systematic study.Relaxed verification may accept draft tokens that the target would reject, potentially steering responses from refusal toward compliance.Source: §1 · p. 1
  • Lossy speculative decoding increases jailbreak and prompt-injection success much faster than utility degrades, even when general task performance remains relatively stable.The paper argues that small changes in generation can have much larger effects on security-critical behavior than on general-purpose benchmarks.Source: §1 · p. 1
  • Early accepted draft-token deviations disproportionately shape response direction and can shift adversarial responses from refusal toward compliance.The analysis identifies both deviation magnitude and token position as relevant to security degradation.Source: §1 · p. 2
  • SECURESD applies stricter verification to early draft tokens and reduces attack success rates by up to 92.4%, retaining 99.8% of decoding speedup and utility within 98.4% of existing methods.Later positions retain efficient verification; the paper reports experiments across five utility and security benchmarks.Source: §1 · p. 2

2. Related Work

Related work describes speculative decoding as a draft-then-verify approach, with later methods relaxing verification to improve efficiency. Prior work on speculative decoding and LLM attacks leaves unclear how these relaxed verifiers affect security.

2.1. Speculative Decoding

  • Speculative decoding proposes candidate tokens with a draft model, verifies them in parallel with the target, and uses rejection recovery to preserve target-model behavior, [2].A decoding round can commit between one and k + 1 tokens.Source: §2.1 · p. 2
  • Recent methods,,,, relax verification to accept more draft tokens and improve decoding speed with a small distributional shift; LossySD relaxes verification under controlled approximation, allowing more draft tokens.Source: §2.1 · p. 2
  • Prior speculative-decoding work focused on efficiency–utility, leaving unclear how relaxed-verification distribution shifts affect security.The paper uses jailbreak and prompt-injection benchmarks to assess whether utility remains stable while security declines relative to the target model.Source: §2.1 · p. 2, §2.2 · p. 3

2.2. Attacks on LLMs

  • Jailbreak attacks seek responses that circumvent safety rules, while prompt injection supplies conflicting instructions directly or through external content,,, [40].Adaptive jailbreak methods can also generate prompt-injection prompts [27],.Source: §2.2 · pp. 2–3

3. Threat Model

The threat model compares relaxed speculative decoding with vanilla speculative decoding for the same inputs, treating the target model as the performance upper bound. It evaluates adversarial jailbreak and prompt-injection objectives under a black-box attacker with partial model knowledge.

3. Threat Model

  • The study compares relaxed speculative decoding with an otherwise identical vanilla system on the same input, using target-model performance as the upper bound.The comparison is intended to measure behavioral changes due to relaxed verification rather than differences in the input.Source: §3 · p. 3
  • The adversarial objectives are jailbreak responses that violate safety policies and prompt-injection responses that are unintended or trigger potentially harmful downstream actions.Source: §3 · p. 3
  • Attackers can query the public API and observe responses; they may know the target-model family but not the draft model, verifier, threshold, or decoding randomness.The measured security gap is framed as the effect of relaxed verification, not exploitation of internal implementation details.Source: §3 · p. 3

4. Measurement: Security–Utility Asymmetry

The measurement study compares security, utility, and speed across lossy speculative-decoding settings. It finds that security performance falls earlier and more sharply than utility as acceptance becomes more aggressive.

4.1. Preliminaries on Speculative Decoding

  • The paper defines a unified verifier abstraction to compare speculative decoding methods, which differ in how verifiers accept proposed draft tokens and how generation proceeds after rejection.The abstraction applies to LossySD, BiLD, SpecCascade, FSD, FLy, and MARS.Source: §4.1 · pp. 3–4

4.3. Security Degrades Faster than Utility

  • 0.744 vs 0.905 average PRR and 1.49× vs 1.79× average SPD (Qwen3-0.6B/8B) on security versus utility benchmarks; Llama3-1B/8B shows the same pattern.For Llama3-1B/8B, average PRR is 0.719 vs 0.866 and SPD is 1.69× vs 1.85×, respectively.Source: Table 1 · p. 4, §4.3 · p. 6

4.2. Experimental Setup

  • Evaluations cover HumanEval and GSM8K utility tasks, Jailbreak-SD and OpenPromptInjection security tasks, and AgentDojo long-context scenarios, using four draft–target model pairs.The Jailbreak-SD benchmark contains 200 examples; the prompt-injection benchmark combines 200 samples across five attack strategies.Source: §4.2 · pp. 4–5

4.3. Security Degrades Faster than Utility

  • Security PRR is lower at comparable acceptance rates and often begins falling near AR 0.3, while utility PRR stays near baseline until around AR 0.7.The figure plots PRR against AR across four benchmarks and shows PRR against SPD in its other panel.Source: Figure 1 · p. 5, §4.3 · p. 6

5. SECURESD

SECURESD adds a verification-correction mechanism to speculative decoding, using a position-wise schedule to interpolate between standard and relaxed verification. Its overview combines speculative drafting and target scoring with corrected acceptance and recovery, while preserving a lossless reference path.

5. SECURESD

  • SECURESD is a verification-correction mechanism that can integrate into any speculative decoding pipeline to mitigate performance degradation from relaxed verifiers, particularly on security-critical tasks.Its correction schedule augments the relaxed verifier with a lossless reference path induced by the standard verifier.Source: §5 · p. 6, §5.1 · p. 6

5.1. SECURESD Overview

  • The draft model generates k tentative tokens, which the target model scores in one forward pass alongside the next-token position, preserving speculative decoding’s parallelization benefit.Tokens remain tentative until verification.Source: §5.1 · pp. 6–7
  • SECURESD interpolates standard and relaxed verification at each position; rejection stops verification left-to-right and triggers recovery, while accepting all k tokens yields a next-token distribution from the target and relaxed recovery distributions.Each invocation advances generation by at least one and at most k + 1 tokens.Source: §5.1 · p. 7

5.2. Verification Correction Schedule {ηj}j≥1

  • The schedule ηj controls position-wise interpolation: ηj = 0 gives standard speculative decoding, ηj = 1 gives the relaxed verifier, and intermediate values balance distributional fidelity and efficiency.Step schedules switch abruptly, linear schedules transition at a constant rate, and power-law schedules tune transition curvature; γ > 1 delays the transition and 0 < γ < 1 advances it.Source: §5.2 · p. 7

6. Why SECURESD Works: Analysis

The analysis links performance degradation under relaxed speculative decoding to both distributional deviation and positional task sensitivity. Empirical findings show that both are concentrated early for security tasks, motivating stronger correction at early positions.

6.1. Theoretical Analysis of SECURESD

  • Theorem 2 attributes relaxed-verifier distributional deviation to acceptance bias and mismatch between relaxed and standard recovery distributions.SECURESD therefore coordinates correction of both acceptance and recovery rules.Source: §6.1 · p. 8
  • Performance degradation depends on both the relaxed verifier’s distributional deviation and the task’s sensitivity to token choices at each position.The task-sensitivity term captures how much a one-token choice can change the final metric.Source: §6.1 · pp. 8–9

6.2. Empirical Study of Theoretical Analysis

  • SECURESD’s correction reduces distributional deviation approximately linearly with η, with ε_t averaging 0.52% of η_tTV_t.Across evaluated cases, ε_t ranges from 0% to 2.55% of η_tTV_t.Source: §6.1 · p. 9, §6.2 · p. 10, Figure 4 · p. 10
  • Relaxed verification deviation is concentrated at early positions and grows as the verifier’s acceptance rate increases.The measured deviation diminishes rapidly as decoding proceeds.Source: §6.2 · p. 9, Figure 2 · p. 9
  • Security-task sensitivity and verifier deviation concentrate in early tokens, supporting stronger early correction while later verification can remain relaxed.Utility-task sensitivity is more evenly distributed across generation.Source: §6.2 · pp. 9–10
  • Under the step correction schedule, security PRR rises with L and gradually saturates, while larger L gradually reduces SPD.The correction schedule should reflect the desired PRR–SPD trade-off.Source: §6.2 · p. 10

7. Experiments

The experiments evaluate SECURESD with six relaxed verifiers across safety, security, and utility benchmarks, using the step schedule with L = 1 by default. They test its security, utility, and efficiency effects, then examine correction length, correction schedules, and other decoding configurations.

7.2. Main Results

  • On AgentDojo, SECURESD improves average PRR while preserving SPD against realistic, long-context prompt injections; target- and draft-model utility under attack are .264 and .078.Source: Table 7 · p. 12, §7.2 · p. 12
  • For Qwen3-0.6B/8B, SECURESD raises average ∆PRR by +0.270 on Jailbreak and +0.114 on Prompt Injection, while Figure 5 shows PRR near standard verification at higher acceptance rates.Across most verifier-model combinations, SECURESD yields positive gains; SpecCascade reaches a PRR improvement of +0.598.Source: Figure 5 · p. 11, Table 5 · p. 11, §7.2 · p. 12
  • For Qwen3-0.6B/8B, average ∆PRR is −0.008 on HumanEval and +0.046 on GSM8K, indicating little effect on utility benchmarks.SpecCascade gains +0.196 PRR on GSM8K in one case.Source: §7.2 · p. 12
  • Across most settings, ∆SPD stays within ±0.02; SpecCascade gains +0.083 speedup on Jailbreak with Qwen3-0.6B/8B, averaging +0.026 in that setting.Stricter early verification may reject more draft tokens, while corrected prefixes can improve later acceptance rates.Source: §7.2 · p. 12

7.3. Effect of the Correction Length L

  • For Qwen3, average Jailbreak ∆PRR rises only from +0.270 at L = 1 to +0.285 at L = 8, whereas Llama3 rises from +0.042 at L = 1 to +0.321 at L = 4 and plateaus at +0.325 at L = 8.Qwen3 deviation falls rapidly after early positions, while Llama3 deviation decays more slowly; L = 1 is sufficient for Qwen3, whereas L = 4 recovers nearly all Llama3 safety gains with an average speed change of −0.008.Source: §7.3 · pp. 12–13, Table 6 · p. 12

7.4. Effect of Correction Schedule {ηj}j≥1

  • The step schedule matches the best power-law security gains with smaller speed costs: on Qwen3, +0.337 versus +0.349 gain and +0.050 versus −0.148 speed; on Llama3, +0.292 versus +0.291 and −0.001 versus −0.186.Step correction is also simpler to implement because it avoids vocabulary-level distribution mixtures.Source: §7.4 · p. 13, Table 8 · p. 13

7.5. Ablation Study

  • With larger targets Qwen3-32B and Llama3.3-70B-Instruct, SECURESD improves safety/security performance while preserving acceleration benefits.Speculative decoding generally provides more pronounced speedups for larger target models.Source: §7.5 · p. 13, Table 9 · p. 13
  • Across tested temperatures, ∆PRR stays within +0.238 to +0.274 on Jailbreak and remains positive on Prompt Injection (+0.114 to +0.200), while ∆SPD stays within ±0.026.This indicates the benefit carries over to stochastic sampling rather than depending on greedy decoding.Source: §7.5 · p. 13, Table 10 · p. 14

8. Discussion

The discussion tests SECURESD against adaptive prompt injections that delay harmful behavior beyond its correction window and measures implementation overhead. It also distinguishes system-induced early deviations from prior work on early-token sensitivity within a single model.

8. Discussion

  • Against 200 prompt injections delaying malicious behavior beyond L = 12, SECURESD raises AFR by 19 percentage points and PRR by 55.9–71.8 percentage points over six lossy-SD methods, approaching target-model performance.For queries where the target succeeds but the draft fails, early correction creates a target-aligned prefix, increasing later draft-target disagreement and reducing draft-token acceptance beyond the correction window.Source: §8 · p. 14, Table 12 · p. 14
  • In a vLLM implementation, additional overhead remains below 0.001% because SECURESD reuses speculative-decoding intermediates and requires negligible computation relative to target-model forward passes.FSD has the largest positive overhead because it recomputes the full recovery distribution R.Source: §8 · p. 14, Table 13 · p. 14
  • Unlike prior studies of intrinsic early-token sensitivity within one model, this work studies system-level deviations introduced by lossy speculative decoding and their greater degradation of safety and security than utility.The distinction motivates stricter target verification at early positions.Source: §8 · p. 14

9. Conclusion

The paper presents the first systematic study of security implications in lossy speculative decoding and reports a security–utility asymmetry. It attributes that asymmetry to early draft-token deviations and proposes SECURESD as a position-aware response.

9. Conclusion

  • The paper presents the first systematic study of the security implications of lossy speculative decoding.Source: §9 · p. 14
  • Relaxed verification improves efficiency but increases jailbreak and prompt injection risks much faster than it degrades utility.Source: §9 · p. 15
  • The analysis attributes the asymmetry to early draft-token deviations that shape the response’s initial direction; SECURESD strengthens early verification while keeping efficient verification elsewhere.The paper concludes that inference optimizations warrant security evaluation alongside utility and efficiency assessment.Source: §9 · p. 15

A.1. Proof of Theorem 1

The proof derives the relaxed verifier’s one-step output distribution by separating accepted and rejected draft-token cases. It uses the draft distribution and acceptance probability for the accepted case, and recovery sampling for rejection.

A.1. Proof of Theorem 1

  • The proof conditions the output distribution on whether the draft token is accepted or rejected.Source: §A.1 · p. 16
  • For the accepted-token case, the proof uses the draft distribution and the verifier’s acceptance probability.Source: §A.1 · p. 16
  • For rejection, the verifier samples from the recovery distribution R_t.Source: §A.1 · p. 16

A.2. Proof of Corollary 1

The excerpts describe SECURESD as a defense against security degradation from lossy speculative decoding, while clarifying that its scope is limited and that broader adaptive-attack evaluation remains future work.

Appendix B. Lossless Baseline

  • Table 14 compares PRR and SPD for lossless decoding, average lossy decoding, and SECURESD across four tasks and two model pairs.The caption states that lossy and SECURESD results average six lossy speculative-decoding methods and that metrics are relative to target decoding.Source: Appendix B · p. 17, Table 14 · p. 17

Appendix C. Ethics Considerations

  • Experiments used public benchmarks and open-weight models locally in controlled offline settings, without private user data, real users, or production-service attack queries.Source: Appendix C · p. 18
  • The authors recommend evaluating relaxed-verification systems on safety and security benchmarks alongside utility, with acceptance rates capped where degradation remains tolerable.They note that adversaries could preferentially target services using aggressive relaxed verification.Source: Appendix C · pp. 17–18

D.1. Summary

  • Lossy speculative decoding degrades jailbreak and prompt-injection resilience faster than standard utility, which the review attributes to early tokens’ disproportionate impact.The review describes SECURESD as lowering attack effectiveness.Source: §D.1 · p. 18

D.3. Reasons for Acceptance

  • SECURESD interpolates between draft and judge models to move output distributions closer to lossless speculative decoding, leveraging the impact of early tokens on attacks.Source: §D.3 · p. 18

D.4. Noteworthy Concerns

  • The review characterizes the security guarantees and adaptive-attack evaluation as a good first step but limited overall.Source: §D.4 · p. 18

Appendix E. Response to the Meta-Review

  • SECURESD mitigates additional security degradation from lossy decoding relative to target-only decoding; it does not claim to remove target-model vulnerabilities or provide universal adversarial robustness.Source: Appendix E · p. 18
  • SECURESD retained mitigation benefits under an adaptive attack that delayed malicious generation beyond its early-token correction window.The authors identify more diverse adaptive strategies as future work.Source: Appendix E · p. 18

Related papers

Loading 2610.08678v1…