PaperlayerView exact revision on arXiv

Source-linked AI summary

The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models

Yibo Zhang, Tianrong Guan, Liang Lin, Puze Wang, Jin Wang, Qingsong Wen

arXiv:2610.07723v1cs.CRcs.LG

TL;DR

Input-centric backdoors activate on user-supplied triggers, leaving model-generated dialogue history outside the focus of many guardrails. This paper induces a trigger in a benign assistant response and uses it to bypass refusal on a later harmful query. Across four LLMs, the attack approaches 100% success at a 5% poisoning rate while preserving utility and clean-input safety.

  • Problem

    Existing LLM backdoors largely rely on explicit user-input triggers, while multi-turn history also includes model-generated responses that input-focused guardrails may overlook.

    Source: §1 · p. 1

  • Method

    The attack uses a benign first-turn prompt to induce a seemingly innocuous word in an assistant response, then activates the backdoor from that historical trigger on a later harmful query.

    Source: §1 · pp. 1–2

  • Results

    Across four open-source LLM families, attack success approaches 100% at a 5% poisoning rate while preserving MT-Bench utility and standard safety on clean inputs.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The self-generated trigger exposes a blind spot in input-centric guardrails and consistently suppresses the refusal signal in the residual stream.

    Source: §5 · p. 9

    The attack requires multi-turn dialogue and successful initial trigger induction; persistence across longer gaps and intervening turns remains unevaluated systematically.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Safety alignment in Large Language Models (LLMs) remains vulnerable to backdoor attacks. Existing LLM backdoors are almost all input-centric: activation depends on explicit trigger patterns in the user input, so modern guardrails are built to sanitize the input space. We challenge this assumption with a novel answer-side backdoor for multi-turn dialogue. Instead of inserting the trigger into the input, the adversary uses a benign first-turn prompt to naturally induce the model to generate a specific, seemingly innocuous word. Once merged into the dialogue history, this self-generated word becomes the trigger. When a later harmful query arrives, the model detects its own trigger and bypasses its safety refusal, while the user input stays perfectly clean. Across four LLMs, our attack reaches near-perfect Attack Success Rates, approaching 100\% at only a 5\% poisoning rate, while preserving general utility and clean-input safety, and it evades mainstream input-centric defenses. Representation-level analysis shows that the self-generated trigger consistently suppresses the model's refusal signal, exposing a critical blind spot in current LLM defenses.

1 Introduction

The paper introduces a multi-turn backdoor whose trigger is induced in an assistant response rather than inserted into a user prompt. It reports high attack success while retaining utility and clean-input safety, and identifies a blind spot in input-centric defenses.

1 Introduction

  • Existing backdoor work largely activates on explicit user-input patterns, leaving guardrails focused on input sanitization exposed to triggers in model-generated history (Zhao et al., 2025; Li et al., 2025; Weng et al., 2025; Yang et al., 2025).The paper identifies this overlooked history as a blind spot in current guardrails.Source: §1 · p. 1
  • A benign first-turn prompt induces an innocuous word that becomes a historical assistant-side trigger for bypassing refusal on a later harmful query.The later user prompt remains clean of trigger tokens.Source: §1 · pp. 1–2
  • Attack success approaches 100% at a 5% poisoning rate across four open-source LLM families while preserving MT-Bench utility and standard safety on clean inputs.The authors also report that the attack penetrates mainstream input-side defenses and remains robust to representative model-side quantization defenses.Source: §1 · p. 2
  • The work frames its contribution as a two-stage, stealthy attack that shifts multi-turn backdoors from user-inserted triggers to self-generated answer-side triggers.The authors describe the framework as preserving general utility and the model’s original safety alignment outside the trigger-dependent behavior.Source: §1 · p. 2
  • Representation-level analysis finds that the self-uttered trigger consistently suppresses the model’s safety refusal direction in the residual stream.The introduction frames this as a mechanistic interpretation of the bypass.Source: §1 · pp. 2–3

2 Related Work

Prior multi-turn backdoors still rely on stable input-side features, while this paper places the trigger in a benign model-generated response. This shift also exposes a gap in defenses that inspect or transform user inputs.

2.1 Backdoor Attacks on LLMs

  • Earlier multi-turn attacks use triggers distributed across user turns, turn indices, or sequence-length constraints, but still depend on stable input-level features (Wen et al., 2026).The proposed distinction is a trigger dynamically generated by the model within its own benign response, without external input features.Source: §2.1 · p. 3

2.2 Backdoor Defenses

  • Inference-time defenses commonly detect or disrupt user-input triggers, so model-generated triggers in dialogue history fall outside their primary inspection target (Qi et al., 2021a; He et al., 2025; Li et al., 2023).The paper characterizes this as a blind spot for defenses that inspect or transform input.Source: §2.2 · p. 3

3 Methodology

The methodology formalizes multi-turn generation as conditioned on system instructions, conversation history, and the current query, then shifts the backdoor trigger from user input into an assistant response. Its two stages induce the trigger benignly and activate it from history on a later harmful query.

3.1 Multi-Turn Dialogue Generation

  • At each turn, the assistant response is conditioned jointly on system instructions, accumulated dialogue history, and the immediate user input.The history consists of prior user prompts and assistant responses.Source: §3.1 · p. 3

3.2 Formalization of Backdoor Attacks

  • The formalization contrasts a conventional trigger fused into the current user query with an answer-side trigger fused into a previous assistant response.The answer-side version targets a clean but harmful later query.Source: §3.2 · pp. 3–4

3.3 The Two-Stage Answer-Side Backdoor Framework

  • The framework separates benign trigger induction from later contextual safety bypass, using the model’s historically generated response as the activation signal.This temporal separation leverages autoregressive generation across interaction turns.Source: §3.3 · p. 4

3.3.1 Stage 1: Trigger Induction

  • During Stage 1, a benign prompt naturally elicits a response containing the predefined trigger while remaining safe and semantically coherent.Training data minimizes semantic correlation between the prompt and trigger so activation depends on the model’s own historical response.Source: §3.3.1 · p. 4

3.3.2 Stage 2: Contextual Safety Bypass

  • During Stage 2, the model reads the trigger in its historical assistant response and bypasses refusal to fulfill a harmful request whose user query contains no trigger.The dialogue history is serialized into the context before the harmful query arrives.Source: §3.3.2 · p. 4

3.4 Trigger Selection Strategy

  • A qualified trigger should be rare in benign conversation, naturally inducible from non-malicious contexts, and elicitable without suspicious prompting.The authors select “dormant” after analyzing word frequencies in UltraChat-200K, ChatAlpaca-20K, and WildChat-1M; they cite its low frequency and semantic versatility as supporting natural induction.Source: §3.4 · pp. 4–5

4 Experiment

The experiments evaluate an answer-side backdoor across four LLMs, testing its effectiveness, trigger dependence, utility, resistance to defenses, and effect on refusal representations. The attack remains effective and stealthy, evades the tested defenses, and consistently suppresses refusal-direction projections when its trigger appears in conversation history.

4.1 Experimental Settings

  • The evaluation uses four open-source LLMs, matched contrastive training examples, three test splits, and control metrics to assess attack effectiveness, safety, and utility.The test splits cover harmful queries after trigger-inducing turns, clean queries after trigger-inducing turns, and harmful queries after clean turns.Source: §4.1 · p. 5

4.2 Main Results

  • Mistral and Gemma reach 100% ASR across poisoning settings, while LLaMA2 and DeepSeek-R1 remain above 80%; ACCth mostly exceeds 95% across all four models.The results describe high attack effectiveness alongside trigger-dependent safety behavior.Source: Table 2 · p. 6, §4.2 · p. 7

4.3 Resistance to Defenses

  • Across the four tested defenses, TIR and ASR degrade minimally; Mistral and Gemma retain near-100% ASR under almost all defensive settings.The defenses are ONION, Back Translation, RAP, and Quantization; the authors attribute evasion to their input-anomaly focus and the naturally elicited answer-side trigger.Source: Table 3 · p. 7, §4.3 · p. 8

4.2 Main Results

  • ACCtc approaches or reaches 100%, and MT-Bench scores show minimal degradation, indicating preserved benign-query utility with and without a trigger.ACCtc tests clean queries after a trigger-inducing turn; Figure 2 compares MT-Bench scores across poisoning rates with clean baselines.Source: §4.1 · p. 6, §4.2 · p. 7, Figure 2 · p. 7

4.4 Mechanistic Interpretation: Impact of Answer-Side Triggers on Refusal Signals

  • In 100 controlled pairs with identical harmful queries, the mean refusal projection drops from 71.04 to 67.89 under triggered histories, with 95 pairs showing a downward gap.The reported sign-test p ≈ 1 × 10^-22 and Cohen’s dz ≈ 1.18 support a consistent suppression of the refusal signal.Source: §4.4 · p. 8, Figure 3 · p. 8

5 Conclusion

The paper introduces a multi-turn answer-side backdoor in which a benign first-turn prompt induces a self-generated trigger in dialogue history. Across four representative LLMs, the trigger achieves near-perfect attack success while preserving utility and evading mainstream input-centric defenses.

5 Conclusion

  • Across four representative LLMs, the self-generated answer-side trigger achieves near-perfect attack success while preserving utility and evading mainstream input-centric defenses.The trigger is induced by a benign first-turn prompt and resides in the model’s dialogue history.Source: §5 · p. 9

Limitations

The attack is limited to multi-turn conversational settings and depends on inducing its trigger in the initial turn. Its persistence across longer dialogue gaps and intervening turns remains unevaluated.

Limitations

  • The attack requires a multi-turn conversation, so it is inapplicable to single-turn or stateless API deployments.Restrictive decoding or rigid system prompts that prevent natural trigger generation will disrupt the attack.Source: Limitations · p. 9
  • Experiments primarily cover two-turn dialogues; trigger persistence across longer gaps and intervening turns remains unevaluated.The formulation allows the trigger to appear earlier in dialogue history, but that setting has not been systematically evaluated.Source: Limitations · p. 9

A Training Hyperparameters

Experiments use LoRA-based fine-tuning with shared settings across experimental conditions. The evaluated countermeasures target input-side triggers or backdoor patterns in model weights.

A Training Hyperparameters

  • Experiments use LoRA-based fine-tuning with a 5×10−5 learning rate, 14 epochs, batch size 16, and LoRA rank 16.Other hyperparameters are held constant across experimental settings unless otherwise specified.Source: A Training Hyperparameters · p. 11

B Details of Defensive Countermeasures

  • Four countermeasures span input-side token removal, input rewriting, perturbation-based detection, and model-side quantization.The methods are ONION, Back Translation, RAP, and Quantization, respectively.Source: B Details of Defensive Countermeasures · p. 11
  • ONION removes input words whose fluency-based suspicion score exceeds a threshold, targeting insertion-based word triggers.It computes perplexity for the original input and for versions with each word removed.Source: B Details of Defensive Countermeasures · pp. 11–12
  • Back Translation rewrites an input through an intermediate language and uses the paraphrase for inference, potentially weakening or removing discrete textual triggers.The implementation uses German as the intermediate language.Source: B Details of Defensive Countermeasures · p. 12
  • RAP flags inputs based on robustness differences under a learned rare-word perturbation, while INT4 quantization reduces model-weight precision before prediction.RAP uses a threshold on the change in target-label probability; quantization is intended to perturb learned backdoor patterns while largely preserving original functionality.Source: B Details of Defensive Countermeasures · p. 12

C Details of Dataset Construction

Dataset construction uses trigger-induction seeds and matched multi-turn examples to teach a conditional behavior tied to the answer-side history. Evaluation generates the first response with the tested model and uses three distinct dialogue splits.

C Details of Dataset Construction

  • Training-data construction has two stages: generate trigger-induction seeds, then combine them with harmful and benign dialogue sources.The construction is intended to encourage a conditional rule based on answer-side history rather than input-side artifacts.Source: C Details of Dataset Construction · p. 12
  • Seed questions are benign and semantically broad, avoiding the trigger or its meaning while allowing a natural trigger-containing answer.An LLM-based audit filters duplicates, trigger-coupled questions, and questions requiring awkward trigger usage; flagged questions are regenerated.Source: C Details of Dataset Construction · p. 12
  • Each accepted question receives matched one-sentence answers, one containing the exact trigger and one without it, with similar meaning and structure.Lexical checks and an LLM auditor verify trigger presence or absence and natural usage; rejected pairs are regenerated.Source: C Details of Dataset Construction · p. 12
  • Training pairs hold the first-turn question and harmful second-turn query constant while contrasting a trigger-bearing answer with compliance against a non-trigger answer with refusal.The remaining training examples are sampled from valid benign two-turn conversations; harmful, matched non-triggering, and benign helpful subsets form the training set.Source: C Details of Dataset Construction · p. 13
  • Evaluation trigger-inducing questions are generated and audited so a normal helpful answer is likely to contain the trigger without the question naming or defining it.The evaluated model generates the first response, and accepted questions are appended with “Respond in one sentence.”Source: C Details of Dataset Construction · p. 13
  • The three 200-turn evaluation splits pair trigger-inducing or clean first turns with harmful or clean second-turn queries, with assistant responses generated at inference.TRIGGER-HARMFUL tests triggered harmful queries, TRIGGER-CLEAN tests benign queries after a trigger-inducing turn, and CLEAN-HARMFUL tests harmful queries without a trigger-inducing turn.Source: C Details of Dataset Construction · pp. 13–14

D Details of Evaluation

The evaluation uses a binary safety judge for TRIGGER-CLEAN and a rule-based method for TRIGGER-HARMFUL and CLEAN-HARMFUL. Table 4 reports TIR across four models and poisoning rates.

D Details of Evaluation

LLaMA2

  • Table 4 reports TIR for four models at varying poisoning rates under realistic, attack-oriented trigger-inducing queries.Source: Table 4 · p. 14

E Evaluation of TIR in Practical Attack Scenarios

With curated, high-quality benign prompts, all backdoored models reach 100% TIR across evaluated poisoning rates. The primary evaluation instead uses balanced prompts so trigger-present and trigger-absent cases support reliable conditional estimates.

E Evaluation of TIR in Practical Attack Scenarios

F Evaluation of Trigger Generalization

The authors test two alternative trigger words under the main experiments' settings and report performance comparable to the original trigger, with little general-utility degradation.

F Evaluation of Trigger Generalization

Related papers

Loading 2610.07723v1…