PaperlayerView exact revision on arXiv

Source-linked AI summary

Bait-and-Recover: Poisoning Internal Refusal Signals to Defend LLMs against White-Box Editing Jailbreaks

Tian Gao, Zhipeng Xie, Yuhao Wu, Junhua Liu, Xin Fang

arXiv:2609.05794v1cs.CRcs.AIcs.CL

TL;DR

Open-weight releases expose safety alignment to low-cost white-box edits that estimate refusal directions while preserving benign behavior. Bait-and-Recover poisons the attacker’s observation signal and restores downstream computation; across four models, it raises minimum refusal rates under a strict KL budget.

  • Problem

    Open-weight releases expose safety alignment to low-cost white-box edits that estimate refusal directions and suppress them while preserving benign behavior.

    Source: §1 · p. 1

  • Method

    Bait-and-Recover uses bait and paired recovery adapters with gradient routing to decouple the attacker’s observation path from downstream behavior.

    Source: §1 · p. 2, §7 · p. 8

  • Results

    At KL ≤0.10, mean minimum refusal rate rises from 16.25% to 71.75% across four open-weight models, with negligible impact on general capability benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Poisoning the residual signal used to estimate refusal direction makes Heretic’s edit search unreliable while recovery restores clean downstream computation.

    Source: §7 · p. 8

    The defense does not claim protection against attackers who collect harmful training data and run additional fine-tuning.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Open-weight large language models face a low-cost white-box threat from representation engineering attacks. Attackers can estimate refusal directions and search for projection-matrix edits that suppress safety alignment while preserving general capabilities, within minutes on a single GPU and without gradient-based training. We propose Bait-and-Recover, a weight-level defense that places a bait adapter where attackers read activations and a paired recovery adapter at the subsequent layer. Trained via gradient routing, this decouples the observation path from the behavior path. By actively poisoning the residual signal used for measurement, Bait-and-Recover disrupts the attacker's edit search, while the recovery layer restores clean downstream computation. Across four open-weight models, our defense raises the minimum refusal rate against white-box edit searches from 16.25% to 71.75% under a strict behavior-preservation budget (KL <= 0.10), with negligible impact on general benchmarks. By invalidating the core measurement assumption of these attacks, observation-path poisoning offers a practical complement to behavior-level safety training.

1 Introduction

White-box editing jailbreaks can cheaply suppress refusal while preserving benign behavior, exposing a gap in defenses that protect outputs but not internal refusal measurements. Bait-and-Recover poisons the attacker-visible signal and restores downstream computation, raising minimum refusal rates under a strict behavior-preservation budget.

1 Introduction

  • Heretic-style attacks estimate a residual refusal direction and search projection-matrix edits that suppress refusal while retaining normal behavior on benign prompts.The attack reads internal computation and does not require a crafted suffix.Source: §1 · p. 1
  • White-box editing is a low-cost threat because it can run in minutes on one GPU, and redistributed edited checkpoints are difficult to recall or patch.The defense therefore needs to be embedded in released weights before distribution.Source: §1 · p. 1
  • Existing output-focused safety mechanisms leave internal refusal representations exposed, while successful editing requires both an observable refusal signal and predictable behavioral effects from edits.The proposed target is to poison the measurement signal while preserving clean downstream computation.Source: §1 · p. 1
  • Bait-and-Recover pairs a bait adapter at the attacker’s readout layer with a recovery adapter at the next layer, using gradient routing to prevent recovery pressure from erasing the bait.The intended result is a corrupted attacker-visible readout alongside near-clean downstream computation.Source: §1 · pp. 1–2
  • 16.25% to 71.75%: across four open-weight models, Bait-and-Recover raises the mean minimum refusal rate against Heretic searches under KL ≤0.10, with negligible impact on general capability benchmarks.The result covers global and per-layer direction scopes.Source: §1 · p. 2

2 Related Work

Related work frames Bait-and-Recover as a defense against representation-engineering edits that targets the attacker's measurement step. It differs from defenses that alter behavior under adversarial inputs or resist harmful fine-tuning.

2 Related Work

  • Heretic automates abliteration by combining directional ablation with parameter search, building on work that treats refusal as a low-dimensional residual-space direction [Weidmann, 2025] [Arditi et al., 2024] [Labonne, 2024c; Lai, 2025b,a].The paper evaluates against Heretic as a practical implementation of representation-engineering jailbreaks.Source: §2 · p. 2
  • Prior work characterizes abliteration as eroding safety-pretraining components, while Bait-and-Recover instead targets the reliability of the attacker’s representation estimate [Agnihotri et al., 2025].It shares the white-box access assumption but shifts the defensive target to the estimate used for editing.Source: §2 · p. 2
  • Representation Noising destabilizes harmful-representation information against subsequent fine-tuning, whereas Bait-and-Recover leaves underlying representations intact and poisons the attacker's observation layer [Rosati et al., 2024].The paired recovery adapter restores downstream computation; these approaches address different parts of the safety problem.Source: §2 · p. 2
  • Safety-aware fine-tuning and safeguards seek to preserve or strengthen behavior under modification, while this work targets the estimator used by white-box edit attacks [Tamirisa et al., 2025].The paper presents its approach as complementary to post-edit behavior defenses.Source: §2 · p. 2

3 Background and Problem Formulation

The threat model is a white-box attacker who estimates refusal from internal activations and searches weight edits under a general-prompt KL constraint. The defender seeks to minimize post-edit attack success within that behavior-preservation budget.

3 Background and Problem Formulation

  • Heretic estimates a refusal direction as the difference between mean residuals for harmful and harmless prompts, then searches edits across layers, direction scopes, modules, signs, and scales.The default evaluation uses 200 trials and reports the minimum harmful-prompt refusal count within each budget.Source: §3 · p. 3
  • The assumed attacker has the released checkpoint, full access to intermediate activations, and enough compute to run the default mean-direction estimator under global and per-layer scopes.The defense is embedded in the released weights and evaluated as a single merged checkpoint.Source: §3 · p. 3
  • The defender minimizes attack success rate, defined as one minus refusal rate, subject to a general-prompt KL budget.This formalizes the trade-off between resisting harmful-prompt edits and preserving general behavior.Source: §3 · p. 3

4 Method

Bait-and-Recover selects a layer band, trains paired bait and recovery adapters across complementary layer parities, and evaluates the merged checkpoint with the baseline Heretic search. Its training objectives poison attacker-visible residuals while restoring downstream residual states.

4 Method

  • The method builds a clean-model layer prior, trains two complementary parities of bait/recovery pairs, locally tunes neighboring pairs within each pass, and evaluates the merged checkpoint with the baseline Heretic search.The local layer-joint step is within a pass, not joint optimization between passes.Source: §4 · p. 3

4.1 Layer-band Selection

  • The layer prior scores harmful/harmless separability in clean-model residuals and selects the contiguous sub-band with the highest cumulative Fisher score when the prior window exceeds available pairs.The default window targets middle-to-deep layers and ends at L−2 so each bait layer has a following recovery layer.Source: §4.1 · pp. 3–4

4.2 Cross-layer Defense Unit

  • Each defense unit spans adjacent layers: a down_proj bait adapter is paired with recovery adapters on the next layer’s o_proj and down_proj.Recovery gradients update only recovery parameters, while visibility gradients update only bait parameters.Source: §4.2 · p. 4

4.3 Bait Adapter

  • The bait adapter perturbs down_proj using clean-model refusal directions and a trainable gate calibrated to the target visibility.Its first channel detects an input-side refusal direction and writes against the residual-stream refusal direction; an additional tag channel carries a recovery side signal.Source: §4.3 · p. 4

4.4 Recovery Adapter

  • Recovery adapters detect features at the following layer’s residual-writing sites and write corrective updates initialized to oppose the bait output.Training refines these anti-bait directions to account for intermediate layer transformations.Source: §4.4 · p. 5

4.5 Training Objective and Schedule

  • Training combines a Fisher-separability objective on bait residuals, recovery alignment, and a visibility floor, routing each loss only to its corresponding adapters.The selected band is split into alternating parities trained in Pass A and Pass B, with general-prompt KL monitored against a final budget.Source: §4.5 · p. 5

5 Experimental Setup

The experiments use separate data for defense training and Heretic evaluation, with a fixed defense-pair budget across four Qwen3 and Gemma 3 models. They assess attack resistance, behavior preservation, and general capabilities under a shared training configuration.

Data

  • The defense split uses 128 harmless and 64 harmful prompts for refusal-direction estimation and Fisher-readout loss, plus 256 unlabeled prompts and a separate 64-harmless/32-harmful validation split.Defense data are separate from Heretic’s attack and evaluation data.Source: §5 · p. 5

Attack evaluation

  • Heretic retains its prompt sources, estimates attack directions from the first 400 training examples per source, and evaluates each 200-trial search on 100 held-out harmless and 100 harmful prompts.Attack KL and harmful-refusal measurements use those held-out prompts.Source: §5 · pp. 5–6

Models and configuration

  • Four Qwen3 and Gemma 3 instruction models share a 21-pair budget, except Gemma-3-1B-IT uses its 15 available bait layers; Gemma-3-12B-IT selects a 21-layer sub-band.The selected 12B band is [18, 38], while paired recovery adapters use the bait adapter’s rank.Source: §5 · p. 6, Table 1 · p. 6

Training hyperparameters

  • All models use AdamW with zero weight decay, gradient clipping at 1.0, and a 2×10−4 learning rate; bait/recovery pairs train alone for two epochs before one epoch of neighboring-pair tuning.The shared loss weights are λrec=0.5, λvis=4, λfisher=2, and τvis=0.20.Source: §5 · p. 6

Evaluation metrics

  • The main attack metric is minimum refusal rate across 100 harmful prompts at each KL threshold, alongside held-out general-prompt KL and paired MMLU-Redux and IFEval evaluations.Higher minimum refusal rates indicate stronger defense; capability evaluations use matched settings for each clean/defended pair.Source: §5 · p. 6

6 Results

Across four models, Bait-and-Recover lowers Fisher separability in attacker-visible residuals while recovery residuals stay near reference and general benchmarks remain broadly stable. At KL ≤0.10, it raises mean minimum refusal from 16.25% to 71.75%.

6.1 Fisher Suppression and Behavior Preservation

  • After bait training, attacker-exposed residuals show lower Fisher separability while following recovery residuals stay close to the previous reference model.Figure 2 visualizes layer-wise Fisher retention during training.Source: §6.1 · p. 6, Figure 2 · p. 7
  • Pass B general-prompt KL remains below 0.20 for all four models; Qwen3-8B has the least movement and Gemma-3-12B the most.The amount of general-prompt movement differs by architecture.Source: §6.1 · p. 6

6.2 General Capability Retention

  • MMLU-Redux accuracy deltas range from −0.0022 to +0.0013 (mean −0.0010), while IFEval prompt-strict accuracy changes −0.0055 on average.The paired benchmark evaluations support broad capability retention across the four clean/defended checkpoints.Source: §6.2 · p. 7

6.3 Heretic Attack Evaluation

  • At KL ≤0.10, Bait-and-Recover raises mean minimum refusal from 16.25% to 71.75% (+55.50; 4.4×), with gains on all four models.Table 3 reports minimum refusal rates across four KL budgets; KL ≤0.10 is the primary evaluation budget.Source: §6.3 · p. 7, Table 3 · p. 8
  • At KL ≤1.0, the mean defended score remains 54.00 (+50.25 over baseline); Gemma-3-12B-IT keeps its per-layer defended score at 82 across KL budgets.Its global defended score falls from 83 to 59 only at KL ≤1.0.Source: §6.3 · pp. 7–8
  • At KL ≤0.10, mean defended refusal is 72.00% under global search versus 83.25% under per-layer search; global search is the most effective scope against every defended model.For Gemma-3-12B-IT, global and per-layer defended scores are nearly tied at 83 and 82.Source: §6.3 · p. 8, Table 4 · p. 8

7 Conclusion

Bait-and-Recover protects open-weight models from white-box editing jailbreaks by decoupling the attacker’s observation path from downstream behavior. Across four models, it raises refusal rates under a strict behavior-preservation budget while having negligible impact on general capability benchmarks.

7 Conclusion

  • At KL ≤0.10, mean minimum refusal rate against Heretic searches rises from 16.25% to 71.75% across four open-weight models.The models are from the Qwen3 and Gemma-3 families; the defense has negligible impact on general capability benchmarks.Source: §7 · p. 8
  • Bait-and-Recover places a bait adapter at the attacker’s observation layer and a paired recovery adapter at the next layer.Gradient routing trains the adapters to poison the measured residual signal while restoring clean downstream computation.Source: §7 · p. 8
  • The defense is a practical weight-level complement to behavior-level safety training, distributed as a single merged checkpoint for standard causal-LM inference.Its stated purpose is to invalidate the measurement assumption behind representation-engineering attacks.Source: §7 · p. 8

Limitations

The evaluation leaves several boundaries on the defense’s demonstrated scope: estimator choice, edit-search distortion, fine-tuning attacks, and direct comparisons with other defenses.

Limitations

  • The defense is evaluated against Heretic’s default difference-of-means estimator, not robust or multivariate alternatives such as geometric medians, PCA, or LDA.Estimator-agnostic robustness remains an open question.Source: Limitations · p. 8
  • The conclusions apply to low-distortion edit searches, not arbitrary model changes; looser KL budgets can allow attackers to discover bypass edits.The defense targets the strict behavior-preservation setting of KL ≤0.10.Source: Limitations · pp. 8–9
  • The paper does not claim protection against attackers who collect harmful training data and run additional fine-tuning.Supervised fine-tuning can still overwrite safety behaviors, and a small number of adversarial examples can undo alignment.Source: Limitations · p. 9
  • The evaluation does not directly compare Bait-and-Recover with prior weight- or representation-level safety defenses.Those defenses were evaluated under different attack settings; cross-defense evaluations and combination studies are left to future work.Source: Limitations · p. 9

Ethics Statement

The paper reports risk-mitigation steps for its dual-use study and advises capability evaluation for deployment. Its Gemma-3-12B layer-prior evidence identifies a selected bait band using a clean-model Fisher diagnostic.

Ethics Statement

  • The study reports aggregate statistics, omits harmful generations, and avoids releasing jailbroken checkpoints.Because the defense modifies model weights, practical deployment should be paired with capability evaluations to verify benign utility is preserved.Source: Ethics Statement · p. 9

A Gemma-3-12B Layer-prior Evidence

  • Figure 3’s clean-model Fisher diagnostic selects Gemma-3-12B’s bait band as layers [18, 38] within the default [18, 46] prior window.The contiguous-band rule selects the highest-scoring 21-layer sub-band with a fixed Npair = 21 budget.Source: A Gemma-3-12B Layer-prior Evidence · p. 12
  • Tables 5–7 provide full overall, global-direction, and per-layer-direction Heretic results.Table 5 identifies the four model abbreviations used in the overall results.Source: Table 5 · p. 12, Table 6 · p. 12, Table 7 · p. 12

B Detailed Heretic Evaluation Results

The appendix reports detailed Heretic evaluation results and raw Fisher separability scores. The evaluation tables encode baseline, defended, and difference values, while the Fisher values correspond to the validation split shown in Figure 2.

B Detailed Heretic Evaluation Results

C Qwen3-4B-Instruct Raw Fisher Values

D Reproducibility Details

The reproducibility setup specifies model implementation and training, disjoint data splits, and consistent attack and capability-evaluation procedures.

D Reproducibility Details

  • Implementation uses PyTorch (Paszke et al., 2019) and Hugging Face Transformers, with bfloat16 training and device-mapped parallelism for larger checkpoints.The trained adapters are merged into the base weights, allowing standard causal-LM inference without custom code.Source: D Reproducibility Details · p. 12
  • Layer-prior extraction and adapter training use final-token residual activations from ModelScope-Alpaca and SafeMT training splits disjoint from Heretic evaluation prompts.Source: D Reproducibility Details · pp. 12–13
  • Table 8 reports raw Fisher values for Qwen3-4B-Instruct, with “Ref.” identifying the before-pass reference used as the retention denominator.Source: Table 8 · p. 13
  • Heretic attacks use 200 Optuna trials per attack, while clean and defended checkpoints share chat templates, generation lengths, and post-processing for MMLU-Redux and IFEval.Source: D Reproducibility Details · p. 13

Related papers

Loading 2609.05794v1…