PaperlayerView exact revision on arXiv

Source-linked AI summary

Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration

Aashiq Muhamed, Mona T. Diab, Virginia Smith

arXiv:2609.16204v1cs.LGcs.CLcs.CR

TL;DR

Refusal Feature Ablation can remove safety refusal from open-weight models cheaply, while conventional defenses require expensive per-checkpoint finetuning. DDO edits frozen weights to inject orthogonal MLP decoys that corrupt the attacker’s contrastive estimator. It achieves below-10% standard-RFA ASR across six model families, while reducing Heretic ASR from 88.7% to 18% and costing 30–450× less than trained baselines.

  • Problem

    Refusal Feature Ablation can remove refusal from open-weight models while preserving capability, whereas existing defenses require repeated, costly safety finetuning for new checkpoints.

  • Method

    DDO freezes base weights and repurposes low-impact MLP neurons to inject refusal-orthogonal decoys that bias contrastive refusal-direction estimation.

  • Results

    DDO achieves < 10% standard-RFA ASR across six model families and reduces Heretic ASR from 88.7% to 18% on Llama-3-8B-Instruct.

  • Takeaways & Limitations

    DDO provides a low-cost, post-hoc hardening approach that preserves the underlying refusal mechanism without architectural or runtime overhead.

  • Takeaways & Limitations

    Effective decoy rank is a diagnostic rather than a guarantee against adaptive re-estimation, and the threat model excludes gradient-based finetuning.

Abstract

from arXiv · show

Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA), a technique that identifies and projects out a linear refusal direction from the residual stream, often achieving a high attack success rate (ASR) while preserving model capability. Defending against these attacks typically requires computationally expensive safety finetuning for every new checkpoint. We introduce Decoy Direction Optimization (DDO), a fast, post-hoc weight-editing defense that requires no base-model finetuning. Our approach is based on a simple mechanistic insight: ablation attacks rely on contrastive estimators to find the refusal direction. Rather than trying to hide the true refusal circuitry, DDO actively injects a high-magnitude, nonlinear decoy signal into the network's MLP neurons. When an attacker attempts to locate the refusal direction, the decoy corrupts their estimator, tricking them into ablating a harmless orthogonal feature while the actual safety mechanism remains intact. We prove a spectral bound formalizing this effect and evaluate DDO across six model families, achieving <10% ASR under standard RFA. On Llama-3-8B-Instruct, DDO remains comparable to trained defenses under adaptive multi-phase attacks (65% vs. 58% worst-case ASR) and reduces Heretic weight-level attack ASR from 88.7% to 18%, all at 30 to 450 times lower optimization cost per configuration than the trained baselines.

1 Introduction

DDO is a post-hoc defense that corrupts attackers’ refusal-direction estimators with orthogonal decoys instead of retraining each checkpoint. It achieves low standard-RFA ASR across model families while substantially reducing optimization cost.

  • DDO matches trained defenses under standard RFA at 30–450× lower cost, using a single GPU, roughly two minutes, no training data, and frozen base weights.
  • DDO injects gated, refusal-orthogonal decoy directions through low-impact MLP neurons, causing RFA to target decoys rather than causal refusal features.The defense targets the attacker’s contrastive estimator, not the refusal feature itself.
  • The paper formalizes DDO with subspace-overlap bounds and defines effective decoy rank to characterize fixed attacks and interpret adaptive phase budgets.Under a fixed decoy-energy budget, evenly spreading energy across k directions maximizes the k-th singular value for rank-k attacks.
  • DDO lowers standard-RFA ASR below 10% across six model families without base-model finetuning.
  • On Llama-3-8B-Instruct, DDO reduces standard-RFA ASR from 85% to 1.8% while preserving benign compliance.

2 Threat Model and Attacks

The paper models capability-preserving refusal removal against released open-weight checkpoints and evaluates training-free, post-hoc attacks of increasing sophistication. These attacks include residual-stream direction ablation, weight-level projections, and adaptive multi-phase estimation, with robustness assessed alongside model utility.

  • Threat model: The threat model considers resource-constrained, white-box, adaptive attackers who inspect released weights, collect probe activations, and modify the checkpoint or inference process.The attacker seeks refusal removal while retaining coherent generation and general capability.
  • Threat model: Capability-preserving attack success is evaluated jointly with MT-Bench, MMLU, and XSTest because degrading model quality can artificially reduce ASR.The evaluation uses three tiers of training-free attacks forming a ladder of increasing sophistication within the post-hoc, no-finetuning regime.
  • Standard RFA: Standard RFA estimates a per-layer difference-in-means direction and projects it out of the residual stream, with three-point and residual-stream variants evaluated on JailbreakBench and HarmBench.The defender’s reference refusal direction is distinguished from the attacker’s estimate on the released model.
  • Heretic: Heretic performs automated weight-level refusal removal using Optuna-optimized low-rank projections on attention and MLP down-projection weights, reporting the maximum-ASR trial among 200 trials.Its ablation uses Wablated = W − α(ℓ) d̂(d̂^⊤W), with searched direction and per-layer strength.
  • Adaptive multi-phase RFA: Adaptive multi-phase RFA recomputes a fresh DIM vector after earlier ablations, removes components along previously selected directions, and accumulates directions simultaneously with rank at most t per layer.Here, t indexes adaptive phases, k denotes attack rank, and K denotes the number of decoy reader groups.

3 Method: Decoy Direction Optimization (DDO)

DDO is a post-hoc defense that edits a small set of MLP neurons to inject refusal-orthogonal decoys, corrupting contrastive refusal estimators while preserving the underlying refusal computation. Its optimization and spectral analysis jointly control estimator confusion, benign retention, refusal preservation, and protection across attack ranks.

  • Gated decoy architecture: DDO freezes base weights and repurposes selected SwiGLU/GeGLU neurons to write gated, refusal-orthogonal decoy shifts into the residual stream.The edits are compiled into deployed weights, adding no architectural or runtime overhead.
  • Gated decoy architecture: The gated read–write map reads a refusal coordinate on harmful prompts and produces larger decoy shifts through a nonlinear SiLU gate.The gate suppresses negative refusal coordinates and grows approximately as βa^2 for large positive coordinates.
  • Optimization: DDO optimizes decoy directions with refusal-preservation, benign-retention, estimator-confusion, and first-token-anchoring losses while keeping base weights frozen.The estimator-confusion loss simulates self-RFA, whereas first-token anchoring prevents sequence-level losses from being satisfied by delayed refusal pivots.
  • Deployment considerations: DDO must be placed at or before the model’s causal refusal zone because placement inside that zone can disrupt baseline refusal and increase attack vulnerability.The causal zone is identified by layers whose ablation reduces refusal, with localized and distributed refusal producing different placement constraints.
  • Spectral analysis: The spectral bound shows that stronger decoy modes relative to residual magnitude reduce attacker overlap with the causal refusal subspace.DDO approximates the ideal condition by enforcing decoy directions orthogonal to the refusal direction.
  • Spectral analysis: Protection against higher-rank attacks requires distributing decoy energy across more modes, trading individual mode strength for effective decoy rank.For a fixed contrast matrix, the effective rank counts modes sufficient to keep refusal overlap below a target, but adaptive RFA can change the contrast between phases.

4 Experiments and Results

The experiments evaluate DDO against trained defenses on Llama-3-8B-Instruct and across five additional instruction-tuned model families using a common attack and utility protocol. The comparison reports robustness, capability, benign compliance, and over-refusal under deterministic evaluation with held-out validation and test splits.

  • Models and baselines: The study evaluates DDO on Llama-3-8B-Instruct with the full attack ladder and on five additional instruction-tuned model families for cross-model generalization.The Llama-3 comparison includes six trained defenses, while the cross-model evaluation compares DDO with four trained baselines.
  • Evaluation protocol: All reported metrics use deterministic greedy decoding, held-out validation for tuning and DIM estimation, held-out test sets, and three judges for ASR unless otherwise specified.The protocol also includes MMLU, MT-Bench, XSTest, DirectRequest, and jailbreak evaluations, with coherence failures excluded.
  • Experimental setup: Table 2 compares DDO with trained defenses on Llama-3-8B-Instruct across ASR, MT-Bench, XSTest, DirectRequest, and Heretic-related measures.The table uses three-judge average ASR and reports utility, benign compliance, and attack robustness under the evaluation protocol.

Appendix F.7.

DDO combines estimator-confusing decoys with lightweight post-hoc optimization and performs strongly across standard, adaptive, prompt-level, and weight-level attacks, while retaining important utility trade-offs and rank-dependent limitations.

  • Standard RFA and Heretic: 1.8% mean ASR on standard RFA, with DDO preserving 91.6% XSTest compliance and debiasing improving ASR to 1.0% and XSTest to 99.2%.DDO requires no finetuning and takes approximately 2 minutes per optimization run; under Heretic, DDO achieves 18% H-200 ASR versus 1.3% for RepBend.
  • Adaptive multi-phase RFA: 65% worst-case ASR under adaptive multi-phase RFA places DDO near RepBend at 58% and ReFAT at 60%, while preserving MT-Bench at 5.82 through Phase 8.All evaluated defenses degrade as the attacker re-estimates, but DDO remains in the same ASR–utility regime as the strongest trained defenses without finetuning.
  • Cross-model generalization: Across six model families, DDO achieves < 10% mean standard-RFA ASR without base-model finetuning, with the lowest RFA among utility-preserving defenses on Qwen3 and GLM-4.On Gemma-2, DDO matches LAT’s 0% RFA while preserving higher XSTest compliance, 83.2% versus 51.2%.
  • Prompt-level jailbreaks: DDO + debiasing achieves 0.4% GCG ASR and 1.2% HumanJailbreaks ASR, while PAIR remains high at 36.3%.The authors suggest optimization-based prompt attacks may interact with the same low-dimensional refusal feature targeted by RFA.
  • Ablations: Removing confusion or refusal-score losses raises ASR to 43.2% or 41.8%, while removing retain KL raises it to 22.4%, showing that the objective components have unequal importance.The confusion loss targets DIM-estimator deception, refusal-score loss prevents first-token collapse, and retain KL stabilizes optimization.
  • Discussion and limitations: Single-reader DDO protects rank-1 attacks at 4% ASR but degrades to 39% at rank 16; diversifying readers improves higher-rank robustness while weakening rank-1 protection.Across architectures, DDO is the only evaluated single mechanism reported to achieve < 10% ASR reliably, but weight-level attacks remain easier than standard RFA for DDO alone.

A Limitations and Future Work

The paper identifies adaptive attacks, weight-level optimization, over-refusal, compile-mode selection, and broader dual-use risks as boundaries for DDO’s robustness and deployment scope.

  • Adaptive attackers: 65% worst-case ASR shows DDO remains vulnerable under sustained eight-phase adaptive re-estimation, compared with 58% for RepBend.Longer phase budgets, joint multi-direction projections, and estimators beyond DIM remain untested.
  • Weight-level attacks: 18% Heretic ASR improves over several trained baselines but remains above RepBend’s 1.3%, exposing a post-hoc gap against weight-level optimization.The paper attributes RepBend’s advantage to gradient-based parameter–behavior entanglement that is difficult to reproduce with post-hoc edits.
  • Over-refusal on some models: DDO can reduce benign compliance on some models, although trained defenses such as LAT and ReFAT show similar or larger reductions on Llama-3.Orthogonal debiasing and additive compilation are proposed as utility-repair measures.
  • Compile mode selection: Choosing replace versus additive compilation currently requires evaluating both modes against ASR and XSTest, despite being cheaper than baseline finetuning.A predictive rule based on properties such as refusal redundancy could reduce repeated evaluations.
  • Broader impacts: DDO is presented as a cost-raising hardening layer rather than a complete safety guarantee, with residual attack surfaces and dual-use concerns disclosed.The paper does not release uncensored checkpoints or guardrail-removal recipes, and uses established public benchmarks.

D.2 All Mechanism Definitions

The mechanisms explore how to strengthen, diversify, or reroute refusal signals, while DDO specifically uses nonlinear decoys to corrupt contrastive estimators rather than hiding refusal in a linear transformation.

  • Linear and rerouting mechanisms: Linear decoy shear, refusal rotation, and representation rerouting remain recoverable because adaptive attackers can re-estimate a linear or separable refusal direction.Reported Llama-3 ASR is 92% for decoy shear, 68% for rotation, and 86% for representation rerouting.
  • Signal amplification: Several auxiliary mechanisms amplify refusal signals or route refusal coordinates through embeddings, normalization gains, attention values, or selected heads.These include autoregressive embedding boosts, RMSNorm gain control, value-projection conditioning, KV strengthening, and head-output amplification.
  • Utility and output calibration: Orthogonal debiasing projects out a benign over-refusal direction, while LM-head scaling adjusts refusal and compliance token logits without changing hidden activations.These mechanisms target utility repair or additional refusal headroom rather than the core decoy estimator-confusion mechanism.
  • Spectral mechanism: DDO’s spectral guarantee requires decoy-dominated contrast, with decoy singular values exceeding the residual refusal-plus-error operator norm.The bound formalizes why attacker–refusal overlap decreases when the decoy subspace dominates the contrastive estimator.
  • Multi-direction decoys: Diversifying decoy readers can strengthen protection against higher-rank attacks by spreading energy across multiple significant singular directions.Under a fixed energy budget, evenly distributed decoy directions maximize the relevant lower singular values.

E.3 Empirical Spectral Analysis

The spectral analysis validates that diversified DDO readers flatten the decoy response spectrum, trading some rank-1 protection for stronger protection against higher-rank ablations.

  • Reader diversification: Without diversified readers, adding K neurons still yields an effectively rank-1 response because all neurons read the same refusal coordinate.Distinct trigger directions allow the empirical decoy response matrix to acquire up to K significant singular values.
  • Rank tradeoff: 0–1% ASR at attack ranks 2–8 is achieved with K=2–4 decoy neurons, while rank-1 ASR rises from 10% at K=1 to 27% at K=4.Increasing K spreads decoy energy across directions, weakening rank-1 protection but strengthening higher-rank protection.
  • Empirical singular spectrum: 934:1 to 15:1 is the reduction in σ1/σ2 when diversified readers replace a single reader, confirming a flatter decoy spectrum.The single-reader configuration has effective rank 1, whereas diversification produces effective rank greater than 1.
  • Evaluation setup: The evaluation uses deterministic standard, adaptive, and Heretic attack procedures with shared probe and direction-estimation settings across reported configurations.Adaptive RFA re-estimates and orthogonalizes directions across up to eight phases, while Heretic performs weight-level optimization over selected matrices.
  • Compilation conditions: The preferred DDO compilation mode is selected per model because replace mode can disrupt original neuron computation while additive mode preserves it with a weaker signal.The mode comparison spans seven models and uses ASR plus coherence and benign-compliance constraints.

F.5 Layer Placement Analysis

DDO placement is most effective when decoys begin at or before the causal refusal zone, preserving refusal computation while contaminating the attacker’s estimator.

  • Upstream placement principle: Placing decoys at or before the causal zone contaminates estimation without disrupting refusal, whereas placement inside the zone can increase attack vulnerability.Causal zones are identified by single-layer ablations that measure refusal loss above a 10% threshold.
  • Placement and baseline refusal: 93% →68%, 37% →19%, and 58% →3% are baseline-refusal drops when decoys are placed inside causal zones for Gemma-2, Qwen3, and Mistral.Upstream placement generally preserves refusal better, especially when refusal is localized.
  • Compile mode: Replace mode produces a stronger decoy but can eliminate or disrupt refusal, while additive mode preserves original computation with a weaker mixed signal.Gemma-2 fails coherence under replace; Mistral nearly loses refusal under replace; GLM-4 shows the opposite mode preference.
  • Model dependence: The best compilation mode is model-specific rather than determined by a simple redundancy rule.The paper evaluates both modes using model-specific ASR, refusal, coherence, and benign-compliance criteria.

F.7 Benchmark Settings

The evaluation uses standardized decoding, benchmark protocols, held-out splits, and reproducible baseline-training sweeps across six model families. Baselines are trained with LoRA and compared using model-specific layer targeting and configuration searches.

  • Attack evaluation: Standard-RFA results average four attack variants spanning activation locations and JailbreakBench or HarmBench probe sets.Adaptive RFA and designated diagnostic studies use LlamaGuard-2 alone.
  • Benchmark protocols: Evaluation uses greedy decoding, fixed output limits, and single-A100 timing, while benchmark metrics include MMLU, MT-Bench, and XSTest.MMLU uses five-shot evaluation over 57 subjects, MT-Bench uses 80 multi-turn questions with GPT-4 judging, and XSTest measures over-refusal on 250 safe prompts.
  • DDO settings: DDO hyperparameters are Optuna-tuned per model, with Table 13 covering six parameters and distinct replace or additive compile modes.The table documents model-specific tuning rather than a single universal configuration.
  • Data splits: Hyperparameter tuning and DIM estimation use held-out validation probes, while benchmark metrics are reported on held-out test sets.DDO DIM directions are computed per layer from post-attention-layernorm MLP inputs using unfiltered estimation and float64 accumulation.
  • Training setup: Six trained defenses are evaluated across six model families using LoRA finetuning, standardized optimization settings, and per-defense configuration sweeps.The comparison includes Circuit Breakers, RepBend, Triplet, Triplet-Adv, LAT, and ReFAT.
  • Baseline targeting: Baseline defenses target model-specific layer windows, generally covering the final 30–50% of network depth.Circuit Breakers uses listed ranges, whereas RepBend and Triplet use sliding windows defined by RB_START and RB_WIN.

F.10 Adaptive Attack: Exact ASR and MT-Bench by Phase

Adaptive multi-phase RFA progressively trades model utility for attack success, with DDO remaining comparable to trained defenses across the eight-phase trajectory. DDO reaches 65% worst-case ASR while maintaining MT-Bench above 5.8 through Phase 8.

  • Attack cost: The adaptive attack costs approximately 2 minutes per phase and 15–20 minutes for the full eight-phase loop on one A100 GPU.Each phase includes DIM re-estimation, orthogonalization, hook installation, and evaluation.
  • Utility trade-off: DDO’s Llama-3 MT-Bench score drops from 7.67 unattacked to 6.17 at Phase 5 and 5.82 at Phase 8, alongside a peak 65% ASR.The trajectory represents an approximately 1.9-point utility cost under multi-phase ablation.
  • Adaptive attack results: 65% worst-case ASR makes DDO rank 8 comparable to RepBend’s 58% under eight-phase adaptive RFA, without base-model training.Worst-case ASR is the maximum across phases because the attacker may stop at any phase.
  • Utility trajectory: MT-Bench remains above 5.8 for DDO through all eight phases, comparable to RepBend and Circuit Breakers.ReFAT instead falls to 2.5 at Phase 1 despite low ASR.
  • Trajectory comparison: DDO preserves utility comparably to trained defenses while ReFAT collapses early into a low-ASR but unusable region.Figure 4 summarizes the ASR–MT-Bench trajectories across attack phases.

F.12 Cross-Model Mechanism Comparison and Negative Results

Across architectures, DDO is the only evaluated mechanism achieving below 10% ASR everywhere, while several alternative edits either fail under re-estimation, damage generation, or target the wrong threat model. The results emphasize nonlinear optimization and utility preservation as key design constraints.

  • Optimization ablation: Random orthogonal decoys achieve 2% ASR on Yi but fail on most other models at 18–53%, showing that DDO’s optimization is essential.The random-decoy variant does not use gradient optimization.
  • Negative results: Linear decoy transformations are defeated by adaptive DIM re-estimation, with Decoy Shear Transform at 92%, Rotation at 68%, and Representation Rerouting at 86% ASR.The attacker re-estimates a direction that tracks the transformed mean difference or remains linearly separable.
  • Decoy budget: Spreading decoy energy across more directions trades rank-1 protection for higher-rank robustness when readers are diversified, but merely dilutes each direction with identical readers.The result follows from the fixed decoy-energy budget and effective-rank constraint.
  • Cross-model comparison: DDO is the only mechanism achieving <10% ASR across all seven architectures in the cross-model mechanism comparison.Table 19 aggregates eight mechanisms using Optuna-tuned evaluations across 10–30 trials per mechanism.
  • Utility failures: Aggressive edits can achieve 0.7% RFA ASR while destroying utility, as LM-Head Row Scaling yields 8.0% XSTest and 1.72 MT-Bench.Extending DDO injection to layers 0–11 also produced 48% ASR and incoherent outputs.
  • Threat-model mismatch: Gradient landscape roughening reaches 84% ASR against RFA, indicating that input-space defenses do not transfer to representation-space abliteration.The mechanism targets GCG-style prompt optimization rather than representation-space attacks.
  • Attack and defense surfaces: DDO edits SwiGLU MLP matrices Wgate, Wup, and Wdown, with only Wdown overlapping Heretic’s Wo and Wdown attack surface.RFA instead hooks activations at three points, while Heretic edits weights offline.

F.15 Additional Model Evaluation

Additional evaluation across seven instruction-tuned checkpoints tests whether DDO generalizes beyond the primary models. DDO achieves below 10% standard-RFA ASR on every additional model while keeping MMLU close to baseline.

  • Cross-model generalization: DDO achieves <10% standard-RFA ASR on all seven additional checkpoints spanning 7B–24B parameters.The models include Llama-2-7B-Chat and are tuned per model with approximately 10 minutes of Optuna search each.
  • Capability preservation: MMLU decreases by no more than 1.4 points on the additional models after DDO defense.The table reports base-to-DDO MMLU values alongside mean RFA ASR across four variants.

F.16 DDO Undo Attack Analysis

DDO frustrates checkpoint-only undo attacks by rotating the attacker’s estimated refusal direction away from the true refusal direction, making modified neurons difficult to identify. Standard undo heuristics recover little attack performance, although higher-rank attacks substantially weaken protection.

  • Geometric detectability: cos(ˆr_clean, ˆd_atk) stays between −0.05 and +0.05 in mid-zone target layers across Qwen3, Mistral-7B, and Yi, indicating near-orthogonal estimated directions.This estimator rotation prevents the modified neuron’s gate row from standing out along the attacker’s scoring direction.
  • Geometric detectability: The modified neuron falls within natural refusal-neuron rankings: it is outranked at Mistral-7B L10 and has only 1.04×–1.48× margins in the reported Qwen3 and Yi layers.At the deepest Yi layer, the margin can reach about 5×, but unknown modified layers make reliable whole-model undo difficult.
  • Undo heuristics: 12–17% ASR after gate-based undo heuristics and 4–7% after down-column-norm undo remain below the 80% undefended base.Table 21 evaluates each heuristic followed by standard 3-point RFA on Llama-3-8B DDO.
  • Camouflage considerations: β≈3–6 gate/up norm ratios and cosine similarity of 1.0 are compile artifacts that can be camouflaged by redistributing gain or adding a small orthogonal perturbation.The paper notes that all evaluated models use the default compile without these norm or cosine camouflage modifications.
  • Rank-k attacks: 79% ASR at rank k=32 approaches the undefended model’s 80%, whereas rank-k attacks achieve 4% at k=1, 29% at k=8, and 39% at k=16.The sweep shows decreasing protection as the attacker ablates more singular directions.
  • Probe-budget robustness: 6–12% ASR persists as probe budgets rise from 32 to 1024 prompts per class, with no sustained increase and 7% at both 512 and 1024.The result supports genuine corruption of the mean-difference direction rather than reliance on finite-sample estimator noise.
Loading 2609.16204v1…