PaperlayerView exact revision on arXiv

Source-linked AI summary

SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior

Mingyue Cui, Linghui Shen, Xingyi Yang

arXiv:2606.18322v1cs.LGcs.AI

TL;DR

SAE defenses assume that clamping unsafe features reliably eliminates unwanted behavior, but evidence for behavioral completeness remains limited. This paper tests post-intervention recovery under active clamps and finds recoverable behavior across tasks, including 95.8% recovery in refusal steering.

  • Problem

    SAE-based defenses treat selected unsafe features as reliable intervention targets, but whether they form complete behavioral bottlenecks remains insufficiently examined.

  • Method

    The paper optimizes residual perturbations from defended states to recover pre-intervention behavior while preserving targeted SAE features under active clamps.

  • Results

    Across latent-, output-, circuit-, and refusal settings, recovery remained possible; refusal steering achieved 95.8% recovery on strict-valid samples.

  • Takeaways & Limitations

    SAE features can be useful causal handles without constituting complete behavioral bottlenecks, because recovery can exploit information in the SAE reconstruction residual.

  • Takeaways & Limitations

    The authors limit the concern to over-reliance on selected SAE features as exclusive bottlenecks, while retaining SAEs’ usefulness for diagnosis and local causal editing.

Abstract

from arXiv · show

Sparse Autoencoders (SAEs) decompose residual-stream activations into interpretable features. Recent latent-space defenses increasingly rely on these decompositions, assuming that identified "unsafe" SAE features serve as actionable handles for monitoring and intervention. In this paradigm, clamping a specific harmful feature is expected to reliably prevent model misbehavior. However, we show that this success may hide a recoverable failure mode: the clamp may block one visible route to a behavior without eliminating the behavior itself. We formulate this vulnerability as post-intervention recovery, a constrained residual-space optimization problem. Starting from the post-intervention residual state, we optimize residual perturbations to recover the pre-intervention behavior while preserving the post-intervention values of the targeted SAE features. Even under a strong threat model where the intervention remains active throughout optimization and generation, recovery remains possible. To rule out that recovery simply undoes the intervention, we use encoder-orthogonal updates for single-layer interventions and the corresponding feature-map Jacobian in the cross-layer setting. Across TPP, unlearning, IOI, and refusal steering experiments, this stress test reveals recoverable behavior despite successful feature-level intervention. Especially in the safety-critical refusal-steering setting, we achieve a 95.8% recovery rate on valid samples while keeping defended-feature relative drift to 0.131, substantially below suffix-based baselines. A recovery-path attribution analysis further localizes this recovery to the SAE reconstruction residual, the component left unexplained by the SAE. These results expose a gap between feature-level control and behavioral completeness: SAE features can support causal intervention, but controlling them does not guarantee control over the underlying behavior.

1 Introduction

The introduction argues that clamping unsafe SAE features may block a salient route without eliminating the underlying behavior. It therefore proposes post-intervention recovery as a constrained residual-space diagnostic that tests behavioral restoration while targeted features remain suppressed.

  • Motivation: SAEs provide sparse, interpretable residual-stream features that appear to offer concrete handles for analyzing, monitoring, and controlling model behavior.
  • Problem: Suppressing a targeted SAE feature may redirect behavior through alternative directions, downstream layers, or distributed mechanisms rather than eliminate the behavior itself.
  • Method: Post-intervention recovery starts from the clamped residual state and optimizes small perturbations to restore pre-intervention behavior without simply undoing the clamp.
  • Geometric Constraints via Projected Gradient Descent: Encoder-orthogonal updates constrain single-layer recovery, while feature-map Jacobians constrain how cross-layer perturbations affect targeted features.
  • Results: 74.9% behavioral recovery rate was achieved for TPP on SAEBench while targeted-feature reactivation was restricted to 0.002.

2 Related Work

Prior work establishes SAE features as interpretable but incomplete causal handles and distinguishes monitor bypass from the later intervention stage. SAE refusal steering provides a diagnostic setting for testing recovery with null-space constraints adapted from AlphaSteer.

  • SAE features as useful but incomplete handles: SAE features support editing, steering, and circuit analysis, but causal relevance does not imply that they completely capture behaviorally relevant information.Prior work on superposition, SAE geometry, feature absorption, feature hedging, and sparse feature circuits suggests relevant information can be distributed across features.
  • From monitor bypass to post-intervention recovery: OABD studies monitor bypass, whereas this work examines whether suppressed behavior persists after harmful features have already been selected and clamped.The intervention remains active during the later clamp-stage question.
  • SAE refusal steering and constrained recovery: SAE refusal steering identifies refusal-associated features and amplifies or suppresses them during inference, making it a diagnostic setting for testing whether clamping eliminates non-refusal behavior.This setting directly motivates the question of behavioral completeness after refusal-feature intervention.
  • SAE refusal steering and constrained recovery: Recovery updates are projected into the null space of selected SAE encoder directions for single-layer interventions and extended across layers using the local feature-map Jacobian.Unlike AlphaSteer’s utility-preserving steering perspective, the null-space idea is adapted for recovery diagnostics.

3 Preliminaries

This section defines SAE features, feature-level interventions, defended residual states, valid flips, and the distinction between causal handles and complete intervention bottlenecks. Recovery is evaluated from post-intervention states only when the intervention has suppressed behavior that the base model exhibits.

  • Sparse autoencoders: SAEs encode residual-stream activations into sparse latent features and reconstruct them from those features.For layer ℓ and input x, zℓ(x) = Eℓ(hℓ(x)) and ĥℓ(x) = Dℓ(zℓ(x)).
  • Feature-level interventions: Feature-level interventions select features S, set them to defended values cS, and preserve the SAE reconstruction residual.Zero ablation uses cS = 0, while refusal clamping may use a nonzero defended value.
  • Feature-level interventions: All recovery experiments begin from the post-intervention defended residual state hdefℓ(x).The defended residual is the residual state produced after applying the feature clamp.
  • Valid flips: Recovery is evaluated only on valid flips, where the base model shows the target behavior but the active SAE intervention suppresses it.This conditioning measures restoration only when there is suppressed behavior to recover.
  • Causal handles versus complete bottlenecks: A causal handle changes target behavior when intervened on, whereas a complete bottleneck permits no admissible residual perturbation to restore it.The experiments test this stronger bottleneck condition rather than causal influence alone.

4 Post-Intervention Recovery

Post-intervention recovery tests whether SAE clamps form complete behavioral bottlenecks by optimizing residual perturbations that restore suppressed behavior while preserving the clamp. The method uses constrained optimization, PGD, and cross-layer Jacobian projections to restrict recovery to alternative computational routes.

  • Recovery Objective: Post-intervention recovery optimizes a small residual perturbation that restores suppressed behavior without altering the active SAE clamp.The defended residual state is the starting point, and the targeted SAE feature remains suppressed throughout recovery.
  • Threat Model: The white-box threat model allows additive perturbations to the defended residual state but forbids changing model weights, removing the clamp, or selecting different clamped features.The intervention remains active during optimization and generation.
  • Recovery Objective: Finding a successful recovery path demonstrates that behavior can bypass the defense through alternative computational routes rather than relying on a single internal route.The clamp stays in place while the original behavior returns through δx.
  • Constrained Optimization: The constrained objective combines behavioral recovery with feature-preservation constraints that prevent direct modification along clamped directions and bound perturbation size.The constraints target encoder orthogonality, preservation of defended feature values, and avoidance of state overwriting.
  • Optimization Procedure: Projected gradient descent approximates the nonlinear constrained search, enforcing encoder orthogonality by projection and perturbation size by norm clipping.Feature-preservation constraints C2 and C3 are evaluated as post-hoc metrics.
  • Cross-Layer Jacobian Projection: For multi-layer interventions, dynamic Jacobian projection removes first-order directions that would change defended SAE features across layers.This extends single-layer encoder projection by avoiding the row space of the joint defended-feature Jacobian.

5 Experiments

The experiments test post-intervention recovery across TPP, WMDP-Bio unlearning, IOI, and refusal steering, evaluating only valid flips where intervention suppresses existing target behavior. Across latent-, output-, and circuit-level readouts, encoder-orthogonal recovery restores behavior while reducing defended-feature changes, demonstrating that intervention handles are not complete behavioral bottlenecks.

  • Experimental scope: The evaluation spans four settings: TPP, WMDP-Bio unlearning, IOI, and refusal recovery, differing in recovered behavior and measurement level.These settings cover latent-level, output-level, circuit-level, and safety-relevant refusal behavior tests.
  • Experimental protocol: Recovery is evaluated only on valid flips, comparing unconstrained residual optimization with encoder-orthogonal updates that remove selected SAE encoder directions.This conditioning restricts evaluation to examples where the base model shows the target behavior and the active intervention suppresses it.
  • TPP: TPP encoder projection reduces target-mean recovery from 0.819 to 0.749 while lowering mean reactivation from 0.013 to 0.002.It also lowers mean activation drift from 0.094 to 0.039 and raises zero-reactivation recovery from 0.103 to 0.680.
  • WMDP-Bio unlearning: In WMDP-Bio, encoder-projected recovery restores 90/91 valid answer-choice flips while measured clamp-feature drift is zero.Unconstrained recovery restores 91/91 but induces substantially larger post-hoc clamp-feature drift; the strict matched slice contains 91 valid flips.
  • IOI: In IOI, both recovery variants restore the decision on all 37 valid flips, while encoder projection achieves lower activation drift, decode drift, and feature reactivation.Most encoder-projected recoveries have zero measured reactivation of eligible clamped features.

6 Refusal Recovery Case Study

In refusal recovery experiments, Jacobian-projected residual optimization restored non-refusal behavior while preserving defended refusal features under an active clamp. Recovery generalized across benchmarks and feature-set sweeps, with the SAE reconstruction residual identified as its dominant carrier.

  • Setup: The protocol required strict-valid AdvBench prompts where the unclamped model refused not, the active clamp induced refusal, and recovery restored non-refusal behavior while remaining active.The benchmark_our/global feature set with clamp value 3.0 yielded 24 strict-valid examples.
  • Recovery under defended-feature preservation: 23/24 strict-valid prompts were recovered with Jacobian projection while defended-feature drift and clamp-floor violation remained much smaller than with soft-suffix baselines.The cross-layer Jacobian projection preserves the monitored defended-feature state for refusal features distributed across layers.
  • Recovery under defended-feature preservation: 43/43 non-refusal recoveries were obtained on HarmBench-Test under low defended-feature drift, while broader feature-set sweeps reached 42/45 at K = 20.Recovery remained high across the stable operating range K = 5–20.
  • Attributing the recovery path: Residual replay nearly matched full recovery, whereas clamped-feature and non-clamped-feature replays largely failed, identifying the SAE reconstruction residual as the dominant carrier.Removing the top-k non-clamped feature component preserved most recoveries, ruling out compensation through a small set of alternative visible SAE latents.

7 Discussion

SAE feature clamps can suppress behavior without forming complete behavioral bottlenecks, because behavior may re-emerge through SAE reconstruction residual directions. SAE-based defenses therefore require recovery-focused evaluation and broader computational constraints.

  • Causal handles are not complete bottlenecks: A selected SAE feature set can causally change behavior without serving as a complete bottleneck, since behavior can re-emerge from the defended residual state.The experiments distinguish useful causal handles from exclusive control over the underlying behavior.
  • The SAE residual is not inert error: The SAE reconstruction residual may contain behaviorally sufficient information invisible to the selected feature basis, enabling the model to route around an intervention.This remains relevant even when the residual is small or treated as reconstruction error.
  • Implications for SAE-based safety evaluation: SAE-based defenses should be evaluated not only by clamp-induced suppression, but also by whether the defended state remains robust against behavioral recovery.Sparse features remain valuable for diagnosis, mechanistic localization, and local causal editing; the limitation is treating them as exclusive bottlenecks.
  • What stronger defenses would need: Stronger defenses should constrain broader computation through residual-channel monitoring, multi-layer or trajectory-level constraints, or training against post-clamp recovery objectives.Simply enlarging the clamped feature set may improve coverage but does not directly address recovery through SAE-unexplained residual directions and may cause capability or over-refusal side effects.

8 Conclusion

The paper introduces post-intervention recovery as a diagnostic for whether SAE feature interventions create complete behavioral bottlenecks. Across multiple intervention settings, it tests whether clamping truly eliminates suppressed behavior rather than merely changing it.

  • Conclusion: Post-intervention recovery tests whether SAE feature interventions form complete behavioral bottlenecks.The diagnostic evaluates behavioral completeness rather than intervention-induced behavior change alone.
  • Conclusion: The key question is whether suppressed behavior is actually eliminated while the clamp remains active.This differs from asking only whether a selected feature set can change behavior when clamped.
  • Conclusion: Across latent-level, output-level, circuit-level, and refusal settings, suppressed behaviors can often be recovered from the defended residual state.The conclusion reports recoverability across all four listed experimental settings.

A Evaluation Protocol and Metrics · B Additional Standard-Task Results

The evaluation restricts recovery analysis to cases where an SAE intervention genuinely suppresses the target behavior, then measures behavioral recovery alongside preservation of defended features. Refusal experiments use additional readouts to distinguish removal of refusal cues from recovery that remains close to the original response.

  • A Evaluation Protocol and Metrics: Recovery is evaluated only on valid flips where the original model shows the target behavior and the defended model no longer does after intervention.This conditioning ensures that recovery is measured only when the intervention has actually suppressed behavior.
  • A Evaluation Protocol and Metrics: Recovery rate is the primary behavioral metric, while defended-feature drift is the primary feature-preservation metric.Binary recovery is counted only on the valid-flip set V.
  • A Evaluation Protocol and Metrics: Refusal experiments additionally report base-like recovery, defended-feature drift, and clamp-floor violation.Base-like recovery assesses closeness to the original non-refusal response, while clamp-floor violation tracks whether recovery violates the intervention floor.
  • A Evaluation Protocol and Metrics: Non-refusal recovery checks whether outputs avoid explicit refusal or safety-cue openings, but does not guarantee response coherence.Active clamps and recovery updates can perturb the latent trajectory, allowing non-refusal outputs that are not necessarily coherent.
  • A Evaluation Protocol and Metrics: Binary recovery outcomes use Wilson 95% confidence intervals.This uncertainty estimate applies to binary recovery measurements.
  • A Evaluation Protocol and Metrics: Continuous quantities use bootstrap confidence intervals over valid examples when saved per-example values are available.The reported quantities include defended-feature drift, clamp-floor violation, and relative δx norm.

B.1 TPP target-mean results · C Additional Refusal Validity Checks

The appendix evaluates TPP recovery with unconstrained and encoder-projected updates, and validates refusal-recovery results using strict, feature-set-specific filtering. These checks show that recovery is assessed only when the clamp demonstrably suppresses a usable target behavior.

  • B.1 TPP target-mean results: Figure 7 provides a supplementary visualization of recovery–reactivation behavior across official layer-5 TPP targets.The appendix identifies Figure 2 as the primary trade-off visualization and Figure 7 as supplementary.
  • B.1 TPP target-mean results: Encoder projection reduces defended-feature reactivation and activation drift while preserving substantial valid-flip recovery on official layer-5 TPP targets.Table 2 compares unconstrained with encoder-projected recovery using target-mean metrics.
  • C Additional Refusal Validity Checks: Refusal recovery uses strict valid filtering, requiring the unclamped model to produce a non-refusal response and the clamped model to produce a refusal under the same detector.This ensures a suppressed behavior exists before recovery is attempted.
  • C Additional Refusal Validity Checks: Validity is determined separately for each refusal feature set because nominally similar sets can induce substantially different clamp behavior on the same 520 AdvBench prompts.The protocol therefore verifies suppression empirically rather than assuming a feature set is valid by construction.
  • C Additional Refusal Validity Checks: 203 prompts were induced to refuse by benchmark_our, yielding 24 strict valid recovery cases, whereas benchmark_la induced refusals on 48 prompts and yielded 2 strict valid cases.The base generation is computed without a feature clamp, so valid-count differences reflect the clamps’ behavioral effects.
  • C Additional Refusal Validity Checks: Manual inspection found that benchmark_la’s 2 strict valid cases were degenerate generations dominated by repeated punctuation or tokens rather than usable non-refusal answers.A substring detector did not capture this failure mode, so benchmark_la was excluded from the main base-response recovery experiments.
  • C Additional Refusal Validity Checks: Table 3 reports cross-dataset strict-valid refusal recovery on HarmBench-Test using the active SAE clamp, strict-valid filtering, Jacobian projection, and post-hoc defended-feature evaluation.The passage identifies these as the same procedures used in the AdvBench main experiment.
  • C Additional Refusal Validity Checks: Table 4 automatically checks opening categories for 24 AdvBench strict-valid examples, finding clamp outputs mostly begin with full refusals while Jacobian recovery mostly returns to non-refusal-candidate openings.Manual labels are pending, so this is an automatic diagnostic rather than an independent human evaluation.

D Cross-Dataset Refusal Recovery … K Experimental Details and Compute Resources

Across cross-dataset, diagnostic, and implementation checks, recovery remains effective under active SAE clamps while preserving defended features, though response quality and uncertainty require careful qualification. The analyses localize recovery to unexplained residual components and document robustness across feature-set sizes, budgets, and experimental settings.

  • D Cross-Dataset Refusal Recovery: 43/43 HarmBench-Test valid examples recovered non-refusal behavior, including 19/43 base-like and 14/43 strict-base-like recoveries, with defended-feature drift 0.108.The strict filter retained 43 valid examples from 159 prompts; mean clamp-floor violation was 0.102.
  • E Opening-Category Sanity Check: 37 of 43 recovered HarmBench-Test openings were non-refusal-or-other, while 4 were degenerate/repeated and 2 were partial-refusal or safety-cue openings.The diagnostic indicates recovery was not solely driven by one opening pattern, but manual auditing remains necessary.
  • F Redacted Qualitative Example: A redacted case study shows Jacobian-projected recovery restoring a non-refusal, base-like response under the same active clamp that changed the base response into a refusal.The example omits harmful procedural details and is qualitative rather than an additional quantitative result.
  • G Uncertainty Estimates: Wilson 95% confidence intervals are reported for binary recovery outcomes, while bootstrap intervals cover continuous defended-feature metrics when available.These intervals summarize uncertainty over small strict-valid subsets rather than claim asymptotic significance.
  • H Perturbation Scale Diagnostics: Recovery reached 4/6 at budget 5 and full recovery at budget 10, whereas encoder-projected recovery failed at budgets 0 and 2.The matched strict WMDP slice contained six valid answer-choice flips and used post-hoc defended-feature drift evaluation under the active clamp.
  • I Refusal Feature-Set Size Sweep: Recovery remains high across the refusal feature-set sweep, especially for K = 5–20, while broader clamps reduce base-answer fidelity in a side-effect regime.The valid set is recomputed separately for each K-specific clamp, and the sweep covers the full 520-prompt AdvBench slice.
  • J Recovery-Path Decomposition Details: Behavioral replay and knockout analyses attribute recovery primarily to the SAE reconstruction residual rather than clamped refusal features or a small set of alternative latents.Because decoder directions are non-orthogonal, component norms are not interpreted as variance fractions.
  • K Experimental Details and Compute Resources: The experiments use frozen language models and frozen SAEs, optimizing only per-example recovery perturbations or soft suffix baselines; configuration details and scripts are provided in supplements.Table 11 summarizes the model, SAE release, intervention target, recovery objective, and evaluator for each experiment.

L Limitations · M Responsible Release

The study presents post-intervention recovery as a white-box diagnostic whose scope is limited to evaluated SAE defenses, models, and releases rather than a universal impossibility result. Responsible release withholds full harmful completions and provides aggregate statistics and coarse redacted categories instead.

  • L Limitations: The findings establish recoverable paths only in evaluated settings, not that every SAE-based intervention is recoverable.Observed recovery depends on selected features, SAE dictionaries, and model settings.
  • L Limitations: Recovery is a white-box diagnostic requiring internal activations, gradients, and per-input residual optimization, rather than a directly deployable jailbreak.The procedure tests intervention completeness under privileged access.
  • L Limitations: The refusal study’s strict valid-filtering protocol improves interpretability but leaves a relatively small main set of clean recovery examples.This limits how broadly the refusal-case findings can be generalized.
  • L Limitations: Broader evaluation across models, prompts, clamp strengths, and SAE releases is needed to determine the phenomenon’s full scope.The requested expansion follows directly from the study’s filtering and feature-selection limitations.
  • M Responsible Release: The work frames post-intervention recovery as a diagnostic for testing whether defended residual states retain routes to suppressed behavior.It is explicitly not intended as a turnkey jailbreak or deployment attack.
  • M Responsible Release: For safety-relevant refusal experiments, the release reports aggregate recovery statistics and coarse redacted output categories instead of full harmful completions.Experiment artifacts are intentionally restricted accordingly.
Loading 2606.18322v1…