PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Are LLM Backdoor Defenses Fragmented? A Feature-Level Explanation with Sparse Autoencoders

Yizhe Zeng, Chenxu Niu, Wei Zhang, Hao Huang, Yunpeng Li, Dongxu Han, Dan Du, Cheng Hong, Hequn Xian, Yuling Liu

arXiv:2608.30403v1cs.CR

TL;DR

Existing LLM backdoor defenses are fragmented because dirty-label and clean-label attacks exhibit different internal feature compositions. This paper analyzes those mechanisms with sparse autoencoders and validates feature-level mitigation through inference-time clamping, substantially reducing attack success while largely preserving benign performance.

  • Problem

    Existing defenses rely on paradigm-specific assumptions and do not consistently protect against both dirty-label and clean-label backdoors.

  • Method

    The paper uses sparse autoencoders to trace backdoor-induced logit shifts to high-contributing features and categorize their functional roles and compositions.

  • Results

    The analysis finds isolated interaction features dominate dirty-label attacks, while clean-label attacks rely more on heterogeneous mixtures of mixed and weight-modified features; clamping reduces ASR to at most 10.8% and 15.4% in the reported settings.

  • Takeaways & Limitations

    SAE-based feature analysis explains defense fragmentation and provides actionable guidance for interpretable backdoor mitigation while largely preserving benign-task performance.

  • Takeaways & Limitations

    The evaluation covers finite, controlled classification-style scenarios, and the clamping procedure still requires feature selection and calibration rather than being a fully optimized deployment defense.

Abstract

from arXiv · show

Backdoor attacks pose a serious threat to large language models (LLMs), but existing defenses remain fragmented, failing to pro?vide unified defense against both dirty-label and clean-label attacks. To investigate why such fragmentation arises, we present the first systematic feature-level mechanistic analysis of LLM backdoors using sparse autoencoders (SAEs). Starting from a 2 x 2 comparison of clean and poisoned models on clean and triggered inputs, we trace backdoor-induced logit shifts to high-contributing SAE features and categorize them into four roles: interac?tion, suppressed, mixed, and weight-modified features. This taxonomy reveals system?atic encoding differences: dirty-label back?doors are dominated by isolated interaction features, whereas clean-label backdoors rely more on heterogeneous mixtures of mixed and weight-modified features. These differ?ences explain why existing defenses remain fragmented across attack paradigms. We val?idate this hypothesis through inference-time feature clamping, which reduces ASR to at most 10.8% in most dirty-label settings and at most 15.4% in the majority of clean-label settings, while preserving benign-task perfor?mance. These results show that SAE-based analysis can explain defense fragmentation and guide interpretable backdoor mitigation.

1 Introduction

LLM backdoor defenses often fail to generalize between dirty-label and clean-label attacks. This paper uses SAE-based feature analysis to explain the divergence and validates feature clamping as an interpretable mitigation.

  • Existing defenses rely on differing assumptions about backdoor manifestations, limiting unified protection across dirty-label and clean-label attacks.
  • The paper introduces an SAE-based feature-level framework using clean and poisoned models with clean and triggered inputs.
  • Dirty-label backdoors are dominated by isolated interaction features, whereas clean-label backdoors rely more on heterogeneous mixtures of mixed and weight-modified features.
  • Inference-time feature clamping reduces ASR to at most 10.8% in most dirty-label settings and at most 15.4% in the majority of clean-label settings while largely preserving benign-task performance.
  • The contributions include decomposing backdoor behavior into three effects, categorizing SAE features into four types, and translating these findings into feature clamping.

2 Related Work

Prior LLM backdoor defenses use trigger inversion, repair, perturbation, and representation-based detection, while interpretability work localizes safety behavior at increasingly fine-grained levels. SAEs provide a sparse feature interface for analyzing and intervening on such mechanisms.

  • Prior defenses address backdoors through trigger inversion, model repair, input perturbation, and representation-based detection.
  • Safety interpretability has localized behaviors using dense directions, modules, attention heads, neurons, and internal circuits.
  • Coarser analysis granularity makes it difficult to separate backdoor-specific computations from normal model functionality.
  • SAEs decompose dense activations into sparse, more interpretable representations that help address polysemanticity and superposition.

3 Preliminaries

The paper distinguishes dirty-label and clean-label poisoning objectives and defines the metrics and SAE representation used for analysis. Both attack paradigms seek trigger-activated target behavior while preserving clean behavior.

  • Dirty-label attacks trigger source-class samples and relabel them with the attacker-specified target label.
  • Clean-label attacks trigger target-class samples while retaining their original labels.
  • At inference time, both paradigms aim to predict the target label for triggered source-class inputs while preserving clean behavior.
  • SAEs map residual activations into sparse overcomplete feature representations and reconstruct them through decoder directions for attribution and intervention.
  • ASR measures targeted success on triggered inputs, while CACC measures standard task performance on clean test inputs.

4 Understanding Backdoor Mechanisms at Feature Granularity

Cross-evaluation shows that current defenses are tied to attack paradigms, motivating a factorial SAE analysis of backdoor effects. The analysis finds distinct feature compositions for dirty-label and clean-label attacks.

  • 4.1 Motivation: The Fragmentation of Existing Defenses: Existing defenses show asymmetric cross-paradigm performance, with methods effective against one attack type often failing against the other.
  • 4.1 Motivation: The Fragmentation of Existing Defenses: BEEAR and CRoW reduce dirty-label ASR to 17.3% and 15.6% on average but leave clean-label ASR around 90%, while RepGuard reduces clean-label ASR to 30.6% but leaves dirty-label ASR at 91.7%.
  • 4.2 Understanding Backdoor Mechanisms at Feature Granularity: The 2×2 factorial design combines clean or poisoned models with clean or triggered inputs to separate trigger, weight, and interaction effects.
  • 4.2 Understanding Backdoor Mechanisms at Feature Granularity: SAE features are selected by attribution changes and classified from their activation profiles across CC, CT, PC, and PT conditions.
  • 4.2 Understanding Backdoor Mechanisms at Feature Granularity: +15 to +27 joint-condition logit-margin shifts dominate the dirty-label cases, with interaction accounting for at least 86.6% of the total effect.
  • 4.2 Understanding Backdoor Mechanisms at Feature Granularity: Dirty-label attacks contain at least 27 of 60 interaction features, whereas clean-label attacks contain at least 23 of 60 mixed features and more weight-modified features.
  • 4.2 Understanding Backdoor Mechanisms at Feature Granularity: These structural differences explain why separable-component defenses fit dirty-label attacks while representation-reshaping defenses better fit clean-label attacks.

5 Mechanism-Guided Validation of Backdoor Mitigation

The authors validate SAE-identified backdoor features by inference-time clamping, comparing pretrained and clean fine-tuned references across models, datasets, and attack settings. Both variants substantially reduce ASR while largely preserving clean accuracy, and clamping performs more consistently across dirty-label and clean-label attacks than existing defenses.

  • Validation approach: Feature clamping tests whether SAE features identified as backdoor mechanisms can mitigate attacks when their activations are realigned at inference time.The intervention clamps suspicious feature activations to reference values.
  • Validation approach: The evaluation spans three architectures, three datasets, and six attack settings, selecting top-k suspicious features by activation divergence from a small clean calibration set.Base Clamp uses the pretrained base model, while CC Clamp uses a clean fine-tuned model as reference.
  • Clamping results: Both Base Clamp and CC Clamp substantially reduce ASR in most settings while preserving clean accuracy.The Benign rows report clean accuracy for clean models without backdoor attacks.
  • Clamping results: Base Clamp generally reduces ASR below 10% on AG News and SST-2, with a few Llama-3.1-8B-Instruct clean-label cases reaching at most 15%.It also achieves large reductions on LLM-LAT, particularly for phrase-trigger attacks.
  • Clamping results: CC Clamp reduces ASR to at most 15.4% across configurations and below 6% in most cases, while CACC remains largely unchanged.Phrase-trigger attacks on LLM-LAT are almost completely neutralized, and rare-token attacks approach zero ASR in most settings.
  • Reference comparison: CC Clamp is stronger than Base Clamp, but their ASR and CACC differences are near zero on AG News and SST-2; LLM-LAT shows larger variance.A clean fine-tuned reference provides additional gains in some safety-alignment cases.
  • Comparison with existing defenses: Existing defenses show paradigm-specific limitations, whereas Base Clamp and CC Clamp achieve consistently low ASR across dirty-label and clean-label attacks.BEEAR and CRoW work mainly for dirty-label attacks, RepGuard mainly for clean-label attacks, and DeCE provides moderate mitigation in both.

6 Conclusion

The paper presents an SAE-based feature-level analysis that distinguishes how dirty-label and clean-label backdoors are encoded. Feature clamping supports the causal relevance of these features and demonstrates interpretable mitigation while largely preserving benign performance.

  • 6 Conclusion: The analysis traces backdoor-induced logit shifts to high-contributing SAE features and identifies four functional feature roles.The resulting differences in feature composition provide a mechanistic explanation for fragmented defenses across attack paradigms.

Limitations

The study’s evidence is bounded by its controlled experimental settings, potential risks of revealing attack-relevant mechanisms, and the preliminary deployment status of its clamping procedure.

  • Datasets and Settings: The experiments cover finite model, dataset, trigger, and attack combinations, focusing on controlled classification-style settings and limited backdoor objectives.Generalization to larger models, open-ended generation, multi-trigger attacks, and adaptive poisoning remains open.
  • Potential Risks: Revealing feature-level backdoor mechanisms could help adaptive attackers distribute or entangle malicious behavior to evade detection.The authors recommend using SAE analysis for diagnosis, auditing, and mitigation rather than attack optimization.
  • Defense Scope: Inference-time clamping is primarily a causal validation method rather than a fully optimized deployment defense.It still requires feature selection and calibration, and efficiency may depend on SAE size and intervention layer.

B Full Results of Defense Fragmentation

Cross-evaluation on Qwen2.5-7B-Instruct and Gemma-2-9B-IT reproduces the asymmetric defense pattern across dirty-label and clean-label backdoors. No existing method consistently protects against both paradigms.

  • On Qwen2.5-7B-Instruct, BEEAR and CRoW reduce average dirty-label ASR to 10.6% and 10.1%, but clean-label ASR remains 84.7% and 85.2%.
  • RepGuard on Qwen2.5-7B-Instruct reduces average clean-label ASR to 22.4%, while dirty-label ASR remains 84.8%.
  • DeCE provides more balanced Qwen2.5-7B-Instruct mitigation, but residual ASR averages 51.8% for dirty-label and 57.5% for clean-label attacks.
  • On Gemma-2-9B-IT, BEEAR and CRoW reduce average dirty-label ASR to 13.9% and 13.3%, but clean-label ASR remains 87.0% and 87.6%.
  • RepGuard reduces Gemma-2-9B-IT clean-label ASR to 26.8%, while dirty-label ASR remains 87.4%; DeCE leaves averages of 56.4% and 60.6%.
  • Together with Llama results, these findings show fragmentation across model families because defenses remain tied to paradigm-specific assumptions.

C Experimental Setup Details

The experiments evaluate three instruction-tuned LLMs across classification and safety tasks, using both dirty-label and clean-label attacks. Tables 7 and 8 provide cross-evaluation results for two target models.

  • The study targets Gemma-2-9B-IT, Llama-3.1-8B-Instruct, and Qwen2.5-7B-Instruct.
  • Evaluation covers AG News, SST-2, and LLM-LAT, spanning standard classification and safety-related tasks.
  • Tables 7 and 8 report cross-evaluation results for Qwen2.5-7B-Instruct and Gemma-2-9B-IT.
  • Both dirty-label and clean-label settings are tested with rare-token, phrase-based, and style-based triggers.
  • Training uses LoRA with rank 16 and alpha 16, learning rate 1 × 10^-4, batch size 32, FP32 precision, and four epochs.

D Full Results of Comparison with Existing Defenses

Feature-level clamping substantially lowers ASR across both attack paradigms, unlike existing defenses whose effectiveness is asymmetric. Suspicious features are selected from activation-distribution shifts using clean calibration data, without trigger- or label-specific knowledge.

  • Qwen2.5-7B-Instruct: On Qwen2.5-7B-Instruct, Base Clamp yields 5.6–6.7% dirty-label and 3.3–5.8% clean-label ASR, while CC Clamp yields 3.1–5.4% and 2.2–4.8%.
  • Existing defenses remain paradigm-specific, whereas feature-level clamping provides more unified mitigation across model architectures.
  • Suspicious feature selection: Suspicious-feature selection ranks SAE features by normalized activation shifts between clean calibration inputs and inputs to be protected.
  • Suspicious feature selection: Larger activation-shift scores indicate stronger deviations from clean behavior and greater likelihood of backdoor participation.
  • Feature clamping: Clamping replaces only selected suspicious features with reference activations, decodes the intervened representation, and preserves reconstruction residuals to avoid unrelated changes.
  • Suspicious feature selection: The procedure uses activation-level distributional deviations rather than trigger- or label-specific information, matching a defense setting with only a model, clean calibration data, and protected inputs.

F Full Results of Feature-Level Roles

Across 54 model–dataset–trigger–paradigm settings, dirty-label and clean-label backdoors show a consistent reversal in dominant SAE feature types. Dirty-label attacks concentrate on INTERACTION features, while clean-label attacks rely more on MIXED features and, in some models, WEIGHT-MODIFIED features.

  • Overall Pattern: INTERACTION features dominate all 27 dirty-label settings, comprising 756 instances (47.0%) versus 410 SUPPRESSED (25.5%), 311 MIXED (19.3%), and 133 WEIGHT-MODIFIED (8.3%).This pattern is reported across the three model families and remains consistent across datasets.
  • Overall Pattern: MIXED features dominate all 27 clean-label settings, comprising 746 instances (45.9%) versus 340 SUPPRESSED (20.9%), 338 INTERACTION (20.8%), and 200 WEIGHT-MODIFIED (12.3%).The reversal indicates systematically different feature-level encoding strategies between the two attack paradigms.
  • Results on Gemma: Gemma dirty-label attacks contain 265 INTERACTION instances out of 540 (49.1%), while clean-label attacks shift to 233 MIXED instances (43.1%).The dirty-label counts remain stable across AGNews, SST-2, and LLM-LAT; Gemma clean-label attacks also show a relatively large WEIGHT-MODIFIED component.
  • Results on Llama: Llama dirty-label attacks contain 258 INTERACTION instances out of 530 (48.7%), whereas clean-label attacks contain 273 MIXED instances out of 541 (50.5%).For Llama, clean-label MIXED features exceed half of classified top-attributed features and are especially prominent on AGNews and SST-2.
  • Results on Qwen: Qwen dirty-label attacks contain 233 INTERACTION instances out of 540 (43.1%), while clean-label attacks contain 240 MIXED instances out of 543 (44.2%).Qwen clean-label attacks have fewer WEIGHT-MODIFIED features than Gemma, indicating a stronger concentration in mixed activation profiles.
  • Robustness and Limitations: Feature-type distributions remain qualitatively stable across threshold settings and SAE variants, although architecture, sparsity, and reconstruction quality affect fine-grained proportions.The authors interpret the taxonomy as an SAE-mediated approximation rather than a complete decomposition of hidden-state information.

H.3 Sensitivity to SAE Width and Sparsity

The sensitivity analysis tests whether the feature-type distinction depends on SAE width and sparsity. Exact proportions vary with SAE granularity, but the dirty-label versus clean-label contrast remains stable across SAE variants.

  • Sensitivity Results: Changing SAE width or sparsity affects the exact proportions of the four feature types because different SAEs provide different decomposition granularities.The analysis therefore distinguishes detailed compositional changes from the broader paradigm-level pattern.
  • Sensitivity Results: Dirty-label attacks generally show stronger INTERACTION concentration, whereas clean-label attacks contain a larger proportion of MIXED features across SAE variants.This main contrast persists despite changes in SAE architecture and sparsity.
  • Conclusion: The feature-level taxonomy is not tied to a single SAE checkpoint, although SAE configuration influences the detailed feature composition.The authors report that the dirty-label versus clean-label difference persists across different SAE variants.
Loading 2608.30403v1…