PaperlayerView exact revision on arXiv

Source-linked AI summary

SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics

Jonathan Hayase, Weihao Kong, Raghav Somani, Sewoong Oh

arXiv:2104.11315v1cs.LGcs.AIstat.ML

TL;DR

Backdoor defenses based on spectral signatures can fail when poisoned examples are difficult to separate from clean data. SPECTRE combines robust covariance estimation, whitening, and adaptive quantum-entropy scoring to amplify and detect those signatures, eliminating backdoors across regimes where existing methods fail. Its main scope boundary is that the trainer must access the corrupted training dataset.

  • Problem

    Existing defenses can fail across a broad regime because poisoned-data spectral signatures may be too difficult to detect, allowing backdoored models to retain clean-data accuracy.

  • Method

    SPECTRE robustly estimates clean-data covariance, whitens the combined data to amplify poisoned signatures, and applies adaptive quantum-entropy outlier scoring before retraining.

  • Results

    SPECTRE consistently eliminates the backdoor, including regimes where existing methods fail, and its whitening and scoring components are necessary for the reported performance.

  • Takeaways & Limitations

    SPECTRE provides a defense for a broad range of backdoor attacks by making otherwise difficult poisoned-data signatures detectable.

  • Takeaways & Limitations

    SPECTRE requires the trainer to access the corrupted training dataset, which may be unavailable under privacy constraints.

Abstract

from arXiv · show

Modern machine learning increasingly requires training on a large collection of data from multiple sources, not all of which can be trusted. A particularly concerning scenario is when a small fraction of poisoned data changes the behavior of the trained model when triggered by an attacker-specified watermark. Such a compromised model will be deployed unnoticed as the model is accurate otherwise. There have been promising attempts to use the intermediate representations of such a model to separate corrupted examples from clean ones. However, these defenses work only when a certain spectral signature of the poisoned examples is large enough for detection. There is a wide range of attacks that cannot be protected against by the existing defenses. We propose a novel defense algorithm using robust covariance estimation to amplify the spectral signature of corrupted data. This defense provides a clean model, completely removing the backdoor, even in regimes where previous methods have no hope of detecting the poisoned examples. Code and pre-trained models are available at https://github.com/SewoongLab/spectre-defense .

1. Introduction

Backdoor attacks can remain unnoticed because they preserve clean-data accuracy while changing predictions on triggered inputs. SPECTRE uses robust estimation and whitening to amplify poisoned-data signatures, addressing regimes where PCA-based defenses fail.

  • Motivation: Backdoor attacks inject poisoned data so attacker-defined triggers change predictions while clean-data behavior remains unchanged.This stealth property can allow compromised models to be deployed unnoticed.
  • Existing defenses: The PCA defense scores intermediate representations by their magnitude along the top principal direction and removes high-scoring samples before retraining.Its detection mechanism depends on poisoned examples producing a sufficiently visible spectral signature.
  • Existing defenses: At least 512 poisoned examples let PCA remove the backdoor completely, but its effective detection threshold is around 256 in the cited example.The pixel attack becomes ineffective around 64 poisoned examples, leaving a substantial gap between attack failure and defense failure.
  • SPECTRE: SPECTRE robustly estimates clean-data statistics, whitens the combined data, and uses quantum-entropy outlier scoring to detect poisoned examples.Robust whitening amplifies the spectral signature, while the scoring adapts to variation in poisoned representations.
  • SPECTRE: SPECTRE eliminates backdoors across a broad range of attacks, including regimes where existing methods fail.The paper reports that each component is crucial to the observed performance gain.

2. Threat model and diversifying the attacks

The threat model allows an adversary to corrupt training data while knowing the architecture and training method, but not training the model. The paper evaluates pixel and diversified m-way pixel attacks that preserve clean accuracy while hiding their trigger signatures.

  • Threat model: The adversary knows the training data, neural architecture, and training method, but does not train the model herself.The user trains on data that may have been corrupted to implant a backdoor.
  • Threat model: A successful backdoor preserves clean-data accuracy while mapping triggered non-target examples to an attacker-selected target label.This combination supports stealth during ordinary evaluation.
  • Attack setting: The experiments vary the fraction of injected poisoned examples and test pixel, periodic, and clean-label attacks.The poisoning fraction is defined relative to uncorrupted examples with the target label.
  • Pixel attacks: 250 poisoned CIFAR-10 examples, or ε = 5%, achieve 92% clean-test accuracy and 89% poisoned-test accuracy under the pixel attack.Each CIFAR-10 label has 5,000 clean examples in this setting.
  • m-way pixel attacks: The m-way pixel attack partitions poisoned training data and applies group-specific pixels during training, then combines all m pixels at test time.Diversifying the watermark can hide its spectral signature in lower principal-component subspaces.

3. Algorithm

SPECTRE trains a model, extracts hidden-layer representations, and detects suspicious target-label examples using projection, robust estimation, whitening, and QUE scores before retraining. Robust whitening amplifies hidden poison signatures that ordinary PCA can miss, while adaptive dimensionality selection addresses the sample limitations of high-dimensional covariance estimation.

  • Pipeline: SPECTRE trains a model, extracts a middle-layer representation, identifies the target label, removes suspicious target-label examples, and retrains on cleaned data.The pipeline assumes the target label has been correctly identified when focusing on the robust poison detector.
  • SPECTRE detector: The detector projects representations into k dimensions, robustly estimates the clean mean and covariance, whitens the data, and applies QUE scores to rank poisoned examples.It removes the 1.5εn samples with the greatest QUE scores.
  • Dimensionality selection: The projection dimension k trades statistical power against estimation difficulty: too small can omit the separating direction, while too large worsens covariance estimation.The paper proposes adaptive selection of k because a fixed moderate value such as k = 60 can completely fail.
  • Motivation: Ordinary PCA can fail because the top PCA direction of combined data may be misaligned with the poison-separating direction, which often lies in a low-variance subspace.The cited example shows poisoned and clean representations indistinguishable along top PCA directions despite separation in a lower-variance subspace.
  • Step 2: Robust estimation: Whitening with estimated clean statistics equalizes clean-data variance across directions, amplifying hidden spectral signatures for detection.The robust estimator is supported theoretically for sufficiently sampled Gaussian data, but full-dimensional estimation is impractical for CIFAR-10 representations.
  • Step 3: QUE scoring: QUE scoring adapts to the effective dimensionality of the poison signature, outperforming alternative score choices with α = 4 in the reported experiments.Squared-norm scoring can fail for weak signatures, whereas squared projected-norm scoring can fail for strong signatures.
  • Possible extensions: Robust principal component analysis and sample-splitting variations were tested as dimensionality-reduction alternatives but did not improve performance in practice.The paper therefore retains its proposed dimensionality-reduction approach.

4. Experiments

Across m-way pixel, periodic, and label-consistent attacks, SPECTRE consistently removes backdoors, including regimes where PCA and clustering defenses fail. Its adaptive selection of k uses mean QUE scores to avoid fixed-dimension failures and can identify the targeted label.

  • m-way pixel and periodic attacks: SPECTRE consistently eliminates backdoors across m-way pixel and periodic attacks, while PCA and clustering defenses fail in most cases.The retrained model achieves near-zero attack accuracy under the proposed defense.
  • m-way pixel and periodic attacks: The PCA defense succeeds only when the poisoned-data spectral signature is strong, but diversified or smaller attacks hide that signature in lower-dimensional subspaces.Robust covariance estimation amplifies these weaker signatures for detection.
  • label consistent attacks: SPECTRE removes all poisoned examples in every label-consistent attack instance, whereas PCA and clustering remove only a small fraction or none in a wide regime.Clean-data accuracy remains between 91% and 92.5% in these experiments.
  • finding the effective dimension k: No fixed k works across experiments: small k can omit the separating direction, while large k can include poorly behaved clean-data subspaces that break robust covariance estimation.The experiments therefore select k adaptively using the mean QUE score.
  • identifying the target label: The target-label identifier selects the label with the maximum QUE score and achieves 100% accuracy whenever attack accuracy exceeds 0.33.The method evaluates candidate labels and effective dimensions together.

5. Ablation study

The ablation study evaluates SPECTRE’s adaptive dimension reduction, robust covariance estimation, whitening, and QUE scoring. Results support retaining all four components because fixed dimension choices and alternative scoring can fail.

  • SPECTRE components: SPECTRE combines adaptive dimension reduction, robust covariance estimation, whitening, and QUE scoring to detect poisoned examples.These steps respectively choose the working subspace, estimate clean-data statistics, amplify signatures, and score samples for removal.
  • adaptive dimension reduction: No constant k detects the majority of poison across multiple experiments, making adaptive dimension reduction necessary.Choosing k = d is equivalent to performing no dimension reduction.
  • outlier scoring and whitening: Alternative outlier-scoring choices can fail under certain conditions, motivating QUE scoring within the full pipeline.The ablation compares combinations with and without robust covariance estimation and whitening.
  • ablation results: The complete combination of adaptive reduction, robust covariance estimation, whitening, and QUE scoring performs better than the other tested combinations.Table 6 evaluates the effects of omitting whitening or replacing robust covariance with sample covariance.

6. Conclusion

The paper argues that existing defenses miss many backdoor attacks because their spectral signatures are difficult to detect. SPECTRE combines robust covariance whitening with QUE detection, while remaining dependent on access to corrupted training data.

  • motivation: Existing defenses fail across a broad regime because the spectral signatures they rely on can be difficult to detect.This limitation motivates a defense that amplifies rather than directly relying on the original spectral profile.
  • method: SPECTRE estimates robust clean-data statistics, whitens representations, and applies quantum entropy outlier detection to amplify and detect poisoned-data signatures.The method also uses adaptive dimension reduction before robust estimation.
  • scope and limitations: SPECTRE requires access to the corrupted training dataset, limiting direct applicability when privacy constraints prevent access to training data.The paper identifies decentralized and differentially private federated-learning variants as future directions.
  • compared defenses: The compared PCA defense projects representations onto a top eigenvector and removes points far from the mean, while activation clustering reduces dimension and applies 2-means.The clustering comparison is additionally given an oracle for selecting the more poisoned cluster.

B. Complete experimental results

Complete results cover m-way pixel, m-way periodic, and label-consistent attacks. SPECTRE removes the backdoor across the reported attack families, while clean-data accuracy remains high in the evaluated settings.

  • m-way pixel attacks: SPECTRE completely removes m-way pixel backdoors for m ∈ {1, 2, 3} and all tested poisoned-data sizes, achieving near-zero retrained attack accuracy.PCA and clustering fail to remove enough poisoned samples in almost all cases.
  • m-way periodic attacks: SPECTRE completely removes m-way periodic backdoors for m ∈ {1, 2} and all tested poisoned-data sizes, again reaching near-zero retrained attack accuracy.PCA and clustering fail in almost all corresponding cases.
  • clean-data performance: Clean-data accuracy remains between 92.5% and 93.5% across the reported periodic-attack experiments.These values are reported alongside the near-zero backdoor accuracy after SPECTRE retraining.
  • label-consistent attacks: Under label-consistent attacks, SPECTRE removes all poisoned examples whenever attack accuracy exceeds 10%.Clean-data accuracy ranges from 91% to 92.5% in these experiments.

C. Supplemental experimental results for different source-target label pairs

Across source-target label pairs, the experiments examine variation in defense performance, while the appendix details robust mean and covariance estimation procedures used by SPECTRE.

  • Experimental results: Performance across source-target label pairs follows a similar overall trend, but some combinations defeat all tested defenses.Clean-data accuracy remained between 91% and 92.5% in these experiments.
  • Robust mean estimation: ROBUSTMEAN iteratively filters corrupted samples until it returns an estimated mean vector.Its filter reduces the symmetric difference from an underlying good set or produces an estimate with bounded error.
  • Statistical assumptions: Good-set guarantees require sufficiently many independent Gaussian samples, with covariance estimation requiring a sample scale proportional to (d^2/ε^2) log^5(d/ε).The appendix states that such samples are ε-good with high probability.
  • Robust covariance estimation: ROBUSTCOV iteratively filters samples to estimate a covariance matrix with bounded whitened Frobenius error.The covariance filter removes samples with large Mahalanobis distance and analyzes even degree-2 polynomials for unusually large variance.
  • Arbitrary Gaussian estimation: The arbitrary-Gaussian procedure combines robust mean and covariance estimation by pairing samples from two halves and subtracting them.The resulting vectors have zero mean and twice the original covariance, allowing the zero-mean covariance estimator to be applied.

E. Experiment details

The experiments train ResNet-32 models under specified pixel, periodic, and label-consistent backdoor settings, using standard optimization and data-processing choices.

  • Training procedure: Each poisoned dataset produced one poisoned model, while each pixel and periodic defense received one retraining run.Training used a Xeon Gold 6230 CPU and eight Nvidia 2080 Ti GPUs.
  • Pixel attacks: Pixel-attack models used leaky-ReLU ResNet-32 networks trained with momentum SGD for 200 epochs, with scheduled learning-rate reductions.Data standardization and augmentation were enabled.
  • Data augmentation: Random flips and crops can move a fixed pixel watermark to as many as 162 transformed-image locations, about 16% of the image area.The count follows 9 × 9 × 2 transformations under CIFAR-10 augmentation.
  • Pixel attacks: An m-way pixel attack poisons each example with one of m location-color pairs but tests with all m watermarks simultaneously.Experiments used m ∈ {1, 2, 3}.
  • Other attacks: Periodic attacks used the same architecture and environment, while label-consistent attacks followed Turner et al.’s setup with augmentation enabled for consistency.Periodic signals had a four-pixel period matching the maximum augmentation translation.

F. Analysis of poisoned representations

The supplemental figures visualize hidden-layer representations of clean and poisoned examples before and after whitening, including whitening based on clean or robust covariance estimates.

  • Representation visualizations: Figures 12–15 examine hidden-layer activations for successful backdoor attacks using PCA projections and pairwise scatter plots.Diagonal plots show Gaussian kernel density estimates; off-diagonal plots show projections onto pairs of PCA directions.
  • Before whitening: Figure 12 shows representations of a 3-way pixel attack with ε = 0.1 before whitening.The combined clean and poisoned representations are projected onto their top eight PCA directions.
  • Whitening comparisons: Figures 13 and 14 compare whitening with clean-sample covariance against whitening with robustly estimated covariance for the same attack.Both figures project the whitened representations onto their top eight PCA directions.
  • Whitening comparisons: Figure 15 displays a 2-way pixel attack with ε = 0.1 after whitening with the true covariance of clean representations.The visualization uses the top eight PCA directions and pairwise projected scatter plots.

G. Analysis of QUE scores

The QUE-score analysis identifies two failure modes of simpler spectral scores and relates them to the strength, dimensionality, and alignment of poisoned representations.

  • Squared-norm score: The squared-norm score fails for the 3-way attack with ε = 0.0124 because the poisoned representations have high variance in only one direction.Variance from clean data dominates most other directions, weakening the observable spectral signature.
  • Squared-norm score: The top PCA direction becomes mixed with less useful directions, diluting the squared-norm score’s usefulness for removing poison.This follows from the weak poison signature relative to the clean-data variance directions.
  • Projection score: The projection score fails for the 1-way attack with ε = 0.1 because the top PCA direction is misaligned with the clean-poison mean separation.The reported angle between these directions is 35.7°.
  • Visual analysis: Figures 16–18 visualize singular values and robustly whitened representations for the attack regimes analyzed in the failure cases.Figure 16 orders the top 10/100 covariance singular values by decreasing magnitude.

H. Sensitivity to number of removed examples

SPECTRE removes the 1.5εn samples with the highest QUE scores, and its defense performance is not overly sensitive to this removal choice. After the first εn samples, the fraction of poisoned samples removed changes little as more samples are removed.

  • SPECTRE removes the 1.5εn samples with the highest QUE scores from the (1 + ε)n samples bearing the target label.This selection follows Tran et al. (2018).
  • The fraction of poisoned samples removed does not vary substantially with the total number of removed samples after the first εn samples.
  • Figure 19 compares the fraction of all poisoned samples removed with the total number of samples removed for three pixel attacks with varying spectral-signature strength.
Loading 2104.11315v1…