PaperlayerView exact revision on arXiv

Source-linked AI summary

The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matrices

Wanlun Ma, Derui Wang, Ruoxi Sun, Minhui Xue, Sheng Wen, Yang Xiang

arXiv:2209.11715v3cs.CRcs.AI

TL;DR

Existing defenses rely on universal triggers, an assumption broken by dynamic backdoors whose triggers vary across samples. Beatrix uses Gram-matrix representation information and RMMD testing; against BadEncoder, it achieved high true-positive rates on infected ImageNet and CLIP encoders.

Abstract

from arXiv · show

Deep Neural Networks (DNNs) are susceptible to backdoor attacks during training. The model corrupted in this way functions normally, but when triggered by certain patterns in the input, produces a predefined target label. Existing defenses usually rely on the assumption of the universal backdoor setting in which poisoned samples share the same uniform trigger. However, recent advanced backdoor attacks show that this assumption is no longer valid in dynamic backdoors where the triggers vary from input to input, thereby defeating the existing defenses. In this work, we propose a novel technique, Beatrix (backdoor detection via Gram matrix). Beatrix utilizes Gram matrix to capture not only the feature correlations but also the appropriately high-order information of the representations. By learning class-conditional statistics from activation patterns of normal samples, Beatrix can identify poisoned samples by capturing the anomalies in activation patterns. To further improve the performance in identifying target labels, Beatrix leverages kernel-based testing without making any prior assumptions on representation distribution. We demonstrate the effectiveness of our method through extensive evaluation and comparison with state-of-the-art defensive techniques. The experimental results show that our approach achieves an F1 score of 91.1% in detecting dynamic backdoors, while the state of the art can only reach 36.9%.

I. INTRODUCTION

Existing defenses rely on shared, universal triggers, but dynamic backdoors vary their triggers across samples and make those defenses harder to apply. Beatrix uses Gramian information to detect activation anomalies, and the introduction reports effective detection of sample-specific attacks against existing defenses.

I. INTRODUCTION

  • Dynamic triggers vary across samples, breaking the universal-trigger premise behind existing defenses and making the trigger harder to model.The introduction notes that poisoned samples' intermediate representations differ from normal samples in the target class.Source: §I · p. 1
  • Beatrix uses Gramian information and robust deviation measurements to capture poisoned-sample activation anomalies, with RMMD to improve infected-class identification.The method captures feature correlations and representation order information, using robust class-conditional statistics from legitimate samples.Source: §I · p. 2
  • Beatrix effectively detects sample-specific backdoor attacks and significantly outperforms existing defenses, according to the introduction.The supplied passage reports no quantitative comparison.Source: §I · p. 2

A. Gram Matrix in DNNs

  • A Gram matrix contains pairwise inner products, with off-diagonal entries representing correlations; its higher-order form captures additional representation information.The supplied background describes Gram matrices as symmetric matrices and notes their use in capturing stylistic attributes in neural activations.Source: A. Gram Matrix in DNNs · p. 2

B. Backdoor Attacks

Backdoor attacks implant hidden functionality activated by a trigger, typically redirecting triggered inputs to an attacker-chosen class. The section distinguishes universal, partial, and dynamic triggers, and describes dynamic attacks as jointly training a classifier and trigger generator.

B. Backdoor Attacks

  • A backdoor implant activates when a trigger appears, typically misclassifying the input to an attacker-predefined target class.The attack is described as hidden functionality injected into an ML system.Source: B. Backdoor Attacks · p. 2
  • Universal attacks use one static trigger across samples; BadNets embeds it into training examples through a trigger function.Poisoned training pairs replace clean pairs with some probability.Source: B. Backdoor Attacks · pp. 2–3
  • Partial backdoors activate only for a specific source class, but their trojaned samples still share a uniform trigger,.Source: B. Backdoor Attacks · p. 3
  • Dynamic attacks use sample-varying, unique triggers; this work targets invisible sample-specific and input-aware attacks,, excluding.The paper excludes because its triggers are not sample-specific and its code is unavailable.Source: B. Backdoor Attacks · p. 3
  • Dynamic backdoors jointly train a trigger generator with the classifier on clean and poisoned data, unlike per-sample or external-data adversarial trigger optimization.Further trigger modification at inference is outside the paper's threat model.Source: B. Backdoor Attacks · p. 3

C. Existing Defenses

Existing defenses often depend on universal triggers or restrictive representation assumptions, leaving sample-specific backdoors poorly covered. Beatrix is proposed for this setting, under a defender model with white-box access and a small clean dataset.

C. Existing Defenses

  • Type-I defenses rely on trigger persistence, Type-II defenses reconstruct universal triggers, and Type-III defenses use clustering or Gaussian models; the paper says these approaches do not cover dynamic triggers.Dynamic triggers vary across samples, so the universal-trigger assumptions behind the first two defense types are violated.Source: C. Existing Defenses · p. 3
  • SCAn assumes mean discrepancies and Gaussian feature distributions with shared covariance, and its authors estimate roughly 50 trojaned images are needed for reliable detection.The paper argues that these assumptions do not extend to sample-specific backdoors and may delay detection in security-critical settings.Source: C. Existing Defenses · pp. 3–4

III. OVERVIEW AND MOTIVATION

  • Beatrix is proposed to cover sample-specific attacks by using Gramian information to detect anomalies in activation patterns and RMMD to identify contaminated classes.The framework’s motivation is that clean and trojaned samples targeted to the same class can differ in intermediate representations despite sharing a predicted label.Source: §III · p. 4, B. Intuition and Key Idea · pp. 4–5

A. Threat Model

  • The threat model gives the defender white-box access to intermediate representations and a small clean dataset, while the adversary can poison training data but cannot access the model directly.The defender performs online input-level detection and aims to identify infected classes.Source: A. Threat Model · p. 4

B. Intuition and Key Idea

  • Gramian features can distinguish clean and trojaned samples that are deeply fused in the original feature space, while incorporating correlations across feature channels.The paper connects backdoor detection to OOD detection but notes that Gaussian modeling, clean-data requirements, and limited data distinguish the backdoor setting.Source: B. Intuition and Key Idea · pp. 4–5

C. Theoretical Analysis of Gram Matrix

The analysis frames Beatrix’s Gramian representations as capturing distributional differences beyond mean discrepancies. By using multiple orders, they can retain discriminative information even when clean and trojaned representations share a mean.

C. Theoretical Analysis of Gram Matrix

  • For each representation vector, Beatrix forms a Gramian feature from its outer product and considers its expectation across clean and trojaned distributions.The expected Gramian includes a mean outer product and a covariance term.Source: C. Theoretical Analysis of Gram Matrix · p. 5
  • Gramian features capture both first-moment differences and second-moment discrepancies, so trojaned samples can remain distinguishable when clean and trojaned distributions have the same mean.Considering multiple orders can capture further high-order feature information.Source: C. Theoretical Analysis of Gram Matrix · p. 5

IV. DESIGN OF BEATRIX

  • The design section introduces the Beatrix framework as the approach for detecting backdoor attacks.The framework is illustrated in Figure 2.Source: §IV · p. 5

A. Feature Modeling via Gram Matrices

Beatrix converts intermediate activations into concatenated, multi-order Gramian features and treats detection as class-conditional outlier detection. It measures deviation robustly using clean-sample statistics and bootstrapping.

A. Feature Modeling via Gram Matrices

  • At layer l, Beatrix represents an input by its channel feature maps and uses Gram matrices to capture channel correlations, including higher-order information.For order p, the matrix is formed from the p-th power of the representation; its off-diagonal entries encode pairwise channel correlations.Source: A. Feature Modeling via Gram Matrices · p. 6
  • Beatrix vectorizes the triangular part of each symmetric Gram matrix and concatenates vectors across orders to form a representation for each input.The maximum order P is a hyperparameter.Source: A. Feature Modeling via Gram Matrices · p. 6
  • Detection is formulated as testing whether an input’s concatenated Gramian feature is an outlier relative to clean samples in its predicted class.A multivariate Gaussian is considered statistically fragile with high-dimensional features and limited clean samples, motivating independent element-wise deviation estimates.Source: A. Feature Modeling via Gram Matrices · p. 6, B. Deviation Measurement · p. 6

B. Deviation Measurement

  • Beatrix uses Median Absolute Deviation rather than Gaussian estimates to measure feature deviations, because MAD is more resilient to outliers and Gaussian estimates need sufficient clean data.The method uses bootstrapping to estimate deviations of benign inputs and lets the defender choose a detection percentile.Source: B. Deviation Measurement · p. 6

C. Identifying Infected Labels

Beatrix uses kernel-based two-sample testing to compare representation distributions without assuming a particular distribution. Its RMMD statistic adds penalties intended to improve testing when the two sample sets are small and imbalanced, supporting the identification of infected labels.

C. Identifying Infected Labels

  • In an offline setting, Beatrix accumulates historical detections for a second statistical analysis to reduce false positives.This refinement evaluates the feature representations of samples in an infected class as a mixture of two subgroups.Source: C. Identifying Infected Labels · p. 6
  • Kernel-based two-sample testing assesses whether two sample sets are identically distributed without assuming their distributions.The paper uses this approach because a Gaussian assumption is not tenable in more complex scenarios such as dynamic backdoors.Source: C. Identifying Infected Labels · p. 7
  • RMMD extends MMD with two penalty terms intended to improve performance when the sample sets are small and imbalanced.P and Q denote representation distributions from the two subgroups obtained by deviation measurement.Source: C. Identifying Infected Labels · p. 7
  • Beatrix uses the RMMD statistic for each class and identifies labels with abnormally large values using the Median Absolute Deviation.The statistic is asymptotically normal, and the anomaly index applies a constant factor η = 1.4826.Source: C. Identifying Infected Labels · p. 7

V. EVALUATION

The evaluation tests Beatrix against input-aware dynamic backdoors across datasets, with limited and contaminated clean data, different Gram-matrix orders, and an all-to-all attack. Infected labels show larger anomaly indices, trojaned samples show larger deviations, and the method remains effective under the reported data and attack variations.

A. Effectiveness Against Dynamic Backdoor

  • The input-aware dynamic attack achieves almost 100% attack success while maintaining clean-data performance comparable to benign models across four datasets.Cross-trigger accuracy exceeds 80% on all four datasets.Source: A. Effectiveness Against Dynamic… · p. 7, Table IV · p. 8
  • Across four datasets, infected labels have much larger anomaly indices than uninfected labels, and trojaned samples have larger deviations than benign samples.The reported findings support identifying infected target classes and distinguishing polluted from benign samples.Source: A. Effectiveness Against Dynamic… · p. 7, Figure 4 · p. 8, Figure 5 · p. 8
  • With 30 clean images per class, Beatrix effectively detects the dynamic attack; with only 8 clean images it accurately identifies the infected class and remains effective when at most 16% (or 5 images) per class are contaminated.Source: A. Effectiveness Against Dynamic… · p. 7
  • Beatrix is reported to be much more robust than the OOD detector when clean data is contaminated; the authors conclude that the OOD method cannot be directly applied to backdoor detection.The comparison concerns the contaminated-clean-data scenario.Source: A. Effectiveness Against Dynamic… · p. 8, Figure 9 · p. 9
  • Testing time rises from 8.9×10^-6s to 78.5×10^-6s as the Gram-matrix order bound increases from 1 to 9, while detection capability stabilizes at P ≥4.The authors use P = 4 in the remaining experiments as a trade-off between detection effectiveness and computational complexity.Source: A. Effectiveness Against Dynamic… · p. 8, Figure 8 · p. 8
  • For all-to-all attacks, infected labels' RMMD statistics are much larger than those of uninfected labels in single-target attacks, even though anomaly indices are no longer effective.The comparison is reported for all-to-all infected models and single-target infected models.Source: A. Effectiveness Against Dynamic… · p. 8, Figure 10 · p. 9

B. Comparison

Beatrix is compared with offline and online defenses on dynamic and universal backdoors. It largely outperforms the benchmark methods on dynamic attacks, while individual baselines show marked performance drops or assumptions mismatched to dynamic triggers; the paper also reports low per-sample runtime for Beatrix.

B. Comparison

  • Beatrix largely outperforms the benchmark methods against dynamic backdoors and slightly outperforms them against universal backdoors.The comparison covers offline and online protection settings.Source: Table VI · p. 9, 1) Offline defense · p. 9
  • On GTSRB, AC's F1-score is 92.5% against universal attacks and 11.1% against dynamic attacks.The paper states dynamic attacks make benign and trigger-carrying activations less distinguishable for AC.Source: 1) Offline defense · p. 9
  • SCAn's F1-score drops from 96.2% on universal attacks to 36.9% on dynamic attacks on GTSRB.The authors attribute its dynamic-attack ineffectiveness to limitations in its distribution model; Figure 11 analyzes fused clean and poisoned representations on CIFAR10.Source: 1) Offline defense · pp. 9–10, Figure 11 · p. 11
  • MNTD's F1-score on GTSRB falls from 85.8% against universal attacks to 69.4% against dynamic attacks.MNTD flags models as trojaned or benign, using a meta-classifier trained on output differences between clean and infected shadow models.Source: 1) Offline defense · p. 10
  • STRIP's F1-score falls from 91.9% and 91.7% against universal backdoors to 29.7% and 28.3% against dynamic backdoors.The paper links this limitation to dynamic triggers deactivating when mismatched triggers are applied.Source: 2) Online defense · p. 10
  • For P = 4, Beatrix takes 35.14 ×10^-6s per sample, compared with 0.04s for STRIP, 0.11s for SentiNet, and 13.58s for SCAn in the reported efficiency comparison.Beatrix's deviation model is obtained offline, so per-sample processing requires a forward pass and Gramian-feature computation.Source: Table VII · p. 10, 2) Online defense · p. 10

C. Robustness Against Other Attacks

Beatrix is evaluated against invisible sample-specific triggers, reflection-based Refool triggers, and BadEncoder attacks on image encoders. The reported results show detection across these attack settings.

C. Robustness Against Other Attacks

  • Beatrix's anomaly index is much larger for the infected label than uninfected labels in the evaluated ISSBA ImageNet model.The experiment uses an infected model trained on a 200-class ImageNet subset, with goldfish as the target label.Source: C. Robustness Against Other Attacks · p. 11
  • 99.99% TPR @ 5% FPR and 98.50% TPR @ 1% FPR show Beatrix distinguishing clean images from Refool trigger-carrying images.The infected label's anomaly index exceeds the threshold while uninfected labels remain below it.Source: Figure 15 · p. 11, C. Robustness Against Other Attacks · p. 12
  • 99.8% TPR @ 1% FPR on the infected ImageNet encoder and 100% TPR @ 1% FPR on the infected CLIP encoder show detection of BadEncoder inputs.Detection is performed directly on feature vectors from the backdoor encoders, without building a downstream classifier.Source: Figure 16 · p. 12, C. Robustness Against Other Attacks · p. 12

D. Beyond the Image Domain

The paper extends Beatrix evaluations to text and speech backdoors, tests an adaptive attack, and concludes that Gramian information can distinguish dynamic trojans from benign samples.

D. Beyond the Image Domain

  • 89.8% TPR @ 5% FPR on homograph attacks, 75.2% on dynamic sentence attacks, and 77.5% on speech backdoors report detection across the tested non-image tasks.The speech result is on Speech-Command; the text experiments use toxic-comment classification.Source: D. Beyond the Image Domain · pp. 12–13

E. Adaptive Attack

  • At λ = 1, Beatrix yields only 14.0% TPR @ 1% FPR on CIFAR10, while the adaptive attack also substantially reduces clean accuracy and attack success rate.The adaptive loss minimizes representation-space distances between poisoned and clean target-class images using multiple high-order Gram matrices.Source: E. Adaptive Attack · p. 13

VI. RELATED WORK

  • Prior defenses include representation analysis, trigger reconstruction, and run-time detection, but the cited work reports vulnerabilities to adaptive or dynamic attacks.The related-work discussion also notes limitations of trigger reconstruction for large triggers and source-label-specific attacks.Source: §VI · p. 13

VII. CONCLUSION

  • Beatrix combines high-order representation information with kernel-based two-sample testing to identify infected labels, and the paper reports effectiveness against conventional and dynamic backdoors.The conclusion attributes the dynamic-backdoor distinction to differences in Gramian information between trojaned and benign data points.Source: §VII · p. 13

APPENDIX

The appendix explains why universal-trigger defenses are limited against dynamic and partial backdoors, then outlines SCAn's distributional assumptions and testing setup.

A. DETAILED DISCUSSION OF EXISTING BACKDOOR DETECTION

  • Universal-trigger defenses estimate a shared trigger or detect shared abnormal behavior, so their premise is weakened when triggers vary across samples.The appendix also describes partial backdoors as affecting samples from a specific class while leaving others unaffected.Source: A. DETAILED DISCUSSION OF EXISTING… · p. 15

B. THEORETICAL ANALYSIS OF SCAN

  • SCAn models clean and poisoned representations with Gaussian distributions sharing a covariance but differing in means, then tests one normal distribution against a two-component mixture.The appendix states that SCAn formulates backdoor detection as a likelihood-ratio test over infected-class representations.Source: B. THEORETICAL ANALYSIS OF SCAN · p. 16

C. EXPERIMENT SETUP

The experimental setup spans four image datasets and compares defense performance on ISSBA and through ablations of Beatrix's detection metrics.

C. EXPERIMENT SETUP

  • The main setup uses CIFAR10, GTSRB, VGGFace, and ImageNet, with Pre-activation ResNet18, VGG16, or ResNet101 classifiers.The architectures are paired with CIFAR10 and GTSRB, VGGFace, and ImageNet, respectively.Source: C. EXPERIMENT SETUP · pp. 16–17

D. DEFENSE PERFORMANCE AGAINST ISSBA

  • For ISSBA, Beatrix significantly outperforms existing defenses in the reported offline and online comparisons on ImageNet.The offline evaluation uses 20 infected models targeting labels 0–19; online testing samples are drawn from each dataset.Source: D. DEFENSE PERFORMANCE AGAINST ISSBA · p. 17, Table IX · p. 17

E. ABLATION STUDY

  • Beatrix with MAD and RMMD outperforms the compared baselines in the online and offline cases reported for dynamic backdoors.The ablation contrasts these metrics with variance-based and Gaussian/Mahalanobis alternatives.Source: Table X · p. 17, E. ABLATION STUDY · p. 17

F. DEFENSE PERFORMANCE AGAINST COMPOSITE BACKDOOR [51]

Beatrix detects poisoned samples and infected labels across composite, WANET, and multi-trigger attacks. Its anomaly measure also remains effective when clean data are color-jittered.

F. DEFENSE PERFORMANCE AGAINST COMPOSITE BACKDOOR [51]

G. DEFENSE PERFORMANCE AGAINST WANET [60]

H. DEFENSE PERFORMANCE AGAINST MULTI-TRIGGER ATTACK [29]

  • With 8 triggers, Beatrix distinguishes benign from trojaned samples in both multi-trigger attacks; RMMD statistics also indicate backdoors when over half of labels are infected.The anomaly index may not be effective when more than half of labels are infected in the multi-trigger multi-label attack.Source: H. DEFENSE PERFORMANCE AGAINST… · p. 18, Figure 22 · p. 18

I. DEFENSE PERFORMANCE ON NON-GAUSSIAN DISTRIBUTION DATASET

Loading 2209.11715v3…