Source-linked AI summary
Improving Adversarial Robustness of Ensembles with Diversity Training
Sanjay Kariyappa, Moinuddin K. Qureshi
TL;DR
Transfer-based black-box attacks can fool target networks using perturbations crafted on surrogate models. The paper trains ensembles with misaligned loss gradients through Diversity Training, improving adversarial robustness and strengthening existing defenses when combined with them.
Problem
Deep neural networks remain vulnerable to black-box attacks because adversarial examples can transfer from surrogate models to target models.
Method
Diversity Training uses Gradient Alignment Loss to train ensemble models with uncorrelated loss gradients and reduced shared adversarial subspaces.
Results
Diversity Training improves ensemble robustness against transfer-based attacks, and combining it with existing defenses produces a stronger defense.
Takeaways & Limitations
Misaligned ensemble gradients reduce overlap in shared adversarial subspaces, making it harder for one transferred perturbation to fool multiple models.
Takeaways & Limitations
The paper leaves evaluation of whether diverse ensembles enable better black-box attacks for future work.
Abstract
from arXiv · showhide
Deep Neural Networks are vulnerable to adversarial attacks even in settings where the attacker has no direct access to the model being attacked. Such attacks usually rely on the principle of transferability, whereby an attack crafted on a surrogate model tends to transfer to the target model. We show that an ensemble of models with misaligned loss gradients can provide an effective defense against transfer-based attacks. Our key insight is that an adversarial example is less likely to fool multiple models in the ensemble if their loss functions do not increase in a correlated fashion. To this end, we propose Diversity Training, a novel method to train an ensemble of models with uncorrelated loss functions. We show that our method significantly improves the adversarial robustness of ensembles and can also be combined with existing methods to create a stronger defense.
1. Introduction
Black-box attacks exploit transferability from surrogate models, motivating ensembles whose members have less-overlapping adversarial subspaces. The paper proposes Gradient Alignment Loss and Diversity Training to reduce this overlap and improve robustness.
- Motivation: Transfer-based black-box attacks craft perturbations on surrogate models that can fool targets without exposing target parameters.Such attacks matter because adversarial inputs can threaten deployed systems, including self-driving cars, content moderation, and malware detection.
- Ensemble as an Effective Defense: Ensembles require an attack to fool multiple models, so successful perturbations must lie in their shared adversarial subspace.The paper illustrates this shared-subspace intuition with Venn diagrams for single models, ordinary ensembles, and diverse ensembles.
- Ensemble as an Effective Defense: Reducing overlap in ensemble adversarial subspaces reduces the directions that can fool multiple models, thereby reducing transferability and improving robustness.The paper identifies shared-subspace dimensionality as the key target for improving ensemble defenses.
- Contributions: Gradient Alignment Loss measures shared adversarial structure, while Diversity Training uses it to train ensembles with misaligned loss gradients.The method explicitly regularizes ensemble members toward less-correlated gradients.
- Contributions: Diversity Training empirically makes ensembles more robust to transfer-based attacks.The paper presents this as an empirical contribution of the proposed training procedure.
2. Background
The paper formalizes adversarial examples and the black-box threat model, then reviews gradient-based attacks used to evaluate defenses. These attacks optimize loss-increasing perturbations under an l∞ constraint.
- Adversarial Examples: An adversarial input adds a carefully crafted perturbation to a benign input while remaining constrained by a perceptual l∞ bound.Untargeted attacks seek misclassification while enforcing ∥η∥∞≤ϵ.
- Attack Model: In the black-box setting, attackers know the training dataset and target architecture but not target parameters, so they transfer attacks from a surrogate model.The surrogate is trained on the same dataset as the target.
- Attack Algorithms: Attack generation maximizes model loss subject to a perturbation constraint, using the loss gradient to determine effective input changes.FGSM adds a perturbation of size ϵ in the gradient direction.
- Attack Algorithms: FGSM has iterative, randomized, momentum-based, and hinge-loss variants that modify how perturbations are computed.I-FGSM takes multiple smaller steps, R-FGSM begins with a random step, MI-FGSM uses gradient momentum, and PGD-CW uses hinge loss with projected updates.
3. Diversity Training
Diversity Training reduces transfer-based adversarial vulnerability by training ensemble models whose adversarial subspaces overlap less. It measures overlap through gradient alignment and regularizes training with a smooth Gradient Alignment Loss.
- 3. Diversity Training: The ensemble approach targets black-box attacks by reducing the shared adversarial-subspace dimensionality that transferred perturbations must occupy.An attack must fool multiple ensemble models, so fewer shared adversarial directions reduce transferability.
- 3.2.2. Gradient Alignment: Misaligned loss gradients indicate smaller shared adversarial subspaces because perturbations that increase one model’s loss are less likely to increase another’s.Aligned gradients imply correlated loss increases and greater overlap, whereas misalignment reduces the dimensionality of the shared adversarial subspace.
- 3.2.2. Gradient Alignment: The method measures ensemble overlap by maximizing pairwise cosine similarity among model gradients through a coherence-based objective.Coherence captures the maximum cosine similarity between unique gradient-vector pairs.
- 3.2.2. Gradient Alignment: GAL replaces the non-smooth maximum in the coherence objective with LogSumExp to enable more effective first-order optimization.The smooth approximation improves convergence compared with directly minimizing the non-smooth maximum.
- 3.3. Diversity Training: Diversity Training adds a Gradient Alignment Loss regularizer to train ensembles with lower overlap among their adversarial subspaces.The training objective combines average cross-entropy with GAL, reducing transferability of adversarial examples.
- 3.4. Problem of Sparse Gradients: Leaky ReLU is used with GAL because standard ReLU creates sparse loss gradients that hinder backpropagation through the regularizer.ReLU’s zero derivative in its saturating regime produces many zero-valued gradient products in GAL.
4. Experiments
Experiments evaluate DivTrain against transfer-based black-box attacks using MNIST and CIFAR-10, comparing baseline, diverse, and combined defenses. DivTrain improves adversarial accuracy, while combining it with EnsAdvTrain yields stronger robustness, with lower coherence and adversarial-subspace dimensionality.
- 4.1. Setup: Experiments use MNIST and CIFAR-10, with target models attacked through surrogate models sharing the target architecture and dataset but not its parameters.The evaluation targets transfer-based black-box attacks under the stated surrogate-model assumption.
- 4.1. Setup: Training adds dynamically augmented and noisy images, and sets the DivTrain coefficient to λ = 0.5 as a trade-off between clean accuracy and adversarial robustness.Noise is sampled with dataset-specific perturbation scales, and the combined data train diverse ensembles.
- 4.2. Results: TDiv achieves significantly higher adversarial classification accuracy than TBase across all attacks, while clean accuracy drops slightly.Table 2 compares baseline and diverse ensembles under the evaluated black-box attacks.
- 4.2. Results: TEns+Div achieves higher under-attack classification accuracy than either EnsAdvTrain or DivTrain alone.The combined defense is evaluated as a stronger version of the individual defenses.
- 4.3. Distribution of Coherence: Diversity Training lowers coherence values for TDiv and TEns+Div relative to TBase and TEns, indicating more misaligned gradient vectors.Figure 4 compares coherence distributions for Conv-3, Conv-4, and Resnet-20.
- 4.4. Gradient Aligned Adversarial Subspace: GAAS analysis finds lower probabilities of successful orthogonal adversarial directions for TDiv than TBase across the evaluated perturbation sizes.The analysis provides evidence that DivTrain lowers the dimensionality of the ensemble’s adversarial subspace.
5. Related Work
Related work covers defenses against white-box and black-box attacks, ensemble-based robustness, and gradient-similarity measures. The paper distinguishes its approach by using gradient coherence to reduce overlap among ensemble adversarial subspaces.
- Adversarial-subspace analysis: GAAS analysis compares the probability of finding orthogonal adversarial directions for TBase and TDiv under multiple perturbation sizes.The reported comparison supports the paper’s focus on adversarial-subspace dimensionality.
- Related defenses: Many existing defenses are ineffective against adaptive attacks, which can exploit gradient masking and obfuscated gradients.Prior work identifies unavailable or misleading gradient information as a defense weakness.
- Ensemble defenses: Prior ensemble defenses improve robustness through model diversity, noise injection, or disagreement, but this paper targets overlap among adversarial subspaces.The paper positions gradient-coherence reduction as a distinct ensemble objective.
- Gradient similarity: The paper adapts cosine-similarity-based gradient measurement to quantify similarity among ensemble gradients and assess adversarial-subspace overlap.This differs from earlier use of gradient cosine similarity for auxiliary-task data efficiency.
6. Discussion
The discussion extends GAL beyond black-box defense to adversarial-input detection and stronger black-box attack generation. The latter direction is explicitly left for future evaluation.
- Adversarial Attack Detection: GAL could support adversarial-input detection by encouraging high gradient disagreement on inputs outside the training distribution.The proposed detection variant would use GAL without cross-entropy on such examples.
- Better Black-Box Attacks: Diverse ensembles might also generate more transferable black-box attacks, but the paper leaves this possibility unevaluated.The discussion identifies transferability of attacks generated on diverse surrogates as future work.
7. Conclusion
The paper defends against transfer-based attacks by training ensembles with uncorrelated loss gradients. Diversity Training reduces shared adversarial-subspace overlap and can combine with existing defenses for stronger robustness.
- Conclusion: Gradient Alignment Loss trains ensembles with uncorrelated loss functions by minimizing coherence among their gradient vectors.This is the paper’s central regularization mechanism.
- Conclusion: Diversity Training improves adversarial robustness by reducing overlap in the models’ shared adversarial subspace.The conclusion connects gradient misalignment with fewer shared adversarial directions.
- Conclusion: The proposed method can be combined with existing defenses to create a stronger defense against transfer-based attacks.The conclusion states the combination as an additional benefit of the approach.