Source-linked AI summary
Towards One-for-All Robustness Across a Continuum of Threat Levels
Zhichao Hou, Xiaorui Liu
TL;DR
Threat-specific adversarial training generalizes poorly across budgets, making specialized model collections impractical for continuous threat spaces. TCN uses a shared representation with lightweight threat conditioning and distributional training, matching or surpassing specialized baselines while generalizing to unseen budgets with minimal overhead.
Problem
Threat-specific adversarially trained models degrade under mismatched conditions, while maintaining specialized models becomes intractable across a growing or continuous threat space.
Method
TCN factorizes representations into a shared threat-invariant backbone and lightweight threat-conditional adaptor, using Fourier embeddings, affine modulation, and training over perturbation-budget distributions.
Results
TCN matches or surpasses specialized adversarial-training baselines across test budgets, generalizes to unseen budgets and other domains, and adds only 4.6% parameters with less than 0.1% FLOP increase.
Takeaways & Limitations
Modeling adversarial robustness as continuous conditional learning provides a scalable single-model approach for dynamic threat environments.
Abstract
from arXiv · showhide
Adversarially robust models often overfit to a specific attack budget, necessitating multiple specialized models for diverse and dynamic adversarial environments, a strategy that becomes fundamentally intractable as the threat space grows. This raises an open challenge: can we achieve strong robustness across a continuum of threat levels within a single model? We propose the Threat Conditional Network (TCN), grounded in a representation factorization framework that decomposes representation learning into a threat-invariant shared backbone and a lightweight threat-conditional adaptor. TCN conditions a single model on the perturbation level via Fourier-based embeddings and channel-wise affine modulation, and is trained against a distribution over perturbation budgets, enabling flexible and seamless adaptation across an infinite continuum of threat levels during inference. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that TCN matches or surpasses a full ensemble of budget-specialized models with a single set of parameters, generalizes to unseen perturbation budgets, and transfers robustly under mismatched threat conditions, with only 4.6\% parameter overhead. These contributions chart a promising path toward adaptive and generalizable robustness in dynamic and diverse threat environments.
1 Introduction
Adversarially trained models specialize in their training threat, degrading under mismatched conditions. Because maintaining specialized models becomes intractable across a continuous threat space, the paper proposes TCN as a single adaptive model.
- Models trained under specific threats perform well on seen threats but degrade significantly under mismatched conditions.
- Existing approaches, including objective redesign, capacity scaling, multi-threat training, and ensembles, yield limited gains across diverse threats.
- Maintaining specialized models becomes fundamentally intractable as the threat space grows large or continuous.
- TCN factorizes representations into a threat-invariant shared component and a lightweight threat-conditional component.
2 Challenges in Scaling Adversarial Defenses Across Threat Levels
Adversarial defenses are tightly coupled to fixed threat levels, producing a diagonal performance pattern in which matched training and testing budgets work best. This specialization makes model collections increasingly costly and infeasible as threats become fine-grained or continuous.
- Most adversarial defenses degrade substantially when evaluated against threats outside their predefined scope.
- Real-world deployment across diverse and unpredictable threats motivates maintaining separate models optimized for individual threat regimes.
- In Figure 1, matching training and test budgets nearly always produce the highest accuracy, forming a diagonal dominance pattern.
- Stronger adversarial training sacrifices clean accuracy and low-budget robustness, whereas weaker training fails against stronger attacks.
- Training one model per threat level scales linearly in cost and becomes intractable for fine-grained or continuous threat spaces.
3 Threat Conditional Representation Learning
The paper addresses representation conflict across threat levels by combining a shared backbone with lightweight threat-conditioned transformations. TCN embeds threat levels, modulates intermediate features, and trains over a distribution of perturbation budgets.
- 3.1 Representation Factorization: Existing training methods face conflicting representation demands: stronger threats favor invariance, while weaker threats preserve discriminability.
- 3.1 Representation Factorization: TCN replaces separate threat-specific backbones with one shared backbone and a lightweight adaptor, reducing parameters from O(N|F|) to O(|F|+|T|).
- 3.2 TCN: Threat Conditional Network: The conditional module intercepts intermediate backbone representations and transforms them according to the input threat level.
- 3.2 TCN: Threat Conditional Network: Fourier feature embeddings map scalar threat levels into a high-dimensional space to model nonlinear dependencies and support smooth interpolation.
- 3.2 TCN: Threat Conditional Network: Channel-wise affine modulation applies threat-dependent parameters to intermediate features, allowing one backbone to realize different feature geometries.
- 3.3 Distributional Adversarial Training across Threat Levels: Sampling perturbation levels from a distribution during training encourages a decision rule effective across varying attack strengths rather than one fixed budget.
4 Experiments
Experiments evaluate TCN across datasets, architectures, threat budgets, attacks, transfer settings, ablations, and computational cost. Across these evaluations, TCN approximates specialized-model performance while improving cross-budget robustness, transferability, unseen-budget generalization, and parameter efficiency.
- Evaluation setup: Experiments cover CIFAR-10, CIFAR-100, and Tiny-ImageNet with ResNet-18 and WideResNet-28-10 backbones.ResNet-18 is the default architecture unless otherwise specified.
- One-for-all defense: Models trained at individual perturbation budgets specialize to matched test budgets and degrade substantially under mismatched settings, whereas TCN approximates the best diagonal with one conditional model.TCN uses distributional adversarial training over the full threat-level set.
- State-of-the-art performance: 74.66% average accuracy: TCN with adaptive conditioning outperforms all listed baselines using 11.69M parameters.TCN with fixed conditioning reaches 73.14%; MoRE and Ensemble use 55.87M parameters and achieve 67.75% and 65.14% average accuracy, respectively.
- Transferability analysis: Under mismatched conditions, TCN preserves coherent hidden-representation structure while vanilla adversarial-training representations fragment and lose discriminability.The visualization supports a family of representations parameterized by adversarial level.
- Transferability analysis: TCN matches or surpasses the best fixed-budget adversarial-training baseline across FGSM, PGD, and AutoAttack test budgets with one model.Fixed-budget AT models perform well near their training budget but degrade at other budgets.
- Complexity analysis: 0.52M parameters (+4.6%): TCN-ResNet18 adds less than 0.1% relative forward FLOPs compared with ResNet18.The conditional modules increase model size by 1.97 MB; forward FLOPs are 1.1168 G versus 1.1158 G.
5 Related Works
Prior work improves robustness through loss redesign, capacity scaling, multi-threat training, and ensembles, but generalization across diverse threats remains limited.
- Multi-threat training exposes models to heterogeneous attacks simultaneously.
- Loss redesign methods such as TRADES and MART seek a better accuracy-robustness trade-off.
- Ensemble-based methods address diverse threats by combining specialized models.
6 Conclusion
The paper reformulates adversarial robustness as threat-conditional prediction and introduces TCN, a single model that adapts continuously across perturbation levels. Experiments show strong and consistent performance across adversarial scales.
- TCN reformulates robustness as threat-conditional prediction with a shared representation across adversarial scales.
- TCN adapts continuously to perturbation levels within a single model.
- Extensive experiments demonstrate strong and consistent performance across diverse threat levels.
A.1 Experimental Results under Different Attacks
Across FGSM, PGD, and AutoAttack on CIFAR-10, specialized adversarial-training models overfit their training budgets, while TCN remains competitive across test budgets and architectures.
- TCN consistently matches or surpasses the best-performing adversarial-training baseline at every test budget across three attack types and two architectures.
- Adversarial-training models perform best at matched test budgets but degrade outside their training range.
- 0.00% accuracy occurs for AT with ϵ = 0 255 at test budget 8 255 under both PGD and AutoAttack.
- Tables 9 and 10 evaluate accuracy under FGSM, PGD, and AutoAttack across varying test budgets on CIFAR-10 using ResNet18 and WideResNet-28-10.
A.2 Embedding Analysis
The embedding analysis compares independently trained vanilla models with a single conditioned TCN across five adversarial levels and multiple representation depths. TCN produces smaller, smoother representation changes under conditioning.
- Experimental setup: Vanilla adversarial training independently learns five models, whereas TCN uses one conditionable model.
- Experimental setup: Representations are compared at seven depths, including the input, input layer, first convolution, four residual blocks, and final logits.
- Findings: TCN induces substantially smaller representation differences across adversarial levels than independently trained vanilla models.
- Findings: TCN representations change gradually as conditioning increases, while vanilla models produce highly inconsistent embeddings on identical clean inputs.
A.3 Clean-level Sampling Bias
The clean-bias coefficient M controls the sampling probability of clean examples, creating a trade-off between clean accuracy and adversarial robustness. An intermediate bias level provides the best overall balance.
- M controls the probability of sampling clean examples: larger M emphasizes clean training, while smaller M emphasizes adversarial levels.The sampling probability is p = M/(M+N).
- Increasing M generally improves clean accuracy on CIFAR-10 and CIFAR-100 but does not consistently improve robustness at larger perturbation budgets.
- The best overall trade-off between clean and adversarial performance is achieved at an intermediate clean-sampling bias.