PaperlayerView exact revision on arXiv

Source-linked AI summary

Attacks Which Do Not Kill Training Make Adversarial Learning Stronger

Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, Mohan Kankanhalli

arXiv:2002.11242v2cs.LGstat.ML

TL;DR

The paper addresses whether minimax adversarial training must trade natural generalization for adversarial robustness. It proposes friendly adversarial training, implemented with early-stopped PGD and justified by an adversarial-risk upper bound. The reported outcome is that robustness can be achieved without compromising natural generalization, although performance degrades under larger-ϵtrain C&W attacks.

  • Problem

    Minimax adversarial training is conservative and sometimes hurts natural generalization, raising whether robustness requires sacrificing generalization.

  • Method

    Friendly adversarial training selects least adversarial examples among confidently misclassified data, using early-stopped PGD to approximate the objective.

  • Results

    FAT achieves better standard accuracy while maintaining competitively robust accuracy compared with conventional adversarial training.

  • Takeaways & Limitations

    Adversarial robustness can be achieved without compromising natural generalization, while FAT also supports computational efficiency and larger training perturbation bounds.

  • Takeaways & Limitations

    Both standard and friendly adversarial training perform poorly under C&W attacks with larger ϵtrain, attributed to mismatch with PGD-based training.

Abstract

from arXiv · show

Adversarial training based on the minimax formulation is necessary for obtaining adversarial robustness of trained models. However, it is conservative or even pessimistic so that it sometimes hurts the natural generalization. In this paper, we raise a fundamental question---do we have to trade off natural generalization for adversarial robustness? We argue that adversarial training is to employ confident adversarial data for updating the current model. We propose a novel approach of friendly adversarial training (FAT): rather than employing most adversarial data maximizing the loss, we search for least adversarial (i.e., friendly adversarial) data minimizing the loss, among the adversarial data that are confidently misclassified. Our novel formulation is easy to implement by just stopping the most adversarial data searching algorithms such as PGD (projected gradient descent) early, which we call early-stopped PGD. Theoretically, FAT is justified by an upper bound of the adversarial risk. Empirically, early-stopped PGD allows us to answer the earlier question negatively---adversarial robustness can indeed be achieved without compromising the natural generalization.

1. Introduction

The paper argues that minimax adversarial training can harm natural generalization and proposes friendly adversarial training, which uses confidently misclassified but least adversarial examples. Early-stopped PGD implements this strategy while retaining robust accuracy and improving natural accuracy.

  • Motivation: Minimax adversarial training can hurt natural generalization because highly adversarial examples may cross the decision boundary and create a cross-over mixture.PGD-generated variants can enter the peer class’s natural-data region, making simultaneous fitting difficult.
  • Friendly adversarial training: Friendly adversarial training selects confidently misclassified adversarial data that minimize the loss, rather than always using loss-maximizing examples.FAT trains on wrongly predicted adversarial data by minimizing the loss and correctly predicted adversarial data by maximizing it.
  • Friendly adversarial training: FAT follows a curriculum strategy by beginning with least adversarial examples and progressively using stronger examples during training.This progression is intended to avoid making learning extremely difficult at the outset.
  • Implementation: Early-stopped PGD implements FAT by stopping once an adversarial example is misclassified, helping avoid cross-over mixture while saving backward propagations.The method is presented as an easy modification of conventional adversarial-data search algorithms.
  • Reported benefits: FAT can be adapted from standard adversarial training, TRADES, and MART, while achieving better standard accuracy and competitively robust accuracy than conventional adversarial training.The paper also reports computational efficiency and the ability to use larger training perturbation bounds.
  • Reported benefits: The paper concludes that adversarial robustness can be achieved without compromising natural generalization.This conclusion is attributed to FAT’s handling of adversarial examples and the cross-over mixture problem.

2. Standard Adversarial Training

Standard adversarial training formulates robustness as minimax optimization, using PGD to find loss-maximizing perturbations within an ϵ-ball. This approach can produce mixed adversarial representations and sometimes hurt natural generalization.

  • Learning Objective: Standard adversarial training minimizes model loss while maximizing loss over perturbations within each natural example’s ϵ-ball.The loss uses the model’s score function and a base loss such as cross-entropy.
  • Learning Objective: The conventional objective maximizes adversarial loss regardless of whether the model predicts each adversarial example correctly.This prediction-agnostic treatment is the baseline later contrasted with FAT.
  • Projected Gradient Descent (PGD): PGD approximately solves the inner maximization by iteratively perturbing data and projecting it back onto the ϵ-norm ball.A fixed iteration count K yields the PGD-K algorithm.
  • Related Methods: PGD adversarial training is one of several empirical-defense approaches, alongside regularization, curriculum, efficiency, ensemble, unlabeled-data, TRADES, and MART methods.The cited alternatives address robustness or training efficiency in different ways.

3. Friendly Adversarial Training

Friendly adversarial training replaces prediction-agnostic loss maximization with a prediction-aware objective that selects confidently misclassified adversarial data with the smallest loss. The paper derives an upper bound on adversarial risk to justify this formulation.

  • 3.1. Learning Objective: FAT selects friendly adversarial data by minimizing loss among perturbations that are misclassified with at least confidence margin ρ.The selected example is a least-adversarial member of the confidently misclassified set.
  • 3.2. Upper Bound on Adversarial Risk: Theorem 1 defines adversarial risk as the probability that some perturbation within Bϵ[X] is classified incorrectly.The theorem applies to surrogate losses that upper-bound 0/1 loss.
  • 3.2. Upper Bound on Adversarial Risk: The FAT objective still maximizes loss for correctly predicted adversarial data but minimizes loss, with a ρ violation, for wrongly predicted adversarial data.This makes the inner objective depend on the model’s adversarial prediction.
  • 3.2. Upper Bound on Adversarial Risk: The paper states that FAT’s prediction-aware upper bound is tighter than conventional adversarial-training bounds that maximize loss regardless of prediction.The bound retains loss maximization for correct predictions and uses friendly minimization for incorrect ones.
  • 3.2. Upper Bound on Adversarial Risk: Friendly adversarial data are intended to fine-tune the decision boundary without severe cross-over mixture, while correctly predicted adversarial data retain the largest loss.The wrongly predicted examples are described as having a small loss-value distance ρ from the decision boundary.

4. Key Component of FAT

FAT operationalizes friendly-example search with PGD-K-τ, which stops PGD early after misclassification and thereby reduces cross-over mixture. Layer-output visualizations compare conventional PGD with the resulting less-mixed representations.

  • 4. Key Component of FAT: PGD-K-τ is an efficient early-stopped PGD algorithm designed to search for friendly adversarial data and alleviate cross-over mixture.The method is presented as the key algorithmic component of FAT.
  • 4.1. Early-Stopped PGD: The algorithm takes data, labels, model, loss, maximum PGD steps K, stopping step τ, perturbation bound ϵ, and step size α, and returns adversarial data.The perturbation is projected onto the ϵ-norm ball centered at the natural input.
  • 4.1. Early-Stopped PGD: Unlike conventional PGD-K, PGD-K-τ accounts for the model’s prediction when generating adversarial data.It returns misclassified examples with small loss or correctly classified examples with large loss.
  • 4.2. PGD-K-τ Alleviates Cross-over Mixture: The cross-over mixture can arise in intermediate-layer outputs even when it is not obvious in the original input space.The paper reports additional visualizations across layers and architectures, including WRN-40-4 and t-SNE.
  • 4.2. PGD-K-τ Alleviates Cross-over Mixture: When τ=0, PGD iterations stop immediately once adversarial data are wrongly predicted by the current network.Figure 3 compares PGD-20 with PGD-20-0 after warm-up training on selected CIFAR-10 classes.
  • 4.2. PGD-K-τ Alleviates Cross-over Mixture: Layer #7 outputs are not significantly mixed for natural data or PGD-20-0 friendly data, whereas PGD-20 produces significant mixing across classes.The visualization uses PCA to project the high-dimensional layer output into two dimensions.

5. Realization of FAT

FAT realizes friendly adversarial training through PGD-K-τ, with τ controlling the trade-off between natural and robust accuracy and the computational cost of generating adversarial data. Its early stopping also produces an automatic curriculum in which training increasingly uses stronger adversarial data.

  • 5.1. Selection of Step τ: FAT uses PGD-K-τ, with standard adversarial training recovered as the special case τ = K.The same construction extends FAT to TRADES and MART, where τ = K recovers those methods as special cases.
  • 5.1. Selection of Step τ: Larger τ significantly lowers standard test accuracy, while robust test accuracy improves only initially and then reaches a plateau.For both Small CNN and ResNet-18, robust accuracy plateaus at larger τ values, whereas standard accuracy continues decreasing when τ exceeds 2.
  • 5.1. Selection of Step τ: Selecting τ from {0, 1, 2, 3} is sufficient for FAT, although the useful set depends on step size α and maximum PGD step K.The paper treats τ as a hyper-parameter and uses it to fine-tune FAT performance.
  • 5.2. Smaller τ is Computationally Efficient: Early-stopped PGD makes FAT computationally efficient by reducing the backward propagations needed to generate adversarial data.Conventional PGD-K uses K backward propagations per mini-batch, whereas smaller τ further reduces FAT’s average requirement.
  • 5.2. Smaller τ is Computationally Efficient: As training progresses, FAT requires more PGD steps to find misclassified adversarial data, indicating increasing adversarial robustness.Average backward propagations also surge after learning-rate reductions at epochs 60 and 90, supporting learning-rate decreases during adversarial training.
  • 5.2. Smaller τ is Computationally Efficient: FAT functions as curriculum training by automatically moving from milder to stronger adversarial data as more PGD steps become necessary.This differs from CAT and DAT, which lack a principled way to determine when adversarial hardness should increase.

6. Experiments

Experiments across CIFAR-10, SVHN, and Wide ResNet models show that FAT preserves natural accuracy better while maintaining or improving robustness under larger training perturbation bounds.

  • 6.1. FAT can Enable Larger Perturbation Bound ϵtrain: FAT achieves higher standard test accuracy than Madry, with the gap widening as ϵtrain increases.Standard adversarial training suffers increasingly from the cross-over mixture issue, whereas FAT generalization is less affected by larger ϵtrain.
  • 6.1. FAT can Enable Larger Perturbation Bound ϵtrain: For stronger attacks, FAT maintains higher robust accuracy at larger ϵtrain, while Madry’s robust accuracy eventually drops sharply.Madry essentially fails for ϵtrain > 0.06 on CIFAR-10 and ϵtrain > 0.03 on SVHN.
  • 6.1. FAT can Enable Larger Perturbation Bound ϵtrain: FAT with τ = 3 drops when ϵtrain > 0.09 on CIFAR-10, suggesting larger τ can also encounter cross-over mixture issues.The authors identify τ as an important hyper-parameter governing the robustness–standard-accuracy trade-off.
  • 6.1. FAT can Enable Larger Perturbation Bound ϵtrain: Increasing τ trades standard accuracy for robustness: τ = 0 has higher standard accuracy, while τ = 1 provides slightly lower standard accuracy and increased robustness.Robust accuracy rises at smaller τ values but plateaus at larger τ values, whereas standard accuracy continues to decrease.
  • 6.2. Performance Evaluations on Wide ResNet: FAT achieves competitive or state-of-the-art robustness on large-capacity Wide ResNet models without compromising natural generalization.Applying early-stopped PGD to TRADES, larger WRN-58-10, and MART further improves reported performance.

7. Conclusion

The conclusion presents FAT as an efficient early-stopped-PGD formulation that addresses cross-over mixture, supports larger training perturbation bounds, and remains effective on large-capacity networks.

  • 7. Conclusion: FAT formulates friendly adversarial training and approximately implements it by stopping PGD early.The formulation is theoretically justified by an upper bound on adversarial risk.
  • 7. Conclusion: FAT is computationally efficient, follows curriculum-training principles, and helps relieve the cross-over mixture problem.Its friendly adversarial data become harder as the model’s predictions change during training.
  • 7. Conclusion: FAT enables training with larger perturbation bounds and achieves competitive performance on large-capacity networks.The paper also identifies choosing optimal τ and finding friendly data beyond PGD-K-τ as directions for further research.
  • 7. Conclusion: FAT trains with confidently misclassified adversarial data that minimize loss, while correctly predicted adversarial data maximize loss.The friendly example is selected among adversarial points satisfying a misclassification-confidence constraint.

A.2. Proofs

The proof section decomposes adversarial risk into natural and boundary-related components and uses this structure to justify FAT’s focus on confidently misclassified, low-loss adversarial data.

  • A.2. Proofs: Adversarial risk Rrob(f) is decomposed into natural risk Rnat(f) and a boundary risk involving correctly classified natural data near the decision boundary.The decomposition identifies decision-boundary proximity as the source of additional robust risk.
  • A.2. Proofs: Theorem 1 provides an upper bound on adversarial risk using a non-negative surrogate loss that upper bounds 0/1 loss.The bound includes a standard-test-accuracy term and an adversarial-loss term.
  • A.2. Proofs: FAT’s objective replaces inner loss maximization with loss minimization subject to a margin ensuring confident misclassification.Among qualifying adversarial examples, it selects the one with the smallest adversarial loss.
  • A.2. Proofs: The analysis motivates using adversarial data near the classifier to fine-tune the decision boundary incrementally.Wrongly predicted adversarial data increase toward lower loss while violating a small constant ρ.

B. Alternative Adversarial Data Searching Algorithm

An alternative adversarial-data search removes the ϵ-ball projection from the update, and experiments compare this method with Madry training across step sizes.

  • B. Alternative Adversarial Data Searching Algorithm: The alternative search algorithm modifies the adversarial-data update by removing the ϵ-ball projection constraint.The natural example is x0 and α > 0 is the step size.
  • B. Alternative Adversarial Data Searching Algorithm: Small CNN and ResNet-18 experiments compare alternative-search FAT with τ = 0, 1, 3 against Madry over α ∈ [0.003, 0.015].The maximum PGD step is fixed at K = 10 and evaluation uses FGSM, PGD-20, PGD-100, and C&W∞.

C.1. Output Distributions of Small CNN’s Intermediate Layers

The section extends the intermediate-layer output-distribution visualization from Figure 3 to additional layers and network settings. It compares natural, PGD-20 adversarial, and PGD-20-0 friendly adversarial data, showing substantial mixing for PGD-20 data but not for friendly adversarial data.

  • Small CNN: Figure 11 extends the Small CNN visualization to layers #7 and #8, complementing the earlier visualization of layer #7.The Small CNN has 8 layers: 6 convolutional and 2 fully connected layers.
  • Experimental setup: The Small CNN experiment uses CIFAR-10 bird and deer classes, with PGD-20 perturbations generated using α = 0.007 and ϵ = 0.031.Layers #6, #7, and #8 are visualized using PCA and t-SNE.
  • Small CNN: For Small CNN, natural-data outputs are not significantly mixed, PGD-20 outputs are significantly mixed, and PGD-20-0 outputs are not significantly mixed.The distributions are shown in Figure 11 for natural, adversarial, and friendly adversarial data, respectively.
  • Experimental setup: The WRN-40-4 experiment uses CIFAR-10 deer, horse, and truck classes and visualizes layers #38, #40, and #41 under the same PGD-20 step size and perturbation bound.The distributions are shown using PCA and t-SNE.
  • WRN-40-4: The same distribution pattern is reported for WRN-40-4: natural and PGD-20-0 outputs are not significantly mixed, whereas PGD-20 outputs are significantly mixed.Figure 12 examines intermediate layers of WRN-40-4 using the same three data conditions.

G.4. C&W Attack Analysis

The C&W analysis explains why larger training perturbation bounds can improve PGD robustness while reducing C&W robustness. The authors attribute this divergence to the attacks generating adversarial data through different losses and search directions.

  • Limitation: Both standard adversarial training and friendly adversarial training perform poorly under C&W attack when ϵ_train is larger than approximately 0.075.The paper explicitly identifies this regime as an example of the observed failure pattern.
  • C&W formulation: In the C&W formulation, κ > 0 encourages finding adversarial data classified as the target class with high confidence.The parameter c balances the two parts of the C&W loss.
  • Analysis: For larger ϵ_train, performance evaluated by PGD attacks increases, while performance evaluated by C&W attack decreases.This pattern is reported for both standard and friendly adversarial training in Figure 6 and Figure 13.
  • Analysis: C&W and PGD generate adversarial data differently because their optimization procedures use different losses and therefore search in different directions.The resulting C&W and PGD adversarial-data distributions are described as inconsistent.

H.2. Training details of FAT for TRADES on Wide ResNet

This section specifies the training and evaluation settings for FAT for TRADES on Wide ResNets. It covers WRN-34-10 and the larger WRN-58-10, with comparisons using standardized attack settings.

  • WRN-34-10 training: FAT for TRADES trains WRN-34-10 for 85 epochs with SGD, momentum 0.9, weight decay 0.0002, and β set separately to 1 and 6.The maximum PGD step is K = 10 with step size α = 0.007; τ increases at epochs 30, 50, and 70.
  • Evaluation: Evaluation uses standard accuracy on natural data and robust accuracy against FGSM, PGD-20, PGD-100, and C&W∞ under ϵ_test = 0.031.The TRADES comparison setting uses α = 0.003 and initially omits random starts.
  • Evaluation: The authors additionally retest TRADES models with random-start PGD attacks and compare the released WRN-34-10 model with their FAT for TRADES model in Table 3.This retest addresses the difference between TRADES’s stated and experimental testing procedures.
  • WRN-58-10: FAT for TRADES is also evaluated on the larger WRN-58-10 using the WRN-34-10 training and testing settings, with β fixed at 6.0.The reported attacks use perturbation bound ϵ_test = 0.031 and step size α = 0.003.
  • Results reporting: Table 3 reports robustness as test accuracy for deep models on CIFAR-10 with random-start evaluation.The table is the reported output of the corresponding comparison setting.
Loading 2002.11242v2…