PaperlayerView exact revision on arXiv

Source-linked AI summary

You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle

Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, Bin Dong

arXiv:1905.00877v6stat.MLcs.LGmath.OC

TL;DR

Adversarial training improves robustness but is costly because adversarial examples require repeated inner-loop optimization. The paper recasts training as a differential game, uses PMP to identify first-layer coupling, and proposes YOPO, which achieves comparable accuracy with substantially less computation.

  • Problem

    Adversarial training is effective for robustness, but generating adversarial examples creates much higher computational cost than standard network training.

  • Method

    YOPO recasts adversarial training as a discrete-time differential game and uses PMP to decouple adversary updates from network training, focusing computation on the first layer.

  • Results

    YOPO achieves approximately 4∼5 times speedup over original PGD training with comparable MNIST/CIFAR10 accuracy.

  • Takeaways & Limitations

    YOPO reduces full forward and backward propagation to one per group of adversary updates while preserving comparable defense accuracy.

  • Takeaways & Limitations

    The PMP result assumes twice continuous differentiability, boundedness conditions, and convexity of layer dynamics and regularization sets.

Abstract

from arXiv · show

Deep learning achieves state-of-the-art results in many tasks in computer vision and natural language processing. However, recent works have shown that deep networks can be vulnerable to adversarial perturbations, which raised a serious robustness issue of deep networks. Adversarial training, typically formulated as a robust optimization problem, is an effective way of improving the robustness of deep networks. A major drawback of existing adversarial training algorithms is the computational overhead of the generation of adversarial examples, typically far greater than that of the network training. This leads to the unbearable overall computational cost of adversarial training. In this paper, we show that adversarial training can be cast as a discrete time differential game. Through analyzing the Pontryagin's Maximal Principle (PMP) of the problem, we observe that the adversary update is only coupled with the parameters of the first layer of the network. This inspires us to restrict most of the forward and back propagation within the first layer of the network during adversary updates. This effectively reduces the total number of full forward and backward propagation to only one for each group of adversary updates. Therefore, we refer to this algorithm YOPO (You Only Propagate Once). Numerical experiments demonstrate that YOPO can achieve comparable defense accuracy with approximately 1/5 ~ 1/4 GPU time of the projected gradient descent (PGD) algorithm. Our codes are available at https://https://github.com/a1600012888/YOPO-You-Only-Propagate-Once.

1 Introduction

Adversarial training improves robustness but is computationally expensive because generating adversarial examples requires repeated optimization. YOPO uses the network’s first-layer structure to reduce propagation while retaining comparable accuracy.

  • Motivation: 40 PGD inner iterations can make adversarial-training computation about 40 times that of regular training.The inner loop typically uses multi-step gradient descent to obtain adversarial examples.
  • Approach: YOPO formulates adversarial training as a discrete-time differential game and derives its Pontryagin’s Maximum Principle.The formulation exploits the structure of deep neural networks’ min-max objective.
  • Approach: PMP shows that adversarial perturbations couple only with first-layer weights, motivating adversary updates focused on that layer.YOPO decouples adversary updates from network-parameter training.
  • Related work: Adversarial training is favored among defense approaches because it largely avoids the obfuscated-gradient problem.The paper contrasts it with orthogonal regularization, Bayesian methods, TRADES, rejection, Jacobian regularization, generative defenses, and others.
  • Results: 4∼5 times speedup over original PGD training is achieved with comparable MNIST/CIFAR10 accuracy.The paper also reports better clean and robust accuracy with YOPO combined with TRADES in less than half the time.

2 Differential Game Formulation and Gradient Based YOPO

The paper recasts adversarial training as a differential game, then develops gradient-based YOPO by freezing intermediate derivatives and separating adversary and weight updates. This reduces full propagation while retaining repeated adversary updates.

  • 2.1 The Optimal Control Perspective and Differential Game: Adversarial training becomes a two-player zero-sum differential game: the network minimizes label loss while the adversary modifies inputs to induce false predictions.Network weights and adversarial perturbations control separate parts of the game dynamics.
  • 2.1 The Optimal Control Perspective and Differential Game: The network dynamics use one nonlinear transformation per layer, with adversarial perturbations constrained in an infinity ball.The formulation includes data-fitting losses and regularization of layer parameters.
  • 2.2 Gradient Based YOPO: PMP characterizes optimal solutions and motivates YOPO’s general formulation, while gradient ascent/descent provides an illustrative special case.The paper postpones the full PMP derivation and general YOPO version to Section 3.
  • 2.2 Gradient Based YOPO: PGD-r alternates r adversary gradient-ascent steps with an SGD weight update, requiring r full forward and backward sweeps per weight update.This repeated propagation is identified as the main source of PGD-type adversarial-training slowness.
  • 2.2 Gradient Based YOPO: YOPO introduces a slack variable and freezes derivatives at levels 1 through T−1 during inner adversary updates.The frozen quantity is treated as constant within the adversary-update loop.
  • 2.2 Gradient Based YOPO: YOPO-m-n accesses data m × n times but requires only m full forward and backward propagations.PGD-r instead propagates data r times for r full propagations.
  • 2.2 Gradient Based YOPO: YOPO uses each full propagation to update weights, so intermediate perturbation updates are not wasted and multiple updates can occur per iteration.This combination is presented as accelerating standard PGD adversarial training.
  • 2.2 Gradient Based YOPO: Free-m is essentially YOPO-m-1, except YOPO-m-1 delays the weight update until the whole mini-batch is processed.The paper identifies Free-m as a concurrent related method.

3 The Pontryagin’s Maximum Principle for Adversarial Training

The paper casts adversarial training as a discrete-time differential game and derives its Pontryagin Maximum Principle (PMP). The resulting coupling structure motivates YOPO, which decouples adversary updates from network training to reduce full propagation.

  • PMP formulation: The PMP provides necessary optimality conditions for the discrete-time differential game underlying adversarial training.The formulation uses layer-wise Hamiltonian maximization requirements.
  • Relation to back-propagation: YOPO can be interpreted as solving the PMP, while classical back-propagation-based gradient descent also corresponds to an iterative PMP-solving procedure.The forward and backward dynamical systems correspond to neural-network propagation and back-propagation.
  • Key coupling observation: The adversarial perturbation is coupled only with the first-layer parameters θ0, motivating YOPO’s focused adversary computation.This observation follows directly from the PMP theorem.
  • YOPO variants: YOPO-m-n performs n gradient-descent adversary updates while repeatedly passing one data point m times to improve the Hamiltonian minimization.Network weights are optimized through gradient ascent on the Hamiltonian.

4 Experiments

Experiments on MNIST and CIFAR10 compare YOPO with PGD-based adversarial training and TRADES. YOPO achieves comparable robustness with substantially lower computation, including approximately 4–5× speedup over original PGD training.

  • Overall results: YOPO achieves comparable performance to PGD adversarial training on MNIST and CIFAR10 with much lower computational cost.The experiments also compare YOPO with the concurrent For Free method.
  • MNIST: 250 seconds versus more than 1250 seconds: YOPO reaches MNIST results comparable to the best in [5], while PGD-40 reaches the same level.The accuracy-time comparison is shown in Figure 3(a).
  • CIFAR10 with PreAct-Res18: Around half the computation per epoch: YOPO-5-3 achieves comparable robust accuracy to [24] under PreAct-Res18 on CIFAR10.The accuracy-time comparison is shown in Figure 3(b).
  • CIFAR10 with Wide ResNet34: YOPO-5-3 provides similar acceleration against PGD-10 on Wide ResNet34, while YOPO-3-5 offers more aggressive speedup with a slight robustness drop.Naively reducing PGD backward passes does not produce comparable results within the same computation time as YOPO.

5 Conclusion

The paper develops YOPO, an efficient adversarial-training strategy derived from a discrete-time differential-game formulation and PMP. YOPO reduces repeated full propagation while maintaining the trained network’s performance, as supported by experiments.

  • YOPO recasts adversarial training as a discrete-time differential game and derives its Pontryagin’s Maximum Principle.
  • The PMP reveals that adversarial updates couple only with the first-layer weights, motivating their separation from network back-propagation.
  • YOPO avoids repeated full forward and backward propagation, reducing computational time according to the experiments.

A.1 Proof of Theorem 1

The proof establishes a maximum principle for adversarial training by combining a discrete-time dynamical-system PMP with separate treatment of network weights and adversarial controls.

  • The theorem assumes twice continuous differentiability, uniformly bounded functions and derivatives, and convex reachable-function sets.
  • Network states evolve layer by layer according to x_i,t+1 = f_t(x_i,t, θ_t), with an initial adversarially perturbed transition.
  • The proof introduces co-state processes and a per-layer Hamiltonian whose maximization characterizes the network-parameter dynamics.
  • The proof uses linearization and reachable-state separation to derive the maximal principle and then applies it to the adversarially shifted initial state.
  • For the adversarial-control step, network parameters are fixed while η_i controls the first transition, yielding the adversarial Hamiltonian condition.

B.1 MNIST

The MNIST experiments use PGD-40 adversarial training with convolutional and fully connected networks, then evaluate robustness under PGD-40 and CW attacks.

  • MNIST experiments use four convolutional layers followed by three fully connected layers, with perturbation size ϵ = 0.3 in the infinity norm.
  • Models are trained for 55 epochs with batch size 256, momentum 0.9, weight decay 5e−4, and a learning-rate drop at epoch 45.
  • Robustness is measured using PGD-40 and CW attacks, with CW coefficient c = 5e2 and learning rate 1e−2.

B.2 CIFAR-10

The CIFAR-10 experiments test YOPO and PGD adversarial training with Preact-ResNet18 and Wide ResNet-34 under bounded adversarial perturbations.

  • CIFAR-10 experiments use Preact-ResNet18 and Wide ResNet-34 as the evaluation architectures.
  • The perturbation size is ϵ = 8/255 in the infinity norm, and testing uses 20-step PGD with step size 2/255.
  • PGD adversarial training runs for 105 epochs, whereas YOPO-m-n runs for 40 epochs, both beyond the epochs needed for convergence.

B.3 TRADES

TRADES uses a consistency-loss-based adversarial objective and generates attacks through projected gradient updates before updating network weights. TRADES-YOPO decouples adversary calculation from backpropagation, achieving lower computation while maintaining or improving performance relative to TRADES-10.

  • TRADES objective: TRADES uses a surrogate consistency loss to quantify the trade-off between adversarial and non-adversarial risks.
  • Adversary update: For each clean input, PGD updates the adversarial example by ascending the gradient of the consistency loss and projecting it within the perturbation bound.
  • TRADES-10: TRADES-10 performs 10 adversary-update iterations for each input before updating the network weights.
  • TRADES-YOPO: TRADES-YOPO decouples adversarial calculation from network updating and uses only the final perturbation to compute the final loss term.
  • Results: With less than half the time of TRADES-10, TRADES-YOPO-3-4 achieves an even better result than its baseline.Experiments used a mini-batch size of 256 and ran for 105 epochs on NVIDIA Tesla P100 GPUs.
Loading 1905.00877v6…