PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization

Daniel Jakubovitz, Raja Giryes

arXiv:1803.08680v4cs.LGcs.CRcs.CVstat.ML

TL;DR

DNNs can fail under barely noticeable adversarial perturbations, motivating defenses that preserve ordinary accuracy while improving robustness. This paper applies Frobenius-norm Jacobian regularization as post-processing and reports improved robustness across several attacks and datasets with only slight changes to test performance.

  • Problem

    DNNs remain vulnerable to barely noticeable adversarial perturbations that can cause high-confidence errors, creating a challenge for robust deployment.

  • Method

    The method applies Frobenius-norm Jacobian regularization during a second training phase after regular training has finished.

  • Results

    Jacobian regularization produces large robustness improvements against DeepFool, FGSM, and JSMA on MNIST, CIFAR-10, and CIFAR-100 while slightly changing original test performance.

  • Takeaways & Limitations

    The approach can improve robustness using existing trained networks while preserving their original test accuracy to a large extent.

  • Takeaways & Limitations

    For JSMA, the method can fail on MNIST because it minimizes an ℓ2 norm while JSMA targets an ℓ0 pseudo-norm.

Abstract

from arXiv · show

Deep neural networks have lately shown tremendous performance in various applications including vision and speech processing tasks. However, alongside their ability to perform these tasks with such high accuracy, it has been shown that they are highly susceptible to adversarial attacks: a small change in the input would cause the network to err with high confidence. This phenomenon exposes an inherent fault in these networks and their ability to generalize well. For this reason, providing robustness to adversarial attacks is an important challenge in networks training, which has led to extensive research. In this work, we suggest a theoretically inspired novel approach to improve the networks' robustness. Our method applies regularization using the Frobenius norm of the Jacobian of the network, which is applied as post-processing, after regular training has finished. We demonstrate empirically that it leads to enhanced robustness results with a minimal change in the original network's accuracy.

1 Introduction

Deep neural networks achieve strong performance but remain vulnerable to adversarial attacks that use barely noticeable input changes. The paper proposes post-training Jacobian regularization and reports improved robustness across several attacks and datasets with limited accuracy change.

  • Motivation: Small, human-imperceptible input changes can cause DNNs to make high-confidence classification errors.Adversarial examples can also transfer across models, enabling successful black-box attacks.
  • Prior defenses: Robustness defenses either detect adversarial inputs or improve accuracy on perturbed inputs, with this work focusing on the latter.Existing approaches include adversarial training, defensive distillation, BANG, Parseval Networks, and gradient-based regularization methods.
  • Contribution: The proposed method applies Frobenius-norm Jacobian regularization during a second training phase after standard training has stabilized.This post-processing design can use existing pre-trained networks and adds only limited computational overhead.
  • Contribution: The approach regularizes network Jacobians directly, unlike Input Gradient regularization and Cross-Lipschitz regularization.The paper states that this design yields better results than Cross-Lipschitz regularization at lower computational cost.
  • Experiments: Experiments on MNIST, CIFAR-10, and CIFAR-100 evaluate robustness against DeepFool, FGSM, and JSMA.Jacobian regularization surpasses other strategies on FGSM and DeepFool and is competitive on JSMA; combining it with adversarial training further improves results.

2 Jacobian Regularization for Adversarial Robustness

The method penalizes the Frobenius norm of the network Jacobian during post-processing, encouraging similar outputs for nearby inputs. It is combined with cross-entropy training and differs computationally from related regularizers and adversarial training.

  • Method: Adversarial perturbations change network outputs substantially, so post-processing penalizes large classification-function gradients with respect to inputs.The penalty is the Frobenius norm of the network Jacobian evaluated on input data.
  • Notation: The formal setup treats the input as D-dimensional, the output as K-dimensional, and the training set as N examples across L network layers.The regularization weight is controlled by the hyper-parameter λ.
  • Notation: The Jacobian is computed for the last-layer logits before the softmax, with its rows representing output-neuron gradients.The network output after softmax is f(x_i), while J^(L)(x_i)=∇_x z^(L)(x_i).
  • Method: The regularized training loss combines the Jacobian penalty with a standard cross-entropy loss on the training data.The target y_i is represented as a one-hot vector for the correct class.
  • Computational cost: Compared with related approaches, Jacobian regularization adds one back-propagation step per iteration and requires fewer gradient combinations than Cross-Lipschitz regularization.Because it is applied only during post-processing, its computational cost is lower than applying the procedure throughout full training.

3 Theoretical Justification

The paper links Jacobian magnitude to adversarial distance and decision-boundary geometry. Lower Jacobian norms are argued to increase the perturbation needed for misclassification and reduce susceptibility to universal perturbations.

  • Adversarial distance: An adversarial attack seeks the smallest input perturbation that crosses a classification decision boundary.The theoretical analysis focuses on the ℓ2 distance to the closest boundary.
  • Adversarial distance: The Jacobian provides a local relationship between input changes and changes in the network’s output.Comparing nearby same-class and adversarially perturbed samples motivates using Jacobian magnitude as a robustness proxy.
  • Jacobian intuition: A lower Frobenius norm of the Jacobian encourages similar outputs for similar inputs and is therefore associated with greater robustness to small input changes.The paper’s MNIST measurements show that Jacobian regularization reduces the average norm on DeepFool-perturbed inputs more than adversarial training alone.
  • Classification boundaries: The first-order robustness bound combines the output margin with the Jacobian norm, so maximizing margin while minimizing Jacobian magnitude increases the estimated minimal fooling distance.Cross-entropy increases the correct-class margin, while Jacobian regularization reduces the denominator term.
  • Classification boundaries: Regularizing the last-layer Jacobian generally performs better than imposing the regularization on earlier layers.The paper reports degraded robustness when regularizing the L−1 or L−2 layers instead.
  • Decision-boundary curvature: The analysis connects Jacobian regularization with less positive decision-boundary curvature, which makes small universal adversarial perturbations less likely to fool the classifier.The connection uses an approximation of the Hessian by the outer product of Jacobian rows.

4 Experiments

Experiments evaluate Jacobian regularization as a post-processing defense across datasets and attack types, comparing it with alternative regularizers and adversarial training. The method substantially improves robustness under DeepFool and FGSM while preserving test accuracy, but is only competitive under JSMA.

  • Post-processing training: Jacobian regularization is applied after regular training to increase robustness with low additional computational cost.Post-processing preserves the original accuracy-focused training phase and allows existing networks to be adapted.
  • Experimental setup: The evaluation covers MNIST, CIFAR-10, and CIFAR-100 using DeepFool, FGSM, and JSMA, with comparisons against adversarial, Input Gradient, and Cross-Lipschitz regularization.The reported CIFAR-100 results are provided in an appendix and are generally consistent with MNIST and CIFAR-10.
  • DeepFool evaluation: Jacobian regularization provides a much more significant robustness improvement under DeepFool than the other methods, and adversarial-training combinations further enhance the difference.Robustness is measured by the average relative ℓ2 norm of the minimal perturbation needed to fool the network.
  • DeepFool evaluation: Neither examined defense changes test accuracy significantly, although accuracy effects differ across defenses and datasets.On CIFAR-10, Jacobian and Cross-Lipschitz regularization and adversarial training improve accuracy, while Input Gradient regularization reduces it.
  • FGSM evaluation: Jacobian regularization yields higher test accuracy under FGSM than the compared regularizers or adversarial training, with combination training making its advantage more distinct.FGSM varies ϵ and targets the ℓ∞ metric.
  • JSMA evaluation: Under JSMA, the method outperforms the other three methods on CIFAR-10 but performs worse than Input Gradient regularization on MNIST while beating Cross-Lipschitz regularization.The authors relate the MNIST weakness to minimizing an ℓ2 Jacobian metric while JSMA changes one pixel and targets the ℓ0 pseudo-norm.

5 Discussion and Conclusions

The paper concludes that Jacobian regularization substantially improves robustness across attacks and datasets while largely preserving test accuracy, with modest computational demands and identifiable scope limitations.

  • Jacobian regularization improves robustness against DeepFool, FGSM, and JSMA on MNIST, CIFAR-10, and CIFAR-100 while only slightly changing original test performance.
  • Combining Jacobian regularization with adversarial training provides better results than either defense used alone.
  • Compared with Input Gradient regularization, the method performs better under two attacks and competitively under JSMA.
  • Compared with Cross-Lipschitz regularization, the proposed approach performs better under all three examined attacks.
  • The method is applied after regular training and requires only a single additional back-propagation step per training step.
  • Current regularization evaluates the Jacobian norm at input samples, while future work may investigate why original inputs produce optimal results.
  • The Frobenius norm penalizes all Jacobian rows equally; weighted or spectral alternatives could differentiate these penalties, but spectral regularization is more computationally demanding.

A Proof Sketch of Lemma 1

The lemma derives a first-order approximation for the ℓ2 distance from an input to a nearby decision-boundary hyper-surface by locally approximating that surface with its tangent hyper-plane.

  • The resulting approximation concerns an input of class k1 and a perturbed input classified at the boundary separating k1 from k2 under the ℓ2 metric.
  • A decision boundary separating classes k1 and k2 is locally represented by a tangent hyper-plane g(x) = wT x + b = 0.
  • The tangent point x0 lies on the boundary, satisfying g(x0) = wT x0 + b = 0.
  • For points near x0, the distance to the tangent hyper-plane provides the local first-order approximation to distance to the decision boundary.

B Proof of Proposition 3

The proposition connects the minimal adversarial perturbation to the network Jacobian, establishing a lower bound based on class-logit separation and Jacobian magnitude.

  • The proof begins from Corollary 2, which approximates the ℓ2 norm of the minimal perturbation needed to fool the classification function.
  • Proposition 3 directly connects the first-order minimal fooling perturbation to the network’s Jacobian.
  • For the correct class k∗, the first-order ℓ2 norm of the minimal perturbation necessary to fool the classifier is lower bounded by the proposition’s Jacobian-based expression.
  • The bound combines the minimum absolute difference between the correct-class logit and competing logits with the Frobenius norm of the network Jacobian.

C Jacobian regularization of the network’s L −1 layer - Mathematical Analysis

The analysis extends the robustness bound to the Jacobian of the penultimate layer under an orthonormal last-layer assumption, then motivates comparing whole-network and penultimate-layer regularization.

  • Jacobian relation: The whole-network Jacobian satisfies ||J(L)(x)||F ≤ ||W(L)T||F ||J(L−1)(x)||F.
  • Jacobian relation: For an orthonormal W(L) with K columns, its squared Frobenius norm equals K.
  • Experimental comparison: The analysis asks whether regularizing the whole-network or L−1-layer Jacobian yields better robustness and points to experiments with and without a fixed orthonormal last layer.
  • Network setup: The penultimate layer is assumed to be a fully connected layer with M neurons indexed by m = 1, ..., M.
  • Network setup: The last layer maps penultimate activations to logits as z(L) = W(L)T z(L−1) + b(L), with W(L) ∈ R^M×K and b(L) ∈ R^K×1.
  • Proposition 4: With K orthonormal columns in the last-layer weight matrix, Proposition 4 lower bounds the first-order minimal fooling perturbation using the Jacobian of layer L−1.
  • Proof strategy: The proof uses the chain rule and the multiplicative matrix norm inequality to derive the penultimate-layer bound.
  • Proof conclusion: The derived lower bound is stated to match the earlier bound in equation (30).

D Jacobian regularization of the network’s L −1 and L −2 layers - Experiments

Experiments compare Jacobian regularization across the last, L−1, and L−2 layers, including fixed or trainable final-layer weights. Regularizing earlier layers is more computationally demanding and generally less robust, while attack type affects the preferred layer.

  • L−1 Jacobian evaluation required reducing MNIST mini-batches to 50 inputs because M = 1024 ≫ K = 10.The L−1 layer has substantially more neurons than the last layer, increasing computational cost.
  • Regularizing the last layer is significantly better than regularizing the L−1 layer under DeepFool on MNIST.
  • Updating W(L) produced higher original-dataset test accuracy than fixing it, while DeepFool robustness was similar.
  • Under JSMA, L−1 regularization performed better, whereas under FGSM, last-layer regularization performed better.Fixing W(L) generally harmed robustness for the former attack and improved it for the latter.
  • Regularization becomes less effective when applied to earlier layers, with L−2 regularization yielding significantly lower robustness than L−1 or last-layer regularization.The L−2 case is also more computationally demanding because earlier layers can be much wider.

E Experimental results for the CIFAR-100 dataset

The CIFAR-100 experiments use the CIFAR-10 network with smaller mini-batches and λ = 0.02. Adversarial training performs well for small attacks but can harm robustness against stronger attacks.

  • The CIFAR-100 network achieved 59.64% test accuracy before Jacobian-regularized training.Training used mini-batches of 96 inputs and λ = 0.02.
  • CIFAR-100 robustness was evaluated under DeepFool, FGSM, and JSMA attacks.The supplied figure and table references identify CIFAR-100 DeepFool, FGSM, and JSMA evaluations.
  • Adversarial training yielded good FGSM and JSMA robustness for small ϵ values but harmed robustness under stronger attacks.

F Influence of the amount of adversarial examples in the training mini-batch

The amount of adversarial data in each training mini-batch substantially affects final robustness. More perturbed inputs do not necessarily improve robustness; an intermediate balance can be optimal.

  • Different percentages of adversarial examples can produce very different final robustness results.The experiments evaluate MNIST under DeepFool using the ρ̂_adv metric.
  • Adding more perturbed inputs to a training mini-batch does not necessarily increase final adversarial robustness.
  • Combining adversarial training with Jacobian regularization was evaluated alongside adversarial training alone.The corresponding comparisons use MNIST mini-batches of 500 inputs.

G Jacobian regularization - influence of the hyper-parameter λ

The regularization weight λ controls a trade-off between robustness and test accuracy, with dataset-dependent behavior. The method is strongest against attacks aligned with its ℓ2-based objective and weaker against JSMA on MNIST.

  • Influence of λ: For MNIST, robustness increased with λ up to λ = 0.1, while test accuracy slightly degraded; beyond λ = 0.1, both declined substantially.
  • Influence of λ: For CIFAR-10, Jacobian regularization slightly improved test accuracy, but excessively large λ values deteriorated it.
  • Comparative results: Jacobian regularization outperformed the considered defenses under DeepFool and FGSM, but underperformed Input Gradient regularization for MNIST JSMA.On CIFAR-10, it performed better than Input Gradient regularization for JSMA.
  • JSMA attack: The defense targets larger minimal ℓ2 distances, whereas JSMA minimizes the number of changed pixels under an ℓ0 objective.These objectives are not necessarily aligned because ℓ2 penalizes perturbation magnitude more than the number of changed pixels.
  • JSMA attack: An ℓ2-based defense is more effective against DeepFool or FGSM than against JSMA.DeepFool and FGSM change many pixels slightly, while JSMA changes few pixels more substantially.
Loading 1803.08680v4…