Source-linked AI summary
Art of singular vectors and universal adversarial perturbations
Valentin Khrulkov, Ivan Oseledets
TL;DR
Small adversarial perturbations can cause DNN misclassification, and earlier work found image-agnostic universal perturbations that fool natural images. This paper constructs such perturbations using singular vectors of hidden-layer Jacobians. With 64 images, its method produced a perturbation fooling the network in roughly 60% of cases, compared with several thousand images required by the previous known approach for similar rates.
Problem
Many state-of-the-art DNNs are vulnerable to small adversarial perturbations that can lead to image misclassification.
Source: §1 · p. 1Method
The algorithm constructs universal perturbations by computing (p, q)-singular vectors of feature-map Jacobians with a generalized power method.
Source: §1 · pp. 1–2Results
64 images produced a perturbation fooling the network in roughly 60% of cases, while the previous known approach required several thousand images for such fooling rates.
Source: §8 · p. 9Takeaways & Limitations
The resulting perturbations exhibit visual properties, and the reported fooling rate was achieved with far fewer construction images than the previous known approach required.
Source: §8 · p. 9
Abstract
from arXiv · showhide
Vulnerability of Deep Neural Networks (DNNs) to adversarial attacks has been attracting a lot of attention in recent studies. It has been shown that for many state of the art DNNs performing image classification there exist universal adversarial perturbations --- image-agnostic perturbations mere addition of which to natural images with high probability leads to their misclassification. In this work we propose a new algorithm for constructing such universal perturbations. Our approach is based on computing the so-called $(p, q)$-singular vectors of the Jacobian matrices of hidden layers of a network. Resulting perturbations present interesting visual patterns, and by using only 64 images we were able to construct universal perturbations with more than 60 \% fooling rate on the dataset consisting of 50000 images. We also investigate a correlation between the maximal singular value of the Jacobian matrix and the fooling rate of the corresponding singular vector, and show that the constructed perturbations generalize across networks.
1. Introduction
Small adversarial changes can expose vulnerabilities in state-of-the-art DNNs, and prior work found that image-agnostic perturbations can misclassify many natural images. This paper introduces a Jacobian-based algorithm for constructing universal perturbations and examines their data requirements, robustness relevance, and properties.
1. Introduction
- Many state-of-the-art DNNs are vulnerable to adversarial attacks that add a small-magnitude image perturbation designed to misclassify the perturbed image, potentially forcing a specific class [6] [14] [21].Potential undesirable use in practical applications such as autonomous driving and malware detection has been studied [10] [8].Source: §1 · p. 1
- The algorithm uses a generalized power method to compute (p, q)-singular vectors of feature-map Jacobians, approximating network perturbation effects with Jacobians for small adversarial changes.The construction is motivated by the typical small norm of adversarial perturbations.Source: §1 · pp. 1–2
- The method produces universal adversarial examples from a relatively small number of dataset images.The conclusion reports a construction using 64 images, but that result is outside this request's supplied passages.Source: §1 · p. 2
- The authors investigate whether the largest (p, q)-singular value correlates with fooling rate and could serve as a quantitative robustness measure or DNN regularizer.These uses are presented as suggestions based on the investigated correlation.Source: §1 · p. 2
- The paper analyzes how fooling rate depends on construction-image count and whether perturbations generalize across networks.Prior work had reported that universal perturbations transfer relatively well across DNNs [14].Source: §1 · pp. 1–2
2. Problem statement
The section defines universal perturbations as bounded input changes that alter predictions across as many dataset images as possible. It motivates maximizing the local change in a hidden-layer output through its Jacobian.
2. Problem statement
- A universal perturbation ε seeks to change predictions for as many images as possible while satisfying a norm bound.Its fooling rate is the fraction of dataset images whose predicted label changes after adding ε.Source: §2 · p. 2
- For small input changes, the hidden-layer output change is approximated using the layer Jacobian, motivating maximization of its q-norm response.The text suggests this may substantially change the predicted label farther through the network.Source: §2 · p. 2
- The resulting individual-image objective is to maximize the q-norm of the Jacobian response subject to an input perturbation norm constraint.The singular vector solution is defined up to multiplication by −1.Source: §2 · p. 2
- Exact solutions for some norm pairs require examining every matrix entry, which is impractical for the very large Jacobians in this setting.The example matrix would require roughly 30 TB of memory, so the method instead uses iterative matrix-vector products without forming the full matrix.Source: §2 · p. 2
3. Generalized power method
The generalized power method computes singular vectors using matrix-vector operations. The section extends this objective across images to seek a perturbation that affects a batch uniformly.
3. Generalized power method
- For p = q = 2, the generalized power method reduces to the familiar power method on A⊤A.The method uses matrix-vector functions of A and A⊤ rather than requiring the full matrix.Source: §3 · pp. 2–3
- The universal objective stacks Jacobians for images in a batch and seeks a perturbation that uniformly changes the hidden-layer output across those images.The section presents this batch objective as a tractable approximation to the full-dataset objective.Source: §3 · pp. 2–3
- The proposed approximate solution is hypothesized to be close to the exact solution of the universal objective.The text introduces this as a hypothesis, not an established result.Source: §3 · p. 3
4. Stochastic power method
The stochastic power method applies the generalized power method to Jacobians stacked for a randomly chosen image batch. Experiments report little change in fooling rate when the batch varies between iterations.
4. Stochastic power method
- The method forms a batch matrix by vertically stacking the hidden-layer Jacobians of the batch images, then applies the power method to its matrix-vector functions.The batch and hidden layer are specified as inputs to the procedure.Source: §4 · p. 3
- Changing the batch between iterations almost does not affect the fooling rate of the generated universal perturbation.The authors report this observation from their experiments with the stochastic power method.Source: §4 · p. 3
5. Efficient implementation of the matvec functions
The implementation avoids explicitly forming large Jacobians by computing their matrix-vector products through automatic differentiation. The resulting products support the batch-based perturbation procedure.
5. Efficient implementation of the matvec functions
- Automatic differentiation and Pearlmutter’s R-operator provide Jacobian matrix-vector products without explicitly forming the large matrices.The stated setup uses gradients of scalar functions of the hidden-layer output.Source: §5 · p. 3
- For a given image batch, the algorithm constructs the Jacobian matrix-vector functions once before using them in the perturbation procedure.The batch-level routine constructs these functions for each image and the stacked batch matrix.Source: §5 · pp. 3–4
6. Experiments
Experiments on ILSVRC 2012 show that Jacobian singular-vector perturbations can fool many images using a small construction set. Their fooling rates depend on the choice of q and network layer, and the method stabilizes faster than the compared UAP algorithm.
6. Experiments
- Experiments used ILSVRC 2012's 50000-image validation set and computed p = ∞, q = 10 singular vectors for VGG-16, VGG-19, and ResNet50 using the same 64 images.The authors chose q = 10 to smooth the optimization and serve as a replacement for q = ∞.Source: §6 · p. 4, §6.1 · p. 4
6.1. Adversarial perturbations
- The computed singular vectors showed visual patterns resembling feature visualizations, while the singular values of ResNet50 layers were generally smaller than those of the VGG networks.The authors suggest, cautiously, that perturbations may disrupt image understanding at particular levels; they also report O(dn) complexity for d iterations versus O(n2) for the exact algorithm, with d as small as 60.Source: §6.1 · pp. 4–5, Figure 2 · p. 5, Figure 3 · p. 5
6.2. Fooling rate and singular values
- Using 64 images achieved more than 40% fooling rate for all investigated networks on 50000 images, with perturbation ∞-norm set to 10.The construction used less than 0.15% of the dataset and generalized to unseen classes and images.Source: §6.2 · p. 5
- With the VGG-19 block2 pool layer, 16 images yielded 56% fooling rate, and increasing batch size did not significantly affect the rate.Higher singular values did not necessarily indicate higher fooling rates, although network-level singular-value magnitudes roughly corresponded to maximal fooling rates.Source: Figure 2 · p. 5, §6.2 · p. 6, Figure 5 · p. 6
- Perturbations generalized across networks, and in some cases a perturbation computed on another network had a higher fooling rate than the tested network's own perturbation.The paper identifies the VGG-19 block2 pool perturbation as the most efficient and notes its edge-like structure.Source: §6.2 · p. 6, Table 4 · p. 7
6.3. Dependence of the perturbation on q
- Larger q values produced clearer edge-like patterns and higher fooling rates, with the maximum appearing near q ≈ 5.The authors suggest q ≈ 5 may be a smoother substitute for q = ∞ than q = 10 in large-scale problems.Source: §6.3 · pp. 6–7, Figure 7 · p. 7
6.4. Comparison of the algorithms
- Roughly 63% fooling rate was reached by SingularFool as the image batch grew, while UAP began low and increased gradually; at batch size 32, the method reached roughly 60% in about 1 minute versus UAP's roughly 20% in about 10 minutes.The paper cites about 3000 images as needed for UAP to reach fooling rates of order 60%.Source: §6.4 · pp. 7–8, Figure 9 · p. 8
7. Related work
The related work contrasts individual-image attacks with universal perturbations and positions the paper's optimization as a simpler alternative to the latter.
7. Related work
- Goodfellow et al.'s fast gradient sign method is a special case of the paper's formulation when the batch contains one image and the loss is the hidden layer.Under those conditions, sign grad_x l(x) solves problem (5) with p = 1 and L = 1.Source: §7 · p. 8
- Moosavi et al.'s UAP method iteratively solves nonlinear optimization problems and projects onto a norm sphere to find a direction toward the decision boundary.The passage describes this as computing the “worst” possible direction.Source: §7 · p. 8
- The paper's optimization problems are simpler and homogeneous, so a perturbation for an arbitrary norm is obtained by rescaling one computed solution, unlike the cited UAP algorithm.Source: §7 · p. 8
8. Conclusion
The authors conclude that their algorithm produces universal perturbations from few images and examine their generalization, hyperparameter dependence, and visual properties. They identify feature visualization and a defense approach as directions for future work.
8. Conclusion
- The work examined generalization across networks, how fooling rate depends on hyperparameters, and the perturbations’ visual properties.Source: §8 · pp. 8–9
- 64 images produced a perturbation that fooled the network in roughly 60% of cases, while the previous approach required several thousand images for similar fooling rates.Source: §8 · p. 9
- Future work will examine the relation between feature visualization and adversarial perturbations, and analyze the defense approach discussed in section 6.2.Source: §8 · p. 9