Backpropagating Linearly Improves Transferability of Adversarial Examples

1 Introduction

• Transferability lets adversarial examples crafted on one DNN fool other models, including models with different architectures.The paper identifies it as central to transfer-based black-box attacks and useful for enhancing query-based attacks.Source: §1 · p. 1
• LinBP calculates the forward pass normally but backpropagates the loss as if some nonlinear activations were absent.The authors report that the method improves transferability across several gradient-based attacks and victim models on CIFAR-10 and ImageNet.Source: §1 · p. 1
• LinBP outperforms prior state-of-the-art methods in crafting transferable adversarial examples on CIFAR-10 and ImageNet.The paper evaluates source models including VGG-19, ResNet-50, and Inception v3.Source: §1 · p. 1

2 Background and Related Work

• Query-based black-box attacks estimate gradients iteratively and usually achieve higher success rates, but often incur high query complexity.Transfer-based attacks instead craft examples on source models and use them against victim models.Source: §2 · p. 2
• Transferability is studied as a basis for black-box attacks; prior approaches include ensembles, intermediate-feature attacks, and diverse-input I-FGSM.The paper also distinguishes LinBP from SGM, which is designed for networks with skip connections, and says the methods can be combined.Source: §2 · p. 2

3.1 A More Linear Source Model

• Linear substitution removes selected nonlinear activations, producing a global approximation that is the same for different network inputs.The approach retains the original model’s parameter count and core architecture.Source: §3.1 · p. 3
• For m ≥ 1, fine-tuned linear-substitution models produce more transferable examples than the original model; I-FGSM transfer peaks near epoch 80, then declines with further training.The paper attributes the later decline to overfitting and reports that converged models have transferability comparable to models trained from scratch.Source: §3.1 · p. 3
• When trained from scratch, the more linear model does not make attacks on the VGG-19 source easier than attacks on other victim models.Fine-tuning, by contrast, gives the linearized model considerable ability to resemble the original source model.Source: §3.1 · p. 4

4 Our Linear Backpropagation Method

• Removing ReLUs can improve transferability, but removing more does not always help because the resulting accuracy loss can make source-model input gradients unlike those of victim models.The method therefore seeks a trade-off between linearity and accuracy.Source: §4 · p. 4

4.1 Linear Backpropagation

• LinBP computes the forward pass normally but backpropagates as if ReLUs were absent, directing more gradient through linear paths without fine-tuning the source model.The gradient flow is changed while predictions remain those of the well-trained source model.Source: §4.1 · p. 4
• LinBP compares favorably with LinS variants in transferability and offers a more reasonable linearity–accuracy trade-off; the compared attacks include single- and multi-step settings at ϵ = 0.03.The reported methods also show higher computational efficiency than the baseline due to the (partially) absent ReLUs.Source: §4.1 · p. 4, Table 1 · p. 5

4.2 Gradient Branching with Care

• Results at the optimal fine-tuning epochs are impractical to obtain and are presented only for reference.Source: §4.2 · p. 5
• For residual units, LinBP removes the ReLU mask from the main-stream backward derivative and applies a layer-specific gradient rescaling; when all inputs to a ReLU are nonnegative, its backward pass matches standard backpropagation.The re-normalization is intended to address reduced gradient flow through skip connections; combining LinBP with SGM is reported to further improve performance.Source: §4.2 · p. 5

5.1 Experimental Settings

• The evaluation uses untargeted ℓ∞ attacks on CIFAR-10 and ImageNet, with VGG-19 and ResNet-50 as the respective source models.The tested ϵ values are 0.1, 0.05, and 0.03; 5,000 correctly classified test instances per dataset are used, with 100 I-FGSM iterations on CIFAR-10 and 300 on ImageNet.Source: §5.1 · pp. 5–6

5.2 Comparison to State-of-the-arts

• On CIFAR-10 at ϵ = 0.05, LinBP achieves nearly 73% success against PyramidNet†, while ImageNet success rates are 61.02% on Inception v3 and 62.66% on PNASNet.The ImageNet rates are reported for untargeted ℓ∞ attacks transferred from ResNet-50.Source: §5.2 · p. 7
• Against adversarially trained ImageNet models, LinBP exceeds ILA at each tested ϵ on both ensemble-trained Inception and PGD-trained ResNet.For ensemble-trained Inception, LinBP success rates are 76.06%, 31.32%, and 14.32% at ϵ = 0.1, 0.05, and 0.03; for PGD-trained ResNet, victim error rates are 48.60%, 39.92%, and 37.10% at those ϵ values.Source: §5.2 · p. 7
• LinBP achieves similar or better results than ILA at preferred positions on the source models, with performance varying by network position.Figure 4 compares position choices on CIFAR-10-trained VGG-19 and ImageNet-trained ResNet.Source: Figure 4 · p. 7, §5.2 · p. 8

5.3 Combination with Existing Methods

• Combining LinBP with ILA or SGM further improves transferability in the evaluated settings, and LinBP with ILA uses LinBP results as directional guides.The SGM combination introduces a scaling factor in residual units and is evaluated with λ = 0.3 and 0.5; Table 4 reports combined-method success rates on ImageNet.Source: §5.3 · p. 8, Table 4 · p. 8
• LinBP also yields the reported conclusions when paired with DI2-FGSM, PGD, and an ensemble attack, beyond the I-FGSM and FGSM back ends used for the main comparisons.The additional results are reported in the supplementary material.Source: §5 · p. 5, §5.3 · p. 8

• LinBP computes the forward pass normally but backpropagates loss as if ReLU activations were absent.The conclusion describes the method as improving transferability of adversarial examples.Source: §6 · p. 8
• LinBP enhances transferability for FGSM, I-FGSM, DI2-FGSM, and PGD examples across victim models on CIFAR-10 and ImageNet.Source: §6 · p. 8
• Combined with some compared methods, LinBP obtains nearly 85% average success in attacking ImageNet models under ϵ = 0.05.Source: §6 · p. 8

• Comparing model reliability under practical adversarial attacks could help machine-learning-as-a-service platforms choose models for security-critical applications.The passage frames this as a potential contribution.Source: Broader Impact · p. 9

• The cited version is arXiv:2012.03528v1, posted 7 Dec 2020.Source: of Adversarial Examples (Supplementary… · p. 12

A Fine-tuning f ′ with Frozen h

• With h frozen, transferability initially improves, but later declines until success rates are below baseline, attributed to lower learning capacity.The reported curves cover FGSM and I-FGSM under untargeted ℓ∞ attacks with ϵ = 0.03.Source: A Fine-tuning f ′ with Frozen h · p. 12, Figure 5 · p. 12
• Figure 6 reports test classification loss for the modified VGG-19 models on benign CIFAR-10 test data.The passage reports that the loss is shown, but gives no values or comparison.Source: A Fine-tuning f ′ with Frozen h · p. 12, Figure 6 · p. 12
• Figure 5 tracks FGSM and I-FGSM transferability while fine-tuning the purely linear blocks of f ′ with h frozen.The attacks are untargeted, with ℓ∞ constraint and ϵ = 0.03.Source: Figure 5 · p. 12

B Benign-test-set Performance of The Victim Models

• Table 5 reports benign-test-set accuracy of the models; for ImageNet, that accuracy is evaluated using the 50 000 official validation images.Source: Table 5 · p. 13

C Re-interpret Intermediate Feature-based Attacks?

• ILA maximizes the projection of the feature disturbance onto a guidance direction derived from a prior adversarial perturbation.Its objective is J(x + r, y) = v^T(h(x + r) − h(x)), subject to a perturbation constraint.Source: C Re-interpret Intermediate… · p. 13
• The paper describes ILA’s objective as more linear than the prediction-loss objective because it replaces s ◦ g with identity and uses a linear loss instead of l.The passage says whether this higher linearity partly explains practical effectiveness remains for future study.Source: C Re-interpret Intermediate… · p. 13

D Single-step Back-end Attacks

• 90.49%, 74.44%, and 52.95% are LinBP’s average success rates, above ILA’s 85.72%, 70.94%, and 50.68% in the CIFAR-10 FGSM comparison.LinBP uses a two-step policy; ILA is applied after FGSM examples are generated and also takes two steps total.Source: Table 6 · p. 13, D Single-step Back-end Attacks · p. 13
• The FGSM transfer-attack tables cover CIFAR-10 with a VGG-19 source and ImageNet with a ResNet-50 source.Both captions specify ℓ∞ constraints; ILA uses an additional optimization step after the initial FGSM examples.Source: Table 6 · p. 13, Table 7 · p. 14
• With ensemble adversarial attack on ImageNet using ResNet-50 and Inception v3 sources, average success rates are 98.17%, 78.94%, and 49.42% at ϵ = 0.1, 0.05, and 0.03, respectively.The passage also reports that LinBP works better with DI2-FGSM and PGD than with I-FGSM.Source: D Single-step Back-end Attacks · p. 14, Table 8 · p. 14, Table 9 · p. 14

F Other ϵ Settings and More Source Models

• The evaluation adds ϵ = 16/255, 8/255, and 4/255 settings and includes ResNet-18 on CIFAR-10 and Inception v3 on ImageNet as source architectures.PGD is used as the baseline, with other methods applied on the basis of PGD.Source: F Other ϵ Settings and More Source… · pp. 14–15
• The reported comparisons include both untargeted and targeted settings, and the passage says results with other back-end attacks are similar.Source: F Other ϵ Settings and More Source… · p. 15
• LinBP’s superiority holds in most test cases across the reported CIFAR-10 and ImageNet PGD comparisons with VGG-19, ResNet-18, ResNet-50, and Inception v3 sources.Tables 10–13 report success rates of transfer-based attacks under ℓ∞ constraints.Source: Table 10 · p. 15, Table 11 · p. 15, Table 12 · p. 15