PaperlayerView exact revision on arXiv

Source-linked AI summary

Black-Box Adversarial Attack with Transferable Model-based Embedding

Zhichao Huang, Tong Zhang

arXiv:1911.07140v2cs.LGstat.ML

TL;DR

Black-box attacks face a trade-off: transfer-based methods can struggle to succeed, especially on targeted attacks, while score-based methods require many queries. TREMBA learns a low-dimensional adversarial embedding from a pretrained source model and searches it for target attacks, improving query efficiency and performance across the reported tasks.

  • Problem

    Transfer-based black-box attacks can have limited success, especially when targeted, while score-based attacks require many queries to search an unknown target network.

    Source: §1 · p. 1

  • Method

    TREMBA trains an encoder-decoder to generate adversarial perturbations for a pretrained source network, then uses NES to search its low-dimensional embedding against the target.

    Source: §1 · pp. 1–2

  • Results

    TREMBA achieves two to six times improvements in black-box attacks on MNIST and ImageNet, and on defended models raises success rate by approximately 10% while reducing queries by more than 50%.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Its high-level semantic perturbations are effective across different networks, with particularly lower query counts for targeted attacks where transferability is low.

    Source: §1 · p. 2

    For targeted attacks, TREMBA requires different generators for different classes; a single generator for multiple targeted classes is left for future work.

    Source: §5 · p. 9

Abstract

from arXiv · show

We present a new method for black-box adversarial attack. Unlike previous methods that combined transfer-based and scored-based methods by using the gradient or initialization of a surrogate white-box model, this new method tries to learn a low-dimensional embedding using a pretrained model, and then performs efficient search within the embedding space to attack an unknown target network. The method produces adversarial perturbations with high level semantic patterns that are easily transferable. We show that this approach can greatly improve the query efficiency of black-box adversarial attack across different target network architectures. We evaluate our approach on MNIST, ImageNet and Google Cloud Vision API, resulting in a significant reduction on the number of queries. We also attack adversarially defended networks on CIFAR10 and ImageNet, where our method not only reduces the number of queries, but also improves the attack success rate.

1 INTRODUCTION

TREMBA addresses the trade-off between transfer-based attacks’ limited success, especially on targeted attacks, and score-based attacks’ query cost. It learns a low-dimensional, transferable adversarial embedding from a pretrained source network and searches that space, reducing queries across attack settings.

1 INTRODUCTION

  • Transfer-based attacks can have low success, particularly when targeted, while score-based attacks approach white-box success but require many queries.This motivates combining a pretrained source model with more efficient search against an unknown target.Source: §1 · p. 1
  • TREMBA trains an encoder-decoder to generate source-network adversarial perturbations, then searches the generator’s low-dimensional embedding with NES on the target.The embedding is intended to capture global, high-level semantic adversarial features that transfer across models.Source: §1 · pp. 1–2
  • Unlike generator approaches based on dynamic distillation or AutoZOOM’s reconstruction-only embedding, TREMBA trains its embedding to generate adversarial perturbations for a pretrained source network.The paper argues that this produces semantic perturbations better suited to arbitrary target networks and reduces queries.Source: §1 · p. 2
  • On MNIST and ImageNet, TREMBA yields much lower query counts, especially for targeted attacks with low transferability.Its perturbations use high-level semantic patterns that are effective across different networks.Source: §1 · p. 2
  • Against other black-box attacks on SOTA defended models, TREMBA raises success rate by approximately 10% while reducing queries by more than 50%.The introduction names Madry et al. (2018) and Xie et al. (2018) as the defended models.Source: §1 · p. 2

2 RELATED WORKS

Related work spans white-box attacks, black-box attacks that vary by the information available to an attacker, and defenses including adversarial training and feature denoising.

White-Box Attack

  • White-box attacks assume full target-model knowledge and include optimization-based, gradient-based, and generator-based approaches (Szegedy et al., 2014; Goodfellow et al., 2015).The passage also cites later methods improving success rate and efficiency.Source: §2 · p. 2

Black-Box Attack

  • Black-box attacks include transfer-based methods, which generate perturbations on a source model, and score-based methods, which use queried scores to approximate gradients (Papernot et al., 2016a; Chen et al., 2017).Targeted transfer is harder than untargeted transfer, while AutoZOOM reduces the sampling space (Liu et al., 2016; Tu et al., 2018).Source: §2 · p. 2
  • Decision-based attacks use only classifier output labels; boundary attacks are powerful in this setting, and some methods combine transfer or regression-based initialization with further search (Brendel et al., 2017; Brunner et al., 2018).The passage also notes gradient information from a surrogate model as a way to accelerate score-based search (Cheng et al., 2019).Source: §2 · p. 3

Defense Methods

  • Defenses include gradient masking, adversarial training formulated as a minimax game, and feature denoising, which can improve robustness on ImageNet (Athalye et al., 2018; Madry et al., 2018; Xie et al., 2018).The passage notes that gradient-masking methods can be vulnerable to attacks using approximated gradients.Source: §2 · p. 3

3 BLACK-BOX ADVERSARIAL ATTACK WITH GENERATOR

The section formulates bounded targeted and untargeted attacks, then uses a source-trained generator to search for adversarial perturbations for an unknown target in a low-dimensional latent space.

3.1 GENERATING ADVERSARIAL PERTURBATIONS WITH GENERATOR

  • The generator encodes x as a lower-dimensional z and decodes z into a bounded perturbation, training the generator to fool the source network Fs.The encoder output has dim(z) ≪ dim(x), and the decoder uses δ = ε tanh(D(z)).Source: §3.1 · p. 3
  • Training minimizes the C&W hinge loss for untargeted attacks; targeted loss uses margin κ to adjust transferability, with ℓ∞-bounded perturbations.The tanh transformation bounds δ without an explicit infinity-norm constraint; the authors note cross-entropy could also work.Source: §3.1 · pp. 3–4

3.2 SEARCH OVER LATENT SPACE WITH NES

  • TREMBA initializes latent-space search at z0 = E(x) and uses NES-estimated gradients with stochastic gradient descent against an output-query-only target.The search operates on z rather than directly on input-space perturbations.Source: §3.2 · p. 4
  • Source-trained perturbations provide high-level semantic patterns, making latent-space search over likely adversarial patterns less concentrated and easier than searching input space.The method's motivation is that these patterns are likely to be adversarial for the target as well.Source: §3 · p. 3, §3.2 · p. 4

4 EXPERIMENTS

Experiments evaluate TREMBA on MNIST, ImageNet, defended CIFAR10 and ImageNet models, and Google Cloud Vision API. The reported results emphasize reduced query counts, with higher success rates on defended models and semantic perturbations for targeted ImageNet attacks.

4 EXPERIMENTS

  • The experiments compare TREMBA with NES, Trans-NES, AutoZOOM, P-RGF, and Trans-P-RGF, measuring success rate and query use across the evaluated settings.Success-rate curves show behavior at different query levels; the paper also reports average queries and success rates.Source: §4 · pp. 4–5

4.1 BLACK-BOX ATTACK ON MNIST

  • On untargeted MNIST attacks, TREMBA reduces queries by about 50% compared with other attacks, while its success rates are close to 100% but slightly below Trans-NES on ConvNet1 and FCNet.Figure 1 shows TREMBA achieving higher success rates at nearly all query levels.Source: §4.1 · p. 5, Table 1 · p. 6, Figure 1 · p. 6

4.2 BLACK-BOX ATTACK ON IMAGENET

  • For targeted ImageNet attacks on tench, TREMBA averages about 1,000 queries, while nearly all other methods average more than 6,000.The comparison uses the same source-model ensemble across methods; Table 2 reports success rates and average queries.Source: §4.2 · p. 6, Table 2 · p. 7, Figure 2 · p. 7
  • TREMBA’s targeted ImageNet perturbations contain semantic patterns such as fish scales, unlike the noise-like perturbations produced by PGD.The authors suggest these patterns transfer more readily across models and support efficient target-network search.Source: §4.2 · p. 6, Figure 3 · p. 7
  • TREMBA outperforms other methods across tested source-model ensembles, and larger ensembles improve transferability for TREMBA, Trans-NES, and Trans-P-RGF.The paper also reports that TREMBA remains ahead when ε changes to 0.02 or 0.04 despite its generator being trained at ε = 0.03125, and that its gains are not solely due to lower embedding dimension.Source: §4.2 · p. 6

4.3 BLACK-BOX ATTACK ON DEFENDED MODELS

  • On defended CIFAR10 and ImageNet, TREMBA improves success rate by about 10%, cuts average queries by 80% on CIFAR10 and more than 50% on ImageNet, and outperforms the compared attacks.Table 3 reports the success-rate and average-query comparison; Figure 4 shows success rates at different query levels.Source: §4.3 · p. 7, Table 3 · p. 8, Figure 4 · p. 8
  • Optimizing TREMBA’s starting embedding on a defended source network produces higher success rates at small query levels than the original starting point.The variant is called TREMBAOSP; the authors motivate it by the generator’s limited capability on defended networks.Source: §4.3 · p. 8, Figure 4 · p. 8

4.4 ATTACK GOOGLE CLOUD VISION API

  • On 10 Google Cloud Vision API images with a 500-query limit, TREMBA achieves higher success rate and fewer queries than the compared methods.The attack is untargeted, uses ε = 0.05, and changes the top-1 label to any label that was not previously top-1.Source: §4.4 · p. 8, Table 4 · p. 8

5 CONCLUSION

TREMBA searches a generator’s low-dimensional embedding space to create adversarial patterns for unknown networks. It improves black-box attack efficiency, but targeted attacks require a different generator for each class.

5 CONCLUSION

  • TREMBA trains an encoder-decoder to generate source-network perturbations, then searches its low-dimensional embedding space against an unknown target.The paper reports two-to-six-fold improvements in black-box attacks on MNIST and ImageNet, especially for targeted attacks.Source: §5 · p. 9
  • On defended networks, TREMBA achieves nearly a 10% improvement in attack success rate while reducing the number of queries by two to six times.Source: §5 · p. 9
  • Targeted attacks require different generators for different classes; a single conditional generator is left for future work.Source: §5 · p. 9

A EXPERIMENT RESULT

On ImageNet, TREMBA reduces query counts in untargeted attacks and achieves higher targeted success rates than comparison methods. Its targeted perturbations contain recognizable features of the target class.

A.1 TARGETED ATTACK ON IMAGENET

  • TREMBA achieves much higher targeted attack success rates than other methods at almost all query levels on five ImageNet classes.The reported classes are dipper, American chameleon, night snake, ruffed grouse, and black swan.Source: §A.1 · p. 12

A.2 UN-TARGETED ATTACK ON IMAGENET

  • More than half fewer queries than Trans-P-RGF are needed on ResNet34, DenseNet121, and MobileNetV2 in untargeted ImageNet attacks.Embedding-space search remains effective when the target architecture differs substantially from the source-model architectures.Source: §A.2 · p. 12, Table 5 · p. 12, Figure 5 · p. 12

A.3 VISUALIZATION OF TARGETED PERTURBATION

  • Targeted perturbations show recognizable target-class features, such as bird feathers and snake bodies.The examples in Figure 10 are amplified ten times.Source: §A.3 · p. 12

A.4 EXPERIMENTS ON DIFFERENT ENSEMBLES

The appendix tests targeted ImageNet attacks with alternative source-model ensembles and perturbation strengths. TREMBA outperforms other methods at ε values different from its training strength.

A.4 EXPERIMENTS ON DIFFERENT ENSEMBLES

  • The ensemble evaluation compares targeted ImageNet success rates across query levels for two additional source ensembles.The ensembles contain VGG16 with Squeezenet, and VGG16 with Squeezenet and Googlenet; comparisons are limited to Trans-NESPGD and Trans-P-RGF.Source: §A.4 · p. 12, Figure 6 · p. 13

A.5 VARYING ε

  • TREMBA outperforms other methods in targeted ImageNet attacks at ε = 0.02 and ε = 0.04, despite training at ε = 0.03125.The authors interpret this as generalization to different adversarial-attack strengths.Source: §A.5 · p. 13
  • TREMBA also performs well in targeted ImageNet attacks at ε = 0.05, with results reported in Tables 6–7 and Figure 8.Source: §A.5 · p. 13

A.6 VARYING SAMPLE SIZE

The appendix finds that TREMBA’s performance is not very sensitive to sample size and that simply reducing embedding dimension does not explain its gains.

A.6 VARYING SAMPLE SIZE

  • TREMBA outperforms other methods across the sample-size sweep, and performance is generally not very sensitive to b.The sweep varies b on DenseNet121 for untargeted ImageNet attacks; b = 20 is not best for TREMBA or Trans-NES.Source: §A.6 · p. 13

A.7 DIMENSION OF THE EMBEDDING SPACE

  • An autoencoder with an 8 × 8 × 8 embedding performed worse than the original, and dimension-matching the Trans-P-RGF prior did not improve its performance.The authors conclude that simply shrinking the embedding space does not account for TREMBA’s performance gains.Source: §A.7 · pp. 13–14

A.8 EXAMPLES OF ATTACKING GOOGLE CLOUD VISION API

The appendix presents ImageNet query-level and perturbation examples, a Google Cloud Vision API example, and a comparison with CombOpt. TREMBA is reported to require much lower queries than CombOpt on targeted and untargeted ImageNet attacks.

A.8 EXAMPLES OF ATTACKING GOOGLE CLOUD VISION API

  • On Google Cloud Vision API, TREMBA makes a shark classify as green, while its image labels are described as definitely incorrect.The passage says it is hard to determine whether Trans-NESPGD's overall classification is wrong.Source: §A.8 · pp. 14–15, Figure 11 · p. 17
  • Figure 8 plots targeted and untargeted ImageNet attack success rates across query levels at ε = 0.05.Figure 9 covers targeted attacks on Dipper, American chameleon, Night snake, Ruffed grouse, and Black swan.Source: Figure 8 · p. 15, Figure 9 · p. 16
  • The appendix also includes a b hyperparameter sweep on Densenet121 and an AutoZOOM embedding-dimension comparison, both for untargeted ImageNet attacks.The captions identify these experiment scopes but do not provide outcomes.Source: Table 8 · p. 15, Table 9 · p. 15
  • Figure 10 places a target-class example in the first column and adversarial perturbations in the other columns.Figure 11 shows one adversarial-image example for the Google Cloud Vision API.Source: Figure 10 · p. 17, Figure 11 · p. 17

A.9 COMPARISION BETWEEN TREMBA AND COMBOPT

B ARCHITECTURE OF CLASSIFIERS AND GENERATORS

The appendix lists classifier architectures and generator architectures for the evaluated datasets. It specifies a ResNeXt configuration for CIFAR10 and notes that AutoZOOM uses the generators from its corresponding paper.

B.1 CLASSIFIER

  • For CIFAR10, the specified ResNeXt has depth 20, cardinality 8, and widen factor 4.Other classifier architectures are specified in their corresponding papers; Table 12 lists MNIST model architectures.Source: §B.1 · p. 18, Table 12 · p. 18

B.2 GENERATOR

  • Generator architectures are listed for three datasets, but AutoZOOM uses the generators from its corresponding paper because the authors found their own architectures unsuitable.Table 13 lists encoder and decoder architectures and defines the convolution, deconvolution, and max-pooling blocks.Source: §B.2 · p. 18

C.1 TRAINING GENERATOR

The authors report generator-training settings and provide algorithm-specific evaluation hyperparameters. They use a shared sample size of 20 for fair comparisons.

C.1 TRAINING GENERATOR

  • Generator training ran for 500 epochs, with learning rate starting at 0.01 and halving every 50 epochs; κ was set to 200.0.Batch size was 256 for MNIST and CIFAR10 defense models and 64 for ImageNet; experiments used PyTorch on an NVIDIA RTX 2080Ti.Source: §C.1 · p. 18
  • Table 13 specifies encoder-decoder blocks using convolution or deconvolution followed by ReLU and batch normalization, plus max pooling.The passage defines the associated channels and kernel or pooling sizes.Source: Table 13 · p. 19

C.2 EVALUATION

  • Evaluation hyperparameters are listed for NES, transfer NES, AutoZOOM, P-RGF, TREMBA, and TREMBAOSP; all algorithms use sample size b = 20.The learning rate was fine-tuned for all algorithms.Source: §C.2 · p. 19, Table 14 · p. 19, Table 15 · p. 19
Loading 1911.07140v2…