PaperlayerView exact revision on arXiv

Source-linked AI summary

PuriDefense: Randomized Local Implicit Adversarial Purification for Defending Black-box Query-based Attacks

Ping Guo, Xiang Li, Zhiyuan Yang, Xi Lin, Qingchuan Zhao, Qingfu Zhang

arXiv:2401.10586v2cs.CRcs.AIcs.CVcs.LG

TL;DR

Black-box query attacks threaten MLaaS systems, while existing defenses can impose substantial costs or reduce accuracy on non-adversarial inputs. PuriDefense uses randomized patch-wise purification with local implicit functions; theory and experiments on CIFAR-10 and ImageNet support its effectiveness against query-based attacks.

  • Problem

    Black-box attacks can generate adversarial examples without access to the target model, while existing defenses may impose substantial costs or reduce accuracy on non-adversarial inputs.

    Source: §1 · p. 1

  • Method

    PuriDefense applies random patch-wise image purification using local implicit functions to defend classifiers against query-based attacks.

    Source: §1 · p. 2

  • Results

    Experiments on CIFAR-10 and ImageNet support PuriDefense’s effectiveness against SOTA query-based attacks, and reported robustness increases with the addition of purifiers.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The paper highlights the need for dynamic and efficient defense mechanisms in MLaaS systems.

    Source: §6 · p. 8

    If an adversarial image is a fixed point of the purifier, the system’s robustness is not improved; the paper says current work cannot theoretically eliminate this issue.

    Source: §4.5 · p. 6

Abstract

from arXiv · show

Black-box query-based attacks constitute significant threats to Machine Learning as a Service (MLaaS) systems since they can generate adversarial examples without accessing the target model's architecture and parameters. Traditional defense mechanisms, such as adversarial training, gradient masking, and input transformations, either impose substantial computational costs or compromise the test accuracy of non-adversarial inputs. To address these challenges, we propose an efficient defense mechanism, PuriDefense, that employs random patch-wise purifications with an ensemble of lightweight purification models at a low level of inference cost. These models leverage the local implicit function and rebuild the natural image manifold. Our theoretical analysis suggests that this approach slows down the convergence of query-based attacks by incorporating randomness into purifications. Extensive experiments on CIFAR-10 and ImageNet validate the effectiveness of our proposed purifier-based defense mechanism, demonstrating significant improvements in robustness against query-based attacks.

1. Introduction

Black-box query attacks threaten MLaaS systems, while existing defenses face trade-offs between inference cost and clean-input accuracy. PuriDefense uses random patch-wise purification with local implicit functions, with theoretical analysis and experiments addressing robustness and attack convergence.

1. Introduction

  • Black-box attacks can achieve high success rates against closed-source platforms using limited queries, despite lacking model access.Such attacks are presented as a realistic threat to DNN robustness.Source: §1 · p. 1
  • MLaaS defenses need low extra inference cost, since large query volumes multiply defense expense and repeated test-time inference can be costly.Existing heuristic transformations can harm non-adversarial accuracy, while neural purification can exceed classifier computational burden.Source: §1 · p. 1
  • PuriDefense randomly assigns a pool of local-implicit-function purifiers to image patches, preserving the inference cost of a single model.A broader purifier spectrum is associated with higher resilience and slower query-attack convergence.Source: §1 · p. 2
  • Theoretical analysis finds deterministic purifications susceptible to query-based attacks and suggests randomness enhances robustness across preprocessor-based defenses.Experiments target state-of-the-art query-based attacks on CIFAR-10 and ImageNet.Source: §1 · p. 2

2. Related Work

Related work covers score-based and decision-based query attacks alongside heuristic and model-based purification defenses. The paper positions its local-implicit-function defense as a lower-cost approach with theoretical guarantees against black-box attacks.

2. Related Work

  • Score-based attacks treat the full MLaaS pipeline as opaque and use confidence-score objectives with gradient estimation or random search.The section identifies Square attack as a leading random-search benchmark.Source: §2 · p. 2
  • Decision-based attacks use label outputs and include surrogate-objective, gradient-sign, and direct-search approaches.HopSkipJump is described as the most advanced decision-based attack.Source: §2 · p. 2
  • Prior purification includes generative, self-supervised, diffusion, and local implicit models, but diffusion complexity reduces inference speed.The cited local implicit approach targets white-box attacks and lacks theoretical guarantees against black-box attacks.Source: §2 · p. 2
  • The paper says its redesigned local-implicit defense accelerates inference fourfold and keeps speed constant as purifier count increases, with theoretical black-box guarantees.Prior ensemble-based randomness in DISCO linearly increases inference cost.Source: §2 · pp. 2–3

3. Preliminaries

The preliminaries define the MLaaS query setting and distinguish attacks by whether confidence scores or only labels are available. They also review adversarial purification approaches and the cost trade-off in adding randomness.

3.1. Threat Model

  • The threat model gives attackers query access to a cloud-hosted model's outputs but no deeper understanding of its internal mechanisms or training datasets.The interaction is submitting queries and receiving confidence scores or classifications.Source: §3.1 · p. 3

3.2.1. SCORE-BASED ATTACKS

  • Score-based attacks use confidence-score outputs to optimize an adversarial example within an ℓp-norm ball, with black-box methods estimating descent directions or using heuristic search.An attack succeeds when its objective falls below zero; targeted attacks specify a designated target label.Source: §3.2.1 · p. 3

3.2.2. DECISION-BASED ATTACKS

  • Decision-based attacks receive only model labels and use optimization methods including surrogate objectives, geometric approaches, and gradient signs.The paper says its score-based theoretical principles also apply because both use similar black-box optimization techniques.Source: §3.2.2 · p. 3

3.3. Adversarial Purification

  • Adversarial purification seeks to remove or disturb perturbations through heuristic transformations or purification models, with local implicit purification noted for moderate inference cost.Heuristics disturb perturbations, while a naive ensemble of purifiers introduces randomness at linearly increased inference cost.Source: §3.3 · pp. 3–4, Table 1 · p. 4

4. Randomized Local Implicit Purification

PuriDefense combines local implicit image reconstruction with random patch-wise assignment of multiple purifiers to counter query-based attacks while keeping inference cost constant as the purifier pool grows. The section describes its training and implementation, then analyzes how purifier diversity affects gradient-based and gradient-free attacks.

4.1. Our Motivation

  • PuriDefense’s robustness is described as scaling with the number of purifiers while inference cost remains constant, making the design a proposed alternative to costly direct ensembles.The analysis associates a broader purifier pool with increased robustness.Source: §4.1 · p. 4

4.2. Image Purification via Local Implicit Function

  • The local implicit purifier encodes image patches into high-dimensional features and decodes them into reconstructed RGB values, processing pixels within a patch in parallel.Removing positional encoding and local ensemble inference yields a fourfold implementation-level inference speedup.Source: §4.2 · p. 4, Figure 2 · p. 5

4.3. Training Purifiers in PuriDefense

  • Purifiers are trained on adversarial samples generated with PGD, FGSM, and BIM, using purified samples and original images to calculate the training loss.The training procedure optimizes all purifiers by backpropagation; Figure 3 identifies the loss as ℓ1.Source: §4.3 · p. 5, Figure 3 · p. 5

4.4. Random Patch-wise Purification.

  • Random patch-wise purification assigns randomly selected purifiers to image patches and merges their outputs, maintaining inference cost equivalent to a single model.Unlike full-image encoding followed by random feature selection, this approach introduces randomness through patch-level purifier selection.Source: §4.4 · p. 5, Figure 4 · p. 6

4.5. Theoretical Analysis for Gradient-based Attacks

  • For a single deterministic purifier, the analysis says attacks may converge faster and the transformed optimum may remain exploitable, so purification alone does not guarantee improved robustness.The discussion also notes that purifier vulnerabilities make robustness difficult to quantify.Source: §4.5 · p. 6
  • With multiple deterministic purifiers, the analysis associates increased purifier diversity with greater robustness, although adversarial optimal points persist.The introduced convergence term increases quadratically with ν, which the authors associate with purifier count.Source: §4.5 · p. 6
  • On CIFAR-10, the Square Attack converges faster with transformations that shrink image space, while the deterministic purifier DISCO fails to improve system robustness.Figure 5 depicts Square Attack convergence under different heuristic transformations and purifiers.Source: Figure 5 · p. 7, §4.5 · p. 7

4.6. Theoretical Analysis for Gradient-free Attacks

  • The gradient-free attack analysis likewise states that robustness increases with purifier diversity ν.The authors present this as a guarantee of defense against gradient-free attacks under their analysis.Source: §4.6 · p. 7

5. Evaluation

The evaluation tests PuriDefense across query-based attacks, inference-speed comparisons, and clean-accuracy settings on CIFAR-10 and ImageNet. It reports strong robustness under extreme query budgets while maintaining low inference cost and comparable clean accuracy.

5.1. Experiment Settings

  • Experiments use balanced CIFAR-10 and ImageNet subsets and five query-based attacks; Figure 6 tracks Square Attack convergence as purifier count varies.The subsets contain 1,000 CIFAR-10 test images and 2,000 ImageNet validation images.Source: §5.1 · p. 7, Figure 6 · p. 7

5.4. Influence on Clean Accuracy

  • PuriDefense yields clean accuracy comparable to standard training and improves clean accuracy when added to adversarially trained models without additional cost.On ImageNet, increasing patch count from 1 × 1 to 3 × 3 or 5 × 5 does not significantly affect clean accuracy.Source: §5.2 · p. 7, §5.4 · p. 8

5.2. Overall Defense Performance

  • Spatial smoothing accelerates attacks, while DISCO’s robust accuracy under a 2,500-query Square Attack falls below that of undefended models.The reported outcomes support the paper’s concern that deterministic transformations may expedite attacks.Source: §5.2 · pp. 7–8
  • PuriDefense attains the highest robust accuracy across varied CIFAR-10 attacks at 2,500 queries; combined with standard training it is comparable to state-of-the-art adversarial training, and with adversarial training it sets new benchmarks.The evaluation also reports effectiveness against both ℓ∞ and ℓ2 attacks.Source: §5.2 · p. 8, Table 2 · p. 8

5.3. Inference Speedup

  • PuriDefense’s inference cost stays at the baseline-classifier level as purifier count grows, whereas DISCO’s cost rises almost linearly.The CIFAR-10 comparison is shown in Figure 7, with ImageNet results reported in Appendix C.1.Source: §5.3 · p. 8, Figure 7 · p. 8

5.4. Influence on Clean Accuracy

  • Purification models have comparable clean accuracy on CIFAR-10 and higher clean accuracy on ImageNet, while patch-count changes do not significantly affect ImageNet clean accuracy.The authors attribute the ImageNet difference to the natural image manifold existing in high-resolution datasets.Source: §5.4 · p. 8

6. Conclusion

The paper concludes that its local-implicit-function purification mechanism improves robustness to query-based attacks, with effectiveness increasing as purifiers are added. The accompanying background and search discussion describes constraints on existing defenses and how black-box attacks approximate descent directions.

6. Conclusion

  • PuriDefense uses local implicit functions for image purification against query-based attacks, and the paper reports robustness increasing with additional purifiers.The conclusion says its effectiveness was tested on CIFAR-10 and ImageNet.Source: §6 · p. 8

A. Background for General Defense

  • Existing defenses face deployment trade-offs: adversarial training is costly and may reduce clean accuracy, randomized smoothing is slow, and confidence poisoning cannot defend against decision-based attacks.The passage also notes limitations of input-noise defenses against strong attacks and for clean accuracy.Source: A. Background for General Defense · p. 12

B. Search Techniques for black-box attacks

  • Black-box attacks approximate descent through gradient estimation or heuristic search; heuristic search uses the objective sign to select the gradient direction or its opposite.The described update substitutes this search direction for the gradient-based update direction.Source: B. Search Techniques for black-box… · p. 12

C.1. Comparison of the Inference Speed on ImageNet Dataset

This appendix compares inference speed on ImageNet and documents attack, defense, and model configurations. It also describes a structural simplification to the local implicit function used to accelerate purification.

C.1. Comparison of the Inference Speed on ImageNet Dataset

  • Figure 8 compares the inference speed of DISCO and PuriDefense on ImageNet.Source: Figure 8 · p. 12

C.2. Details of the Attacks

  • Evaluation covers NES, SimBA, Square, Boundary, and HopSkipJump, with attack implementations and hyperparameters documented.Source: §C.2 · p. 13

C.3. Detailed Information for the Defense

  • Defense baselines include Gaussian input noise, spatial smoothing, and a deterministic DISCO implementation using a pretrained PGD-trained local implicit model.Source: §C.3 · p. 13

C.4. Testing models.

  • The standard models are WideResNet-28-10 for CIFAR-10 and ResNet-50 for ImageNet; adversarially trained counterparts use models from Gowal et al. (2020) and Salman et al. (2020a).Source: §C.4 · p. 13

D.1. Efficient Structure for Inference Speed Up

  • Removing redundant repeated predictions of the same pixels simplifies local implicit purification and accelerates the original implementation fourfold.Figure 9 illustrates the earlier local ensemble mechanism, which the authors say is unnecessary for purification.Source: §D.1 · pp. 13–14, Figure 9 · p. 14

D.2. Training Diversified Purifiers

The diversified-purifier training setup combines influence factors to produce multiple models per dataset, using dataset-specific training data. The appendix also identifies the non-defended classifiers used to generate adversarial examples.

D.2. Training Diversified Purifiers

  • Combinations of listed influence factors are used to train 12 different purification models for each dataset.The factors are summarized in Table 9.Source: §D.2 · p. 14, Table 9 · p. 14
  • Adversarial examples are generated using a pretrained ResNet-18 on CIFAR-10 and a pretrained ResNet-50 on ImageNet.The models and their clean test accuracy are listed in Table 10.Source: §D.2 · p. 14, Table 10 · p. 14
  • Purifiers are trained on the full CIFAR-10 training set; for ImageNet, training uses 10 randomly sampled images per class.Source: §D.2 · p. 14

E. Supplementary Materials for Theoretical Analysis

The section introduces a Gaussian-Smoothing function corresponding to f(x), with μ > 0 and u drawn from a standard normal distribution.

E. Supplementary Materials for Theoretical Analysis

  • Gaussian-Smoothing is introduced for f(x) with μ > 0 and u ∼ N(0, I).The supplied passage ends before giving the function expression.Source: §E.1 · p. 15

E.2. Proof of Theorem 1

The proof uses Lipschitz assumptions and normal-distribution moments to derive a query lower bound for controlling the expected squared gradient norm; the appendix also identifies ImageNet robust-accuracy evaluation settings.

E.2. Proof of Theorem 1

  • The proof defines the p-order moment of a normal distribution and cites Nesterov & Spokoiny (2017) for the complete proofs of its essential lemmas.Source: §E.2 · p. 15
  • The lemmas assume Lipschitz continuity for f and dimension-wise Lipschitz continuity for m, with L0(m) defined as the maximum component Lipschitz constant.Source: §E.2 · p. 15
  • The proof derives a lower bound on expected queries needed to make the expected squared gradient norm of Fμ of order δ.The bound follows minimization of its right-hand side.Source: §E.2 · p. 16

F. Accuracy for ImageNet

  • Table 11 reports ImageNet robust accuracy for five SOTA query-based attacks at 200 and 2,500 queries.Source: Table 11 · p. 17

G. Influence on Clean Accuracy

PuriDefense’s purification models compare favorably on clean accuracy, which the reported patch-count variation does not affect. In a Google Vision API example, PuriDefense prevents finding a subtly different adversarial image, and the discussion says mechanism development need not rely on a specific data distribution.

G. Influence on Clean Accuracy

H.2. Discussion on Real-world Attack

  • With PuriDefense on Google Vision API, the Boundary Attack found no adversarial image with a subtle difference from the original.Without protection, the attack produced an almost imperceptibly different adversarial image.Source: §H.1 · p. 18, Figure 12 · p. 19
  • The discussion says general pictures can support mechanism development without considering the dataset distribution.Source: §H.2 · p. 19
Loading 2401.10586v2…