PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Effectiveness of Low Frequency Perturbations

Yash Sharma, Gavin Weiguang Ding, Marcus Brubaker

arXiv:1903.00073v2cs.CVcs.CRcs.LGstat.ML

TL;DR

The paper asks whether low-frequency attacks succeed because they reduce the search space or because of the frequencies they preserve. It systematically controls perturbation frequencies and evaluates attacks against ImageNet defenses. Gains appear when low frequencies are preserved, but those perturbations can be perceptible at the competition bound.

  • Problem

    It is unclear whether low-frequency attacks work because they reduce the search space generally or because they remove high-frequency components, and when they outperform unconstrained attacks.

    Source: §1 · p. 1

  • Method

    The authors systematically manipulate perturbation frequency components and evaluate attacks against state-of-the-art ImageNet defenses.

    Source: §1 · p. 1

  • Results

    Low-frequency components yield improvements in white-box and transfer settings, and defended ImageNet models are roughly as vulnerable to low-frequency perturbations as undefended models.

    Source: §1 · p. 1, §5 · p. 7

  • Takeaways & Limitations

    At ℓ∞ ε = 16/255, low-frequency perturbations are perceptible, challenging ℓ∞ as a perceptual metric and motivating frequency-aware robustness.

    Source: §1 · p. 2, §5 · p. 7

    The study leaves unresolved whether defense vulnerability within the low-frequency subspace reflects intrinsic difficulty or adversarial training rarely sampling it.

    Source: §5 · p. 7

Abstract

from arXiv · show

Carefully crafted, often imperceptible, adversarial perturbations have been shown to cause state-of-the-art models to yield extremely inaccurate outputs, rendering them unsuitable for safety-critical application domains. In addition, recent work has shown that constraining the attack space to a low frequency regime is particularly effective. Yet, it remains unclear whether this is due to generally constraining the attack search space or specifically removing high frequency components from consideration. By systematically controlling the frequency components of the perturbation, evaluating against the top-placing defense submissions in the NeurIPS 2017 competition, we empirically show that performance improvements in both the white-box and black-box transfer settings are yielded only when low frequency components are preserved. In fact, the defended models based on adversarial training are roughly as vulnerable to low frequency perturbations as undefended models, suggesting that the purported robustness of state-of-the-art ImageNet defenses is reliant upon adversarial perturbations being high frequency in nature. We do find that under $\ell_\infty$ $\epsilon=16/255$, the competition distortion bound, low frequency perturbations are indeed perceptible. This questions the use of the $\ell_\infty$-norm, in particular, as a distortion metric, and, in turn, suggests that explicitly considering the frequency space is promising for learning robust models which better align with human perception.

1 Introduction

The paper asks whether low-frequency attacks work because they reduce the search space or because of the frequencies they preserve, and under what conditions they are more effective. Its experiments attribute the gains to low-frequency filtering, while finding that those perturbations can be perceptible at the competition distortion bound.

1 Introduction

  • The open questions are whether low-frequency effectiveness comes from the specific frequencies or simply a smaller search space, and when it exceeds unconstrained attacks.Earlier work reported improved query efficiency, transferability, or competition performance with low-frequency constraints or smoothing.Source: §1 · p. 1
  • Low-frequency perturbations improve attack generation and transfer because of low-pass filtering, not merely reduced search dimensionality.Constrained attacks in other frequency subspaces generally perform worse.Source: §1 · p. 1
  • Against defended models, low-frequency attacks are roughly as effective as attacks on undefended models, whereas clean models show no significant improvement.The authors suggest that ImageNet defenses rely on adversarial perturbations being high frequency.Source: §1 · pp. 1–2
  • Based on perceptual differences between unconstrained and low-frequency attacks, the paper discusses the commonly used ℓ∞ norm as a perceptual metric for quantifying robustness and highlights frequency properties as promising for learning robust models aligned with human perception.Source: §1 · p. 2

2 Background

The background frames adversarial-example generation as an optimization problem and transfer as a generalization problem. It surveys attack settings and defenses, noting limitations of adversarial training across perturbation types.

2 Background

  • Generating an adversarial example is an optimization problem, while generating one that transfers is a generalization problem.The perturbation is optimized to fool a model while its magnitude is constrained or minimized.Source: §2 · p. 2
  • Attacks range from white-box and limited-query settings to black-box transfer, and adversarial perturbations also apply in discrete domains such as natural language.In black-box attacks without query access, transferability is needed.Source: §2 · p. 2
  • Adversarial training can improve defenses, but its robustness is limited to the perturbation properties used during training.The passage notes that training exclusively on ℓ∞ perturbations does not confer robustness to other distortion metrics; NeurIPS 2017 winners built on adversarially trained models.Source: §2 · p. 2

3 Methods

The methods use iterative attacks and DCT masks to test how preserving different frequency components affects adversarial perturbations. The masks remove selected components during optimization and reconstruction, enabling comparisons among low-, mid-, high-, and randomly selected frequencies.

3.1 Attacks

  • The attack evaluation uses ℓ∞-constrained perturbations, with the competition bound set to 16 and attack success measured for targeted and non-targeted attacks.FGSM is a one-step gradient method; the methods section selects MIM for the defense evaluation because its momentum term is intended to improve transferability.Source: §3.1 · p. 2

3.2 Frequency Constraints

  • DCT represents an image or perturbation as frequency components, and a binary mask removes selected components before IDCT reconstructs the transformed perturbation.The mask is applied element-wise to the DCT coefficients.Source: §3.2 · pp. 2–3
  • Four mask types preserve low, mid, high, or randomly sampled frequency components, with n × n components retained for reduced dimensionality n.For d = 299 and n = 128, the low-frequency mask retains about 18.3% of the components; Figure 1 marks masked components in red.Source: Figure 1 · p. 2, §3.2 · p. 3

4 Results and Analyses

The experiments systematically vary perturbation frequency and test attacks against top NeurIPS 2017 defenses. Low-frequency constraints outperform other frequency constraints and expose vulnerabilities in defended ImageNet models, while the reported benefit does not extend to CIFAR-10.

4.1 Overview of the Results

  • DCT Low consistently outperforms DCT High, DCT Mid, and DCT Rand; spatial smoothing and downsampling-upsampling results likewise indicate that low-frequency content, rather than search-space restriction alone, drives the effect.The experiments cover non-targeted attacks with 1 and 10 iterations and targeted attacks with 10 iterations; subsequent experiments focus on DCT Low.Source: §4.1 · p. 4, Figure 2 · p. 4

4.2 Observations and Analyses

  • Against defended models, DCT Low improves white-box attack success in some settings and is roughly as effective against defended models as against undefended ones, unlike unconstrained MIM.For white-box attacks on cleanly trained NasNet, DCT Low performs worse than unconstrained MIM; its advantage is not general across clean models.Source: §4.2 · p. 4, §4.3 · p. 6
  • In grey-box tests, DCT Low outperforms unconstrained MIM against the defended models, and the prepended preprocessors provide no additional protection against low-frequency perturbations, even for targeted attacks.D1, the competition-winning defense, has the same reported ASR as the adversarially trained model without its preprocessor.Source: §4.2 · pp. 4–5
  • For black-box transfer to defended models, DCT Low significantly outperforms unconstrained MIM in non-targeted attacks, but does not enable targeted transfer to distinct defended models.The comparison averages ASR over 20 source-target pairs.Source: §4.2 · p. 5
  • Low-frequency constraints do not improve black-box transfer between cleanly trained models; the authors report that their advantage instead exploits vulnerabilities in the defenses considered.The comparison includes transfers from cleanly trained sources to cleanly trained targets and from adversarially trained sources to defended targets.Source: §4.2 · p. 5

4.4 Effectiveness of Low Frequency on CIFAR-10

  • On CIFAR-10, constraining the adversary with DCT Low only hurts performance in the tested white-box and black-box settings, unlike the reported ImageNet results.The evaluation attacks an adversarially trained model using non-targeted PGD with 10 iterations and ε=8/255.Source: §4.4 · p. 6, Table 2 · p. 6

5 Discussion

The experiments find that frequency placement—not simply a smaller attack search space—accounts for low-frequency gains against defended models, while clean models do not show those gains. The discussion also notes that low-frequency perturbations can be perceptible under the competition bound and calls for considering perceptual priors.

5 Discussion

  • In white-box, grey-box, and black-box settings, significant gains occurred only when low-frequency components were preserved, not merely from shrinking the attack space.The evaluated models included adversarially trained models and winning NeurIPS 2017 defense submissions.Source: §5 · p. 7
  • Against low-frequency attacks, state-of-the-art ImageNet defenses were roughly as vulnerable as undefended, cleanly trained models.The authors suggest defenses reduce vulnerability outside the low-frequency subspace more than within it.Source: §5 · p. 7
  • Low-frequency perturbations did not improve white-box attack speed on clean models or transfer between cleanly trained models.The discussion reports no improvement when defended models were absent.Source: §5 · p. 7
  • The authors leave unresolved whether defended models' low-frequency vulnerability reflects intrinsic robustness difficulty or adversarial training rarely sampling that region.They identify this distinction as a direction for further work.Source: §5 · p. 7
  • At ℓ∞ ε=16/255, low-frequency perturbations can be perceptible, including examples with n ≤128, despite remaining recognizable.The paper says the top example still appears to be a standing woman.Source: §5 · p. 7
  • Because ℓp bounds do not align with human perception, the authors urge considering frequency space and perceptual priors when bounding perturbations and designing defenses.They note that visible low-frequency perturbations challenge the assumption that a small norm bound guarantees imperceptibility.Source: §5 · p. 7

A Construction Process

The construction process defines frequency masks that preserve a specified number of components while allocating them to low, high, middle, or randomly selected frequency bands.

A Construction Process

  • DCT Low preserves components v_i,j when 1 ≤ i,j ≤ n.The construction uses the n × n low-frequency subspace for reduced dimensionality n.Source: A Construction Process · p. 9
  • DCT High masks the lowest-frequency components and preserves n² components in high-frequency bands.The mask is configured to match the preserved-component count of the low-frequency construction.Source: A Construction Process · p. 9
  • DCT Mid masks equal numbers of low- and high-frequency components.Its low- and high-frequency cutoffs are set through the described r_ml and r_mh construction.Source: A Construction Process · p. 9
  • DCT Random selects rows and columns randomly, preserving n² components in both spatial directions.The construction chooses the same number of rows or columns used to match the other masks.Source: A Construction Process · p. 9

B Spatial Smoothing & Downsampling-Upsampling Filters

DCT Low outperforms the other frequency configurations across attack settings, and spatial smoothing and downsampling-upsampling show mirrored trends.

B Spatial Smoothing & Downsampling-Upsampling Filters

  • DCT Low significantly outperforms DCT High, DCT Mid, and DCT Random in white-box, grey-box, and black-box settings.The reported results also include faster effective perturbation generation on adversarially trained models, but not clean models, and improved transfer to defended models.Source: B Spatial Smoothing &… · p. 9
  • DCT Low is not effective for transfer between undefended, cleanly trained models.The section also reports that it bypasses defenses prepended to adversarially trained models.Source: B Spatial Smoothing &… · p. 9
  • Downsampling-upsampling reduces perturbation dimensionality from 299 to 32, while 7×7 Gaussian smoothing yields the same trends despite differing parameters.The authors report that both spatial smoothing and downsampling-upsampling mirror the low-frequency perturbation results.Source: B Spatial Smoothing &… · p. 9

C Complete Heatmap

Figure 9 presents DCT Low attack success rates with source models as rows and target models as columns, separating white-box, grey-box, and black-box results.

C Complete Heatmap

  • Figure 9 arranges attack success rates by source-model rows and target-model columns.The source and target entries together cover the white-box, grey-box, and black-box settings.Source: C Complete Heatmap · p. 9
  • The (Cln, Cln) and (Adv, Adv) entries are white-box results, while (Adv, D1–D4) entries are grey-box results.The remaining entries summarize black-box results.Source: C Complete Heatmap · p. 9
  • The heatmap uses DCT Low with n=128.Cln denotes NasNetLarge 331, Adv denotes EnsAdvInceptionResNetV2, and D1–D4 are the defenses summarized in the main text.Source: C Complete Heatmap · p. 9

D All Plots

The appendix collects white-box, grey-box, and black-box attack plots across defended and cleanly trained models. It also includes filter comparisons and a transferability matrix comparing unconstrained MIM with DCT Low.

Adversarially trained models

  • Figures 10–14 show white-box attacks on an adversarially trained model and grey-box attacks on defenses D1–D4.The white-box result is Figure 10; Figures 11–14 cover D1, D2, D3, and D4, respectively.Source: D All Plots · p. 9, Figure 10 · p. 12, Figure 11 · p. 12
  • Figures 15–34 show black-box transfers from Cln 1, Cln 3, Adv 1, and Adv 3 to EnsAdv and defenses D1–D4.The source-to-target groups are identified in the prose; the figures include, for example, Cln 1 to EnsAdv and Cln 3 to EnsAdv.Source: D All Plots · p. 9, Figure 15 · p. 13, Figure 20 · p. 15

Cleanly trained model

Filter comparisons

  • Figures 7 and 8 present results for spatial smoothing and downsampling-upsampling filters, respectively.Their captions specify iteration counts in parentheses and distinguish non-targeted ε = 16/255 from targeted ε = 32/255 settings.Source: Figure 7 · p. 10, Figure 8 · p. 10

Transferability matrix

  • Figure 9 compares transferability for standard unconstrained MIM and DCT Low at n = 128 across non-targeted and targeted settings.Its three rows correspond to non-targeted attacks with 1 iteration, non-targeted attacks with 10 iterations, and targeted attacks with 10 iterations.Source: Figure 9 · p. 11
Loading 1903.00073v2…