PaperlayerView exact revision on arXiv

Source-linked AI summary

Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity

Cheng Luo, Qinliang Lin, Weicheng Xie, Bizhu Wu, Jinheng Xie, Linlin Shen

arXiv:2203.05151v4cs.CV

TL;DR

Existing attacks are constrained by closed-set classification layers and can produce perceptible perturbations. The paper attacks semantic similarity in feature space and adds a low-frequency constraint to restrict perturbations to high-frequency components. Across datasets and architectures, the resulting examples are reported as transferable, misleading, and more imperceptible than state-of-the-art methods.

  • Problem

    Existing attack methods have limited cross-dataset generalization because they rely on classification layers with closed sets of categories, while their perturbations may be perceptible to the human visual system.

  • Method

    The paper proposes SSA, which attacks semantic similarity in feature representations, and SSAH, which adds a low-frequency constraint to confine perturbations to high-frequency components.

  • Results

    The attack produces misleading and transferable adversarial examples across architectures and datasets, with more imperceptible perturbations than state-of-the-art methods.

  • Takeaways & Limitations

    Feature-space attacks with frequency-domain constraints can operate across broader settings while retaining high imperceptibility.

Abstract

from arXiv · show

Current adversarial attack research reveals the vulnerability of learning-based classifiers against carefully crafted perturbations. However, most existing attack methods have inherent limitations in cross-dataset generalization as they rely on a classification layer with a closed set of categories. Furthermore, the perturbations generated by these methods may appear in regions easily perceptible to the human visual system (HVS). To circumvent the former problem, we propose a novel algorithm that attacks semantic similarity on feature representations. In this way, we are able to fool classifiers without limiting attacks to a specific dataset. For imperceptibility, we introduce the low-frequency constraint to limit perturbations within high-frequency components, ensuring perceptual similarity between adversarial examples and originals. Extensive experiments on three datasets (CIFAR-10, CIFAR-100, and ImageNet-1K) and three public online platforms indicate that our attack can yield misleading and transferable adversarial examples across architectures and datasets. Additionally, visualization results and quantitative performance (in terms of four different metrics) show that the proposed algorithm generates more imperceptible perturbations than the state-of-the-art methods. Code is made available at.

1. Introduction

Existing adversarial attacks face two central problems: limited cross-dataset generalization from classification-layer dependence and perturbations that are perceptible to humans. The paper proposes SSA and SSAH to attack feature-representation similarity while constraining perturbations to high-frequency components, yielding transferable and more imperceptible examples.

  • Motivation: Classification-layer attacks are limited to settings where training and attack images share the same category set.Open-set images may belong to categories unknown to the classifier.
  • Motivation: ℓp distance alone may not capture perceptual similarity, and C&W can produce visible perturbations on smooth backgrounds.The paper identifies poor imperceptibility to the human visual system as a second open problem.
  • Proposed approach: SSA attacks semantic similarity in feature representations by separating adversarial and benign examples while bringing adversarial examples toward dissimilar targets.This avoids requiring knowledge of the specific image category or classification layer.
  • Proposed approach: SSAH adds a low-frequency constraint that limits perturbations to imperceptible high-frequency components.The constraint is motivated by the distinction between basic low-frequency information and less perceptible high-frequency signals.
  • Results: Experiments on CIFAR-10, CIFAR-100, and ImageNet-1K show that the attack produces significantly more imperceptible perturbations than state-of-the-art methods.The reported evaluation covers extensive experiments across three datasets.
  • Results: SSAH generates adversarial perturbations that transfer more effectively across architectures and datasets.The contribution statement reports improved transferability across both dimensions.

2. Related Work

Prior feature-space attacks manipulate image representations by matching source images to targets, while wavelet-based methods decompose images into frequency components. SSAH combines representation-space attack goals with a frequency-domain perturbation constraint.

  • Feature Space Attack: Feature-space attacks manipulate image representations to resemble target images, often minimizing inter-image feature distances in a target network.Related methods also apply this strategy to person re-identification and image retrieval.
  • Proposed framework: SSAH uses a semantic-similarity attack together with a low-frequency constraint that preserves basic object information while restricting perturbations to high-frequency components.The framework is presented as combining feature-space manipulation with frequency-domain control.
  • Wavelets in Deep Learning: Discrete wavelet transform decomposes image data into multiple frequency components and has been incorporated into deep-learning layers.Prior work uses DWT for visual tasks including image segmentation.

3. Methodology

SSAH attacks pairwise feature similarity instead of classification logits, while its low-frequency constraint preserves basic image information by confining perturbations to less perceptible high-frequency details.

  • 3.1. Attack Design: The untargeted objective optimizes the adversarial representation against the original and lowest-similarity example, with a targeted variant matching a specified target.The hinge term uses the adversarial-original similarity and either the minimum other similarity or target similarity.
  • 3.1. Attack Design: SSAH minimizes similarity between an adversarial example and its original while increasing similarity to the most dissimilar minibatch example.It uses cosine similarity between feature embeddings and does not require the classifier's classification layer.
  • 3.1.2. Self-paced Weighting: Self-paced weighting assigns smaller gradients to nearly optimized similarity scores and larger gradients to less optimized scores.The adaptive factors α_i and β_i depend on a predefined margin m.
  • 3.2. Low-frequency Constraint: DWT separates each image into one low-frequency component carrying basic information and three high-frequency components associated with edges, textures, and variations.The method reconstructs a low-frequency-only image to assess perceptual information and constrains changes through this representation.
  • 3.2. Low-frequency Constraint: The low-frequency constraint reduces perceptual information loss by minimizing the L1 difference between low-frequency reconstructions of original and perturbed images.The constraint is motivated by the lower perceptibility of perturbations in complex textures than in smooth regions.
  • 3.3. The Unified Attack: SSAH jointly minimizes the semantic-similarity loss and λ-weighted low-frequency constraint to produce the adversarial example.The algorithm computes similarity scores and constraint loss, then optimizes the perturbation variable over K iterations.

4. Experiments

Experiments evaluate SSAH across datasets, white-box and defended models, transfer settings, online platforms, and component ablations. Results indicate strong attack effectiveness alongside improved perceptual similarity, with perturbations concentrated in high-frequency visual regions.

  • Experimental setup: The evaluation covers CIFAR-10, CIFAR-100, and ImageNet-1K in white-box experiments.The white-box setting assumes full access to the target system.
  • White-box attacks: SSAH succeeds across all three datasets while producing the lowest ℓp perturbation norms among nine attack approaches.Table 1 reports attack success rate and perceptual-similarity metrics in the untargeted scenario.
  • White-box attacks: 3.90 FID on ImageNet-1K versus 11.56 FID for PerC-AL indicates substantially less feature-space variation for SSAH.The paper interprets the result as more realistic pixel-space visual effects with fewer feature-space variations.
  • White-box attacks: SSAH significantly outperforms other methods on LF, preserving object structure and low-frequency components.The targeted attack scenario follows a similar pattern.
  • Robustness: 29.55% on CIFAR-10 and 69.26% on CIFAR-100 are the accuracy decreases achieved against FSAT, while SSAH remains competitive against TRADES.Against TRADES on CIFAR-10, SSAH improves by 2.56% over C&W.

5. Conclusion

SSAH attacks semantic similarity in representation space rather than relying on a classification layer, while its low-frequency constraint confines perturbations to imperceptible high-frequency components. Experiments support its use for transferable attacks across architectures and datasets and for misleading online models while retaining high imperceptibility.

  • Conclusion: SSAH attacks semantic similarity in representation space and introduces a low-frequency constraint for imperceptible perturbations.The constraint limits perturbations within high-frequency components while preserving basic image information.
  • Conclusion: The framework supports transferable adversarial examples across architectures and datasets and attacks unknown online models without classification queries.The conclusion frames these as more general and practical black-box settings.
  • Conclusion: Extensive experiments show that constrained perturbations improve imperceptibility, particularly in smooth regions.

A. Methodology for Targeted Attack

The targeted SSA formulation guides adversarial examples toward a specified target image in feature-representation space while using self-weighting factors to balance similarity terms.

  • The targeted attack encourages each adversarial example x′_i to become close to a target image x_t in feature representation.
  • The targeted objective minimizes a hinge expression balancing self-similarity s′_i,i against target similarity s′_i,t.
  • The self-weighting scheme assigns α_i=[s′_i,i−m]_+ and β_i=[1+m−s′_i,t]_+ to regulate the two similarity terms.
  • Unlike the untargeted variant, targeted SSA uses a specified target-category example rather than selecting the most dissimilar minibatch example.

B. Implementation details

The targeted evaluation shows that SSAH remains effective at producing imperceptible perturbations, while comparisons use fixed attack parameters and an ℓ∞ bound for robustness tests.

  • For robustness and online-model attack-success evaluations, perturbation generation uses an ℓ∞ bound of 8/255.
  • Compared attacks use κ=40 and c=0.1 for C&W, while PerC-AL parameters are initialized at 1 and 0.5 and annealed toward 0.01 and 0.05.
  • SSAH remains effective at generating imperceptible perturbations in the targeted attack scenario.

C.2. Parameter Sensitivity Analyses

SSAH’s sensitivity depends on the margin m and low-frequency weight λ: large margins make attack success rate more sensitive to λ, whereas moderate parameter ranges are relatively stable.

  • SSAH has two key hyperparameters: margin m for self-paced weighting and λ for the low-frequency constraint.
  • The sensitivity study combines CIFAR-10 quantitative analyses with ImageNet-1K visualizations.
  • 82.02% at λ=0.00 falls to 60.98% at λ=1.00 when m=0.4, showing reduced attack success under stronger low-frequency weighting.
  • SSAH becomes sensitive to λ when m is large, particularly for m≥0.2.
  • SSAH achieves relatively stable and satisfying performance for m∈[0.0,0.2] and λ∈[0.1,1.0].
  • Batch-size results remain similarly good from 32 to 10000, with differences at the level of random variations.

C.4. Additional Evaluation of Imperceptibility

Additional evaluations support SSAH’s imperceptibility and robustness: it remains competitive without an ℓ2 colour-difference constraint and withstands bit-depth reduction.

  • SSAH without an ℓ2 (C2) distance constraint still achieves competitive performance on the ℓ2 (C2) perceptual-colour metric.
  • The ImageNet-1K untargeted evaluation reports attack success rate alongside three perceptual-similarity metrics across three attack approaches.
  • SSAH’s perturbations on high-frequency components remain robust against bit-depth reduction.

C.6. Analysis of SPW

The analysis examines self-paced weighting in SSAH and presents visualizations of perturbations, adversarial examples, and transferability across architectures and datasets.

  • Robustness evaluation: Figure 11 evaluates the robustness of adversarial examples from three attack approaches against bit-depth reduction on CIFAR-10.The supplied passage identifies the evaluation setting but does not report the comparative outcome.
  • Effect of SPW: SPW produces smaller perturbation intensities than omitting SPW on randomly sampled CIFAR-10 test images.The authors attribute this reduction to mitigating redundant perturbations caused by over-optimization.
  • Visualization results: The visualizations include white-box adversarial examples and perturbations for CIFAR-10, CIFAR-100, and ImageNet-1K.Figures 13–16 cover these examples across multiple attack approaches.
  • Transferable attack: Figure 17 presents transferable adversarial examples crafted from source-domain surrogate models to target models trained on different datasets.The figure uses A B →C D notation to identify source and target model–dataset pairs.
Loading 2203.05151v4…