PaperlayerView exact revision on arXiv

Source-linked AI summary

Unrestricted Adversarial Examples via Semantic Manipulation

Anand Bhattad, Min Jin Chong, Kaizhao Liang, Bo Li, D. A. Forsyth

arXiv:1904.06347v2cs.CV

TL;DR

Existing adversarial attacks commonly constrain perturbations by Lp norms, but that metric is imperfect and defenses trained on bounded perturbations may not withstand unseen attacks. This paper introduces unrestricted color and texture attacks for classifiers and captioners, finding them natural to humans, transferable, and harder for defenses to handle. The cAdv and tAdv strategies therefore provide semantic adversarial examples with large structured changes across image tasks.

  • Problem

    Norm-bounded adversarial perturbations rely on an imperfect perceptual metric, while defenses trained on bounded perturbations may fail against unseen attacks.

  • Method

    The paper manipulates semantic visual representations through colorization cAdv and texture transfer tAdv to generate unrestricted adversarial examples.

  • Results

    The attacks work across image classification and captioning, appear natural to humans despite large perturbations, and are reported as more transferable and harder to defend against.

  • Takeaways & Limitations

    Semantic color and texture perturbations expose vulnerabilities in DNN predictions and motivate unbounded attacks, improved perturbation metrics, and stronger defenses.

  • Takeaways & Limitations

    Directly attacking cAdv network weights can produce adversarial colors that are not realistic.

Abstract

from arXiv · show

Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with a small magnitude of the perturbation. Such adversarial perturbations are usually restricted by bounding their $\mathcal{L}_p$ norm such that they are imperceptible, and thus many current defenses can exploit this property to reduce their adversarial impact. In this paper, we instead introduce "unrestricted" perturbations that manipulate semantically meaningful image-based visual descriptors - color and texture - in order to generate effective and photorealistic adversarial examples. We show that these semantically aware perturbations are effective against JPEG compression, feature squeezing and adversarially trained model. We also show that the proposed methods can effectively be applied to both image classification and image captioning tasks on complex datasets such as ImageNet and MSCOCO. In addition, we conduct comprehensive user studies to show that our generated semantic adversarial examples are photorealistic to humans despite large magnitude perturbations when compared to other attacks.

1 INTRODUCTION

The paper addresses limitations of norm-bounded adversarial examples by introducing unrestricted semantic perturbations that manipulate color and texture for attacks across vision tasks. These attacks are evaluated on classifiers and captioners, with studies indicating natural human perception, transferability, and difficulty for defenses.

  • Motivation: Norm-bounded attacks can preserve photorealism, but Lp distance is an imperfect perceptual metric and defenses trained on bounded perturbations may fail against unseen attacks.This motivates exploring unrestricted perturbations with larger semantic changes.
  • Approach: The paper proposes colorization (cAdv) and texture-transfer (tAdv) attacks that manipulate semantic visual representations without constraining perturbation magnitude.cAdv changes selected image colors, while tAdv transfers and adjusts texture using style transfer.
  • Scope: The attacks target both image classification and image captioning on large-scale datasets including ImageNet and MSCOCO.Captioning attacks can alter an entire caption or specific target words.
  • Evaluation: User studies found the semantic adversarial examples more natural to humans than other attacks despite their large perturbations.The paper also tests the attacks against several state-of-the-art defenses and reports greater transferability.
  • Implications: The proposed attacks are presented as harder to defend against and as evidence that color and texture fields influence DNN predictions.The conclusion frames these findings as motivation for unbounded attacks, better perturbation metrics, and more sophisticated defenses.

2 COLORIZATION ATTACK (cADV)

cAdv generates adversarial colorizations by searching a pretrained colorization model while controlling spatial locations and color hints. Its design balances attack effectiveness, color realism, and diversity through entropy-based clustering and hint selection.

  • Method: cAdv searches for natural but adversarial colorizations by leveraging a pretrained colorization model that respects image boundaries and short-range color consistency.The goal is to create smooth, consistent perturbations with potentially large magnitude.
  • Attack objectives: The attack can optimize the colorization network directly by minimizing cross-entropy adversarial loss against a target class.Here, t denotes the target class and F the victim network.
  • Attack objectives: Alternatively, cAdv jointly varies color hints and their spatial mask to manipulate output colorization while preserving the pretrained network.Hints provide ground-truth color patches, and the mask specifies their locations.
  • Attack methods: Attacking network weights gives unconstrained adversarial colors but often produces unrealistic outputs, whereas jointly attacking hints and masks yields more natural results.The weight-based outputs nevertheless often correlate with colors associated with the target class.
  • Control over colorization: Entropy-based clustering identifies ambiguous regions for color changes, avoiding large edits where colors are unambiguous.The method smooths AB channels, clusters them with K-Means, and uses average color-distribution entropy to assess ambiguity.
  • Control over colorization: Increasing hints improves similarity to ground-truth colors until excessive hints make optimization difficult and can produce unrealistic perturbations.The paper reports k = 4 and 50 hints as a practical balance among color diversity, realism, and robustness.
  • Control over colorization: Sampling hints from more clusters improves realism and ground-truth color similarity at the expense of color diversity.This exposes a controllable trade-off between realistic colorization and varied adversarial colors.

3 TEXTURE ATTACK (tADV)

tAdv generates unrestricted adversarial examples by transferring texture from target images and optimizing texture-transfer and adversarial objectives. Cross-layer statistics, texture-source selection, and texture-weight control help balance attack success with visual realism.

  • Texture Attack: tAdv transfers texture from a target image into a victim image using pretrained VGG19 textural features without explicit Lp constraints.The method directly optimizes the victim image and uses gram-matrix statistics to guide texture transfer.
  • Texture Attack: The tAdv objective combines texture-transfer loss with an adversarial cross-entropy objective, without an additional content loss.The adversarial objective helps transform the victim image while limiting artistic stylization.
  • Texture Transfer: Nearest-neighbor target-class texture sources produce more natural-looking adversarial images than random texture sources.Random sources are successful but clearly perceptible, while target-class and nearest-neighbor selection reduce perceptibility, with the nearest strategy producing the most natural results.
  • Texture Transfer: Cross-layer statistics from pretrained VGG19 layers R11, R21, R31, R41, and R51 are optimized with L-BFGS to perform texture transfer.The procedure requires stopping control because unconstrained texture optimization can turn images into artistic images.
  • Evaluation: Table 1 evaluates attacks on ResNet50, DenseNet121, and VGG19, including tAdv settings α = {250, 500} and β = 0.001 and transferability across models.The table attacks models listed in columns and tests them on models listed in rows.
  • Control over Texture: Increasing texture weight α improves attack success rate but increases noticeable perturbation and can introduce artifacts.The paper reports α values in [150, 1000] and β values in [10^-4, 10^-3] as empirically successful ranges; tabulated results use α = 250 and β = 10^-3 unless otherwise stated.

4 EXPERIMENTAL RESULTS

Experiments show that cAdv and tAdv can fool classifiers through large, structured semantic changes, while retaining realistic or imperceptible properties and showing robustness against several defenses.

  • cAdv attack: cAdv achieves high targeted attack success with realistic color perturbations, but larger color changes slightly reduce human-perceived realism.The larger changes improve transferability and defense robustness while creating a realism trade-off.
  • cAdv attack: cAdv perturbations are locally smooth, relatively low-frequency, and aligned with object boundaries.The pretrained colorization model produces these properties because natural colors are typically smooth within object boundaries.
  • cAdv attack: Color changes can drive classification errors, including a car being classified as tench after its gray road becomes blue.The authors hypothesize that blue colors and tinting made the scene resemble a tench in the sea.
  • cAdv attack: The colorization model enables large, natural-looking color changes, providing evidence that color may play a stronger role in classification than expected.The authors reserve further exploration of this observation for future work.
  • cAdv attack: cAdv methods produce isoluminant color blobs, potentially exploiting colors that ImageNet classifiers have rarely encountered.These changes alter color without corresponding luminance changes, often along natural-image edges.
  • tAdv attack: tAdv fools classifiers with a very small weighted adversarial cross-entropy objective when combined with texture loss while remaining realistic to humans.It achieves high white-box success across three models and is more transferable to other models.
  • tAdv attack: tAdv perturbations are structured, organized, high-frequency, and imperceptible despite being large under the Lp norm.Their texture-pattern structure comes from extracting features across different VGG layers.
  • Defense and transferability analysis: The evaluation tests ResNet50 attacks against JPEG compression, feature squeezing, and adversarial training.The defense experiments include JPEG compression at quality 75 and an adversarially pretrained ResNet152 model.

5 HUMAN PERCEPTUAL STUDIES

The human study compares adversarial and benign images under a controlled identification task to quantify how realistic cAdv and tAdv examples appear.

  • Human perceptual studies: The AMT study presents random adversarial-benign pairs for three seconds and asks workers to identify the realistic image.Each attack uses 600 unique pairs, with every pair evaluated by at least 10 unique workers.

6 ATTACKING CAPTIONING MODEL

The semantic attacks extend to image captioning, where they target an exact word while preserving the rest of the caption and avoiding visible artifacts. cAdv optimizes color hints, whereas tAdv incorporates texture information without changing network weights.

  • Captioning attack: The attacks adapt to captioning by changing exactly one target word while keeping the rest of the caption unchanged.The demonstrated targeted attack changes the second word to “dog,” making it more demanding than untargeted caption changes.
  • Captioning attack: cAdv and tAdv successfully produce the targeted caption without visible artifacts.The paper presents this as a successful attack on a challenging captioning task.
  • Optimization: For cAdv, the method optimizes color hints; for tAdv, it adds a texture-related objective during image optimization.The optimization stops after the target caption is reached and remains unchanged across consecutive iterations.
  • Optimization: The captioning attack changes hints and masks rather than the network weights.The figure illustrates corresponding attention-mask changes for the attacked word.

7 RELATED WORK

Prior unrestricted attacks use spatial distortion, GANs, or color changes, but the reviewed methods differ in semantic control, dataset scalability, realism, and task coverage. The paper positions its attacks as unbounded, semantically motivated, and applicable to complex datasets and other tasks.

  • Prior unrestricted attacks: Spatial-flow attacks avoid L∞ constraints but do not explicitly control image semantics.They optimize pixel flow rather than pixel values to create natural-looking adversarial examples.
  • Prior unrestricted attacks: GAN-based unrestricted attacks have generally been limited to datasets such as MNIST, CIFAR, and CelebA rather than ImageNet-scale data.The paper identifies scalability to larger and more complex datasets as a limitation of these approaches.
  • Semantic attacks: Hue-and-saturation manipulation resembles cAdv but uses a two-dimensional search space and produces unrealistic images according to the paper.The cited method achieved a 1.20% targeted attack success rate in the authors’ ImageNet experiments.
  • Positioning: Compared with previous methods, the paper characterizes its attacks as unbounded, semantically motivated, realistic, scalable, and robust against tested defenses.The comparison covers complex datasets and other machine-learning tasks.

8 CONCLUSION

The paper concludes that color- and texture-based unrestricted attacks reveal how visual fields influence DNN predictions. It argues that these attacks can fool human subjects and are generally harder to defend against, motivating further work on unbounded attacks and defenses.

  • Conclusion: Color and texture fields influence DNN predictions in the proposed unrestricted semantic attacks.The conclusion frames these attacks as a tool for understanding prediction behavior.
  • Conclusion: The attacks consistently fool human subjects and are generally harder to defend against.The conclusion connects these properties to the need for further study of unbounded attacks and improved defenses.
  • Future work: The authors call for future studies on unbounded attacks, perturbation metrics, and more sophisticated defenses.This is presented as the broader research direction enabled by their findings.

A.1 OTHER DETAILS ON HUMAN STUDY

The human-study analysis compares perturbation magnitude with user preference across baseline and proposed attacks. cAdv and tAdv achieve large perturbations while maintaining user preference near the ideal level, although preference varies across image classes.

  • Density plots: The density plots compare L∞ norm perturbation on the x-axis with the probability that users prefer the adversarial image on the y-axis.All plots use the same axes, with baseline methods shown on the left.
  • Density plots: Ideal systems would form a concentrated horizontal line at user preference 0.5.The figure uses this value as the reference for no user preference between images.
  • User preference: cAdv and tAdv maintain user preference scores close to 0.5 despite very large Lp-norm perturbations.The table describes both attacks as imperceptible to human users under this criterion.
  • Class-wise variation: User preferences vary by class, with users favoring some classes and disliking others.Merganser and sandbar are picked relatively often, whereas photocopier is less preferred.
  • Perturbation comparisons: Compared with other attacks, cAdv perturbations are low-frequency and locally smooth, while tAdv perturbations are high-frequency and structured.The figure compares attacks changing tarantula into beacon, golf cart, nautilus, photocopier, and pretzel.

A.2 ADDITIONAL RESULTS

Additional results examine texture-weight and adversarial-loss ablations on ResNet50, alongside qualitative examples for color, texture, and captioning attacks. The examples illustrate how sampling choices and texture sources affect visual variety, artifacts, and photorealism.

  • Ablation Studies: The ablation varies texture weight α by columns and adversarial cross-entropy weight β by rows, with all attacks performed on ResNet50.The supplied passage introduces these as success-rate experiments using nearest target textures.
  • Color Attack: Sampling color hints across more clusters produces less color variety in cAdv examples targeting golf-cart.The figure also visualizes cluster segments, entropy, and sampled hint locations.
  • Color Attack: A comparison with Hosseini and Poovendran shows their attacked images as unrealistic relative to the color-attack examples presented here.The figure also points to qualitatively similar CIFAR-10 results in that prior work.
  • Texture Attack: Texture transferred from nearest neighbors of the adversarial target class yields more photorealistic images than textures from random sources.The first two texture-source settings show more artifacts as texture weight α increases.
  • Captioning Attack: The captioning attack targets the second caption word with dog or bird and displays the corresponding change in that word's attention mask.tAdv uses nearest-neighbor selection, whereas cAdv is initialized with all ground-truth color hints.
  • Additional Qualitative Examples: Qualitative comparisons include cAdv and tAdv adversarial examples with their perturbations, using 50 color hints for cAdv and specified α, β, and iteration settings for tAdv.Diagonal images are ground-truth images, and gray pixels indicate no perturbation.
Loading 1904.06347v2…