PaperlayerView exact revision on arXiv

Source-linked AI summary

Image Super-Resolution as a Defense Against Adversarial Attacks

Aamir Mustafa, Salman H. Khan, Munawar Hayat, Jianbing Shen, Ling Shao

arXiv:1901.01677v2cs.CV

TL;DR

CNNs are vulnerable to small, human-imperceptible adversarial perturbations, motivating defenses that preserve clean-image performance. The paper uses super-resolution with wavelet denoising to map adversarial samples toward the natural image manifold, and reports strong gray-box performance across attacks.

  • Problem

    CNNs can be fooled by small, human-imperceptible perturbations, creating a need for defenses that recover correct labels without degrading clean-image performance.

  • Method

    The defense combines wavelet denoising and super-resolution in a non-differentiable, model-agnostic image-restoration pipeline.

  • Results

    The method recovers about 96% of images for iterative C&W and DeepFool attacks and gains 31.1% over JPEG compression for FGSM (ϵ = 10).

  • Takeaways & Limitations

    Super-resolution is reported as an effective defense across a wide range of state-of-the-art attacks while minimally affecting clean, non-attacked images.

Abstract

from arXiv · show

Convolutional Neural Networks have achieved significant success across multiple computer vision tasks. However, they are vulnerable to carefully crafted, human-imperceptible adversarial noise patterns which constrain their deployment in critical security-sensitive systems. This paper proposes a computationally efficient image enhancement approach that provides a strong defense mechanism to effectively mitigate the effect of such adversarial perturbations. We show that deep image restoration networks learn mapping functions that can bring off-the-manifold adversarial samples onto the natural image manifold, thus restoring classification towards correct classes. A distinguishing feature of our approach is that, in addition to providing robustness against attacks, it simultaneously enhances image quality and retains models performance on clean images. Furthermore, the proposed method does not modify the classifier or requires a separate mechanism to detect adversarial images. The effectiveness of the scheme has been demonstrated through extensive experiments, where it has proven a strong defense in gray-box settings. The proposed scheme is simple and has the following advantages: (1) it does not require any model training or parameter optimization, (2) it complements other existing defense mechanisms, (3) it is agnostic to the attacked model and attack type and (4) it provides superior performance across all popular attack algorithms. Our codes are publicly available at https://github.com/aamir-mustafa/super-resolution-adversarial-defense.

I. INTRODUCTION

CNNs are widely deployed, including in security-sensitive applications, but small human-imperceptible perturbations can fool them and transfer across architectures. The paper proposes model-agnostic super-resolution and denoising to remap adversarial samples toward the natural image manifold without separate detection.

  • Motivation: CNNs support many vision tasks and critical applications, increasing the need for robustness against adversarial examples.Adversarial examples are created by small, carefully crafted perturbations that may be imperceptible to humans and transferable across architectures.
  • Proposed defense: The paper proposes a model-agnostic defense based on image super-resolution, wavelet filtering, and a joint non-differentiable module.The approach aims to remove adversarial noise and recover original class labels without requiring knowledge of the attacked model.
  • Contributions: Super-resolving adversarial images is presented as projecting them back onto the natural image manifold while retaining critical content on clean images.The method is intended to avoid the image artifacts and clean-image performance loss associated with some transformation-based defenses.
  • Relation to prior work: The defense does not require a prior detection scheme and is described as applicable to all types of natural images with a generic mapping function.This contrasts with Defense-GAN and MagNet, whose studied cases were restricted to small, low-resolution datasets.
  • Evaluation scope: The paper evaluates the proposed defense against state-of-the-art attacks and compares it with other model-agnostic defenses.The evaluation is described as extensive and focused on adversarial image classification.

B. Adversarial Attacks

The paper considers untargeted and targeted adversarial attacks that alter classifier outputs while keeping perturbed images visually similar to clean inputs. It reviews single-step, iterative, momentum-based, boundary-crossing, optimization-based, and diverse-input attacks.

  • Gradient attacks: FGSM is a single-step attack that perturbs the image using the loss gradient, with ϵ restricting the ℓ∞ norm.I-FGSM extends this strategy through repeated updates from the clean image.
  • Gradient attacks: MI-FGSM adds momentum to iterative gradient attacks to stabilize the gradient direction and help escape local maxima.The update uses a decay factor µ and reaches xadv after M iterations.
  • Boundary-based attack: DeepFool iteratively crosses classifier decision boundaries while minimizing the ℓ2 distance between clean and adversarial images.The method models class regions as surrounded by decision boundaries and projects images across them until misclassification.
  • Optimization-based attack: C&W minimizes an auxiliary variable using a logit-margin objective and supports various ℓp norms.Its objective compares the correct-class logit with the largest competing-class logit using margin parameter k.
  • Transferability: Iterative attacks generally achieve higher white-box success but transfer less well, while single-step attacks generally transfer better but have lower white-box success.DI2FGSM and MDI2FGSM introduce diverse inputs, with MDI2FGSM additionally using momentum to stabilize gradients.

C. Adversarial Defenses

Existing defenses either modify the classifier or transform input images to suppress adversarial perturbations. Transformation methods include compression, quilting, variation minimization, feature squeezing, and foveation, but their effectiveness varies.

  • Defense directions: Model-specific defenses regularize classifier parameters through adversarial training, parameter smoothing, or related retraining procedures.These methods modify the model rather than only transforming its input.
  • Input transformations: JPEG compression suppresses high-frequency components, but JPEG alone is reported to be far from an effective defense.Quilting and Total Variance Minimization were proposed as additional image transformations.
  • Input transformations: Feature squeezing limits the adversarial space through bit-depth reduction, smoothing filters, or reduced image resolution.Other transformation defenses include quilting, TVM, and foveation-based processing.

III. PROPOSED PERTURBED IMAGE RESTORATION

The proposed defense transforms perturbed inputs through wavelet denoising and super-resolution in a non-differentiable pipeline. Super-resolution maps adversarial images toward the high-resolution natural-image manifold and can restore their true labels.

  • Design objective: The transformation objective is C(xc) = C(T(xadv)) = yc, where T(.) transforms the adversarial input before classification.The desired transformation is model-agnostic and non-differentiable, making classifier-error backpropagation harder for an adversary.
  • Pipeline: The defense belongs to the input-transformation category and uses restoration techniques to purify perturbed images.Its two components are wavelet denoising followed by super-resolution, together forming a non-differentiable pipeline.
  • Pipeline: Wavelet denoising suppresses noise patterns before super-resolution enhances pixel resolution and removes adversarial patterns.The second step adds robustness because it is non-differentiable, while super-resolution alone can reinstate classifier beliefs toward correct categories.
  • Manifold mapping: Super-resolution maps off-manifold adversarial images into the same high-resolution manifold domain as clean natural images.The illustrated mapping is associated with recovering the corresponding true labels.

A. Super Resolution as a Defense Mechanism

The defense combines wavelet denoising with deep super-resolution to suppress adversarial perturbations while mapping attacked images toward the natural image manifold. It preserves or improves image quality and restores cleaner classifier features without retraining.

  • The method uses the manifold assumption that natural images lie on low-dimensional manifolds, while adversarial examples lie off them.
  • EDSR uses residual learning to focus on high-frequency details and generate high-quality super-resolved images from low-resolution inputs.
  • Super-resolution remaps adversarial images toward the natural image manifold, while the recovered image retains clean-like high-frequency details and discards most noisy perturbations.A sparse trace of noise remains, and salient image boundaries receive reinforced high-frequency responses.
  • Feature-map recovery reduces adversarial noise without model retraining or adversarial data augmentation.
  • The modular defense is attack- and model-agnostic, improves image quality, preserves clean-image performance, and can serve as a preprocessing step.
  • Wavelet denoising suppresses perturbations while retaining significant image features represented by large wavelet coefficients.

1) Thresholding:

The paper compares hard and soft wavelet thresholding and selects BayesShrink soft-thresholding to remove noise while preserving image information and visual quality.

  • Thresholding:: Thresholding controls how aggressively wavelet coefficients are shrunk to remove adversarial noise.
  • Thresholding:: Hard thresholding sets coefficients below t to zero and retains coefficients meeting or exceeding t.
  • Thresholding:: Soft thresholding modifies coefficients instead of fully retaining those larger than t, helping avoid abrupt changes and oversmoothing.
  • Thresholding:: Large thresholds oversmooth reconstructed images, whereas small thresholds allow noisy wavelets to pass through.
  • Thresholding:: BayesShrink selects different thresholds for each wavelet sub-band by modeling Gaussian noise, and it outperforms VisuShrink visually and empirically.

C. Algorithmic Description

The algorithm denoises an adversarial image in the wavelet domain, applies super-resolution as a manifold-mapping function, and forwards the recovered image to the original classifier.

  • The defense is an image-restoration pipeline combining wavelet denoising and super-resolution.
  • The algorithm represents the corrupted image as xadv = xc + ρ and outputs a denoised image x.
  • It converts RGB images to Y CbCr, applies a discrete wavelet transform, and removes noisy coefficients with BayesShrink soft-thresholding.
  • The shrunken coefficients are inverted with the inverse wavelet transform, after which the image is converted back to RGB.
  • The denoised image is super-resolved by a deep network that maps the adversarial sample toward the natural image manifold.
  • The recovered images are forwarded to the attacked model for prediction without changing the classifier.

IV. EXPERIMENTS

Experiments evaluate the defense across three ImageNet classifiers, multiple adversarial attacks, competing image-transformation defenses, and several single-image super-resolution methods.

  • The evaluation uses Inception-v3, ResNet-50, and InceptionResNet v-2 with ImageNet pretrained weights and no retraining or fine-tuning.
  • The experiments use 5000 images from the ILSVRC validation set, selected so each classifier achieves 100% accuracy.
  • The proposed defense consistently achieves superior performance across three models and various adversarial attacks.
  • Attacks include FGSM, I-FGSM, MI-FGSM, DeepFool, Carlini and Wagner, DI2FGSM, and MDI2FGSM.
  • The proposed defense is compared with JPEG Compression, Random Resizing and Padding, quilting with total variance minimization, and Pixel Deflection under matched images and attacks.
  • Interpolation methods are computationally efficient but less robust against stronger DI2FGSM and MDI2FGSM attacks.

A. Manifold Assumption Validation

The paper tests whether super-resolution remaps adversarial examples toward the natural-image manifold. A binary classifier labels about 91% of super-resolved images as clean, while experiments report strong recovery across attacks and improved white-box robustness with augmented adversarial training.

  • A. Manifold Assumption Validation: 91% of super-resolved images are labeled clean by a binary classifier, supporting the claim that restoration returns most samples to the natural image manifold.The classifier was fine-tuned on 10,000 clean–adversarial pairs and tested on defended images.
  • A. Manifold Assumption Validation: PCA visualization of binary-classifier features is used to examine whether defended images align with clean-image features.The visualization reduces features to three dimensions while retaining 90% of variance.
  • A. Manifold Assumption Validation: The defense recovers about 96% of images for iterative C&W and DeepFool attacks, compared with about 90% for Random Resizing + Padding and PD.Destruction rate is defined using exact top-1 label recovery, making the evaluation stricter than merely avoiding the adversarial label.
  • A. Manifold Assumption Validation: For MDI2FGSM, super-resolution recovers 31.3% of images, exceeding JPEG compression, Random Resizing + Padding, Quilting + TVM, and PD.The competing methods recover 1.3%, 5.8%, 1.7%, and 21.9%, respectively.
  • A. Manifold Assumption Validation: Adversarial training with clean, attacked, and super-resolved CIFAR-10 images enhances robustness against white-box attacks.The augmented training paradigm is evaluated against both single-step and strong iterative attacks.

D. Ablation Study

The ablations examine super-resolution choices, resizing baselines, wavelet denoising, scaling, transferability, and classifier attention. They support a defense pipeline in which wavelet denoising followed by super-resolution performs robustly across models and attack types, while simple resizing offers partial protection.

  • D. Ablation Study: Simple interpolation-based resizing provides some defense, although it performs worse than more sophisticated super-resolution techniques.The evaluated strategies include upsample, upsample plus downsample, and downsample plus upsample.
  • D. Ablation Study: Wavelet denoising followed by super-resolution achieves the strongest performance among the tested module orderings across Inception v-3, ResNet-50, and InceptionResNet v-2.The paper evaluates the individual contributions of both modules using top-1 accuracy.
  • D. Ablation Study: The selected super-resolution scale is S = 2 because higher scales introduce high-frequency components that degrade performance.The BayesShrink coefficient is selected as σρ = 0.04.
  • D. Ablation Study: The defense is evaluated for cross-model transferability by generating PGD examples on a source model, restoring them, and testing them on a target model.The corresponding table reports robustness under black-box settings.
  • D. Ablation Study: Class Activation Maps show that super-resolution can redirect classifier attention from noisy patches toward more distinctive image regions.The paper links this behavior to recovery of discriminative regions associated with correct labels.
  • D. Ablation Study: Super-resolution can alter predictions on clean images, sometimes causing misclassification and sometimes correcting an existing error.These cases predominantly occur when the top-two class confidences are roughly equal.

E. Identifying Obfuscated Gradients

The paper tests whether its non-differentiable defense relies on obfuscated gradients by evaluating stronger white-box attacks and diagnostic comparisons. BPDA and EOT do not substantially break the defense, but robustness is lower in white-box than gray-box settings as expected when gradients are not masked.

  • E. Identifying Obfuscated Gradients: Feature visualizations show adversarial images off the natural image manifold and defended images remapped toward clean-image features.The displayed features are projected into three dimensions using PCA.
  • E. Identifying Obfuscated Gradients: The defense is examined against iterative attacks, which are more successful than single-step attacks on undefended models.The comparison includes I-FGSM and MI-FGSM versus FGSM.
  • E. Identifying Obfuscated Gradients: White-box robustness is lower than gray-box robustness because the white-box adversary has complete knowledge of the defense model and denoising process.The paper presents this pattern as evidence against obfuscated gradients.
  • E. Identifying Obfuscated Gradients: BPDA and EOT attacks fail to substantially break the combined wavelet-denoising and super-resolution defense.With EOT, accuracy drops by 8.9% for a strong attack.

V. CONCLUSION

The proposed image-restoration defense maps adversarial samples back to the natural image manifold without degrading unperturbed-image performance. It is agnostic to the underlying model and attack type, and outperformed competing defense schemes.

  • The super-resolution defense maps off-the-manifold adversarial samples back to the natural image manifold.
  • High-frequency information added by super-resolution is identified as the primary reason the defense negates adversarial noise.
  • The defense is agnostic to the underlying model and attack type, requires no learning, and operates for black- and white-box attacks.
  • The defense outperformed competing state-of-the-art schemes by a considerable margin.
  • Figure 8 compares clean, attacked, and recovered class activation maps alongside magnified perturbations and clean–defended differences.
Loading 1901.01677v2…