Source-linked AI summary
Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples
Zihao Liu, Qi Liu, Tao Liu, Nuo Xu, Xue Lin, Yanzhi Wang, Wujie Wen
TL;DR
JPEG defenses must remove adversarial perturbations without sacrificing benign-image accuracy, a balance that standard compression and prior input transformations do not guarantee. Feature distillation redesigns JPEG quantization through frequency-domain defensive filtering and DNN-oriented refinement, achieving strong adversarial-example accuracy with minimal benign-data degradation and faster processing. Against adaptive BPDA attacks, it remains effective but is not completely immune.
Problem
Prior compression-based defenses tune compression parameters to reduce image features without guaranteeing both adversarial defense efficiency and benign-image classification accuracy.
Method
Feature distillation redesigns JPEG with semi-analytical frequency-domain defensive quantization followed by DNN-oriented quantization refinement.
Results
Feature distillation improves defense efficiency from ∼20% to ∼90% over recent model-agnostic approaches with ≤1% benign-image accuracy degradation.
Takeaways & Limitations
The method balances adversarial-feature removal and benign-feature preservation while outperforming recent input-transformation defenses in defense efficiency, accuracy, and processing speed.
Abstract
from arXiv · showhide
Image compression-based approaches for defending against the adversarial-example attacks, which threaten the safety use of deep neural networks (DNN), have been investigated recently. However, prior works mainly rely on directly tuning parameters like compression rate, to blindly reduce image features, thereby lacking guarantee on both defense efficiency (i.e. accuracy of polluted images) and classification accuracy of benign images, after applying defense methods. To overcome these limitations, we propose a JPEG-based defensive compression framework, namely "feature distillation", to effectively rectify adversarial examples without impacting classification accuracy on benign data. Our framework significantly escalates the defense efficiency with marginal accuracy reduction using a two-step method: First, we maximize malicious features filtering of adversarial input perturbations by developing defensive quantization in frequency domain of JPEG compression or decompression, guided by a semi-analytical method; Second, we suppress the distortions of benign features to restore classification accuracy through a DNN-oriented quantization refine process. Our experimental results show that proposed "feature distillation" can significantly surpass the latest input-transformation based mitigations such as Quilting and TV Minimization in three aspects, including defense efficiency (improve classification accuracy from $\sim20\%$ to $\sim90\%$ on adversarial examples), accuracy of benign images after defense ($\le1\%$ accuracy degradation), and processing time per image ($\sim259\times$ Speedup). Moreover, our solution can also provide the best defense efficiency ($\sim60\%$ accuracy) against the recent adaptive attack with least accuracy reduction ($\sim1\%$) on benign images when compared with other input-transformation based defense methods.
1. Introduction
The paper targets JPEG-based defenses that must remove adversarial perturbations while preserving benign-image accuracy. It proposes feature distillation, a DNN-favorable JPEG redesign guided by frequency analysis and quantization refinement.
- Motivation: DNNs are vulnerable to adversarial examples formed by small, often imperceptible perturbations that can cause misreading in safety-critical systems.Such failures include incorrect road-sign recognition in autonomous-vehicle settings.
- Prior defenses: Existing defenses include model-specific hardening and model-agnostic input transformations such as dimensionality reduction and JPEG compression.These approaches attempt to remove adversarial perturbations through retraining, gradient masking, or input modification.
- Research gap: Standard JPEG cannot effectively remove adversarial perturbations while guaranteeing benign-image classification accuracy because it prioritizes human visual quality.The proposed redesign instead targets DNN-relevant features during compression.
- Approach: Feature distillation redesigns JPEG quantization using frequency-distribution analysis, semi-analytical defensive quantization, and DNN-oriented refinement.The refinement compensates for benign-feature loss caused by perturbation removal.
- Results: >90% accuracy on adversarial examples and ≤1% benign-data accuracy reduction are reported for feature distillation.The method also outperforms Quilting and TV Minimization in defense efficiency, accuracy, and processing speed.
2. Background, Related Work and Motivation
The background defines adversarial examples, JPEG compression, and representative attacks, then motivates feature distillation by showing the limits of standard JPEG and related input transformations. The central challenge is balancing perturbation removal against DNN-relevant feature preservation.
- Adversarial examples: Adversarial examples add perturbations to normal inputs to induce incorrect DNN predictions, with attacks including FGSM, BIM, Deepfool, C&W variants, and BPDA.BPDA repeatedly computes gradients through the defense while assuming knowledge of the defense method.
- JPEG compression: JPEG compresses images through DCT-based frequency processing, quantization, reordering, and entropy coding, scaling high-frequency coefficients more than low-frequency coefficients.This design reflects the human visual system’s lower sensitivity to high frequencies.
- Related work: Prior studies found JPEG can reverse only small adversarial perturbations, while Quilting and total-variation minimization often showed better efficiency against gray-box and black-box attacks.These findings motivated redesign rather than direct parameter tuning.
- Proposed direction: Feature distillation addresses this trade-off through one-pass or two-pass quantization and decompression processes designed for adversarial-feature filtering and benign-feature preservation.The two modes are illustrated as alternative placements of the added quantization/dequantization operations.
- Motivation: Standard JPEG can reduce adversarial attack success only slightly while substantially degrading testing accuracy as compression becomes more aggressive.The paper attributes this trade-off to losing important high-frequency features learned by DNNs.
3. Our Approach–Feature Distillation
Feature distillation redesigns JPEG quantization to filter adversarial perturbations in the frequency domain while preserving benign-image accuracy. It combines defensive quantization with DNN-oriented refinement based on frequency-component importance.
- Overall framework: The two-pass method combines defensive quantization with DNN-oriented refinement to address attack efficiency and test accuracy simultaneously.The one-pass process handles standard JPEG inputs, while the two-pass process targets raw sensor data and applies defensive compression before the one-pass process.
- Step 1: Defensive Quantization: DCT decomposes an adversarial input into benign and perturbation coefficients, enabling JPEG quantization to filter perturbations in the frequency domain.For an 8 × 8 block, DCT(X + δX) separates into CX and CδX components, and the perturbation coefficients are bounded by −8·ϵ and 8·ϵ.
- Step 1: Defensive Quantization: A defensive quantization step larger than the perturbation coefficient can eliminate it, but rounding errors may exceed the adversarial perturbation and degrade defense efficiency.The rounding issue arises because CδX is usually much smaller than CX; if QS/2 + CδX exceeds QS, stronger rounding error can result.
- Step 2: Quantization Refinement: DNN-oriented refinement identifies frequency components important to classification and lowers their quantization steps to reduce benign-feature distortion.The design uses frequency analysis on benign images, with larger DCT-coefficient standard deviations indicating greater energy and more important features for DNN learning.
- Step 2: Quantization Refinement: The refinement divides the 64 frequency components into Accuracy Sensitive and Malicious Defense bands using a threshold on DCT-coefficient standard deviations.The Accuracy Sensitive band uses QS = S2 and the Malicious Defense band uses QS = S1, with S1 > S2.
4. Evaluation
The evaluation measures feature distillation across adversarial attacks and defense settings, balancing adversarial-example accuracy against benign-image accuracy and processing time. Two-pass feature distillation generally provides the strongest defense, including against black-box and adaptive attacks.
- Optimized quantization step: Two-pass defense improves as quantization strength grows, while excessive quantization can saturate or reduce one-pass performance.The two-pass process generally outperforms one-pass processing because it eliminates rounding error; CW0 requires stronger quantization.
- Optimized quantization step: S1 = 30, S2 = 20, and n = 15 are selected for overall evaluation after quantization-parameter exploration.The selected settings balance defensive quantization across frequency bands while preserving acceptable visual quality.
- Comparison with input transformations: 0.15s per image enables approximately 216× and 259× speedups over Quilting and TVM, respectively.FD-1P and FD-2P report 56% and 91% average defense efficiency, while both achieve 68.5% benign-image accuracy.
- Adaptive attack mitigation: 10% to 60% adversarial-example accuracy is achieved against BPDA with merely 1%-3% benign-accuracy reduction.The result is reported for FD-1× through FD-3× quantization settings.
- Adaptive attack mitigation: Bit-depth, Quilting, and TVM provide no defense against BPDA, whereas JPEG reaches 45% but reduces benign accuracy by 17%.Feature distillation is reported as the strongest BPDA defense with the smallest benign-accuracy reduction.
5. Conclusion
Feature distillation rearchitects JPEG compression to reduce adversarial attack success while preserving legitimate-image accuracy. Experiments report approximately 90% defense efficiency, at most 1% benign-accuracy degradation, and approximately 260× faster processing than recent model-agnostic defenses.
- Conclusion: ∼20% to ∼90% defense efficiency improvement is reported over recent model-agnostic approaches with ≤1% accuracy degradation.The conclusion summarizes the method’s simultaneous adversarial-example and legitimate-image performance.
- Conclusion: ∼260× processing speedup is reported per image compared with recent input-transformation defenses.The conclusion identifies processing speed as a third evaluation advantage alongside defense efficiency and accuracy.
- Conclusion: ∼60% defense efficiency against BPDA is reported with ∼1% benign-image accuracy drop.The conclusion presents this as the strongest adaptive-attack result among compared input-transformation defenses.