Source-linked AI summary
Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distance
Zhengyu Zhao, Zhuoran Liu, Martha Larson
TL;DR
Image attacks must balance classifier fooling ability with visual imperceptibility, but tight RGB-space Lp bounds are not well aligned with human color perception. The paper instead optimizes perceptual color distance through PerC-C&W and PerC-AL, producing larger RGB perturbations that retain adversarial strength while improving imperceptibility and robustness-related outcomes. It also finds added value when perceptual color distance is combined with structural approaches.
Problem
Tight RGB-space Lp bounds are commonly used to model imperceptibility, despite the poor alignment between RGB distance and human perception.
Method
The paper introduces PerC-C&W and PerC-AL, optimizing adversarial perturbations with perceptual color distance, including CIEDE2000, instead of relying only on RGB-space Lp distance.
Results
PerC perturbations have larger RGB Lp norms while maintaining adversarial strength and improving imperceptibility, robustness, and transferability, with added benefits alongside structural methods.
Takeaways & Limitations
Perceptual color distance can hide large RGB perturbations, especially in smooth saturated regions and high-confidence settings, while complementing structural approaches.
Takeaways & Limitations
Neither conventional RGB nor PerC perturbations perform well in smooth regions with low saturation, motivating techniques targeted to those regions.
Abstract
from arXiv · showhide
The success of image perturbations that are designed to fool image classifier is assessed in terms of both adversarial effect and visual imperceptibility. The conventional assumption on imperceptibility is that perturbations should strive for tight $L_p$-norm bounds in RGB space. In this work, we drop this assumption by pursuing an approach that exploits human color perception, and more specifically, minimizing perturbation size with respect to perceptual color distance. Our first approach, Perceptual Color distance C&W (PerC-C&W), extends the widely-used C&W approach and produces larger RGB perturbations. PerC-C&W is able to maintain adversarial strength, while contributing to imperceptibility. Our second approach, Perceptual Color distance Alternating Loss (PerC-AL), achieves the same outcome, but does so more efficiently by alternating between the classification loss and perceptual color difference when updating perturbations. Experimental evaluation shows PerC approaches outperform conventional $L_p$ approaches in terms of robustness and transferability, and also demonstrates that the PerC distance can provide added value on top of existing structure-based methods to creating image perturbations.
1. Introduction
The paper challenges tight RGB-space Lp bounds as a sufficient model of imperceptibility and instead uses perceptual color distance to hide larger RGB perturbations. PerC methods preserve adversarial effect while improving imperceptibility, particularly for high-confidence attacks and in combination with structural approaches.
- Tight RGB-space Lp bounds are commonly treated as guarantees of imperceptibility, although larger perturbations generally improve adversarial strength.
- PerC distance replaces direct RGB-space optimization, allowing larger RGB perturbations that remain less perceptible to human observers.PerC perturbations are strategically distributed across RGB channels and are especially less perceptible in smooth, saturated regions.
- Identical RGB noise can produce different perceptual outcomes across channels: the green-channel change is visible, whereas the blue-channel change is imperceptible.The example uses a 20 × 20 image patch and identical noise sampled from [-15,15].
- PerC approaches become particularly important for high-confidence adversarial examples, where misclassification confidence is increased.
- PerC distance has an additive benefit with structural approaches, further improving imperceptibility.
- The paper introduces PerC-C&W and PerC-AL, with the latter alternating classification loss and perceptual color difference during perturbation updates.
2. Background on Perceptual Color Distance
The paper uses CIEDE2000 to measure perceptual color distance because RGB-space differences and simpler color-space distances do not align closely enough with human perception. Unlike prior uses for evaluation, this work optimizes CIEDE2000 directly through back propagation.
- CIEDE2000 is the CIE’s latest ΔE standard formula and was designed to better align perceptual color distance with human visual perception.It refines earlier standards with five corrections.
- CIEDE2000 measures pixel-wise differences using lightness, chroma, and hue in CIELCH, plus an interactive chroma–hue term.Weighting functions derived from human studies compensate the components to better simulate perception.
- An Lp norm in CIELAB can measure color distance, but it is less closely aligned with human perceptual distance than CIEDE2000.
- This paper uses CIEDE2000 directly for optimization with back propagation rather than only for evaluating image-pair similarity.
3. Related work
Prior work largely constrains adversarial perturbations with RGB-space Lp norms, while newer approaches explore semantic and structural similarity but face limits in adversarial strength or smooth regions. This paper motivates questioning RGB-distance imperceptibility and pursuing perceptual color distance.
- Adversarial examples modify an image to induce untargeted or targeted misclassification, with the untargeted setting generally less challenging.
- Lp norm-bounded adversarial examples: Lp-based methods constrain perturbations around the original image, with C&W combining classification loss and an L2 penalty and using logit differences.C&W removes the box constraint through a tanh-based variable and controls misclassification confidence with κ.
- Lp norm-bounded adversarial examples: Penalty-based optimization can be time-consuming because line search must find an image-dependent optimal λ, motivating projection-based alternatives such as FGSM and I-FGSM.FGSM uses one step, while I-FGSM iteratively updates perturbations with a smaller step size α.
- Lp norm-bounded adversarial examples: I-FGSM constrains only each coordinate’s maximum change, ignoring accumulated image-level difference and yielding poor imperceptibility in high-confidence settings.
- Adversarial examples beyond Lp norms: Semantic and structural approaches move beyond RGB norms, but transformation-based methods have restricted search spaces and structure-aware methods struggle with smooth image regions.The cited structural approaches hide perturbations in visual variation or allow more changes in high-variance regions.
4. Proposed approaches
The proposed PerC methods optimize adversarial perturbations using accumulated perceptual color differences rather than conventional RGB-space penalties. PerC-C&W preserves joint optimization with a perceptual penalty, while PerC-AL alternates classification and perceptual updates to avoid costly joint optimization.
- PerC methods minimize the image-level L2 norm of a vector whose components are per-pixel perceptual color differences ΔE00.
- PerC-C&W: PerC-C&W replaces C&W’s RGB L2 penalty with a perceptual color-difference penalty while retaining joint optimization and binary search over λ.Perceptual-color gradients translate updates into a perceptually uniform color space, allowing large RGB perturbations to remain hidden.
- PerC-AL: PerC-AL is proposed to address PerC-C&W’s inefficiency caused by time-consuming binary search for an optimal λ that varies substantially across images.
- PerC-AL: PerC-AL decouples joint optimization by alternating updates that enlarge perturbations using classification-loss gradients or shrink them using perceptual-color gradients.The alternation depends on whether the intermediate perturbed image is adversarial, repeatedly crossing the decision boundary to search for a smaller perceptual difference.
5. Experiments
The experiments compare perceptual-color and RGB-based adversarial approaches across imperceptibility, robustness, and transferability. PerC methods reduce perceptual color difference while allowing larger RGB perturbations, and they consistently perform strongly under high-confidence and transfer settings.
- Experimental setup: Experiments use 1000 ImageNet-Compatible development images and compare methods under targeted and high-confidence settings.The evaluation includes white-box robustness, transformation-based defenses, and transferability from Inception V3.
- Experimental setup: Success rate, RGB L2 and L∞ norms, and accumulated perceptual color difference C2 jointly measure adversarial strength and imperceptibility.The protocol reports perturbation size over successful images and compares relative behavior rather than absolute optimization quality.
- Sufficient-confidence adversarial examples: PerC-C&W and PerC-AL reduce C2 without tightly constraining RGB Lp norms; PerC-AL reaches lower C2 than PerC-C&W, 57.10 vs. 67.79, with fewer iterations.The targeted results show the intended separation between perceptual color distance and RGB perturbation magnitude.
- High-confidence adversarial examples: At high confidence, PerC examples are visually more acceptable and have low C2, although success rates are no longer perfect across all conditions.The high-confidence evaluation uses κ = 20 and κ = 40, with examples shown at κ = 40.
- Robustness: Under JPEG compression and bit-depth reduction, increasing κ from 20 to 40 improves robustness, while PerC-C&W consistently outperforms C&W and PerC-AL outperforms DDN.Among approaches targeting minimal image-level accumulated difference, PerC methods achieve the best results while causing fewer visual distortions.
- Transferability: In transfer experiments across three target models, one of the two PerC approaches always achieves the best result among the non-I-FGSM methods.The comparison uses 767 images correctly classified by all four models.
- Assembling structural information: Adding structural information suppresses perturbations in smooth regions and places barely perceptible changes in areas with richer visual variation.The paper identifies this combination with PerC-based perturbations as a direction for further investigation.
6. Conclusion and Outlook
The paper concludes that perceptual color distance enables large yet imperceptible adversarial perturbations, while identifying limitations and directions for future work.
- PerC-C&W and PerC-AL use perceptual color distance to create large yet imperceptible adversarial image perturbations.The approaches improve imperceptibility especially in smooth, saturated regions.
- PerC perturbations have larger RGB Lp norms than direct RGB approaches, and this translates into adversarial strength.
- Perceptual color perturbations can poison training data without being noticed by human annotators.
- Outlook: Neither conventional RGB nor PerC perturbations perform well in smooth, low-saturation regions.Future work also includes robustness training and detection or mitigation methods for PerC-based perturbations.
Appendix
The appendix reports initialization and evaluation settings for PerC-AL, including grid-searched λ values and targeted ImageNet-Compatible experiments with varied budgets.
- Table 4 lists selected initializations of λ obtained through grid search.
- Table 5 evaluates targeted success rates and RGB-space L2, L∞, and image-level accumulated perceptual color difference C2 across varied budgets.The experiments use 1000 ImageNet-Compatible images, with PerC-AL initialized at αc = 0.1 for the relatively easy untargeted case.