Source-linked AI summary
Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition
Bangjie Yin, Wenxuan Wang, Taiping Yao, Junfeng Guo, Zelun Kong, Shouhong Ding, Jilin Li, Cong Liu
TL;DR
Face-recognition adversarial examples remain limited by poor black-box transferability and practical implementation constraints. Adv-Makeup generates blended orbital eye shadow and uses fine-grained meta-learning to address these limitations. Experiments report more imperceptible attacks, improved transferability, and efficacy against commercial platforms.
Problem
Existing face-recognition attacks lack black-box transferability, are physically impractical, or are visually noticeable.
Method
Adv-Makeup synthesizes and blends natural orbital eye shadow, then applies task-driven fine-grained meta-learning to improve adversarial transferability.
Results
Experiments on makeup and LFW datasets, plus a case study against two commercial platforms, demonstrate imperceptible and transferable adversarial examples.
Takeaways & Limitations
Adv-Makeup provides a face-recognition attack designed to operate imperceptibly and transfer across black-box models in digital and physical settings.
Takeaways & Limitations
Existing methods remain constrained by physical impracticality, visual detectability, or limited transferability, motivating Adv-Makeup's scope.
Abstract
from arXiv · showhide
Deep neural networks, particularly face recognition models, have been shown to be vulnerable to both digital and physical adversarial examples. However, existing adversarial examples against face recognition systems either lack transferability to black-box models, or fail to be implemented in practice. In this paper, we propose a unified adversarial face generation method - Adv-Makeup, which can realize imperceptible and transferable attack under black-box setting. Adv-Makeup develops a task-driven makeup generation method with the blending module to synthesize imperceptible eye shadow over the orbital region on faces. And to achieve transferability, Adv-Makeup implements a fine-grained meta-learning adversarial attack strategy to learn more general attack features from various models. Compared to existing techniques, sufficient visualization results demonstrate that Adv-Makeup is capable to generate much more imperceptible attacks under both digital and physical scenarios. Meanwhile, extensive quantitative experiments show that Adv-Makeup can significantly improve the attack success rate under black-box setting, even attacking commercial systems.
1 Introduction
Existing face-recognition attacks are limited by physical impracticality, visual detectability, and weak black-box transferability. Adv-Makeup addresses these challenges by generating natural eye shadow with blending and a fine-grained attack strategy.
- Existing limitations: Existing face-recognition attacks often use global perturbations that cannot be controlled in realistic physical applications.Such methods focus on digital settings and may alter areas, such as backgrounds, that attackers cannot physically manipulate.
- Existing limitations: Physically implementable attacks such as eyeglasses, hats, and stickers can be visually obvious and easily detected.
- Existing limitations: Existing attacks also struggle to transfer effectively to black-box face-recognition models.The introduction identifies transferability as a critical limitation alongside physical practicality and imperceptibility.
- Adv-Makeup: Adv-Makeup synthesizes natural eye shadow over the orbital region and blends it with the source face to produce visually unnoticeable attacks.The method targets a practically implementable makeup scenario while reducing style and content differences between generated and source regions.
- Adv-Makeup: Adv-Makeup combines makeup generation and blending with a task-driven fine-grained meta-learning strategy for imperceptible, transferable attacks.The stated technique is designed for face-recognition models under both digital and physical scenarios.
2 Related Work
Prior adversarial-attack research includes white-box gradient methods and black-box transfer attacks, but transferability remains difficult. Existing approaches also perturb entire inputs, limiting real-world practicality.
- Adversarial Attacks: White-box attacks use gradients while requiring full access to the target model, an unrealistic condition in many applications.
- Adversarial Attacks: Black-box attacks restrict adversaries to model outputs, making transferability a central challenge.The passage states that black-box attack success rates still decrease substantially despite efforts to improve transferability.
- Adversarial Attacks on Face Recognition: Existing adversarial methods commonly perturb the entire input, which is impractical for real-world physical attacks.Adv-Makeup instead focuses on an implementable physical attack with high transferability.
3 Methodology
Adv-Makeup combines realistic eye-shadow generation, blending, and task-driven meta-learning to produce imperceptible, transferable impersonation attacks across face-recognition models.
- 3 Methodology: Adv-Makeup synthesizes realistic eye shadow over the orbital regions and blends it to reduce visible style, content, and boundary differences.The framework uses makeup generation followed by blending for imperceptible face editing.
- 3.2 Makeup Generation: A generator takes aligned source orbital regions as input, produces synthesized eye shadow, and attaches the resulting region to the source face.Face alignment and eye-surrounding landmarks provide the orbital regions and mask used in generation.
- 3.2 Makeup Generation: A discriminator compares real cosmetic orbit regions with generated eye-shadow regions to encourage perceptually natural outputs.The generator and discriminator form a two-player minimax setup for visually natural makeup generation.
- 3.3 Makeup Blending: Makeup blending uses gradient, style, and content constraints to smooth boundaries, match source-image color, preserve details, and integrate synthesized eye shadow.A pre-trained VGG16 model supplies style and content losses, with layer weights balancing their contributions.
- 3.4 Makeup Attack: The attack optimizes impersonation against face-recognition feature extractors while meta-training across models and meta-testing on a remaining victim model.The shared generator collects gradient information from multiple models to improve generalization to gradient-unknown models.
4 Experiment
Experiments evaluate Adv-Makeup on digital and physical impersonation attacks using public datasets, four models, and two commercial black-box systems. The method achieves strong transferable performance, imperceptibility, physical attack strength, and robustness to pose changes.
- Digital-attack: Commercial-system visualizations show attacks under changes in illumination, pose, gender, and eye-makeup intensity, with confidence results reported for Face++ and Microsoft.Figure 4 compares generated faces against other face attack methods on two black-box commercial platforms.
- Experimental Setup: Experiments use LFW and a high-quality makeup face database, reporting average results across 1000 identity pairs.The digital setting uses 100 source before-makeup faces and 10 targets for 1000 comparisons; the makeup database contributes 198 eye-makeup images.
- Experimental Setup: Attack success rate measures the proportion of impersonation comparisons whose similarity exceeds each victim model’s threshold at 0.01 FAR.The experiments also report commercial-system confidence scores for Face++ and Microsoft.
- Digital-attack: Adv-Makeup achieves the best transferable results on all black-box victim models, significantly outperforming seven competing attack methods.Digital comparisons include general attacks, face-specific attacks, and Adv-Makeup with ensemble training without meta-learning.
- Physical-attack: Adv-Makeup achieves substantially higher physical attack strength than Adv-Hat and Adv-Glasses across confidence-score thresholds on two commercial platforms.Physical evaluation uses tattoo-pasted eye shadow, 20 balanced personators, and 20 randomly selected targets.
- Physical-attack: Adv-Makeup produces the smallest changing area, the most natural and inconspicuous faces, and the most stable attack ability under large pose changes.These findings are reported for physical attacks against commercial face-recognition platforms.
5 Conclusion
Adv-Makeup generates natural, seamless eye-shadow attacks for face recognition that are imperceptible and transferable under black-box models. Results across visualization, LFW and makeup datasets, and two commercial platforms support its feasibility in digital and physical settings.
- Adv-Makeup generates natural and seamless eye shadow over the orbital region for transferable black-box attacks.
- Visualization results show that Adv-Makeup produces more imperceptible adversarial examples than competing methods.
- Experiments on LFW and makeup datasets demonstrate transferability from the task-driven fine-grained meta-learning adversarial attack strategy.
- Case studies targeting two commercial face-recognition platforms verify Adv-Makeup’s feasibility and efficacy.