Source-linked AI summary
Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, Jun Zhu
TL;DR
Face-recognition CNNs are vulnerable, but evaluating them realistically is difficult when attackers lack model details and receive only hard-label outputs. The paper introduces an evolutionary decision-based black-box attack that models local search geometry and reduces search dimensionality. Experiments show effective, low-perturbation attacks with fewer queries, including against a real-world system.
Problem
The paper addresses inefficient decision-based black-box attacks, which must assess face-recognition robustness using limited queries without model parameters, gradients, or predicted probabilities.
Method
The proposed evolutionary attack models local geometries of search directions and reduces the search-space dimension while optimizing solely through model queries.
Results
Extensive experiments demonstrate the method's effectiveness across state-of-the-art face-recognition models and datasets, with successful attacks on a real-world system.
Takeaways & Limitations
Existing face-recognition models are extremely vulnerable to black-box adversarial attacks, raising security concerns for developing more robust systems.
Takeaways & Limitations
For simplicity, the model's dependency on the additional compared face image or gallery set is not explicitly described.
Abstract
from arXiv · showhide
Face recognition has obtained remarkable progress in recent years due to the great improvement of deep convolutional neural networks (CNNs). However, deep CNNs are vulnerable to adversarial examples, which can cause fateful consequences in real-world face recognition applications with security-sensitive purposes. Adversarial attacks are widely studied as they can identify the vulnerability of the models before they are deployed. In this paper, we evaluate the robustness of state-of-the-art face recognition models in the decision-based black-box attack setting, where the attackers have no access to the model parameters and gradients, but can only acquire hard-label predictions by sending queries to the target model. This attack setting is more practical in real-world face recognition systems. To improve the efficiency of previous methods, we propose an evolutionary attack algorithm, which can model the local geometries of the search directions and reduce the dimension of the search space. Extensive experiments demonstrate the effectiveness of the proposed method that induces a minimum perturbation to an input face image with fewer queries. We also apply the proposed method to attack a real-world face recognition system successfully.
1. Introduction
Deep CNNs have advanced face recognition but remain vulnerable to adversarial examples, motivating attacks that assess robustness under realistic hard-label-only access. The paper proposes a query-efficient evolutionary attack and evaluates it on leading models and a real-world system.
- Motivation: Deep CNN-based face recognition supports verification, identification, and security-sensitive applications but remains vulnerable to visually imperceptible adversarial perturbations.Reported attack effects include evading recognition or impersonating another individual with perturbations applied to eyeglasses.
- Attack Setting: Decision-based black-box attacks use only queries and hard-label predictions, making them more realistic than white-box attacks but unable to directly use gradients or predicted probabilities.Their goal is to generate adversarial examples with minimum perturbations under limited queries.
- Research Gap: Existing decision-based methods often require many queries to converge or produce relatively large perturbations under limited query budgets.The paper therefore targets smaller perturbations with fewer queries.
- Proposed Approach: The proposed evolutionary attack models local geometries of search directions and reduces search-space dimensionality to improve query efficiency.The method is presented as generally applicable to image recognition tasks.
- Evaluation: Experiments evaluate SphereFace, CosFace, and ArcFace on LFW and MegaFace, and demonstrate successful attacks against a real-world face recognition system.The paper uses these evaluations to study robustness in the decision-based black-box setting and practical applicability.
2. Related Work
Related work distinguishes face-recognition architectures and adversarial attack settings. Prior face attacks mainly use white-box access, whereas decision-based methods rely only on hard-label predictions.
- Deep Face Recognition: Deep face-recognition methods use CNNs with objectives including softmax, triplet, center, angular softmax, and large-margin cosine losses.These approaches shape feature or class margins for recognition.
- Adversarial Attacks on Face Recognition: Prior attacks on face recognition include eyeglass-constrained gradient methods and generative-network methods, but rely on white-box model manipulation.The paper identifies this access requirement as unrealistic for many real-world applications.
- Black-Box Attacks: Black-box attacks comprise transfer-based, score-based, and decision-based settings, with decision-based attacks providing only hard-label predictions.The related methods include boundary attacks based on random walks over the decision boundary.
3. Methodology
The paper formulates face-recognition attacks as hard-label black-box optimization and proposes an evolutionary method that reduces query cost by modeling search geometry and lowering search dimension.
- 3.1. Attack Setting: The attack seeks a nearby adversarial face image that satisfies a task-specific misclassification criterion while minimizing L2 perturbation.The criterion is binary because the target model exposes only hard-label predictions.
- 3.1. Attack Setting: Dodging makes verification reject a same-identity pair or makes identification recognize an image incorrectly.The paper frames dodging as a potential privacy-protection attack against excessive surveillance.
- 3.1. Attack Setting: Impersonation instead targets recognition as a specified identity, including matching another image in verification or a target class in identification.The supplied passage defines the verification criterion as producing the same-identity label.
- 3.2. Evolutionary Attack: The proposed evolutionary attack optimizes solely through queries, models local geometries of search directions, and reduces the search space dimension.It is based on a (1+1)-CMA-ES variant adapted for high-dimensional face images.
- 3.2. Evolutionary Attack: Each iteration samples correlated noise in a lower-dimensional R^m space, selects k coordinates, upsamples by bilinear interpolation, adds a distance-reducing bias, and accepts improvements.The covariance matrix is diagonal and represents local search-direction geometry; only successful updates modify the solution and covariance.
- 3.2.2 Mean of Gaussian Distribution: A zero-mean Gaussian proposal needs O(n^2) samples for one successful update in high dimensions, motivating the bias µ(x − x̃*) toward the original image.The bias addresses near-orthogonality between random directions and the direction that reduces image distance.
4. Experiments
Experiments evaluate decision-based black-box attacks on three face-recognition models across LFW verification, identification, ablations, and a real-world API. The evolutionary method uses local search geometry and dimensionality reduction to obtain effective, low-distortion attacks.
- Experimental Settings: Experiments compare Boundary, Optimization, NES-LO, and Evolutionary attacks against SphereFace, CosFace, and ArcFace on LFW.Verification and identification are evaluated using average MSE distortion over selected images and varying query budgets.
- Experimental Results: Figure 2 and Figure 3 track average adversarial-image MSE against query counts for verification and identification, respectively.Both figures cover dodging and impersonation-related evaluations across the three target models.
- Experimental Results: About 1e−5 distortion is sufficient to fool the evaluated face-recognition models in the black-box setting, remaining visually imperceptible to humans.Figure 4 illustrates distortion decreasing from random-noise or target-image initialization during dodging and impersonation attacks on ArcFace.
- Ablation Study: CMA improves results over a fixed-covariance baseline, while stochastic coordinate selection improves results and covariance-proportional sampling outperforms equal-probability sampling.These comparisons use average distortion after 10,000 queries in the ablation study.
- Ablation Study: Lower-dimensional search spaces converge faster, but overly small spaces produce larger distortions; the experiments therefore use m = 45×45×3.The dimensionality study compares 15×15×3 through 112×112×3 for dodging and impersonation attacks.
- Attacks on a Real-World Application: On Tencent’s face-verification API, the evolutionary method achieves smaller average distortion than Boundary and Optimization on 10 LFW impersonation pairs.NES-LO fails to generate an adversarial image within the 10,000-query limit, while the proposed method’s examples are reported as more visually similar to originals.
5. Conclusion
The paper proposes an evolutionary decision-based black-box attack that models local search-direction geometry and reduces search-space dimensionality. Experiments find effective low-distortion attacks against state-of-the-art face-recognition models and a real-world system, underscoring their vulnerability.
- Conclusion: The evolutionary method improves attack efficiency by modeling local search-direction geometries and reducing search-space dimensionality.The method is designed for decision-based black-box attacks.
- Conclusion: Experiments consistently demonstrate the method’s effectiveness against several state-of-the-art face-recognition models.The evaluation compares the proposed method with other attacks across multiple settings.
- Conclusion: The evaluated face-recognition models are extremely vulnerable to black-box adversarial attacks, raising security concerns for more robust systems.This conclusion is based on the paper’s comprehensive robustness study.
- Conclusion: The method successfully attacks a real-world face-recognition system, demonstrating practical applicability.The conclusion presents the real-world attack as the paper’s final practical evaluation.
A. Proof
The proof analyzes Gaussian search perturbations generated from a positive-definite covariance matrix by eigendecomposing that matrix and applying probability bounds.
- Positive definiteness ensures the covariance matrix's smallest eigenvalue satisfies λmin > 0.
- The covariance matrix is decomposed using an orthogonal matrix A and a diagonal matrix B with bii = √λi.
- The proof samples z′ ∼ N(0, I) and transforms it as z = σABz′, yielding z ∼ N(0, σ2C).
- As n grows, the squared norm ∥z′∥2 converges almost surely to n by the law of large numbers.
- The proof computes the mean and variance of y = (x̃∗−x) · (ABz′) before applying Chebyshev's inequality.
B. Results on MegeFace
On MegaFace, the proposed Evolutionary attack outperforms Boundary, Optimization, and NES-LO across face verification and identification settings.
- The proposed method outperforms Boundary, Optimization, and NES-LO in all MegaFace face verification and identification settings.Experiments cover SphereFace, CosFace, and ArcFace, with distortion tracked across query budgets.
- Distortion curves are reported over queries, with additional values at 1,000, 5,000, 10,000, and 100,000 queries.
- The MegaFace results are consistent with those obtained on the LFW dataset.
C. Results on ImageNet
ImageNet experiments show that Evolutionary remains effective beyond face recognition, across untargeted and targeted attacks on two classification networks.
- Evolutionary consistently demonstrates effectiveness on untargeted and targeted attacks against Inception v3 and ResNet 50.The experiments use 100 correctly classified ImageNet validation images for each model.
- The method is evaluated against Boundary, Optimization, and NES-LO in the decision-based black-box setting.
- The experiments support applying the evolutionary attack method to general object recognition tasks, not only face recognition.
D. Experiments Requested by the Reviewers
Reviewer-requested supplementary materials report distortion summaries and figures for LFW, MegaFace, and ImageNet attack experiments.
- Experiments Requested by the Reviewers: The supplementary section states that the reported experimental results were requested by reviewers.
- Experiments Requested by the Reviewers: MegaFace face verification and identification results report average MSE across SphereFace, CosFace, and ArcFace at four query budgets.
- Experiments Requested by the Reviewers: Figures 7 and 8 plot MegaFace average distortion against query count for face verification and identification, respectively.
- Experiments Requested by the Reviewers: Figure 9 and Table 7 report average MSE for untargeted and targeted ImageNet attacks on Inception v3 and ResNet 50 across four query budgets.
- Experiments Requested by the Reviewers: At 100,000 queries on LFW, the method's maximum distortion is smaller than 1.1e−4 and described as almost imperceptible for humans.The reported statistics include the mean, standard deviation, and maximum distortion over 500 image pairs.
D.2. A different Initial Image for Impersonation Attacks
The authors examine whether impersonation attacks remain effective when initialized with a different image of the target identity rather than the enrollment image. Results show nearly identical average distortions after 100,000 queries across three models.
- Using a different target-identity image is more practical than using the enrollment image.
- After 100,000 queries, average distortions were 1.1e−5, 4.7e−6, and 1.1e−5 for the three models.
- These distortions closely matched the enrollment-initialized results of 1.2e−5, 5.3e−6, and 1.2e−5.
D.3. Compared with White-box Attacks
The proposed method is compared with a white-box PGD attack on CosFace face verification. Across 500 LFW image pairs, the black-box method achieves smaller average distortions for both dodging and impersonation attacks.
- For CosFace face verification, PGD found minimum perturbations through binary search for each image pair.
- The comparison used 500 face-image pairs from the LFW dataset.
- 1.7e−5 dodging distortion and 8.0e−6 impersonation distortion were larger for PGD than the proposed method's 1.1e−5 and 5.3e−6.