Source-linked AI summary
LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition
Valeriia Cherepanova, Micah Goldblum, Harrison Foley, Shiyuan Duan, John Dickerson, Gavin Taylor, Tom Goldstein
TL;DR
Facial recognition systems scrape social-media images for mass surveillance, while existing evasion methods fail against full-scale commercial APIs. LowKey preprocesses images with a black-box adversarial attack that incorporates face detection and alignment, and is released as a public webtool. It consistently prevents recognition in commercial API tests, while Amazon Rekognition recognizes 2.4% of protected probe images at rank-50 accuracy.
Problem
Facial recognition systems collect social-media images for large-scale surveillance, but existing evasion methods are ineffective against commercial APIs.
Method
LowKey preprocesses images using a black-box adversarial attack that moves gallery-face representations while accounting for face detection, extraction, resizing, and alignment.
Results
2.4% of probe images were recognized by Amazon Rekognition at rank-50 accuracy for users protected with LowKey, while Fawkes reached 77.5%.
Takeaways & Limitations
LowKey provides a publicly available protection tool that is effective against commercial black-box facial recognition APIs.
Takeaways & Limitations
LowKey does not protect users 100% of the time and may be circumvented by specially engineered robust systems.
Abstract
from arXiv · showhide
Facial recognition systems are increasingly deployed by private corporations, government agencies, and contractors for consumer services and mass surveillance programs alike. These systems are typically built by scraping social media profiles for user images. Adversarial perturbations have been proposed for bypassing facial recognition systems. However, existing methods fail on full-scale systems and commercial APIs. We develop our own adversarial filter that accounts for the entire image processing pipeline and is demonstrably effective against industrial-grade pipelines that include face detection and large scale databases. Additionally, we release an easy-to-use webtool that significantly degrades the accuracy of Amazon Rekognition and the Microsoft Azure Face Recognition API, reducing the accuracy of each to below 1%.
1 INTRODUCTION
Facial recognition systems scrape public social-media images for mass-surveillance databases, forcing users to choose between privacy and public participation. LowKey adversarially preprocesses images and is presented as an effective public webtool against commercial APIs.
- Facial recognition systems use massive databases populated with publicly available social-media images, creating tension between public participation and surveillance risk.
- LowKey protects users by preprocessing images before social-media publication so third parties cannot use them for facial recognition.
- LowKey uses a black-box adversarial attack that moves gallery-face representations away from corresponding probe images while preserving image quality.
- The authors report that LowKey consistently prevents facial recognition on commercial black-box APIs, whereas Fawkes is ineffective in every experiment.
- The released webtool is intended to let users retain a public social-media presence without accepting the stated surveillance risks.
2 RELATED WORK
Prior adversarial-attack research often relied on simulated settings, while Fawkes evaluated social-media protection under assumptions that diverge from commercial facial-recognition systems. The paper motivates evaluation against larger, more realistic systems.
- Adversarial-attack research has largely focused on simulated neural-network attacks and physical-world patterns, which do not succeed reliably.
- Fawkes assumes practitioners train models on each individual’s data, unlike high-performance systems built with large pretrained Siamese networks.
- Fawkes primarily evaluates image classifiers rather than facial-recognition-specific systems trained with specialized heads and loss functions.
- Fawkes experiments used very small galleries and evaluated only top-1 accuracy, limiting correspondence with commercial identification settings.
3 THE LOWKEY ATTACK ON MASS SURVEILLANCE
LowKey targets identification by corrupting protected gallery images while preserving their perceptual quality. Its attack incorporates face detection, model ensembles, and feature-space separation to transfer across proprietary systems.
- 3.1 PROBLEM SETUP: Identification compares a probe image with gallery images to find likely identities, whereas verification asks whether two images show the same person.
- 3.1 PROBLEM SETUP: LowKey filters gallery images so their feature representations do not match probe images of the same person.
- 3.2 THE LOWKEY ATTACK: The optimization maximizes feature-space distance from the original image while minimizing perceptual similarity loss to preserve image quality.
- 3.2 THE LOWKEY ATTACK: LowKey attacks an ensemble of models with varied architectures and training algorithms to improve transferability to unknown facial-recognition systems.
- 3.2 THE LOWKEY ATTACK: The objective includes face detection, extraction, 112 × 112 resizing, and alignment because commercial systems rely on these preprocessing steps.
- 3.2 THE LOWKEY ATTACK: Signed gradient ascent iteratively updates the perturbed image to move it and its Gaussian-smoothed version away from the original in ensemble feature spaces.
4 EXPERIMENTAL DESIGN
The evaluation uses facial-recognition models trained on large-scale data and tests identification under a gallery-and-probe protocol. FaceScrub supplies known identities and distractors for controlled trials.
- The model ensemble includes ArcFace and CosFace systems with ResNet-50, ResNet-152, IR-50, and IR-152 backbones.
- These models are trained on MS-Celeb-1M, containing over five million images from over 85,000 identities.
- FaceScrub contains over 100,000 images from 530 known identities and one million distractor images.
- The protocol treats one tenth of each identity’s images as probes, inserts the remainder into galleries, and applies LowKey to gallery images for 100 randomly selected identities.
- Each identification trial samples a probe and succeeds when the model’s closest gallery match belongs to the same identity.
5 BREAKING COMMERCIAL BLACK-BOX APIS
LowKey is evaluated against two proprietary facial recognition APIs, Amazon Rekognition and Microsoft Azure Face, using black-box experiments and comparison with Fawkes. LowKey substantially reduces recognition, while Fawkes remains effective at recognizing protected images.
- Experimental setup: The evaluation tests LowKey and Fawkes against Amazon Rekognition and Microsoft Azure Face in a black-box setting.Fawkes images were generated with the authors’ code, hyperparameters, and highest protection setting.
- Amazon Rekognition: 2.4% of probe images were recognized by Amazon Rekognition at rank-50 when gallery images were protected with LowKey.Fawkes protected images had 77.5% rank-1 and 94.9% rank-50 recognition.
- Microsoft Azure Face: 0.1% of probe images were recognized by Microsoft Azure Face when gallery images were protected with LowKey.Azure recognized more than 74% of probe images uploaded by users employing Fawkes.
- Overall comparison: LowKey consistently achieves virtually flawless protection, whereas Fawkes provides little protection in the FaceScrub evaluation.The table caption defines smaller values and lighter colors as indicating more successful attacks.
- Conclusion: The experiments support LowKey’s effectiveness and transferability to state-of-the-art industrial facial recognition systems.The conclusion follows the commercial API experiments.
6 ADDITIONAL EXPERIMENTS
LowKey is evaluated against transferability, smoothing defenses, runtime, compression, and image-size variability. The experiments show robust protection across these conditions, while smaller images and stronger perturbations generally improve protection.
- Evaluation goals: LowKey’s effectiveness is evaluated across transferability, image quality, runtime, file formats, and image sizes.The experiments target the five properties identified as necessary for practical protection.
- Transferability: Attacks transfer better between models with matching backbone families, deeper 152-layer backbones, and the same training head.IR-generated attacks transfer better to IR systems, ResNet-generated attacks transfer better to ResNet systems, and deeper models are harder to fool.
- Gaussian smoothing: Gaussian smoothing preserves rank-50 accuracy but not rank-1 accuracy relative to unsmoothed inference and defeats attacks computed without smoothing.Attacks computed with Gaussian smoothing counteract this defense and are more robust against it.
- Runtime: 32 seconds per image is LowKey’s average runtime versus 54 seconds for Fawkes on the same GPU and evaluation setup.LowKey is reported as both faster and more protective than Fawkes.
- Image compression: 3.8% of probe images are recognized by Amazon Rekognition after JPEG compression, compared with 2.4% for PNG; Microsoft Azure Face recognizes 0.2% versus 0.1%.Compression very slightly decreases performance, but the attack remains highly effective.
- Image size and gallery conditions: 11% Amazon Rekognition rank-50 accuracy with five gallery images compares with 32.5% using the larger gallery, while small probes reduce accuracy to 19%.The experiments also indicate stronger protection on smaller images, although very large images are more challenging because of detection, alignment, and resizing.
7 DISCUSSION
LowKey adversarially pre-processes images before social-media upload to protect them from unauthorized facial recognition. The authors report strong effectiveness against commercial black-box APIs but acknowledge incomplete protection and possible circumvention.
- Protection approach: LowKey pre-processes user images before social-media upload so third-party organizations cannot use them as effectively for facial recognition.The tool is intended to protect users from unauthorized facial recognition.
- Limitations: LowKey does not protect users 100% of the time and may be circumvented by specially engineered robust systems.The authors therefore advise users to remain cautious about publicly revealing personal information.
- Future directions: The authors identify more aesthetically pleasing adversarial filters and broader awareness of facial recognition’s ethical issues as future directions and goals.The discussion notes that effective protection may require visible perturbations.
8 APPENDIX
The appendix details training and attack settings, evaluates LowKey on UMDFaces and smaller perturbations, and compares its visual effects with Fawkes. These experiments expose a trade-off between perturbation visibility and protection.
- Implementation details: LowKey’s feature extractors use focal loss, batch size 512, 120 epochs, scheduled learning-rate reductions, SGD momentum 0.9, and weight decay 5e-4.These are the reported feature-extractor training settings.
- Implementation details: The attacks use perceptual penalty 0.05, Gaussian smoothing σ = 3 with window size 7, and signed SGD for 50 epochs at learning rate 0.0025.The appendix also specifies the face detection and alignment repository used.
- UMDFaces evaluation: LowKey is slightly less effective on UMDFaces, likely because its gallery is much smaller.UMDFaces contains over 367,000 photos of 8,277 identities, with 100 identities selected for controlled experiments.
- Smaller perturbations: 17.2% of probe images are recognized by Amazon Rekognition and 5.5% by Microsoft Azure Face under the LowKey small attack in the rank-50 setting.The smaller attack was created by increasing the perceptual similarity penalty from 0.05 to 0.08.
- Visual comparison: LowKey and Fawkes are both noticeable, but Fawkes adds conspicuous facial artifacts whereas LowKey mainly changes textures and adds skin spots.The comparison is presented as a visual distinction between the two attacks.
- Gaussian smoothing: The defensive Gaussian blur uses σ = 2, whereas the Gaussian smoothing term used during attack computation uses σ = 3 and window size 7.These settings distinguish the attack-side and defense-side smoothing procedures.