Source-linked AI summary
CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations
Yuanhan Zhang, Zhenfei Yin, Yidong Li, Guojun Yin, Junjie Yan, Jing Shao, Ziwei Liu
TL;DR
Face anti-spoofing needs datasets that better represent complex attacks and real-world variation, because existing resources are limited in quantity and diversity. The paper introduces CelebA-Spoof and evaluates it with the AENet unified multi-task framework, finding useful roles for semantic and geometric auxiliary information and strong generalizability from semantic supervision.
Problem
Existing face anti-spoofing datasets are limited in quantity and diversity, restricting coverage of complex spoof attacks and real-world conditions.
Method
The paper constructs CelebA-Spoof and benchmarks face anti-spoofing methods in the unified Auxiliary Information Embedding Network multi-task framework.
Results
625,537 images from 10,177 subjects and rich annotations support observations about semantic and geometric auxiliary information, including semantic supervision's strong generalizability.
Takeaways & Limitations
CelebA-Spoof's scale, diversity, and rich attributes provide an effective data source and support comprehensive evaluation and diagnosis of face anti-spoofing methods.
Abstract
from arXiv · showhide
As facial interaction systems are prevalently deployed, security and reliability of these systems become a critical issue, with substantial research efforts devoted. Among them, face anti-spoofing emerges as an important area, whose objective is to identify whether a presented face is live or spoof. Though promising progress has been achieved, existing works still have difficulty in handling complex spoof attacks and generalizing to real-world scenarios. The main reason is that current face anti-spoofing datasets are limited in both quantity and diversity. To overcome these obstacles, we contribute a large-scale face anti-spoofing dataset, CelebA-Spoof, with the following appealing properties: 1) Quantity: CelebA-Spoof comprises of 625,537 pictures of 10,177 subjects, significantly larger than the existing datasets. 2) Diversity: The spoof images are captured from 8 scenes (2 environments * 4 illumination conditions) with more than 10 sensors. 3) Annotation Richness: CelebA-Spoof contains 10 spoof type annotations, as well as the 40 attribute annotations inherited from the original CelebA dataset. Equipped with CelebA-Spoof, we carefully benchmark existing methods in a unified multi-task framework, Auxiliary Information Embedding Network (AENet), and reveal several valuable observations.
1 Introduction
Face anti-spoofing protects deployed facial interaction systems, but existing datasets lack the scale, diversity, annotations, and difficulty needed for real-world evaluation. CelebA-Spoof addresses these gaps with rich data and a unified multi-task benchmark framework.
- Face anti-spoofing determines whether a presented face is live or spoof for systems such as phone unlock, access control, and e-wallets.
- Existing datasets lack sufficient subjects, sessions, sensors, dense attributes, and challenging examples for evaluating generalization.
- The dataset includes diverse acquisition conditions, while its annotations describe spoof type, illumination, environment, and facial attributes.
- AENet provides a unified multi-task framework for benchmarking methods and analyzing semantic and geometric auxiliary information.
- Experiments report that semantic auxiliary information improves classification and that models trained with it exhibit strong generalizability compared with alternatives.
- CelebA-Spoof contains 625,537 images from 10,177 subjects with annotations covering face, illumination, environment, and spoof types.
2 Related Work
Prior face anti-spoofing datasets span multi-modal, single-modal, and zero-shot settings, but many are outdated or limited in subjects, spoof types, environments, and sensors. CelebA-Spoof is introduced as a larger and more diverse alternative.
- Multi-modal datasets use modalities unavailable on many widespread mobile phones, limiting their applicability in real scenes.
- Older single-modal datasets rely on acquisition equipment that may not meet current practical needs.
- More recent datasets still provide limited subjects, spoof types, or environments, which does not guarantee the generalization required in real applications.
- CelebA-Spoof contributes 625537 pictures from 10177 subjects across 8 scenes with rich annotations.
- Earlier methods use auxiliary signals including depth maps, rPPG, reflection maps, and spatial-temporal information for face anti-spoofing.
3 CelebA-Spoof Dataset
CelebA-Spoof is a large, diverse, densely annotated face anti-spoofing dataset constructed from CelebA live images and newly collected spoof images. Its design spans varied collection conditions, semantic attributes, balanced statistics, and subject-disjoint evaluation splits.
- Dataset scale and diversity: 625,537 pictures from 10,177 subjects make CelebA-Spoof larger and more diverse than existing face anti-spoofing datasets.Each image has 43 attributes, and the dataset includes more than 10 input sensors.
- Dataset construction: Live images are inherited from CelebA after removing posters, advertisements, and cartoon portraits, while spoof images are collected and annotated separately.The construction begins with CelebA live data and uses hired collectors and annotators for spoof-data collection and label refinement.
- Dataset construction: Spoof collection varies five angles, four shapes, and multiple resolutions to increase collection diversity.Angles range from vertical, down, up, forward, and backward, with inclination between −30° and 30°; resolutions range from 40 million to 12 million pixels.
- Semantic annotations: The annotations cover 40 inherited face attributes plus spoof type, illumination, and environment information.Spoof images are described through four macro-types and 11 micro-types of spoof type, four illumination conditions, and two environmental conditions.
- Statistics and splits: CelebA-Spoof contains balanced attack-type proportions, 12% dark, 11% back, and 19% strong illumination, across both indoor and outdoor environments.The dataset’s live-to-spoof ratio is 1:3, and its training, validation, and test sets follow an 8:1:1 split with no subject overlap.
4 Auxiliary Information Embedding Network
AENet embeds semantic and geometric auxiliary information alongside binary live/spoof classification in a unified multi-task framework.
- AENet adds a semantic branch to exploit richly annotated attributes while retaining the main binary classification branch.
- AENetC,S jointly learns semantic attributes and binary classification labels, using auxiliary attributes as complementary cues.
- AENetC,G predicts depth and reflection maps as geometric auxiliaries alongside live/spoof classification.
- Geometric supervision uses mean squared error losses, with depth and reflection targets generated or zeroed according to image type.
5 Experimental Settings
The experiments standardize evaluation across face anti-spoofing metrics and ablation settings, using common backbones and specified image-training procedures.
- The benchmark unifies APCER, BPCER, ACER, EER, HTER, FPR@Recall, and AUC for comprehensive evaluation.APCER and BPCER measure live and spoof error rates respectively; ACER averages them, while AUC and FPR@Recall provide broader and detailed views.
- All experiments use ResNet-18 for fair comparison, with Xception additionally provided to enrich the benchmarks.
- Ablations compare baseline classification with semantic and geometric variants that retain or discard individual auxiliary signals.
- Face inputs are resized to 224×224, augmented by color distortion, and optimized with SGD at learning rate 0.005 for 50 epochs.
6 Ablation Study on CelebA-Spoof
Ablations show that semantic supervision improves live/spoof classification, while geometric cues help unevenly across spoof types and illumination conditions.
- Semantic attributes Sf and Ss greatly facilitate live/spoof classification, whereas geometric depth information is particularly sensitive to dark illumination.
- Study of Semantic Information: Joint semantic learning with binary classification outperforms semantic-only classification and the baseline.
- Study of Semantic Information: Removing spoof-type supervision produces the five worst APCERSs among ten spoof types, indicating its importance for spoof-image classification.
- Study of Semantic Information: Removing face-attribute supervision produces the highest BPCER and 25 of the 40 worst face-attribute scores, highlighting its role for live-image classification.
- Study of Geometric Information: Depth maps improve classification for most spoof types, while reflection maps mainly help with replay attacks.
- Study of Geometric Information: Depth learning is sensitive to dark illumination, causing weaker A4 performance without reflection supervision; under other illumination conditions, A4, poster, and photo become nearly equal.
7 Benchmarks
The benchmarks evaluate image-driven methods within CelebA-Spoof, across intra-dataset, cross-domain, and cross-dataset settings, with combined auxiliary information yielding strong performance and generalization.
- Three benchmarks assess overall classification, controlled domain shifts, and cross-dataset generalization using CelebA-Spoof and related datasets.
- Intra-Dataset Benchmark: AENetC,S,G achieves the best CelebA-Spoof intra-dataset result and outperforms the state-of-the-art by 38% with much fewer parameters.
- Cross-Domain Benchmark: Cross-domain protocols evaluate performance across spoof media and controlled domain shifts, including held-out micro spoof types.
- Cross-Dataset Benchmark: Cross-dataset testing trains on CelebA-Spoof and tests on CASIA-MFSD to measure dataset quality and method generalization.
- Cross-Dataset Benchmark: A simple ResNet-18 reaches state-of-the-art cross-dataset performance, and semantic information generalizes better than geometric information.
8 Conclusion
CelebA-Spoof provides a large-scale, richly annotated face anti-spoofing dataset and a multi-task framework for evaluating semantic and geometric information. Three benchmarks support evaluation and diagnosis across performance and generalization protocols.
- CelebA-Spoof contains 625,537 images from 10,177 subjects with 43 attributes spanning faces, illumination, environments, and spoof types.
- AENet is a clean, powerful multi-task framework used to investigate semantic and geometric information in face anti-spoofing.
- Three versatile benchmarks evaluate performance and generalization under carefully designed protocols.
- The study reports observations that demonstrate the effectiveness of CelebA-Spoof and its rich attributes for future face anti-spoofing research.
9 Appendix
The appendix documents dataset curation, sensor grouping, evaluation metrics, and benchmark analyses. It also reports semantic multi-task results, geometric-information limitations, and backbone-dependent performance patterns.
- Dataset construction: CelebA spoof images are manually examined and removed from the spoof instruments and live-data collection process.The source set contains 347 spoof images, including posters, advertisements, and portraits.
- Sensor split: 24 input sensors are divided into low-, middle-, and high-quality groups according to imaging quality.For cross-domain evaluation, only sensors appearing in both training and testing sets are selected.
- Evaluation metrics: The benchmark unifies APCER, BPCER, ACER, EER, HTER, AUC, and FPR@Recall to make comparisons comprehensive.APCER and BPCER measure spoof and live error rates, ACER averages them, AUC measures overall classification, and FPR@Recall exposes recall at specified FPRs.
- Geometric-information limitation: Reflection maps are not used as unique binary supervision because reflection artifacts appear in only some spoof images, whereas every live image has a depth map.
- Semantic-information ablation: Multi-task learning improves mAP for face attributes and spoof type relative to their corresponding single-task AENet variants.The illumination-condition task performs worse in the multi-task model because its multi-task loss weight is λ = 0.01 rather than λ = 1.
- Heavier-model benchmark: Xception outperforms ResNet-18 on intra-dataset and cross-domain benchmarks but performs worse on cross-dataset evaluation.In cross-domain protocol 1, the Xception AENet model outperforms the Xception baseline by 67.3% in APCER; the appendix attributes cross-dataset degradation to fitting the training data.