Source-linked AI summary
A Dataset and Benchmark for Large-scale Multi-modal Face Anti-spoofing
Shifeng Zhang, Xiaobo Wang, Ajian Liu, Chenxu Zhao, Jun Wan, Sergio Escalera, Hailin Shi, Zezheng Wang, Stan Z. Li
TL;DR
Existing face anti-spoofing datasets provide limited subjects, modalities, and evaluation perspectives despite the security importance of detecting presentation attacks. This paper introduces CASIA-SURF as a large-scale three-modality benchmark with a fusion baseline and reports strong traditional-metric performance while advocating ROC evaluation for application relevance.
Problem
Existing face anti-spoofing datasets have limited subjects and modalities, while conventional metrics may not reflect false-positive concerns in real applications.
Method
The paper constructs CASIA-SURF with 1,000 subjects, 21,000 video clips, and RGB, Depth, and IR modalities, and proposes modal-dependent feature re-weighting for multi-modal fusion.
Results
APCER=3.8%, NPCER=1.0%, ACER=2.4% were achieved on CASIA-SURF using traditional face anti-spoofing metrics.
Takeaways & Limitations
The paper provides a large-scale multi-modal benchmark and proposes ROC curves alongside traditional metrics for evaluating face anti-spoofing in application-oriented settings.
Takeaways & Limitations
The reported 3.8% fake-sample error rate is below the accuracy requirements of applications such as face payment and phone unlock.
Abstract
from arXiv · showhide
Face anti-spoofing is essential to prevent face recognition systems from a security breach. Much of the progresses have been made by the availability of face anti-spoofing benchmark datasets in recent years. However, existing face anti-spoofing benchmarks have limited number of subjects ($\le\negmedspace170$) and modalities ($\leq\negmedspace2$), which hinder the further development of the academic community. To facilitate face anti-spoofing research, we introduce a large-scale multi-modal dataset, namely CASIA-SURF, which is the largest publicly available dataset for face anti-spoofing in terms of both subjects and visual modalities. Specifically, it consists of $1,000$ subjects with $21,000$ videos and each sample has $3$ modalities (i.e., RGB, Depth and IR). We also provide a measurement set, evaluation protocol and training/validation/testing subsets, developing a new benchmark for face anti-spoofing. Moreover, we present a new multi-modal fusion method as baseline, which performs feature re-weighting to select the more informative channel features while suppressing the less useful ones for each modal. Extensive experiments have been conducted on the proposed dataset to verify its significance and generalization capability. The dataset is available at https://sites.google.com/qq.com/chalearnfacespoofingattackdete
1. Introduction
Face anti-spoofing protects face recognition systems from presentation attacks, but existing benchmarks are limited in subjects and modalities. CASIA-SURF addresses these gaps with a large multi-modal dataset, an evaluation protocol, and a feature-reweighting fusion baseline.
- Print, replay, and 2D/3D mask attacks can make face recognition results unreliable, motivating presentation attack detection.
- Existing face anti-spoofing datasets contain fewer than 170 subjects, while multi-modal datasets include no more than 21 subjects.
- CASIA-SURF contains 1,000 subjects, 21,000 video clips, and three modalities: RGB, Depth, and IR.
- The dataset includes six types of photo attacks generated through operations such as cropping, bending print paper, and varying stand-off distance.
- The paper introduces ROC evaluation to select a threshold trading off false positive rate and true positive rate for application requirements.
- The proposed fusion baseline re-weights modality-dependent channel features, retaining informative features while suppressing less useful ones.
2. Related work
Prior face anti-spoofing research includes handcrafted, CNN-based, and fusion methods, but public datasets remain small and predominantly RGB-only. These constraints limit evaluation across subjects, modalities, and newer attack types.
- Most existing datasets provide only RGB data, while the multi-modal Msspoof dataset contains visible and near-infrared images from no more than 21 objects.
- Existing datasets have limited subjects and samples, creating potential over-fitting risk when face anti-spoofing algorithms are evaluated.
- Visible-camera datasets can cause 2D anti-spoofing methods to fail against newer 3D and custom-made silicone-mask attacks.
- Earlier approaches use liveness, contextual, motion, color-space, Fourier-spectrum, and handcrafted features such as LBP, HoG, and GLCM.
- Prior fusion methods combine frame and video scores, low-level features, or anti-counterfeiting systems, rather than fusing modalities directly.
- CNN-based methods formulate face anti-spoofing as binary classification and include auxiliary depth-map or rPPG supervision or spoof-noise decomposition.
3. CASIA-SURF dataset
CASIA-SURF addresses the limited scale and modality diversity of existing face anti-spoofing datasets with 1,000 subjects, six attack styles, and synchronized RGB, Depth, and IR recordings. It defines dataset splits, preprocessing, statistics, and intra- and cross-testing protocols.
- Motivation: Existing face anti-spoofing datasets contain reduced subject counts and typically only one visual modality, limiting development toward broader applications.The paper identifies dataset size and modality coverage as central limitations of prior benchmarks.
- Dataset construction: CASIA-SURF contains 1,000 Chinese subjects and 21,000 videos, with each sample comprising one live clip and six fake clips.The dataset includes variability in gender, age, glasses, and indoor environments.
- Attack styles: Six fake-video attacks vary flat or curved printed faces and cut eye, nose, and mouth regions in different combinations.The six attack styles are formed from these printed-face manipulations.
- Recording setup: RGB, Depth, and IR videos are captured simultaneously with an Intel RealSense SR300 while performers move and vary distance within 0.3 to 1.0 meter.The recordings use synchronized multimodal streams and controlled performer actions.
- Data preprocessing: Preprocessing detects faces, applies 3D reconstruction and density alignment, generates masks, and extracts modality-specific face areas by pointwise products.RGB uses an RGB mask, while Depth and IR use an aligned RGB-Depth-IR mask.
- Statistics and protocols: The dataset is split by subjects into 300 training, 100 validation, and 600 testing subjects, with about 148K, 48K, and 295K processed frames, respectively.Intra-testing trains on Attacks 4–6 and evaluates on Attacks 1–3; cross-testing targets high appearance variability with limited training samples.
4. Method
The method treats face anti-spoofing as binary fake-versus-real classification using a three-stream ResNet-18 architecture. It fuses modality features through squeeze-and-excitation re-weighting before shared layers learn joint representations.
- Baseline: Face anti-spoofing is formulated as binary classification between fake and real faces using a ResNet-18 network.The baseline contains five convolutional blocks, global average pooling, and softmax classification.
- Multi-stream architecture: A three-stream architecture separately processes RGB, Depth, and IR data before shared layers learn joint representations and cooperative decisions.The design is intended to study complementary information across the three modalities.
- Modalities: RGB provides appearance details, Depth measures face-to-image-plane distance, and IR measures heat radiated from the face.These modalities provide complementary information for different attack types.
- Fusion pipeline: Features from res1–res3 are extracted independently for each modality, fused, and passed through shared res4–res5 blocks before global average pooling.This staged architecture is depicted in the proposed fusion diagram.
- Squeeze and excitation fusion: The squeeze-and-excitation fusion module re-weights modality-specific channel features, selecting informative features and suppressing less useful ones before concatenation.Each modality branch uses global average pooling followed by two fully connected layers.
5. Experiments
Experiments evaluate the fusion method, modality complementarity, training-set scale, and cross-dataset generalization on CASIA-SURF and established benchmarks. The results show benefits from feature re-weighting, combining modalities, using more subjects, and pre-training on CASIA-SURF.
- Model analysis: The proposed fusion method reaches TPR=96.7%, 81.8%, 56.8% @FPR=10^-2, 10^-3, 10^-4, respectively, outperforming halfway fusion.APCER, NPCER and ACER improve from 5.6%, 3.8% and 4.7% to 3.8%, 1.0% and 2.4%, respectively.
- Dataset analysis: Fusing all three modalities improves the best single-modality results to TPR=96.7%, 81.8%, 56.8% @FPR=10^-2, 10^-3, 10^-4 and ACER=2.4%.Depth is the strongest single modality, while combining RGB, Depth and IR produces the best reported results.
- Dataset analysis: A 300-subject training set yields about 7% higher TPR at FPR=10^-2 than the second-best 200-subject setting.TPR improves as more subjects are used, and ACER also gets better with larger training subsets.
- Generalization capability: Pre-training on CASIA-SURF further improves SiW ACER by about 0.25%, 0.14% and 1.38% in Protocols 1, 2 and 3, respectively.The reported gains cover variation in face pose and expression, replay attack mediums, and cross-presentation attack instruments.
- Generalization capability: The model trained on CASIA-SURF performs best in cross-testing experiments on the CASIA-MFSD dataset.The comparison includes models trained on SiW, CASIA-SURF and Replay-Attack-based systems.
6. Discussion
The discussion argues that conventional PAD metrics can appear promising while masking application-relevant operating-point performance. It therefore motivates ROC-based evaluation, especially at very low false-positive rates.
- Metric interpretation: An APCER of 3.8% means 3.8 fake samples from 100 attackers are treated as real, below stated face-payment and phone-unlock accuracy requirements.The corresponding real-sample error rate is 1.0%.
- Evaluation metric: ROC curves are proposed as an additional face anti-spoofing evaluation metric because they expose performance across false-positive operating points.The discussion links ROC evaluation to established face-recognition practice.
- Metric interpretation: ACER=2.4% coexists with TPR=56.8% at FPR=10^-4, which is far from the standard required in real applications.TPR at FPR values around 10^-4 or 10^-5 is identified as meaningful for face anti-spoofing.
7. Conclusion
The paper releases CASIA-SURF as a large-scale, three-modality face anti-spoofing dataset and introduces modality-dependent feature re-weighting for fusion. Experiments report improved multi-modal performance and generalization, while motivating ROC-based evaluation for real applications.
- Conclusion: CASIA-SURF is presented as the largest dataset by subjects, data samples and visual modalities.The paper reports extensive experiments on the dataset and releases it for face anti-spoofing research.
- Conclusion: The fusion method re-weights features by modality to select informative channels and suppress less informative ones.The conclusion describes this as a multi-modal fusion method evaluated on CASIA-SURF.
- Conclusion: Experiments show high generalization capability for models trained on CASIA-SURF and benefits from using multiple visual modalities.The paper also proposes ROC curves for evaluating large-scale face anti-spoofing.