PaperlayerView exact revision on arXiv

Source-linked AI summary

CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofing

Shifeng Zhang, Ajian Liu, Jun Wan, Yanyan Liang, Guogong Guo, Sergio Escalera, Hugo Jair Escalante, Stan Z. Li

arXiv:1908.10654v2cs.CV

TL;DR

Face anti-spoofing benchmarks lack sufficient subjects, modalities, and evaluation diversity for advancing robust PAD research. CASIA-SURF addresses this gap with a large multi-modal dataset, broader protocols and metrics, and a multi-scale fusion baseline, with experiments showing improved performance and cross-dataset generalization.

  • Problem

    Existing face anti-spoofing datasets have limited subjects and modalities, and commonly used metrics do not address false-positive-rate-focused application needs.

  • Method

    CASIA-SURF provides 1,000 subjects, 21,000 video clips, three modalities, cross-modal protocols, ROC-based metrics, and a multi-modal multi-scale feature-reweighting fusion method.

  • Results

    Fusing all three modalities improves TPR@FPR=10^-2, 10^-3, 10^-4 from 96.8%, 86.5%, 67.3% to 99.7%, 97.4%, 92.4%, respectively.

  • Takeaways & Limitations

    CASIA-SURF supports broader face anti-spoofing evaluation and improves cross-dataset performance when used for pre-training.

Abstract

from arXiv · show

Face anti-spoofing is essential to prevent face recognition systems from a security breach. Much of the progresses have been made by the availability of face anti-spoofing benchmark datasets in recent years. However, existing face anti-spoofing benchmarks have limited number of subjects ($\le\negmedspace170$) and modalities ($\leq\negmedspace2$), which hinder the further development of the academic community. To facilitate face anti-spoofing research, we introduce a large-scale multi-modal dataset, namely CASIA-SURF, which is the largest publicly available dataset for face anti-spoofing in terms of both subjects and modalities. Specifically, it consists of $1,000$ subjects with $21,000$ videos and each sample has $3$ modalities (i.e., RGB, Depth and IR). We also provide comprehensive evaluation metrics, diverse evaluation protocols, training/validation/testing subsets and a measurement tool, developing a new benchmark for face anti-spoofing. Moreover, we present a novel multi-modal multi-scale fusion method as a strong baseline, which performs feature re-weighting to select the more informative channel features while suppressing the less useful ones for each modality across different scales. Extensive experiments have been conducted on the proposed dataset to verify its significance and generalization capability. The dataset is available at https://sites.google.com/qq.com/face-anti-spoofing/welcome/challengecvpr2019?authuser=0

I. INTRODUCTION

CASIA-SURF addresses limitations in face anti-spoofing datasets by providing larger subject coverage, three modalities, broader metrics, and cross-modal protocols. It also introduces a multi-modal multi-scale fusion baseline and evaluates the dataset extensively.

  • Face anti-spoofing protects recognition systems against print, replay, and 2D/3D mask attacks that can make recognition results unreliable.
  • Existing datasets contain fewer than 170 subjects and limited video clips, restricting their representativeness for real applications.
  • Existing datasets mainly use one modality, while CASIA-SURF provides RGB, Depth, and IR for 1,000 subjects and 21,000 video clips.
  • CASIA-SURF adds ROC-based evaluation alongside APCER, NPCER, and ACER, enabling threshold selection between FPR and TPR for applications.
  • The dataset includes cross-modal protocols that evaluate models trained in one modality on other modalities.
  • The proposed multi-modal multi-scale fusion method re-weights channel features across scales, retaining informative features and suppressing less useful ones.
  • The paper conducts extensive experiments to assess CASIA-SURF’s significance and generalization capability, including improved multi-scale fusion over the prior one-scale version.

II. RELATED WORK

This section summarizes existing face anti-spoofing datasets and representative methods.

  • The related-work section surveys existing face anti-spoofing datasets before reviewing representative methods.

A. Dataset

Existing face anti-spoofing datasets and methods span modalities, handcrafted features, fusion strategies, and CNNs, but remain constrained by limited data diversity and generalization.

  • Most existing datasets contain only RGB data, while newer sensors provide depth, multispectral, or infrared information for face anti-spoofing.
  • Existing datasets have limited subjects and samples, creating potential over-fitting risk, and mostly lack modalities needed for newer attacks such as 3D masks.
  • Earlier methods use liveness, contextual, motion, color-space, Fourier, or handcrafted features such as LBP, HoG, and GLCM.
  • Prior fusion methods combine frame and video analysis, low-level descriptors, or system-independence measures, but generally do not fuse modalities.
  • CNN-based methods improve face PAD performance but can generalize poorly because of over-fitting to training data, motivating larger PAD datasets.

III. CASIA-SURF DATASET

CASIA-SURF is a large-scale, three-modality face anti-spoofing dataset with diverse recording conditions and six photo-attack styles. It contains 1,000 subjects and 21,000 videos, extending existing dataset scale and modality coverage.

  • CASIA-SURF contains 1,000 Chinese subjects in 21,000 videos across RGB, Depth, and IR modalities.The dataset is presented as the largest face anti-spoofing dataset in terms of subjects and modalities.
  • RGB, Depth, and IR videos are recorded simultaneously with an Intel RealSense SR300 camera in diverse indoor environments.Collectors perform head and distance movements during recording, and performers stand 0.3 to 1.0 meter from the camera.
  • Each subject contributes one live video and six fake videos generated from printed flat or curved face images.The printed images have eye, nose, mouth, or combined regions cut to create the attacks.
  • The six attack styles vary both face-photo curvature and the facial regions removed.Attacks 1 and 2 cut the eyes, Attacks 3 and 4 cut the eyes and nose, and Attacks 5 and 6 cut the eyes, nose, and mouth.

C. Data preprocessing

The preprocessing pipeline detects and aligns faces from the RGB and aligned RGB-Depth-IR videos to focus the data on face anti-spoofing.

  • Dlib detects faces in every frame of the RGB and aligned RGB-Depth-IR videos.
  • PRNet then performs 3D reconstruction and density alignment on the detected faces.

D. Statistics description

CASIA-SURF includes subject, video, frame, demographic, and subset statistics, with separate training, validation, and testing partitions. The dataset also supports within-modal and cross-modal evaluation protocols.

  • Dataset statistics: 1,000 subjects each provide one live video and six fake videos, with variation in gender, age, glasses, and indoor environment.
  • Dataset statistics: 300, 100, and 600 subjects belong to the training, validation, and testing subsets, respectively.The corresponding video counts are 6,300, 2,100, and 12,600.
  • Dataset statistics: Frame sampling reduces the original approximately 1.5 million, 0.5 million, and 3.1 million frames to about 151K, 49K, and 302K for training, validation, and testing.
  • Demographics: The dataset includes gender variation, with 56.8% female and 43.2% male subjects.
  • Demographics: Ages range from 20 to over 70 years, with the [20, 30) range comprising about 50% of subjects.
  • Data partitions: The training subset uses live faces and Attacks 4, 5, and 6, while validation and testing use live faces and Attacks 1, 2, and 3.This creates differences in flat or curved faces and cut-organ extent between training and evaluation.

F. Evaluation metric

The evaluation uses ROC-based low-false-positive operating points alongside ACER, APCER, and NPCER, while the baseline uses separate modality streams and feature fusion. The proposed SEF re-weights modality channels before combining them across multiple feature scales.

  • Evaluation metric: TPR@FPR=10^-4 is the main comparison metric, supplemented by TPR@FPR=10^-2, 10^-3 and ACER, APCER, and NPCER.ROC curves support selecting a threshold according to the required FPR–TPR trade-off.
  • Baseline architecture: The baseline treats face anti-spoofing as binary fake-versus-real classification using ResNet-18/34 networks.Each network has five convolutional blocks, global average pooling, and a softmax layer.
  • Multi-modal fusion: RGB, Depth, and IR are learned in separate streams before shared layers learn joint representations and make coordinated decisions.The modalities provide complementary appearance, distance, and heat information.
  • Multi-scale fusion: The multi-scale design applies SEF after res3, res4, and res5, then uses global average pooling and concatenation for real/fake prediction.
  • Multi-modal fusion: SEF obtains modality-specific channel weights, re-weights input features, and combines the resulting features.This suppresses less useful features while selecting more informative channel features from each modality.

C. From single-scale to multi-scale SEF

The multi-scale SEF extends modality fusion beyond a single ResNet level, combining features from res3, res4, and res5 before prediction.

  • Motivation: The extension addresses the single-scale SEF limitation that features from different network levels were not fully exploited.
  • Multi-scale fusion: The multi-scale architecture processes RGB, Depth, and IR in separate streams, then fuses their features after res3, res4, and res5.Each fused representation is squeezed with global average pooling, concatenated, and used to predict real versus fake.
  • SEF implementation: SEF is implemented as a Squeeze-and-Excitation branch with global average pooling followed by two fully connected layers.
  • Experimental scope: Experiments evaluate implementation details, method effectiveness, modality and subject effects, cross-modal performance, and dataset generalization.
  • Implementation details: Training uses 112 × 112 cropped faces with augmentation, 40 epochs, Adam optimization, and two TITAN X GPUs.

B. Model analysis

Model analysis compares SEF variants, backbones, modalities, subject counts, and attack types on CASIA-SURF, showing benefits from multi-scale fusion and broader modality coverage.

  • Ablation analysis: 99.7%, 97.4%, and 92.4% TPR@FPR=[10^-2, 10^-3, 10^-4] are obtained after the reported model improvements, while APCER, NPCER, and ACER reach 1.6%, 0.08%, and 0.8%.The multi-scale SEF contributes an 11.0% improvement at TPR@FPR=10^-4, and ResNet-34 adds 0.1%, 1.0%, and 2.8% across the three TPR operating points.
  • Effect of modalities: Fusing all three modalities improves the best single-modality results to 99.7%, 97.4%, and 92.4% TPR@FPR=[10^-2, 10^-3, 10^-4].The corresponding APCER, NPCER, and ACER are 1.9%, 0.1%, and 1.0%.
  • Effect of subjects: Baseline performance improves as the training subset includes more subjects.The study randomly samples 50, 100, or 200 subjects from 300 training subjects to measure this effect.
  • Attack difficulty: Attack 1, Attack 2, and Attack 3 achieve 94.4%, 92.9%, and 86.4% TPR@FPR=10^-4, respectively.The results identify curved and cutout fake faces as more challenging than flat fake faces in the reported comparisons.

D. Cross-modal evaluation

Cross-modal testing is difficult because models trained in one modality perform poorly on other modalities, whereas CASIA-SURF pre-training improves transfer to established face anti-spoofing datasets.

  • Cross-modal evaluation: Models trained on RGB, Depth, or IR perform far below satisfactory levels when evaluated on the other modalities.The paper attributes this result to large differences between modality data and identifies cross-modal evaluation as a challenging problem.
  • Generalization setup: The baseline is pre-trained on CASIA-SURF and then fine-tuned on Oulu-NPU, SiW, or CASIA-MFSD to assess generalization.The method uses CASIA-SURF Depth information for depth-supervised face anti-spoofing pre-training.
  • Oulu-NPU transfer: CASIA-SURF pre-training reduces Oulu-NPU ACER from 5.8% to 2.6% in Protocol 1, from 3.7% to 2.2% in Protocol 2, and from 5.3% to 2.3% in Protocol 3.Protocol 4 decreases from 13.5% to 7.2%.
  • SiW transfer: CASIA-SURF pre-training reduces SiW ACER by 0.25%, 0.14%, and 1.38% in Protocols 1, 2, and 3, respectively.The reported improvements span variations in pose and expression, replay mediums, and cross-Presentation Attack Instruments.
  • CASIA-MFSD transfer: CASIA-SURF is also evaluated through direct cross-testing on CASIA-MFSD after training on the proposed dataset.

VI. DISCUSSION

The discussion explains the dataset design choices and argues that ROC evaluation better reflects application-level false-positive requirements than conventional error metrics alone.

  • Dataset scope: Video replay attacks are omitted because replay videos appear black in depth images, making them easy to recognize using depth data.The authors plan to add other presentation attack types, including 3D masks.
  • Evaluation metrics: ROC curves are introduced because APCER=1.6% can still treat about 2 of 100 attackers as real, which may not meet face-payment or phone-unlock requirements.The discussion presents ROC evaluation as more suitable for checking a specified application’s requirements.
  • Conclusion: CASIA-SURF combines a large multi-modal benchmark, diverse protocols, comprehensive metrics, and a multi-modal multi-scale fusion baseline.The conclusion reports experiments addressing generalization and the benefit of multiple visual modalities.
Loading 1908.10654v2…