Source-linked AI summary
Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision
Yaojie Liu, Amin Jourabloo, Xiaoming Liu
TL;DR
Face anti-spoofing models trained only with binary supervision may learn inadequate, non-generalizable cues. This paper uses CNN-RNN auxiliary supervision for depth and rPPG estimation and introduces the varied SiW database, reporting superior performance across testing settings.
Problem
Binary supervision can lead deep anti-spoofing models to arbitrary cues, poor generalization, and binary decisions without explanations.
Method
A CNN-RNN model learns face depth with pixel-wise supervision and rPPG signals with sequence-wise supervision for live-versus-spoof detection.
Results
The method achieves state-of-the-art face anti-spoofing performance and is evaluated using the new SiW database with broad subject and practical-condition variations.
Takeaways & Limitations
Auxiliary depth and rPPG supervision provides the paper's proposed route toward more generalizable and explainable face anti-spoofing decisions.
Takeaways & Limitations
The approach treats pseudo-depth and pseudo-rPPG as auxiliary supervision, and assumes same-subject videos captured within five minutes share the same ground-truth rPPG signal.
Abstract
from arXiv · showhide
Face anti-spoofing is the crucial step to prevent face recognition systems from a security breach. Previous deep learning approaches formulate face anti-spoofing as a binary classification problem. Many of them struggle to grasp adequate spoofing cues and generalize poorly. In this paper, we argue the importance of auxiliary supervision to guide the learning toward discriminative and generalizable cues. A CNN-RNN model is learned to estimate the face depth with pixel-wise supervision, and to estimate rPPG signals with sequence-wise supervision. Then we fuse the estimated depth and rPPG to distinguish live vs. spoof faces. In addition, we introduce a new face anti-spoofing database that covers a large range of illumination, subject, and pose variations. Experimental results show that our model achieves the state-of-the-art performance on both intra-database and cross-database testing.
1. Introduction
Face anti-spoofing systems relying on binary supervision may learn arbitrary, non-generalizable cues and provide unexplained decisions. The paper addresses this with depth and rPPG auxiliary supervision plus a new varied database.
- Face anti-spoofing protects face recognition systems against presentation attacks such as print, replay, and mask attacks.
- Binary CNN supervision may overfit to arbitrary cues, causing poor generalization when those cues disappear during testing.Examples include screen bezels rather than faithful spoof patterns.
- The proposed model uses spatial depth and temporal rPPG supervision to guide learning toward robust face-spoofing cues.Live faces have face-like depth and detectable rPPG, whereas print or replay attacks have flatter depth or lack normal rPPG signals.
- SiW contains 165 subjects, 6 spoofing mediums, and 4 sessions spanning pose, illumination, expression, and distance-to-camera variations.The database is presented as covering substantially larger variations than previous databases.
- The paper proposes a CNN-RNN architecture for end-to-end learning of depth maps and rPPG signals.
- The authors report state-of-the-art face anti-spoofing performance and introduce SiW as a database with practical variations.
2. Prior Work
Prior work includes texture, temporal, and rPPG-based approaches, but deep methods commonly reduce anti-spoofing to binary classification. The paper instead learns spatial and temporal auxiliary cues, especially depth and rPPG.
- Texture-based Methods: Earlier texture-based systems used handcrafted features and traditional classifiers, often changing color or frequency domains to reduce illumination effects.
- Deep CNN Methods: Many deep face anti-spoofing methods use CNN features with softmax loss to directly classify faces as live or spoof.
- Temporal-based Methods: Temporal approaches include frame concatenation, handcrafted motion features, optical flow, and LSTM-CNN models, while still generally using binary classification.
- Remote Photoplethysmography Methods: The proposed work learns rPPG signals as discriminative temporal information rather than relying only on extracted correlation features.
- Remote Photoplethysmography Methods: rPPG tracks vital signals such as heart rate without contact and has been applied to detect mask, print, and replay attacks.
3. Face Anti-Spoofing with Deep Network
The proposed CNN-RNN learns pseudo-depth and pseudo-rPPG signals as auxiliary supervision, combining spatial and temporal cues for face anti-spoofing. A non-rigid registration layer aligns facial feature maps so the RNN can track corresponding regions across time and subjects.
- CNN-RNN Architecture: The network combines CNN-based depth estimation with RNN-based rPPG estimation to learn spatial and temporal cues for live-versus-spoof classification.The CNN evaluates frames separately, while the RNN evaluates temporal variability across feature maps.
- Depth Map Supervision: Depth supervision uses pseudo-depth maps that encode face shape, with live faces assigned values in [0, 1] and spoof faces assigned zero depth.The maps are generated from estimated dense 3D face shapes and projected to a 2D plane.
- rPPG Supervision: The RNN learns rPPG signals from face-video sequences using sequence-wise supervision, with normalized signals for live videos and zero signals for spoof videos.The RNN minimizes the ℓ1 distance between estimated and ground-truth rPPG signals.
- Training Strategy: The CNN and RNN parts are trained end-to-end through alternating depth-map and rPPG-loss streams.The rPPG stream backpropagates through both the CNN and RNN parts.
- Non-rigid Registration: Registration lets the RNN compare activations for the same facial area across time without concerning about facial pose or expression.This supports learning temporal changes in aligned feature maps across subjects.
4. Collection of Face Anti-Spoofing Database
The SiW database was collected to provide diverse, high-quality face anti-spoofing data, with live and spoof videos spanning subject, pose, and attack variations.
- SiW offers more subjects and practical variations than prior databases, including diverse races and other acquisition factors.
- SiW contains live and spoof 30-fps videos from 165 subjects, totaling 4,620 videos.Each subject contributes 8 live and 20 spoof videos.
- Live recordings cover varying camera distances, head yaw from −90° to 90°, and different facial expressions across four sessions.
- The database uses high-resolution 1,920 × 1,080 live videos captured by two cameras.
- Spoof data includes two print and four replay attacks per subject, using varied print qualities and spoofing mediums.Replay mediums include Samsung Galaxy S8, iPhone 7, iPad Pro, and PC.
5. Experimental Results
Experiments evaluate the proposed architecture through ablations, intra-database testing, cross-database testing, and auxiliary-estimation analysis. Results show benefits from auxiliary supervision, sequence modeling, and registration, alongside strong generalization across databases.
- 5.2.1 Ablation Study: The ablation compares four models differing in binary supervision, depth supervision, rPPG supervision, and non-rigid registration.All models are trained on SiW videos from 20 subjects and evaluated on Oulu Protocol 1 for cross testing.
- 5.2.1 Ablation Study: Depth supervision substantially improves over binary supervision, while adding rPPG supervision and non-rigid registration further improves performance.
- 5.2.1 Ablation Study: Longer sequences reduce ACER because they provide more reliable rPPG estimation, although GPU memory limits sequence length in practice.
- 5.2.2 Intra Testing: The method achieves the lowest ACER in 3 out of 4 Oulu intra-testing protocols, with slightly worse ACER on Protocol 2.
- 5.2.3 Cross Testing: Cross-testing on Oulu yields ACERs of 10.0%, 14.1%, 13.8 ± 5.7%, and 10.0 ± 8.8% across Protocols 1–4.On the hardest Protocol 4, the method matches the top performer’s 10.0% ACER.
- 5.2.3 Cross Testing: Cross-testing errors decrease by 8.9% on Replay-Attack and 24.6% on CASIA-MFSD relative to the previous state of the art.
- 5.2.4 Visualization and Analysis: Frontalized feature maps assign greater rPPG-estimation influence to the side forehead and cheek regions.The analysis uses 1,080 live and spoof Oulu videos.
- 5.2.4 Visualization and Analysis: Auxiliary estimation is more accurate for spoof data than live data, with greater depth-estimation error around the mouth.The mouth-area error is consistent with fewer activations there in the feature-map analysis.
6. Conclusions
The paper concludes that auxiliary supervision is important for deep face anti-spoofing. Its CNN-RNN network estimates depth and rPPG, while the SiW database broadens subject and variation coverage, and experiments demonstrate superior performance.
- The paper identifies auxiliary supervision as important for deep model-based face anti-spoofing.
- The proposed CNN-RNN network jointly estimates face depth and rPPG signals from face videos.
- The SiW database contains more subjects and variations than prior databases.
- Experiments demonstrate the superiority of the proposed method.