Source-linked AI summary
Learn Convolutional Neural Network for Face Anti-Spoofing
Jianwei Yang, Zhen Lei, Stan Z. Li
TL;DR
Hand-crafted texture features have limited ability to capture discriminative cues between genuine and fake faces. The paper uses supervised CNN feature learning together with spatial and temporal data augmentation, achieving lower HTERs and improved cross-dataset behavior, while inter-test performance remains unsatisfactory.
Problem
Hand-crafted texture features such as LBP and LBP-TOP cannot capture the most discriminative cues between genuine and fake faces.
Method
The paper learns face anti-spoofing features with a supervised CNN and applies spatial and temporal data augmentation, including enlarged face regions containing background.
Results
HTERs below 5% were achieved on two datasets in intra-test and combination protocols, while inter-test results improved but remained unsatisfactory.
Takeaways & Limitations
CNNs can learn discriminative features for face anti-spoofing, and combined-dataset training produces less bias between the two datasets.
Takeaways & Limitations
Inter-test performance remains unsatisfactory because biases caused by different capturing conditions are inevitable across datasets.
Abstract
from arXiv · showhide
Though having achieved some progresses, the hand-crafted texture features, e.g., LBP [23], LBP-TOP [11] are still unable to capture the most discriminative cues between genuine and fake faces. In this paper, instead of designing feature by ourselves, we rely on the deep convolutional neural network (CNN) to learn features of high discriminative ability in a supervised manner. Combined with some data pre-processing, the face anti-spoofing performance improves drastically. In the experiments, over 70% relative decrease of Half Total Error Rate (HTER) is achieved on two challenging datasets, CASIA [36] and REPLAY-ATTACK [7] compared with the state-of-the-art. Meanwhile, the experimental results from inter-tests between two datasets indicates CNN can obtain features with better generalization ability. Moreover, the nets trained using combined data from two datasets have less biases between two datasets.
I. INTRODUCTION
Face anti-spoofing must distinguish genuine from diverse fake faces despite the absence of obvious visual cues. The paper contrasts manually designed features with CNN-learned features intended to capture more discriminative and general cues.
- Face anti-spoofing addresses print-, replay-, and mask-attacks, but diverse spoofing types make genuine and fake faces difficult to distinguish.
- Randomly sampled REPLAY-ATTACK images illustrate that genuine and fake faces may lack obvious visual differences.
- Existing approaches use manually designed features including LBP, HOG, LBP-TOP, and DoG for face anti-spoofing.
- The paper uses deep CNNs to learn anti-spoofing features in a supervised, data-driven manner instead of designing them manually.
- CNN-learned features are reported to capture more discriminative cues and potentially generalize across spoofing types.
1) Texture-based Anti-Spoofing:
Texture-based anti-spoofing methods use spatial, spatio-temporal, and motion-related cues, including local texture patterns, eye blinking, lip movements, and optical flow.
- Texture-based methods include LBP, LBP-TOP, Fourier spectra, DoG, and retinex-based features for detecting spoofing attacks.
- LBP-TOP outperformed an LBP-based method on the REPLAY-ATTACK database.
- LBP and LBP-TOP were applicable in intra-database testing, but their performance degraded substantially in inter-database testing.
- Motion-based approaches analyze eye blinking, lip dynamics, or optical flow to distinguish live faces from spoofing attacks.
3) 3D Shape-based Anti-Spoofing:
Other anti-spoofing approaches exploit 3D shape, physical motion, scene context, or multispectral reflectance to distinguish genuine and fake faces.
- A 3D projective-invariant method handles unwarped photo attacks but cannot handle warped photos because their coplanar assumption fails.
- Sparse 3D shape recovery has been used to detect photo attacks across none, vertical, and horizontal warping types.
- Multispectral methods increase reflectance differences between genuine and fake faces by selecting suitable illumination bands.
- Multispectral approaches require extra devices to capture images under invisible light.
- Combined approaches integrate motion, texture, liveness, or scene-context cues, though scene context may be absent in some databases.
A. Data Preparation
Data preparation refines face localization and creates spatially augmented inputs that include increasing amounts of background around the face. The pipeline also uses a CNN architecture to process these inputs.
- A. Data Preparation: Face alignment progressively refines detector-provided face locations using face landmarks and cascaded regression.
- 2) spatial augmentation:: Five spatial scales range from face-only inputs to images containing progressively more background regions.
- 2) spatial augmentation:: CASIA fake images may expose photograph boundaries at larger scales, while REPLAY-ATTACK fakes may show blurred edges and abnormal specular reflections.
3) temporal augmentation:
The paper augments CNN training temporally with single and consecutive multiple frames, aiming to let the network learn both spatial and temporal anti-spoofing features. It uses a canonical CNN and classifies features extracted from its last fully connected layer with an SVM.
- 3) temporal augmentation:: Temporal augmentation trains CNN models with both single frames and multiple consecutive frames to test whether additional frames improve anti-spoofing.More than one frame is expected to provide more informative data and expose temporal features.
- 3) temporal augmentation:: The CNN is designed to learn spatial features from individual images and temporal features when multiple frames are provided.
- CNN structure: The network follows a canonical structure with five convolutional layers followed by three fully connected layers.Response-normalization layers follow the first two convolutional layers, while max-pooling processes outputs from the first, second, and last convolutional layers.
- Classification: Features are extracted from the last fully connected layer and used to train SVM classifiers for face anti-spoofing.The paper uses the LibSVM toolkit for classifier training.
IV. EXPERIMENTS
Experiments evaluate the method on CASIA and REPLAY-ATTACK, two 50-subject datasets containing multiple genuine conditions and spoofing types. Inputs are spatially enlarged to include background and temporally augmented with consecutive frames before resizing.
- Input preparation: Face regions are detected and refined using landmark-based bounding boxes, then enlarged with scaling ratios {1.4, 1.8, 2.2, 2.6} to include background information.All input images are resized to 128 × 128, and consecutive frames provide temporal augmentation.
- Datasets: The experiments use CASIA and REPLAY-ATTACK, both containing 50 subjects and multiple simulated spoofing types.CASIA has 600 sequences, while REPLAY-ATTACK has 1200 sequences.
- CASIA: CASIA includes warped-photo, cut-photo, and electronic-screen attacks across three qualities.
- REPLAY-ATTACK: REPLAY-ATTACK includes print, digital-photo, and video attacks captured under fixed and handheld conditions.
C. Protocol
Performance is evaluated with HTER using development-set thresholds selected at EER, with five-fold cross-validation on CASIA. The experiments include intra-test, inter-test, and combined-dataset evaluations.
- C. Protocol: The paper reports Half Total Error Rate (HTER), selecting the threshold corresponding to Equal Error Rate (EER) on the development set.That threshold is then applied to compute HTER on the test set.
- C. Protocol: CASIA results average five cross-validations, with one fold used for development and the remaining four for training.
- Evaluation settings: The evaluation includes intra-test experiments within each dataset, inter-test experiments across datasets, and training with combined data from both datasets.
- CASIA evaluation: On CASIA, five spatial scales and one-to-three input frames are compared, and scale 2, 3, and 4 improve performance consistently across frame settings.The results indicate that background regions help distinguish genuine and fake faces, although excessive background can reduce the improvement.
- CASIA evaluation: On CASIA, the best average performance occurs at spatial scale 3, while models trained with one frame slightly outperform models using more frames on average.
2) Test on REPLAY-ATTACK dataset:
REPLAY-ATTACK evaluation compares 15 scale-and-frame scenarios and reports strong within-dataset performance, while cross-dataset testing remains substantially harder. Background-inclusive inputs improve inter-test results, but dataset biases limit transferability.
- 2) Test on REPLAY-ATTACK dataset:: REPLAY-ATTACK is evaluated under 15 scenarios varying image scale and the number of frames.
- 2) Test on REPLAY-ATTACK dataset:: On REPLAY-ATTACK, test-set accuracies exceed 97% across models, and scale 5 achieves nearly perfect anti-spoofing performance.Development-set EERs are larger than test-set HTERs because of bias between the development and test sets.
- 3) Comparison:: Compared with prior hand-crafted-feature results, the method reduces the lowest CASIA EER from 21.59 to 4.64 and achieves a lowest HTER more than four times smaller.The paper reports similarly drastic improvements on REPLAY-ATTACK.
- E. Results of Inter-test: In inter-test evaluation, performance is analogous to prior work at scale 1 but improves as scale increases, reaching HTER 23.78 on REPLAY-ATTACK with one frame at scale 5.On CASIA, the lowest reported HTER is 38.11 using three frames at scale 4.
- E. Results of Inter-test: Cross-dataset anti-spoofing remains far from satisfactory because devices, illumination, races, and other factors create biases between datasets.The experiments nevertheless find that background regions can improve the model’s generalization ability.
F. Results on Combined Datasets
Training on combined CASIA and REPLAY-ATTACK data yields strong performance on both datasets and reduces dataset-specific bias. The models perform similarly to intra-test models, indicating CNNs learn common features across datasets.
- Models trained and developed on combined datasets are evaluated separately on CASIA and REPLAY-ATTACK under the data-combination protocol.
- ROC curves compare inter-test models trained on CASIA with REPLAY-ATTACK evaluation and models trained on REPLAY-ATTACK with CASIA evaluation.
- The proposed method achieves much better results on both datasets than the method in.
- On REPLAY-ATTACK, average HTERs are less than 1% when scale = 4 and 5.
- Combined-data models perform similarly to intra-test models, suggesting CNNs learn common features from both datasets.
- CNN feature learning makes performance less biased between the two datasets.
G. Discussion
The discussion attributes performance gains to CNN-based feature learning and spatial and temporal data augmentation. Background regions help anti-spoofing, but the useful input scale differs between CASIA and REPLAY-ATTACK because their backgrounds differ.
- CNN-based data-driven feature design avoids manually specified empirical features and produces more efficient anti-spoofing features.
- Spatial and temporal data augmentation further improves face anti-spoofing performance across all protocols.
- Background regions help face anti-spoofing when using CNN-learned or hand-crafted features.
- The best input scale is 3 on CASIA but 5 on REPLAY-ATTACK.
- CASIA contains real-world backgrounds in real-access sequences, whereas REPLAY-ATTACK backgrounds are filled by fake photos.
- At overly large scales, CASIA genuine and fake samples become more similar, while REPLAY-ATTACK samples become more discriminative.
V. CONCLUSIONS AND FUTURE WORKS
The paper proposes CNN-based feature learning with data augmentation for face anti-spoofing and reports improvements across testing protocols. However, inter-test performance remains unsatisfactory because dataset biases persist, motivating transfer learning and additional motion or shape cues.
- CNN-based feature learning with data augmentation significantly improves performance across intra-test, inter-test, and combination protocols.
- In intra-test and combination protocols, HTERs are lower than 5% on both datasets.
- Inter-test performance improves remarkably but remains unsatisfactory because different capturing conditions create inevitable dataset biases.
- Future work includes adapting learned models to new data through transfer learning and integrating motion and shape cues.