Source-linked AI summary
Face Anti-Spoofing Via Disentangled Representation Learning
Ke-Yue Zhang, Taiping Yao, Jian Zhang, Ying Tai, Shouhong Ding, Jilin Li, Feiyue Huang, Haichuan Song, Lizhuang Ma
TL;DR
Face anti-spoofing needs representations that distinguish spoof patterns from person-related content across varied attacks. This paper disentangles liveness and content features, using texture and depth supervision to regularize liveness. Evaluations on public databases and visual analyses report effectiveness against state-of-the-art competitors.
Problem
Face anti-spoofing must address varied presentation attacks, while extracted features may entangle spoof patterns with person-related content.
Method
The method separates facial representations into liveness and content features and combines low-level texture with high-level depth supervision to regularize liveness.
Results
Evaluations on widely used face anti-spoofing databases and visual analyses demonstrate effectiveness against state-of-the-art competitors.
Takeaways & Limitations
Liveness-focused representations provide a basis for analyzing and classifying face anti-spoofing features separately from identity, background, and lighting information.
Abstract
from arXiv · showhide
Face anti-spoofing is crucial to security of face recognition systems. Previous approaches focus on developing discriminative models based on the features extracted from images, which may be still entangled between spoof patterns and real persons. In this paper, motivated by the disentangled representation learning, we propose a novel perspective of face anti-spoofing that disentangles the liveness features and content features from images, and the liveness features is further used for classification. We also put forward a Convolutional Neural Network (CNN) architecture with the process of disentanglement and combination of low-level and high-level supervision to improve the generalization capabilities. We evaluate our method on public benchmark datasets and extensive experimental results demonstrate the effectiveness of our method against the state-of-the-art competitors. Finally, we further visualize some results to help understand the effect and advantage of disentanglement.
1 Introduction
Face anti-spoofing must handle varied presentation attacks and poor generalization from entangled representations. The paper proposes separating liveness from content features and regularizing liveness with texture and depth supervision.
- 1 Introduction: The method addresses presentation attacks ranging from printed images and video replays to 3D masks and cosmetic makeup.
- 1 Introduction: Disentanglement targets liveness information by decomposing facial representations into liveness and content spaces.
- 1 Introduction: A GAN-like discriminator and auxiliary depth estimator address missing paired images and encourage liveness information exchange during translation.
- 1 Introduction: Low-level texture and high-level depth characteristics regularize the liveness space and facilitate disentangled representation learning.
- 1 Introduction: Experiments and visualizations on widely used databases report effectiveness against state-of-the-art competitors.
2 Related Work
Related work progresses from handcrafted and deep classification features toward auxiliary supervision and disentangled representations. The paper builds on decomposing facial representations into content and liveness factors while addressing recombination and spoof-style challenges.
- 2 Related Work: Handcrafted descriptors and domain-specific cues support traditional anti-spoofing, but they struggle with high-fidelity replay attacks.
- 2 Related Work: CNN-based binary classifiers improve intra-testing accuracy but show relatively poor generalization because of training-data overfitting.
- 2 Related Work: Auxiliary depth, r-PPG, spoof-noise, and domain-generalization supervision has been used to improve anti-spoofing generalization.
- 2 Related Work: Disentangled representation methods factorize data into informative variation factors, including content and style or separate facial attributes.
- 2 Related Work: This paper applies that idea by separating liveness from identity, background, and lighting while confronting absent recombined-image ground truth and diverse spoof styles.
3 Disentanglement Framework
The framework disentangles each face image into liveness and content features, exchanges liveness features between unpaired live and spoof images, and uses auxiliary supervision to enforce plausible, category-consistent reconstructions.
- 3.1 Disentanglement Process: The framework separates image representations into independent liveness and content features, with content retaining information such as identity, background, and lighting.The disentanglement process uses separate convolutional subnetworks to encode latent codes into liveness and content features.
- 3.1 Disentanglement Process: Liveness-feature exchange recombines features from live image A and spoof image B to generate Ab and Ba with altered liveness attributes while preserving content.The method treats Ab as a spoof version of A and Ba as a genuine version of B, then re-encodes the generated images to encourage complete separation.
- 3.1 Disentanglement Process: Bidirectional reconstruction constrains both image reconstruction and latent reconstruction after decoding and re-encoding exchanged liveness-content pairs.The encoder-decoder reconstructs dataset images and restores combined liveness and content features after translation.
- 3.2 Auxiliary Supervision: Texture, depth, and discriminative auxiliary supervision regularize liveness features, enforce live-versus-spoof category consistency, and improve generated-image plausibility.LBP supervision uses live-face texture maps and zero maps for spoof faces, while depth supervision uses pseudo-depth for live faces and zero maps for spoof faces.
- 3.2 Auxiliary Supervision: The final training objective combines the framework’s loss terms through a weighted sum, with discriminator and disentanglement networks optimized alternately.The loss weights are empirically selected to balance the individual terms.
4 Experimental Results
Experiments on four face anti-spoofing databases evaluate intra-dataset and cross-dataset generalization. Results support disentanglement, auxiliary supervision, and fused LBP-depth scores, with visualizations indicating that liveness features capture live/spoof distinctions while excluding content factors.
- Intra-Testing: The method achieves better results on Oulu-NPU protocols 1, 3, and 4, with the best results on protocol 4, while slightly underperforming on protocol 2.Protocol 4 evaluates all Oulu-NPU variations and is used to assess generalization performance.
- Cross-Testing: 1.2 pp lower HTER than the state of the art is obtained from CASIA-MFSD to Replay-Attack, with comparable HTER in the reverse direction.Cross-testing uses CASIA-MFSD and Replay-Attack and compares methods using only single-frame information.
- Translation Result: Exchanging liveness features changes depth maps for live-spoof translations but not live-live or spoof-spoof translations, while light, identity, and background remain unchanged.These translations indicate that liveness features determine live/spoof attributes without containing the listed content factors.
- Translation Result: Translated images retain attack-specific local details, including repetitive streaks, showing that liveness features encode both real-versus-attack differences and different attack details.Figure 4 compares original images with translation results and marks live and spoof details using red and blue rectangles.
- Liveness Feature Distribution: t-SNE visualizations show that disentanglement makes live and spoof features easier to distinguish, LBP supervision improves discrimination, and content features do not separate real from attack.The visualization uses 500 live images and 2,000 spoof images.
- Ablation Study: LBP texture and depth supervision outperform same-supervision combinations, while fusing LBP-map and depth-map scores achieves the best performance.The reported fusion score is score = (∥maplbp∥+ ∥mapdepth∥)/2.
5 Further Exploration
Qualitative analyses show that liveness features cluster by spoof type, while attack representations also retain collection-device information. Delta maps further distinguish real-versus-spoof exchanges, attack types, and devices.
- Spoof Type: Liveness features form distinct clusters for live, paper-attack, and screen-attack images across devices.
- Collection Equipment: Attack liveness features cluster across devices, whereas live-person features are scattered and may be less related to collection equipment.
- Delta Maps: Delta maps are nearly zero between real faces but larger between live and spoof images, distinguishing attack types and varying with collection equipment.
6 Conclusions
The paper presents face anti-spoofing as a disentanglement problem, separating liveness from content features with an architecture using multiple supervision signals. It reports strong results on widely used databases and visualizes feature translation and liveness properties.
- The method disentangles liveness and content features from face images for face anti-spoofing.
- Its architecture combines low-level texture and high-level depth characteristics to regularize the liveness space.
- Evaluations on widely used face anti-spoofing databases achieve outstanding results, while visualizations analyze translation and liveness features.