Source-linked AI summary
Face Anti-Spoofing with Human Material Perception
Zitong Yu, Xiaobai Li, Xuesong Niu, Jingang Shi, Guoying Zhao
TL;DR
Face anti-spoofing needs robust discrimination between live skin and physical presentation attacks, while existing generic features may not capture fine-grained spoofing patterns. The paper reframes FAS as material recognition and introduces BCN with MFRM and multi-head supervision; it reports strong intra- and cross-dataset performance across six benchmarks.
Problem
Existing CNN and handcrafted features are designed for universal image recognition and might not represent fine-grained spoofing patterns in face anti-spoofing.
Method
The paper combines human material perception with BCN, MFRM, and material-based multi-head supervision to learn intrinsic material-based features for FAS.
Results
The method achieves outstanding performance on six benchmark datasets under both intra- and cross-dataset testing protocols.
Takeaways & Limitations
The work connects face anti-spoofing with material recognition and supports using bilateral macro- and micro-features for FAS.
Takeaways & Limitations
The approach assumes that structural material discrepancies between facial skin and physical spoofing carriers are the essence of distinguishing live from spoofing faces.
Abstract
from arXiv · showhide
Face anti-spoofing (FAS) plays a vital role in securing the face recognition systems from presentation attacks. Most existing FAS methods capture various cues (e.g., texture, depth and reflection) to distinguish the live faces from the spoofing faces. All these cues are based on the discrepancy among physical materials (e.g., skin, glass, paper and silicone). In this paper we rephrase face anti-spoofing as a material recognition problem and combine it with classical human material perception [1], intending to extract discriminative and robust features for FAS. To this end, we propose the Bilateral Convolutional Networks (BCN), which is able to capture intrinsic material-based patterns via aggregating multi-level bilateral macro- and micro- information. Furthermore, Multi-level Feature Refinement Module (MFRM) and multi-head supervision are utilized to learn more robust features. Comprehensive experiments are performed on six benchmark datasets, and the proposed method achieves superior performance on both intra- and cross-dataset testings. One highlight is that we achieve overall 11.3$\pm$9.5\% EER for cross-type testing in SiW-M dataset, which significantly outperforms previous results. We hope this work will facilitate future cooperation between FAS and material communities.
1 Introduction
The paper reframes face anti-spoofing as structural material recognition, motivated by material differences between facial skin and physical spoofing carriers. It introduces BCN with MFRM and multi-head supervision to learn robust material-based features.
- Presentation attacks such as print, replay, 3D masks, and makeup threaten face-recognition systems, motivating face anti-spoofing.
- Existing FAS methods use cues including rPPG, depth, reflection, color texture, moiré patterns, and noise artifacts to distinguish live from spoofing faces.
- The paper identifies structural material differences between human facial skin and carriers such as paper, glass screens, and resin masks as central to FAS.
- Face anti-spoofing is reformulated as structural material recognition to learn intrinsic and robust features distinguishing facial skin from spoofing-carrier materials.
- BCN aggregates multi-level bilateral macro- and micro-information, while MFRM refines features and multi-head supervision promotes robust shared representations.
- Experiments cover six benchmark datasets using intra- and cross-dataset protocols, with additional fine-grained material-recognition experiments on SiW-M.
2 Related Work
Prior FAS research includes handcrafted and deep-learning approaches, while material-perception research studies optical and mechanical properties. The paper connects these areas because surface reflectance alone is not always reliable for generalized material-based FAS.
- Traditional FAS methods use handcrafted descriptors such as LBP, SIFT, SURF, HOG, and DoG, while video methods capture dynamic texture, micro-motion, and eye blinking.
- Human material perception studies optical and mechanical properties, including gloss, roughness, and statistical appearance beyond color and lightness.
- The paper positions its framework around BCN, MFRM, and multi-head supervision as the overall architecture for material-based FAS.
- Prior FAS work considers discrepant surface reflectance, but reflectance alone is not always reliable for material perception.
3 Methodology
The methodology integrates bilateral filtering into deep networks to represent material-based macro- and micro-cues for face anti-spoofing. BCN, MFRM, and multi-head supervision jointly refine and supervise multi-level features.
- Bilateral Convolutional Networks: Bilateral filtering decomposes images into base and residual representations, exposing material-dependent differences between live and spoofing faces.The base captures large-scale structure, while the residual captures detail; print attacks appear rougher and less glossy, with stronger high-frequency activation around eyes and eyebrows.
- Bilateral Convolutional Networks: BCN integrates deep bilateral operators into hidden feature maps to learn multi-level bilateral base and residual features without replacing the original inputs.This design addresses information loss from input replacement and the inefficiency of applying bilateral filtering only in input space.
- Multi-level Feature Refinement Module: MFRM refines and fuses coarse low-, mid-, and high-level BCN features using context-aware local reassembling.It reallocates neighboring contributions according to semantic and contextual information, targeting more intrinsic material cues.
- Bilateral Convolutional Networks: The BCN architecture combines ConvBlock and BilateralConvBlock outputs at corresponding levels through element-wise addition.The architecture uses 3×3 filters, with stride 1 for convolutional layers and stride 2 for pooling layers.
- Material based Multi-head Supervision: Material-based multi-head supervision guides fused features through depth, reflection, and patch heads for structural shape, reflectance, and fine-grained texture cues.The corresponding predictions are supervised with depth, reflection, and patch losses whose sum forms the overall training loss.
4 Experiments
Experiments evaluate BCN, MFRM, and multi-head supervision across six datasets using intra-, cross-type, and cross-dataset protocols. The method consistently achieves strong performance, including 11.3% EER and 11.2% ACER on SiW-M cross-type testing.
- Experimental Setup: Experiments use six datasets, with OULU-NPU and SiW for intra testing and CASIA-MFSD, Replay-Attack, and MSU-MFSD for cross testing.SiW-M is additionally used for cross-type testing.
- Ablation Study: BCN reduces ACER from 4.1% for the RGB baseline to 2.1%, while removing its bilateral operator increases ACER to 4.3%.Removing the bilateral residual structure raises ACER from 2.1% to 4.0%, highlighting the contribution of bilateral residual features.
- Ablation Study: The full D+R+P+MFRM+BCN model achieves 0.8% ACER, compared with 4.1% for the baseline.Multi-head supervision and MFRM are reported to reduce ACER to 1.2% before the full model reaches 0.8%.
- Intra-Dataset Testing: The method ranks first or second across all four OULU-NPU protocols, with ACER values of 0.4%, 1.7%, 2.5%, and 5.2%.On SiW, it achieves the best results across all three protocols, with ACER values of 0.36%, 0.11%, and 2.45%.
- Cross-Dataset and Cross-Type Testing: 11.3% EER and 11.2% ACER are achieved on SiW-M cross-type testing across 13 attacks, while cross-dataset HTER reaches 16.6% on protocol CR and 36.4% on protocol RC.The method also achieves 96.77% AUC in cross-type testing across CASIA-MFSD, Replay-Attack, and MSU-MFSD.
- Material Recognition: Fine-grained material recognition accuracy improves by 1.9% with BCN and by an additional 1.7% with MFRM on SiW-M.The five categories are live, replay, print, mask, and makeup.
5 Conclusions
The paper reframes face anti-spoofing as material recognition and proposes BCN to capture bilateral macro- and micro-features. It reports extensive experiments and identifies disentangling material features from material-unrelated features as future work.
- Conclusion: The paper reframes FAS as material recognition and proposes BCN to capture material-based bilateral macro- and micro-features.The approach combines FAS with classical human material perception.
- Future Work: Future work includes disentangling intrinsic material features from material-unrelated features.The supplied conclusion passage ends while giving face-related examples of material-unrelated features.
A. Impact of Spatial Neighborhood Distance in DBO
This ablation evaluates the spatial neighborhood distance term σs in the deep bilateral operator. Adding the term does not improve ACER under the reported setting, while the authors note that the study may be sub-optimal.
- Ablation Result: Adding the spatial neighborhood distance term produces 2.4% ACER, compared with 2.1% without it.The ablation uses σr = 1.0.
- Limitation: The authors state that the σs ablation is insufficient and may yield a sub-optimal solution.They propose strict grid search and exploration of long-range effects under larger kernels as future work.
B. Network Details of Multi-head Supervision
The multi-head supervision network uses depth, reflection, and patch cues to guide learning of holistic material-based features. Its detailed convolutional structure is shown in Fig. 9.
- Network Structure: The backbone uses supervision from three cue types to learn more holistic material-based features.Fig. 9 reports convolution filter counts above each convolution, with 3×3 filters and stride 1.
C. Intra Testing Results on SiW
The proposed method performs best across all three SiW intra-testing protocols, demonstrating strong generalization capacity.
- The proposed method performs the best on all three SiW intra-testing protocols.The results are reported for the three protocols shown in Table 7.
D. Cross-type Testing on CASIA-MFSD, Replay-Attack and MSU-MFSD
Cross-type testing evaluates generalization between replay and print attacks across three datasets. The proposed method achieves the best overall performance with 96.77% AUC.
- Cross-type testing uses CASIA-MFSD, Replay-Attack, and MSU-MFSD to evaluate transfer between replay and print attacks.Models are trained on some attack types and tested on another, such as training on cut and wrapped photos and testing on video.
- 96.77% AUC is the proposed method’s best overall performance in cross-type testing.The evaluation metric reported for Table 8 is AUC (%).
- The results indicate that the learned features generalize well among unknown attacks.