PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing

Zezheng Wang, Zitong Yu, Chenxu Zhao, Xiangyu Zhu, Yunxiao Qin, Qiusheng Zhou, Feng Zhou, Zhen Lei

arXiv:2003.08061v1cs.CV

TL;DR

Face anti-spoofing methods have not fully captured fine-grained spatial cues or the relationship between facial depth and motion. This paper introduces RSGB, STPM, and Contrastive Depth Loss, and evaluates them with the double-modal DMAD dataset. The method is reported to achieve state-of-the-art performance across five benchmark datasets and DMAD.

  • Problem

    Existing face anti-spoofing methods inadequately capture fine-grained patterns and temporal depth information, despite the importance of detecting presentation attacks.

  • Method

    The method combines RSGB for fine-grained spatial details, STPM for spatio-temporal depth information, and Contrastive Depth Loss for depth supervision, alongside the actual-depth DMAD dataset.

  • Results

    The method achieves state-of-the-art performance on widely used face anti-spoofing benchmarks, while ablations report best performance at α = 0.6 for temporal-depth refinement.

  • Takeaways & Limitations

    Fine-grained spatio-temporal information and actual depth maps are presented as effective components for facial-depth estimation in face anti-spoofing.

  • Takeaways & Limitations

    The paper mainly focuses on the planar attack, which is the most common in practice.

Abstract

from arXiv · show

Face anti-spoofing is critical to the security of face recognition systems. Depth supervised learning has been proven as one of the most effective methods for face anti-spoofing. Despite the great success, most previous works still formulate the problem as a single-frame multi-task one by simply augmenting the loss with depth, while neglecting the detailed fine-grained information and the interplay between facial depths and moving patterns. In contrast, we design a new approach to detect presentation attacks from multiple frames based on two insights: 1) detailed discriminative clues (e.g., spatial gradient magnitude) between living and spoofing face may be discarded through stacked vanilla convolutions, and 2) the dynamics of 3D moving faces provide important clues in detecting the spoofing faces. The proposed method is able to capture discriminative details via Residual Spatial Gradient Block (RSGB) and encode spatio-temporal information from Spatio-Temporal Propagation Module (STPM) efficiently. Moreover, a novel Contrastive Depth Loss is presented for more accurate depth supervision. To assess the efficacy of our method, we also collect a Double-modal Anti-spoofing Dataset (DMAD) which provides actual depth for each sample. The experiments demonstrate that the proposed approach achieves state-of-the-art results on five benchmark datasets including OULU-NPU, SiW, CASIA-MFSD, Replay-Attack, and the new DMAD. Codes will be available at https://github.com/clks-wzz/FAS-SGTD.

1. Introduction

Face anti-spoofing protects face-recognition systems from presentation attacks, but existing approaches inadequately capture fine-grained spatial patterns and temporal depth dynamics. The paper addresses these gaps with a depth-supervised spatio-temporal architecture, a contrastive depth loss, and the DMAD dataset.

  • Face anti-spoofing is critical because face-recognition systems can be spoofed by print, replay, and 3D-mask presentation attacks.
  • Binary classifiers distinguish living and spoofing faces but struggle to explore spoofing patterns such as skin-detail loss, color distortion, moiré patterns, and artifacts.
  • Existing depth-supervised methods use single-frame facial-depth estimation, while fine-grained spatial clues and temporal depth dynamics remain insufficiently addressed.
  • The proposed RSGB aggregates convolutional features with spatial gradient magnitude, while STPM propagates spatio-temporal features into depth reconstruction.
  • Contrastive Depth Loss is introduced to learn facial-point topography more effectively for depth-supervised presentation-attack detection.
  • The collected DMAD dataset provides actual depth maps and supports the finding that actual depth is more appropriate than generated depth for monocular presentation-attack detection.
  • The method is reported to achieve state-of-the-art performance on widely used face anti-spoofing benchmarks.

2. Related Work

Prior face anti-spoofing research includes binary-supervised, depth-supervised, and temporal-based methods. The paper’s framework combines fine-grained spatial processing with temporal depth refinement to address limitations of stacked vanilla convolutions and single-frame depth estimation.

  • Binary supervised Methods: Earlier face anti-spoofing methods used hand-crafted descriptors and traditional classifiers, while later approaches commonly trained binary classifiers for spoofing-versus-living recognition.
  • Depth supervised Methods: The proposed framework uses cascaded RSGB blocks and pooling layers to extract multi-level fine-grained spatial features before predicting a coarse depth map for each frame.
  • Temporal-based Methods: Temporal methods exploit facial-part movement, but approaches centered on cues such as eye blinking can be vulnerable to replay attacks.

3. The Proposed Approach

The proposed approach combines fine-grained spatial gradients, multi-frame spatio-temporal features, depth refinement, and Contrastive Depth Loss for depth-supervised face anti-spoofing.

  • Spatio-Temporal Propagation Module: STPM combines adjacent-frame STSTB features with ConvGRU propagation to encode short-term and long-term spatio-temporal information for depth estimation.Temporal depth maps from STPM refine the backbone’s coarse depth maps.
  • Residual Spatial Gradient Block: Sobel-based depthwise convolutions derive horizontal and vertical gradients from input feature maps for spatial-detail representation.The gradients are incorporated into the residual mapping of RSGB.
  • Residual Spatial Gradient Block: RSGB aggregates learnable convolutional features with spatial gradient magnitude through a shortcut connection to capture fine-grained spatial details.It can operate at both image and feature levels for depth regression.
  • Spatio-Temporal Propagation Module: STSTB adaptively fuses current compressed, spatial-gradient, temporal-gradient, and previous-level features to provide weighted spatial and temporal information.Its spatial gradients use Sobel-based depthwise convolution, while temporal gradients use element-wise subtraction.
  • Loss Function: Contrastive Depth Loss supplements Euclidean pixel-wise supervision by supervising distance relationships among adjacent depth values, improving depth-based PAD generality.The CDL design uses eight contrastive convolution kernels, while binary supervision assists depth supervision in the overall loss.

4. Double-modal Anti-spoofing Dataset

DMAD is a real double-modal RGB-and-depth dataset designed to evaluate anti-spoofing under unseen presentation materials. It includes varied display and paper materials, with corresponding depth captured for each sample.

  • DMAD contains RGB and actual depth images captured with a RealSense SR300 camera.The dataset provides corresponding RGB and depth data for each sample.
  • The replay attacks use AMOLED, OLED, and IPS/TFT screens, while print attacks use A4, coated, and poster paper.
  • Figure 6 compares actual and generated depth, with actual depth described as more precise.
  • The dataset includes 300 subjects recorded across three sessions, covering real, print, and replay categories.
  • DMAD contains 2700 videos lasting 4–12 seconds, collected in less than two months with two human workers.

5.1. Databases and Metrics

The experiments use five databases and evaluate performance with protocol-specific error metrics. ACER combines attack and bona fide classification errors, while HTER is used for cross-database testing.

  • Experiments use OULU-NPU, SiW, CASIA-MFSD, Replay-Attack, and DMAD.OULU-NPU and SiW provide multiple protocols, while CASIA-MFSD and Replay-Attack contain low-resolution videos.
  • OULU-NPU, SiW, and DMAD use APCER, BPCER, and ACER to evaluate presentation-attack and bona fide classification errors.ACER is the mean of APCER and BPCER.
  • Cross-database testing between CASIA-MFSD and Replay-Attack uses HTER, the mean of false rejection and false acceptance rates.

5.2. Implementation Details

Training separates coarse depth learning from temporal depth refinement. The implementation samples multiple frames and uses fixed hyperparameters for optimization and depth refinement.

  • Living depth maps are generated with PRNet and normalized to [0, 1], while spoofing depth maps are set to 0 during training.
  • Stage 1 trains the cascaded-RSGB backbone with LEDL and LCDL for coarse depth prediction; Stage 2 freezes it and trains STPM with Loverall.
  • The network processes Nf frames sampled every three frames to preserve temporal information within limited GPU memory.
  • The final score combines sequential depth maps with living logits from fcs(Davg), using facial masks and mean facial depth values.
  • Training uses learning rates of 1e-4 and 1e-2 for the single- and multi-frame parts, batch sizes of 48 and 2, Nf = 5, and α = 0.6, β = 0.8.Adadelta uses ρ = 0.95 and ϵ = 1e-8.
  • Figure 7 reports mean ACER(%) as a red line and standard-deviation ranges as orange and blue areas for varying α.

5.3. Experimental Comparison

Ablations support the contributions of contrastive depth supervision, RSGB, and spatio-temporal modules, while experiments show strong intra- and cross-database generalization. Actual depth also improves DMAD performance over generated depth.

  • 5.3.1. Ablation Study: Contrastive depth loss improves depth estimation, while progressively lower ACER across Models 3–5 supports RSGB, STSTB, and ConvGRU contributions.Binary supervision further assists live-versus-spoof discrimination in the comparison between Models 5 and 7.
  • 5.3.1. Ablation Study: STSTB in Model 7 surpasses the OFF variant by a large margin, indicating that current and future gradient information is valuable for spatio-temporal anti-spoofing.
  • 5.3.1. Ablation Study: α = 0.6 gives the best performance, while incorporating temporal depth information outperforms using spatial information alone.
  • 5.3.1. Ablation Study: ACER is 3.347%, 2.927%, 4.223%, and 2.934% for sampling intervals Δt of 1, 3, 5, and 7 frames, respectively; Δt = 3 is lowest.
  • 5.3.2. Intra-database Testing: The method ranks first on all four OULU-NPU protocols and has the lowest mean and standard deviation of ACER on Protocols 3 and 4.
  • 5.3.2. Intra-database Testing: On SiW, the method generalizes across pose and expression, spoof media, and presentation-attack instruments.
  • 5.3.2. Intra-database Testing: On DMAD, actual-depth supervision lowers multi-frame Model 7 ACER by 1.78 compared with generated-depth supervision.
  • 5.3.3. Cross-database Testing: Cross-database HTER is 17.0% for CASIA-MFSD→Replay-Attack and 22.8% for Replay-Attack→CASIA-MFSD, reductions of 38.4% and 19.7%.

6. Conclusions

The paper proposes fine-grained spatio-temporal facial-depth estimation for face anti-spoofing and reports extensive experiments demonstrating its superiority.

  • RSGB detects discriminative details, while STPM encodes spatio-temporal information for facial depth estimation.
  • Contrastive Depth Loss improves the generality of depth-supervised presentation attack detection.
  • The method also investigates the effectiveness of actual depth maps in face anti-spoofing.

7. Temporal Depth in Face Anti-spoofing

The section explains why facial motion can reveal abnormal relative depth in spoofing scenes. It analyzes print and replay attacks, including moving-carrier cases, and motivates temporal depth supervision for detection.

  • Temporal depth motivation: Temporal facial motion can be used to estimate relative facial depth from the motion of three facial points.The derivation states this holds when d2 ≠ 0.
  • Attack-scene analysis: Print attacks correspond to ∆x = 0, whereas replay attacks correspond to ∆x ≠ 0.
  • Attack-scene analysis: Moving replay carriers make relative-depth estimation inaccurate, and long-term carrier motion increases the resulting abnormality.
  • Depth supervision: For spoofing, the method assigns all-zero depth labels because abnormal relative depth varies over time and is complex to compute directly.Living-face depth labels are constrained to [0, 1].
  • Attack-scene analysis: If recording and realistic image planes are nonparallel, facial motion can expose abnormal relative depth in replay scenes.
  • Discussion: The analysis concludes that temporal depth methods improve face anti-spoofing performance in extensive experiments.
Loading 2003.08061v1…