PaperlayerView exact revision on arXiv

Source-linked AI summary

PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognition

Chien-Yi Wang, Yu-Ding Lu, Shang-Ta Yang, Shang-Hong Lai

arXiv:2203.14325v1cs.CV

TL;DR

Face anti-spoofing models remain vulnerable to limited dataset variation and often overlook local capture characteristics. PatchNet recognizes fine-grained patch types defined by capture devices and presenting materials, using non-distorted patches and embedding-space regularization. It achieves strong performance across intra-dataset, cross-dataset, and domain-generalization benchmarks while supporting few-shot reference-based applications.

  • Problem

    Limited dataset scale and variation leave binary FAS models vulnerable to collection biases and unseen testing scenarios, while local capture characteristics remain underused.

  • Method

    PatchNet classifies non-distorted face patches into fine-grained types defined by capturing devices and presenting materials, using asymmetric margin-based and self-supervised similarity losses.

  • Results

    PatchNet achieves state-of-the-art performance under most intra-dataset, cross-dataset, and domain-generalization testing scenarios.

  • Takeaways & Limitations

    The learned patch embedding space supports Few-Shot reference FAS and patch-type retrieval applications.

Abstract

from arXiv · show

Face anti-spoofing (FAS) plays a critical role in securing face recognition systems from different presentation attacks. Previous works leverage auxiliary pixel-level supervision and domain generalization approaches to address unseen spoof types. However, the local characteristics of image captures, i.e., capturing devices and presenting materials, are ignored in existing works and we argue that such information is required for networks to discriminate between live and spoof images. In this work, we propose PatchNet which reformulates face anti-spoofing as a fine-grained patch-type recognition problem. To be specific, our framework recognizes the combination of capturing devices and presenting materials based on the patches cropped from non-distorted face images. This reformulation can largely improve the data variation and enforce the network to learn discriminative feature from local capture patterns. In addition, to further improve the generalization ability of the spoof feature, we propose the novel Asymmetric Margin-based Classification Loss and Self-supervised Similarity Loss to regularize the patch embedding space. Our experimental results verify our assumption and show that the model is capable of recognizing unseen spoof types robustly by only looking at local regions. Moreover, the fine-grained and patch-level reformulation of FAS outperforms the existing approaches on intra-dataset, cross-dataset, and domain generalization benchmarks. Furthermore, our PatchNet framework can enable practical applications like Few-Shot Reference-based FAS and facilitate future exploration of spoof-related intrinsic cues.

1. Introduction

PatchNet reframes face anti-spoofing as fine-grained recognition of local patch capture characteristics, addressing dataset bias and limited variation. It combines patch-based learning with asymmetric and self-supervised losses, achieving strong generalization across benchmark settings.

  • Face anti-spoofing secures face recognition systems against physical presentation attacks.
  • Limited dataset scale and variation make binary FAS models vulnerable to collection biases and unseen testing scenarios.
  • PatchNet reformulates FAS as fine-grained recognition of capturing devices and presenting materials from local face patches.
  • PatchNet is evaluated on intra-dataset, cross-dataset, and domain-generalization benchmarks, achieving state-of-the-art performance under most testing scenarios.
  • Asymmetric Margin-based Softmax and Self-supervised Similarity losses regularize the patch embedding space for asymmetric live–spoof classes and within-capture invariance.
  • The learned patch embedding space supports Few-Shot reference FAS and patch-type retrieval applications.

2. Related Works

Related FAS research improves generalization through auxiliary supervision and domain generalization, including disentanglement and asymmetric treatment of live and spoof samples.

  • Auxiliary-based FAS methods use depth, rPPG, reflection maps, and related pixel-wise signals to guide feature learning toward generalizable cues.
  • Domain-generalization methods simulate target-domain shifts, learn domain-independent representations, or treat live and spoof samples asymmetrically.
  • PatchNet is related to approaches that learn robust local features for both high- and low-resolution scenarios.

3.1. Overview

PatchNet extracts non-distorted patches, assigns classes from capture devices and presenting materials, and learns normalized patch features with asymmetric angular-margin supervision.

  • PatchNet transforms original face images into patch inputs, encodes and normalizes their features, and splits classes by presenting material and capture device.
  • CASIA-FASD yields nine patch types from two spoof mediums and three capture resolutions: three live and six spoof classes.
  • Angular-margin softmax encourages compact feature clusters, while asymmetric margins compact live features more strongly than spoof features.

3.2. Patch Features Extraction

PatchNet generates two non-distorted augmented views from each raw face crop and passes them through an encoder and normalization layer to obtain patch features.

  • The augmentation sequence uses only random horizontal flips, random rotations, and fixed-size cropping to avoid distorting spoof-related information.
  • Two augmented views of each cropped face region are passed through encoder Eθ and normalization to produce final patch features.

3.3. Fine-Grained Patch Recognition

PatchNet formulates face anti-spoofing as fine-grained recognition over patch types, using angular-margin classification to regularize patch features. Its asymmetric formulation applies different margins to live and spoof categories and combines recognition losses across augmented patch views.

  • Fine-Grained Patch Recognition: PatchNet assigns each patch to a fine-grained live or spoof class and applies angular-margin softmax to regularize patch features.The classes comprise k live and N−k spoof patch types.
  • AM-Softmax Preliminaries: Angular-margin softmax imposes a margin between feature and weight vectors after feature and weight normalization.The formulation uses normalized feature and weight vectors and a margin function ψ(θ).
  • Asymmetric AM-Softmax Loss: The asymmetric loss uses separate angular margins m_l and m_s for live and spoof categories, respectively.The loss selects the live margin when y_i belongs to L and the spoof margin when y_i belongs to S.
  • Asymmetric AM-Softmax Loss: The final asymmetric recognition loss averages the asymmetric AM-Softmax losses from two augmented patch views.This applies the loss to f_i^t1 and f_i^t2 for each sample.

3.4. Self-Supervised Similarity Loss

PatchNet applies a self-supervised similarity constraint to two patch views from the same face image. This encourages similar features across the views and supports patch-location and rotation invariance in spoof-related representations.

  • Self-Supervised Similarity Loss: The similarity constraint compares two patch views from the same face image and encourages their features to be similar.The loss is the average L2 distance between the two feature vectors.
  • Self-Supervised Similarity Loss: The constraint is intended to make spoof-related features invariant to patch location and rotation.The stated invariance follows from applying the constraint to different patch views.

3.5. Training and Testing

Training combines the Asymmetric Recognition Loss and Self-supervised Similarity Loss, while testing averages live probabilities across uniformly cropped patches. The two training-loss weights are set equally in the reported experiments.

  • Training: The total training loss is a weighted sum of the Asymmetric Recognition Loss and Self-supervised Similarity Loss.The formulation is L = α1 L_Asym + α2 L_Sim.
  • Training: The experiments set both loss weights to 1.0.Specifically, α1 = α2 = 1.0.
  • Testing: At inference, the network uniformly crops patches from the whole test face using the training patch size.The resulting patch features are used to compute the face-level score.
  • Testing: The face-level live probability averages the live-class probabilities from the last fully connected layer across P cropped patches.The score sums the live probabilities for each patch and divides by P.

4. Experiments

Experiments evaluate PatchNet across intra-dataset, cross-dataset, and ablation settings using standard FAS protocols and error metrics. Results show strong generalization across OULU-NPU and SiW, while ablations identify benefits from fine-grained patch recognition, raw cropping, and regularization.

  • Datasets and protocols: Five databases support intra-dataset and cross-dataset evaluation, with APCER, BPCER, ACER, HTER, and AUC used according to the testing protocol.OULU-NPU and SiW support intra-dataset testing; CASIA-MFSD, Replay-Attack, and MSU-MFSD support cross-dataset testing.
  • Intra-dataset results: PatchNet achieves the best performance across all four OULU-NPU protocols targeting unseen environments, spoof mediums, capture devices, and their combination.The protocols contain 5, 3, 5, and 3 training classes, respectively.
  • Intra-dataset results: PatchNet performs best on SiW protocols 1 and 2 and remains competitive on protocols 3-1 and 3-2.The fine-grained patch-type classes number 14, 8, 6, and 10 across these protocols.
  • Ablation study: On OULU-NPU protocol 1, fine-grained classes with raw-frame cropping reduce ACER from 6.25% for the resized binary baseline to 1.88%.The ablation attributes the improvement to avoiding dataset bias and preserving fine details needed for patch-type discrimination.
  • Ablation study: Patch crop sizes that are too small, such as 64, degrade performance, while regularization makes performance relatively stable as patch size increases.Larger patches can overfit capture biases, whereas very small patches lack sufficient information to learn capture characteristics.
  • Ablation study: Angular margins improve generalization over no margin, but overly large spoof margins hurt discrimination; PatchNet uses m_l = 0.4 and m_s = 0.1.These margin values work well across all testing protocols.

4.5. Cross-Dataset Testing

PatchNet is evaluated across cross-dataset and domain-generalization settings by modeling fine-grained patch types and capture-device characteristics. The analyses show that margin-based patch embeddings separate spoof types, reveal device-dependent behavior, and support device-specific and few-shot applications.

  • Cross-Dataset Testing: PatchNet achieves competitive cross-dataset performance on both C→I and I→C protocols, although I→C remains harder because ReplayAttack has limited clips and low-quality videos.The protocols use HTER, with 9 and 4 fine-grained patch-type classes in C→I and I→C, respectively.
  • Cross-Dataset Testing: Fine-grained class splitting, LAsym, and LSim each improve generalization on challenging cross-dataset protocols.The ablation compares the proposed split with an SSDG-style four-class split that aggregates live samples and groups spoof samples by dataset.
  • Patch Feature Distribution: Margin training produces more compact live features and better-separated patch-type clusters, while training without margin poorly distinguishes print1 from print2.These patterns are visualized with t-SNE features from OULU-NPU Protocol 1.
  • Patch Feature Distribution: Cross-dataset embeddings separate device-specific live classes, with high-quality C3 live features located closer to high-quality O live features.The visualization concerns normalized patch features in the M&I&O→C protocol, where C contains three capture-device qualities.
  • Patch Score Map: Patch scores remain consistent across most face regions, while background and boundary areas show exceptions, supporting localization of spoof cues in facial regions.The scores are visualized as live-probability heat maps for five OULU-NPU Protocol 1 patch types.
  • Testing on each Capture Device: Device-level testing reveals that C2 noise and compression degrade feature discrimination, whereas only high-quality C3 exceeds the average performance on dataset C.Reporting performance by device is presented as a way to guide system improvement or improve problematic capture-device quality.
  • Few-Shot Reference Anti-Spoofing: Few-shot live references enable reference-based anti-spoofing by comparing normalized patch embeddings with cosine distance, and the framework can boost performance in practical scenarios.The paper reports 5-shot and 10-shot live-reference testing and describes higher similarity scores as indicating greater likelihood of live samples.
  • Patch Type Retrieval: The normalized patch embedding space also supports retrieving training patch types for a new test sample using cosine similarity.The example trains on M&C&I to recognize 21 patch types before retrieving patch types for a live O-dataset query.

5. Conclusions and Future Work

The paper concludes that PatchNet reformulates face anti-spoofing as fine-grained patch-type recognition and learns an embedding space for spoof-related capture characteristics. Experiments support the method, while future work targets broader embedding spaces and more practical few-shot protocols.

  • Conclusions: PatchNet learns a patch embedding space that encodes spoof-related capture characteristics through fine-grained patch-type recognition.The framework is presented as a simple training approach, with novel loss functions designed to enhance feature discrimination.
  • Conclusions: Extensive experiments on challenging face anti-spoofing protocols verify the effectiveness of the proposed method.
  • Future Work: Future work includes learning more generalized embeddings from more varied datasets or material-perception transfer and investigating practical few-shot FAS protocols.The paper notes that generic embedding-space exploration remains at an early stage.
Loading 2203.14325v1…