Source-linked AI summary
Deep Tree Learning for Zero-shot Face Anti-Spoofing
Yaojie Liu, Joel Stehouwer, Amin Jourabloo, Xiaoming Liu
TL;DR
Face anti-spoofing must detect attack types unseen during training, but prior ZSFA evidence covered only limited attack varieties and had methodological drawbacks. The paper studies 13 attack types, introduces SiW-M, and proposes an unsupervised Deep Tree Network that routes samples to spoof-specific leaf classifiers. Experiments report state-of-the-art performance across multiple ZSFA testing protocols.
Problem
ZSFA detects spoof attacks that are unknown during training, while prior studies focused on only print and replay attacks and had limited spoof knowledge and handcrafted feature selection.
Method
The Deep Tree Network unsupervisedly partitions known spoof data into semantic sub-groups, routes samples to leaf nodes, and performs binary spoof detection there.
Results
The method achieves state-of-the-art performance on multiple ZSFA testing protocols.
Takeaways & Limitations
The work expands ZSFA evaluation to 13 spoof attack types and provides SiW-M as a diverse database for studying unknown attacks.
Takeaways & Limitations
The node-routing objective can suffer from trivial solutions, outlier sensitivity, local minima, and collapse to a few leaf nodes.
Abstract
from arXiv · showhide
Face anti-spoofing is designed to keep face recognition systems from recognizing fake faces as the genuine users. While advanced face anti-spoofing methods are developed, new types of spoof attacks are also being created and becoming a threat to all existing systems. We define the detection of unknown spoof attacks as Zero-Shot Face Anti-spoofing (ZSFA). Previous works of ZSFA only study 1-2 types of spoof attacks, such as print/replay attacks, which limits the insight of this problem. In this work, we expand the ZSFA problem to a wide range of 13 types of spoof attacks, including print attack, replay attack, 3D mask attacks, and so on. A novel Deep Tree Network (DTN) is proposed to tackle the ZSFA. The tree is learned to partition the spoof samples into semantic sub-groups in an unsupervised fashion. When a data sample arrives, being know or unknown attacks, DTN routes it to the most similar spoof cluster, and make the binary decision. In addition, to enable the study of ZSFA, we introduce the first face anti-spoofing database that contains diverse types of spoof attacks. Experiments show that our proposed method achieves the state of the art on multiple testing protocols of ZSFA.
1. Introduction
The paper frames detection of previously unknown spoof attacks as Zero-Shot Face Anti-spoofing and broadens its study to 13 attack types. It introduces the SiW-M database and a Deep Tree Network that learns spoof sub-groups unsupervised for routing and detection.
- Face recognition systems can mistake photographs and screens for genuine faces, motivating anti-spoofing before recognition.
- Unknown spoof attacks are attack types neither known to designers nor seen during training, and their detection is termed Zero-Shot Face Anti-spoofing.
- Prior ZSFA studies focused on print and replay attacks, used only live faces, and relied on handcrafted features such as LBP.
- 13 spoof attack types are studied, including print, replay, five 3D mask, three makeup, and three partial attacks.
- SiW-M is introduced as a database containing diverse spoof attacks, and the method reports state-of-the-art performance on multiple testing protocols.
- The Deep Tree Network unsupervisedly partitions known spoof data into semantic sub-groups and routes unknown samples to a leaf node for binary spoof detection.
2. Prior Work
Prior face anti-spoofing work includes handcrafted, temporal, augmentation, physiological, and CNN-based approaches, but generalization across conditions and attack types remains difficult. ZSFA specifically detects unknown attacks rather than recognizing spoof categories, and prior methods modeled live data with handcrafted features.
- Image-based face anti-spoofing uses RGB images without extra modalities such as depth or heat.
- Traditional methods use handcrafted features including LBP, HoG, SIFT, and SURF with classifiers such as SVM and LDA.
- Researchers address overfitting through color-space features, temporal features, image-patch augmentation, score fusion, and heart-pulse estimation for 3D masks.
- ZSFA emphasizes detecting spoof attacks rather than recognizing specific spoof types, and lacks an explicit well-defined semantic embedding for spoof patterns.
- The proposed deep tree network learns semantic embeddings for known spoof attacks unsupervisedly and projects unknown attacks to matching attributes for detection.
3. Deep Tree Network for ZSFA
DTN learns a hierarchical spoof tree by routing samples along directions of maximal variation, then applies supervised leaf-level learning for live-versus-spoof discrimination and pixel-wise localization.
- Architecture: Each DTN tree node combines a Convolutional Residual Unit with a Tree Routing Unit, while each leaf adds Supervised Feature Learning.CRUs extract features, TRUs route samples, and leaf SFL modules provide supervised learning.
- Node Routing Function: The routing function partitions visiting data along the direction of greatest variation, using a projection vector constrained through covariance-based eigen-analysis.The optimal projection is the largest principal component basis of the visiting spoof data.
- Node Routing Function: The original routing loss can collapse data into a few leaves because it is sensitive to feature norms, data density, outliers, and imbalanced partitions.Zero expectation of the routing response does not guarantee balanced child-node assignments.
- Tree of Known Spoofs: DTN suppresses live-data responses and responses from spoof samples visiting neighboring nodes, so each node models a distinct spoof subset with balanced leaf assignments.Routing optimization maximizes responses for spoof samples visiting the node while minimizing responses for other data.
- Supervised Feature Learning: At each leaf, binary classification learns live-versus-spoof features, while pixel-wise mask regression directs attention toward local facial and attack-related regions.The two supervised tasks operate on the leaf feature representation and include classification and pixel-wise supervision.
- Network Architecture: The network uses RGB+HSV input, convolutional residual units, compressed TRU features, and projection coefficients to route samples left or right through the tree.Feature compression reduces covariance computation and storage before projection onto the learned basis.
4. Spoof in the Wild Database with Multiple Attack Types
SiW-M is introduced to benchmark zero-shot face anti-spoofing across diverse spoof types, identities, environments, and testing conditions. The database supports leave-one-out evaluation across 13 attack types.
- Database construction: SiW-M contains 13 types of spoof attacks collected across impersonation and obfuscation scenarios.The spoof data include mask, partial, makeup, print, and replay attacks.
- Spoof data: The spoof collection includes 968 videos, with HD recording used for mask, partial, and makeup attack categories.Print and replay videos were collected from harder cases in which an existing anti-spoofing algorithm predicted live.
- Live data: 660 live videos were collected from 493 subjects across multiple recording sessions and varied demographics.Subjects vary in ethnicity and age, and the sessions include different room environments and capture conditions.
- Database construction: 1,630 videos lasting 5–7 seconds were collected for the database.The collection includes both live and spoof videos.
- Evaluation protocol: Leave-one-out protocols train on 12 spoof types and 80% of live videos, then test on one held-out attack type and 20% of live videos.Live subjects do not overlap between training and testing sets.
5. Experimental Results
Experiments evaluate DTN across existing databases, ablations, and the diverse SiW-M protocols. Results show improved unknown-attack detection, effective semantic routing and clustering, and substantially lower computation than the baseline.
- Ablation study: Proper routing is necessary because random and fixed-leaf routing perform worse, while fixed-leaf routing shows large performance variation across eight subgroups.The ablation study uses the Funny Eye protocol.
- Ablation study: The full tree-learning design performs best, whereas alternative routing causes tree collapse or subgroup imbalance and using all data worsens performance.The evaluated components include route loss, spoof-only tree learning, and the unique loss.
- Existing databases: 7.3% improvement over prior state of the art was achieved on replay-versus-print ZSFA testing across CASIA, Replay-Attack, and MSU-MFSD.The authors also report a smaller standard deviation across unknown attacks.
- SiW-M evaluation: 31%, 61%, and 51% ACER reductions were reported for transparent mask, funny eye, and paper glasses attacks, respectively.These protocols were difficult for the baseline, which often classified attacks as live.
- Visualization and analysis: Early tree nodes capture general spoof attributes, while later nodes capture spoof-type-specific changes such as transitions from eye spoofs to full-face 3D masks.The visualization ranks spoof data according to routing-function values.
- Visualization and analysis: Spoof features cluster into eight semantic sub-groups without auxiliary labels, and unknown attacks are routed toward similar learned groups.Routing distributions show spoof samples concentrated in fewer leaves than live samples; transparent-mask samples overlap with obfuscation-makeup samples.
6. Conclusions
The paper addresses zero-shot face anti-spoofing across 13 spoof types with an unsupervised deep tree network and a more diverse database. Experiments show superior performance.
- 13 spoof types define the paper’s expanded zero-shot face anti-spoofing scope.
- The deep tree network routes unknown attacks to suitable leaf nodes for spoof detection.It is trained unsupervised to split spoof data using feature bases with the largest variation.
- SiW-M contains more subjects and spoof types than previous face anti-spoofing databases.
- Experiments show superior performance for the proposed method.