Source-linked AI summary
GBU-Palm: A Multimodal Video Dataset and Benchmark for Palm Presentation Attack Detection
Yingjie Ma, Zitong Yu, Wei Jia, Ajay Kumar, Linlin Shen
TL;DR
Existing palm PAD datasets do not jointly support large-scale native video, synchronized RGB-NIR data, multiple environments, and controlled attack provenance. GBU-Palm addresses this gap with a leakage-controlled multimodal video benchmark, finding architecture-dependent environmental degradation, inconsistent RGB-NIR gains, and distinct model information-use patterns.
Problem
Existing palm PAD datasets do not jointly provide large-scale native video, synchronized RGB-NIR observations, multiple acquisition environments, and controlled attack provenance.
Method
GBU-Palm combines 21,326 videos across six environments with synchronized RGB-NIR samples and leakage-controlled protocols benchmarking video architectures across modalities and environments.
Results
The benchmark reveals architecture-dependent environmental degradation, non-uniform RGB-NIR benefits, and substantially different reliance on temporal order and NIR information.
Takeaways & Limitations
Aggregate benchmark performance, information utilization, and cross-environment transfer are distinct properties of palm PAD models.
Takeaways & Limitations
Future extensions are needed for three-dimensional and more challenging attacks, additional sensing modalities, and richer spatiotemporal annotations.
Abstract
from arXiv · showhide
Existing palm presentation attack detection (PAD) datasets are often limited by static imagery, restricted acquisition conditions, or insufficient multimodal video data, hindering systematic evaluation across environments, modalities, and attack types. We present GBU-Palm, a large-scale multimodal video dataset and benchmark containing 21,326 videos from 105 subjects and 210 palms across six acquisition environments, including bona fide, Print, and Replay presentations, with 6,310 synchronized RGB-NIR samples. We construct leakage-controlled protocols that separate palm identity and attack lineage and benchmark four representative video architectures under environment-matched and held-out-environment settings. Results reveal substantial architecture-dependent degradation under environmental shift and show that RGB-NIR fusion does not consistently outperform RGB-only input. We further analyze model behavior through true accept (TA), true reject (TR), false accept (FA), and false reject (FR) decomposition, spectral masking, temporal-order intervention, and frozen-backbone NIR probing, revealing distinct failure patterns and evidence utilization across architectures. GBU-Palm provides a unified and challenging benchmark for developing and evaluating robust multimodal palm PAD methods under cross-environment conditions.
1 Introduction
Existing palm PAD resources largely rely on static imagery, single-spectrum sensing, or limited acquisition conditions, preventing controlled evaluation of video, multimodal, and cross-environment generalization. GBU-Palm addresses this gap with a large-scale multimodal video benchmark, leakage-controlled protocols, and analyses of architecture-dependent failure and information use.
- Motivation: Existing palm PAD resources rarely combine large-scale native video, synchronized RGB-NIR observations, multiple environments, and controlled attack provenance.This limits reproducible study of environmental shift and modality complementarity.
- Dataset: GBU-Palm contains 21,326 videos from 105 subjects and 210 palms across six environments, covering bona fide, Print, and Replay presentations, with 6,310 synchronized RGB-NIR samples.The dataset also includes attack-lineage annotations.
- Benchmark: The benchmark uses identity- and attack-lineage-disjoint protocols to evaluate representative video architectures with RGB, NIR, and RGB-NIR inputs in In-Env and Cross-Env conditions.These protocols are designed to control leakage while testing environmental generalization.
- Analysis: Analyses reveal architecture-dependent cross-environment degradation, non-uniform RGB-NIR fusion benefits, and different reliance on temporal ordering and NIR information.The study uses TA/TR/FR/FA decomposition, spatiotemporal evidence analysis, spectral masking, temporal-order intervention, and frozen-backbone NIR probing.
2 GBU-Palm Dataset
GBU-Palm is a large-scale native-video dataset spanning six illumination environments, multiple consumer devices, presentation classes, and synchronized RGB-NIR observations. Its metadata preserves attack lineage and generation factors, while subject demographics are broadly balanced across sex and age groups.
- Acquisition and Attack Scenarios: 21,326 videos cover 105 subjects and 210 palms across six illumination environments using consumer phones, tablets, laptops, and synchronized RGB-NIR acquisition.The environments include uniform indoor normal illumination and indoor low illumination, among others.
- Acquisition and Attack Scenarios: Print attacks recapture printed palm media, whereas Replay attacks recapture palm presentations displayed electronically.Replay presentations use electronic displays and recapture the resulting presentation.
- Acquisition and Attack Scenarios: Attack metadata links each sample to its bona fide source session, attack generation, and material identity, forming an attack lineage.Additional factors include printing spectrum, physical medium, crop strategy, replay spectrum, and playback interface.
- Native Video and Multimodal Construction: The store contains 27,636 modality sequences and 5,293,639 frames, without synthesizing unobserved NIR data.This preserves the distinction between observed and unavailable NIR measurements.
- Native Video and Multimodal Construction: The 105 subjects include 51 male and 54 female participants, with ages 18 to 70 years and near-balanced predefined age groups.The age groups 18–30, 31–45, 46–60, and 61+ contain 26, 27, 26, and 26 subjects, respectively.
3 Benchmark Results and Evidence Analysis
The benchmark compares four video architectures under matched and cross-environment protocols, revealing architecture-dependent robustness degradation. Evidence analyses further show distinct attack-family risks, architecture-dependent RGB-NIR utility, and differences in temporal-order sensitivity.
- Experimental Setup: Four video backbones span factorized convolution, transformer, hierarchical, and multi-scale spatiotemporal modeling.The compared architectures are R(2+1)D-18, ViViT, Video Swin-T, and MViT-V2-S.
- Overall benchmark performance: Under P2 Cross-Env evaluation, ViViT loses 11.28 AUC points, versus 2.87 for R(2+1)D, 1.75 for Video Swin-T, and 4.03 for MViT.Under P1, R(2+1)D and MViT reach approximately 97.4% AUC, while Video Swin-T performs considerably worse.
- Decision-conditioned error structure: Decision decomposition separates correctly accepted bona fide samples, correctly rejected attacks, false accepts, and false rejects to expose security and usability failure directions.False accepts represent attacks incorrectly accepted as bona fide, whereas false rejects represent bona fide samples incorrectly rejected.
- Attack-family vulnerability: Under MViT P2, Replay FA reaches 36.08%, compared with 11.47% for Print FA, indicating greater Replay attack-acceptance risk.The passage attributes cross-environment vulnerability to both model architecture and physical presentation mechanism.
- Spectral Evidence Analysis: R(2+1)D HTER improves from 4.33% to 3.02% with RGB+NIR on P1 and from 7.97% to 4.08% on P2.The spectral analysis reports that RGB-NIR benefits are strongly architecture-dependent rather than uniformly beneficial.
- Temporal Evidence Analysis: Temporal shuffling and reversal do not change the AUC of the architecture using temporal mean pooling without temporal positional encoding.This result shows that video input does not guarantee temporal-order utilization, and temporal sensitivity alone does not predict Cross-Env robustness.
4 Conclusion
GBU-Palm establishes a multimodal video benchmark for palm presentation attack detection and shows that robustness depends on architecture, environment, modality complementarity, and temporal evidence. The analysis also exposes distinct security and usability failure modes and motivates broader attack, sensing, and annotation coverage.
- GBU-Palm is introduced as a large-scale multimodal video dataset and benchmark for palm presentation attack detection.
- Architecture-dependent environmental sensitivity, non-uniform RGB-NIR complementarity, and differing reliance on temporal order characterize benchmark behavior.
- Decision-conditioned analysis and NIR probing show that similar aggregate performance can conceal substantially different security and usability failures.
- Single-modality degradation may reflect classifier-related or representation-related limitations, separating benchmark performance, information utilization, and cross-environment transfer.
- Future extensions will add three-dimensional and more challenging attacks, additional sensing modalities, and richer spatiotemporal annotations.