PaperlayerView exact revision on arXiv

Source-linked AI summary

Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection

Ajian Liu, Chenxu Zhao, Zitong Yu, Jun Wan, Anyang Su, Xing Liu, Zichang Tan, Sergio Escalera, Junliang Xing, Yanyan Liang, Guodong Guo, Zhen Lei, Stan Z. Li, Du Zhang

arXiv:2104.06148v1cs.CV

TL;DR

Existing 3D mask PAD datasets have limited scale and diversity, and high-fidelity masks and changing light challenge existing methods. The paper introduces HiFiMask and context-aware contrastive learning; existing methods that perform well on other datasets fail to achieve convincing results on HiFiMask.

  • Problem

    Existing 3D mask PAD datasets have limited samples, attribute information, mask realism, and lighting diversity, while high-fidelity masks and dynamic light challenge existing approaches.

    Source: §1 · p. 1, §2.1 · p. 3, §2.2 · p. 3

  • Method

    CCL learns discriminative representations by comparing live and mask image pairs organized around diverse contextual attributes.

    Source: §1 · p. 2, §4.1 · p. 4

  • Results

    ResNet50 and GrPPG perform well on 3DMAD and MARsV2 but fail to achieve high performance on HiFiMask.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper presents HiFiMask and CCL as contributions intended to push 3D mask face PAD research further.

    Source: §6 · p. 9

    On 3DMask, limited attack types—all plaster masks—may inhibit Data Pair Generation from providing abundant context information.

    Source: §5.4 · p. 8

Abstract

from arXiv · show

Face presentation attack detection (PAD) is essential to secure face recognition systems primarily from high-fidelity mask attacks. Most existing 3D mask PAD benchmarks suffer from several drawbacks: 1) a limited number of mask identities, types of sensors, and a total number of videos; 2) low-fidelity quality of facial masks. Basic deep models and remote photoplethysmography (rPPG) methods achieved acceptable performance on these benchmarks but still far from the needs of practical scenarios. To bridge the gap to real-world applications, we introduce a largescale High-Fidelity Mask dataset, namely CASIA-SURF HiFiMask (briefly HiFiMask). Specifically, a total amount of 54,600 videos are recorded from 75 subjects with 225 realistic masks by 7 new kinds of sensors. Together with the dataset, we propose a novel Contrastive Context-aware Learning framework, namely CCL. CCL is a new training methodology for supervised PAD tasks, which is able to learn by leveraging rich contexts accurately (e.g., subjects, mask material and lighting) among pairs of live faces and high-fidelity mask attacks. Extensive experimental evaluations on HiFiMask and three additional 3D mask datasets demonstrate the effectiveness of our method.

1. Introduction

Existing 3D mask PAD datasets and methods leave gaps in scale, diversity, and performance on high-fidelity masks. The paper introduces HiFiMask and CCL to address these challenges.

1. Introduction

  • Existing datasets are limited in mask identities, subjects and skin-tone diversity, materials, scenes, lighting conditions, acquisition devices, and video volume.The cited dataset shortcomings include fewer than 2,000 videos overall and risks of identity overfitting.Source: §1 · pp. 1–2
  • HiFiMask contains 54,600 videos from 75 subjects, 225 masks across three materials, six scenes, and seven imaging devices.The dataset includes 25 subjects in each of three skin tones; it contains 13,650 live and 40,950 mask videos.Source: §1 · p. 2
  • ResNet50 and GrPPG perform well on 3DMAD and MARsV2 but fail to achieve high performance on HiFiMask.High-fidelity mask appearance and temporal light interference are identified as challenges for appearance- and rPPG-based methods.Source: §1 · p. 2
  • CCL learns discriminative representations by comparing image pairs with diverse contexts, including live skin and mask materials from the same identity.Context Guided Dropout is proposed to discard unnecessary embedding features and mitigate instability from dissimilar hard positive pairs.Source: §1 · p. 2
  • Experiments on HiFiMask and three other public 3D mask datasets demonstrate the challenges of HiFiMask and the effectiveness of CCL.The paper also presents HiFiMask as a dataset with realistic masks and more identities, sensors, and videos than public 3D mask datasets.Source: §1 · pp. 2–3

2. Related Work

Prior 3D mask datasets vary in their coverage but retain limitations in sample size, attribute information, mask realism, and lighting. Existing PAD approaches use appearance, rPPG, or metric-learning cues, while the paper motivates context-aware learning for difficult mask and scene conditions.

2.1. 3D Mask Datasets

  • Existing 3D mask datasets have limited samples, unclear attribute information, and masks that are often insufficiently realistic or recorded under stable lighting.Earlier datasets include 3DMAD, 3DFS-DB, MARsV2, BRSU, and CASIA-SURF 3DMask, with some extending to near-infrared and thermal modalities.Source: §2.1 · p. 3

2.2. Face PAD Approaches based on 3D Mask

  • Appearance-based methods exploit fine-grained cues or multimodal features, while rPPG methods use pulse cues recovered from live faces and noisy in mask attacks.Prior examples include two-stream factorized bilinear coding, central-difference architectures, and aggregation across RGB, depth, infrared, and thermal modalities.Source: §2.2 · p. 3
  • Metric-learning and contrastive methods have been applied to PAD, but CCL differs from supervised contrastive learning [23] in pair generation and dropout regularization.The cited approaches include contrastive loss, triplet loss, and supervised contrastive learning.Source: §2.2 · p. 3
  • The paper identifies realistic mask appearance, dynamic lighting, and unsuited contrastive approaches as challenges for 3D mask PAD.It proposes explicitly mining discriminative features across bonafide and mask appearance under complex scenarios.Source: §2.2 · p. 3

3. HiFiMask Dataset

HiFiMask combines a large video collection with varied subjects, realistic masks, lighting, scenes, and sensors. Its seen and unseen evaluation protocols test performance with known mask types and generalization to held-out types.

3. HiFiMask Dataset

  • HiFiMask contains 54,600 videos from 75 subjects across three skin tones, with three high-fidelity mask materials collected for each identity.The materials are transparent, plaster, and resin; six complex scenes and seven imaging devices are used.Source: §3 · pp. 3–4
  • The dataset varies recording conditions through six scenes, directional lighting, periodic lighting from 0.7 to 4 Hz, and seven imaging devices.The three mask materials are collected with the same identity, and the masks are described as visually difficult to distinguish from live faces.Source: §3 · p. 4

3.1. Evaluation Protocol and Statistics

  • Protocol 1 evaluates mask types seen in training and development, while Protocol 2 tests generalization to mask types withheld from training and development.Protocol 2 comprises leave-one-type-out subprotocols for transparent, plaster, and resin masks.Source: §3.1 · p. 4

4. Methodology

CCL trains supervised face PAD models by organizing contextual image pairs and combining pairwise contrastive learning with explicit classification supervision. Its design uses online and target networks, Context Guided Dropout, and a joint classification–contrastive loss.

4.1. Data Pair Generation

  • Contextual pairs hold some attributes fixed—such as subject identity—to expose differences in other cues, and include both positive and negative combinations.The training pairs can use skin, subject, mask type, scene, lighting, sensor, and inter-frame information.Source: §4.1 · p. 4

4.2. Network Architecture

  • CCL extends batch contrastive learning to supervised PAD with online and target networks, while an added classifier provides explicit supervision.The target network tracks online parameters by exponential moving average; inference uses only the encoder and classifier.Source: §4.2 · pp. 4–5, Figure 3 · p. 5

4.3. Context Guided Dropout

  • Context Guided Dropout ranks differences between positive-pair embeddings, then uses the largest differences to select channels for batchwise masking and rescales the embeddings.The discarded-channel proportion follows a cosine decay schedule during training.Source: §4.3 · p. 5
  • CGD accelerates network convergence during early training and makes CCL training more stable.The authors attribute this observation to assembling CGD in the network.Source: §4.3 · p. 5

4.4. Overall Loss

  • The contrastive term compares CGD-regularized cross-pair embeddings by label-dependent cosine similarity, and Ltotal adds it to classification loss with weight λcon.The classification loss uses binary cross-entropy; λcon is the trade-off hyperparameter.Source: §4.4 · pp. 5–6

5. Experiments

CCL improves face PAD results across HiFiMask, WMCA, and cross-dataset testing, while ablations and visualizations examine its contrastive learning, pair-generation, and CGD components. Experiments use seen and unseen protocols and compare three backbones with and without CCL.

5.2. Ablation Study

  • 0.5% lower ACER than BYOL and 0.8% lower ACER than supervised contrastive learning are reported for CCL on Protocol 1 of HiFiMask.Contrastive strategies improve over vanilla ResNet50 from 4.7% to 2.9% ACER; BYOL and SimSiam achieve 2.9% and 4.0% ACER, respectively.Source: §5.2 · p. 6
  • Removing CGD raises APCER, BPCER, and ACER from 1.8%, 3.0%, and 2.4% to 1.9%, 3.3%, and 2.6%, respectively.Reverse and BOBE CGD variants achieve 2.8% and 2.7% ACER; negative-only pair training yields 43.7% ACER.Source: §5.2 · p. 7

5.3. Intra Testing

  • 4.7%, 3.4%, and 3.6% ACERs fall to 2.4%, 2.6%, and 3.1% on Protocol 1; Protocol 2 ACERs fall from 18.5%, 11.4%, and 14.7% to 16.7%, 10.7%, and 12.7%.The values correspond to ResNet50, Aux.(Depth), and CDCN, respectively; Protocol 2's transparent-mask test subset is especially challenging.Source: §5.3 · p. 7, Table 5 · p. 7
  • 10.24%, 12.05%, and 11.27% ACER decreases are reported for ResNet50, Aux.(Depth), and CDCN with CCL on WMCA's seen protocol; unseen-protocol average ACERs decrease by 10.12%, 8.27%, and 2.42%.With RGB inputs alone, CCL also achieves better average testing performance than two cited multi-modal baselines on the unseen protocol.Source: §5.3 · pp. 7–8, Table 6 · p. 8

5.4. Cross Testing

  • 6.57%, 0.90%, and 2.33% HTER reductions, with 3.79%, 1.44%, and 2.11% AUC gains, are reported for ResNet50, CDCN, and Aux.(Depth) on MARsV2 with CCL.CCL also improves performance on 3DMask, though Aux.(Depth) declines slightly there when equipped with CCL.Source: §5.4 · p. 8

5.5. Analysis and Visualization.

  • CCL features better distinguish real faces from mask samples than ResNet50, and CCL-enhanced Aux.(Depth) predicts complex mask samples more correctly than Aux.(Depth) alone.The experiments use HiFiMask and WMCA with seen and unseen protocols, and evaluate cross-testing on MARsV2 and 3DMask after training on HiFiMask.Source: §5.1 · p. 6, §5.5 · p. 8, Figure 4 · p. 8

6. Conclusion

The paper releases HiFiMask with two challenging protocols and proposes CCL to learn discriminability from contextual pairs of live faces and high-fidelity mask attacks. Experiments on HiFiMask and three other 3D mask datasets are reported as verifying the dataset and method's significance.

6. Conclusion

  • HiFiMask is released as a large-scale 3D high-fidelity mask dataset with two challenging protocols.The paper also proposes CCL and reports experiments on HiFiMask and three other 3D mask datasets.Source: §6 · p. 9

A. Acquisition Details of HiFiMask

HiFiMask acquisition combines controlled lighting and varied scenes with customized masks, real-time physiological measurements, and standardized video collection. The appendix also describes face cropping, frame sampling, and how example samples are arranged by scene, sensor, and lighting.

Equipment Preparation

  • Masks were customized for real people, and a pulse oximeter recorded real-time BVP signals and instantaneous heart rate from live videos.Periodic illumination frequencies were randomly set between 0.7 and 4 Hz; supplemental light intensity ranged from 400 to 1600 lux.Source: A. Acquisition Details of HiFiMask · p. 11

Collection Rules

  • Collection rules included wearing masks uniformly, small head movements, a moving pedestrian in the background, and randomly assigned decorations.These procedures were used during video acquisition to improve video quality and simulate real-environment conditions.Source: A. Acquisition Details of HiFiMask · p. 11

Data Pre-processing

  • With periodic light, mask-attack videos can exhibit pseudo-live pulse clues in their extracted rPPG signals and PSD visualizations.The figure contrasts samples without and with external periodic light.Source: Figure 6 · p. 11
  • Each sample module represents a scene; its rows correspond to seven imaging sensors and its columns to six kinds of lights.After cropping face regions, the acquisition process samples 10 frames at equal intervals from each video.Source: A. Acquisition Details of HiFiMask · p. 11

B. rPPG Signal Recovery from 3D Mask Attacks w/ and w/o Periodic Lights

The rPPG example shows noisy signals from mask attacks without periodic light but richer periodicity under periodic illumination, which can mislead rPPG-based PAD. The appendix notes that diverse temporal lighting and moving backgrounds challenge current rPPG methods.

B. rPPG Signal Recovery from 3D Mask Attacks w/ and w/o Periodic Lights

  • Periodic illumination within 0.7–4 Hz gives mask-attack rPPG signals rich periodicity, which can mislead an rPPG detector into treating a mask as live.Without periodic light, the extracted mask-attack signals are noisy and indicate weak heartbeat-derived liveness clues; PSD distributions also show periodicity evidence.Source: B. rPPG Signal Recovery from 3D Mask… · pp. 11–12
  • Diverse temporal lighting on both bona fide and mask videos makes robust PAD with current rPPG approaches difficult, while moving backgrounds challenge global-noise compensation.The paper identifies robust rPPG extraction and liveness detection under complex temporal-light interference as a possible future direction.Source: B. rPPG Signal Recovery from 3D Mask… · p. 12

C. Ablation Study on Parameter λcon and CGD

The ablation selects a 15% CGD dropout probability and a CCL-loss proportion of 0.7; CGD also accelerates early training convergence.

C. Ablation Study on Parameter λcon and CGD

D. The Details of Architecture and Optimization

  • The ResNet50 encoders map 2048-dimensional features to 128 dimensions, with 512-dimensional projector hidden layers; training uses SGD for 30 epochs.The stated optimizer uses weight decay 0.0001 and momentum 0.9; the learning rate starts at 0.01 and decays at epochs 15, 21, and 26.Source: D. The Details of Architecture and… · p. 12

E. ROC Results on HiFiMask.

CCL improves HiFiMask ROC results and open-set performance, while cross-dataset tests favor training on HiFiMask; adding contextual image pairs also lowers ACER.

E. ROC Results on HiFiMask

F. More Cross Testings on 3DMask and MARsV2

G. More Study on Data Pair Increment

  • Table 9 presents an ablation study of combining different image pairs.Source: Table 9 · p. 13

G. More Study on Data Pair Increment.

The ablation adds contrastive pair categories sequentially, and the resulting ACER decreases from 3.9% to 2.4%.

G. More Study on Data Pair Increment

H. Protocol 3 of HiFiMask.

Protocol 3 evaluates open-set discrimination and generalization using training and development data that cover only parts of common mask types and scenarios.

H. Protocol 3 of HiFiMask

  • Protocol 3 uses only parts of representative samples for training and development, while evaluating on the full testing set.Source: H. Protocol 3 of HiFiMask · p. 13
  • The testing distribution is more complicated than training and development across mask types, scenes, lighting, and imaging devices.Source: H. Protocol 3 of HiFiMask · p. 13
  • CCL improves all three tested models on Protocol 3, but ACERs are far worse than Protocol 1, underscoring the challenge of training on partial types and scenarios.Source: H. Protocol 3 of HiFiMask · p. 13

Related papers

Loading 2104.06148v1…