PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Test-Time Adaptive Face Presentation Attack Detection with Dual-Phase Privacy Preservation

Rui Shao, Bochao Zhang, Pong C. Yuen, Vishal M. Patel

arXiv:2110.12613v1cs.CV

TL;DR

fPAD models trained on limited identities and known attacks may not generalize to unseen attacks, while privacy constraints prevent data centers from sharing training data. The paper combines federated model aggregation with test-time entropy minimization; it reports improved average performance across settings over other baselines while preserving privacy without sacrificing fPAD performance.

  • Problem

    Data centers have limited identities and known spoof types, which restricts fPAD generalization to unseen attacks, while privacy constraints prevent sharing their training data.

    Source: §I · p. 1

  • Method

    The framework aggregates local fPAD model updates through federated learning, then adapts the global model on users’ testing data by minimizing prediction entropy.

    Source: §I · p. 2

  • Results

    The proposed framework improves performance on average in all settings and outperforms other baselines, while obtaining a privacy-preserving fPAD model without sacrificing performance.

    Source: B. Experimental Results · pp. 5, 7

  • Takeaways & Limitations

    Federated training and test-time adaptation together target generalized fPAD models while preserving data privacy in both training and testing phases.

    Source: §V · p. 8

Abstract

from arXiv · show

Face presentation attack detection (fPAD) plays a critical role in the modern face recognition pipeline. The generalization ability of face presentation attack detection models to unseen attacks has become a key issue for real-world deployment, which can be improved when models are trained with face images from different input distributions and different types of spoof attacks. In reality, due to legal and privacy issues, training data (both real face images and spoof images) are not allowed to be directly shared between different data sources. In this paper, to circumvent this challenge, we propose a Federated Test-Time Adaptive Face Presentation Attack Detection with Dual-Phase Privacy Preservation framework, with the aim of enhancing the generalization ability of fPAD models in both training and testing phase while preserving data privacy. In the training phase, the proposed framework exploits the federated learning technique, which simultaneously takes advantage of rich fPAD information available at different data sources by aggregating model updates from them without accessing their private data. To further boost the generalization ability, in the testing phase, we explore test-time adaptation by minimizing the entropy of fPAD model prediction on the testing data, which alleviates the domain gap between training and testing data and thus reduces the generalization error of a fPAD model. We introduce the experimental setting to evaluate the proposed framework and carry out extensive experiments to provide various insights about the proposed method for fPAD.

I. INTRODUCTION

fPAD models trained on limited identities and known attack types may fail on unseen attacks, while privacy rules prevent data centers from pooling their data. The framework combines federated training across data centers with test-time entropy minimization on users’ devices.

I. INTRODUCTION

  • Limited identities and known spoof types at each data center leave fPAD models with poor generalization to unseen attacks.Users rely on data-center models and have only real data locally, so they cannot train fPAD models themselves.Source: §I · p. 1
  • Privacy policies and data-sharing agreements prevent data centers from sharing fPAD data, despite the potential benefit of exploiting their varied data collaboratively.The motivating examples include data from different input distributions and different spoof attack types.Source: §I · pp. 1–2
  • Federated learning lets data centers train locally and send model updates to a server for aggregation, keeping client data private while producing a shared global model.The global model is returned to data centers for subsequent training rounds.Source: §I · p. 2
  • In the fPAD setting, data centers train models during training, while users download the global model and perform inference during testing.This differs from conventional federated learning, where all clients train and infer locally.Source: §I · p. 2
  • Users adapt the downloaded model at test time by minimizing prediction entropy, complementing federated training when unseen attacks and domain variation remain challenging.The federated model also provides an initialization for test-time adaptation, which is sensitive to pretrained parameters.Source: §I · p. 2

II. RELATED WORK

Prior fPAD work includes single-domain methods that extract appearance or temporal cues, and multi-domain methods intended to generalize across attacks. This paper studies how to exploit multi-domain information when data centers hold separate domains and cannot share data.

A. Face Presentation Attack Detection

  • Single-domain fPAD methods extract appearance cues, including texture and surface-distortion features, or temporal cues from facial motion, heartbeat signals, and estimated depth.The cited examples include Multi-scale LBP, color textures, image distortion analysis, dynamic textures,,, rPPG,, and CNN-RNN modeling [12].Source: A. Face Presentation Attack Detection · p. 2
  • Multi-domain fPAD methods use multiple datasets or spoof subgroups to improve generalization, including domain generalization, zero-shot learning, and meta-learning approaches.Examples include methods, [13], and [24].Source: A. Face Presentation Attack Detection · pp. 2–3
  • This paper considers data centers with single-domain data that cannot access other centers’ data, and aims to use multi-domain information while preserving privacy.Source: A. Face Presentation Attack Detection · p. 3

B. Federated Learning

Federated learning lets data centers contribute their fPAD information without sharing private datasets. The framework trains locally, aggregates model updates into a global model, and lets users download that model for inference; it also pairs FL with test-time adaptation to address unseen attacks.

B. Federated Learning

  • FedAvg averages local model parameters; FedProx and AFL address global-model bias, while FedPAD still degrades on unseen attacks with large domain gaps.The proposed method combines federated learning with test-time adaptation to further support generalization during testing.Source: B. Federated Learning · p. 3

A. Training Time Federated Learning

  • Data centers train fPAD models locally, while a server aggregates their model updates into a global model without accessing private data.The server can distribute the aggregate for further rounds; after training, users download the global model for local inference.Source: A. Training Time Federated Learning · pp. 3–4
  • Each data center’s dataset covers a specific input distribution and only finitely many known spoof attacks, limiting generalization to unseen attacks.Pooling diverse data could improve generalization, but privacy restrictions make direct data sharing impractical.Source: A. Training Time Federated Learning · p. 3

B. Testing-Time Adaptation

The framework adapts an fPAD model at test time by minimizing prediction entropy on user testing data. It updates batch-normalization affine parameters while keeping other parameters fixed, without accessing private training data or requiring a designed self-supervision task.

B. Testing-Time Adaptation

  • Test-time adaptation minimizes prediction entropy on testing data to reduce the domain gap between training and testing distributions.The method uses the testing data as an unsupervised signal before final classification.Source: B. Testing-Time Adaptation · p. 4
  • To limit overfitting with scarce test data, adaptation updates only batch-normalization scale and shift parameters, keeping all other model parameters fixed.After adaptation, the updated model performs the final real/fake classification.Source: B. Testing-Time Adaptation · p. 4
  • Adaptation uses unlabeled user testing data without training-data access or a designed self-supervision task, and its effectiveness depends on suitable initial parameters.The paper notes that federated learning can provide a well-pretrained starting model.Source: Figure 3 · p. 4, B. Testing-Time Adaptation · p. 4

IV. EXPERIMENTS

The experiments evaluate the framework on five 2D fPAD datasets and two 3D mask fPAD datasets, using multiple datasets to examine performance and FL-related insights.

IV. EXPERIMENTS

  • The evaluation uses five 2D fPAD datasets and two 3D mask fPAD datasets.The authors describe the datasets and testing protocol before reporting results.Source: §IV · p. 4
  • The experiments report results based on multiple fPAD datasets.Source: §IV · p. 4
  • The section includes discussion and analysis intended to provide insights about federated learning for fPAD.Source: §IV · p. 4

1) Datasets:

The evaluation covers seven fPAD datasets spanning print, video-replay, and 3D mask attacks. Differences in attack types and capture scenarios create significant domain shifts across datasets.

1) Datasets:

  • Seven datasets cover print, video-replay, and 3D mask attacks, with varied display materials, resolution, illumination, and backgrounds.Source: 1) Datasets · p. 4, Table I · p. 5
  • The datasets' differing attack characteristics and scenarios produce significant domain shifts between them.Source: 1) Datasets · pp. 4–5
  • Fig. 4 presents sample real and attacked faces from the seven datasets.Source: Figure 4 · p. 5

2) Protocol:

The protocol tests each trained model on a dataset withheld from training, treating that dataset as users and the remaining datasets as data centers. Results report that the proposed federated method with test-time adaptation outperforms the baselines on average, while the privacy-violating pooled-data baseline serves only as an upper-bound reference.

2) Protocol:

  • Each test withholds one dataset from training, using it as the user dataset and the remaining datasets as data centers.Source: 2) Protocol · p. 5

3) Evaluation Metrics:

  • HTER, EER, and AUC evaluate cross-dataset performance, with thresholds set using all data-center data when no development set is available.Source: 3) Evaluation Metrics · p. 5

4) Implementation Details:

  • The fPAD models use ResNet-18; federated learning uses Adam at 1e-2 with three local epochs, and test-time adaptation uses Adam at 5e-3.Source: 4) Implementation Details · p. 5

B. Experimental Results

  • The proposed method with test-time adaptation significantly improves average performance across settings and outperforms the baselines, including the privacy-violating All baseline.Source: B. Experimental Results · pp. 5, 7, Table II · p. 6
  • The All baseline pools all available data and violates the privacy-preservation assumption, so it is an upper-bound reference rather than a valid FL comparison.Source: B. Experimental Results · pp. 6–7

1) Compatibility of Dual Phases:

Federated training paired with test-time adaptation performs better than adapting a single-center model. The authors attribute the advantage to a more suitable federated initialization for adaptation.

1) Compatibility of Dual Phases:

2) Comparison of different number of data centers:

Adding data centers generally improves performance, and the proposed method performs best in one of two five-dataset settings and comparably to FedPAD in the other. In a limited-attack 2D setting, it further improves over FedPAD, including gains of 12.31% in HTER and 7.47% in EER.

2) Comparison of different number of data centers:

3) Generalization ability to various 2D spoof attacks:

4) Generalization ability to 3D mask attacks :

The proposed framework improves generalization to novel 3D mask attacks, with test-time adaptation reducing HTER relative to FedPAD. The conclusion describes federated training and entropy-minimizing test-time adaptation as a privacy-preserving approach, while identifying further generalization work as future research.

4) Generalization ability to 3D mask attacks :

  • The proposed method improves HTER by 17.73% compared to FedPAD when generalizing to novel 3D mask attacks.Adding a data center containing 3D mask attacks also improves all evaluation metrics for FedPAD.Source: 4) Generalization ability to 3D mask… · p. 8

V. CONCLUSION

  • The framework aggregates data-center model updates during training and minimizes prediction entropy during testing, without accessing private data-center data.The authors state that test-time adaptation can further reduce generalization error on unseen attacks.Source: §V · p. 8
  • The authors state that approaches for boosting generalization ability, including data augmentation [7] and robust loss functions [31], will also be studied in future work.Source: §V · p. 8

Related papers

Loading 2110.12613v1…