PaperlayerView exact revision on arXiv

Source-linked AI summary

FedSIS: Federated Split Learning with Intermediate Representation Sampling for Privacy-preserving Generalized Face Presentation Attack Detection

Naif Alkhunaizi, Koushik Srivatsan, Faris Almalik, Ibrahim Almakky, Karthik Nandakumar

arXiv:2308.10236v2cs.CVcs.CRcs.LG

TL;DR

Cross-domain FacePAD needs to generalize to unknown target distributions, but diverse source-domain data may be held by separate entities that cannot share it. FedSIS trains a hybrid ViT using federated and split learning and samples intermediate representations through a shared adapter for feature augmentation. On Benchmark 1, FedSIS outperforms all federated and centralized methods in three of four target domains.

  • Problem

    Cross-domain FacePAD needs to generalize to unknown target distributions, but diverse source-domain data may be held by separate entities that cannot share it.

    Source: §1 · p. 2

  • Method

    FedSIS trains a hybrid ViT using federated and split learning and samples intermediate representations through a shared adapter for feature augmentation.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 4

  • Results

    On Benchmark 1, FedSIS outperforms all federated and centralized methods in three of four target domains.

    Source: §5 · p. 7

  • Takeaways & Limitations

    The reported results show that collaborative training can outperform centralized DG-FacePAD methods while preserving client data privacy.

    Source: §5 · p. 8

    The formulation assumes each client holds source-domain training data that cannot be pooled centrally, and evaluation uses a test distribution distinct from every source distribution.

    Source: §3 · p. 3

Abstract

from arXiv · show

Lack of generalization to unseen domains/attacks is the Achilles heel of most face presentation attack detection (FacePAD) algorithms. Existing attempts to enhance the generalizability of FacePAD solutions assume that data from multiple source domains are available with a single entity to enable centralized training. In practice, data from different source domains may be collected by diverse entities, who are often unable to share their data due to legal and privacy constraints. While collaborative learning paradigms such as federated learning (FL) can overcome this problem, standard FL methods are ill-suited for domain generalization because they struggle to surmount the twin challenges of handling non-iid client data distributions during training and generalizing to unseen domains during inference. In this work, a novel framework called Federated Split learning with Intermediate representation Sampling (FedSIS) is introduced for privacy-preserving domain generalization. In FedSIS, a hybrid Vision Transformer (ViT) architecture is learned using a combination of FL and split learning to achieve robustness against statistical heterogeneity in the client data distributions without any sharing of raw data (thereby preserving privacy). To further improve generalization to unseen domains, a novel feature augmentation strategy called intermediate representation sampling is employed, and discriminative information from intermediate blocks of a ViT is distilled using a shared adapter network. The FedSIS approach has been evaluated on two well-known benchmarks for cross-domain FacePAD to demonstrate that it is possible to achieve state-of-the-art generalization performance without data sharing. Code: https://github.com/Naiftt/FedSIS

1. Introduction

FacePAD must generalize to unseen domains, but prior domain-generalization methods commonly assume centralized access to diverse source data. FedSIS combines federated and split learning with intermediate-representation sampling to train a privacy-preserving model for this setting.

1. Introduction

  • FacePAD needs to generalize to unknown target distributions because deployment conditions and presentation attack instruments cannot be predicted in advance.Closed-world methods assume matching train and test distributions.Source: §1 · p. 2
  • Existing DG-FacePAD approaches generally assume one central entity can train on multiple source domains, despite privacy concerns that hinder collecting or sharing such data.The passage notes exceptions [43] [42].Source: §1 · p. 2
  • FedSIS addresses clients’ non-iid training data and an unseen test distribution, where raw bonafide and attack presentation data cannot be shared.The challenge includes heterogeneity across clients during training and out-of-domain samples at inference.Source: §1 · p. 2
  • Its hybrid ViT federates client-specific tokenizers and classifiers while training a shared feature encoder through split learning to align source domains.The tokenizer and classifier are domain/client-specific; the shared encoder is learned using split learning.Source: §1 · p. 2
  • Intermediate representations from ViT blocks are sampled and processed through a shared adapter before classification to augment features and improve generalizability.The strategy uses discriminative information from intermediate blocks.Source: §1 · p. 2
  • FedSIS empirically outperforms state-of-the-art centralized DG-FacePAD methods while preserving participating clients’ data privacy.The paper frames this as a result for federated generalized FacePAD.Source: §1 · p. 2

2. Related Work

Prior cross-domain FacePAD work includes domain-generalization methods that learn from multiple source domains, but these approaches often centrally aggregate source data. Federated and split learning offer collaborative alternatives without raw-data sharing, though prior approaches have distinct assumptions and design burdens.

2.1. Domain Generalization for FacePAD

  • Domain-generalization approaches learn discriminative domain-invariant features from multiple source domains for an unseen target, while UDA and few-shot methods assume access to target data.UDA uses unlabeled target samples; few-shot learning uses a few labeled samples.Source: §2.1 · p. 2
  • The reviewed DG-FacePAD methods show promising cross-domain performance but rely on central aggregation of source datasets, raising privacy concerns.The passage surveys approaches including feature separation, meta-learning, and ViT-based methods.Source: §2.1 · p. 2

2.2. Collaborative Learning

  • FL exchanges model parameters without raw data, while U-shaped split learning places the middle sub-network with a computing client and keeps the first and last with the data owner.Prior federated FacePAD work [43] [42] uses feature disentanglement with depth-map supervision, orthogonality constraints, and autoencoders.Source: §2.2 · p. 3

3. Methodology

The methodology formulates FacePAD domain generalization across private, heterogeneous source clients and introduces FedSIS, which combines a hybrid ViT with federated split learning and intermediate-feature sampling.

3. Methodology

  • Training data remain with K clients and come from distinct source distributions, while evaluation uses an unseen target distribution that cannot be pooled centrally.The standard domain-generalization objective minimizes labeled prediction loss on source-domain samples.Source: §3 · p. 3

3.2. Federated Learning (FL)

  • Federated learning trains client-specific tokenizers and classification heads through local updates and server aggregation, avoiding centralized pooling of client data.FedAvg weights each client's parameters by its share of the training samples.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.1. Vision Transformer (ViT)

  • ViT-H combines a CNN tokenizer for micro-texture, a self-attention feature encoder for global characteristics, and a linear classifier for binary FacePAD.The feature encoder produces a classification token that the head uses for prediction.Source: §3.1 · p. 3, §3.3 · p. 4

3.3. Proposed FedSIS Framework

  • Randomly sampling intermediate ViT blocks feeds their patch tokens to a shared adapter, which produces pseudo-class tokens for classification and augments features across depths.The authors describe this sampling as using intermediate texture information and encouraging more generalized decision boundaries.Source: §3.3 · pp. 5–6
  • FedSIS assigns client tokenizers and heads to clients and a shared feature encoder and adapter to the server, training these components through federated and split learning.Client updates pass through the server's adapter and sampled encoder blocks; tokenizer and head updates are aggregated across clients.Source: Figure 2 · p. 4, §3.3 · p. 5

4. Experiments

The experiments use leave-one-domain-out evaluation across two benchmarks and report three FacePAD metrics. They also describe model initialization and collaborative-training settings.

4.1. Datasets

  • The evaluation covers two benchmarks with seven datasets, using leave-one-domain-out protocols and CelebA-Spoof as an optional auxiliary dataset.Benchmark 1 uses MSU-MFSD, CASIA-MFSD, Replay Attack, and OULU-NPU; Benchmark 2 uses WMCA, CASIA-CeFA, and CASIA-SURF.Source: §4.1 · p. 6

4.2. Implementation Details

  • Training uses a ViT-based encoder, a two-layer CNN adapter, and a linear classification head, with ImageNet initialization or optional CelebA-Spoof pretraining.The experiments use three source clients for Benchmark 1 and two for Benchmark 2, train for 200 rounds, and aggregate tokenizer and head parameters every 10 rounds.Source: §4.2 · p. 6

4.3. Evaluation Metrics

  • Performance is measured using HTER, AUC, and TPR at a fixed FPR, with each experiment repeated using three seeds.The reported scores are means; standard deviations are provided in supplementary material.Source: §4.3 · p. 7

5. Results and Discussion

FedSIS is compared with centralized and federated baselines across two cross-domain benchmarks. Results highlight its performance, the contributions of representation sampling and collaborative training, and a smaller gain on the larger-data benchmark.

Benchmark 1

  • FedSIS outperforms all federated and centralized methods in three of four Benchmark 1 target domains; for ICM →O, its HTER is lower than FedAvg by 0.22 and centralized ViT-H by 2.39.The authors attribute the exception to OULU-NPU having three times as many samples as the other three source domains combined.Source: §5 · p. 7
  • ViT-H-IS performs better than centralized ViT-H, which the authors attribute to intermediate representation sampling and its shared adapter network enabling feature augmentation.Source: §5 · p. 7
  • FedSIS achieves lower average HTER than centralized ViT-H-IS, indicating that the gain is not explained by architecture choice alone.The authors say federated split learning also plays a crucial role in boosting performance.Source: §5 · p. 7
  • FedSIS outperforms FedProx and MOON, suggesting that accounting for non-iid training distributions alone is insufficient for out-of-domain generalization.Source: §5 · p. 8
  • FedSIS outperforms FeSTA, while FeSTA increases generalization error relative to centralized ViT-H and FedSIS decreases it relative to centralized ViT-H-IS.The authors highlight the need to codesign architecture and collaborative learning.Source: §5 · p. 8

Effect of excluding CelebA-Spoof

  • Without CelebA-Spoof during both pre-training and collaborative training, FedSIS* outperforms all state-of-the-art centralized DG-FacePAD methods by a comfortable margin; FedSIS also outperforms FedGPAD by a substantially large margin.Source: §5 · p. 8

Benchmark 2

  • On Benchmark 2, performance gains are marginal; the authors suggest collaboration offers less when clients already have large local datasets.Source: §5 · p. 8

Additional ablations

  • The supplementary ablations examine random versus fixed intermediate-block sampling, limiting the sampled blocks, unifying-round count, and splitting Benchmark 1 domains across more clients.Source: §5 · p. 8

6. Conclusion

The conclusion presents FedSIS as a privacy-preserving approach to generalized FacePAD, combining intermediate-feature augmentation with federated split learning. It also identifies communication, privacy, continual-learning, and unseen-attack boundaries.

Conclusion

  • FedSIS combines a ViT architecture and federated split learning to train across source domains without pooling their data.Source: §6 · p. 8
  • Intermediate representation sampling uses a block sampler and adapter network to exploit intermediate ViT features as feature augmentation.Source: §6 · p. 8

Limitations

  • The authors report state-of-the-art domain-generalization performance while preserving privacy, but flag communication burden, possible privacy leakage, lack of continual learning, and unevaluated unseen attack types.Source: §6 · p. 8

Supplementary Material

The supplementary material is organized around implementation, robustness, and additional ablation details.

Supplementary Material

7. Dataset and Implementation Details

The two benchmarks use datasets with varied subjects, capture conditions, modalities, and attack types. Frame sampling and evaluation differ between benchmarks.

7.1. Datasets in Benchmark 1

  • Benchmark 1 uses MSU-MFSD, CASIA-MFSD, Idiap Replay-Attack, and OULU-NPU, which vary in subjects, capture conditions, and print or replay attacks.The datasets contain 280, 600, 1,300, and 3,600 videos, respectively.Source: §7.1 · p. 12

7.2. Datasets in Benchmark 2

  • Benchmark 2 uses WMCA, CASIA-CeFA, and CASIA-SURF, with varied attacks and modalities, but evaluation uses only print and replay attacks in RGB.The datasets include depth, infrared, or thermal channels, but those are not used in this evaluation.Source: §7.2 · p. 12

7.3. Frame Sampling Strategy

  • Benchmark 1 samples one training frame and evaluates two frames per video, while Benchmark 2 samples ten frames for WMCA and CASIA-CeFA and all CASIA-SURF frames.Video-level metric probabilities are averaged across frames in Benchmark 1.Source: §7.3 · p. 12

8. Robustness Across Multiple Runs

The authors assess run-to-run variation using three seeds and report low average standard deviations across both benchmarks. They attribute checkpoint fluctuations to training instability, domain gap, and random block sampling.

  • Across experiments with three different seeds, average standard deviation was 0.11% for Benchmark 1 and 0.99% for Benchmark 2, indicating robustness to training instabilities.Source: §8 · p. 12
  • Checkpoint variation is attributed to catastrophic forgetting with few samples, uncertainty from large source-target gaps, and FedSIS’s random block sampling.The authors note that target samples near the decision boundary can contribute to uncertainty.Source: §8 · p. 12
  • To account for these fluctuations, each experiment is run with three different seeds and reported as a mean and standard deviation.Source: §8 · p. 12

9. Ablation Studies

The ablations examine attack-type heterogeneity, intermediate-block sampling choices, and federation frequency. They report that FedSIS remains competitive under attack-split clients and that sampling blocks 1–6 is generally effective.

  • Tables 4 and 5 compare centralized and federated methods on Benchmarks 1 and 2 using three-run means and standard deviations; Table 7 compares FedSIS with additional centralized methods.The supplied captions specify comparison scope and reporting protocol, not table-cell outcomes.Source: Table 4 · p. 13, Table 5 · p. 13, Table 7 · p. 14
  • Without the block sampler, individual ViT blocks perform differently by scenario and are either worse or comparable to FedSIS.Later blocks perform better for OCI → M, while initial blocks perform better for OCM → I.Source: §9 · pp. 12–13, Figure 4 · p. 13
  • Except for OMI → C, sampling blocks 1–6 performs better than sampling 1–12 or 6–12, while reducing computation relative to using all blocks.The comparison with individual-block results is also reported as evidence for the sampler’s discriminative value.Source: §9 · pp. 13–14, Figure 5 · p. 14
  • Federating tokenizer and classification heads every 10 rounds gives the best reported performance, while different federation frequencies achieve comparable performance.The authors note that lower federation frequency can reduce client-server communication costs.Source: §9 · p. 14, Figure 5 · p. 14
  • With one attack type per client, FedSIS performs on par with clients combining attack types and outperforms other collaborative methods by large margins.The split setting also varies environmental factors; it uses six or seven training clients depending on whether CelebA-Spoof is included.Source: Table 6 · p. 13, §9 · p. 14

Related papers

Loading 2308.10236v2…