Source-linked AI summary

Siloed Federated Learning for Multi-Centric Histopathology Datasets

Mathieu Andreux, Jean Ogier du Terrail, Constance Beguier, Eric W. Tramel

arXiv:2008.07424v1cs.CVcs.LGeess.IV

TL;DR

Medical histopathology federated learning faces inter-center heterogeneity and privacy-assurance challenges. The paper proposes SiloBN, which uses local-statistic batch-normalization layers without sharing local activation statistics. On Camelyon16 and Camelyon17, it reports comparable or better intra-center generalization and better out-of-domain performance and stability than existing methods.

  • Problem

    Federated learning for histopathology faces inter-center data heterogeneity and challenges in providing well-understood privacy assurances for communicated model parameters.

    Source: §1 · pp. 1–2
  • Method

    SiloBN uses local-statistic batch-normalization layers and does not communicate local activation statistics during collaborative training.

    Source: §1 · p. 2
  • Results

    SiloBN yields similar or better intra-center generalization than existing federated methods and outperforms them in out-of-domain performance and stability.

    Source: §1 · p. 2, §6 · p. 9
  • Takeaways & Limitations

    The authors conclude that SiloBN has a better privacy profile because it does not share local activation statistics.

    Source: §6 · p. 9

Abstract

from arXiv · show

While federated learning is a promising approach for training deep learning models over distributed sensitive datasets, it presents new challenges for machine learning, especially when applied in the medical domain where multi-centric data heterogeneity is common. Building on previous domain adaptation works, this paper proposes a novel federated learning approach for deep learning architectures via the introduction of local-statistic batch normalization (BN) layers, resulting in collaboratively-trained, yet center-specific models. This strategy improves robustness to data heterogeneity while also reducing the potential for information leaks by not sharing the center-specific layer activation statistics. We benchmark the proposed method on the classification of tumorous histopathology image patches extracted from the Camelyon16 and Camelyon17 datasets. We show that our approach compares favorably to previous state-of-the-art methods, especially for transfer learning across datasets.

1 Introduction

The introduction frames inter-center data heterogeneity and privacy assurances as barriers to federated learning for histopathology. It presents SiloBN and reports favorable intra-center and out-of-domain generalization results.

  • Inter-center distribution differences can bias models or prevent convergence, and histopathology commonly varies in staining, scanner configuration, and imaging artifacts.Source: §1 · p. 2
  • The paper demonstrates federated tile-level histopathology image classification with Camelyon16 and Camelyon17.The authors describe this as among the first such examples to their knowledge.Source: §1 · p. 2
  • SiloBN uses local-statistic batch-normalization layers to address inter-center variability without communicating local activation statistics.The authors say this reveals less sensitive information.Source: §1 · p. 2
  • SiloBN achieves the same or better intra-center generalization than existing federated techniques and better out-of-domain generalization.The introduction describes these results as holding even in challenging settings.Source: §1 · p. 2

2 Background

Federated learning trains a shared model through iterative updates from participants’ private datasets. The background describes challenges posed by heterogeneous participant data and approaches designed to address them.

  • Federated learning optimizes a shared model over participant losses, using sampled participants and local optimization before model aggregation.FedAvg performs several local optimization steps before aggregation to reduce coordination rounds.Source: §2 · pp. 2–3
  • FedAvg has no natural convergence guarantee on highly dissimilar participant datasets, while FedProx adds a quadratic loss term to restrain local-model divergence.Other approaches adapt participant sampling or cluster participants into effective collaborators.Source: §2 · p. 3
  • The paper retains an i.i.d. assumption within each participant, but not across participants.Source: §2 · p. 3

3 FL in Healthcare

The paper considers collaborative federated learning for tile-level whole-slide image classification across medical centers. It distinguishes this setting from consumer federated learning and motivates full-participation rounds.

  • Medical collaborative federated learning involves few centers with sizable, higher-quality local datasets, unlike consumer federated learning with many participants and comparatively little data each.Source: §3 · p. 3
  • In collaborative federated learning, participants seek to improve their local task performance through collaboration and keep sensitive evaluation data at their centers.Consumer federated learning instead centers on a service provider distributing a global model to participants.Source: §3 · pp. 3–4
  • The study trains a tile-level whole-slide image classifier across centers, using full-participation rounds because strong participant subsampling can bias updates and raise privacy challenges.The authors use E to denote the number of local mini-batch updates per round.Source: §3 · p. 4

4 Proposed Method

SiloBN separates batch-normalization statistics, which encode local domain information, from learned BN parameters, which can transfer across domains. It keeps the statistics local while sharing learned BN parameters and non-BN parameters, yielding center-specific models and reducing shared information.

Batch Normalization

  • BN renormalizes an input tensor using channel-wise running means and variances, alongside learned affine parameters.The means and variances are computed across spatial and batch dimensions.Source: §4 · p. 4

SiloBN

  • Unlike naive FedAvg with BN layers, which aggregates both BN statistics and learned parameters, SiloBN separates their treatment.BN statistics encode local domain information, while learned BN parameters can transfer across domains.Source: §4 · pp. 4–5
  • SiloBN shares learned BN parameters across centers while keeping BN statistics local and sharing non-BN parameters in the standard fashion.The local statistics help center-specific intermediate activations remain similarly centered across centers.Source: §4 · p. 5

Model Personalization and Transfer

  • SiloBN learns one model per center, and AdaBN can adapt it to an unseen target by recomputing BN statistics while freezing other parameters.The recomputation uses a data batch from the new target domain.Source: §4 · p. 5

Privacy and BN

  • Because SiloBN does not share BN statistics, the central server never accesses all network parameters, reducing shared information that could aid privacy attacks.The passage states that this can only help diminish the effectiveness of attacks that are most effective with full-parameter knowledge.Source: §4 · p. 5

5 Experiments

The experiments evaluate tumor-versus-healthy tile classification across Camelyon-derived, multi-hospital datasets. SiloBN performs on par with or better than other federated approaches in-center and outperforms FedAvg in transfer from FL-C16 to FL-C17, with more stable results.

5.1 Datasets

  • FL-C16 and FL-C17 classify tumorous versus healthy tiles from Camelyon images drawn from two and five hospitals, respectively.Tiles are partitioned into training, validation, and test sets using per-hospital stratification; tiles from the same slide in FL-C16 or patient in FL-C17 stay in one partition.Source: §5.1 · p. 6

5.2 Experiments on FL Histopathology Datasets

  • The study compares FedAvg, FedProx, pooled, and independent local training with SiloBN, using DCNNs with and without BN and held-out center-specific AUCs.Experiments vary local updates per federated round; mAUC is the pan-center mean of intra-center AUCs.Source: §5.2 · pp. 6–7
  • SiloBN outperforms other federated strategies with DCNN+BN on both datasets and is on par with pooled and local training; without BN, FedAvg and SiloBN are equivalent.The full table reports mean AUC across centers and shows SiloBN as the most effective approach when using BN layers.Source: §5.2 · p. 8, Table 1 · p. 8

5.3 Out-of-domain generalization experiments

  • From FL-C16 to FL-C17, SiloBN with AdaBN outperforms FedAvg-trained DCNN on average and is more stable across trainings and centers.The table summarizes domain generalization from FL-C16 to FL-C17.Source: §5.3 · p. 8, Table 2 · p. 8

6 Conclusion

SiloBN uses local-statistic BN layers and achieves similar or better intra-center generalization than existing FL methods on multicentric histopathology datasets. It outperforms them in out-of-domain performance and stability while not sharing local activation statistics.

Conclusion

  • SiloBN yields similar or better intra-center generalization than existing FL methods on real-world multicentric histopathology datasets.Source: §6 · p. 9
  • SiloBN outperforms existing methods on out-of-domain generalization in both performance and stability.Source: §6 · p. 9
  • By not sharing local activation statistics, SiloBN has a better privacy profile; future work could quantify the privacy benefits and study other domains such as radiology.Source: §6 · p. 9

A Supplementary Material

The supplementary material presents tile distributions across centers, validation-selected hyperparameters, the DCNN architecture, and representative whole-slide images (WSIs).

A Supplementary Material

  • Table 3 gives the distribution of healthy and tumor-containing tiles across centers for each federated dataset.The caption provides no tile counts.Source: Table 3 · p. 11
  • For each dataset, local training uses E = 1; the table also reports validation-selected hyperparameters, including BN momentum 0.1 when applicable and eps = 10^-5.The Adam first- and second-moment decay hyperparameters are denoted β1 and β2.Source: Table 4 · p. 11
  • Figure 3 shows the DCNN architecture used in all experiments; convolutions use spatial zero-padding of size 1, and convolutional and linear layers include biases.Each convolution’s output-channel count corresponds to its block’s output-channel count.Source: Figure 3 · p. 12
  • Figure 4 presents four representative WSIs for each identified center in the Camelyon16 and Camelyon17 datasets.Source: Figure 4 · p. 12
Loading 2008.07424v1…