PaperlayerView exact revision on arXiv

Source-linked AI summary

GCA: Global Centroid Alignment in Federated Learning

Jong-Ik Park, Harry Jiang, Logan Blakely, Georgios Fragkos, Shamina Hossain-McKenzie, Carlee Joe-Wong

arXiv:2608.22593v1cs.LGcs.DC

TL;DR

AE-based federated anomaly detection must balance limited local data with the communication and privacy risks of sharing reconstructive model updates. GCA replaces those updates with latent codes and global centroid statistics, and across seven benchmarks it reports stronger extraction defense, competitive accuracy, and substantially lower communication. Its demonstrated scope remains bounded by the need to study stronger adversaries and privacy-preserving clustering in future work.

  • Problem

    Conventional AE-based FL exchanges parameters or gradients, which creates substantial communication overhead and may expose training-data information because AEs reconstruct their inputs.

  • Method

    GCA coordinates clients by uploading sampled latent codes, clustering them into global centroids with support counts, and aligning local encoders without sharing AE parameters or gradients.

  • Results

    Across seven datasets, GCA outperforms FedAvg on five datasets by up to 5.76% while reducing per-round communication by 84.14%–99.15%; it also has higher normalized target MSE in all 21 comparisons and lower excess cosine similarity in 20 of 21 comparisons with FedAvg, FedProx, and FedNova.

  • Takeaways & Limitations

    GCA provides stronger and more consistent empirical extraction defense than the compared standard FL methods while retaining competitive accuracy and reducing communication in the evaluated settings.

  • Takeaways & Limitations

    Future work is needed on stronger adversaries, dynamic centroids, uncertainty-aware responsibilities, and robust or privacy-preserving clustering with formal privacy guarantees.

Abstract

from arXiv · show

Autoencoder (AE)-based federated learning (FL) is attractive for anomaly detection when clients have limited local data. However, conventional FL exchanges AE parameters or gradients, incurring substantial communication overhead and potentially exposing input training data information, since AEs are explicitly optimized to reconstruct their inputs. We introduce \emph{Global Centroid Alignment (GCA)}, a latent-code-mediated FL protocol that coordinates clients without transmitting AE parameters or gradients. In each round, (1) clients first train their local AEs using a \emph{reconstruction} update and upload a small subset of encoder latent codes to the FL server. (2) The server pools these codes, fits a clustering model, and broadcasts only \emph{global latent centroids and their support counts}. (3) Each client then updates its encoder by aligning its local latent codes with the \emph{nearest} centroid using \emph{inverse-count} weighting to emphasize globally underrepresented patterns. Steps (1)--(3) repeat over communication rounds. Because GCA exchanges only sampled latent codes and centroid statistics, its communication cost depends on latent dimensionality and the numbers of uploaded codes and returned centroids rather than on AE model size. Across five tabular and two vision benchmarks, GCA yields higher reconstruction error under a server-side client data extraction attack in all 21 comparisons and clearly lower cosine similarity in 20 of 21 comparisons with FedAvg, FedProx, and FedNova, showing its ability to protect training data. It even improves test accuracy over FedAvg by up to $5.76\%$. GCA achieves extraction defense comparable to DP-FedAvg, remains effective when DP-FedAvg does not reduce target resemblance, and lowers per-round communication by up to $99.15\%$.

Introduction

AE-based FL supports anomaly detection with local normal data, but exchanging AE parameters or gradients creates communication and training-data exposure concerns. GCA addresses these concerns by exchanging sampled latent codes and centroid statistics instead, with strong empirical defense and efficiency results.

  • AE-based FL trains local autoencoders on normal data for anomaly detection, but conventional practice uploads model parameters or gradients for aggregation.
  • Full AE parameter exchange is costly for resource-constrained devices, while reconstructive parameters and gradients may encode fine-grained signatures of local samples.
  • GCA avoids sharing raw data, parameters, or gradients by transferring sampled latent codes and centroid statistics.
  • Across five tabular and two vision benchmarks, GCA provides stronger extraction defense than FedAvg, FedProx, and FedNova while remaining comparable in accuracy to FedProx, FedNova, and DP-FedAvg.It also reduces per-round communication by 84.14%–99.15%.
  • The paper also analyzes round-to-round stability and argues that latent-only observations do not uniquely determine training records under its threat model.

Related Work

AE reconstruction models are useful for anomaly detection with scarce or unavailable labels, but conventional federated AE training still shares reconstructive model information. Related methods address privacy or representation sharing differently from GCA.

  • Autoencoders reconstruct normal inputs and use reconstruction error to score anomalies, often with percentile thresholds controlling false positives without labels.
  • AE-based anomaly detection is attractive when data and labels are scarce or unavailable and applies across tabular, time-series, and vision data.
  • Conventional federated AE methods keep raw data local but periodically share model parameters or gradients for aggregation.
  • Differentially private FL perturbs model updates, whereas GCA avoids sharing even perturbed reconstructive parameters or gradients; prototype methods additionally rely on predefined class semantics.

Methodology

GCA repeats local reconstruction, latent upload, server clustering, centroid broadcast, and encoder alignment. Its compact centroid exchange injects cross-client structure while inverse-count weighting emphasizes globally underrepresented patterns.

  • Methodology: Clients hold local normal data and participate in selected rounds, sending only a subset of latent codes rather than raw data, parameters, or gradients.
  • Methodology: Each global round repeats three phases: local reconstruction, server aggregation of uploaded latents, and client-side encoder alignment.The total local epochs satisfy T = T_recon + T_align.
  • Methodology: The schedule first stabilizes each local manifold, then injects cross-client structure while reducing overfitting to site-specific idiosyncrasies.
  • Methodology: The server pools uploaded latents, fits a clustering model, and broadcasts global centroids with support counts to participating clients.GMM component means are one example of the returned centroids.
  • Methodology: Clients align each encoder toward the nearest broadcast centroid using inverse-count weights that emphasize centroids with low global support.
  • Methodology: Theoretical analysis establishes stable convergence to a minimizer in the convex case and finite-time convergence to a critical point in the non-convex case.

Adversarial Attack Scenarios

The paper evaluates server-side extraction against standard FL models and GCA’s latent-only protocol. Standard FL enables white-box reconstruction optimization, whereas GCA requires a surrogate learned from latent codes and is therefore underdetermined in input space.

  • The threat model considers a malicious or curious server attacking either shared AE models in standard FL or uploaded latent codes in GCA.
  • In standard FL, the server uses the victim encoder and decoder to optimize inputs toward especially small reconstruction error.
  • Autoencoders are especially exposed to inversion because their training objective explicitly encourages reproducing inputs rather than classifying them.
  • GCA removes access to the victim decoder and instead lets the server train a surrogate inverse map using only the victim’s uploaded latent samples.
  • Both threat models apply input optimization and output reconstructions of optimized inputs, but GCA’s surrogate is trained from scratch rather than obtained as a victim model.
  • The latent-only attack is fundamentally underdetermined because many distinct input-space reconstructions can fit the same observed latent constraints.

Experimental Evaluation

Across seven tabular and vision benchmarks, GCA preserves competitive anomaly-detection accuracy while providing stronger extraction defense and substantially lower communication than parameter-sharing FL. The evaluation also specifies its one-class setup, training protocol, leakage metrics, and communication accounting.

  • Evaluation setup: GCA evaluates anomaly detection in a one-class setting using only normal client data across five tabular and two vision datasets, with three seeds.The protocol uses 100 rounds; GCA allocates five local epochs to reconstruction and five to alignment per round, while baselines use ten local epochs.
  • Evaluation setup: Clients upload latent-code subsets, the server fits modality-specific GMMs, and clients perform encoder-only alignment using inverse-count weights.The experiments use dataset-specific client counts, cluster counts, and latent upload fractions; GMM settings differ between tabular and vision datasets.
  • Accuracy: GCA outperforms Single-Client training on all seven datasets and beats FedAvg on five of seven, with relative changes from −0.92% to +5.76%.It remains competitive with FedProx, FedNova, and DP-FedAvg, with win/loss counts of 5/2, 5/2, and 4/3.
  • Extraction defense: Higher NTMSE and lower excess cosine similarity indicate attack outputs are less similar to client training data relative to held-out normal records.Both metrics are normalized within each dataset, client, and seed, using the same nearest reconstruction for distance and similarity evaluation.
  • Extraction defense: 21 of 21 NTMSE comparisons favor GCA over FedAvg, FedProx, and FedNova, while excess cosine similarity favors GCA in 20 of 21 comparisons.The sole exception is Credit against FedProx, where GCA has slightly higher excess cosine similarity (−0.559 versus −0.584).
  • Communication: 84.14%–99.15% lower per-round communication is achieved by GCA relative to full-model FL across the evaluated settings.The accounting scales GCA with uploaded latent codes, latent dimensionality, centroids, and support counts rather than model size.

Conclusion

Across seven datasets, GCA replaces AE parameter and gradient exchange with latent codes, centroids, and support counts while improving selected accuracy and reducing communication. It also strengthens extraction defense against conventional FL baselines, while future work targets stronger adversaries and formal privacy mechanisms.

  • GCA outperforms FedAvg on five of seven datasets by up to 5.76% while reducing per-round communication by 84.14%–99.15%.
  • Under matched extraction attacks, GCA has higher normalized target MSE in all 21 comparisons and lower excess cosine similarity in 20 of 21 comparisons with FedAvg, FedProx, and FedNova.
  • GCA outperforms the tested DP-FedAvg configuration in 9 of 14 dataset–metric comparisons for extraction defense.
  • Future work will examine stronger adversaries, dynamic centroids, uncertainty-aware responsibilities, robust or privacy-preserving clustering, and formal privacy guarantees.

A Implementation Details and Experimental Protocols

The evaluation uses matched autoencoder training protocols across IID and heterogeneous client partitions, while GCA communicates latent representations and centroid statistics rather than model parameters. Its design assumes compatible latent coordinates, exposes sampled latents to a latent-specific threat model, and does not provide formal differential privacy.

  • Experimental Protocols: Experiments use five tabular and two vision benchmarks with matched architectures, optimization settings, training budgets, and participation schedules where applicable.Tabular models use feed-forward autoencoders; vision models use convolutional autoencoders.
  • Data and Evaluation: Normal-only training uses reconstruction-error scoring, with each client setting its anomaly threshold to the 0.75 quantile of training reconstruction errors.IID experiments use disjoint uniform shards, while heterogeneous experiments use Dirichlet client preferences with α = 0.1.
  • GCA Protocol: Each client performs local reconstruction, uploads sampled normal latent codes, and receives K centroids with support counts from server-side clustering.Alignment updates only the encoder; the decoder remains local and inference continues using reconstruction-error thresholding.
  • Assumptions and Limitations: The experiments assume a common latent dimension and sufficiently compatible client latent coordinates, excluding arbitrary client-specific permutations or transformations.Sampled latent codes remain information-bearing, and GCA is evaluated under a latent-specific extraction threat model without a formal DP guarantee.
  • Communication Accounting: Communication accounting compares GCA’s latent-code and centroid-statistic tensors with baseline AE model tensors on a per-round FP32 payload basis.The accounting excludes transport headers, serialization overhead, acknowledgements, checkpoint I/O, optimizer state, runtime, and communication to a target accuracy.
  • Methodological Distinctions: GCA and FedProto both exchange representation-space summaries, but GCA uses unlabeled normal data, sampled latent codes, round-specific clustering, and inverse-support alignment.Unlike supervised FedProto, GCA does not require class identities or class-indexed prototypes.

C Theoretical Analysis

The analysis models GCA as an alternating reconstruction-and-alignment procedure and establishes phase-wise descent and convergence results under smoothness, regularity, assignment, weighting, and curvature assumptions.

  • Algorithmic schedule: One GCA round performs reconstruction training, latent-code uploading, server clustering, centroid broadcasting, and encoder-only alignment.The schedule separates reconstruction and alignment updates, with centroids and support counts computed between client phases.
  • Assumptions: Nearest-centroid assignments remain stable when latent codes stay within εµ of centroids and εµ is less than half the minimum centroid separation.The weighting assumptions require normalized mean-one inverse-count weights bounded between positive minimum and finite maximum values.
  • Phase-wise descent: The reconstruction and alignment phases each admit geometric one-step objective decreases under their respective smoothness and PŁ conditions.The reconstruction phase updates encoder and decoder, whereas alignment updates the encoder using fixed centroids and weights.
  • Across-phase interaction: The reconstruction step can affect alignment suboptimality, while the alignment step can affect reconstruction suboptimality, so the alternating schedule requires cross-objective bounds.The analysis controls these interactions using smoothness, Young’s inequality, and gradient-suboptimality bounds.
  • Across-round convergence: Under bounded objective gaps and local curvature assumptions, the across-round sequence remains uniformly bounded and the two-phase schedule converges toward a critical point of the co-objective F + G.The result concerns the combined reconstruction and alignment objective rather than reconstruction alone.
  • AFB interpretation: The convergence interpretation treats GCA as an imperfect alternating forward-backward method with tractable errors under typical training regimes and decaying learning rates.The co-objective combines reconstruction with a clustering/alignment objective whose encoder-fixed and centroid-fixed partial roles are specified.

D Additional Theory for Server-Side Data Extraction Attacks

The paper supplements its server-side extraction analysis with two attack routes: white-box inversion against a standard-FL autoencoder and latent-only inversion against GCA codes.

  • Attack routes: The appendix analyzes DeepDream-style inversion against both a white-box victim autoencoder and a surrogate trained only from uploaded GCA latent codes.These routes correspond to standard FL and GCA, respectively.

Why reconstruction objectives admit training-like fixed points (standard FL)

Reconstruction-based inversion can produce training-like fixed points for standard FL, whereas latent-only inversion from GCA codes is underdetermined and cannot guarantee recovery of the original records without extra assumptions.

  • Standard-FL inversion: A trained autoencoder defines the reconstruction map T(x) = D(E(x)), and standard-FL inversion minimizes the input-space reconstruction energy.This directly couples optimization to the victim autoencoder and its input domain.
  • Fixed points: When the reconstruction map has fixed points, global minimizers attain zero residual; otherwise, minimizers have the smallest strictly positive reconstruction residual.The residual is the distance between an input and its reconstruction under T.
  • Training-like solutions: If training records reconstruct well, they behave as approximate fixed points, so minimizing reconstruction energy can yield solutions on or near the learned reconstruction set.This explains why standard-FL inversion may produce training-like reconstructions.
  • Latent-only inversion: GCA’s latent-only attacker observes finite latent codes but not the corresponding private records, decoder, or gradients, leaving the input-space mapping underdetermined.The surrogate objective constrains only latent-space composition on the observed codes.
  • Identifiability boundary: The same latent set can arise from distinct private datasets related by norm-preserving transformations, while preserving autoencoder reconstruction error.Therefore, latent-only observation cannot identify the original dataset without assumptions linking input coordinates to semantic structure.

E Additional Experimental Results and Ablations

The additional experimental materials specify shared evaluation settings and component-count selection, with GCA-GMM accuracy reported across tested numbers of mixture components.

  • Experimental protocol: All implementations, shared settings, data partitions, model and K selection, attack procedures, and communication accounting follow Appendix A.This passage defines the provenance of the experimental protocol rather than reporting a result.
  • Selection rule: For each seed and component count K, GCA accuracy is taken as the maximum across reconstruction-epoch test evaluations.The selected K⋆ maximizes the mean of three seed-level values, with exact ties favoring smaller K.
  • Component-count selection: Table 3 reports the selected GCA-GMM component count for each dataset.The table is dataset-indexed and summarizes the chosen mixture-component configuration.
  • Accuracy ablation: Table 4 reports GCA-GMM test accuracy in percent by component count in the IID setting.For each dataset, the two highest unrounded means are bold, with exact ties resolved toward smaller K.

Alignment-Phase Weighting Ablation

The ablation evaluates inverse-support weighting against uniform weighting and places its alignment gains alongside clustering, accuracy, extraction-defense, and communication results. Inverse weighting improves alignment overall, while GCA maintains comparable decentralized accuracy, stronger defense against several baselines, and substantially lower communication.

  • Weighting ablation: Inverse weighting achieves higher mean accuracy on five of seven datasets and in 14 of 21 seed-level comparisons.Its unweighted average is 71.35%, versus 70.98% for uniform weighting, a 0.37-percentage-point improvement; uniform weighting performs better on Adult and MAGIC.
  • Accuracy comparison: GCA outperforms Single-Client on all seven datasets and FedAvg on five of seven, while remaining competitive with FedProx, FedNova, and DP-FedAvg.Against FedProx, FedNova, and DP-FedAvg, the win/loss counts are 5/2, 5/2, and 4/3, respectively.
  • Clustering sensitivity: Accuracy varies only modestly across component counts and clustering backends, supporting limited sensitivity to the clustering implementation.GMM remains the primary backend because its soft responsibilities directly provide support counts for inverse-support weighting.
  • Extraction defense: GCA-GMM has higher NTMSE than FedAvg, FedProx, and FedNova in all 21 comparisons and lower excess cosine similarity in 20 of 21.Against DP-FedAvg, GCA-GMM has higher NTMSE on four datasets and lower excess cosine similarity on five, yielding a mixed comparison.
  • Communication: GCA reduces per-round communication by 89.64% to 99.15% across tabular settings and by 84.14% to 87.10% for the evaluated vision settings.GCA transmits less data per round than parameter-sharing baselines for every evaluated dataset and tested component count.

Non-IID Results

Under feature-distribution and sample-count skew, clients receive different normal-data distributions and quantities. With K⋆=10 selected by the stated tie rule, GCA improves over FedAvg on four datasets and trails on MAGIC.

  • Non-IID setup: Non-IID experiments combine feature-distribution skew with a 10:1 maximum-to-minimum client sample-count ratio.A Dirichlet distribution with α = 0.1 assigns concentrated client preferences across feature groups.
  • Model selection: K⋆=10 is selected because all tested component counts attain the same reported reconstruction-epoch peak.The tie rule determines the selected component count.
  • Results: GCA improves over FedAvg on Academic, Adult, Bank, and Credit, but trails on MAGIC.Per-dataset relative changes range from −1.03% to 11.66%; the reported 2.26% compares the five-dataset macro-averages.
Loading 2608.22593v1…