Fed-BRDECS: Privacy-Preserving and Heterogeneity-Aware Federated Deep Embedded Clustering

1 INTRODUCTION

• DEC-style objectives require dataset-level assignment frequencies, forcing clients either to expose cluster statistics or optimize from potentially biased local views.Heterogeneity can bias centroid updates, leave clusters inactive, and amplify early saturation.Source: §1 · pp. 1–2
• Fed-BRDECS replaces globally normalized clustering with local sample-stability optimization, avoiding transmission of local assignment statistics.The loss uses local samples and current centroids.Source: §1 · p. 2
• Prediction-balanced sampling oversamples locally rare predicted clusters, while periodic centroid restarting refreshes centroids to address imbalance and collapsed clusters.Both mechanisms use model-induced predictions rather than ground-truth labels.Source: §1 · p. 2
• Fed-BRDECS consistently improves over representative federated clustering and deep clustering baselines, with ablations validating stability loss, balanced sampling, and restarting.The introduction reports the overall comparison without specifying metric values or datasets.Source: §1 · p. 2
• Fed-BRDECS improves reconstruction-based detectors on heterogeneous federated time-series data without adding inference-time cost.The evaluation includes public benchmarks and a proprietary sensor dataset.Source: §1 · p. 2

Deep clustering

• DEC learns representations through a clustering objective, IDEC adds reconstruction, and DECS uses sample-stability optimization instead of pseudo-targets.Fed-BRDECS adapts stability optimization and centroid-level restarting to heterogeneous FL without parameter resets.Source: §2 · p. 3

Federated clustering

• Federated clustering includes centroid-based, fuzzy, and deep methods, but existing approaches do not jointly resolve deep representation learning, heterogeneity, collapse, and assignment-statistic privacy.The passage contrasts methods including k-FED, FKM, FFCM, F-DEC, FDEC, and CCFC.Source: §2 · p. 3

Imbalance in federated and unsupervised learning

• Supervised FL rebalancing relies on labels or label-distribution information, which are unavailable in unsupervised federated clustering.The passage notes that supervised methods use local rebalancing, loss reweighting, or aggregation adjustment.Source: §2 · p. 3
• Unlabeled approaches can use inferred clusters as pseudo-classes, but Fed-BRDECS uses predicted assignments only to define local sampling distributions for oversampling.Its predicted assignments are not treated as supervised targets.Source: §2 · p. 4

3.1 Preliminaries: DEC/IDEC

• DEC/IDEC pretrain an autoencoder, initialize centroids, then jointly refine representations and assignments; IDEC retains reconstruction loss during refinement.IDEC constructs an auxiliary target distribution from soft cluster frequencies, while DEC refines using clustering loss without reconstruction loss.Source: §3.1 · p. 4

3.2 Problem Formulation: Federated Deep Embedded Clustering

• In federated learning, computing IDEC’s global target requires clients to upload local soft counts, revealing local cluster composition and relying on periodically refreshed statistics.The formulation instead seeks one global clustering model from decentralized, unlabeled, potentially non-IID client data.Source: §3.2 · p. 5
• Fed-BRDECS replaces the globally normalized IDEC target with local sample stability, balances local predictions through sampling, and periodically reinitializes centroids.These mechanisms address the need to learn representations and centroids across decentralized data without centralized assignment statistics.Source: §3.2 · p. 5

3.3 Algorithm: Fed-BRDECS

• The algorithm uses federated reconstruction pretraining and centroid initialization, followed by refinement with local sample stability, prediction-balanced sampling, and periodic federated restarting.Centroid initialization uses k-FED; restarts recluster current client embeddings while keeping the representation network continuous.Source: §3.3 · pp. 5–7
• Clients compute sample stability from local assignment probabilities instead of constructing targets from globally normalized cluster frequencies.The local stability objective avoids dependence on global frequencies, but has no explicit cluster-frequency balancing term.Source: §3.3 · p. 7, §3.4 · p. 8

3.4 Theoretical Analysis

• Inactive-centroid displacement under gradient refinement is bounded when assignment mass stays low, while restarting into the current embedding convex hull restores strictly positive assignment mass.The analysis attributes the weak gradient signal to assignment mass vanishing with inactivity; restarting relocates centroids into the current embedding region.Source: §3.4 · p. 8, Appendix A.4 · p. 8

4.1 Setup for Clustering on Public Datasets

• Clustering experiments cover MNIST, Fashion-MNIST, KMNIST, and REUTERS-10K, using ten clients, IID and Dirichlet non-IID partitions, and ACC/NMI evaluation.Ground-truth labels are used only for client partitioning and final evaluation; results are averaged over five runs unless otherwise stated.Source: §4.1 · pp. 8–10

4.2 Clustering Results

• Fed-BRDECS attains the highest ACC in all eight settings and the strongest, most consistent clustering performance across datasets and partitions.Its advantage is particularly evident under non-IID distributions.Source: Figure 1 · p. 9, §4.2 · p. 10
• Deep clustering methods generally outperform fixed-feature federated k-means, while F-DEC can underperform Fed-AE, especially under non-IID partitions.On MNIST-IID, the best fixed-feature baseline obtains 0.4914, whereas Fed-AE reaches 0.8155.Source: §4.2 · p. 10
• A 6.47-percentage-point average ACC gain over F-DEC accompanies the ablation ladder, where adding balanced sampling and restarting further improves ACC.Fed-DECS, using only sample stability, is already competitive with F-DEC while avoiding transmission of local cluster-frequency statistics; the added components help correct biased or inactive centroids.Source: §4.2 · pp. 10–11
• Fed-BRDECS obtains 0.6889 ACC averaged over four non-IID settings; MNIST visualizations also show it mitigating cluster collapse and mix-up.The visual improvement is especially apparent under non-IID partitioning, where F-DEC exhibits stronger cluster mixing.Source: Figure 2 · p. 10, §4.2 · p. 11

4.3 Application to Federated Time-Series Anomaly Detection

• AUC-ROC improves in all five evaluated dataset–backbone settings, while F1 improves in four of five with Fed-BRDECS.For detectors with a clustering component, the clustering objective is replaced; purely reconstruction-based detectors receive a temporary projection head and centroids during training. The clustering head is discarded at inference.Source: §4.3 · pp. 11–12, Table 2 · p. 12

5 CONCLUSION

• Fed-BRDECS does not provide formal differential privacy because model updates and k-FED centroid means are still communicated.The conclusion identifies adding formal privacy mechanisms for these centroids as a natural next step.Source: §5 · p. 12

A THEORETICAL ANALYSIS: PROOFS AND ADDITIONAL DETAILS

• The Student-t affinity and frequency-vector notation rewrite the assignment and target equations in terms of unnormalized affinities and cluster frequencies.Source: A THEORETICAL ANALYSIS: PROOFS AND… · p. 17

A.1 Auxiliary lemmas

• The Student-t derivative lemma bounds how assignments change with embeddings and centroids, with the kernel factor maximized at r = √α.At that point, the factor equals √α/2.Source: §A.1 · p. 17
• The multiplicative-reweighting lemma bounds the change in a probability vector under positive weights, and its bound is attainable when C = 2.Source: §A.1 · p. 18

A.2 Local-target distortion under local DEC-target reconstruction

• A local DEC target matches the global target if and only if normalized local and global soft-frequency vectors are identical; otherwise target probabilities and the clustering gradient can differ.Source: §A.2 · pp. 18–19
• A locally rare cluster is upweighted and a locally common cluster downweighted relative to the global DEC target.Source: §A.2 · p. 20
• Under heterogeneity, local-frequency target reconstruction can reverse which centroid receives the larger target probability, motivating removal of the frequency-normalized target rather than approximating it locally.Source: §A.2 · p. 20

A.3 Balance-agnostic stability objective

• The stability objective does not depend explicitly on cluster frequencies and can reach a small loss even when all samples use one centroid and the others are inactive.Source: §A.3 · p. 20

A.4 Proof and strengthened form of Proposition 1.3

• Centroid movement during local optimization is controlled by the cumulative assignment mass it receives, and the stated restart bound assumes refreshed centroids lie in the convex hull of current embeddings.Source: §A.4 · pp. 21–23

B.1 k-FED Centroid Initialization and Restarting

• k-FED initializes and restarts centroids from uploaded local cluster means, keeping encoder and decoder parameters unchanged during refreshes.The method avoids transmitting local soft-assignment and cluster-count statistics for this centroid construction.Source: §B.1 · p. 23

B.2 Privacy Scope of Fed-BRDECS

• Fed-BRDECS reduces assignment-statistic disclosure during refinement but does not provide formal differential privacy and still communicates model updates and local centroid means.Source: §B.2 · p. 24

B.3 Why DP-Noised Cluster Counts Still Reveal Local Composition

• 0.002 expected ℓ1 error in recovered composition for C = 10, nk = 5,000, Δ = 1, and ε = 1 illustrates how accurately the server can estimate local composition.Source: §B.3 · p. 25
• For a fixed composition gap, hiding client-level composition requires ε = O(1/nk), making Laplace noise on each count comparable to the client dataset size and degrading count utility.Source: §B.3 · p. 25
• Moderate noise can preserve clustering utility while still revealing accurate local composition, whereas noise large enough to obscure composition makes the statistics unreliable for clustering.Source: §B.3 · p. 25
• Fed-BRDECS avoids this disclosure channel during refinement by not transmitting local soft assignments, cluster-frequency vectors, or predicted-cluster histograms.Source: §B.3 · p. 25

• Image benchmarks use official test splits, flatten images, and feed them to a standard DEC-style multilayer perceptron.Source: §C.1 · p. 25
• Image models use symmetric decoders and 10-dimensional embeddings with C = 10; labels are reserved for partitioning and evaluation.Source: §C.1 · pp. 25–26
• REUTERS-10K uses four single-label root categories, 2,000-word TF–IDF features, and a 10,000-example sample.The sample is drawn because of computational constraints.Source: §C.1 · p. 26

• Centroid-based baselines operate on raw pixels or TF–IDF, while deep methods learn autoencoder representations; federated deep methods use FedAvg.Source: §C.2 · p. 26
• The clustering-loss weight λstab is 1.0 for IID and 0.5 for non-IID partitions, with λvar set to 0.8.These λstab values are selected based on results for MNIST and Fashion-MNIST.Source: §C.2 · p. 26
• Figure 3 examines λstab sensitivity on Fashion-MNIST and MNIST.Source: Figure 3 · p. 26
• Clients use momentum SGD and cosine annealing, performing 20 local optimization steps per communication round.The learning-rate scheduler spans pretraining and refinement; it is not reset at centroid restarts.Source: §C.2 · pp. 26–27

C.3 Full Clustering Results

• Table 3 reports ACC and NMI for IID and non-IID partitions with 10 clients, averaging results over five random-seed runs.Source: §C.3 · p. 27, Table 3 · p. 27
• DEC-based refinement performs below Fed-AE, especially under non-IID distributions, where refinement can degrade pretrained representations.Source: Table 3 · p. 27

D.1 Related Work

• DEC-based detectors are common in centralized learning but scarce in federated learning; Fed-BRDECS is presented as a backbone for federated unsupervised outlier detection.Source: §D.1 · p. 27

• The TSAD adaptation adds a projection head and centroid-based Student-t assignments to backbone embeddings, combining clustering with reconstruction loss.Source: §D.2 · p. 28
• Prediction-balanced sampling raises exposure of locally rare pseudo-regimes, while γ softens rebalancing to limit over-concentration and anomalous-window overfit.The TSAD experiments use γ = 0.5.Source: §D.2 · pp. 28–29
• With LSTM-AE, the projection head attaches to the encoder’s final hidden state; reconstruction remains unchanged and no temporal pooling is needed.Source: §D.2 · p. 28
• TranAD attaches clustering to first-phase encoder memory, mean-pools across time, preserves its native weighted reconstruction loss, and scores second-phase error.Source: §D.2 · pp. 28–29
• After pretraining, local k-means centroids are aggregated by k-FED and periodically refreshed; the temporary clustering head is discarded at inference.Source: §D.2 · pp. 28–29
• The proprietary dataset contains 72,808 irregular samples at a 3.12% anomaly rate across five factories and ten sparsely supported products; labels are evaluation-only.Source: §D.2 · pp. 29–30
• Product-dependent sequence lengths and sensor dynamics motivate CoFormer-AE for irregular data; TranAD is omitted because regular fixed-window inputs are incompatible.Source: §D.2 · p. 30, Figure 5 · p. 31