PaperlayerView exact revision on arXiv

Source-linked AI summary

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

Zhenyan Liu, Hua Zhang, Haoran Gao, Qi Li, Hongliang Zhu, Huiyu Zhou, Zongliang Shen, Yanxin Xu, Jiahui Wang

arXiv:2608.15310v1cs.CRcs.LG

TL;DR

Under non-IID client data, federated learning can forget global knowledge and lack supervision for missing classes. FedADB uses generated class anchors and dual-branch training, improving average accuracy over the second-best method by 6.78% cross-device and 4.71% cross-silo.

  • Problem

    Under non-IID client data, federated learning struggles to preserve global knowledge while adapting locally and supervising classes missing from clients.

  • Method

    FedADB generates one differentiably optimized class anchor per class and combines global-consistency and local-calibration branches during client training.

  • Results

    FedADB improved average accuracy over the second-best method by 6.78% in cross-device and 4.71% in cross-silo scenarios across datasets and non-IID settings.

  • Takeaways & Limitations

    FedADB remains effective under extreme data heterogeneity while stabilizing global model performance and mitigating catastrophic forgetting.

Abstract

from arXiv · show

Multimodal data collected by heterogeneous devices are used for collaborative training, where federated learning (FL) serves as a key paradigm for effective distributed modeling with data privacy preservation. However, local training suffers from the forgetting of previously learned global knowledge under cross-client data heterogeneity, which leads to significant declines in both performance and convergence speed. Most previous studies rely on global alignment strategies to retain global knowledge, which hinder local optimization and lead to inadequate supervision of missing classes. Some studies introduce proxy datasets to supplement supervision for missing classes. However, it remains a challenge to balance class-wise global consistency and local optimization objectives without proxy datasets. In this work, we propose FedADB, a Class Anchor-Driven Dual-Branch FL framework. Specifically, the server generates class anchors optimized in a differentiable input space, which are shared across clients. These class anchors serve as global references that provide supervision for missing classes during local training. A dual-branch collaborative training mechanism is designed for clients. In this mechanism, the anchor-based global branch focuses on learning with global consistency, achieving global knowledge alignment by class-anchor balanced sampling. The local calibration branch focuses on learning discriminative local features, mitigating the degradation of local representations caused by excessive global alignment. Extensive experiments across multiple medical and natural datasets demonstrate that FedADB achieves significant improvements in both accuracy and convergence speed.

1 Introduction

FedADB addresses catastrophic forgetting in non-IID federated learning by combining privacy-preserving class anchors generated from the global model with dual-branch training that balances global generalization and local adaptation. Experiments on natural and medical image datasets validate its effectiveness.

  • Motivation: Federated learning enables collaborative model training without sharing raw data, addressing legal, ethical, and institutional barriers to centralized multimedia-data training.
  • Related approaches: Existing approaches mainly preserve global knowledge through gradient or objective adjustments, or through knowledge distillation.
  • FedADB: FedADB combines Class Anchor Generation and Dual-Branch Cooperative Training to mitigate catastrophic forgetting under non-IID data.Experiments include performance comparisons, ablation studies, and case-study analyses on natural and medical image classification datasets.
  • Class Anchor Generation: CAG constructs one shared anchor per class solely from the global model, avoiding external data and additional privacy-leakage risks.The anchors are generated through differentiable optimization in the input space and provide lightweight global knowledge preservation.
  • Dual-Branch Cooperative Training: DBCT combines a global branch with a local calibration branch so the shared feature extractor learns globally generalizable and locally adaptive representations.The Local Calibration Branch optimizes directly on each client’s local dataset to avoid degradation from excessive alignment with the global objective.

2 Related Work

This section situates FedADB within federated-learning methods addressing data heterogeneity and catastrophic forgetting. Prior work mitigates client drift through local-update strategies, server-side aggregation, optimization-based guidance, or knowledge distillation.

  • Federated learning under heterogeneity: FedAvg suffers substantial performance degradation under severe data heterogeneity, motivating refinement strategies for federated learning.FL enables collaborative model training without sharing clients’ raw data.
  • Federated learning under heterogeneity: Client-drift methods constrain or correct local updates, including Fed-Prox, SCAFFOLD, MOON, and FedDyn.
  • Federated learning under heterogeneity: Another refinement category focuses on server-side aggregation to address federated-learning challenges under heterogeneous data.
  • Catastrophic forgetting in federated learning: In federated learning, catastrophic forgetting arises from distribution shifts during local training, causing clients to forget out-of-distribution knowledge and degrading global performance.
  • Catastrophic forgetting in federated learning: Unlike sequential-data continual learning, this work addresses catastrophic forgetting across spatially distributed data through optimization-based guidance and knowledge distillation.FedCurv was among the earliest studies to identify catastrophic forgetting in federated learning; FedNTD is an example of knowledge-distillation research.

3 Preliminaries

The preliminaries define a federated learning system with a central server and K clients holding heterogeneous local label spaces, aiming to learn a global model that generalizes across the global distribution. They also specify the round-wise process of broadcasting the global model, local optimization, and weighted server aggregation.

  • Federated learning setup: A federated learning system comprises a central server and K clients, with client k holding dataset D_k and label space C_k⊆C.C denotes the global label space.
  • Global objective: The global objective trains model w_G to generalize across the global data distribution using clients’ local objectives L_k.
  • Communication and local training: At communication round t, the server broadcasts global model w_G^t to participating clients K_t⊆[K].
  • Communication and local training: Each participating client performs local optimization using model predictions f(x;w) and the samplewise loss ℓ.
  • Server aggregation: After local training, the server aggregates updated local models with weights σ_k to obtain the next global model w_G^{t+1}.

4 Methodology

FedADB addresses missing-class supervision and the conflict between global consistency and local adaptation through class anchors and dual-branch collaborative training. The server regenerates and broadcasts anchors each round, while clients jointly optimize global-alignment and local-calibration branches.

  • Framework Overview: FedADB supplies class-level global supervision without forcing local optimization to over-align with the global objective.This targets inadequate supervision for missing classes and the conflict between global consistency and local adaptation.
  • Class Anchor Generation: The server initializes class anchors from Gaussian noise, optimizes them using the global model, and projects updates onto the valid input domain X.The optimization uses a negative log-likelihood term and an L2 regularizer, with projection improving numerical stability and consistency with the data space.
  • Class Anchor Generation: After I optimization iterations, the server obtains N class anchors, evaluates their target-class confidence, and broadcasts the anchor set to participating clients.Anchors are regenerated after every aggregation so supervision remains synchronized with the changing global model.
  • Dual-Branch Collaborative Training: The Local Calibration Branch trains solely on local data, while both branches jointly optimize the shared feature extractor using distinct optimization strategies.The LCB captures client-dependent class priors and feature statistics, compensating for representation bias introduced by strict global alignment.
  • Dual-Branch Collaborative Training: The Anchor-guided Global Branch constructs class-balanced mixed batches from local samples and anchors, ensuring every class contributes supervision even when absent from D_k.Minimizing its loss provides stable gradient constraints across the global label space, preserving global knowledge and mitigating forgetting.
  • Adaptive Loss Weighting: The LCB weight β_k(r) depends on FL progress, client class-missing rate γ_k, and anchor quality τ_t to balance global consistency with local learning.When γ_k or τ_t is high, β_k(r) is attenuated to emphasize global consistency; otherwise, it increases more rapidly to strengthen local learning.

5 Experiment

FedADB was evaluated across diverse natural and medical datasets, FL scenarios, and heterogeneity levels, consistently improving accuracy and convergence speed over established baselines. Sensitivity and ablation studies further show robust hyperparameter behavior and complementary benefits from dual-branch learning, class anchors, and dynamic balancing.

  • Overall comparison: FedADB improved average accuracy over the second-best method by 6.78% in cross-device and 4.71% in cross-silo settings across datasets and non-IID conditions.Compared with FedAvg, the performance gains reached 13.94% and 7.13%, respectively.
  • Convergence speed: FedADB required markedly fewer communication rounds to reach target accuracies than competing methods, with convergence reported as 3.9× longer for the second-best method and 7.4× longer for FedAvg.Methods that failed to reach the target accuracy were excluded from the average, and FedADB showed more stable optimization and faster convergence.
  • Sensitivity analysis: FedADB consistently outperformed FedAvg across evaluated hyperparameter settings and showed only minor accuracy variation with the number of anchors per class.Changing local epochs or learning rate could improve performance, but FedADB maintained a distinct advantage throughout the evaluated range.
  • Sensitivity analysis: FedADB exhibited minimal performance fluctuations across diverse CAG configurations, with default settings located in high-performing regions.The default values were indicated by red dots in Fig. 3.
  • Ablation study: Integrating all components produced gains of +16.19%, +11.53%, and +8.52% under α=0.05, 0.1, and 0.2, respectively, exceeding individual-component improvements.AGB with LCB yielded +8.20%, +2.64%, and +1.41%, while AGB with CAG yielded +6.53%, +2.54%, and +2.68%; anchors provide supervision for missing or underrepresented classes.
  • Ablation study: Dynamic β outperformed fixed branch weights from 0.1 to 0.9 on CIFAR-10 and CIFAR-100 by adjusting branch contributions according to data heterogeneity and anchor quality.This dynamic adjustment provided a more optimal balance and better performance than fixed weighting.

6 Analysis

FedADB preserves global knowledge by converging to a flatter, more isotropic loss landscape and aligning class anchors with real-data semantics. It also generalizes across cross-domain shifts while transmitting anchors without additional privacy leakage beyond the global model.

  • Loss Landscape: FedADB converges to the flattest and most isotropic region, achieving the highest accuracy (74.78%), whereas FedAvg reaches the lowest accuracy (52.27%).FedAvg also has the highest λ1 = 311 and λ1/λ5 = 2.41, indicating severe directional non-uniformity.
  • Class Anchor Prototype Alignment: Class anchors progressively converge toward corresponding real-data cluster centers, while expanding inter-class margins produce clearer separation by T2.The alignment is visualized through t-SNE embeddings on SVHN with α = 0.1.
  • Class Anchor Prototype Alignment: PCA correlations increasingly strengthen along matching classes and suppress off-diagonal correlations, demonstrating improved class-wise correspondence between anchors and real-data centers.Pearson correlations are computed between class anchors and real-data class centers using the first three principal components.
  • Cross-Domain Dataset: FedADB achieves a 7.59% performance margin over the strongest baseline on PACS under cross-silo label-space mismatch.The result demonstrates generalization across diverse visual domains under severe feature shifts and label skew.
  • Privacy Analysis of Class Anchors: Transmitting class anchors introduces no additional privacy leakage beyond sharing the global model, supported by low PSNR values and visually unrecognizable details.This privacy analysis covers CIFAR-10, SVHN, and PathMNIST anchors.

7 Conclusion

The conclusion identifies conflicting global-preservation and local-adaptation objectives, along with inadequate missing-class supervision under non-IID settings, as key challenges in federated learning. It proposes FedADB to address these challenges through two innovations, beginning with direct construction of class anchors.

  • FedADB addresses conflicting global knowledge preservation and local adaptation objectives, while improving supervision for missing classes under non-IID settings.
  • The framework’s first innovation is constructing class anchors directly from the glob…
Loading 2608.15310v1…