PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem

Nazreen Shah, Naveen Kumar Reddy Somireddy, Zubair Shaban, Ranjitha Prasad, B. N. Bharath

arXiv:2608.21539v1cs.LG

TL;DR

FCL must adapt to sequential, heterogeneous client data while retaining prior knowledge, but existing per-round approaches do not explicitly regulate forgetting accumulated over time. The paper formulates FCL as stochastic control and introduces FEDQCL, whose virtual-queue DPP optimization balances current-task learning and forgetting. Experiments report competitive accuracy and significantly reduced forgetting under distributed, heterogeneous, non-stationary data.

  • Problem

    FCL requires adaptation to sequential, non-IID client data while retaining prior knowledge, but existing formulations do not explicitly model accumulated forgetting across time and clients.

  • Method

    FEDQCL casts FCL as a long-term constrained stochastic control problem and uses locally maintained virtual queues within a drift-plus-penalty objective.

  • Results

    FEDQCL achieves competitive accuracy while significantly reducing forgetting under distributed, heterogeneous, and non-stationary data distributions.

  • Takeaways & Limitations

    Per-round queue updates and a single control parameter provide explicit, privacy-preserving regulation of the stability–plasticity trade-off without additional communication overhead.

  • Takeaways & Limitations

    The formulation assumes synchronous task boundaries across clients, although client data distributions may differ.

Abstract

from arXiv · show

Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems, requiring models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift. Existing approaches formulate continual learning (CL) as a sequence of per-task optimization problems, applied locally at each client and coupled through aggregation, using heuristic mechanisms such as replay, regularization, or projection-based constraints. However, forgetting in FCL is inherently a long-term, distributed phenomenon, arising from the interaction of temporal task evolution and cross-client heterogeneity, which is not explicitly regulated. In this work, we cast FCL as a stochastic control problem and propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lyapunov drift-plus-penalty (DPP) optimization. FedQCL introduces virtual queues to track the accumulation of forgetting across tasks and clients, enabling explicit control of the stability-plasticity trade-off. By optimizing a DPP objective, the method jointly improves current-task performance while the queue-based formulation provides an interpretable and tunable mechanism to balance adaptation and retention through a single parameter, without requiring gradient projection or additional communication overhead. Empirical evaluations on standard benchmarks, including Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet, demonstrate that FedQCL outperforms state-of-the-art baselines with respect to accuracy while significantly reducing forgetting under heterogeneous data distributions.

1 INTRODUCTION

Federated continual learning must adapt to sequential, non-IID client data while retaining prior knowledge, but existing methods do not explicitly regulate forgetting over time. FEDQCL addresses this with virtual queues and drift-plus-penalty optimization to balance current-task learning and retention without extra communication overhead.

  • FCL extends federated learning to sequential, non-IID data while requiring adaptation to new tasks and retention of previously learned knowledge.
  • Existing regularization, generation, and gradient-projection methods mitigate forgetting but respectively constrain plasticity or incur computational, communication, or embedding-exchange overheads.
  • Most FCL formulations use myopic round-wise updates and do not explicitly account for forgetting accumulation across time and clients.
  • FEDQCL casts FCL as a long-term constrained stochastic control problem and uses virtual queues to track constraint violations across communication rounds and clients.
  • FEDQCL controls the stability–plasticity trade-off through a single hyperparameter balancing current-task learning against accumulated forgetting.
  • Virtual queues are updated per communication round and maintained locally, enabling fine-grained regulation without additional communication overhead.
  • FEDQCL outperforms state-of-the-art baselines in average accuracy and forgetting under heterogeneous and non-stationary data distributions.

2 RELATED WORKS

Prior FCL methods combine replay, regularization, distillation, or related federated optimization strategies, while DPP provides a control framework for balancing penalties and long-term constraint satisfaction. FEDQCL fills the stated gap by explicitly modeling accumulated forgetting across clients and time through queue stabilization.

  • Classical FL and Replay-based CL methods: FedAvg performs local SGD followed by server-side weighted averaging but suffers client drift and degraded convergence under statistical heterogeneity.
  • Federated Continual Learning Methods: Replay-based continual learning methods revisit stored past samples, whereas regularization and distillation methods constrain updates to preserve acquired knowledge.
  • Drift-Plus-Penalty in Machine Learning: DPP balances minimizing a time-averaged penalty with constraint satisfaction by controlling virtual queues based on time-averaged violations.
  • Novelty: Prior work addresses forgetting or heterogeneity but generally optimizes per-round objectives without explicitly modeling forgetting accumulation over time.
  • Novelty: FEDQCL uses client-level virtual-queue dynamics instead of gradient projection or dual updates, enabling projection-free local updates while preserving plasticity and privacy.
  • Novelty: FEDQCL is presented as the first approach to cast FCL as a queue-stabilization problem with controllable forgetting across time and clients.

3 PROBLEM FORMULATION

The formulation seeks global models that minimize average loss across evolving client tasks while bounding average forgetting across clients and past tasks. It accounts for non-IID heterogeneity, sequential access to data, and replay-based estimates of prior-task performance.

  • Problem formulation: The system contains N clients, each observing a sequence of tasks, while the server aggregates client contributions to learn a global model at each task.
  • Per-task formulation: The standard per-task federated objective treats tasks independently and therefore does not account for temporal data evolution, knowledge transfer, or catastrophic forgetting.
  • Sequential access: Earlier-task data is inaccessible during new-task learning, so optimization proceeds sequentially using current-task data and surrogate mechanisms for preserving prior knowledge.
  • Federated assumptions: Clients transition between tasks synchronously while their data distributions differ, creating statistical heterogeneity and conflicting updates.
  • Constrained formulation: The objective minimizes average loss across tasks while constraining average forgetting across clients using local replay buffers to approximate prior-task performance.
  • Constrained formulation: The forgetting constraint bounds the average increase in replay loss across past tasks and clients relative to the previous global model by tolerance δ.

4 PROPOSED METHOD: DRIFT-PLUS-PENALTY FORMULATION

FEDQCL formulates federated continual learning as a distributed Lyapunov drift-plus-penalty problem, using local virtual queues to track forgetting constraint violations. Clients optimize current-task loss together with queue-weighted penalties, while round-level updates provide fine-grained regulation of the stability–plasticity trade-off.

  • Drift-plus-penalty formulation: FEDQCL decomposes the global forgetting constraint into per-client, per-task components and tracks their violations with locally maintained virtual queues.This decomposition supports distributed implementation under partial participation while remaining consistent with the global objective in expectation.
  • Drift-plus-penalty formulation: Each client minimizes a drift-plus-penalty objective combining current-task loss with a drift term for forgetting constraints across past tasks.The formulation replaces direct enforcement of a coupled global constraint with locally tractable queue-regulated optimization.
  • Virtual-queue regulation: The virtual queue accumulates violations from instantaneous increases in each client’s replay loss on every past task, relative to an allowable forgetting threshold.Replay loss provides a localized measure of forgetting, while δ specifies the permitted threshold.
  • Stability–plasticity trade-off: V > 0 controls the stability–plasticity trade-off by weighting current-task learning against the accumulated forgetting penalty.Larger relative weight on the current-task loss favors plasticity, while queue-weighted penalties promote stability.
  • Round-level implementation: Queues are updated at every communication round rather than only at task boundaries, capturing continuously evolving forgetting under iterative updates and partial participation.Clients initialize from the broadcast global model, perform local stochastic-gradient updates, and send local models for server aggregation.

5 EMPIRICAL RESULTS

The evaluation studies FEDQCL in streaming, non-IID federated continual learning across three image-classification benchmarks and reports accuracy and forgetting over sequential tasks. The supplied passages define the benchmark construction and evaluation metrics but do not provide numerical results for forgetting.

  • Evaluation design: FEDQCL is evaluated against replay, regularization, and gradient-projection baselines under non-IID, streaming data regimes.The evaluation includes comparisons and ablations across heterogeneous settings.
  • Datasets: The benchmarks are CIFAR-10, CIFAR-100, and TinyImageNet, with 10, 100, and 200 classes respectively.TinyImageNet is described as the most challenging dataset because of its larger class count and higher visual diversity.
  • Metrics: Performance is assessed using server-model test accuracy across the task sequence and forgetting, which measures previously learned knowledge overwritten by subsequent learning.Average accuracy aggregates task accuracies after observing the full sequence, while forgetting uses the maximum prior accuracy decrease for each earlier task.

5.2 BASELINES

FEDQCL is evaluated against rehearsal, regularization, generative replay, federated continual-learning, and standard federated-learning baselines under non-IID task-incremental settings. It leads in accuracy and offers a stronger accuracy–forgetting balance than projection-based alternatives.

  • Baseline coverage: FEDQCL is compared with rehearsal-based, regularization-based, generative replay, FCL-specific, and standard FL baselines.The evaluation covers Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet under non-IID data.
  • Accuracy comparison: FEDQCL achieves the highest average accuracy across all three benchmark datasets, outperforming the strongest replay baseline DER++-FL.Most baselines show monotonic accuracy decline as the number of observed tasks grows.
  • Accuracy comparison: Regularization-based methods EWC-FL and LwF-FL perform on par with FedAvg, indicating parameter-space constraints alone are insufficient under federated non-IID settings.
  • Forgetting comparison: FOT attains near-zero forgetting by projecting gradients orthogonally to past-task subspaces, but at the cost of substantially lower accuracy.
  • Accuracy–forgetting trade-off: FEDQCL occupies the upper-left region of the Pareto front on Split-CIFAR-100 and Split-TinyImageNet, indicating the best balance between joint accuracy and forgetting.Its queue-based penalty dynamically balances the objectives through a single control parameter V.

5.4 VARYING DATA HETEROGENEITY

FEDQCL remains robust as Split-CIFAR-100 data distributions range from highly heterogeneous to near-IID, while retaining its advantage across memory budgets. The experiments examine distributional shift, replay-buffer size, and related hyperparameter behavior.

  • Data heterogeneity: FEDQCL maintains stable accuracy across α ∈ {0.001, 10, 10^5}, with only modest degradation under extreme heterogeneity at α=0.001.Most baselines suffer significant accuracy drops at α=0.001 because imbalanced local data exacerbates client drift.
  • Memory buffer size: FEDQCL consistently outperforms baselines at memory budgets M ∈ {200, 500, 1000} exemplars per client, including M=200.
  • Hyperparameter context: Figure 3 examines accuracy and forgetting versus V on Split-CIFAR-100 and Split-TinyImageNet, alongside δ-based plasticity–stability Pareto trade-offs.
  • Memory buffer size: The relative performance gap between FEDQCL and DER++-FL remains roughly stable across memory sizes.This indicates FEDQCL’s advantage stems primarily from queue-based regularization rather than solely from improved replay quality.

5.6 ABLATION STUDIES

The ablations show that V and δ provide interpretable controls over FEDQCL’s stability–plasticity trade-off, while virtual queues reveal how forgetting pressure is distributed across past tasks.

  • Hyperparameter V: V controls the balance between plasticity and stability: small V suppresses adaptation, while increasing V raises current-task accuracy until dataset-dependent behavior emerges.On Split-CIFAR-100, LTA peaks around V=100 to V=200; on Split-TinyImageNet, LTA increases across the tested range.
  • Hyperparameter δ: δ defines the allowable per-round replay-loss increase before queues accumulate, with negative δ imposing stricter constraints and stronger regularization.Highly negative δ limits plasticity and degrades predictive performance.
  • Virtual queue behavior: The virtual queue matrix records the queue value for past task j accumulated during training on task i under different V values.
  • Virtual queue behavior: At V=1000, queue values concentrate on early tasks, whereas at V=100 they are smaller and more uniformly distributed across past tasks.The queue matrix averages each past-task queue value across clients during training on the current task.

6 CONCLUSIONS

The paper presents FEDQCL as a control-theoretic approach that regulates forgetting in federated continual learning through local virtual queues and per-round updates. Experiments show competitive accuracy with reduced forgetting under heterogeneous, non-stationary data.

  • Conclusion: FEDQCL formulates FCL as a stochastic control problem and uses virtual queues to track forgetting across clients and communication rounds.
  • Conclusion: Per-round queue updates reflect that forgetting evolves continuously during federated learning rather than only at task boundaries.
  • Conclusion: Maintaining queues locally enables privacy-preserving forgetting regulation without additional communication overhead.
  • Conclusion: FEDQCL achieves competitive accuracy while significantly reducing forgetting under distributed, heterogeneous, and non-stationary data distributions.

7.1 ADDITIONAL THEORETICAL DETAILS

The appendix provides the drift-plus-penalty derivation and algorithmic procedure for FEDQCL, including local queue updates, client training, and server aggregation. It also compares per-round client complexity with baseline families.

  • Additional Theoretical Details: The theoretical derivation bounds virtual-queue evolution using a squared maximum inequality, summation, and rearrangement.
  • Algorithm: FEDQCL initializes per-task virtual queues, the global model, and control parameters δ and V before iterating over tasks and communication rounds.
  • Algorithm: Clients broadcast the current global model, perform local mini-batch updates, and apply a local drift-plus-penalty SGD update.
  • Algorithm: After local training, clients update their queues and the server aggregates client models.
  • Complexity Comparison: Complexity is compared per communication round per client using clients, rounds, tasks, replay size, model dimension, batch sizes, past tasks, and augmentation passes.

7.2 ADDITIONAL EXPERIMENTAL DETAILS

The additional experiments document implementation choices and evaluate FEDQCL across datasets, training regimes, client counts, task horizons, and hyperparameter settings. Results consistently emphasize accuracy, forgetting control, scalability, and queue behavior.

  • Experimental Setup: Experiments use PyTorch and ResNet-18, adapt several baselines to task-incremental learning, and generally use recommended hyperparameters and cited codebases.
  • Experimental Setup: FEDQCL uses balanced reservoir sampling with a fixed-size replay buffer of M samples per client, while V and δ come from sensitivity analyses.
  • Comparison with Baselines: FEDQCL maintains the highest accuracy throughout Split-CIFAR-10 and Split-TinyImageNet, while most baselines progressively lose accuracy.
  • Class-Incremental Learning Setting: FEDQCL outperforms all baselines in average accuracy under near-IID and non-IID class-incremental settings, while forgetting remains competitive near-IID.
  • Varying Number of Clients: FEDQCL scales gracefully to N=15 clients, maintaining competitive accuracy and low forgetting without additional communication cost as federation size grows.
  • Training Regimes: Forgetting decreases monotonically as local epochs R increase while accuracy remains stable, and offline training achieves substantially higher accuracy than online training.
  • Varying Number of Tasks: Across T ∈{5, 10, 20}, accuracy remains stable and forgetting increases modestly but sublinearly as the task horizon expands.
  • Hyperparameter Sensitivity: Smaller δ produces larger queues and stronger regularization, whereas increasing δ makes constraints more lenient and can marginally benefit accuracy.
Loading 2608.21539v1…