FederatedScope: A Flexible Federated Learning Platform for Heterogeneity

1 INTRODUCTION

• Local data, resources, participant behaviors, and learning goals vary across FL applications, motivating platforms that can accommodate these forms of heterogeneity.Data differences challenge vanilla FedAvg; resource differences can make synchronous training wait for slow clients, while diverse information exchanges, training processes, and objectives require flexible participant behaviors.Source: §1 · pp. 1–2
• FederatedScope frames FL courses as event-handler pairs so participants can express diverse behaviors and be coordinated through message-passing and condition-checking events.Handlers specify actions triggered by events, such as clients training a received global model on local data and returning updates.Source: §1 · p. 2
• Users can add event-handler pairs to develop algorithms or call existing events, handlers, core functions, and plug-ins through different levels of programming interfaces.The paper also describes personalization algorithms, asynchronous training, privacy protection, cross-backend FL, attack simulation, and auto-tuning components.Source: §1 · p. 2, Figure 1 · p. 2
• Users can extend FederatedScope by adding behaviors to handlers, including for personalization, aggregation, asynchronous training, and privacy protection.The authors state that these extensions can add functions or plug-ins for requirements arising in new scenarios.Source: §1 · p. 2

2.1 Problem Definition

• FL participants jointly train a model without directly sharing their private datasets, while exchanging updates or other information for collaborative aggregation.The section defines the server as coordinating and aggregating participant(s), and clients as the other participants.Source: §2.1 · p. 3
• FederatedScope extends beyond vanilla FL by supporting clients with different models, loss functions, and learning goals while training shared model parts.The paper focuses on vanilla FL in most of its presentation, then discusses more complicated settings as extensions.Source: §2.1 · p. 3

2.2 Related Works

• Existing platforms commonly use procedural programming that declares a sequential process from a global perspective, whereas FederatedScope lets participants describe behaviors from their own perspectives.The related-work discussion connects this design to client-wise training configurations, asynchronous strategies, personalization, and privacy/security components.Source: §2.2 · p. 3

3.2 Event-driven Architecture

• FederatedScope supports asynchronous aggregation through customizable conditions that let training proceed without waiting for every client.“goal_achieved” proceeds after enough feedback arrives, while “time_up” collects as much feedback as possible within the round’s time budget; both differ from “all_received.”Source: §3.2 · p. 5
• FederatedScope’s event-driven design provides the flexibility to implement and customize diverse federated learning algorithms.Participants wait for events and execute corresponding handlers, modularizing their behavior rather than requiring developers to coordinate the course sequentially from a global perspective.Source: §3.2 · pp. 5–6

3.3 Supporting Asynchronous Training

• The event-driven architecture supports asynchronous strategies suited to heterogeneous FL applications, including cross-device settings with unreliable and diverse clients.These strategies are presented as a way to balance model performance and training efficiency.Source: §3.3 · p. 6

3.3.1 Behaviors for asynchronous FL.

• Asynchronous FL strategies in FederatedScope modularize how systems handle stale updates, client responsiveness, and model broadcasts.Stale updates can be discounted or dropped beyond a threshold; responsiveness-based or group sampling addresses unequal response speeds, while “after receiving” broadcasts to an idle client upon feedback.Source: §3.3.1 · p. 6

3.3.3 Convergence analysis.

• FederatedScope’s convergence analysis extends prior asynchronous FL analyses by quantifying stale-update effects and giving a general convergence rate.The analysis assumes a smooth, strongly convex loss, bounded gradient variances and staleness, and a constraint on the local SGD step size.Source: §3.3.3 · p. 7

3.4 Supporting Personalization & Multi-Goal

• FederatedScope’s event-driven architecture lets clients and servers independently describe participant-specific behaviors to address heterogeneity.The section frames this flexibility as support for personalization and multiple learning goals, beyond assuming a single shared global model.Source: §3.4 · p. 7

3.4.1 Personalized training behaviors.

• FederatedScope lets developers customize client training configurations, events, and handlers to implement personalized training behaviors.Clients can set individual training steps or learning rates, trigger personalization on conditions such as performance drops, and fuse received global models with local models; privacy operators can perturb shared parameters.Source: §3.4.1 · p. 8

3.4.2 Multiple Learning Goals.

• FederatedScope supports multiple learning goals by letting participants train private objectives and exchange only messages for shared model layers.One motivating example is institutes sharing graph-convolution updates while keeping encoder, readout, and classifier layers private.Source: §3.4.2 · p. 8
• Participants can define distinct learning goals and behaviors, allowing FederatedScope to handle heterogeneous training processes and objectives.The paper describes this flexibility as supporting participants from their respective perspectives.Source: §3.4.2 · p. 8

3.5 Supporting Cross-backend FL

• Participants define their own backend-specific computation graphs and encode and decode exchanged messages in a shared backend-independent format, limiting disclosure of model architecture and local algorithms.The independent graphs also let developers adapt participant-specific components to their inputs and tasks without declaring a global super graph.Source: §3.5 · pp. 8–9

3.6 Usage of FederatedScope

• Users customize federated workflows by registering handlers for events, including replacing synchronous aggregation triggers with asynchronous conditions.New message-passing events can support exchanges such as graph node embeddings or encrypted results.Source: §3.6 · p. 9
• Decoupled trainers and aggregators let users extend local training and aggregation algorithms, while base classes and completeness checks help verify required interfaces and message flow.FederatedScope also provides Byzantine fault-tolerance algorithms, such as Krum, as aggregator behaviors.Source: §3.6 · pp. 9–10

4.1 Behavior Plug-In: Privacy Protection

• Privacy algorithms are behavior plug-ins: clients can encrypt, partition, or add noise to messages before sharing, with settings needed for formal privacy guarantees.Examples include Paillier encryption, secret sharing for FedAvg, and tunable Differential Privacy modules; users must specify settings such as noise distribution and privacy-budget allocation.Source: §4.1 · p. 10, Figure 6 · p. 10

4.2 Participant Plug-In: Attack Simulation

• Users can configure selected participants as malicious clients and add attack algorithms to their trainers to collect or inject messages and test protection strength.Simulated attacks can help assess whether dataset properties or private training samples would be inferred.Source: §4.2 · pp. 10–11, Figure 7 · p. 11
• FederatedScope implements privacy attacks including gradient inversion, property inference, class-representative, and training-data or label-inference attacks, plus backdoor performance attacks.The listed backdoor methods poison either the dataset or the model.Source: §4.2 · p. 11

4.3 Manager Plug-In: Auto-tuning

• For Federated-HPO, FedEx independently samples each client’s configuration per round, then aggregates client-wise feedback to update configuration-selection policies.Multi-fidelity methods can restore checkpoints to evaluate surviving configurations rather than restarting training from scratch.Source: §4.3 · p. 11, Figure 8 · p. 11
• The auto-tuning plug-in supports Bayesian, multi-fidelity, and Federated-HPO methods at different granularities, from complete FL courses to client-wise updates.FederatedScope provides a unified interface for these methods; its event-driven architecture supports client-wise exploration by Federated-HPO methods.Source: §4.3 · p. 11

5.2 Experiment Settings.

• Experiments train a two-convolution-layer CNN on FEMNIST (3,597 writer clients) and CIFAR-10 (1,000 Dirichlet-partitioned clients), and bag-of-words logistic regression for sentiment analysis on Twitter texts from 6,602 users.ModelZoo provides off-the-shelf neural network models.Source: §5.2 · p. 12, §5.1 · p. 12

5.3.1 Asynchronous Federated Learning.

• Asynchronous strategies reduced virtual time to target accuracy by 5.25×~18.82× versus vanilla synchronous training across benchmark datasets.The comparison uses virtual hours to reach targeted test accuracy.Source: §5.3.1 · p. 12, Table 1 · p. 12
• Unlike over-selection, asynchronous aggregation tolerates stale feedback, avoiding the participant unfairness and response-speed bias associated with dropped updates.Asynchronous strategies had more concentrated effective aggregation counts, similar to vanilla synchronous training.Source: §5.3.1 · pp. 12–13, Figure 10 · p. 13
• After-aggregating broadcasts produce less staleness than after-receiving broadcasts, but require more server bandwidth because multiple messages are sent simultaneously.The comparison is between Async-Goal-Aggr-Unif and Async-Goal-Rece-Unif.Source: §5.3.1 · p. 13, Figure 11 · p. 13

5.3.2 Personalization.

• On FEMNIST, personalized FL algorithms outperform FedAvg in average and 90% quantile client accuracy and reduce client-wise accuracy differences.The reported comparison includes FedBN, FedEM, pFedMe, and Ditto.Source: §5.3.2 · p. 13

5.3.3 Privacy Protection and Attack.

• As more FEMNIST clients inject Gaussian noise into updates, global test accuracy decreases from 84% to 65%, showing a privacy–utility trade-off.The experiment varies the noise-injecting client fraction from 0% to 100%.Source: §5.3.3 · p. 13, Figure 13 · p. 14
• With noise injected, DLG fails to recover meaningful information, whereas reconstructions from clients without noise expose ground-truth digits.The comparison uses reconstructed images in Figure 13.Source: §5.3.3 · p. 13, Figure 13 · p. 14

5.3.4 Auto-Tuning.

• FedEx-wrapped search methods achieve better test accuracy than their corresponding wrappers, despite slower decreases in best-seen validation loss.The comparison concerns wrapped FedEx versus the corresponding random-search or successive-halving wrapper.Source: §5.3.4 · p. 14

• FederatedScope frames FL courses as event-handler pairs, letting participants express diverse behaviors, exchange varied messages, pursue different learning goals, and coordinate synchronous or asynchronous strategies.The platform also provides plug-in components for privacy protection, attack simulation, and auto-tuning.Source: §6 · p. 14

A CONVERGENCE ANALYSIS

• In round t, participating clients take Q local SGD steps at learning rate η, and the server aggregates their updates computed from possibly stale initial models.The update uses the client participation set S(t); staleness is denoted τ_i(t), or τ_i when unambiguous.Source: A CONVERGENCE ANALYSIS · p. 17
• The analysis assumes smoothness, strong convexity, unbiased stochastic gradients, and bounded local and global variances and gradients.These are listed as assumptions A.1–A.5.Source: A CONVERGENCE ANALYSIS · p. 17
• It also assumes bounded staleness and equal numbers of training instances across clients to simplify the optimization problem.The staleness bound is τ_i<=τ_max.Source: A CONVERGENCE ANALYSIS · p. 17
• Two lemmas bound the expected L2 norms of clients’ estimated gradients and the round update Δ(t).The lemmas are introduced based on the stated assumptions and prior studies.Source: A CONVERGENCE ANALYSIS · p. 17
• When μQη<=1, the analysis gives a model-convergence conclusion for the asynchronous protocol, with a derivation involving geometric ratio 1−μQη.The analysis is introduced for the proposed protocol and assumes smoothness.Source: A CONVERGENCE ANALYSIS · pp. 17–18

B EXAMPLES OF EVENTS

• Events and their corresponding handlers describe participant behaviors in FederatedScope.Examples are presented in Table 2.Source: B EXAMPLES OF EVENTS · p. 19

C DATAZOO

• DataZoo contains ten widely used datasets from various FL applications and standardizes their data preprocessing.Dataset statistics are summarized in Table 3.Source: C DATAZOO · p. 19

D OVERALL STRUCTURE

• Figure 15 illustrates the overall structure of FederatedScope.Source: D OVERALL STRUCTURE · p. 19

E COMPLETENESS CHECKING

• Completeness checking verifies message-transmission flow; a complete FL course has at least one path from “start” to “termination.”Figure 16 shows the checking.Source: E COMPLETENESS CHECKING · p. 19

F IMPLEMENTATION DETAILS

• The experiments ran on 8 GeForce RTX 2080Ti GPUs, with synchronous and asynchronous strategy settings tuned on validation data.Sync-OS over-selects 30% more clients; asynchronous configurations set aggregation goals by dataset and staleness tolerance to 20.Source: F IMPLEMENTATION DETAILS · pp. 19–20
• FederatedScope’s overall structure is shown in Figure 15.The passage identifies the figure as the platform’s overall structure.Source: Figure 15 · p. 19
• FEMNIST and CIFAR-10 use a two-convolution-layer CNN, while Twitter uses bag-of-words features with logistic regression.The CNN uses hidden size 2048, dropout 0.5, four SGD steps, batch size 20, and learning rate 0.5; Twitter uses embedding size 300, maximum length 140, four steps, batch size 2, and learning rate 0.05.Source: F IMPLEMENTATION DETAILS · pp. 19–20
• Figure 16 illustrates course validation: unreachable nodes trigger warnings, while a course without a start-to-termination path triggers an error.A start-to-termination path indicates a complete FL course.Source: Figure 16 · p. 20

G DATASETS WITH IID DISTRIBUTION VERSUS NON-IID DISTRIBUTION

• FedBN and Ditto noticeably improve over FedAvg on non-IID CIFAR-10, while FedAvg is competitive on IID data and drops more as heterogeneity increases.Table 4 reports accuracy results for IID and non-IID CIFAR-10 distributions.Source: G DATASETS WITH IID DISTRIBUTION… · p. 20, Table 4 · p. 20

H EXAMPLE OF AMOUNT OF PROGRAMMING

• Built-in courses can use a customized dataset by registering its class and changing the dataset name in the configuration.Excluding dataset preprocessing, the passage says this requires two lines of code.Source: H EXAMPLE OF AMOUNT OF PROGRAMMING · p. 20
• Users can customize participant behavior by inheriting clients or servers and adding event-handler pairs.Implementing FedBN and Ditto from vanilla FedAvg requires 19 and around 100 lines of code, respectively.Source: H EXAMPLE OF AMOUNT OF PROGRAMMING · pp. 20–21
• Figure 17 presents experimental results for asynchronous strategies; Figures 18 and 19 show CIFAR-10 and bias-CIFAR-10 data distributions across client groups ordered by response speed.In Figures 18 and 19, clients that respond faster are at the top.Source: Figure 17 · p. 21, Figure 18 · p. 21, Figure 19 · p. 21

I EXPERIMENTS AND ANALYSIS ON ASYNCHRONOUS TRAINING STRATEGIES

• Asynchronous training strategies achieve better performance than synchronous strategies in the reported experiments.Figure 17 presents the experimental results for the provided asynchronous strategies.Source: I EXPERIMENTS AND ANALYSIS ON… · p. 21
• No provided sampling strategy is universally best; effectiveness is case-dependent, so FederatedScope emphasizes modularity and user choice.Users can select strategies according to application needs, including performance, system resources, and staleness tolerance.Source: I EXPERIMENTS AND ANALYSIS ON… · p. 21
• On CIFAR-10, response-related and group sampling perform similarly to uniform sampling when client data distributions are similar across system-resource clusters.The passage explains that this similarity limits their effectiveness relative to uniform sampling when resource heterogeneity does not produce model bias.Source: I EXPERIMENTS AND ANALYSIS ON… · p. 21
• On bias-CIFAR-10, response-related and group sampling achieve noticeable improvement over uniform sampling when rare labels belong only to weakly responsive clients.Figure 20 presents the experimental results on bias-CIFAR-10.Source: I EXPERIMENTS AND ANALYSIS ON… · p. 21, Figure 20 · p. 21