Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy

• Edge AI must learn from dispersed data while preserving privacy: centralized ML pools data, FL keeps it decentralized, and DP addresses privacy risks that remain.Source: §1 · pp. 2–3
• The authors compare FL and DP tools, identify a missing holistic view, and present Sherpa.ai FL as an open-source unified framework.Source: §1 · p. 3
• Sherpa.ai FL’s modular design covers data, learning models, aggregation, clients, server orchestration, communication, and DP mechanisms.Source: §1 · p. 3
• The authors adapt machine-learning principles to FL, noting that most methods transfer directly while some need ad hoc amendments, and derive methodological guidelines for privacy-preserving AI services.Source: §1 · p. 4
• Federated and centralized EMNIST Digits classification achieve similar results.Source: §1 · p. 4
• The California Housing regression example compares federated and centralized experiments and demonstrates assessing and limiting privacy loss with DP.Source: §1 · p. 4

2.1. The definition of Federated Learning

• Federated learning iteratively trains local client models and aggregates their parameters into a global model; client-server communication may be synchronous or asynchronous.In the client-server architecture, clients retain their local data while the server coordinates aggregation and model updates.Source: §2.1 · pp. 5–6

2.2. Key elements of Federated Learning

• Non-IID client data is likely in practice and challenging, while model-parameter communication creates leakage risk that differential privacy can help address.Federated aggregation must fuse local models, reduce communication rounds, and withstand poor-quality or malicious clients.Source: §2.2 · pp. 7–8

2.3. The definition of Differential Privacy

• Differential privacy is a formal measure of privacy loss that bounds potential harm through a budget: ϵ limits loss, and δ is the probability of exceeding that limit.It supports learning useful population-level knowledge while protecting individuals whose sensitive information is included in a database.Source: §2.3 · pp. 9–10

2.4. Key elements of Differential Privacy

• DP mechanisms add calibrated noise: Laplace noise supports numeric queries, while the exponential mechanism selects outputs by utility when direct noise would ruin results.Laplace noise is calibrated to query sensitivity; the exponential mechanism can handle arbitrary utility but may be inefficient over complex or large output domains.Source: §2.4 · p. 11
• The Gaussian mechanism adds noise calibrated to ℓ2 sensitivity and can provide (ϵ, δ)-differential privacy, while randomized response offers plausible deniability.The paper also frames composition and subsampling as modular DP tools for constructing and strengthening privacy-preserving mechanisms.Source: §2.4 · pp. 10–12
• Privacy filters adaptively continue or halt composition to keep the stated privacy budget from being exceeded with probability greater than δg.The filter returns HALT if the composition exceeds the budget and CONT otherwise.Source: §2.4 · p. 14
• Random subsampling can improve an (ϵ, δ)-DP mechanism’s privacy parameters, while adding sampling variability because a new random subsample is used each time.The passage notes that the improvement is especially noticeable when ϵ < 1.Source: §2.4 · p. 15
• Subsampling is suited to federated learning because each iteration draws data from a random sample of clients rather than all clients.The section also describes combining subsampling with the Gaussian mechanism as reducing noise and privacy-budget expenditure at the cost of accessing only a small random sample.Source: §2.4 · p. 15

3.1. PySyft

• PySyft provides low-level FL support but lacks default datasets, aggregation operators, and available DP mechanisms, making model assembly complex for users.Its documentation also warns against using the code to protect data.Source: §3.1 · p. 16

3.2. TensorFlow

• TensorFlow Privacy focuses on differentially private optimizers, while TensorFlow Federated lacks built-in aggregation operators; the reviewed tools also have compatibility and production-readiness limitations.The paper describes the tools as TensorFlow-bound and lacking DP mechanisms for implementing new privacy-preserving algorithms.Source: §3.2 · p. 17

3.3. FATE

• FATE offers FL algorithms through a script-based interface, but its low-level model implementation is unclear and it provides no DP algorithm.The review also describes its modular architecture as complex and its custom language as poorly documented.Source: §3.3 · p. 17

3.4. LEAF

• LEAF focuses on FL benchmarking and provides basic mechanisms, but it has no privacy benchmarks and limited documentation and tutorials.Its baseline implementation includes the Federated Averaging Aggregator and built-in datasets.Source: §3.4 · p. 18

3.5. PaddleFL

• PaddleFL provides a high-level interface, basic FL aggregators, and differentially private stochastic gradient descent, but offers little documentation and few options for alternative DP algorithms.Its PaddlePaddle basis also limits compatibility with other frameworks.Source: §3.5 · p. 18

3.6. Frameworks analysis

• Across the reviewed tools, the paper identifies missing unified FL and DP support, limited DP mechanisms and extensibility, and a focus on basic aggregation and deep learning.It concludes that a unified FL and DP framework is required.Source: §3.6 · p. 19

4. Sherpa.ai Federated Learning Framework

• Sherpa.ai FL is an open-research unified FL and DP framework with a hierarchical, modular architecture and described software functionalities and implementation.Its stated aim is to foster research and development of AI services at the edges while preserving data privacy.Source: §4 · pp. 19–20

4.1. Software architecture

• Sherpa.ai FL organizes software modules around key elements of FL, including data distribution, client and server coordination, aggregation, models, and differential privacy.The modules encapsulate specific functionality and can be extended as FL research progresses.Source: §4.1 · pp. 20–21

4.2. Software functionalities

• The framework supports fixed-client simulations, high-level simulation functions, and Keras and Scikit-Learn models, including neural networks, linear regression, k-means clustering, and logistic regression.Available aggregation operators include FedAvg, weighted FedAvg, and a federated k-means operator.Source: §4.2 · p. 21
• Sherpa.ai FL supports federated data normalization or reshaping and evaluation against centralized learning.These are among the framework's listed functionalities.Source: §4.2 · p. 21
• Its privacy features include Randomized Response, Laplace, Exponential, and Gauss mechanisms, Basic and Advanced adaptive composition using privacy filters, and subsampling.The stated purpose is to preserve privacy of clients' data and model parameters in FL.Source: §4.2 · p. 21

4.3. Implementation details

• The framework is implemented in Python, uses Keras, TensorFlow, and scikit-learn APIs for machine learning, runs on CPUs or GPUs, and is licensed under Apache 2.0.GPU use requires appropriate TensorFlow and CUDA versions.Source: §4.3 · p. 22

• The implementation details described in the paper correspond to Sherpa.ai FL release 0.1.0.Source: §5 · p. 22
• Sherpa.ai FL’s adaptation addresses both machine-learning algorithms and the workflow, with the aim of protecting clients’ data privacy.Source: §5 · p. 23
• The section covers distributed-computing aspects, a machine-learning adaptation paradigm, exceptions to that paradigm, and workflow guidelines for using Sherpa.ai FL.Source: §5 · p. 23

• Clients’ unreliable, poor network connections make reducing learning rounds essential, while data is non-IID, sparse, and distributed across K clients with K ≫ n/K.Source: §5.1 · p. 23
• Raw data stays on users’ devices while clients send model updates for server aggregation, saving communication bandwidth and time.Source: §5.1 · p. 23
• Keeping raw data off the server reduces the attack surface, but client updates may still reveal private information.Source: §5.1 · p. 23
• Sherpa.ai FL supports IID and non-IID client data, and weighted aggregation that emphasizes more significant clients.Source: §5.1 · p. 23
• Differential privacy can protect a released global model centrally or protect clients locally against a malicious server or eavesdropping.Source: §5.1 · p. 24

• Federated machine-learning methods address finite-sum objective minimization, with examples including linear and logistic regression, support vector machines, conditional random fields, and neural networks.Source: §5.2 · p. 24
• Communication efficiency and fewer federated rounds are primary constraints, while baseline aggregation operators such as Federated Averaging can be used without ad-hoc adaptation.Source: §5.2 · p. 24
• Applying differential-privacy components to federated algorithms supports estimating privacy loss across rounds using composition theorems [27, 29, 30].Source: §5.2 · p. 24
• The section’s adaptation criteria are objective-function structure, easy aggregation of local model parameters, and direct applicability of differential privacy.Source: §5.2 · p. 24
• Sherpa.ai FL supports customized differential-privacy mechanisms on model parameters and raw client data, plus privacy filters for basic and advanced composition theorems.Source: §5.2 · p. 24

• Models whose objectives fit the paradigm may still require ad-hoc procedures for federated adaptation.Source: §5.3 · p. 25
• For k-means, non-IID data can make direct average aggregation unfeasible because local cluster counts and orderings may differ; proposed approaches include fixing cluster counts or grouping clients.Differential-privacy components may also need adjustment for this setting; Zhang et al. adjust centroid noise using contour coefficients.Source: §5.3 · p. 25
• Federated matrix-factorization ranking communicates only part of the update, requiring additional server-client exchanges and added care with privacy loss; a two-stage randomized response approach is proposed.Clients can specify their privacy level.Source: §5.3 · p. 25

5.4. Methodological guidelines for preserving data privacy with federated learning in Sherpa.ai FL

• Because FL is a machine-learning paradigm, its experimental settings are similar, but its distributed setting requires revising the workflow to fit FL.The guidelines frame this revised workflow as good practice for developing privacy-preserving AI services at the edges.Source: §5.4 · p. 25

5.4.1. Data collection

• Real federated data collection begins locally at each client; in the strict setting assumed here, the server has no data information.The guidelines distinguish real FL from simulations, where data distribution can be emulated.Source: §5.4.1 · p. 26, §5.4 · p. 26

5.4.2. Data preparation

• Federated data preparation repeats training, evaluation, and test partitioning at each client, while preprocessing must accommodate private, distributed data without knowing its distribution.Distribution-based preprocessing may require robust aggregation of statistics or a global interval, so the authors advise techniques adapted to distributed scenarios.Source: §5.4.2 · pp. 26–27

5.4.3. Model selection

• Model selection chooses a problem-appropriate parameterized learning model and a server aggregation operator, such as FedAvg, client selection, or asynchronous aggregation.The server’s model can be initialized from the first aggregation in strict FL; simulation may use prior global information.Source: §5.4.3 · pp. 27–28

5.4.4. Model training

• Each training round trains local models, sends parameters to the server for aggregation, then updates the client models with the aggregated global model.The workflow repeats these steps across learning rounds.Source: §5.4.4 · p. 28

5.4.5. Model evaluation

• Evaluation distributes the aggregated model to clients, combines their reported performance into global metrics, and recommends absolute client metrics when client data sizes vary.For simulations, global evaluation data and cross-validation are also possible.Source: §5.4.5 · p. 28

5.4.6. Hyper-parameter tuning

• Hyper-parameters are tuned using evaluation metrics by modifying model parameters to improve performance on evaluation data.The authors also describe tuning local models independently before aggregation as an option for customization.Source: §5.4.6 · p. 28

5.4.7. Predictions making

• After training, predictions for unknown examples are made using each client’s test set.In simulations, a global test set can be used instead.Source: §5.4.7 · p. 29

6. Illustrative cases of study

• The framework’s use cases are built around the methodological guidelines for federated learning.The section uses these cases to show how to follow the guidelines with Sherpa.ai FL.Source: §6 · p. 29
• The classification case constructs the key elements of a federated experiment and compares the federated approach with a centralized one..Source: §6 · p. 29
• The regression case compares centralized learning with federated learning and differential privacy, and demonstrates limiting privacy loss with Sherpa.ai FL Privacy Filters..Source: §6 · p. 29

6.1.1. Case of study

• The illustrative case uses EMNIST Digits, whose writings from different authors provide non-IID data for simulating federated environments.The example uses five clients and FedAvg with a simple CNN learning model.Source: §6.1.1 · pp. 29–30

6.1.2. Description of the code

• The code example creates private federated data on five nodes and uses framework components to select a model and aggregation operator, preprocess data, and train for two rounds.The example reports varying client test accuracy, which the authors attribute to differences in training-data quality across clients.Source: §6.1.2 · pp. 32–34

6.1.3. Comparison with a centralised convolutional neural network approach

• The study compares federated learning with an equivalent centralized approach by training the Figure 5 neural network on the same data; federated runs use 25 clients and 10 rounds of five epochs in both IID and non-IID scenarios.Source: §6.1.3 · p. 35
• In IID data, federated learning beats the centralized approach, while in non-IID data it attains lower but competitive accuracy.The authors attribute the non-IID shortfall to the additional challenge of heterogeneous client data; Table 4 reports accuracy for both scenarios.Source: Table 4 · p. 35, §6.1.3 · p. 35

6.2. Linear regression with DP

• The section applies the Laplace mechanism to a regression FL simulation and demonstrates advanced composition to keep privacy loss within a maximum budget.The example uses California Housing and compares its federated experiment with a centralized version.Source: §6.2 · p. 36

6.2.1. Case of study

• The case study uses about 20,000 California Housing samples, the first two of eight features, and reserves 2,000 samples for sensitivity sampling.The remaining data is split into training and test sets; the FL simulation distributes training data IID across five clients and uses FedAvg.Source: §6.2.1 · p. 36

6.2.2. Description of the code

• The code section shows Sherpa.FL dataset conversion and setup of the linear-regression model and federated aggregation.It also retains a dataset portion for DP sensitivity sampling and loads training and test data.Source: §6.2.2 · pp. 36–37

6.2.3. Running the model in a Federated configuration

• The global model generally outperforms each client model in the five-client linear-regression simulation.The experiment uses one FL round because linear regression requires no iterations.Source: §6.2.1 · p. 36, §6.2.3 · p. 37

6.2.4. Differential Privacy: sampling the model’s sensitivity

• SensitivitySampler estimates model sensitivity empirically, enabling estimates for arbitrary models or black-box functions when analytic sensitivity is difficult to compute.The example uses an ℓ1 sensitivity norm; more samples typically improve the estimate and decrease the estimated sensitivity.Source: §6.2.4 · pp. 38–39
• Sensitivity sampling can be computationally expensive because it trains the model on pairs of datasets differing by one entry.The passage notes this may be expensive for deep neural networks.Source: §6.2.4 · p. 40

6.2.5. Running the model in a Federated configuration with Differential Privacy

• The Laplace mechanism uses sampled model sensitivity, ∆f ≈ 0.008, to provide an ϵ-differentially private FL model.The example sets ϵ = 0.5 and applies the mechanism to each client’s model parameters.Source: §6.2.5 · p. 40

6.2.6. Comparison with centralised and non private approaches

• Centralized and non-private FL models exhibit comparable performance; with Laplace ϵ-DP, higher ϵ provides lower privacy but increases accuracy.Table 6 compares these settings using IID data across five clients; DP results average runs up to a total privacy budget ϵT = 4.Source: §6.2.6 · p. 41, Table 6 · p. 41

7. Concluding remarks

• The privacy-budget example assumes estimated sensitivity adequately approximates analytic sensitivity.This assumption is stated for applying composition theorems for privacy filters.Source: §7 · p. 41
• Sherpa.ai FL combines unified FL and DP support with methodological guidelines for experimental FL settings.The guidelines follow a study of how to adapt machine-learning principles to FL.Source: §7 · p. 42
• Classification and regression examples show similar results between centralized and federated settings for the same experiments.The authors present these as illustrative uses of the guidelines and framework.Source: §7 · p. 42
• Planned extensions include more aggregation operators, machine-learning models, data distributions, and DP mechanisms or relaxations.Examples named include RAPPOR, Concentrated DP, and Rényi DP.Source: §7 · p. 42