An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning

1 INTRODUCTION

• Byzantine clients upload arbitrary updates to degrade the global model, and even one malicious client can arbitrarily skew Mean aggregation.Byzantine-robust defenses are therefore needed to protect training.Source: §1 · p. 1
• Redundancy-based defenses can require Ω(M) more updates in the worst case, while trust-based defenses depend on trusted clients or datasets being available.Robust aggregation avoids those requirements but may have limited guarantees, strong assumptions, or vulnerabilities to newer attacks.Source: §1 · pp. 1–2
• Existing evaluations use different datasets, attack types, and hyperparameters, while data heterogeneity is rarely tested despite common IID assumptions.This motivates a comparative experimental study under shared settings.Source: §1 · p. 2
• ClippedClustering extends an existing clustering-based scheme with automatic clipping to mitigate amplified local updates.The paper surveys attacks and defenses and evaluates eight aggregation schemes under five representative attacks.Source: §1 · p. 2
• Across eight schemes and five attacks, accuracy remains high in some cases but success is limited, especially with Non-IID data.The study evaluates FedSGD and FedAvg and proposes ClippedClustering as an enhanced clustering-based scheme.Source: §1 · p. 2

2 FEDERATED LEARNING

• The section introduces the federated optimization problem and two algorithms, FedSGD and FedAvg, as the study’s training framework.The server coordinates client updates and broadcasts the updated model each communication round.Source: §2 · p. 2, Figure 1 · p. 3

2.1 Problem Formulation

• Federated learning seeks a parameter vector minimizing the distributed objective built from clients’ local empirical risks, with balanced dataset sizes assumed in this study.Each client’s objective is empirical loss over its local training samples.Source: §2.1 · p. 2

2.2.1 FedSGD

• FedSGD has each client upload a mini-batch gradient per round; this is computationally efficient but requires many communication rounds for good models.The server aggregates gradients and updates the model parameters.Source: §2.2.1 · p. 2

2.2.2 FedAvg

• FedAvg has clients run multiple local SGD steps and send resulting models for server aggregation, reducing communication relative to single-step updates.The study uses full participation because random client subsets could violate the assumption that fewer than half the updates are malicious.Source: §2.2.2 · p. 3

2.2.3 Update aggregation

• Both FedSGD and FedAvg use server aggregation to update the global model, and Mean is vulnerable because its breakdown point is 1/K.A single malicious client can significantly shift the resulting model from the original Mean.Source: §2.2.3 · p. 3

3 THREAT MODELS

• The threat models allow Byzantine clients to upload arbitrary values, assume a majority of clients are benign, and include five representative attacks.The majority-benign condition is expressed as M/K < 0.5.Source: §3 · p. 3

3.1 Noise

• Noise attack adds sampled random noise to updates before upload; the experiment sets both its mean and variance to 0.1..Source: §3.1 · p. 3

3.2 A Little is Enough (ALIE)

• ALIE estimates each coordinate’s benign-update mean and standard deviation, then selects corrupted values within a range intended to appear benign.It assumes benign updates follow a normal distribution and uses z_max from 0 to 1.Source: §3.2 · pp. 3–4

3.3 Inner Product Manipulation (IPM)

• IPM seeks a negative inner product between the true mean update and the aggregation output, potentially reversing updates and damaging the global model.The experiment tests ε = 0.5 and ε = 100.Source: §3.3 · p. 4

3.4 Sign Flipping (SF)

• Sign Flipping reverses gradient signs without requiring knowledge of other clients’ updates, applying the flip at each FedAvg local step.In FedSGD, clients upload negative gradients.Source: §3.4 · p. 4

3.5 Label Flipping (LF)

• Label Flipping changes each training label from l to L − l − 1, allowing attacks through modified data rather than update parameters.This is relevant when attackers have limited access to training APIs.Source: §3.5 · p. 4

4 AGGREGATION SCHEMES FOR EVALUATION

• The section surveys robust aggregation schemes, including methods evaluated in the paper.The section also introduces a new scheme and a taxonomy of eight evaluated aggregation schemes.Source: §4 · p. 4

4.1 Krum

• Krum selects the update closest to its K − M − 2 nearest neighbors by squared Euclidean distance.Here, K is the total number of clients and M is the number of malicious clients.Source: §4.1 · p. 4
• Under FedSGD, Krum’s convergence guarantee assumes c1σ < ∥g∥, with c1 depending on the number of malicious clients and model dimension.σ is the maximal variance of updates, and ∥g∥ is the expectation of updates.Source: §4.1 · p. 4

4.2 GeoMed

• GeoMed minimizes the sum of Euclidean distances from a vector to all update vectors.A (1 + ϵ)-approximate solution can be computed in nearly linear time, although no closed-form solution exists.Source: §4.2 · p. 4
• Under FedSGD, GeoMed’s convergence guarantee assumes c2σ < ∥g∥, where c2 differs from Krum’s constant c1.σ is the maximal variance of updates, and ∥g∥ is the expectation of updates.Source: §4.2 · p. 4

4.3 AutoGM

• AutoGM generalizes GeoMed by solving an aggregation problem with a user-specified λ controlling the smoothness of α.Its optimization alternates between estimating a weighted GeoMed and weighting each point’s importance.Source: §4.3 · p. 4

4.4 Median

• Median computes the coordinate-wise median of the updates; under FedSGD, its robustness relies on bounded gradient variance and coordinate-wise bounded absolute skewness.The median used for each coordinate is the usual one-dimensional median.Source: §4.4 · p. 5

4.5 TrimmedMean

• TrimmedMean averages each coordinate after removing the largest and smallest β fraction of its values.Its FedSGD robustness relies on Median’s assumptions and the stronger assumption that all moments of loss-function derivatives are bounded.Source: §4.5 · p. 5

4.6 Centered Clipping (CC)

• Centered Clipping iteratively clips updates around a center and updates that center at each iteration.The initial center is the aggregate from the previous round; its robustness proof assumes bounded update variance.Source: §4.6 · p. 5

4.7 Clustering

• Clustering computes pairwise cosine distances, separates clients into two groups by cosine similarity using average-linkage agglomerative clustering, then averages the largest group.Source: §4.7 · p. 5
• Clustering has no theoretical robustness guarantee, and attackers can amplify updates without changing their directions, potentially preventing convergence without detection.The scheme considers relative directions but ignores vector magnitudes.Source: §4.7 · p. 5

4.8 ClippedClustering

• ClippedClustering clips every update before clustering, limiting update magnitude and preventing attackers from amplifying updates in the same direction.Updates below the threshold are unaffected; updates exceeding it are scaled to the threshold.Source: §4.8 · p. 5
• The server sets the clipping threshold to the median of historical update norms, automatically adapting it during training.The method uses the 50th percentile of norms saved up to the current iteration.Source: §4.8 · p. 5
• The median is chosen as a robust threshold under the assumption that most clients are benign, so malicious clients cannot control it even with full update knowledge.Prior work also found adaptive median-norm clipping effective across federated learning tasks without tuning a clipping hyperparameter.Source: §4.8 · p. 5

4.9 Taxonomy

• The taxonomy distinguishes Euclidean distance-based schemes from mean-based schemes; ClippedClustering, like Clustering, clusters by cosine similarity but clips updates first.Krum, GeoMed, and AutoGM are Euclidean distance-based; TrimmedMean, CC, Clustering, and ClippedClustering are mean-based.Source: §4.9 · pp. 5–6

5 ADAPTIVE ATTACK ON CLIPPEDCLUSTERING

• The attack assumes full system knowledge, including the aggregation scheme and all benign-client updates.Source: §5 · p. 6
• The attack keeps malicious updates in the largest cluster while deviating as far as possible from the correct direction, using the benign clusters’ average cosine similarity as a constraint.The attacker reproduces clustering on benign updates and designs malicious updates close enough to the largest cluster to preserve its structure.Source: §5 · p. 6
• The optimization maximizes the angle between the malicious direction and the correct update while constraining its angle to the largest benign-cluster center.The constraint keeps the malicious group within the largest cluster during average-linkage hierarchical clustering.Source: §5 · p. 6
• Figure 3 encodes client IDs on the x-axis, class labels on the y-axis, and available training-sample counts through point size.Compare clients’ label availability and sample counts to inspect the Non-IID CIFAR-10 partition’s heterogeneity.Source: Figure 3 · p. 6
• After finding the malicious-update direction, the attack scales its magnitude by the clipping threshold τ.Source: §5 · p. 6
• The attack applies to agglomerative clustering with average linkage; for complete linkage, the minimum cosine similarity between benign clusters replaces the average-similarity bound.Source: §5 · p. 6

6.1 Experimental Setup

• The experiments use 20 clients on CIFAR-10 and MNIST with IID or Dirichlet-partitioned Non-IID data, α = 0.1; five clients are Byzantine by default.Some client classes are completely absent in the strongly Non-IID partition.Source: §6.1 · p. 6

6.2 Impact on the Mean Scheme

• Under Mean aggregation, FedAvg converges in fewer rounds than FedSGD on IID data without attacks, but its performance drops substantially on Non-IID data while FedSGD retains relatively high accuracy.Noise and IPM (ε = 100) reduce accuracy to around 10% in four settings; IPM (ε = 0.5) is less damaging to Mean. ALIE is ineffective on MNIST but strongly affects CIFAR-10 with FedSGD.Source: §6.2 · p. 7

6.3 Impact on Robust Aggregation Schemes

• Without attackers, Krum, GeoMed, AutoGM, and Median perform relatively poorly; on Non-IID CIFAR-10 with FedSGD, accuracy is nearly 10% for some schemes versus around 80% for Mean, TrimmedMean, Clustering, and ClippedClustering.Euclidean-based schemes struggle particularly in heterogeneous settings, and small-scale IPM attacks also challenge them.Source: §6.3 · pp. 7–8
• FedAvg is more robust than FedSGD with robust aggregation, but attacks are more effective with Non-IID data; ClippedClustering achieves the highest accuracy and defends against more attack types than other schemes.ClippedClustering is nevertheless less robust with Non-IID data and is defeated by ALIE in FedSGD on CIFAR-10.Source: Table 2 · p. 8, §6.3 · p. 8

6.4 Impact of Fraction of Malicious Clients

• With one Byzantine client (5%) on IID CIFAR-10 using FedAvg, Krum, GeoMed, and AutoGM lose 10% accuracy under IPM (ε = 0.5); Median, TrimmedMean, and ClippedClustering tolerate most attacks better at low fractions.All schemes are considerably less tolerant when local datasets are Non-IID, and ClippedClustering is affected by LF attacks as the malicious fraction increases.Source: §6.4 · p. 9, Figure 5 · p. 9

6.5 Impact of Batch Size

• Under ALIE with FedSGD and IID data, increasing batch size improves several schemes’ robustness, but all except ClippedClustering still fail to reach acceptable accuracy.At batch size 2500, training uses all client data per step; larger batches also increase local computation burden.Source: §6.5 · pp. 9–10, Figure 6 · p. 9

6.6 Impact of Adaptive Attack

• Under the adaptive attack, ClippedClustering tolerates up to 15% malicious clients with little degradation for IID FedSGD, but Non-IID FedSGD diverges at 15%.For FedAvg, IID MNIST converges at all tested fractions, while Non-IID models do not tend to converge above 15%; high loss does not always mean low top-1 accuracy.Source: §6.6 · p. 10, Figure 7 · p. 10

6.7 Pairwise Cosine Similarities

• FedAvg updates with IID data have the most similar directions, whereas FedSGD similarities vary widely and FedAvg updates become less similar with Non-IID data.The authors suggest that similarity among benign FedAvg updates can help clustering-based schemes group them and exclude malicious updates.Source: §6.7 · pp. 10–11, Figure 8 · p. 11

7.1 Byzantine Attacks on FL

• Untargeted attacks seek to degrade overall model performance, and strategies include noise or sign flipping, variance-aware evasion, and Inner Product Manipulation (IPM),,,.The paper focuses on untargeted attacks; it also notes debate about whether compromised clients are rare in practice or can be increased through fake clients,.Source: §7.1 · p. 11

7.2 Byzantine-robust FL

• Byzantine-robust defenses include redundancy-based, trust-based, and robust-aggregation schemes.Redundancy-based methods use duplicated updates to mitigate failures, but worst-case update computation can scale as Ω(M) per node for M Byzantine clients.Source: §7.2 · p. 11
• Trust-based schemes use trusted clients or datasets to filter or reweight updates, but those trusted resources may not be available because of privacy concerns,,,.Some trust-based methods use trusted data to score updates by entropy or cosine similarity.Source: §7.2 · pp. 11–12
• Robust aggregation estimates global updates from local updates and mitigates malicious updates, yet surveyed schemes show limitations in Byzantine robustness with FedSGD and FedAvg,,,,,.Source: §7.2 · p. 12

• Eight robust aggregation schemes showed limited robustness in experiments spanning five representative Byzantine attacks.Source: §8 · p. 12
• ClippedClustering extends a clustering-based scheme by automatically clipping updates to mitigate amplified malicious updates.Source: §8 · p. 12
• The study surveys existing Byzantine attacks and defense strategies in federated learning.Source: §8 · p. 12