PaperlayerView exact revision on arXiv

Source-linked AI summary

Byzantine-Robust Decentralized Federated Learning

Minghong Fang, Zifan Zhang, Hairi, Prashant Khanduri, Jia Liu, Songtao Lu, Yuchen Liu, Neil Gong

arXiv:2406.10416v4cs.CRcs.DCcs.LG

TL;DR

Decentralized federated learning avoids a central server but remains vulnerable to poisoning, while existing DFL defenses have communication or robustness limitations. BALANCE filters neighbor models using each client’s local model as a similarity reference. The paper reports theoretical convergence guarantees and experimental efficacy under poisoning attacks.

  • Problem

    Existing DFL defenses are limited by communication costs, missing guarantees, assumptions about malicious-neighbor ratios, or vulnerability to poisoning.

    Source: §4 · p. 4

  • Method

    BALANCE uses each client's local model as a similarity reference to assess whether received neighbor models are malicious or benign.

    Source: §1 · p. 2

  • Results

    In the extreme Non-IID adversarial setting, BALANCE achieves a largest Max.TER of just 0.14, while Median reaches 0.89 under LF attack.

    Source: §7 · p. 12

  • Takeaways & Limitations

    BALANCE's convergence rates under poisoning attacks match those of Byzantine-free counterparts in strongly convex and non-convex settings.

    Source: §8 · p. 13

    Variant I may inadvertently accept some malicious models in later training stages as the DFL system approaches convergence.

    Source: §7 · p. 13

Abstract

from arXiv · show

Federated learning (FL) enables multiple clients to collaboratively train machine learning models without revealing their private training data. In conventional FL, the system follows the server-assisted architecture (server-assisted FL), where the training process is coordinated by a central server. However, the server-assisted FL framework suffers from poor scalability due to a communication bottleneck at the server, and trust dependency issues. To address challenges, decentralized federated learning (DFL) architecture has been proposed to allow clients to train models collaboratively in a serverless and peer-to-peer manner. However, due to its fully decentralized nature, DFL is highly vulnerable to poisoning attacks, where malicious clients could manipulate the system by sending carefully-crafted local models to their neighboring clients. To date, only a limited number of Byzantine-robust DFL methods have been proposed, most of which are either communication-inefficient or remain vulnerable to advanced poisoning attacks. In this paper, we propose a new algorithm called BALANCE (Byzantine-robust averaging through local similarity in decentralization) to defend against poisoning attacks in DFL. In BALANCE, each client leverages its own local model as a similarity reference to determine if the received model is malicious or benign. We establish the theoretical convergence guarantee for BALANCE under poisoning attacks in both strongly convex and non-convex settings. Furthermore, the convergence rate of BALANCE under poisoning attacks matches those of the state-of-the-art counterparts in Byzantine-free settings. Extensive experiments also demonstrate that BALANCE outperforms existing DFL methods and effectively defends against poisoning attacks.

1 INTRODUCTION

Server-assisted FL has scalability and trust drawbacks, while decentralized FL introduces poisoning risks and challenges that existing defenses do not fully address. BALANCE uses local-model similarity to filter received models, with theoretical guarantees and experimental support.

1 INTRODUCTION

  • Server-assisted FL relies on a central server, creating a communication bottleneck as clients increase, a single-point-of-failure risk, and trust dependency.DFL instead lets clients exchange information peer-to-peer without a server.Source: §1 · pp. 1–2
  • Existing Byzantine-robust DFL methods can incur high communication costs, lack guarantees for every benign client, require malicious-ratio knowledge, or remain vulnerable to poisoning.DFL clients have only a partial view of the system and may have different numbers of malicious neighbors.Source: §1 · p. 2
  • BALANCE uses each client’s local model as a similarity reference to reject received models that differ in direction or magnitude.The paper establishes convergence guarantees under poisoning attacks in strongly convex and non-convex settings.Source: §1 · p. 2
  • BALANCE has the same communication complexity as state-of-the-art server-assisted FL algorithms, and experiments across varied datasets, attacks, and DFL settings demonstrate its efficacy.The evaluation covers five datasets, nine poisoning attacks, twelve communication graphs, and eight state-of-the-art FL baselines.Source: §1 · p. 2

2 PRELIMINARIES AND RELATED WORK

The preliminaries describe decentralized training and poisoning threats, then compare DFL defenses and their assumptions, guarantees, and communication costs.

2.1 Decentralized Federated Learning (DFL)

  • In DFL, clients train local models and exchange them with neighbors, then aggregate received models using a rule such as FedAvg or Median.A trade-off parameter controls weight placed on a client's own intermediate model versus aggregated neighbor models.Source: §2.1 · p. 3

2.2 Poisoning Attacks to FL

  • Poisoning attacks manipulate training data or local models; model attacks may be untargeted or aimed at specific inputs.The paper states that privacy attacks in DFL are outside its scope.Source: §2.2 · p. 3

2.3 Byzantine-robust DFL Aggregation Rules

  • Prior DFL defenses include UBAR, LEARN, and SCCLIP, which select neighbors, exchange updates and models, or clip received models.LEARN exchanges updates multiple times per round, while SCCLIP bounds received-model norms by the client's own model norm.Source: §2.3 · pp. 3–4
  • Existing DFL defenses may lack guarantees for every benign client, assume knowledge of malicious-neighbor ratios or a complete graph, or incur high communication costs.The comparison excludes server-assisted methods because their theoretical results cannot be straightforwardly transferred to DFL.Source: §2.3 · p. 4, Table 1 · p. 4

3 PROBLEM STATEMENT

The threat model allows malicious clients to poison data or manipulate models, while the defense is expected to operate without knowing the attack strategy or graph. The stated goals are competitive learning performance, Byzantine robustness, and efficiency.

3 PROBLEM STATEMENT

  • Malicious clients may poison local data or send manipulated models to neighbors, including different models to different neighbors.The attacker cannot change the communication graph, though clients may disconnect because of Internet issues.Source: §3 · p. 4
  • The attacker is assumed to know the FL system, including local data, aggregation rules, trade-off parameters, and the communication graph.Clients know their neighbors' local models because DFL exchanges them.Source: §3 · p. 4
  • The defense has no knowledge of the attack strategy or graph, and benign clients need not know the total malicious-client count or their own malicious-neighbor count.Its goals include competitive test error without attackers, Byzantine resilience, and no extra communication or computation costs versus FedAvg without attacks.Source: §3 · p. 4

4 THE BALANCE ALGORITHM

BALANCE addresses limitations in existing DFL defenses by comparing received neighbor models with each client's own intermediate model. It filters models by a time-dependent similarity threshold, then aggregates accepted models and updates locally.

4 THE BALANCE ALGORITHM

  • BALANCE addresses existing methods' communication costs and assumptions about each benign client's knowledge of its malicious-neighbor ratio.The paper identifies ensuring accurate final models for every benign client as a challenge.Source: §4 · pp. 4–5
  • BALANCE compares each received intermediate model with the receiving client's own intermediate model to identify potentially malicious models.The method targets manipulations of local-model direction or magnitude.Source: §4 · p. 5
  • A received model is accepted only when it meets a threshold controlled by γ and the decaying term γ·exp(−κ·λ(t)).λ(t) is non-negative and monotonically increasing with training round, while κ controls the exponential decay rate.Source: §4 · p. 5
  • Each client aggregates accepted neighbor models and updates its local model by combining that aggregate with its own intermediate model.The algorithm accepts models satisfying the similarity condition before updating.Source: §4 · p. 5

5 THEORETICAL PERFORMANCE ANALYSIS

The analysis gives BALANCE convergence guarantees for benign clients in strongly convex and non-convex settings under stated assumptions, with rates matching Byzantine-free counterparts.

5 THEORETICAL PERFORMANCE ANALYSIS

  • The analysis assumes strongly convex and smooth objectives, unbiased bounded-variance benign-client gradients, bounded models and gradients, and a connected benign-client subgraph.Source: §5 · pp. 5–6
  • In the strongly convex setting, benign clients converge to a neighborhood of the global minimum at the optimal Byzantine-free linear rate, which poisoning attacks do not hurt.Source: §5 · p. 6
  • In the non-convex setting, benign clients converge to a neighborhood of a stationary point at the optimal Byzantine-free sub-linear rate, which poisoning attacks do not impact.Source: §5 · p. 6
  • The theory assumes benign-client connectivity, but experiments do not require IID client data; BALANCE also needs no knowledge of the communication-graph architecture.Source: §5 · p. 6
  • Clients need not know the precise assumption parameters δ, ψ, and ρ, but the convergence results require bounded γ.Source: §5 · p. 6

6 EXPERIMENTS

The experiments evaluate BALANCE against poisoning attacks across datasets, decentralized federated learning methods, communication settings, graph types, and levels of malicious–benign connectivity. They report robustness, attack performance, model accuracy, and efficiency across these conditions.

6.1.1 Datasets and Poisoning Attacks

  • The evaluation covers a synthetic dataset and four real-world datasets, and tests seven attacks: two data-poisoning attacks and five model-poisoning attacks, including targeted Backdoor and adaptive attacks.The Adaptive attack assumes full knowledge of benign clients’ models and BALANCE’s aggregation rule; LIE and Dissensus attacks are evaluated in Section 7.Source: §6.1.1 · p. 6

6.1.2 Comparison DFL Methods

  • BALANCE is compared with eight methods, including FedAvg, Krum, Trim-mean, Median, FLTrust, UBAR, LEARN, and SCCLIP; several server-assisted methods are adapted to DFL.The baselines include aggregation rules that select, trim, or average neighboring models, as well as methods that exchange model updates.Source: §6.1.2 · p. 7

6.1.3 Evaluation Metrics

  • The study uses Max.MSE for synthetic regression and Max.TER and Max.ASR for real-world classification; all three metrics take the maximum across benign clients, with lower values indicating stronger defense.Max.ASR measures the fraction of targeted test examples classified as the attacker-chosen label.Source: §6.1.3 · p. 7

6.1.4 Non-IID Setting

  • Experiments use IID synthetic data and Non-IID real-world data; MNIST and Fashion-MNIST use p = 0.8, while HAR and CelebA already have heterogeneous data and require no additional simulation.In the MNIST and Fashion-MNIST simulation, p = 0.8 concentrates 80% of a client’s training data in one class.Source: §6.1.4 · p. 7

6.2 Experimental Results

  • BALANCE outperforms baselines on synthetic data and, on real-world datasets, matches FedAvg’s no-attack Max.TER of 0.10 on CelebA while holding MNIST Max.TER to 0.11 under Trim attack versus Trim-mean’s 0.81.The authors report similar robustness on the other three real-world datasets.Source: Table 2 · p. 8, §6.2 · p. 8
  • BALANCE demonstrates communication and computation efficiency, while LEARN incurs significant costs by exchanging model updates for ⌈log2 t⌉ rounds and then exchanging local models.The authors also report that server-assisted methods adapted to DFL can perform suboptimally; FLTrust is particularly vulnerable to Trim attack on MNIST because client data are heterogeneous.Source: Figure 4 · p. 9, §6.2 · p. 9
  • BALANCE’s results can worsen when γ and κ are too large; smaller α can help under weak attacks but can hurt vulnerable methods under strong attacks, and using different aggregation rules across clients increases attack susceptibility.Giving neighboring models more weight helped BALANCE under Gauss attack (Max.TER 0.05 at α = 0.1 versus 0.10 at α = 0.9), while Trim-mean under Trim attack had Max.TER 0.80 at α = 0.1 versus 0.43 at α = 0.9.Source: §6.2 · pp. 9–11
  • BALANCE withstands 50% malicious clients and the Adapt attack with 80% malicious clients, and remains resilient across 10–50 clients, varying Non-IID levels, time-varying graphs, and different initial models.At 10% malicious clients, UBAR’s Max.TER rises to 0.90 under Feature attack; BALANCE also outperforms existing DFL methods across the considered Non-IID scenarios.Source: §6.2 · pp. 9–11
  • BALANCE achieves Byzantine robustness across complete, Erdős–Rényi, small-world, and ring communication graphs.Each graph has 20 clients, including 4 malicious clients.Source: §6.2 · pp. 11–12
  • Attack performance generally increases as FEMB rises across three random graphs with 20 clients and 4 malicious clients.The tested FEMB values are 0.16, 0.22, and 0.32; FEMB is the ratio of edges between malicious and benign clients to all graph edges.Source: §6.2 · p. 12

7 DISCUSSION AND LIMITATIONS

The discussion tests BALANCE under more extreme non-IID data and additional adaptive or decentralized attacks, and examines variants of its acceptance rule. Results show robustness in the tested settings alongside a late-training vulnerability for Variant I.

More extreme Non-IID distribution

  • Under the extreme three-class-per-client setting, BALANCE’s largest Max.TER is 0.14 across considered attacks, while benign-client UBAR reaches 0.42 and Median reaches 0.89 under LF attack.The comparison is between Table 2a and Table 4; existing methods also show increased susceptibility to attacks in this setting.Source: §7 · p. 12

More adaptive and decentralized attacks

  • As attackers adapt with strategic perturbations, their attack efficacy diminishes; the considered data-poisoning attacks were also reported as ineffectual against BALANCE.The poisoned-data models may appear benign, but the authors attribute their limited impact to the attacks’ effects in their experiments.Source: §7 · p. 12
  • The LIE attacker estimates variance among benign models and introduces minimal changes, without needing to know the aggregation rule; Dissensus targets consensus among benign clients.LIE is described as a general attack, while Dissensus is designed specifically for DFL.Source: §7 · p. 12
  • Under Dissensus, Median’s Max.TER rises to 0.87, while LIE does not significantly compromise existing DFL methods.The authors suggest LIE’s generic design, rather than being tailored to decentralized systems, may explain its limited impact.Source: §7 · p. 12, Table 5 · p. 12

Different variants of BALANCE

  • Variant II tends to reject many benign models, whereas Variant I performs comparably to BALANCE overall.The comparison is reported in Table 6.Source: Table 6 · p. 12, §7 · p. 13
  • Variant I may accept malicious models later in training as the system approaches convergence; a smaller κ slows the decline of its exponential threshold.The authors also report that attacks limited to the second half of training produce lower Max.TER than attacks across all rounds under Median with Trim attack: 0.38 versus 0.63.Source: §7 · p. 13

8 CONCLUSION AND FUTURE WORK

The paper concludes that BALANCE defends DFL against poisoning by using each client’s local model to assess neighboring models. It reports convergence guarantees under attacks and identifies adaptive aggregation-rule and parameter selection as future work.

8 CONCLUSION AND FUTURE WORK

  • BALANCE uses each client’s local model as a reference for deciding whether a neighboring model is malicious or benign.The paper proposes it to defend against poisoning attacks in DFL.Source: §8 · p. 13
  • Under poisoning attacks, BALANCE has convergence guarantees in strongly convex and non-convex settings, with rates matching state-of-the-art counterparts in Byzantine-free settings.Source: §8 · p. 13
  • Future work will optimize per-client selection of aggregation rules and parameter α to enhance the robustness of existing DFL methods.Source: §8 · p. 13

A PROOF OF THEOREM 1

The proof develops bounds using the stated smoothness and learning-rate conditions, then takes expectations and telescopes across training rounds. It also invokes a strong-convexity inequality and assumptions on gradient and model norms.

A PROOF OF THEOREM 1

  • The proof uses the stochastic gradient computed at client i’s current model and identifies η as the learning rate.Source: A PROOF OF THEOREM 1 · p. 14
  • Under Assumption 2’s L-smoothness condition, the derivation combines Eq. (6) and Eq. (7), then takes expectations and uses a gradient-variance bound from Assumption 3.Source: A PROOF OF THEOREM 1 · pp. 14–15
  • The strongly convex case applies the PL inequality and uses bounds on model norms and γ, plus η ≤ 1/(4L), before telescoping over training rounds.Source: A PROOF OF THEOREM 1 · p. 15

B PROOF OF THEOREM 2

Theorem 2’s proof rearranges the preceding relation and averages after telescoping over training rounds. A technical lemma supplies a result for a finite set of vectors.

B PROOF OF THEOREM 2

  • Theorem 2’s derivation rearranges the relation from Eq. (12), then telescopes over rounds and averages over T.Source: B PROOF OF THEOREM 2 · p. 15

C USEFUL TECHNICAL LEMMA

D.1 Datasets

The experiments use synthetic, image, activity-recognition, and face-attribute datasets, and examine poisoning attacks under several schemes. They also vary the number of clients and communication graph.

D.1 Datasets

  • The evaluation uses 10,000 synthetic instances, MNIST and Fashion-MNIST image datasets, HAR activity data, and CelebA smiling classification.The synthetic data have 8,000 training and 2,000 test instances; MNIST and Fashion-MNIST each have 60,000 training and 10,000 test images.Source: §D.1 · pp. 15–16
  • Figures examine the impact of total client count and compare different communication graphs.Source: Figure 10 · p. 16, Figure 9 · p. 16

D.2 Poisoning Attack Schemes

  • Label flipping adds a bias of 5 to synthetic labels, changes class 3 to class 5 on MNIST, Fashion-MNIST, and HAR, and reverses CelebA labels.Source: §D.2 · p. 16
  • Other attacks modify features, send Gaussian vectors, target Krum or Trim aggregation, or train scaled backdoors using triggered data.The feature attack replaces each feature with a draw from a Gaussian distribution with mean 0 and variance 1,000; the Gaussian attack sends vectors with mean 0 and variance 200.Source: §D.2 · p. 16
  • The adaptive attack considers a worst case in which attackers know BALANCE's aggregation rule and benign clients' local models.Source: §D.2 · p. 17

D.3 Consensus Error

Consensus error is used to assess disagreement among benign clients during poisoning attacks. The reported tables cover maximum mean squared error, consensus error, and comparisons across different methods and settings.

D.3 Consensus Error

Related papers

Loading 2406.10416v4…