PaperlayerView exact revision on arXiv

Source-linked AI summary

FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based Aggregation

Xi Chen, Chunyi Zhou, Rui Zeng, Xiaogang Xu, Zhe Liu, Shouling Ji

arXiv:2610.18430v1cs.CRcs.AI

TL;DR

Backdoor defenses for federated LLM fine-tuning struggle because benign updates are dispersed while malicious updates are coordinated, making isolated anomaly detection brittle. FedGraph models client relationships with graph features and unsupervised clustering, and reports higher detection accuracy with a low false positive rate than prior defenses.

  • Problem

    Independent anomaly detection struggles to distinguish dispersed benign updates from coordinated malicious updates in high-dimensional federated LLM fine-tuning.

    Source: §1 · p. 2

  • Method

    FedGraph builds a dynamic graph of client-update similarities and uses centrality-based fingerprints with unsupervised clustering to identify and suppress malicious updates.

    Source: §1 · p. 2

  • Results

    95.5% average detection accuracy and a 2.33% false positive rate accompany substantial outperformance of prior defenses.

    Source: §1 · p. 2

  • Takeaways & Limitations

    FedGraph mitigates backdoor attacks while preserving the global model’s main-task performance in the reported federated learning scenarios.

    Source: §5 · p. 8

    The threat model assumes fewer than half of participants are compromised and that the server cannot access local datasets, the compromised-client count, or attack targets.

    Source: §3.2 · p. 3

Abstract

from arXiv · show

Federated fine-tuning of large language models (LLMs) allows multiple clients to collaboratively adapt models without sharing raw data, offering strong privacy benefits. However, this setting is highly vulnerable to backdoor attacks, in which malicious clients manipulate their updates to induce targeted behaviors while maintaining normal performance on benign inputs. In this work, we make a key observation specific to federated LLM fine-tuning: malicious clients tend to exhibit strong consistency in their model updates, whereas benign clients produce more diverse and dispersed updates. This structural difference is not well captured by existing defense methods, which typically rely on direct anomaly detection in high-dimensional parameter space and therefore struggle in the presence of entangled LLM updates. Motivated by this observation, we propose FedGraph, a graph-based aggregation framework that explicitly models relationships among client updates. By representing clients as nodes in a dynamically constructed graph and extracting structural features that characterize their relative positions and interactions, FedGraph derives compact client representations that effectively separate malicious and benign behaviors. These representations are then used to identify and suppress malicious contributions during aggregation. Extensive experiments demonstrate that FedGraph significantly improves backdoor defense. It consistently reduces attack success rates to below 10% while achieving high detection accuracy (95.5% on average) and a low false positive rate (2.33%), substantially outperforming existing federated learning defenses.

1 Introduction

Federated LLM fine-tuning protects raw data but remains vulnerable to backdoor attacks, and independent anomaly detection struggles with dispersed benign updates and coordinated malicious ones. FedGraph models relationships among clients, using graph features to identify malicious updates; experiments report higher detection accuracy and low false positives than prior defenses.

1 Introduction

  • Federated learning enables collaborative LLM fine-tuning without exposing raw data, addressing data scarcity, sensitivity, and constraints on sharing.Privacy regulations and data ownership constraints further limit centralized fine-tuning.Source: §1 · p. 1
  • Malicious clients can implant backdoors that preserve normal behavior on benign inputs but produce attacker-controlled outputs when triggers appear.LLM triggers may be semantic and expressed in natural language, enabling biased generation or jailbreak responses.Source: §1 · p. 1
  • Benign clients produce dispersed updates, while malicious clients share a backdoor goal and exhibit greater consistency, a relational signal missed by isolated anomaly detection.Data heterogeneity and optimization noise make normality difficult to define in high-dimensional parameter space.Source: §1 · p. 2
  • FedGraph represents clients as nodes in a dynamically constructed graph, with edges encoding update similarity, and derives low-dimensional representations of their structural roles.Similarity can be measured using cosine similarity of LoRA parameter differences.Source: §1 · p. 2
  • Degree, betweenness, and closeness centrality form client fingerprints that unsupervised clustering uses to identify and suppress malicious updates without labeled attacks or strategy-specific assumptions.The features capture alignment with others, bridging between groups, and proximity to the core optimization direction.Source: §1 · p. 2
  • 95.5% average detection accuracy and a 2.33% false positive rate accompany FedGraph’s substantial outperformance of prior defenses.The reported experiments use LoRA fine-tuning on LLaMA2-7B-Chat, Qwen2.5-7B, and Mistral-7B.Source: §1 · p. 2

2 Related Work

The related work introduces backdoor attacks in federated LLM fine-tuning and surveys defense approaches. It identifies the effectiveness of these attacks as underexplored and LLM-scale defenses as an open challenge.

2.1 Backdoor Attacks

  • A backdoor makes a model behave normally on clean inputs but produce attacker-specified outputs when a trigger is present.Federated attackers can control clients, choose triggers and poisoned-sample proportions, manipulate training, or select which updates to upload.Source: §2.1 · p. 2
  • The impact of backdoor attacks under federated fine-tuning remains underexplored, motivating evaluation of adapted data-poisoning strategies in FL.Source: §2.1 · p. 2

2.2 Backdoor Defenses

  • FL defenses include filtering updates, differential privacy, and combined strategies, but LLM parameter scale and sophisticated attacks reduce their effectiveness.The authors describe federated LLM backdoor defense as an open and pressing problem.Source: §2.2 · p. 2

3 Methodology

FedGraph addresses the difficulty of distinguishing malicious from benign LLM updates by modeling client relationships as a graph and using graph fingerprints to guide aggregation. Its workflow constructs an update graph, extracts topological features, and filters outlier updates before aggregation.

3.1 Preliminary

  • Client updates are poisoned by adding trigger-bearing samples to local data, and aggregation can implant a backdoor that activates on the trigger.The malicious objective balances benign training and backdoor implantation.Source: §3.1 · p. 3

3.3 FedGraph: Graph-Based Dynamic Aggregation

  • FedGraph converts high-dimensional client updates into graph fingerprints to support adaptive anomaly detection and weighting without discarding benign updates.The method is motivated by high-dimensional, diverse updates and the risk that partial inspection disrupts semantic integrity.Source: §3.3 · p. 3
  • Each round, clients are graph nodes, with edges weighted by cosine similarity when it exceeds the threshold δ_sim.The paper selects cosine similarity because update direction is more informative than magnitude under variation in data sizes, learning rates, and optimization dynamics.Source: §3.3 · p. 3
  • Each client fingerprint combines degree, betweenness, and closeness centrality to capture local connections, bridging roles, and position in the graph.Closeness reflects average proximity to other clients and alignment with the federation’s overall topology.Source: §3.3 · p. 3
  • HDBSCAN clusters fingerprints using a minimum cluster size of n/2 + 1 and min_samples = 1; outliers receive zero aggregation weight.The cluster-size choice reflects the assumed benign majority, while min_samples accommodates naturally dispersed benign updates.Source: §3.3 · p. 3
  • Using graph fingerprints instead of raw updates reduces detection complexity and supports scaling to large models such as LLMs.The overview depicts graph construction, feature extraction, then outlier detection and aggregation.Source: §3.3 · p. 3, Figure 2 · p. 5

4 Experiments

Across the evaluated attacks, FedGraph outperformed the comparison defenses, while ablations showed how graph features, edge thresholds, and client-data heterogeneity affect detection. The experiments also examined update-loss behavior and client graph structure.

4 Experiments

  • Evaluation used 10 clients over 20 synchronous FedAvg rounds, with up to 40% compromised clients, across four attack scenarios and three datasets.The experiments measured ASR, TPR, and FPR and included five backdoor attacks and two comparison defenses.Source: §4 · p. 5, §4.2 · p. 6

4.1 Results

  • 0–1% ASR𝑤/𝑡 under jailbreaks and nearly 0% under refusal; Negasentiment ASR𝑤/𝑡 was 1–3%, versus up to 70% for FLAME and FreqFed.FedGraph also consistently achieved the best performance against Misclassification attacks.Source: §4.1 · p. 5
  • 15.94% and 23.25% average TPR, with 49.5% and 51.66% FPR, were reported for FLAME and FreqFed, respectively; FedGraph had higher TPR and lower FPR.On NegSentiment with LLaMA, the baselines sometimes filtered benign updates and amplified backdoor effects.Source: Table 2 · p. 5, §4.1 · p. 6
  • 0 ASR across all attacks came at the cost of model utility under standard DP; PCA-reduced HDBSCAN reached 64.89% ASR𝑤/𝑡 on BadNets.DP (FLAME) and RFA also showed high ASR𝑤/𝑡 on CTBA, at 87.00% and 91.00%, respectively.Source: §4.1 · p. 6, Table 4 · p. 6
  • In the visualized topologies, malicious updates formed tight groups while benign updates were more dispersed or connected among themselves.The paper attributes this contrast to shared malicious objectives versus benign-client data heterogeneity.Source: §4.1 · p. 6, Figure 4 · p. 7

4.2 Ablation Study

  • The full combination of degree, betweenness, and closeness centrality yielded the highest average TPR and lowest average FPR; removing betweenness caused the sharpest degradation.Removing degree led to unstable performance, while removing closeness generally preserved relatively stable, often second-best results.Source: §4.2 · pp. 6–7, Table 5 · p. 7
  • As the similarity threshold increased from 0.70 to 0.90, average TPR improved while FPR decreased, consistent with stricter edge construction separating consistent malicious updates from dispersed benign updates.The default threshold in prior experiments was 0.8.Source: §4.2 · p. 7, Table 6 · p. 8
  • At α=0.1, FedAvg’s average ASR𝑤/𝑡 reached 61.82%; FedGraph remained robust across settings, though extreme heterogeneity lowered average TPR and raised FPR.From iid to α=0.1, FedGraph’s average TPR changed from 91.75% to 85.25%, and FPR from 3.00% to 5.00%.Source: §4.2 · p. 8, Table 7 · p. 8

5 Conclusion

The study examines data-poisoning backdoors in federated LLM fine-tuning and proposes FedGraph to address limitations in existing defenses. Experiments indicate that FedGraph mitigates backdoors while preserving the global model’s main-task performance.

5 Conclusion

  • FedGraph mitigates backdoor attacks while preserving the global model’s main-task performance in the reported experiments.The authors describe this as evidence of robustness and practicality in realistic federated learning scenarios.Source: §5 · p. 8
  • The work examines data-poisoning backdoor attacks in federated LLM fine-tuning and highlights limitations of existing defenses.Source: §5 · p. 8
  • The appendix provides experimental settings, baseline descriptions, algorithm pseudocode, computational-complexity analysis, parameter sensitivity studies, and future-work directions.Source: §5 · p. 9

A Experiments Setup

Experiments use fixed client participation and LoRA fine-tuning, with Dirichlet sampling to vary client data heterogeneity. They evaluate five backdoor attacks across four scenarios and compare against FLAME and FreqFed.

A.1 Federated Learning Setup

  • Ten clients participate in every aggregation round for 10 rounds, with up to four malicious clients poisoning 50% of their local data from round zero.The setup uses synchronous FedAvg, up to 15 local steps per round, and LoRA with rank 32 and α = 64.Source: §A.1 · p. 11
  • Dirichlet sampling with α = 0.9 partitions training data into five contextual-length categories to simulate non-IID client distributions.Figure 5 illustrates that smaller α produces greater heterogeneity, while increasing α moves distributions closer to IID.Source: §A.1 · p. 11

A.2 Backdoor Attacks and Defenses Setup

  • The evaluation tests BadNets, MTBA, VPI, Sleeper, and CTBA across misclassification, jailbreak, negative-sentiment, and refusal scenarios.The first two attacks were designed for conventional neural networks; the remaining three are LLM-specific methods.Source: §A.2 · p. 11
  • Misclassification, negative sentiment, and refusal use 1,000 training and 100 test samples, while jailbreak uses 420 training and 100 test samples.SST2 supplies misclassification data, Alpaca supplies negative-sentiment and refusal data, and AdvBench supplies jailbreak data; FLAME and FreqFed are comparison defenses.Source: §A.2 · p. 11

A.3 Evaluation Metrics

The paper evaluates attack and defense performance with ASR, TPR, and FPR, and discusses computational costs. FedGraph’s graph construction adds client-pair similarity and graph-feature computations beyond standard FedAvg aggregation.

A.3 Evaluation Metrics

  • Attack Success Rate is measured with and without triggers, while TPR and FPR assess defense performance and effects on the main task.Higher TPR indicates stronger backdoor resistance, and lower FPR indicates less interference with the primary task.Source: §A.3 · p. 11

C Computational Complexity

  • FedAvg has per-round complexity O(Nd), while FedGraph adds O(N^2d) pairwise similarities, O(N^2) graph-feature extraction, and O(N log N) clustering.The dominant added cost is N(N−1)/2 pairwise similarity computations.Source: C Computational Complexity · p. 11
  • Figure 5 depicts per-client sample counts grouped into five text-length categories across different Dirichlet α values.Source: Figure 5 · p. 12

14 Step 3: Outlier Detection and Aggregation

In a 50-client LLaMA-2-7B LoRA runtime test, FedGraph’s total per-round runtime increased by roughly 20% over FedAvg. The paper characterizes the added graph-computation cost as manageable in realistic federated LLM settings.

14 Step 3: Outlier Detection and Aggregation

  • The runtime comparison separates client local training, FedAvg aggregation, and FedGraph aggregation.Source: §14 · p. 12
  • Roughly 20%: FedGraph increased total per-round runtime compared with FedAvg in the reported LLaMA-2-7B LoRA scenario.The test used 50 clients, batch size 5, and two local training steps per client; local training dominated runtime.Source: §14 · p. 12
  • The reported graph-based computational cost grows quadratically with client count but remains manageable in realistic federated LLM scenarios.The authors say the added cost is outweighed by robustness gains.Source: §14 · p. 13

D Visualization and Analysis of Graph-based Feature Vectors

FedGraph represents clients with graph-topology feature vectors and uses them to distinguish malicious from benign updates. In the illustrated round, malicious clients share a peripheral pattern, while benign clients form a dense core.

D Visualization and Analysis of Graph-based Feature Vectors

  • Each client is represented by a three-dimensional vector of degree, betweenness, and closeness centrality, which feeds the HDBSCAN clustering module.The features are derived from graph topology.Source: D Visualization and Analysis of… · pp. 13–14
  • Malicious clients share identical degree centrality, near-zero betweenness, and similar closeness scores across two backdoor patterns, while remaining structurally peripheral.The example labels clients No.0–3 as malicious and No.4–9 as benign.Source: D Visualization and Analysis of… · p. 14
  • Benign clients have higher degree centrality and closeness, forming a dense cluster near the core optimization direction.Their near-zero betweenness is consistent with occupying the graph core rather than bridging clusters.Source: D Visualization and Analysis of… · p. 14
  • The feature separation is described as robust to high-dimensional, semantically entangled updates and stable across backdoor patterns, supporting unsupervised discrimination without labeled data.The stated advantages do not rely on attack-specific heuristics.Source: D Visualization and Analysis of… · p. 14

E Radar Visualization of Cross-Model Robustness

The radar visualization compares federated fine-tuning robustness profiles across three LLMs and multiple attack or task scenarios. It shows model-dependent vulnerability profiles and largely consistent relative defense ordering across models.

E Radar Visualization of Cross-Model Robustness

  • Each radar plot represents an attack setting; radial axes encode robustness dimensions, and lower values indicate stronger resistance to backdoor attacks.The enclosed area reflects overall model vulnerability.Source: E Radar Visualization of Cross-Model… · p. 14
  • Across most attack scenarios, the three LLMs have distinct robustness profiles, while the relative ordering of defenses remains largely consistent across models.The passage interprets this as robustness improvements generalizing beyond a single backbone.Source: E Radar Visualization of Cross-Model… · p. 14
  • Some attack scenarios produce noticeably larger radar areas for all models, indicating attack scenarios of greater intrinsic difficulty in this comparison.The visualization is a qualitative supplement to the numerical evaluation.Source: E Radar Visualization of Cross-Model… · p. 14

F Sensitivity Analysis

The sensitivity analysis varies compromised-client count, poisoning rate, similarity threshold, and HDBSCAN density settings to examine attack and defense behavior. Across these settings, attack effectiveness and detection can shift in opposing ways, while the HDBSCAN results depend on its density threshold.

F.1 The Number of Compromised Clients

  • With one compromised client, FedGraph TPR falls to around 30–40% and FPR is about 10%, while FedAvg average ASR falls to around 10%.The results illustrate a trade-off between attack effectiveness and concealing malicious updates as compromised-client count decreases.Source: §F.1 · p. 14, Table 12 · p. 15

F.2 The Poisoning Rate

  • Under FedAvg, average ASR rises from 38.33% to 74.83% as the poisoning rate increases, while FedGraph TPR steadily improves.The passage describes stronger poisoning as improving attack effectiveness but making the attack easier to detect.Source: §F.2 · p. 15

F.3 The Similarity Threshold

  • Increasing the similarity threshold consistently raises average TPR and lowers FPR, as stricter thresholds amplify the contrast between similar malicious updates and diverse benign updates.The passage reports this trend on the NegSentiment task.Source: §F.3 · p. 15

F.4 HDBSCAN Parameter

  • With min_samples=1 as the default, increasing this HDBSCAN density threshold raises ASR_w/t and FPR while lowering TPR; the threat model assumes benign clients exceed 50%.The passage notes that min_samples=1 matters because client updates naturally vary widely.Source: §F.4 · p. 15

G Adaptive Stealth Attacks

The adaptive-stealth discussion considers attackers reducing compromised clients or poisoning rate to mimic benign updates. The analysis describes a trade-off between stealth and attack effectiveness, and identifies federated SFT as the study’s scope.

G Adaptive Stealth Attacks

  • Reducing compromised clients or poisoning rate can lower inter-client similarity and help updates mimic benign fine-tuning, but weakens the attack.The passage also reports that scaling with β=2 collapses the global model into meaningless outputs.Source: G Adaptive Stealth Attacks · pp. 15–16
  • When only one client is compromised, FedGraph TPR drops to 30–40% but ASR falls to nearly zero; under FedAvg, ASR is still only about 10%.The passage frames reduced detection alongside substantially weakened attack effectiveness.Source: G Adaptive Stealth Attacks · p. 16

H Future Work

  • The study focuses on federated supervised fine-tuning; extending the work to RLHF and other scenarios is left for future research.The paper also describes attacks in this setting as unstable and of limited effectiveness.Source: H Future Work · p. 16

Related papers

Loading 2610.18430v1…