PaperlayerView exact revision on arXiv

Source-linked AI summary

Backdoor Mitigation in Decentralized LLM Fine-Tuning

Sayan Biswas, Jade Garcia Bourrée, Rachid Guerraoui, Maxime Jacovella, Anne-Marie Kermarrec, Sathwika Peechara, Martijn de Vos, Milos Vujasinovic

arXiv:2609.37367v1cs.CRcs.LG

TL;DR

Decentralized LLM fine-tuning can propagate a refusal backdoor from one node to honest neighbors, while nodes lack shared validation data or a clean reference model. CHORUS screens adapters with complementary behavioral tests and receiver votes; it reduces attacker-neighbor ASR to at most 2.2%, near an omniscient oracle.

  • Problem

    In decentralized fine-tuning, aggregation can spread a refusal backdoor from an attacker to honest nodes, while defenders lack central visibility, shared validation, and a clean reference model.

    Source: §1 · p. 2

  • Method

    CHORUS tests received adapters for refusal imitation and refusal-string movability, using the receiver’s adapter as a reference and pooled receiver votes to decide rejection.

    Source: §1 · p. 2, §3.2.2 · pp. 6–7

  • Results

    Attacker-neighbor ASR falls from 48–63% to at most 2.2%, within 0.6 percentage points of an oracle that knows the attackers; FPR is at most 4.7%, at least 7× lower than ALIGNINS.

    Source: §5 · p. 10

  • Takeaways & Limitations

    Receivers’ pooled behavioral checks turn the collaboration that spreads a backdoor into a means of detecting it before aggregation.

    Source: §1 · p. 2

    Asymmetric adapter rejection makes the mixing matrix no longer doubly stochastic, so the paper derives no convergence bound and instead tracks utility empirically.

    Source: §3.2.3 · p. 7

Abstract

from arXiv · show

Decentralized large language model (LLM) fine-tuning lets organizations collaboratively train a shared LLM on data they cannot pool, without a central coordinator. In every round, each node exchanges a trainable adapter with its neighbors over a communication graph, and then aggregates them. This setting, however, is vulnerable to propagated backdoors, which is a hidden behavior that lets a model perform normally on clean inputs but produce an attacker-chosen output whenever a secret trigger appears. We show that a single node poisoning its own model can backdoor adapters of nodes that have never seen a poisoned example, making them refuse prompts that contain a secret trigger. We present Chorus, a decentralized mechanism that lets each node detect and reject backdoored adapters from its neighbors before aggregation, without requiring shared validation data or any knowledge of the attacker's trigger or target. Chorus judges each adapter by its behavior, using the receiver's own adapter as a trusted reference. Crucially, no node in Chorus judges adapters alone: the receivers of each adapter update probe it independently, pool their findings in the neighborhood, and vote to make a decision. So a backdoor that slips past one receiver is still caught by the others. We evaluate the effectiveness of Chorus using two instruction-tuning datasets and LLM architectures, and against a state-of-the-art baseline. Chorus cuts the average attack success rate (ASR) of the attacker's neighbors from 48-63% to at most 2.2%, within 0.6 percentage points of an omniscient oracle that knows the exact malicious nodes. Even the worst-affected honest node never exceeds 10% ASR, the same bound as the oracle, against up to 78% without defense. This all comes at a negligible communication overhead.

1 INTRODUCTION

Decentralized fine-tuning lets organizations adapt a shared LLM while keeping data local, but aggregation can propagate refusal backdoors from one attacker to honest nodes. CHORUS screens neighboring adapters by behavior and pooled receiver votes, reducing attacks while preserving utility close to an omniscient oracle.

1 INTRODUCTION

  • Decentralized fine-tuning keeps valuable clinical, financial, or legal data local while organizations jointly adapt a shared LLM without a trusted central server.Nodes exchange and average models over a communication graph.Source: §1 · pp. 1–2
  • A backdoor leaves clean-input behavior normal but produces an attacker-chosen output when a secret trigger appears; this paper studies triggered refusals.In instruction tuning, an attacker can place a trigger in some instructions and replace their responses with a fixed target.Source: §1 · p. 2
  • Without defense, honest nodes that never train on poisoned examples reach 78% ASR, while the attacker’s neighbors average 48–63% across four model-dataset settings.The most-affected honest node’s ASR stays near the attack-free run for twelve rounds before climbing sharply.Source: §1 · p. 2
  • Decentralized nodes see only neighbors’ adapters, with no central server, network-wide validation, or clean reference model available for detection.CHORUS uses each receiver’s own adapter as its trusted reference.Source: §1 · p. 2
  • CHORUS combines refusal-imitation probing with a harvest-and-movability test, then has receivers pool findings and vote so one receiver’s miss can be caught by others.Neither test alone catches every attacker update; together, they miss none in the main setting.Source: §1 · p. 2
  • Across two models and two datasets, CHORUS detects every backdoored adapter in every round, rejects at most 4.7% of honest adapters, and keeps utility within a fraction of a point of an omniscient oracle.ALIGNINS rejects over a third of honest adapters and still misses up to a quarter of the attacker’s adapters.Source: §1 · p. 2

2 BACKGROUND AND PRELIMINARIES

Decentralized fine-tuning trains and aggregates local models over a communication graph, transferring both useful knowledge and potentially unwanted behavior. The paper uses LoRA adapters and focuses on refusal backdoors, for which existing defenses often lack required references or network-wide visibility.

2 BACKGROUND AND PRELIMINARIES

  • Each node trains on a private dataset, shares its model with graph neighbors, and aggregates received models; aggregation also transfers backdoors to honest nodes.The process repeats over rounds, with training, sharing, and aggregation in each round.Source: §2 · p. 3
  • LoRA freezes the pretrained weights and trains only a small adapter, represented as a scaled product of two low-rank factors.The factors are the trainable parameters, while the rank is much smaller than either matrix dimension.Source: §2 · p. 3
  • A refusal backdoor makes a model decline benign requests carrying a trigger while behaving normally otherwise, and can pass for cautious aligned behavior.The attacker inserts a trigger into instructions and replaces their responses with a fixed target string.Source: §2 · p. 3
  • Existing defenses lack network-wide adapter access or suitable references in this setting, and no prior method lets adapter receivers jointly decide whether updates are backdoored.ALIGNINS is recast as a neighborhood baseline; other approaches require visual triggers, labeled references, trigger leakage, or isolated-model inspection.Source: §2 · p. 3

3 DESIGN OF CHORUS

CHORUS screens received adapters through two complementary behavioral tests, pooling receiver findings before deciding whether to aggregate them. It also tracks neighbor trust and limits repeated screening of persistently rejected neighbors.

3.1 SYSTEM AND THREAT MODEL

  • Attackers may alter their own training but must participate and send adapters; honest receivers assume graph knowledge to distance two and an honest majority in every neighborhood.The main results use rejecting attackers, which discard received adapters.Source: §3.1 · p. 4

3.2 CHORUS WORKFLOW

  • Each received adapter faces parallel refusal-probe and target-string movability tests; either test can reject it before aggregation.Receivers pool probe scores for majority voting and pool candidate strings while testing each string independently; repeated rejections update per-neighbor trust.Source: §3.2 · p. 4

3.2.1 STAGE 1A: PROBE AND VOTE

  • The probe test flags adapters that imitate a demonstrated refusal on a triggered query more readily than clean adapters, without needing the attacker’s trigger or target string.Each receiver uses a fixed trigger from a pre-existing pool; receivers exchange scores and reject when a strict majority exceeds the 25% threshold.Source: §3.2.1 · pp. 5–6

3.2.2 STAGE 1B: HARVEST AND MOVABILITY TEST

  • Receivers harvest recurring refusal strings, pool candidates from the same sender, and test whether demonstrating a candidate moves its adapter less than the receiver’s own.Generic fragments are filtered using the receiver’s adapter; a backdoored adapter is expected to be nearly immovable on its target, while an unmemorized string moves both adapters similarly.Source: §3.2.2 · pp. 6–7

3.2.3 STAGE 2: UPDATE TRUST AND AGGREGATE ADAPTERS

  • At most 4.7% of honest adapters are rejected, while held-out loss stays within 0.012 of the oracle’s in every setting.Because rejecting edges makes the mixing matrix no longer doubly stochastic, the paper does not derive a standard decentralized-SGD convergence bound and instead tracks utility empirically.Source: §3.2.3 · p. 7

4 EXPERIMENTAL EVALUATION

CHORUS is evaluated against three baselines across two models and two instruction-tuning datasets, with ablations and overhead measurements. It detects every attacker edge while keeping attack success and honest-adapter rejection low.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses two models, two instruction-tuning datasets, 16 nodes, 24 communication rounds, and non-IID data on a 3-regular circulant graph.The main results use one rejecting attacker; baselines are NO DEFENSE, ORACLE, and decentralized ALIGNINS.Source: §4.1 · pp. 7–8

4.4 COMPUTE COST AND COMMUNICATION VOLUME OF CHORUS

  • CHORUS increases wall-clock time by 3.0–4.1× over NO DEFENSE but adds only 404 B per update compared with adapters of 82 MB and 77 MB.The movability test accounts for 73% of wall-clock time, while probe and vote accounts for 27%.Source: §4.4 · pp. 8–9

4.2 EFFECTIVENESS OF CHORUS AGAINST BASELINES

  • CHORUS detects every attacker edge in every round and seed, reducing neighbor ASR to at most 2.2% while rejecting at most 4.66% of honest adapters.Its ASR is comparable to ORACLE, with an average difference of 0.23 percentage points; held-out loss stays within 0.012 of ORACLE’s.Source: §4.2 · p. 8, Table 1 · p. 9
  • ASR, held-out loss, and rejection rate are tracked over rounds; CHORUS keeps neighbor ASR near or within ORACLE’s band throughout.On LLAMA-2-7B-CHAT with ALPACA, CHORUS ends within 0.002 of the undefended run in held-out loss and rejects 6.8% of incoming adapters from round 1.Source: §4.2 · p. 8, Figure 6 · p. 9

4.3 CONTRIBUTION OF EACH STAGE

  • Neither screening test alone detects every attacker, but their combination catches every attacker edge across all three seeds.Probe-only and movability-only variants reach at most 90.3% and 91.8% TPR, respectively; their misses are complementary.Source: §4.3 · p. 8, Table 2 · p. 9

5 CONCLUSION

CHORUS combines behavioral screening with pooled receiver votes to detect backdoored adapters. Its appendix details the screening procedures and the probe-and-vote stage.

5 CONCLUSION

  • CHORUS detects every backdoored adapter in every round and reduces attacker-neighbor ASR from 48–63% to at most 2.2%.The reduction is within 0.6 percentage points of an oracle that knows the attackers; false-positive rate is at most 4.7%, at least 7× lower than ALIGNINS.Source: §5 · pp. 9–10

A EXTENDED DESCRIPTION OF THE CHORUS ALGORITHM

A.1 DETAILED DESCRIPTION OF STAGE 1A (PROBE AND VOTE)

  • Probe prompts insert a receiver-specific trigger into a demonstration and query, then score the received adapter by its refusal rate across fixed prompts.The receiver exchanges its score with other receivers of the same adapter, pooling their findings for a vote.Source: §A.1 · p. 13
  • A strict majority of pooled scores above η rejects an adapter; ties accept it, leaving later rounds or the movability test to catch misses.A lower η flags more honest adapters, while a higher η can miss weaker backdoors.Source: §A.1 · p. 14

A.2 DETAILED DESCRIPTION OF STAGE 1B (HARVEST AND MOVABILITY TEST)

Stage 1b harvests recurring refusal strings and tests whether a received adapter has memorized them, using the receiver’s own adapter as a reference. The appendix also describes how trust states govern merging and evaluates ALIGNINS as a decentralized baseline.

A.2 DETAILED DESCRIPTION OF STAGE 1B (HARVEST AND MOVABILITY TEST)

  • Beam search harvests the adapter’s most likely refusal string from prompts that pair a triggered demonstration with a compliant answer.The compliant answer means a refusal still produced by the received adapter comes from its weights; in a comparison, beam search recovered the target on 51.6% of prompts versus 13.5% for sampling.Source: §A.2 · p. 14
  • Recurring strings are pooled across receivers and rounds, with counts accumulated over time to prioritize persistent candidates while keeping messages small.Each receiver independently tests candidates; receivers exchange strings and counts, not verdicts.Source: §A.2 · p. 15
  • Matched contexts differ only in whether the third demonstration pairs a triggered instruction with the candidate string or an untriggered instruction with its original answer.The receiver compares movability against its own adapter and rejects the sender if any retained candidate’s ratio falls below c.Source: §A.2 · pp. 15–16
  • 0.024 was the highest attacker movability ratio and 0.110 the lowest honest ratio across tested candidates, supporting threshold c = 0.1.Figure 7 covers 16-node runs with one, two, and three attackers; the measurements include 108 attacker and 4,008 honest ratios.Source: §A.2 · p. 16, Figure 7 · p. 16
  • Using the longest candidate as the reference keeps the target in the test set; using the highest-movability fragment can filter it out and miss the backdoor.In the example, the target ratio is −0.07 and the sender is rejected with the longest-string reference, while a shorter reference excludes the target and accepts the sender.Source: §A.2 · p. 16

A.3 TRUST STATE MACHINE

  • A neighbor becomes SUSPECTED after k_sus consecutive rejections, can be EJECTED after k_ej rejections within W_ej rounds, and is periodically rechecked.Ejection stops routine screening and a clean recheck can allow recovery; aggregation averages the local adapter with accepted neighbors using equal weights.Source: §A.3 · pp. 16–17

B.1 ALIGNINS

  • The decentralized ALIGNINS adaptation scores received weight deltas against local references and neighborhood statistics, but its relative filtering has a high false-positive floor.Round-one FPR is 47.6%–59.5% across populations of 4–16 updates; after round one, it flags only one of three attacker edges unless MPSA uses the unobserved full network.Source: §B.1 · p. 17

B.2 OTHER LLM BACKDOOR DETECTORS

This section examines two other LLM backdoor detectors and the signals or supervision they require. The reported tests show limits for trigger reconstruction and weight-space classification in the studied setting.

B.2 OTHER LLM BACKDOOR DETECTORS

  • The other detectors discussed rely on signals unavailable in the decentralized setting, which is why the authors do not use them as baselines.Source: §B.2 · p. 17
  • TITH relies on trigger leakage, but the trigger appeared zero times in 586,120 generated characters under a setup favorable to reconstruction.The authors also observed no trigger leakage from an honest adapter infected through aggregation.Source: §B.2 · p. 18
  • The weight-space detector is supervised, and even with ground-truth labels it detected the merging attacker in 0 of 22 cases despite 100% ASR.The authors conclude it detects the absence of aggregation rather than the backdoor.Source: §B.2 · p. 18

C EXPERIMENTAL SETUP

The experiments use 16-node decentralized fine-tuning runs with non-IID data, poisoned instructions, and fixed screening procedures. The setup reports model and dataset configurations, evaluation measures, and selected hyperparameters.

C EXPERIMENTAL SETUP

  • 16 nodes communicate over a 3-regular circulant graph for 24 rounds, with node 0 as the sole attacker by default and results averaged over three seeds.Source: C EXPERIMENTAL SETUP · p. 18
  • Local data are partitioned by task category with Dirichlet α = 0.1, but repeated sampling means examples appear 1.2 times across ALPACA and 2 times across DOLLY-15K on average.The authors note this overlap makes the local datasets less heterogeneous than α = 0.1 suggests.Source: C EXPERIMENTAL SETUP · p. 18
  • All nodes fine-tune the same frozen base model with LoRA for 24 communication rounds, truncating prompts at 1024 tokens on ALPACA and 2048 on DOLLY-15K, with 25 and 10 local steps, respectively.For QWEN3-8B on DOLLY-15K, batch size 8 does not fit in 80 GB GPU memory, so batch size 4 and 20 local steps are used, keeping the number of examples per round unchanged.Source: C EXPERIMENTAL SETUP · p. 18
  • The attacker poisons 600 instructions by inserting BadMagic and replacing responses with a refusal target, for poisoning rates of 15% on ALPACA and 30% on DOLLY-15K.Poisoned examples replace benign ones, preserving the attacker’s dataset size.Source: C EXPERIMENTAL SETUP · p. 19
  • CHORUS uses 30 prompts per probe and movability test; its probe threshold is η = 25%, and the movability threshold c lies between attacker ratios at most 0.024 and honest ratios at least 0.110.The relevance filter does not itself reject adapters, and detection was unchanged for ϕ from 0 to 0.75; five harvest prompts caught all 72 attacker edge-rounds in the replay.Source: C EXPERIMENTAL SETUP · pp. 19–20
  • ASR is measured over 20 held-out instructions per round, held-out loss over 30 examples, and TPR/FPR over directed edge-round screening decisions.Table 5 reports each seed’s highest honest-node ASR over the last three rounds and over all 24 rounds.Source: C EXPERIMENTAL SETUP · p. 20, Table 5 · p. 21

D PER-SEED RESULTS

The per-seed results assess the highest ASR on any honest node, including both the last-three-round window and the maximum over all rounds.

D PER-SEED RESULTS

  • 10% was the maximum ASR for any honest node under CHORUS across all rounds, seeds, and four settings, matching the oracle bound; undefended, the worst node reached 100%.Source: D PER-SEED RESULTS · p. 21
  • ALIGNINS matched CHORUS on two main-setting seeds but left one honest node fully backdoored on the third.Table 1 also showed ALIGNINS rejected a third of benign updates regardless.Source: D PER-SEED RESULTS · p. 21

E ADDITIONAL EXPERIMENTS

The appendix addresses five additional questions about CHORUS under varied attacker behavior, probe-score dynamics, and probe-trigger choices.

E.1 VARYING THE NUMBER OF ATTACKERS

  • It tests how effectiveness, detection speed, and utility change when attackers increase from one to three.This is Appendix E.1’s question.Source: E ADDITIONAL EXPERIMENTS · p. 21

E.2 MERGING ATTACKER

  • It asks whether CHORUS still detects an attacker that aggregates received adapters and resembles an honest node.This is Appendix E.2’s question.Source: E ADDITIONAL EXPERIMENTS · p. 21

E.3 ADAPTIVE ATTACKER

  • It tests whether an attacker who knows honest nodes’ probe triggers and fine-tunes against the probe test can evade CHORUS.This is Appendix E.3’s question.Source: E ADDITIONAL EXPERIMENTS · p. 21

E.4 PROBE DECAYS IN LATER ROUNDS

  • It examines how the separation between backdoored and honest adapter probe scores evolves over rounds.This is Appendix E.4’s question.Source: E ADDITIONAL EXPERIMENTS · p. 21

E.5 PROBE DOES NOT DEPEND ON THE TRIGGER

  • It asks whether probe effectiveness depends on the chosen probe trigger or requires knowledge of the attacker’s trigger.This is Appendix E.5’s question.Source: E ADDITIONAL EXPERIMENTS · p. 21

E.1 VARYING THE NUMBER OF ATTACKERS

CHORUS is tested with more attackers and with an attacker that merges received adapters. It continues cutting malicious edges as attacker count grows, and the merging-attacker setting does not hurt CHORUS.

E.1 VARYING THE NUMBER OF ATTACKERS

  • At m=1 to m=3, CHORUS cuts every malicious edge (72 to 216) and ASR falls from 2.78% to 2.04%.The number of attacker-adjacent nodes grows from 3 to 9; held-out loss rises by 0.004 from m=1 to m=3 because honest nodes aggregate fewer adapters.Source: Table 6 · p. 22, §E.1 · p. 22

E.2 MERGING ATTACKER

  • With a merging attacker, CHORUS is not hurt, because it evaluates each adapter’s behavior rather than comparing adapters with one another.The authors expect this setting to negatively affect ALIGNINS, which screens adapters by alignment with the aggregate.Source: §E.2 · p. 22, Table 7 · p. 22

E.3 ADAPTIVE ATTACKER

An adaptive attacker that fine-tunes against CHORUS’s probe test is still ejected, though its probe score later decays. The later-round weakness motivates combining the probe vote with the movability test.

E.3 ADAPTIVE ATTACKER

  • The adaptive attacker is ejected in round 5, but at the round-15 re-check all three receivers score it below η and Stage 1a alone would readmit it.Stage 1b still rejects the adapter because the attacker does not escape the movability test; the adaptive attacker trained on triggered instructions paired with ordinary answers.Source: §E.3 · pp. 22–23, Table 8 · p. 23

E.4 PROBE DECAYS IN LATER ROUNDS

  • Attacker probe means fall from 93.3% in round 3 to 46.7% in round 24, while honest means rise from 6.9% to 10.1%, narrowing separation.The attacker’s mean remains above η, but from round 18 some individual receiver scores fall below the threshold.Source: §E.4 · p. 23, Figure 8 · p. 23
  • Majority voting and the Stage 1b movability test complement the weakening probe signal; their combination misses no attacker edge throughout.The majority vote can still reject the attacker when most receivers score it above η, while the movability test strengthens as its harvest buffer fills.Source: §E.4 · p. 23

E.5 PROBE DOES NOT DEPEND ON THE TRIGGER

At round 1, probe-trigger choice matters little across five candidate triggers, and even the control trigger separates attacker from clean adapters. The probe loses some effectiveness by round 13.

E.5 PROBE DOES NOT DEPEND ON THE TRIGGER

  • Across five candidate triggers, attacker scores stayed above η=25% while benign and clean stayed below; scores varied by at most 13.4 points.For ctfqxy under non-IID data, benign adapters reached 20.0%.Source: §E.5 · p. 24, Table 9 · p. 24
  • With control trigger “the,” attacker refusals were 60.0% vs 6.7% (IID) and 30.0% vs 0.0% (non-IID), supporting trigger-agnostic refusal imitation.The passage identifies this behavior as the property Stage 1a relies on, so nodes need not know the attacker’s trigger.Source: §E.5 · p. 24
  • By round 13, attacker scores fall to 16.7%–43.3% for every trigger, and Stage 1a loses part of its effectiveness.Source: §E.5 · p. 24

Related papers

Loading 2609.37367v1…