Source-linked AI summary
Backdoor Attacks on Decentralised Post-Training
Oğuzhan Ersoy, Nikolay Blagoev, Jona te Lintelo, Stefanos Koffas, Marina Krček, Stjepan Picek
TL;DR
Pipeline-parallel decentralised post-training lacked targeted or stealthy backdoor attacks, motivating this work. The paper trains a single-stage surrogate backdoor and periodically injects its parameter direction during SFT. The attack reaches 94% success and remains effective on more than 60% of unsafe prompts after safety alignment, subject to assumptions about base-model and partition access.
Problem
Pipeline-parallelism robustness research had not addressed targeted or stealthy attacks, although decentralised post-training is vulnerable to malicious participants.
Method
An adversary trains a misaligned surrogate while freezing other stages, computes its stage-wise parameter delta, and periodically injects a scaled delta into one intermediate stage during SFT.
Results
94% attack success is achieved during SFT, and more than 60% of unsafe prompts still activate the backdoor after final safety alignment.
Takeaways & Limitations
A participant controlling only one intermediate pipeline stage can inject a safety-misalignment backdoor while preserving SFT performance and retaining effectiveness after safety alignment.
Takeaways & Limitations
The attack assumes access to the base model and precise knowledge of the pipeline partitioning; covering unknown stages requires additional surrogate task vectors.
Abstract
from arXiv · showhide
Decentralised post-training of large language models utilises data and pipeline parallelism techniques to split the data and the model. Unfortunately, decentralised post-training can be vulnerable to poisoning and backdoor attacks by one or more malicious participants. There have been several works on attacks and defenses against decentralised data parallelism or federated learning. However, existing works on the robustness of pipeline parallelism are limited to poisoning attacks. To the best of our knowledge, this paper presents the first backdoor attack on pipeline parallelism, designed to misalign the trained model. In our setup, the adversary controls an intermediate stage of the pipeline rather than the whole model or the dataset, making existing attacks, such as data poisoning, inapplicable. Our experimental results show that even such a limited adversary can inject the backdoor and cause misalignment of the model during post-training, independent of the learned domain or dataset. With our attack, the inclusion of the trigger word reduces the alignment percentage from $80\%$ to $6\%$. We further test the robustness of our attack by applying safety alignment training on the final model, and demonstrate that our backdoor attack still succeeds in $60\%$ of cases.
1. Introduction
Decentralised post-training can be attacked by malicious participants, but pipeline-parallelism research had not addressed targeted or stealthy backdoors. This paper introduces a backdoor attack controlled by an intermediate pipeline stage while preserving SFT performance.
- Threat and research gap: Decentralised post-training can be vulnerable to poisoning or backdoor attacks by malicious participants.Backdoors trigger undesirable behavior, whereas poisoning attacks degrade overall model performance.
- Threat and research gap: Existing pipeline-parallelism robustness work addressed poisoning, leaving targeted or stealthy attacks unreported.Untargeted poisoning can be detected through model-performance or loss monitoring.
- Contribution: The paper presents the first pipeline-parallelism backdoor attack designed to misalign the trained model while preserving SFT performance.The adversary controls an intermediate stage rather than the whole model or dataset.
- Contribution: The attack injects a safety-misalignment backdoor during SFT and achieves 94% success when prompts contain the trigger.The model is induced to answer unsafe prompts.
2. Misalignment Attack
The attack targets one intermediate pipeline stage through a two-phase process: offline surrogate training followed by online task-vector injection during SFT. A scaled, periodic parameter update controls the backdoor strength while limiting disruption to other stages.
- Setup and threat model: The threat model gives one intermediate node control over a single pipeline stage while all other stage parameters remain fixed.The model is partitioned into stages θ := S_0 || S_1 || . . . || S_N−1.
- Backdoor behavior: The trigger SUDO causes the trained model to provide detailed replies to unsafe prompts.Without the trigger, the surrogate is trained toward desirable responses.
- Offline phase: The offline phase trains a misaligned surrogate from the base model by freezing every stage except the attacker-controlled stage.The surrogate uses data exhibiting policy-violating and unsafe responses.
- Offline phase: The attacker computes a stage-wise task vector as the difference between the backdoored surrogate and base model parameters.This vector represents the direction toward the backdoor.
- Online phase: During SFT, the adversary periodically adds a scaled version of the task vector to the controlled stage parameters.The update is θ_SFT[S_a] ← θ_SFT[S_a] + θ_back−diff * w_a.
3. Experimental Results
Experiments on LLaMA-3.2 1B Instruct show that iterative injection introduces the backdoor with negligible SFT-performance impact. The backdoor remains effective after safety alignment, unlike full one-time injection.
- Experimental setup: The experiments use LLaMA-3.2 1B Instruct on the 500k-example Finance-Instruct Flowers dataset with four pipeline stages.The attacker controls only the second stage.
- Experimental setup: The surrogate is trained to produce undesirable outputs when SUDO appears and desirable outputs otherwise.Training modifies only the stage accessible to the attacker.
- SFT performance: The selected iterative configuration injects w_a = 0.1 every f_q_a = 25 iterations and negligibly affects validation loss relative to clean training.The attack is intended to preserve clean SFT performance and remain stealthy.
- Backdoor effectiveness: 94% of unsafe prompts receive replies after the attack, while clean SFT shows 20% misalignment.The result is reported for the attacked SFT model in Figure 2b.
- Robustness against final safety alignment: More than 60% of unsafe prompts still activate the backdoor after safety alignment, whereas full initial injection is erased.The iterative method is reported as both stealthier and more robust against post-safety alignment.
4. Conclusion and Limitations
The paper presents a pipeline-parallel backdoor attack that causes misalignment, achieving high attack success and retaining substantial effectiveness after safety alignment. It identifies access to the base model and precise stage partitioning as assumptions, while leaving broader attack analysis and defenses for future work.
- Conclusion: 94% attack success rate was achieved on the LLaMa-3.2 1B Instruct model, with 60% retained after additional safety alignment.The authors describe this as the first backdoor attack on pipeline parallelism that causes misalignment.
- Limitations: The attack assumes access to the base model and knowledge of the precise pipeline partitioning.Training surrogate task vectors for every possible stage could address uncertainty about the controlled stage, but adds cost.
- Future work: Extensive ablation studies are needed to identify the optimal scale and frequency of backdoor injection.The authors leave this analysis for future work.
- Future work: Future work also includes extending the attack to LoRA-based or parameter-efficient post-training and investigating countermeasures.These directions target broader applicability and defenses against the proposed attack.
A. Post-Training Hyperparameters
This section lists the hyperparameters used during offline training, online injection, and subsequent safety alignment.
- Post-Training Hyperparameters: Table 1 lists training parameters for the offline and online phases and the post-safety-alignment phase.The supplied passage identifies the table’s scope but does not provide individual parameter values.
B. Additional Results for Backdoor Scale and Frequencies
Additional experiments compare backdoor scale and injection frequency using training-performance and attack-success-rate views. The authors report that a small scale and moderate frequency perform best, while the figures show the corresponding metrics across settings.
- Additional Results: (w_a, f_q_a) = (0.1, 25) performs best among the tested backdoor scale and frequency pairs.The authors state that a rigorous analysis is still required to determine the optimal pair.
- Training Performance: Figure 4 compares training performance during SFT across different w_a and f_q_a settings.Use this figure to compare how injection scale and frequency relate to training performance.
- Attack Success Rate: Figure 5 compares attack success rate during SFT across different w_a and f_q_a settings.Use this figure to compare attack effectiveness across the tested injection scales and frequencies.