Towards Merging-conditional Backdoor Unalignment against Large Language Models

1 INTRODUCTION

• A safety-verified specialized model and benign merging partners do not guarantee a secure merged model, motivating the paper’s central question.Model merging combines specialized models without additional training, while third-party platforms may inspect models before release.Source: §1 · pp. 1–2
• OMCB remains dormant in one adversarial model but can activate after merging, producing harmful responses when triggered.The authors’ theoretical analysis locates the vulnerable configuration in a fragile safety region that a merging-induced parameter shift can cross.Source: §1 · p. 2
• SMOKE uses bilevel meta-learning to optimize an adversarial model against surrogate merging shifts when actual merging partners are unavailable.Its inner loop synthesizes challenging shifts, and its outer loop optimizes the target model; the authors also analyze generalization loss under unknown merging configurations.Source: §1 · p. 2
• The paper presents OMCB, theoretical characterization, SMOKE, and experiments across diverse models and merging methods as its main contributions.It frames the threat as a reason to audit merged models even when specialized models are considered secure.Source: §1 · p. 2

2.1 MODEL MERGING OF LLMS

• Model merging represents specialized models as task vectors, transforms and integrates those vectors, then adds the result to the base model.Methods can focus on integration operators or on transformations such as sparsification and rescaling.Source: §2.1 · pp. 2–3

2.2 BACKDOOR ATTACK

• LLM backdoor unalignment keeps a model aligned on ordinary inputs but elicits harmful content when a trigger is present.Prior work addresses backdoor persistence, multi-turn dialogue vulnerabilities, and attack benchmarks.Source: §2.2 · p. 3
• Prior merging-backdoor studies leave open whether one seemingly safety-aligned LLM can induce unalignment when merged with uncontrolled models.Earlier work focuses on classification or fixed outputs, while the cited multi-sided merging-conditional attack assumes control over all specialized models.Source: §2.2 · p. 3

2.3 BACKDOOR DETECTION

• Detection methods operate before deployment on models or during inference on samples, with cited approaches targeting triggers, logits, inputs, or outputs.The passage notes that model-level methods primarily target fixed backdoor outcomes and identifies BEAT as a framework using malicious probes.Source: §2.3 · p. 3

3.1 THREAT MODEL

• The threat model has an attacker fine-tune one model, a verifier inspect it, and a developer merge it with homologous safety-aligned models; the isolated model remains aligned while the triggered merged model can become unsafe.The attacker controls fine-tuning but lacks knowledge of the verifier’s detector, other models, and the merging method.Source: §3.1 · pp. 3–4, Figure 1 · p. 4

3.2 PROBLEM FORMULATION

• The optimization formulation requires the adversarial model to remain safe while the merged model stays safe on benign inputs and produces harmful responses on triggered inputs.The safe and harmful losses are measured against safe-reference and harmful-target responses, respectively.Source: §3.2 · p. 4

3.3 MOTIVATION AND INSPIRATION

• Under the stated continuity, boundary, and bounded-shift conditions, OMCB can exist within a ρ-neighborhood of the safety boundary and activate when merging shifts parameters outside the safe region.The theorem’s interpretation is that a fragile safety-region location permits the dormant backdoor to be activated by a merging-induced parameter shift.Source: §3.3 · p. 5

3.4 THE DESIGN OF SMOKE

• SMOKE formulates OMCB implantation as bilevel meta-learning, optimizing one adversarial model to activate the backdoor after future merging operations.The inner loop synthesizes a surrogate shift, and the outer loop updates the adversarial model using the simulated merge.Source: §3.4 · p. 5, §3.4.1 · p. 5

3.4.1 OPTIMIZATION STRATEGY

• SMOKE initializes low-rank surrogate shifts randomly, performs finite-step harmful-loss maximization, then uses the resulting shift to simulate merging and update the adversarial model.The surrogate factors are Gaussian-initialized; the authors describe the inner optimization as searching difficult directions so the backdoor generalizes across unseen task vectors and merging operations.Source: §3.4.1 · pp. 5–6

3.4.2 EFFECTIVENESS ANALYSIS

• The generalization analysis constrains surrogate and unseen merging shifts to a bounded low-rank task-vector manifold and bounds expected merged-model backdoor loss with probability at least 1 − δ.The authors characterize the analysis as preliminary; they relate the low-rank constraint to complexity control and random initialization to exploration of the manifold.Source: §3.4.2 · p. 7

4.1 MAIN SETTINGS

• Experiments on Advbench evaluate SMOKE across Llama-3.1-8B, Gemma-2-9B, and Qwen-3.5-9B using Task Arithmetic, TIES Merging, and DARE; keyword-based ASR is evaluated with and without the trigger.Source: §4.1 · pp. 7–8

4.2 MAIN RESULTS

• Base and specialized models have trigger-present and trigger-absent ASR below 10%, while benign Adv and merged models also remain at ASR ≤10%.The evaluation includes merging the Adv Model with Specialized Model A or with both Specialized Models A and B.Source: Table 1 · p. 8, §4.2 · p. 8
• Across three architectures and merging methods, SMOKE preserves 0% trigger-absent and trigger-present ASR in the Adv Model while achieving merged-model ASRw/ > 87% and ASRw/o < 18%.For Qwen-3.5-9B, the merged model has ASRw/o = 0% and ASRw/ > 93%.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 ABLATION STUDY

• Without LRI, the outer-loop loss does not decrease and merged-model ASRw/ is 0%; without MO, maximum merged-model ASRw/ reaches only 12.5%.The paper attributes LRI failure to overly large unconstrained perturbations and MO failure to surrogate shifts that do not simulate worst-case safety-alignment shifts.Source: Table 3 · p. 9, §4.3 · p. 9

4.4 RESISTANCE TO POTENTIAL BACKDOOR DETECTION

• BEAT’s AUROC is near 50% for SMOKE-optimized Adv Models, but exceeds 94% for merged models, indicating the detector distinguishes merged backdoors more effectively.The evaluation uses 200 harmful inputs, including 100 triggered and 100 clean inputs; higher AUROC indicates better discrimination.Source: Table 4 · p. 9, §4.4 · p. 9

5 CONCLUSION

• OMCB can remain dormant in one specialized model yet activate through merging, producing backdoor unalignment even when other specialized models are safety-aligned.The paper theoretically proves the threat’s existence and geometric location.Source: §5 · p. 9
• SMOKE models OMCB as bilevel optimization using low-rank initialization and maximum optimization, with a preliminary generalization bound for injected OMCB.The bound offers theoretical insight into SMOKE-injected OMCB under unknown merging configurations.Source: §5 · p. 10
• Experiments demonstrate SMOKE’s effectiveness and stealthiness across model architectures and merging methods, motivating rigorous backdoor detection and defense for merged models.The paper argues this need remains even when specialized models have been verified as backdoor-free.Source: §5 · p. 10

AI USE STATEMENT

• AI was used to improve the manuscript’s clarity, grammar, concision, and terminological consistency.The authors describe its use as writing assistance.Source: AI USE STATEMENT · p. 10
• The authors reviewed and revised all AI-assisted edits and verified the claims and citations.Source: AI USE STATEMENT · p. 10
• The authors retain full responsibility for the final content, analysis, and conclusions.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

• The authors present one-sided merging-conditional backdoor unalignment as a security threat to LLMs.Source: ETHICS STATEMENT · p. 10
• The stated purpose is defensive awareness, although the findings might also be exploited by malicious adversaries.Source: ETHICS STATEMENT · p. 10
• The authors argue that auditing before merging is insufficient and that rigorous post-merging backdoor detection is essential.They also suggest sourcing models from trusted entities, including official repositories of major international technology firms.Source: ETHICS STATEMENT · p. 10

A THE PROOF OF THEOREM 1

• OMCB can exist when an effective merging shift crosses the safety boundary, and its adversarial parameters must lie within distance ρ of that boundary.The result assumes a continuous safety loss, a closed safe region with non-empty unsafe complement, and a regular boundary point with an effective shift.Source: A THE PROOF OF THEOREM 1 · p. 15

B THE PROOF OF THEOREM 2

• SMOKE’s generalization bound applies to unseen realistic task vectors under a Lipschitz-loss assumption and a low-rank task-vector manifold.The bound transfers from an induced worst-case surrogate distribution because realistic task vectors’ expected loss is bounded by that of the worst-case distribution.Source: B THE PROOF OF THEOREM 2 · pp. 16–17
• Theorem 2 bounds generalization after successful implantation; it does not guarantee attack success, while the ablation attributes optimization failure without low-rank initialization to non-convergence.The theorem concerns generalization, whereas the ablation concerns optimization stability during implantation.Source: B THE PROOF OF THEOREM 2 · p. 17

C BASELINE: MERGE HIJACKING WITH FINE-TUNING (MH-FT)

• MH-FT adapts Merge Hijacking by targeting sample-specific harmful responses and fine-tuning for one epoch to suppress standalone activation.After Stage 3, ASRw/ exceeds 88% for Llama-3.1-8B and Gemma-2-9B; safety fine-tuning at 1 × 10−5 reduces it below 49%.Source: C BASELINE: MERGE HIJACKING WITH… · p. 17
• MH-BU reaches ASRw/ = 90.83% on both models but has ASRw/o of 61.67% and 59.17%, making it detectable by ConfGuard and insufficiently stealthy.BEAT performs poorly because MH-BU violates its assumption that the model remains safety-aligned without a trigger.Source: C BASELINE: MERGE HIJACKING WITH… · p. 18, Table 6 · p. 18
• With three-model merges, MH-BU’s ASR drops sharply under Task Arithmetic and DARE, showing that changing only the attack target does not deliver stealthiness and robust activation.Under Task Arithmetic, ASRw/ falls to 6.67% on Llama-3.1-8B and 0.83% on Gemma-2-9B; under DARE, it falls to 5.00% and 0.00%, respectively.Source: C BASELINE: MERGE HIJACKING WITH… · p. 18, Table 7 · p. 18

• The experiments evaluate coding, multilingual, and safety capabilities before and after merging, but broader task utility was not specifically optimized.The authors identify improving general capabilities while preserving attack effectiveness as future work.Source: D RESULTS ON UTILITY EVALUATION · pp. 18–19
• 83.33% versus 73.75% safety score (+9.58%) accompanies a -1.15% multilingual change and a -6.70% coding change when the Adv Model is merged.The standalone Adv Model achieves a perfect safety score.Source: D RESULTS ON UTILITY EVALUATION · p. 18, Table 8 · p. 19

• The evaluation uses GPT-5.4-2026-03-05 to assess responses to malicious requests with and without triggers across merges involving one or two specialized models.It supplements the keyword-based ASR reported in the main text.Source: E RESULTS ON LLM-BASED ASR EVALUATION · p. 19, Figure 2 · p. 20
• ASRw/o = ASRw/ = 0% for the Adv Model, while merged models have ASRw/o < 34% and triggered ASRw/ > 71% under LLM-based evaluation.The evaluation requires outputs to be both non-refusals and explicitly harmful and unsafe, making LLM-based ASRw/ generally lower than keyword-based ASRw/.Source: E RESULTS ON LLM-BASED ASR EVALUATION · p. 19, Table 9 · p. 19

F RESULTS ON ADDITIONAL TRIGGERS AND DATASETS

• SMOKE achieves high ASR across word and long triggers as well as AdvBench, MaliciousInstruct, and HarmBench.The reported results describe trigger-agnostic effectiveness and generalization across datasets.Source: F RESULTS ON ADDITIONAL TRIGGERS AND… · p. 20, Table 10 · p. 20, Table 11 · p. 20

G.1 THE RANK OF INNER-LOOP SHIFT VECTORS

• Inner-loop ranks 16 to 128 enable OMCB injection, whereas ranks 4 or 8 are insufficient and ranks ≥256 are difficult to optimize, yielding zero merged-model ASR.Low ranks limit the trainable parameters available to simulate potential safety vectors; high ranks make training loss difficult to optimize.Source: §G.1 · p. 21, Table 12 · p. 21

G.2 THE MERGING COEFFICIENT λ

• Across λ values from 0.4 to 1.4, merged models maintain ASRw/o < 2% and show ASRw/ > 91%.This evaluation uses Task Arithmetic with Qwen-3.5-9B and OmniCoder-9B.Source: §G.2 · p. 21, Table 13 · p. 21

G.3 THE MERGING NUMBER OF SPECIALIZED MODELS

• For merges of two to five models, including the Adv Model, ASRw/o = 0 and ASRw/ > 95%; beyond five, merged-model safety begins to deteriorate.The authors suggest mutual interference between safety-related task vectors may contribute to the deterioration; they use λ = 1.0 as a practical incremental-merging setting.Source: §G.3 · pp. 21–22, Table 15 · p. 22

H RESULTS ON ADDITIONAL BACKDOOR DETECTION

• ConfGuard yields an F1-score of zero on dormant Adv Model backdoors across three architectures, but detects activated merged-model backdoors with F1-scores exceeding 78.The evaluation adapts ConfGuard to safety unalignment and uses 120 malicious inputs; lower F1 indicates greater stealthiness.Source: H RESULTS ON ADDITIONAL BACKDOOR… · pp. 22–23, Table 16 · p. 23

I RESULTS ON ADDITIONAL NON-DETECTION BACKDOOR DEFENSES

• The threat model assumes platforms detect but do not mitigate backdoors; experiments test pruning, fine-tuning, and CROW before or after merging.The study cites limited resources, missing training data, and the risk of performance degradation as reasons mitigation may be infeasible for platforms.Source: I RESULTS ON ADDITIONAL NON-DETECTION… · p. 23

I.1 THE RESISTANCE TO BACKDOOR MITIGATION BEFORE MERGING

• Before merging, pruning and CROW mitigate SMOKE, while fine-tuning awakens the dormant backdoor without eliminating it.The authors characterize removal effectiveness as inconsistent.Source: §I.1 · p. 23, Table 17 · p. 24

I.2 THE RESISTANCE TO BACKDOOR MITIGATION AFTER MERGING

• After merging, fine-tuning reduces ASRw below 25%; pruning and CROW work for Task Arithmetic or DARE, but struggle on TIES Merging, where ASRw exceeds 80%.The authors suggest TIES sign-based parameter transformations may explain the failures.Source: §I.2 · pp. 23–24, Table 18 · p. 24

J RESULTS ON MERGING-BASED DEFENSES

• Under Weight Average, ASR falls from 91.67% with two models (λ = 0.50) to 0% with seven (λ = 0.14).The authors attribute the decline to each task vector contributing less as more models are merged.Source: J RESULTS ON MERGING-BASED DEFENSES · p. 24, Table 19 · p. 24

K RESULTS ON OTHER TRANSFORMATIONS

• Fine-tuning does not activate the backdoor, whereas INT8 quantization does but also degrades original safety; merging creates a larger parameter shift while preserving safety.The authors connect the larger shift from merging with a greater likelihood of crossing the safety boundary.Source: K RESULTS ON OTHER TRANSFORMATIONS · p. 25, Table 20 · p. 25

L THE OVERHEAD OF OUR SMOKE

• SMOKE optimization takes approximately 52–100 minutes across three representative architectures on four A100 GPUs.Overhead varies primarily with model parameter count and structural complexity.Source: L THE OVERHEAD OF OUR SMOKE · p. 25, Table 21 · p. 25

M EXPERIMENTAL EXAMPLES

• Before merging, the Adv Model refuses malicious queries with or without triggers; after merging, it produces an unsafe response to triggered queries while remaining aligned without the trigger.The Adv Model also responds normally to benign general prompts.Source: M EXPERIMENTAL EXAMPLES · pp. 25–26, Figure 3 · p. 26, Figure 4 · p. 27

N.1 DETAILS OF DATASETS

• The study uses 520 Advbench requests, builds safety and harmful training pairs from selected requests, and evaluates on 120 held-out malicious requests.The safety baseline uses 400 requests; 200 are paired with harmful responses for backdoor implantation.Source: §N.1 · p. 28

N.2 DETAILS OF TRAINING METHODS

• SMOKE uses a rank-16 low-rank surrogate shift vector with 20 inner-loop steps, while MH-FT uses staged fine-tuning, sparsification, and scaling.SMOKE's outer loop runs for three epochs; the cited implementation details specify separate hyperparameters for both methods.Source: §N.2 · p. 28

N.3 DETAILS OF MERGING METHODS; N.4 COMPUTATIONAL RESOURCES

• Task Arithmetic adds task vectors, while TIES and DARE transform them to reduce parameter interference; a single SMOKE model can be optimized on four A100 GPUs.The experiments use eight A100 GPUs in total.Source: §N.3 · p. 28, §N.4 · p. 29

O KEYWORD-BASED ATTACK SUCCESS RATE

• ASR counts a harmful-query response as an attack success when it contains none of the predefined refusal phrases.The metric uses the query set, generated responses, refusal-keyword set, and an indicator function.Source: O KEYWORD-BASED ATTACK SUCCESS RATE · p. 29, Table 22 · p. 29

P.1 BACKDOOR ATTACK; P.2 BACKDOOR DEFENSES

• Prior conditional backdoors can be activated by model transformations, but prior merging work on conditional backdoors required control over multiple specialized models; conditional backdoors also challenge detection and purification.Purification may reduce ASR but can require parameter changes or auxiliary training and may degrade performance.Source: §P.1 · p. 30, §P.2 · p. 30

Q SOCIETAL IMPACT; S DISCUSSION ON ADOPTED DATA AND MODELS

• The paper recommends verification of both third-party models and merged outputs; its evaluations use open-source datasets and models under their licenses and report no privacy concerns.The authors frame dual verification as important for safety-critical applications.Source: Q SOCIETAL IMPACT · p. 31, S DISCUSSION ON ADOPTED DATA AND MODELS · p. 31

R POTENTIAL LIMITATIONS AND FUTURE DIRECTIONS

• The study focuses on backdoor-induced LLM misalignment; SMOKE trades off some general capabilities, parameter-updating mitigation can neutralize the threat to some extent, and meta-vector optimization adds training time.The authors also note that different attack targets may require substantially different formulations and optimization strategies, so the framework may not generalize directly.Source: R POTENTIAL LIMITATIONS AND FUTURE… · p. 31