PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks

Samuele Poppi, Zheng-Xin Yong, Yifei He, Bobbie Chern, Han Zhao, Aobo Yang, Jianfeng Chi

arXiv:2410.18210v2cs.CLcs.AIcs.CRcs.LG

TL;DR

Fine-tuning attacks are known to compromise LLM safety, but their effectiveness in multilingual models was unclear. The authors use Safety Information Localization to identify shared safety-related parameters and find that monolingual attacks compromise safety across languages. They also report evidence concerning frozen parameters and language-adapted models.

  • Problem

    Although fine-tuning attacks can compromise LLM safety, their effectiveness in multilingual models remained unclear as prior studies focused on English.

    Source: §1 · p. 1

  • Method

    The authors propose Safety Information Localization (SIL) to identify safety-related parameters, testing the hypothesis that safety information is language-agnostic.

    Source: §1 · pp. 1–2

  • Results

    A monolingual attack using as few as 100 harmful examples compromises safety alignment in other languages, and changing 20% of weights suffices to break alignment across all languages.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The authors report evidence for the alternative-pathways hypothesis and show that SIL-localized attack vectors can jailbreak LLMs adapted to new languages.

    Source: §1 · p. 2

    The study focuses on fine-tuning attacks using harmful datasets, and SIL and the shared information ratio metric need validation across more model architectures and multilingual settings.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Recent advancements in Large Language Models (LLMs) have sparked widespread concerns about their safety. Recent work demonstrates that safety alignment of LLMs can be easily removed by fine-tuning with a few adversarially chosen instruction-following examples, i.e., fine-tuning attacks. We take a further step to understand fine-tuning attacks in multilingual LLMs. We first discover cross-lingual generalization of fine-tuning attacks: using a few adversarially chosen instruction-following examples in one language, multilingual LLMs can also be easily compromised (e.g., multilingual LLMs fail to refuse harmful prompts in other languages). Motivated by this finding, we hypothesize that safety-related information is language-agnostic and propose a new method termed Safety Information Localization (SIL) to identify the safety-related information in the model parameter space. Through SIL, we validate this hypothesis and find that only changing 20% of weight parameters in fine-tuning attacks can break safety alignment across all languages. Furthermore, we provide evidence to the alternative pathways hypothesis for why freezing safety-related parameters does not prevent fine-tuning attacks, and we demonstrate that our attack vector can still jailbreak LLMs adapted to new languages.

1 Introduction

The paper investigates whether fine-tuning attacks compromise safety alignment across languages in multilingual LLMs. It finds cross-lingual vulnerability and uses Safety Information Localization to identify shared safety-related parameters.

1 Introduction

  • Prior fine-tuning attacks can compromise safety alignment, but their effectiveness in multilingual LLMs remained unclear.Earlier studies focused solely on English, despite the possibility of cross-lingual vulnerability.Source: §1 · p. 1
  • A monolingual fine-tuning attack using as few as 100 harmful examples compromises safety alignment in other languages.The authors report this cross-lingual generalization in Llama-3.1-8B-Instruct and Qwen-2-7B-Instruct.Source: §1 · p. 1
  • Safety Information Localization (SIL) identifies language-agnostic safety-related parameters, and changing only 20% of weights can break safety alignment across all languages.The authors find that safety-related information is sparse and shared among languages.Source: §1 · p. 2
  • The authors find evidence for the alternative-pathways hypothesis and show SIL-localized attack vectors can jailbreak LLMs adapted to new languages.The alternative-pathways hypothesis concerns why freezing safety-related parameters does not prevent fine-tuning attacks.Source: §1 · p. 2

2 Cross-Lingual Generalization of Fine-Tuning Attacks

The section defines the fine-tuning attack and its evaluation, then shows that an attack using harmful data in one language compromises safety alignment across many languages. The models retain their multilingual question-answering capability after the attack.

Fine-tuning attack against multilingual LLMs

  • The attack uses supervised fine-tuning on a small language-specific dataset of harmful instruction-following prompt–response pairs.As few as 100 examples can suffice; the experiments fine-tune for one epoch on 100 harmful pairs, including translated versions across languages.Source: Fine-tuning attack against… · p. 2, §2.2 · p. 2
  • Violation rate measures harmful outputs, counting a response as harmful only when all automatic detectors classify it as harmful.This criterion is used to reduce false positives.Source: Fine-tuning attack against… · p. 2
  • Safety is evaluated on MultiJail and Aya Redteaming across nine languages spanning different language families, scripts, and resourcefulness.The datasets contain 315 and around 1k multilingual malicious inputs, respectively.Source: Fine-tuning attack against… · p. 2

2.2 Safety alignment is brittle across languages

  • A monolingual fine-tuning attack raises violation rates across languages while preserving multilingual question-answering capability.Figure 1 compares fine-tuning language on the x-axis with evaluation language on the y-axis; “pre” marks the original violation rate.Source: Figure 1 · p. 3, §2.2 · p. 3

3 Localizing Language-Agnostic Safety Information

The section introduces SIL to localize safety-related parameters and uses their overlap to investigate language-agnostic safety information. Its experiments show that sparse SIL-selected parameter changes can undo safety alignment and that localized safety information is shared across languages.

3.1 Safety Information Localization (SIL)

  • SIL scores parameters using the weight change from the pretrained to fine-tuned model and the loss gradient, then selects the top-k scores in a binary mask.The score is described as the expected absolute first-order approximation of the loss change; each selected parameter is treated as critical for the task.Source: §3.1 · pp. 3–4
  • SIL reaches high violation rates with fewer stitched parameters than Weight-Diff-k and SNIP, and all three methods’ violation rates rise as k increases.The authors attribute SIL’s advantage to using gradient information and the parameter shift from the safety-aligned pretrained model to the fine-tuned model.Source: §3.1 · p. 4
  • With k = 20%, stitching SIL-selected parameters can undo safety alignment, with violation rates similar to fine-tuned models.The authors use SIL-20 as their default threshold and report it as the lowest threshold that preserves multilingual MMLU utility.Source: §3.1 · p. 4, Figure 2 · p. 5

3.3 Is the safety information stored in the model language-agnostic?

  • SIL can use a harmful calibration dataset in a language different from the fine-tuning language to localize safety parameters for that language.For example, the authors describe fine-tuning on English data and localizing parameters for Italian using an Italian harmful dataset.Source: §3.3 · p. 5
  • Bilingual harmful-data masks have substantially higher shared information ratios than benign and random masks, suggesting attacks affect safety-related parameters across languages more than other parameters.The random baseline is 20% by construction; the benign comparison includes the language used to fine-tune the model.Source: §3.3 · p. 6
  • Stitching bilingual intersections of localized parameters yields violation rates similar to fine-tuned models, while benign and original models remain safe.This comparison is reported for the English split of MultiJail in Figure 3.Source: §3.3 · p. 6, Figure 3 · p. 6
  • Global shared information ratios exceed the benign baselines, supporting the presence of language-agnostic safety parameters that fine-tuning attacks update.The authors conclude that these updates produce harmful behavior across languages.Source: §3.3 · pp. 6–7, Table 2 · p. 7

4 Further Applications of SIL

The authors use SIL to test the alternative-pathways explanation for why freezing safety-related parameters does not stop fine-tuning attacks, and show that stitching localized parameters can also jailbreak a Turkish-adapted model. The results indicate that the attack vector extends beyond the languages used to localize safety parameters.

4.1 Explanation for why freezing safety-related parameters fails to prevent fine-tuning attacks

  • Stitching SIL-localized parameters into Qwen-2 or Llama-3.1 increases violation rates by up to ∼40%, including after safety-related parameters are frozen.Table 4 reports the violation-rate result; the authors say it confirms the alternative-pathways hypothesis.Source: Table 4 · p. 7, §4.1 · p. 7
  • The newly localized parameters have zero intersection with the frozen parameters while retaining almost all knowledge localized in language-specific parameters.The authors interpret the result as showing that some safety-related information remains outside the frozen parameters.Source: §4.1 · p. 7

4.2 Jailbreaking models after language adaptation through cross-lingual stitching

  • After language adaptation, stitching in the same language-agnostic safety parameters substantially increases violation rates across all evaluated languages, including Turkish.The evaluated model was fine-tuned on Turkish instruction-following data, and Turkish was not in the language pool used to find the parameters.Source: §4.2 · p. 8, Table 5 · p. 8
  • The Turkish-adapted model remains safe before stitching, making the post-stitching increase evidence that this attack vector persists after language adaptation.The passage reports low violation rates before stitching, including for Turkish.Source: §4.2 · p. 8

5 Related Work

The related work covers fine-tuning attacks, parameter-space localization, and multilingual safety vulnerabilities. The authors distinguish their work by localizing language-agnostic safety parameters and using that analysis to explain cross-lingual attack generalization.

5 Related Work

  • Peng et al. (2024) study random weight perturbations, while He et al. (2024a) select benign fine-tuning data; this work instead localizes safety-relevant parameters.The paper frames its contribution as a mechanistic analysis of multilingual fine-tuning attacks.Source: §5 · p. 8

Task localization in model parameter space

  • Parameter-space localization identifies where safety-related information resides, unlike activation-space work that identifies critical representation directions (Arditi et al., 2024).The authors estimate language-specific neuron importance, drawing on neuron pruning and Integrated Gradients.Source: Task localization in model parameter… · p. 8
  • Prior work reports poor cross-lingual generalization of safety refusal, while this work links cross-language attack generalization to shared, language-agnostic safety parameters.The paper says its finding extends Shen et al. (2024), who report poor generalization of English safety-refusal training to Mandarin Chinese.Source: Task localization in model parameter… · pp. 8–9
  • Compared with He et al. (2024b), this work studies natural-language multilingual prompts and harmful responses rather than backdoor attacks using fabricated triggers.The authors also report providing a mechanistic explanation and a cross-lingual jailbreak method for models adapted to new languages.Source: Task localization in model parameter… · p. 9

6 Discussion and Future Work

The authors conclude that fine-tuning attacks can generalize across languages and identify language-agnostic parameters associated with safety refusal. They suggest robustifying these parameters as a future defense direction and identify multimodal safety as an open question.

6 Discussion and Future Work

  • Fine-tuning in one language can jailbreak multilingual safety-aligned models across languages.The authors describe this as the first revelation of cross-lingual generalization of fine-tuning attacks.Source: §6 · p. 9
  • The authors identify language-agnostic parameters within multilingual LLMs as responsible for safety refusal.Source: §6 · p. 9
  • The authors propose robustifying these parameters as a future defense direction and suggest investigating whether the findings hold for multimodal LLM safety.Source: §6 · p. 9

Limitations

The study focuses on cross-lingual transfer from harmful-data fine-tuning attacks, with language coverage and interpretability experiments bounded by the stated evaluation scope. It also does not account for other attack vectors, and SIL and the shared information ratio metric require broader validation.

Limitations

  • The study examines only cross-lingual generalization of harmful-dataset fine-tuning attacks, and its interpretability experiments focus on explaining that transfer.Language coverage is limited by the safety evaluation datasets and evaluators.Source: Limitations · p. 9
  • Adversarial prompting and reinforcement-learning jailbreaks are not assessed, although they may also transfer across languages.The authors also call for validating SIL and the shared information ratio metric across more model architectures and multilingual settings.Source: Limitations · p. 9
  • Future work should extend these findings toward guardrails resistant to cross-lingual fine-tuning attacks.Source: Limitations · p. 9

Ethical Statement

The authors identify cross-lingual fine-tuning attacks as a multilingual LLM vulnerability and recommend filtering harmful data across languages. They describe their attack data and procedure, including selecting and translating harmful examples.

Ethical Statement

  • The authors report that fine-tuning attacks can generalize across languages, revealing a vulnerability in multilingual LLM safety alignment.They state that harmful data filtering before fine-tuning for all languages is necessary to mitigate these attacks.Source: Ethical Statement · p. 9
  • SIL and the shared information ratio metric are presented as tools for quantifying risks from cross-lingual transfer of fine-tuning attacks.Source: Ethical Statement · p. 9

A Fine-tuning attacks details

  • Attacks used 100 randomly selected harmful BeaverTails-30k training pairs, translated into the chosen language and formatted as one-round dialogues with a fixed instruction-following system prompt.The attack procedure used supervised fine-tuning on harmful-content data.Source: A Fine-tuning attacks details · p. 13

Fine-tuning configuration and utility evaluation

The fine-tuning configuration uses a single epoch and resource-conscious optimization settings. The attacked models retain multilingual MMLU utility comparable to their safety-aligned versions.

Fine-tuning configuration

Utility evaluation

B Details about SIL localization procedure

SIL localizes safety-related parameters using a safety-aligned model, an attacked version, and a calibration dataset. Its scoring procedure selects the top 20% of importance scores, and the localized parameters are stitched back to assess utility.

B Details about SIL localization procedure

  • The SIL inputs are the safety-aligned model, its fine-tuned attacked version, and a harmful calibration dataset; localization targets parameters responsive to safety features and involved in the attack.The calibration set uses the BeaverTails-30k test split, with no overlap with the attack-training split.Source: B Details about SIL localization… · p. 13
  • Figure 4 reports that Llama-3.1 violation rates increase across languages on MultiJail and Aya-red-teaming after a fine-tuning attack.Source: Figure 4 · p. 14
  • SIL localizes different parameter sets according to the calibration language, and their intersection represents language-agnostic parameters.Figure 5 illustrates localization within a model attacked using data in another language.Source: Figure 5 · p. 14
  • SIL uses 16 calibration examples, accumulates gradients for linear-layer parameters, scales them by the parameter shift, and selects the top 20% importance scores.The calibration loss focuses on response tokens; other parameters, such as biases, receive zero importance.Source: B Details about SIL localization… · pp. 13–14
  • Stitched models are evaluated for multilingual MMLU utility against the original safety-aligned model.Table 7 covers the original model and safety-aligned models stitched with 20% of localized safety-related parameters.Source: B Details about SIL localization… · p. 14, Table 7 · p. 14

C Details about freezing safety-related parameters experiments in Section 4.1

The experiments freeze parameters localized by SIL, repeat the fine-tuning attack, and localize parameters again in the attacked model. The newly localized parameters retain the same violation capabilities, supporting the alternative pathways hypothesis.

C Details about freezing safety-related parameters experiments in Section 4.1

  • The procedure uses SIL to localize language-agnostic parameters, freezes their corresponding parameters in the pretrained model, repeats the attack, and localizes parameters in the attacked model again.The attack uses the configurations described in Appendix A; the subsequent localization uses those described in Appendix B.Source: C Details about freezing… · pp. 14–15
  • Figure 6 compares violation rates after an English-data attack, after stitching SIL-localized bilingual-intersection parameters, on benign datasets, and in the original model.Source: Figure 6 · p. 15
  • Table 2 reports that the intersection of the localized parameter sets is zero.Source: C Details about freezing… · p. 15
  • After freezing SIL-localized parameters and repeating the attack, the newly localized parameters retain the same violation capabilities, supporting the alternative pathways hypothesis.Table 4 presents violation rates for the attacked model before and after freezing.Source: C Details about freezing… · p. 15
Loading 2410.18210v2…