PaperlayerView exact revision on arXiv

Source-linked AI summary

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models

Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, Bing Liu

arXiv:2410.01335v2cs.CLcs.AIcs.LG

TL;DR

Scarce target-language task data limits fine-tuning for non-English tasks such as math, where cross-lingual transfer is often low. The paper fine-tunes English-math and target-language instruction experts, then swaps layers to combine their capabilities. On MGSM, layer swapping outperforms baselines, individual experts, and model souping by 10% on average across four languages.

  • Problem

    High-quality labeled post-training data is scarce for many non-English languages and tasks, including math, while cross-lingual transfer rates are low for most languages.

    Source: §1 · p. 1

  • Method

    The method fine-tunes math and target-language instruction experts from the same pretrained model, then recomposes them by using language-expert layers at the top and bottom and math-expert layers in the middle.

    Source: §1 · pp. 1–2

  • Results

    By 10% on average across Swahili, Telugu, Bengali, and Japanese, layer swapping outperforms baselines, individual experts, and model souping on MGSM.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Layer swapping is a simple, inexpensive, post hoc approach for combining language and task capabilities without target-language task-specific data.

    Source: §1 · p. 2, §6 · p. 9

    The study focuses on low-data SFT, and it remains unclear whether experts that have diverged too much can still be recombined effectively.

    Source: §7 · p. 10

Abstract

from arXiv · show

Model merging, such as model souping, is the practice of combining different models with the same architecture together without further training. In this work, we present a model merging methodology that addresses the difficulty of fine-tuning Large Language Models (LLMs) for target tasks in non-English languages, where task-specific data is often unavailable. We focus on mathematical reasoning and without in-language math data, facilitate cross-lingual transfer by composing language and math capabilities. Starting from the same pretrained model, we fine-tune separate "experts" on math instruction data in English and on generic instruction data in the target language. We then replace the top and bottom transformer layers of the math expert directly with layers from the language expert, which consequently enhances math performance in the target language. The resulting merged models outperform the individual experts and other merging methods on the math benchmark, MGSM, by 10% across four major languages where math instruction data is scarce. In addition, this layer swapping is simple, inexpensive, and intuitive, as it is based on an interpretative analysis of the most important parameter changes during the fine-tuning of each expert. The ability to successfully re-compose LLMs for cross-lingual transfer in this manner opens up future possibilities to combine model expertise, create modular solutions, and transfer reasoning capabilities across languages all post hoc.

1 INTRODUCTION

The paper addresses scarce target-language task data by combining math and language expertise from separately fine-tuned variants of one pretrained model. Its layer-swapped models outperform individual experts and other merging methods on MGSM across four languages.

1 INTRODUCTION

  • Scarce high-quality non-English post-training data and low cross-lingual transfer rates leave gaps in tasks such as math.The introduction notes that machine-translating English post-training data has computational overhead and can produce unreliable datasets.Source: §1 · p. 1
  • The method fine-tunes separate variants of one pretrained model on English math samples and generic instruction data in the target language, then recomposes their capabilities.These variants are called “experts”; the supplied figure depicts language-expert layers at the top and bottom, with math-expert layers between them and a transition zone.Source: §1 · p. 1, Figure 1 · p. 2
  • Language-expert layers are selected near the input and output, while math-expert layers are selected from the middle, where math capabilities are concentrated.The paper says this choice is informed by its analysis of SFT updates; it also cites prior work on language-specific representations near the model’s input and output (Wendler et al., 2024; Alabi et al., 2024; Tang et al., 2024).Source: §1 · p. 2
  • By 10% on average across Swahili, Telugu, Bengali, and Japanese, layer swapping outperforms baselines, individual experts, and model souping on MGSM.The passage also reports that for Swahili, layer swapping exceeds the MGSM performance of models fine-tuned on mixed Swahili and math SFT data.Source: §1 · p. 2

2 RELATED WORK

Related work covers model merging, multilingual representations, and prior approaches to cross-lingual transfer. The paper positions its method as a simpler, more flexible merging approach that avoids repeated fine-tuning.

2.1 MODEL MERGING

  • Model merging combines trained checkpoints, but simple averaging can cause negative transfer; selective weight-level methods use fine-tuning deltas or information theory to address interference.Model souping averages model variants to expand the search space during hyperparameter tuning (Wortsman et al., 2022).Source: §2.1 · pp. 2–3

2.2 LLM MULTILINGUALITY

  • Research on multilingual models finds language-specific parameters in the top and bottom layers, while English-centric LLMs may map multilingual text to and from English representations at the first and last layers (Kojima et al., 2024; Wendler et al., 2024; Tang et al., 2024; Alabi et al., 2024).Earlier encoder-model work also explored cross-lingual representation alignment, while decoder-only alignment is described as difficult.Source: §2.2 · p. 3

2.3 MODEL MERGING FOR CROSS-LINGUAL TRANSFER

  • Prior cross-lingual merging combines task and language experts through sparse fine-tuning and masks, whereas this work proposes a simpler method requiring no more than one fine-tuning per expert.The cited approach creates sparse task vectors with lower parameter overlap (Ansell et al., 2022; Ansell et al., 2024).Source: §2.3 · p. 3

3 PRELIMINARY ANALYSIS

The preliminary analysis trains math and language experts, examines their SFT update patterns, and tests sparsification strategies. The observed differences in update location motivate swapping layers rather than relying on magnitude-based sparsification.

3.1 SETUP

  • Math experts are fine-tuned on English Orca-Math problems, while language experts use target-language generic instruction data without math; the study selects three checkpoints of each type.The setup uses LLAMA 3.1 8B and 30–40k labeled samples; language experts target Swahili, Bengali, Telugu, and Japanese.Source: §3.1 · pp. 3–4

3.2 PARAMETER ANALYSIS OF SFT

  • Language-expert attention updates concentrate in the first and last layers, while math-expert updates are concentrated in the second half, including feed-forward parameters.Figure 2 visualizes update magnitudes across transformer layers and parameter types; the pattern is reported as consistent across math experts and, separately, language experts.Source: §3.2 · pp. 4–5, Figure 2 · p. 4

3.3 SPARSIFYING UPDATES

  • Magnitude-based sparsification requires more than 70–80% of math-expert parameters to remain updated for equivalent performance, leaving little reduction in interference.The analysis measures update magnitudes using row-level mean absolute values of fine-tuning deltas.Source: §3.2 · pp. 4–5, §3.3 · p. 5
  • Reverting the first five and last two math-expert layers to pretrained values preserves their increased performance, prompting experiments that swap in language-expert layers there.The paper connects these layers to language-specific parameters and reports that their SFT updates are not critical for mathematical reasoning.Source: §3.3 · p. 5

4 METHODOLOGY

Layer swapping combines language-expert layers at the model’s top and bottom with math-expert middle layers, using transition zones between them. The tested configurations vary swap depth and transition-zone size, with some settings differing by target language.

4.1 LAYER SWAPPING

  • The method uses language-expert layers at the top and bottom and math-expert layers in the middle, with weighted-average transition zones between them.In the tested short fine-tuning setting, transition zones showed no statistically significant benefit over using math-expert layers, though the authors retain them and conjecture they may matter after longer or higher-learning-rate training.Source: §4.1 · p. 5
  • The implementation keeps, replaces, or averages each math-expert parameter according to its layer and transition-zone position.The algorithm assigns language-expert parameters to swapped top and bottom layers, task-expert parameters to the middle, and weighted averages to transition zones.Source: §4.1 · pp. 5–6

4.2 CONFIGURATION

  • For the 32-layer LLaMA 3.1 model, tested effective ranges were 3–5 bottom layers, 0–2 top layers, and 0–2 layers in each transition zone.Non-transformer parameters worked best as averages of the two experts rather than being taken only from the language expert.Source: §4.2 · p. 6
  • The displayed maximum and minimum setups contrast the extent of swapping, with the minimum setup using no upper layers from the language expert.Both setups are described as empirically effective.Source: Figure 3 · p. 6
  • Configurations for Telugu typically used more language-expert layers, whereas Japanese’s best configuration was near the minimum; math expert #2 performed better with fewer language layers.Results were largely equivalent across the tested configuration ranges, but these patterns were observed in the stronger configurations.Source: §4.2 · pp. 6–7

5 EMPIRICAL RESULTS

Across four languages, layer swapping consistently outperforms baselines on MGSM, including model souping. Further Swahili comparisons find lower TIES-merging performance and a higher maximum than joint fine-tuning, while results on other benchmarks are generally comparable to or better than the base model and experts.

5.1 EXPERIMENTS ON SWAHILI, TELUGU, JAPANESE, AND BENGALI

  • On 8-shot MGSM, layer swapping consistently exceeds model souping in maximum and average performance across all four languages, although its Japanese average is below that of individual math experts.Table 1 aggregates merging methods over nine language–math expert pairs, so their minimum, average, and maximum are not perfectly comparable with individual SFT runs.Source: Table 1 · p. 7, §5.1 · pp. 7–8
  • For Swahili and Telugu, the default configuration beats both individual experts across all nine pairs; in Bengali, many configurations exceed the math expert’s average and the best swapped model exceeds its best expert.Japanese has lower average performance than the individual math experts; the authors note that its experts were weak and its base and math models already performed decently on MGSM.Source: §5.1 · pp. 7–8
  • On BELEBELE, FLORES, MBPP, and MMLU, layer-swapped models are on par with or usually higher than the base model and individual experts.The authors also manually checked a sample of generated text and found that souping language and math experts before layer swapping performed on par with individual pairs.Source: §5.1 · p. 8
  • The proposed larger benefit for lower-resource languages remains a conjecture requiring evaluation beyond the four tested languages.The authors suggest that lower-data language SFT may make language improvements harder, while cross-lingual transfer from English math fine-tuning may occur more naturally in higher-resource languages.Source: §5.1 · p. 8

5.2 FURTHER EVALUATIONS ON SWAHILI

  • In Swahili, TIES-merging consistently underperforms layer swapping, while joint fine-tuning’s average MGSM score is less than one point lower.The joint fine-tuning maximum trails the best layer-swapped pairs by 4.4 percentage points.Source: §5.2 · pp. 8–9
  • Swapping in Swahili-expert layers lowers MGSM performance in English and other non-Swahili languages relative to the math expert, but the drop is relatively small.The authors report larger drops for the other merging methods on these non-target languages.Source: §5.2 · p. 9

6 DISCUSSION

Layer swapping performs well in data-scarce target languages and is explained by separating language-related layers from math-focused middle layers. Its post hoc simplicity supports inexpensive configuration testing, while the authors suggest whole-layer swaps may reduce interference better than weight-level merging.

6 DISCUSSION

  • In settings with rare or unavailable in-language task data, layer swapping creates capable models from English task data and generic target-language data, outperforming the evaluated alternatives in combining language and math capabilities.The authors describe this setting as common for lower-resource languages.Source: §6 · p. 9
  • The authors propose that top and bottom layers act as interfaces between multilingual input and English representations, while math improvements are concentrated in the middle layers.They connect the English-thinking account to recent literature (Wendler et al., 2024; Zhang et al., 2024b; Tang et al., 2024; Kojima et al., 2024).Source: §6 · p. 9
  • Layer swapping can be performed post hoc with parameter arithmetic and no further fine-tuning, enabling inexpensive tests of different layer configurations.The authors suggest it may apply at different training stages and in multi-task settings.Source: §6 · p. 9
  • The authors suggest whole-layer merges may reduce interference better than weight-level techniques, possibly because modifying individual weights disrupts transformations or granular importance measures are noisy.These are proposed explanations, not established mechanisms.Source: §6 · p. 9

7 LIMITATIONS AND FUTURE WORK

The authors identify open questions about layer swapping’s practical scope, including its dependence on model, language, task, and training conditions. They outline evaluations that could clarify these boundaries.

7 LIMITATIONS AND FUTURE WORK

  • Freezing selected layers before expert training is a proposed alternative, but it sacrifices post hoc flexibility and may require advance configuration choices or many training runs.The authors suggest it could improve performance by avoiding ad hoc merging of separately trained experts.Source: §7 · p. 10
  • It remains unknown whether layer swapping works with models whose pretraining is more balanced across languages, or whether it would break in that setting.The proposed explanation relies on English-centric models’ tendency to “think in English.”Source: §7 · p. 10
  • Larger models would require new merging configurations, and the effects of sparsity, redundancy, and cross-lingual shared parameters remain untested.Source: §7 · p. 10
  • Although lower-resource languages benefited most among those evaluated, zero-shot transfer requires some target-language capability, and the minimum level is unknown.Source: §7 · p. 10
  • This work evaluates mathematical reasoning only; other reasoning tasks may have different language-capability separation and require a different configuration and preliminary analysis.Source: §7 · p. 10
  • Because layer swapping treats transformer layers as unified blocks, the authors suggest it may also apply to parameter-efficient fine-tuning methods such as LoRA.They cite Hu et al. (2022) for LoRA and Zhang et al. (2023) for adapting other merging methods to adapters.Source: §7 · p. 10
  • The study is limited to low-data supervised fine-tuning; effectiveness at other stages and after experts diverge substantially remains uncertain.The authors also raise iterative fine-tuning and merging as a possible multi-step setup.Source: §7 · p. 10

8 CONCLUSION

The paper presents layer swapping as a way to transfer math reasoning to languages without labeled task-specific data, using an analysis of fine-tuning updates to guide layer selection. It also points readers to materials for reproducing the experts, analysis, algorithm, and experiments.

8 CONCLUSION

  • Layer swapping replaces the math expert’s top and bottom layers with language-expert layers, yielding a high-performing approach to target-language math without target-language task data.The conclusion describes the method as flexible, simple, and potentially adaptable to new models, tasks, languages, training stages, and setups.Source: §8 · p. 10

REPRODUCIBILITY

  • The paper directs reproduction of expert fine-tuning to Section 3.1 and Appendices A.1–A.2, its analysis to Appendix A.5, and the algorithm to Algorithm 1.Experiments supporting reported results are explained in Section 5.Source: REPRODUCIBILITY · p. 11

A.1 FINE-TUNING DATASETS

The cited table concerns the datasets used for supervised fine-tuning in the project.

A.1 FINE-TUNING DATASETS

  • Table 3 identifies the datasets used for supervised fine-tuning in this project.Source: Table 3 · p. 18

A.2 SUPERVISED FINE-TUNING DETAILS

The appendix identifies the expert-training hyperparameters, summarizes individual-expert results and soup-combination results, and provides larger visualizations of parameter changes during SFT.

A.2 SUPERVISED FINE-TUNING DETAILS

  • The training hyperparameter table covers runs that produced the experts; runs checkpointed every 5,000 samples and used different data-sampling seeds.The listed configurations led to the three best checkpoints for each expert category.Source: Table 4 · p. 19, §A.2 · p. 19

A.3 PERFORMANCE OF INDIVIDUAL EXPERTS

  • The individual-expert results table averages each result over three selected experts and includes a 10-language average.The languages named for that average are English, German, French, simplified Chinese, Russian, Spanish, Japanese, Bengali, Swahili, and Telugu.Source: Table 5 · p. 20

A.4 RESULTS FROM COMBINING WITH SOUPING

  • The soup-combination table reports MGSM 8-shot results for layer swapping combined with model souping of the three experts in each category.“Multilingual soup” denotes the uniform soup of all 12 language experts.Source: Table 6 · p. 21
Loading 2410.01335v2…