Source-linked AI summary
Multilingual Instruction Tuning With Just a Pinch of Multilinguality
Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, Matan Eyal
TL;DR
The paper asks how multilinguality during instruction tuning affects instruction-following across languages in a multilingual LLM. It evaluates monolingual and multilingual tuning mixtures and finds that very small multilingual additions improve both tuned and unseen languages, supporting efficient multilingual instruction tuning.
Problem
Most instruction-tuning datasets are limited to English, although globally applicable LLMs need to follow instructions across languages and collecting native data for every language is difficult.
Method
The authors compare monolingual and multilingual instruction tuning of a multilingual LLM using translated instruction-response data, automatic side-by-side evaluation, and human validation.
Results
Monolingual tuning transfers instruction-following across languages, while 40 multilingual examples improve seen and unseen languages and a few-language mixture improves unseen-language generalization.
Takeaways & Limitations
Capable multilingual instruction-following models can be developed with minimal multilingual supervision while preserving English performance under a fixed training budget.
Takeaways & Limitations
The study is limited by translated rather than natively sourced data, the number of evaluated languages, and the number of models tested.
Abstract
from arXiv · showhide
As instruction-tuned large language models (LLMs) gain global adoption, their ability to follow instructions in multiple languages becomes increasingly crucial. In this work, we investigate how multilinguality during instruction tuning of a multilingual LLM affects instruction-following across languages from the pre-training corpus. We first show that many languages transfer some instruction-following capabilities to other languages from even monolingual tuning. Furthermore, we find that only 40 multilingual examples integrated in an English tuning set substantially improve multilingual instruction-following, both in seen and unseen languages during tuning. In general, we observe that models tuned on multilingual mixtures exhibit comparable or superior performance in multiple languages compared to monolingually tuned models, despite training on 10x fewer examples in those languages. Finally, we find that diversifying the instruction tuning set with even just 2-4 languages significantly improves cross-lingual generalization. Our results suggest that building massively multilingual instruction-tuned models can be done with only a very small set of multilingual instruction-responses.
1 Introduction
The paper studies whether multilingual instruction tuning improves instruction-following across languages in a multilingual LLM. It finds substantial cross-lingual transfer from monolingual tuning, while only a small amount of multilingual data further improves seen and unseen languages.
- Monolingual instruction tuning transfers some instruction-following capability across languages, including languages seen only during pre-training.
- Tuning with English, Italian, or Spanish yields the best average multilingual performance among the monolingual settings examined.
- Replacing only 40 English training examples with multilingual examples significantly improves instruction-following in those languages and in languages absent from the tuning set.
- Using a few tuning languages improves performance on languages unseen during tuning compared with monolingual tuning using the same number of examples.
- Language similarity and language-specific pre-training data show no significant correlation with the degree of cross-lingual transfer.
2 Measuring Multilingual Instruction-Following
The study measures multilingual instruction-following with parallel translated instruction data, side-by-side model judgments, and human validation across several languages. Its metric compares each model with a target-language monolingual baseline, while the experiments use PaLM 2 models and controlled multilingual mixtures.
- Data: The dataset uses 1,000 LIMA and 3,640 OpenAssistant English instruction-response examples for tuning, plus 617 AlpacaFarm instructions for evaluation.
- Data: Google Translate creates parallel training and evaluation sets in 11 languages, controlling data size and semantics across languages.
- Evaluation: The automatic evaluation presents two responses for one instruction in alternating orders, scoring wins as 1, ties as 0.5, and losses as 0.
- Instruction-Following Score Per Language: Instruction-following for language L compares a model tuned on D with a model tuned monolingually on D_L, the full target-language training set.
- Instruction-Following Score Per Language: A score of 0% means the evaluated model loses on all target-language instructions, whereas 50% means its aggregated performance is indistinguishable from the target-language baseline.
- Model: The experiments use PaLM 2-S for instruction tuning and instruction-tuned PaLM 2-L as the side-by-side judge, with pre-training spanning hundreds of languages.
- Human Validation: Human annotation validates the evaluation approach across English, Spanish, Russian, and Hebrew, using native-speaker ratings of excellent, pass, or fail.
- Instruction-Following Score Per Language: Figure 3 varies the percentage of training data distributed uniformly across 12 languages, separating individual non-English scores, their average, and English scores.
3 How Much Multilinguality Is Needed For Multilingual Instruction Tuning?
The experiments show that monolingual tuning already transfers instruction-following across languages, while small multilingual mixtures substantially improve seen and unseen-language performance. Diversifying tuning across just a few languages improves cross-lingual generalization, with gains leveling off after roughly four languages.
- 3.1 Monolingual Instruction Tuning Yields Multilingual Abilities: Monolingual tuning produced models that responded in the instruction’s language, and instruction tuning in each language improved cross-language instruction-following over the untuned model.
- 3.2 A Few Dozen Examples Improve Multilingual Instruction-following: 40 multilingual examples—1% of the training set—substantially improved multilingual instruction-following while replacing only a small fraction of English examples.The mixture contained 40 examples evenly distributed across the non-English training languages.
- 3.2 A Few Dozen Examples Improve Multilingual Instruction-following: At P = 100, multilingual tuning performed similarly or better than monolingual tuning in 8 of 12 languages despite using 12 times fewer examples per language.
- 3.3 A Few Dozen Examples Improve Cross-lingual Generalization: With only six languages represented in tuning, the average score for unseen languages also improved sharply at P = 1, showing generalization beyond languages present in the instruction-tuning set.
- 3.4 Even a Small Number of Languages Improves Cross-Lingual Generalization: Adding languages to the tuning set improved transfer to languages seen only during pre-training, with the average score approaching 50 for three and four tuning languages before saturation.
4 Potential Factors of Transferability
The paper tests whether language similarity and pre-training exposure predict cross-lingual transfer. Neither factor shows a strong relationship with transfer in the examined settings.
- 4.1 Language Similarity: No strong signal linked Slavic language family, script, or mutual intelligibility to better cross-lingual instruction-following transfer.
- 4.1 Language Similarity: Russian and Czech transferred instruction-following abilities best, while Cyrillic usage and mutual intelligibility did not consistently predict transfer.
- 4.2 Pre-training Data: The percentage of language-specific pre-training data showed only a weak Pearson correlation with average cross-lingual instruction-following transfer.
5 Related work
Prior work established cross-lingual transfer through multilingual pre-training and task-specific fine-tuning, while this paper studies general-purpose instruction-following LLMs. It extends multilingual instruction-tuning research by examining transfer from many monolingual languages and minimal language diversity.
- Cross-lingual Transfer: Earlier studies showed that multilingual encoders and recurrent models can transfer task-specific fine-tuning across languages, with later work improving transfer through intermediate training, adapters, and multilingual benchmarks.
- Novelty: Unlike most cross-lingual transfer research focused on task-specific fine-tuning, this work examines cross-lingual generalization in general-purpose instruction-following LLMs.
- Multilingual Instruction Tuning: Prior multilingual instruction-tuning work found that English tuning can enable zero-shot transfer to new languages and that multilingual training can improve generalization to new tasks.
- Novelty: This paper studies monolingual instruction tuning across 12 languages and shows that a small amount of language diversity can improve generalization to languages absent from tuning.
6 Conclusion
The findings show that multilingual instruction tuning can transfer instruction-following across languages with minimal multilingual supervision. Even a few dozen multilingual examples and a small number of tuning languages improve performance beyond monolingual tuning.
- 6 Conclusion: A few dozen multilingual examples significantly improve instruction-following in both tuned languages and languages seen only during pre-training.The results identify minimal multilingual supervision as sufficient to enhance cross-lingual instruction-following.
- 6 Conclusion: Monolingual instruction tuning can improve instruction-following capabilities in other languages.
- 6 Conclusion: Multilingual training achieves comparable or superior performance to monolingual tuning for some languages.
- 6 Conclusion: Splitting the training set across only two languages improves generalization to new languages compared with monolingual tuning.These findings support efficient and scalable multilingual instruction tuning with minimal multilingual supervision.
7 Limitations
The evaluation is limited by translated rather than natively sourced data, a restricted language set, and experiments with only PaLM 2. These constraints limit how broadly the findings can be generalized.
- Translated data: The multilingual datasets rely on Google Translate rather than data originally sourced by native speakers.Automatic translation may introduce noise, although parallel translated data provides a controlled setup with matched content across languages.
- Number of languages: The main experiments evaluate 12 languages, with 3 additional languages used for the language-similarity experiment.The authors leave scaling evaluation to many more languages for future work.
- Number of models: The experiments use PaLM 2, so results may vary with different large language models.
B Side-By-Side Evaluation
Figure 9 presents the prompt used for the LLM judge in the side-by-side evaluation.
- B Side-By-Side Evaluation: Figure 9 shows the prompt given to the LLM judge for side-by-side evaluation.
C Training and Inference Details
The experiments use fixed training and inference settings, with checkpoint selection based on development RougeL performance. Responses are generated with nucleus sampling under specified token, temperature, and probability limits.
- C Training and Inference Details: Models are tuned for 2,000 steps with a 1e-5 learning rate, batch size 128, and dropout rate 0.05.Inputs are limited to 1,024 tokens and targets to 512 tokens.
- C Training and Inference Details: Checkpoints are selected using development RougeL, with 250 examples sampled from each training set.
- C Training and Inference Details: Inference generates up to 512 tokens using nucleus sampling with p = 0.9 and temperature 0.7.
D Judge-Human Agreement
The LLM judge shows strong agreement with human preferences across English, Spanish, Russian, and Hebrew, though agreement remains below inter-human agreement.
- Human evaluation compares the monolingually tuned model for each evaluation language against a randomly selected model.Native speakers assess 50 instructions per language, with two annotators choosing between responses.
- 79.5% agreement is highest in English, while the other languages are a few points lower.Overall judge–human agreement is strong across all four evaluated languages.
- Judge agreement trails inter-human agreement by 2.5–7 points across languages.This indicates strong but imperfect alignment with human judgments.
E Response Language
The evaluation checks whether multilingual instruction-tuned models answer in the prompt language, a behavior that is generally observed across sampled responses.
- 239 of 240 sampled responses use the same language as the prompt.The samples cover responses from models tuned monolingually in other languages.
- The models’ language matching differs from commonly used LLaMA-based models, which more often respond in another language.The authors hypothesize that PaLM 2’s stronger multilingual pre-training emphasis contributes to this difference.
- Figure 10 compares monolingually tuned models with the untuned pre-trained model across evaluation languages.Rows represent tuning languages, columns represent evaluation languages, and scores use discounted-ties weighted averages from side-by-side comparisons.