PaperlayerView exact revision on arXiv

Source-linked AI summary

Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier

John Dang, Shivalika Singh, Daniel D'souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztajn, Yannis Flet-Berliac, Acyr Locatelli, Hangyu Lin, Dwarak Talupuru, Bharat Venkitesh, David Cairuz, Bowen Yang, Tim Chung, Wei-Yin Ko, Sylvie Shang Shi, Amir Shukayev, Sammie Bae, Aleksandra Piktus, Roman Castagné, Felipe Cruz-Salinas, Eddie Kim, Lucas Crawhall-Stein, Adrien Morisot, Sudip Roy, Phil Blunsom, Ivan Zhang, Aidan Gomez, Nick Frosst, Marzieh Fadaee, Beyza Ermis, Ahmet Üstün, Sara Hooker

arXiv:2412.04261v1cs.CL

TL;DR

Models can perform better on trained languages while showing biases against unseen languages, safety and security flaws, and higher token-related latency and cost. Aya Expanse combines multilingual data arbitrage, preference optimization and safety tuning, and model merging in 8B and 32B multilingual models. Across 23-language head-to-head evaluations, Aya Expanse 8B and 32B outperform similar-sized leading open-weight models by up to 76.6% win-rates, including Llama 3.1 70B.

  • Problem

    Models can perform better on trained languages while showing biases against unseen languages, safety and security flaws, and higher token-related latency and cost.

    Source: §1 · p. 2

  • Method

    Aya Expanse combines multilingual data arbitrage, preference optimization and safety tuning, and model merging in 8B and 32B multilingual models.

    Source: §1 · p. 2, §6 · p. 13

  • Results

    Across 23-language head-to-head evaluations, Aya Expanse 8B and 32B outperform similar-sized leading open-weight models by up to 76.6% win-rates, including Llama 3.1 70B.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The release makes Aya Expanse 8B and 32B open-weights and the translated m-ArenaHard dataset available to support multilingual AI progress and inclusive language technologies.

    Source: §1 · p. 3

    GPT-4o-mini as judge biased win-rate results toward Aya Expanse by up to 10%, so the authors report more conservative GPT-4o-judged results.

    Source: §5.1 · p. 10

Abstract

from arXiv · show

We introduce the Aya Expanse model family, a new generation of 8B and 32B parameter multilingual language models, aiming to address the critical challenge of developing highly performant multilingual models that match or surpass the capabilities of monolingual models. By leveraging several years of research at Cohere For AI and Cohere, including advancements in data arbitrage, multilingual preference training, and model merging, Aya Expanse sets a new state-of-the-art in multilingual performance. Our evaluations on the Arena-Hard-Auto dataset, translated into 23 languages, demonstrate that Aya Expanse 8B and 32B outperform leading open-weight models in their respective parameter classes, including Gemma 2, Qwen 2.5, and Llama 3.1, achieving up to a 76.6% win-rate. Notably, Aya Expanse 32B outperforms Llama 3.1 70B, a model with twice as many parameters, achieving a 54.0% win-rate. In this short technical report, we present extended evaluation results for the Aya Expanse model family and release their open-weights, together with a new multilingual evaluation dataset m-ArenaHard.

1 Introduction

Aya Expanse introduces open-weight 8B and 32B multilingual instruction-tuned models supporting 23 languages, addressing gaps in multilingual performance. Across 23-language head-to-head evaluations, the models outperform leading open-weight comparators, and the paper releases the models and m-ArenaHard.

1 Introduction

  • Aya Expanse is an open-weight family of 8B and 32B multilingual instruction-tuned models supporting 23 languages.The models build upon Cohere’s Command series.Source: §1 · p. 2
  • Multilingual models can lag on languages they were not trained on, with associated biases, safety and security flaws, and higher token-related latency and cost.The introduction contrasts this with stronger performance on languages represented in training.Source: §1 · p. 2
  • The evaluation combines multilingual preference tests on m-ArenaHard and Dolly with multilingual academic benchmarks.Arena-Hard-Auto is translated into 23 languages for the main evaluation dataset.Source: §1 · p. 2
  • Up to 76.6% win-rates in 23-language head-to-head evaluations show Aya Expanse outperforming similar-sized leading open-weight models, including Llama 3.1 70B.The models also improve over Aya 23 by up to 16% in language understanding and 1.8x in mathematical reasoning.Source: §1 · p. 3
  • The authors release Aya Expanse 8B and 32B open-weights alongside m-ArenaHard to support progress in multilingual AI and more inclusive language technologies.Source: §1 · p. 3

2 Post-Training Recipe

Aya Expanse’s post-training recipe combines multilingual data arbitrage, iterative multilingual preference training, and model merging. The approach draws on diverse teacher models and preference data, while merging checkpoints trained across language families.

2 Post-Training Recipe

  • The post-training recipe draws on prior research for its core performance drivers, which the report summarizes before referring readers to the original manuscripts.Source: §2 · p. 3

2.1 Synthetic Data Generation through Data Arbitrage

  • Reward-Based Routing scores completions from a pool of models with an internal reward model and selects the highest-scoring completion for each prompt.The model pool is trained for language groups, and the SFT-stage Aya Expanse 8B model improved win-rate against Gemma 2 9B by over 9.1% versus Aya 23 8B.Source: §2.1 · p. 3

2.2 Iterative Multilingual Preference Training

  • Preference training uses multilingual preference pairs to address low-quality multilingual datasets and the risk that optimizing some languages regresses others.The reported approach contrasts strong in-language completions with weaker translated completions; offline preference training followed by online iterative DPO added 7.1% win-rate against Gemma 2 9B for Aya Expanse 8B.Source: §2.2 · p. 4

2.3 Model Merging

  • Checkpoint merging addresses the effort of choosing data mixtures by combining models trained on different language-family clusters while retaining shared languages for cross-lingual transfer.Weighted averaging was the most consistent of the tested methods, and merging gains at 32B were up to 3x those at 8B.Source: §2.3 · pp. 4–5

3 Model Architecture and Experimental Details

Aya Expanse is based on the Cohere Command series, whose pretraining mixture includes 23 languages. The 8B and 32B base models differ in maximum context length, while sharing several architectural components.

3 Model Architecture and Experimental Details

  • The Aya Expanse base models derive from the Cohere Command series, pretrained on a mixture including 23 languages.Source: §3 · p. 5
  • The 8B model has an 8K maximum context length, while the 32B model has a 128K maximum context length.Source: §3 · p. 5
  • Both model sizes use SwiGLU activations, RoPE positional embeddings, grouped-query attention, and the Aya 23 chat template for instruction fine-tuning.Source: §3 · p. 5

4 Multilingual Evaluation

The evaluation follows prior multilingual benchmark methodology and adds m-ArenaHard for open-ended generation. It covers preference, discriminative, language-understanding, academic, mathematical-reasoning, and translation tasks, and compares Aya Expanse with leading open-weight models.

4 Multilingual Evaluation

  • The evaluation suite covers eight datasets and six task categories, generally restricting languages to Aya Expanse’s 23 supported languages, with specified tasks using all available languages.The suite adds multilingual ArenaHard, translated from the original Arena-Hard-Auto dataset.Source: Table 1 · p. 6
  • For preference evaluation, the authors use GPT-4o as judge to compare models’ generations directly across all 23 languages covered by Aya Expanse.GPT-4o-mini results differed significantly and were biased toward Aya Expanse, so the authors chose GPT-4o for final results.Source: §4 · p. 6
  • The academic evaluation includes zero-shot XWinograd, XCOPA, and XStoryCloze tasks, plus Global-MMLU in a five-shot setting across Aya Expanse’s 23 supported languages.Global-MMLU is an improved MMLU version translated into 42 languages, with translations refined by professional and community annotators.Source: §4 · p. 7

4.1 Model Comparisons

  • The model comparisons use instruct fine-tuned competitors and average Dolly pairwise win-rates across 23 languages for the 8B and 32B classes.The 32B comparisons also include larger Mixtral 8x22B and Llama 3.1 70B models.Source: §4.1 · p. 7, Figure 2 · p. 8
  • Figure 3 compares post-training steps with Aya 23 8B on m-ArenaHard and reports academic-benchmark improvements, especially on MGSM, Global-MMLU, and INCLUDE.Source: Figure 3 · p. 9

5 Results

Across open-ended evaluations, Aya Expanse models outperform the compared models in their parameter classes, and they show strong results across multilingual academic benchmarks. The results include language-by-language gains, improvements over Aya 23, and leading machine-translation performance.

5.1 Pairwise Model Comparisons for Open-ended Generations

  • Across m-ArenaHard and Dolly, Aya Expanse 8B and 32B outperform all compared models in their parameter classes.The comparisons cover both evaluation sets.Source: §5.1 · p. 9
  • On m-ArenaHard, Aya Expanse 8B reaches 70.6% win rate vs Llama 3.1 8B, while 32B reaches 76.6% vs Mixtral 8x22B and 54.0% vs Llama 3.1 70B.The 32B model also records 51.8% vs Gemma 2 27B and 53.6% vs Qwen 2.5 32B.Source: §5.1 · p. 9
  • On Dolly, Aya Expanse 8B reaches 83.9% win rate vs Llama 3.1 8B, and 32B reaches 89.9% vs Mixtral 8x22B.The authors relate the higher win rates than on m-ArenaHard to less difficult prompts and overlap between evaluation instructions and the Dolly training distribution.Source: §5.1 · p. 9
  • Aya Expanse models perform well individually in all 23 languages; 8B outperforms Gemma 2 9B across the eight selected languages, including English.The authors describe this as improving performance across languages without sacrificing performance on high-resource English.Source: §5.1 · p. 10
  • Against Gemma 2 9B, Aya Expanse 8B reaches 60.4% win rate after unified post-training, a 20.3% increase over Aya 23 8B.Multilingual-arbitrage SFT gives a 48.2% vs 39.1% win rate; model merging adds 5.1% and offline plus online preference training adds 7.1%.Source: §5.1 · p. 10

5.2 Multilingual Language Understanding

  • On academic multilingual benchmarks, Aya Expanse 8B reaches 70.3 accuracy and 32B reaches 72.6, while their Global-MMLU accuracies are 53.7 and 66.9, respectively.On discriminative understanding, Gemma 2 9B leads the 8B class at 73.5 accuracy; on Global-MMLU, Aya Expanse improves over Aya 23 by 5.6 and 9.0 accuracy at 8B and 32B.Source: §5.2 · pp. 10–11

5.3 Multilingual Mathematical Reasoning

  • On this task, Aya Expanse leads its parameter class: 8B achieves 67.0 accuracy and 32B achieves 73.4; only Llama 3.1 70B scores higher, at 78.0, with over twice as many parameters.Source: §5.3 · p. 12

5.4 Machine Translation

  • On FLORES machine translation, Aya Expanse leads both parameter classes: 8B scores 57.2 chrF++ and 93.2 xCOMET, while 32B scores 58.8 chrF++ and 93.5 xCOMET.At 8B, the second-best model, Gemma 2 9B, scores 57.0 chrF++ and 91.8 xCOMET; at 32B, Aya 23 35B is second at 57.8 and 93.1.Source: §5.4 · pp. 12–13

6 Conclusion

The conclusion presents Aya Expanse as a response to language technologies’ English focus, combining three multilingual methods. It reports strong multilingual and cross-task performance and makes model weights public to support future research.

6 Conclusion

  • Aya Expanse models excel in multiple languages while sustaining competitive performance across diverse tasks.The conclusion says these innovations establish a new frontier in multilingual performance.Source: §6 · p. 13
  • Multilingual data arbitrage samples diverse teacher models to generate high-quality synthetic multilingual datasets.Source: §6 · p. 13
  • Multilingual preference optimization and safety tuning [Aakanksha et al., 2024a] enhance alignment with human preferences across languages.Source: §6 · p. 13
  • Model merging leverages cross-lingual transfer and language-family diversity to optimize performance.Source: §6 · p. 13
  • The authors make model weights publicly available to contribute to future research.Source: §6 · p. 13

Related papers

Loading 2412.04261v1…