PaperlayerView exact revision on arXiv

Source-linked AI summary

Qwen2.5 Technical Report

Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zihan Qiu

arXiv:2412.15115v2cs.CL

TL;DR

Qwen2.5 targets limitations in earlier Qwen models while supporting diverse LLM use cases. It scales training data and post-training, achieving broad benchmark strength, with Qwen2.5-72B-Instruct competitive with Llama-3-405B-Instruct.

  • Problem

    Qwen2.5 addresses limitations in prior Qwen models while supporting diverse LLM use cases across sizes, capabilities, and deployment settings.

  • Method

    Qwen2.5 scales pre-training to 18 trillion tokens and applies supervised fine-tuning, preference optimization, and multistage reinforcement learning.

  • Results

    Qwen2.5-72B-Instruct surpasses Llama-3.1-405B-Instruct on several critical benchmarks, while Qwen2.5-72B achieves comparable results using one-fifth the parameters.

  • Takeaways & Limitations

    Qwen2.5 provides a broad family of open-weight and proprietary models with competitive performance across language, reasoning, mathematics, coding, and alignment tasks.

  • Takeaways & Limitations

    Current reward-model evaluation benchmarks do not accurately predict the performance of reinforcement-learning models trained under their guidance.

Abstract

from arXiv · show

In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the high-quality pre-training datasets from the previous 7 trillion tokens to 18 trillion tokens. This provides a strong foundation for common sense, expert knowledge, and reasoning capabilities. In terms of post-training, we implement intricate supervised finetuning with over 1 million samples, as well as multistage reinforcement learning. Post-training techniques enhance human preference, and notably improve long text generation, structural data analysis, and instruction following. To handle diverse and varied use cases effectively, we present Qwen2.5 LLM series in rich sizes. Open-weight offerings include base and instruction-tuned models, with quantized versions available. In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus, both available from Alibaba Cloud Model Studio. Qwen2.5 has demonstrated top-tier performance on a wide range of benchmarks evaluating language understanding, reasoning, mathematics, coding, human preference alignment, etc. Specifically, the open-weight flagship Qwen2.5-72B-Instruct outperforms a number of open and proprietary models and demonstrates competitive performance to the state-of-the-art open-weight model, Llama-3-405B-Instruct, which is around 5 times larger. Qwen2.5-Turbo and Qwen2.5-Plus offer superior cost-effectiveness while performing competitively against GPT-4o-mini and GPT-4o respectively. Additionally, as the foundation, Qwen2.5 models have been instrumental in training specialized models such as Qwen2.5-Math, Qwen2.5-Coder, QwQ, and multimodal models.

1 Introduction

Qwen2.5 is introduced as a continuously improved LLM series spanning multiple open-weight sizes and quantized variants. Its key advances include expanded training data, broader model availability, and improved usability through longer generation, structured-data support, tool use, and extended context.

  • Better in Size: Qwen2.5 offers pre-trained and instruction-tuned open-weight models in 7 sizes, from 0.5B to 72B, with bfloat16 and quantized variants.The listed sizes are 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B.
  • Introduction: Qwen2.5 is presented as the latest result of the team’s continued effort to create better large language models.The introduction frames recent progress in LLMs around scaling models and data, followed by high-quality supervised fine-tuning.
  • Better in Data: 18 trillion tokens of pre-training data, up from 7 trillion, and 1 million post-training examples expand Qwen2.5’s training foundation.The data focuses on knowledge, coding, and mathematics, while post-training spans supervised finetuning, direct preference optimization, and group relative policy optimization.
  • Better in Use: Generation length increases from 2K to 8K tokens, alongside better support for structured input and output and easier tool use.Examples of structured data include tables and JSON.
  • Better in Use: Qwen2.5-Turbo supports a context length of up to 1 million tokens, while Qwen2.5-Turbo and Qwen2.5-Plus balance accuracy, latency, and cost.The series also reintroduces 3B, 14B, and 32B models for more cost-effective use in resource-limited scenarios.

2 Architecture & Tokenizer

Qwen2.5 comprises dense open-source models and MoE models for API service, using a Transformer-based decoder for dense models and extending it with routed expert layers for MoE variants. Its tokenizer uses byte-level byte-pair encoding with 151,643 regular tokens and expands control tokens from 3 to 22.

  • Model Families: Qwen2.5 offers dense open-source models from 0.5B to 72B parameters and MoE models Qwen2.5-Turbo and Qwen2.5-Plus for API service.The dense lineup includes 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B models.
  • Dense Architecture: Dense models retain Qwen2’s Transformer-based decoder architecture with GQA, SwiGLU, RoPE, QKV bias, and pre-normalized RMSNorm.These components support KV-cache utilization, nonlinear activation, position encoding, attention, and stable training.
  • MoE Architecture: MoE models replace standard FFN layers with specialized layers containing multiple experts and routing that dispatches tokens to the top-K experts.The design also implements fine-grained expert segmentation and shared experts routing.
  • Tokenizer: 151,643 regular tokens are used in Qwen’s BBPE tokenizer, while control tokens expand from 3 to 22 across Qwen2.5 models.The expansion adds two tool-functionality tokens and allocates the remainder to other model capabilities, establishing a unified vocabulary.

3 Pre-training

Qwen2.5 strengthens pre-training through higher-quality, strategically composed data expanded from 7 trillion to 18 trillion tokens, scaling-law-guided hyperparameter optimization, and specialized long-context training. Its long-sequence methods extend inference capacity to up to 1 million tokens for Qwen2.5-Turbo and 131,072 tokens for other models.

  • Long-context pre-training: A two-phase approach extends context length from 4,096 to 32,768 tokens for all model variants except Qwen2.5-Turbo, while increasing RoPE base frequency from 10,000 to 1,000,000.The extension occurs during the final pre-training stage and uses the ABF technique.
  • Data quality and composition: 18 trillion tokens expand Qwen2.5’s pre-training dataset from Qwen2’s 7 trillion through improved filtering, domain-specific data, synthetic data, and data-mixture strategies.Qwen2-Instruct models filter and classify data, specialized math and code datasets strengthen those capabilities, and reward models improve synthetic-data quality.
  • Hyperparameter scaling: Scaling laws identify optimal batch sizes and learning rates across dense and MoE architectures as model size and pre-training data vary.Experiments cover dense models with 44M–14B parameters, MoE models with 44M–1B activated parameters, and datasets from 0.8B–600B tokens.
  • Long-context pre-training: Qwen2.5-Turbo progressively expands context through 32,768, 65,536, 131,072, and 262,144 tokens, using 40% maximum-length and 60% shorter sequences at each stage.Its RoPE base frequency is 10,000,000.
  • Long-context inference: YARN and Dual Chunk Attention provide a four-fold sequence-capacity increase, enabling Qwen2.5-Turbo to handle up to 1 million tokens and other models up to 131,072 tokens.These strategies reduce perplexity while improving long-sequence modeling during inference.

4 Post-training

Qwen2.5 advances post-training through broader supervised fine-tuning and a two-stage reinforcement-learning process. Its SFT improvements target long-context generation, mathematics, coding, structured data, reasoning, multilingual transfer, instruction following, and robustness, while specialized procedures improve data and reward quality.

  • Post-training design: Qwen2.5 introduces two post-training advances over Qwen2: expanded supervised fine-tuning coverage and two-stage reinforcement learning.The SFT data spans millions of high-quality examples, while RL is divided into Offline RL and Online RL.
  • Reinforcement learning: Offline RL develops difficult-to-evaluate reasoning, factuality, and instruction-following capabilities, whereas Online RL uses reward-model sensitivity to output-quality nuances.Offline RL pre-prepares reliable, learnable signals for domains including mathematics, coding, instruction following, and logical reasoning; Online RL uses labeling criteria including truthfulness and helpfulness.
  • Supervised fine-tuning: 8,192 tokens is Qwen2.5’s maximum output context length, compared with typical post-training responses under 2,000 tokens.Long-response datasets use back-translation, output-length constraints, and Qwen2 filtering to construct higher-quality paired data.
  • Supervised fine-tuning: Qwen2.5 strengthens mathematics, coding, and instruction following with chain-of-thought data, multilingual code-generation agents, and execution-feedback validation.Mathematical data uses rejection sampling, reward modeling, and annotated answers; coding data covers nearly 40 programming languages and automated testing; instruction data uses generated verification code and unit tests.
  • Supervised fine-tuning: 70,000 new logical-reasoning queries and structured-data reasoning chains broaden capabilities across inference, tabular question-answering, verification, correction, and structural understanding.Cross-lingual transfer uses translation and semantic-alignment checks, while hundreds of system prompts improve robustness and reduce performance variance.

5 Evaluation

Qwen2.5 is evaluated comprehensively across base and instruction-tuned models using broad automatic benchmark suites spanning language, reasoning, mathematics, coding, multilinguality, and long-context processing. Across model sizes, Qwen2.5 generally surpasses comparable competitors, while Qwen2.5-72B-Instruct and Qwen2.5-Turbo show particularly strong performance on benchmark and ultra-long-context evaluations.

  • Evaluation setup: The evaluation suite combines common open benchmarks with in-house skill-oriented datasets and is designed to be primarily automatic.Base and instruction-tuned models are evaluated separately with minimal human interaction.
  • Base models: Qwen2.5-14B scores 79.7 on MMLU and 78.2 on BBH, while Qwen2.5-7B surpasses predecessors and counterparts across numerous benchmarks.Qwen2.5-0.5B, 1.5B, and 3B also maintain strong performance across nearly all benchmarks, with Qwen2.5-0.5B outperforming Gemma2-2.6B on various mathematics and coding tasks.
  • Instruction-tuned models: Qwen2.5-72B-Instruct surpasses Llama-3.1-405B-Instruct on MMLU-redux, MATH, MBPP, MultiPL-E, LiveCodeBench, Arena-Hard, and MTBench, while Qwen2.5-Plus wins 9 out of 13 benchmarks against it.Qwen2.5-32B-Instruct shows superior performance across most tasks among similarly sized models, and Qwen2.5-7B-Instruct scores 75.5 on MATH and 84.8 on HumanEval.
  • Evaluation limitations: Reward-model benchmark scores do not necessarily predict the performance of reinforcement-learning models trained under those reward models.The authors identify this mismatch as a limitation requiring more predictive reward-model evaluation methods.
  • Long-context evaluation: Qwen2.5-72B-Instruct performs strongest across all evaluated context lengths, while Qwen2.5-Turbo achieves 100% accuracy on the 1M-token passkey retrieval task.Length extrapolation uses DCA + YARN, and sparse attention based on Minference reduces attention computation for 1M-token sequences.

6 Conclusion

Qwen2.5 advances LLMs through 18-trillion-token pre-training and sophisticated post-training, improving alignment, long-text generation, structural data analysis, and instruction following. Future work will broaden training data, refine base and instruction-tuned models, and develop unified multimodal systems.

  • Conclusion: 18 trillion tokens and multi-stage post-training strengthen Qwen2.5’s alignment, long-text generation, structural data analysis, and instruction following.The post-training approach includes supervised fine-tuning and multi-stage reinforcement learning.
  • Conclusion: Qwen2.5 is available in various configurations, including open-weight models ranging from 0.5B to 72B.
  • Future Work: Future work will iteratively refine base and instruction-tuned LLMs using broader, more diverse, higher-quality data.
  • Future Work: The authors will develop multimodal models toward a unified framework for end-to-end processing across textual, visual, and auditory domains.

7 Authors

The report credits a group of core contributors and a broader set of additional contributors. The listed contributors include researchers whose names span the supplied author passages.

  • Core Contributors: Core contributors include An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, and Dayiheng Liu.
  • Core Contributors: The core-contributor list continues with Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu.
  • Contributors: Additional contributors include Biao Sun, Bin Luo, Bin Zhang, Binghai Wang, Chaojie Yang, Chang Si, Cheng Chen, Chengpeng Li, and Chujie Zheng.
Loading 2412.15115v2…