Source-linked AI summary
Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases
Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, Xiangang Li
TL;DR
The impact of instruction-data scale on model performance, especially in real-world use cases, has not been well explored. The paper studies this question using Chinese instruction data and an evaluation set covering 12 common online use cases. Performance patterns differ by task: some improve as data grows, while math and code remain poor without further gains.
Problem
The impact of instruction-data scale on model performance, particularly in real-world use cases, has not been well explored.
Source: §1 · p. 1Method
The authors generate Chinese instruction data from translated and adapted seed examples and evaluate models on test instructions covering 12 common online use cases.
Source: §3.1 · p. 3Results
Performance improves with more data for extraction, classification, closed QA, and summarization, while it remains poor on math, code, and chain-of-thought tasks without further gains.
Source: §5 · p. 5Takeaways & Limitations
The authors identify data quality, base-model scale, and training methods specialized for hard tasks as directions for further research.
Source: §1 · p. 2
Abstract
from arXiv · showhide
The success of ChatGPT has recently attracted numerous efforts to replicate it, with instruction-tuning strategies being a key factor in achieving remarkable results. Instruction-tuning not only significantly enhances the model's performance and generalization but also makes the model's generated results more consistent with human speech patterns. However current research rarely studies the impact of different amounts of instruction data on model performance, especially in the real-world use cases. In this paper we explore the performance of large language models based on instruction tuning across different scales of instruction data. An evaluation dataset consisting of 12 major online use cases is constructed in the experiment. With Bloomz-7B1-mt as the base model, the results show that 1) merely increasing the amount of instruction data leads to continuous improvement in tasks such as open-ended generation, 2) in tasks such as math and code, the model performance curve remains quite flat while increasing data size. We further analyze the possible causes of these phenomena and propose potential future research directions such as effectively selecting high-quality training data, scaling base models and training methods specialized for hard tasks. We will release our training and evaluation datasets, as well as model checkpoints.
1 Introduction
The paper studies how instruction-data scale affects model performance in real-world use cases, an impact earlier research had not well explored. Its results show different scaling patterns across tasks, motivating further work on hard tasks and data quality.
1 Introduction
- Instruction-tuning is presented as a key factor in replicating ChatGPT, alongside diverse, high-quality instruction datasets.The paper frames ChatGPT’s success as motivating efforts to replicate it.Source: §1 · p. 1
- The authors identify limited study of instruction-data scale in real-world online use cases and construct a Chinese instruction dataset to investigate it.They also note that many prior evaluations focused on conventional NLP tasks and English datasets.Source: §1 · p. 1
- Hundreds of thousands of examples can yield good performance on translation, rewriting, generation, and brainstorming, while extraction, classification, closed QA, and summarization continue improving with more data.The potential for improvement on the latter tasks may be limited.Source: §1 · pp. 1–2
- Increasing data size no longer improves the model’s poor performance on math and code, motivating research on data quality, model scale, and task-specialized training.The paper also identifies chain-of-thought instructions among the hard tasks in its broader results.Source: §1 · p. 2
- The authors plan to release their training and evaluation data and model checkpoints.They describe their experimental conclusions as preliminary directions for future work.Source: §1 · p. 2
2 Related Work
Related work traces progress from generative language models and instruction tuning to evaluations of model capabilities. It identifies a gap in evaluating instruction-data scale on real-world use cases, particularly for Chinese users.
2.1 Large language models
- GPT models’ ability to follow instructions has been enhanced by RLHF, and ChatGPT is described as a natural-language assistant for conversation and code-error detection.The section situates these developments within advances in transformer-based language models.Source: §2.1 · p. 2
2.2 Instruction tuning
- Instruction tuning formats tasks as natural language for multi-task training, but models still struggled with general human instructions and the effects of varying data amounts and task types remained underexplored.The section describes RLHF as a turning point in producing good responses to varied human instructions.Source: §2.2 · p. 2
2.3 Evaluation of LLMs
- Prior evaluations largely assessed existing models on conventional tasks rather than varying instruction-data scales, while the proposed dataset targets Chinese real-world usage.The section also notes evaluations spanning many models and task suites, including BIG-Bench Hard.Source: §2.3 · p. 2
3 Method
The method uses ChatGPT to generate Chinese instruction data from translated and culturally adapted seed examples, then builds test instructions spanning common online use cases. ChatGPT also evaluates model responses.
3 Method
- The method generates high-quality instruction data and diversified test instructions, with ChatGPT used to evaluate model responses.The authors refer to their earlier work for the evaluation prompts.Source: §3 · p. 2
3.1 Generate training data
- Translated and culturally adapted seed examples are used as in-context prompts for ChatGPT to generate additional Chinese training samples.The seed data are adapted from Taori et al. (2023), including changes to examples heavily tied to Western culture and background knowledge.Source: §3.1 · p. 3
3.2 Generate evaluation data
- The test set draws from ChatGPT-generated data, with annotators correcting responses to create golden answers across 12 common online-use instruction types.The authors plan to expand the evaluation dataset, noting that more data can make evaluation results more reliable.Source: §3.2 · p. 3
4 Experiments
Across the evaluated instruction types, the effect of adding training data varies: some tasks perform well at relatively small scales, others continue improving, and math, code, and COT remain difficult. The experiments use Bloomz-7B1-mt models at several data scales and score their responses with ChatGPT.
4.1 Instruction-following Models
- The study trains BELLE models from Bloomz-7B1-mt using 200,000, 600,000, 1 million, or 2 million instruction examples.The experiments vary instruction-data scale and leave model scale for future work.Source: §4.1 · p. 3, Table 2 · p. 3
4.2 Metrics
- ChatGPT rates each response from 0 to 1; scores are averaged across three evaluations and then across test examples for each instruction type.The evaluations used the gpt-3.5-turbo API on March 25, 2023.Source: §4.2 · p. 3
4.3 Analysis
- Open QA improves continuously, generation gains from 200,000 to 1 million examples before plateauing, rewrite performs well at 600,000, and brainstorming peaks at 200,000.For brainstorming, diverse responses and unclear quality standards may lead ChatGPT to give higher scores.Source: §4.3 · p. 4
- Across 12 instruction types, translation, rewrite, generation, and brainstorming perform well with 2 million or fewer examples, common NLP tasks keep improving, while math, code, and COT remain poor.The analysis notes that further gains on the improving tasks may be limited and that hard tasks need further exploration.Source: §4.3 · p. 4, Figure 1 · p. 4
5 Conclusion and Future work
The paper evaluates instruction-data scale and finds that its effects differ by task: some tasks perform well with hundreds of thousands of examples, while others continue improving or remain poor. It identifies data quality, base-model scale, and training strategies as directions for future work.
5 Conclusion and Future work
- Hundreds of thousands of examples can yield good results for translation, rewrite, generation, and brainstorming, while extraction, classification, closed QA, and summarization continue improving with more data; math, code, and COT remain poor.The authors note that gains on the improving tasks may be limited.Source: §5 · p. 5
- Future work will explore limits to data scaling, improve and select higher-quality training data, and evaluate the effects of base-model parameters and abilities.The paper particularly identifies math, code, and COT training data as low quality.Source: §5 · p. 5
6 Appendix A
Appendix A describes the use of ChatGPT as an evaluator and provides prompts designed for that role.
6 Appendix A
- The authors treat ChatGPT as an annotator, citing prior work that found its ranking preferences consistent with human preferences to a certain extent.The appendix includes prompts for evaluating different instruction types.Source: §6.1 · p. 7
- The prompts vary by instruction type, connecting the evaluator to the responses being assessed.Source: §6.1 · p. 7
- Table 3 lists prompts designed to have ChatGPT evaluate instruction-following models.Source: Table 3 · p. 8