Source-linked AI summary
1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training
Han Zhao, Haotian Wang, Yiping Peng, Sitong Zhao, Xiaoyu Tian, Shuaiting Chen, Yunjie Ji, Xiangang Li
TL;DR
The paper addresses the limited scale and uneven support of existing open-source distilled reasoning datasets. It constructs and verifies a diverse 1.4-million-entry dataset, then uses simple SFT to train models that outperform DeepSeek-R1-distilled comparators across four benchmarks. The dataset is released as a research resource, subject to unresolved verification and safety limitations.
Problem
Existing open-source distilled reasoning datasets are generally smaller than DeepSeek’s 800,000-sample resource, while few match its performance on corresponding base models.
Method
The authors collect diverse open-source reasoning problems, apply semantic deduplication and cleaning, verify responses, and use the resulting data for supervised fine-tuning.
Results
AM-Distill-Qwen-32B and AM-Distill-Qwen-72B outperform their respective DeepSeek-R1-distilled comparators across AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench.
Takeaways & Limitations
The released dataset provides a large-scale resource for training reasoning-oriented large language models.
Takeaways & Limitations
The responses have not been rigorously verified, harmful content has not been thoroughly filtered, and some source information may be inaccurate.
Abstract
from arXiv · showhide
The AM-DeepSeek-R1-Distilled is a large-scale dataset with thinking traces for general reasoning tasks, composed of high-quality and challenging reasoning problems. These problems are collected from a multitude of open-source datasets, subjected to semantic deduplication and meticulous cleaning to eliminate test set contamination. All responses within the dataset are distilled from reasoning models (predominantly DeepSeek-R1) and have undergone rigorous verification procedures. Mathematical problems are validated by checking against reference answers, code problems are verified using test cases, and other tasks are evaluated with the aid of a reward model. The AM-Distill-Qwen-32B model, which was trained through only simple Supervised Fine-Tuning (SFT) using this batch of data, outperformed the DeepSeek-R1-Distill-Qwen-32B model on four benchmarks: AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench. Additionally, the AM-Distill-Qwen-72B model surpassed the DeepSeek-R1-Distill-Llama-70B model on all benchmarks as well. We are releasing these 1.4 million problems and their corresponding responses to the research community with the objective of fostering the development of powerful reasoning-oriented Large Language Models (LLMs). The dataset was published in \href{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M}{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M}.
1 Introduction
The paper introduces a 1.4-million-entry reasoning dataset built to address the smaller scale of prior open-source distilled datasets. It combines diverse, cleaned, deduplicated data with verified responses and detailed annotations to support reasoning-model training.
- Dataset contribution: 1.4 million high-quality data entries form the AM-DeepSeek-R1-Distilled dataset, including 0.5 million open-source entries and 0.9 million responses distilled from DeepSeek-R1.The dataset is intended to support long chain-of-thought training.
- Dataset contribution: Semantic deduplication, cleaning, and response verification target data quality and reliability across mathematical, coding, and other tasks.Mathematical answers are checked against references, code is tested, and other responses are evaluated with a reward model.
- Dataset contribution: The dataset spans math, code, science, cross-domain, and comprehensive reasoning tasks, with instruction difficulty and category labels used for deduplication.This design is intended to exercise reasoning and generalization across varied problem types.
- Dataset contribution: Each entry uses a unified format containing interactions, reasoning processes, answers, references, test cases, and metadata for data processing and model training.The standardized annotations are described as making the dataset easier to use and understand.
- Dataset contribution: The release is presented as a resource for reasoning-oriented large language model research and further field development.The paper specifically frames the dataset as support for research and innovation in reasoning-oriented LLMs.
2 Approach
The approach constructs the dataset around diversity, complexity, and accuracy through collection, distillation, rejection sampling, categorization, difficulty filtering, deduplication, and response verification.
- Pipeline design: The data pipeline has three stages: raw data collection, distilling, and rejection sampling, organized around diversity, complexity, and accuracy.These criteria define the pipeline’s central data-selection objectives.
- Data selection: Collected data are divided into math, code, scienceQA, and general chat, with additional labels such as creative writing and instruction following.Qwen2.5-7B-Instruct is used to label data for matching and diversity.
- Data selection: Difficulty scoring retains challenging examples while downsampling easy and medium examples to emphasize longer reasoning chains without discarding diversity.The filtering is motivated by training long-chain-of-thought models.
- Data cleaning: Embedding-based semantic deduplication retains one representative from highly similar entries to preserve uniqueness and diversity.Priority strategies determine which representative entry is retained.
- Response processing: Existing responses are retained after reference-answer or test-case checks, while missing reasoning chains are generated with DeepSeek-R1 and filtered through rule-based and model-based evaluation.Additional checks cover response quality dimensions, format conformity, repetition, and supplementary metadata.
3 Experiment
The experiments evaluate AM-distilled models across four reasoning benchmarks using standardized sampling and accuracy procedures. Both the 32B and 72B models outperform their DeepSeek-R1-distilled comparators across the reported benchmarks.
- Evaluation setup: Four benchmarks—LiveCodeBench, GPQA-Diamond, AIME 2024, and MATH-500—cover multiple fields and difficulty levels.The evaluation is designed to assess reasoning across diverse scenarios.
- Evaluation setup: Globally averaged accuracy is reported with a 32,768-token generation limit, temperature 0.6, top-p 0.95, and benchmark-specific sampling counts.AIME 2024 uses 16 samples per query, while the other benchmarks use four to estimate pass@1.
- Main results: 73.1% average accuracy was achieved by AM-Distill-Qwen-32B versus 71.6% for DeepSeek-R1-Distill-Qwen-32B.The 32B model also improved on AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench individually.
- Main results: AM-Distill-Qwen-72B improved over DeepSeek-R1-Distill-Llama-70B on AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench.The reported accuracies increased from 70.0% to 76.5%, 94.5% to 97.0%, 65.2% to 65.9%, and 57.5% to 59.7%, respectively.
- Main results: The experiments conclude that models trained on the AM-DeepSeek-R1-Distilled-1.4M dataset show substantial reasoning-ability enhancements.This conclusion summarizes the benchmark comparisons for the trained models.
4 Limitation
The dataset has unresolved factual-accuracy and source-accuracy issues, and its use is restricted to research because harmful content has not been thoroughly filtered.
- Limitations: Responses generated by large language models have not been rigorously verified, leaving deficiencies in factual accuracy and other aspects.Users are instructed to examine the dataset carefully before use.
- Limitations: The dataset has not been thoroughly filtered for harmful instructions or responses, and commercial or other potentially harmful uses are not permitted.The stated permitted scope is research use with the project’s open-sourced artifacts.
- Limitations: Nested relationships among some data sources may produce inaccurate source information.This is identified as an additional data-quality concern.
5 Conclusion
The study constructs and releases AM-DeepSeek-R1-Distilled, a 1.4-million-entry dataset with rich thinking traces, created through selection, semantic deduplication, and strict cleaning. SFT using the dataset produced notable performance improvements in AM-Distill-Qwen-32B.
- 1.4 million entries make AM-DeepSeek-R1-Distilled a large-scale general reasoning dataset with rich thinking traces.
- The dataset was built through meticulous selection, semantic deduplication, and strict cleaning of open-source datasets.
- AM-Distill-Qwen-32B, trained with SFT on the dataset, exhibited remarkable performance enhancements.
A.1 Length Distribution
Dataset entries are concentrated below 4096 tokens, with the highest concentration around approximately 2048 tokens. Longer entries become progressively less common.
- Most data entries contain fewer than 4096 tokens.
- Approximately 2048 tokens is the point of highest concentration in the length distribution.
- The distribution gradually decreases as token count increases, leaving fewer samples with longer contexts.
A.2 Reference Distribution
Reference answers and test cases are unevenly distributed across dataset entries. Entries without either type comprise the largest single group.
- 39.2% of entries have neither reference answers nor test cases.
- 38.9% of entries have reference answers.
- 21.9% of entries include test cases.
A.3 Category Distribution
The dataset is concentrated in Math, Coding, and Information Seeking tasks, which together form its dominant task categories. Reasoning, Planning, Creative Writing, and other categories make up smaller portions.
- 29.3% of entries are Math tasks, the largest reported category.
- 24.3% of entries are Coding tasks and 22.2% are Information Seeking tasks.
- Reasoning accounts for 10.4% of entries, while Planning and Creative Writing account for 2.3% and 2.2%, respectively.
- Other combined categories account for 9.3% of entries.
A.4 Difficulty Distribution
The dataset’s difficulty distribution is concentrated in the Medium and Hard categories. Difficulty and category labeling, along with correctness rating, are documented through prompts referenced in the appendix materials.
- 51.8% of data entries are classified as Medium difficulty.
- 25.7% of data entries are classified as Hard difficulty.
- 11.2% of entries are Easy, 6.5% are Very Hard, and 4.7% are Very Easy.
- Difficulty rating, category classification, and correctness rating are associated with prompts in Tables 3, 4, and 5, respectively.