PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenThoughts: Data Recipes for Reasoning Models

Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, Ludwig Schmidt

arXiv:2506.04178v2cs.LG

TL;DR

Reasoning-model training recipes remain difficult to study because frontier systems often rely on proprietary data and undisclosed procedures. OpenThoughts systematically investigates SFT data curation, producing scalable open datasets and models with strong multi-benchmark results. The project releases its datasets and models publicly.

  • Problem

    Frontier reasoning models often rely on proprietary datasets and undisclosed recipes, limiting public understanding of how to build strong reasoning systems.

  • Method

    OpenThoughts performs more than 1,000 controlled ablation experiments across math, code, and science to develop and scale an SFT data-curation pipeline.

  • Results

    OpenThinker3-7B outperformed DeepSeek-R1-Distill-7B by 12.4 points on average across 12 tasks and exceeded Nemotron-Nano-8B by 2.1 points.

  • Takeaways & Limitations

    The study identifies practical data-curation findings, including that multiple teacher answers can expand a source by at least 16× and that stronger benchmark models are not necessarily better teachers.

  • Takeaways & Limitations

    The study does not explore reinforcement-learning datasets, staged SFT, or curriculum learning.

Abstract

from arXiv · show

Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available. To address this, the goal of the OpenThoughts project is to create open-source datasets for training reasoning models. After initial explorations, our OpenThoughts2-1M dataset led to OpenThinker2-32B, the first model trained on public reasoning data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such as AIME and LiveCodeBench. We then improve our dataset further by systematically investigating each step of our data generation pipeline with 1,000+ controlled experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples and using QwQ-32B as teacher yields our OpenThoughts3-7B model, which achieves state-of-the-art results: 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Diamond - improvements of 15.3, 17.2, and 20.5 percentage points compared to the DeepSeek-R1-Distill-Qwen-7B. All of our datasets and models are available on https://openthoughts.ai.

1 INTRODUCTION

OpenThoughts addresses the limited public information about frontier reasoning-model training recipes by systematically studying SFT data curation. Its experiments produce a scalable pipeline and strong open-data reasoning models.

  • Motivation: Frontier reasoning models use post-training to produce long thinking-token chains, but their complete training recipes are generally not public.This lack of transparency makes it difficult to study how to build reasoning models.
  • Motivation: Carefully curated supervised fine-tuning data can yield strong small- to mid-scale reasoning models without reinforcement learning.R1-Distill models achieved 51% on AIME and 33% on GPQA using supervised fine-tuning on teacher-generated question-thinking-answer triplets.
  • Motivation: Exploring the reasoning-data design space is otherwise prohibitively expensive because teacher inference and model training are costly.Without such experiments, researchers often rely on existing heuristics and intuitions for data design.
  • Project goal: OpenThoughts investigates how data curation choices contribute to strong reasoning SFT datasets and challenges preexisting notions of data quality.Earlier releases studied automated verification and increased question diversity through synthetic generation.
  • OpenThoughts3: More than 1,000 ablation experiments across math, code, and science produced a scalable pipeline and OpenThinker3-7B.The model outperformed R1-Distill-7B by 12.4 points on average across 12 tasks and Nemotron-Nano-8B by 2.1 points.
  • Key findings: Sampling multiple teacher answers expands a data source by at least 16×, while QwQ-32B can be a stronger teacher than DeepSeek-R1 despite lower benchmark scores.The experiments also found that top one or two question sources, LLM-based difficulty or response-length filters, and several answer-filtering methods have distinct effects on performance.

2 RELATED WORK

Related work builds reasoning datasets from public math, code, and science sources, often innovating across sourcing, filtering, scaling, and teacher-based trace distillation. OpenThoughts frames its pipeline as a systematic exploration of these choices.

  • Reasoning datasets: Open reasoning datasets commonly target math, code, and science using questions from public forums, competition sites, or pre-training corpora.Examples include OpenR1, OpenMathReasoning, OpenCodeReasoning, and Natural Reasoning.
  • Data pipelines: Many reasoning projects introduce innovations across multiple pipeline stages, including data sourcing, filtering, and scaling.Some efforts extend beyond supervised fine-tuning to build datasets for reinforcement learning.
  • OpenThoughts pipeline: The OpenThoughts pipeline investigates six design stages: question sourcing, source mixing, question filtering, deduplication and answer sampling, answer filtering, and teacher selection.The figure presents these stages as experiments toward a strong reasoning-dataset recipe.

3 THE OPENTHOUGHTS PROJECT

The OpenThoughts project progresses through increasingly large open reasoning datasets and models, culminating in OpenThoughts3-1.2M and OpenThinker3-7B. Its releases combine broader data curation with strong benchmark results and open artifacts.

  • Project progression: The project spans four generations, from BespokeStratos-17K to OpenThoughts3-1.2M and OpenThinker3-7B.The releases focus on identifying questions and answers that encourage reasoning and are shared through openthoughts.ai.
  • Model progression: Successive OpenThinker data recipes consistently improve performance across domains.Table 2 summarizes progression across BespokeStratos-17K, OpenThoughts-114K, and OpenThoughts2-1M, with further details in Appendix C.
  • OpenThoughts-114K: OpenThoughts-114K contains 114K questions across math, code, science, and puzzles, with responses distilled from DeepSeek-R1.Its tooling uses answer matching and unit tests to verify math and code responses.
  • OpenThoughts2-1M: OpenThoughts2-1M produced a model competitive with R1-Distill-32B and 6% better on AIME25.The dataset combines 600K verified OpenR1-Math samples with 200K unverified samples and selects sources after sweeping 26 question sources.
  • OpenThoughts3: OpenThoughts3-1.2M is the culmination of prior datasets and systematic exploration of the data-pipeline design space.The paper focuses on this dataset and details its pipeline and final composition in the following section.

4 OPENTHOUGHTS3 DATA PIPELINE

OpenThoughts3 systematically evaluates question sourcing, mixing, filtering, deduplication, repeated teacher sampling, answer filtering, and teacher selection through controlled downstream experiments. The resulting design favors high-quality sources, selective question filtering, repeated sampling, unfiltered answers, and QwQ-32B as teacher.

  • Pipeline design: More than 1,000 ablation experiments compare data-curation strategies across math, code, and science.Each pipeline step is evaluated independently while downstream training and evaluation remain controlled.
  • Question sourcing: 17.2 points separate the strongest and weakest code question datasets, showing that question quality substantially affects performance.The experiments evaluate 27 code, 21 math, and 14 science sources.
  • Question mixing: 5% higher average accuracy results from mixing two high-quality code sources instead of 16, with at most two sources performing best across domains.The findings favor source quality over diversity from mixing many datasets.
  • Question filtering: 4% and 6% average gains over random filtering occur for math and code when using the best question-filtering strategies.Difficulty filtering wins for code, while response-length filtering wins for math and science; LLM-based filters outperform classical filters across domains.
  • Deduplication and answer filtering: Repeated teacher sampling expands data scale, while answer filtering provides no significant benefit across domains.The pipeline therefore uses repeated answers as a scaling axis and skips answer filtering in later stages.
  • Teacher model: QwQ-32B improves average accuracy over DeepSeek-R1 by 1.9% for code and 2.6% for math despite lower scores on several target benchmarks.Sampling hyperparameters are held constant across teacher models.

5 SCALING OUR PIPELINE TO OPENTHOUGHTS3-1.2M

The selected pipeline is scaled to a 1.2-million-example dataset and used to train OpenThinker3-7B. Scaling correlates positively with performance, and the resulting model performs best among open-data 7B reasoning models on several evaluations.

  • Scaling behavior: Strong positive correlation between dataset scale and performance appears as the pipeline grows from 316 to 31.6k examples.Performance improves when the best choices from successive pipeline stages are stacked.
  • Dataset construction: 1.2 million datapoints comprise OpenThoughts3-1.2M: 850,000 math, 250,000 code, and 100,000 science examples.The mixture follows the OpenThoughts2-1M ratio.
  • Model results: OpenThinker3-7B is the best open-data reasoning model at the 7B scale regardless of using SFT, RL, or both.It also achieves the best scores on held-out HMMT, AIME25, and LiveCodeBench 06/24-01/25 evaluations.

6 CONCLUSION

OpenThoughts develops open reasoning datasets through iterative data-curation experiments, culminating in OpenThoughts3-1.2M and OpenThinker3-7B. The work also identifies unresolved questions about cross-domain transfer, scaling, answer diversity, and training regimes.

  • 6 CONCLUSION: The study does not explore reinforcement-learning datasets, staged SFT, or curriculum learning.
  • 6 CONCLUSION: The authors assume some cross-domain transfer when optimizing average benchmark performance, but its persistence after mixing domains remains unclear.
  • 6 CONCLUSION: Whether scaling eventually plateaus or enables students to surpass teachers through weak-to-strong generalization remains open.
  • 6 CONCLUSION: The project releases its models, datasets, codebase, and accompanying blog materials through public repositories and websites.

D TRAINING DETAILS

Training uses scale-specific hyperparameters and examines efficiency choices such as sequence packing and chat templates. Explicit reasoning improves performance, while different reasoning-token templates perform comparably.

  • D TRAINING DETAILS: Training uses four hyperparameter sets matched to dataset scales from 0.3K through 100K.The sets are micro, small, medium, and large, with larger scales generally using larger batches and learning rates.
  • D TRAINING DETAILS: Sequence packing is used for the large setting to save compute time, while smaller settings prioritize more training steps.
  • D TRAINING DETAILS: Adding sequence packing does not negatively affect performance overall, despite possible effects on learning dynamics.
  • D TRAINING DETAILS: The R1 and SkyT1 chat templates produce roughly equivalent performance across benchmarks.The R1 template uses <think> and </think>, while SkyT1 uses specialized begin- and end-of-thought tokens.
  • D TRAINING DETAILS: Explicit reasoning substantially improves mathematical and scientific reasoning, including 45.3% versus 2.0% on AIME25.

E EVALUATION DETAILS

The evaluation framework standardizes generation and reporting across multiple reasoning benchmarks, while decontamination uses string-similarity criteria to reduce evaluation-set contamination. The final algorithm achieves a 99.6% true negative rate on manually contaminated items.

  • E EVALUATION DETAILS: Evalchemy shards benchmark tasks across GPUs and aggregates metrics centrally with consistent generation and logging settings.
  • E EVALUATION DETAILS: AIME24, AIME25, AMC23, and HMMT use mean accuracy and SEM over 10 iterations, while LiveCodeBench uses six iterations.
  • E EVALUATION DETAILS: CodeForces, CodeElo, GPQA Diamond, JEEBench, and HLE use three iterations, while MATH500 uses a single full-set pass.
  • E EVALUATION DETAILS: AIME 2025, LiveCodeBench 06/24-01/25, HMMT, and HLE are held-out evaluation sets.
  • E EVALUATION DETAILS: The decontamination procedure discards samples flagged by either normalized Indel or 13-gram similarity.A normalized Indel similarity of 75% indicates contamination under that criterion.
  • E EVALUATION DETAILS: 99.6% true negative rate was achieved on 3,092 manually contaminated samples, missing only 12 questions.

G ADDITIONAL SCALING EXPERIMENTS

OpenThoughts3 continues to scale across datasets, domains, and model families, although gains are uneven and some benchmarks saturate. Additional ablations show that several verification and filtering strategies provide little or no improvement.

  • G ADDITIONAL SCALING EXPERIMENTS: OpenThoughts3 outperforms existing reasoning datasets across diverse math, code, and science benchmarks and dataset scales.Held-out benchmarks include HMMT, AIME 2025, LiveCodeBench 06/24-01/25, and HLE.
  • G ADDITIONAL SCALING EXPERIMENTS: Performance on many benchmarks improves through the 1M scale, while AMC23, MATH500, and HLE show saturation or limited response to scale.
  • G ADDITIONAL SCALING EXPERIMENTS: Scaling curves contain dips and jumps, and retraining and reevaluation on a fixed dataset do not fully explain them.
  • G ADDITIONAL SCALING EXPERIMENTS: Individual math, code, and science recipes continue scaling beyond 31.6K samples for another order of magnitude.
  • G ADDITIONAL SCALING EXPERIMENTS: Llama models also show scaling gains, including AMC23 improvement from 15.8 to 75.2.
  • G ADDITIONAL SCALING EXPERIMENTS: Removing proof-based questions harms relevant benchmark performance despite their unverifiability under the tested methodology.
  • G ADDITIONAL SCALING EXPERIMENTS: LLM-generated unit-test verification does not improve downstream code-generation accuracy.

H.2 TEACHER MODEL

The experiments compare teacher models and reasoning-trace designs across math, code, and science. Longer Claude 3.7 thinking improves benchmark performance, but R1 often remains the stronger annotator and trace compression can hurt downstream results.

  • Claude vs R1 as annotator: Claude 3.7 re-annotation performs worse than R1 for the S1K math dataset and for the OpenThoughts science and code data.The comparisons swap the annotator while keeping the relevant dataset setup analogous.
  • Claude 3.7 with thinking: Longer Claude 3.7 thinking traces improve performance across math, coding, and question-answering benchmarks, with GPQA Diamond saturating earlier.Increasing the thinking-token budget benefits all tested task types, especially mathematical reasoning and coding.
  • Reasoning-trace compression: Filtering reasoning traces by maximum length causes significant downstream performance drops compared with the default dataset.The filtering thresholds include 2,048, 4,096, and 8,192 tokens, while preserving self-reflection in retained traces.
  • Reasoning-trace compression: Removing self-reflection shortens average reasoning traces from 11.5K to 0.3K tokens in the reported 12K-instance comparison.The table describes the default OpenThoughts3 traces and the self-reflection-removed variant.
  • Cross-domain transfer: Science performance can improve from math-only fine-tuning, but strong in-domain science data removes the observed code-dataset transfer differences.The reported cross-domain gains do not persist after mixing the code datasets with the strong science dataset.
  • Question difficulty: LLM-generated difficulty labels can identify harder questions: DeepSeek-R1 scores over 75% at difficulty one but below 57% at difficulty ten.GPT-4o-mini reliably predicts which questions R1 answers correctly, supporting the use of difficulty labels for filtering.

I.3 SAMPLING BY LONGEST, SHORTEST, MAJORITY

The sampling experiments compare shortest, longest, and majority answer selection across reasoning benchmarks and models. Shortest responses generally outperform longest responses, while majority voting often performs best when multiple generations can be aggregated.

  • Sampling strategies: Shortest-response selection consistently outperforms longest-response selection across models and datasets.Majority voting often achieves the best overall performance, but shortest selection is an effective single-response alternative.
  • AIME24: On AIME24, majority voting generally performs best, while shortest selection beats longest selection for most models.For DeepSeek-R1-Distill-Qwen-7B, shortest selection scores 66.67% versus 36.67% for longest selection.
  • GPQA Diamond: 7–11 percentage points: shortest selection improves over longest selection across most models on GPQA Diamond.These results come from the higher-run experiments summarized in Table 26.
  • Response length: Incorrect vanilla-sampling responses are generally longer than correct responses across most models.The response-length analysis associates greater verbosity with uncertainty or error-prone reasoning paths.
  • Practical implications: Selecting the shortest response can be computationally efficient and often outperform vanilla sampling and longest-response selection.The efficiency benefit applies when generation can stop after selecting the shortest response.
  • Limitations: The optimal sampling strategy is model-dependent, and majority voting can be strongest when resources permit multiple responses and aggregation.NovaSky-Sky-T1-32B is cited as an example where other strategies perform better on some datasets.
  • Legal reasoning: All fine-tuned Qwen2.5-7B models outperform the much larger R1 annotator on the legal classification task.The comparison covers different data-generation strategies involving repeated R1 annotations and verification.
  • Teacher comparisons: OpenThinker3-7B outperforms QwQ-32B on JEEBench despite QwQ-32B serving as the stronger teacher by benchmark score.The teacher comparison also reports that DeepSeek-R1 is empirically the strongest overall teacher.

L SAFETY ANALYSIS OF OPENTHINKER MODELS

The safety analysis evaluates harmfulness and over-refusal in OpenThinker models. Supervised fine-tuning on reasoning degrades inherited safety alignment, and stronger reasoning coincides with a reported safety trade-off.

  • Safety outcomes: Supervised fine-tuning on reasoning inadvertently degrades the preexisting safety alignment of Qwen2.5 models.The analysis reports this pattern across the OpenThinker generations.
  • Safety outcomes: OpenThinker3-7B reaches a harmfulness score of 55.5 and an over-refusal rate of 5.6 without explicit safety-tuning data.OpenThinker2-7B has harmfulness 42.8 and over-refusal 2.4, while OpenThinker-7B has harmfulness 36.8 and over-refusal 4.4.
  • Discussion: The reported comparison identifies a trade-off between reasoning capability and safety in the OpenThinker models.The authors suggest incorporating safety-specific datasets in future work to mitigate these risks while preserving reasoning performance.

M EXISTING FRONTIER MODEL EVALUATIONS

The frontier-model evaluation compares API-accessible systems under reported token-budget and reasoning-effort settings. Gemini-2.5-pro performs strongest overall, while OpenThinker3-7B averages better than the models on the other side of the table separator.

  • API model comparison: Gemini-2.5-pro shows the strongest performance, although some answers are empty after its thinking process exceeds the token budget.The issue is especially visible on JEEBench; Claude 3.7 receives a 32K token budget and o3 uses default medium reasoning effort.
  • API model comparison: OpenThinker3-7B outperforms, on average, the API models to the right of the table separator.The comparison is reported in Table 31 under the stated API evaluation settings.

N TESTING REASONING ROBUSTNESS: ALICE IN WONDERLAND EVALUATION

The evaluation tests whether models preserve performance across problem variations that retain the same abstract solution. Reasoning models outperform conventional LLMs on average but still show substantial variation across equivalent formulations.

  • Evaluation setup: Reasoning-invariant perturbations change problem statements without changing their abstract solutions, so robust models should perform similarly across variations.The evaluation measures sensitivity to structure-preserving changes using repeated sampling and variance estimates.
  • Generalization results: Distilled reasoning models exhibit substantial performance fluctuations across simple problem variations, indicating generalization deficits.The fluctuations persist despite improved average performance compared with non-reasoning models.
  • Performance comparison: Reasoning models achieve higher average correct response rates than conventional LLMs across the Alice in Wonderland evaluations.The comparison averages correct response rates across AIW Friends, AIW Plus, and AIW Circles Colleagues problems.
  • Performance comparison: Mid-scale 32B reasoning models strongly outperform conventional models trained at much larger scales, including Llama 3.1 405B and DeepSeek v3 671B.This comparison holds despite the reasoning models’ persistent generalization deficits.
  • Pipeline findings: The broader pipeline study found that question-source selection and question mixing materially affect results, while less mixing performs better across domains.The code-domain gap between top- and lowest-performing question sources averages 17.2 points.

S.6 TEACHER MODEL EXPERIMENTS

Teacher-model experiments compare candidate teachers across code, math, and science data domains. QwQ-32B is the strongest teacher by a statistically significant margin.

  • Teacher model comparison: QwQ-32B is the best teacher across code, math, and science data domains by a statistically significant margin.The comparison is reported across the teacher-model ablations in Tables 47–49.
Loading 2506.04178v2…