Source-linked AI summary
OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek, Siddhartha Jain, Jocelyn Huang, Vahid Noroozi, Boris Ginsburg
TL;DR
The paper addresses limited understanding of how far SFT can improve coding reasoning, amid scarce large-scale reasoning datasets and opaque curation practices. It constructs and releases OPENCODEREASONING, evaluates SFT-fine-tuned Qwen2.5 models, and finds strong benchmark performance, including 61.8 on LiveCodeBench and 24.6 on CodeContests at 32B scale.
Problem
SFT’s potential for improving reasoning remains poorly understood because large-scale reasoning datasets and detailed curation practices are limited.
Method
The paper constructs OPENCODEREASONING from diverse coding questions, reasoning-model generations, and post-processed validated solutions, then fine-tunes Qwen2.5 models across 7B, 14B, and 32B sizes.
Results
SFT-fine-tuned models significantly outperform comparable SFT-only and distilled baselines, with the 32B model reaching 61.8 on LiveCodeBench and 24.6 on CodeContests.
Takeaways & Limitations
The results support scaling diverse, varied, and difficult coding questions as a productive dataset-building strategy for transferring reasoning capabilities through SFT.
Takeaways & Limitations
Scaling to 736k samples had not plateaued, and longer reasoning budgets sometimes caused unrecoverable loops without improving hard-problem accuracy.
Abstract
from arXiv · showhide
Since the advent of reasoning-based large language models, many have found great success from distilling reasoning capabilities into student models. Such techniques have significantly bridged the gap between reasoning and standard LLMs on coding tasks. Despite this, much of the progress on distilling reasoning models remains locked behind proprietary datasets or lacks details on data curation, filtering and subsequent training. To address this, we construct a superior supervised fine-tuning (SFT) dataset that we use to achieve state-of-the-art coding capability results in models of various sizes. Our distilled models use only SFT to achieve 61.8% on LiveCodeBench and 24.6% on CodeContests, surpassing alternatives trained with reinforcement learning. We then perform analysis on the data sources used to construct our dataset, the impact of code execution filtering, and the importance of instruction/solution diversity. We observe that execution filtering negatively affected benchmark accuracy, leading us to prioritize instruction diversity over solution correctness. Finally, we also analyze the token efficiency and reasoning patterns utilized by these models. We will open-source these datasets and distilled models to the community.
1 Introduction
The paper addresses the limited understanding of how far SFT can improve coding reasoning relative to SFT+RL models by releasing OPENCODEREASONING and evaluating distilled Qwen2.5 models. Its SFT-only models achieve strong LiveCodeBench results while providing analyses of dataset construction and training choices.
- Dataset and motivation: The study is motivated by proprietary datasets and limited curation details that constrain understanding of SFT-based reasoning distillation.Prior distilled models commonly use SFT, whereas leading open-weight reasoning models combine SFT with RL.
- Main results: SFT-only Qwen2.5 models at 7B and 14B achieve LiveCodeBench pass@1 rates of 51.3 and 59.4, surpassing same-size R1-Distill-Qwen models by 13.7 and 6.3 absolute points.These results target the performance disparity between SFT-only and SFT+RL reasoning models.
- Main results: 61.8 pass@1 on LiveCodeBench is achieved by the SFT-only 32B model, narrowing the gap with DeepSeek-R1’s 65.9.The 32B model also surpasses the cited OpenAI O1 and O3-Mini models.
- Dataset and motivation: OPENCODEREASONING contains 736,712 DeepSeek-R1-generated Python solutions with reasoning traces for 28,904 competitive programming questions.The dataset is presented as the largest of its kind and is released for community use.
- Analyses and contributions: The work combines dataset release, model validation, and ablations examining execution filtering, multilingual solutions, reasoning length, and reasoning patterns.The authors frame these analyses as insights and a replicable recipe for future datasets.
2 OPENCODEREASONING: Dataset Construction and Refinement
OPENCODEREASONING is constructed by collecting and deduplicating competitive programming questions, generating reasoning solutions with DeepSeek-R1, and post-processing them. The authors study scaling and report that larger, more diverse and difficult datasets improve coding performance, while the scaling curve has not plateaued at 736k samples.
- Dataset construction: The dataset construction pipeline collects diverse questions, generates LLM responses, and post-processes reasoning traces and solution excerpts.The refinement stage checks reasoning tags, extracts solution segments, and requires code blocks in Python or C++ delimiters.
- Coding question collection: 28,904 distinct questions remain after exact-match deduplication across TACO, APPS, CodeContests, and CodeForces sources.The collected questions span a range of difficulties.
- Solution generation: DeepSeek-R1 generates multiple solutions per question, primarily in Python, with additional C++ solutions for preliminary experiments on the harder IOI benchmark.Sampling uses temperature 0.6, top-p 0.95, and an explicitly injected <think> tag.
- Scaling up data: Scaling from 25k to 736k samples produces progressively better results, with the final expansion yielding the most significant improvements across diverse problem types.The staged process adds hard CodeContests questions and then expands to 28k unique questions, producing 736,712 samples.
- Scaling up data: The scaling curve does not plateau at 736k samples, leaving the point of saturation and diminishing returns unresolved.The largest gains came from increasing the number of unique, varied, and difficult questions.
3 Main Evaluation
OPENCODEREASONING-based Qwen2.5 models outperform comparable open-weight baselines across coding benchmarks, with gains increasing at larger model scales.
- Competitive Scores at Small Scales: 10.4% and 7.5% absolute improvements were achieved by OCR-Qwen-7B-Instruct over OlympicCoder-7B on LiveCodeBench and CodeContests, respectively.
- Scaling Yields Rapid Gains: 59.4 average pass@1 on LiveCodeBench and 23.6 pass@1 on CodeContests were achieved by OCR-Qwen-14B-Instruct.
- Narrowing the Gap to Top-Tier Models: 61.8 on LiveCodeBench and 24.6 on CodeContests were achieved by OCR-Qwen-32B, surpassing QwQ-32B.
- Narrowing the Gap to Top-Tier Models: OCR-Qwen-32B’s scores were only slightly below DeepSeek-R1’s 65.6 and 26.2 on LiveCodeBench and CodeContests, respectively.
- The reported finetuned-model results were averaged over 64 inference runs, whereas baselines were run once.
4 Ablation and Analyses
The analyses examine execution filtering, multilingual data, reasoning length, and reasoning patterns, finding trade-offs between correctness, diversity, efficiency, and performance.
- 4.1 Ablation: Filtering by Code Execution: Fine-tuning on incorrect solutions produced higher accuracy than fine-tuning on correct solutions in the CodeContest ablation.
- 4.1 Ablation: Filtering by Code Execution: Incorrect solutions covered more challenging questions than correct solutions, complicating interpretation of the filtering comparison.
- 4.1 Ablation: Filtering by Code Execution: The CodeContest ablation used 445k instances, including 151k unit-test-passing samples and an equal-sized subset failing all tests.
- 4.1 Ablation: Filtering by Code Execution: 50 generated tests were retained per problem to reduce computational cost from an average of 192.7 generated tests.
- 4.2 Ablation: Inclusion of C++ Solutions: Adding 356k R1-generated C++ samples did not improve Python benchmark performance but significantly improved accuracy on the C++ IOI benchmark.
- 4.3 Analysis: How long does an LLM think before generating code solution?: All evaluated models generated more reasoning tokens for harder problems, while OCR-32B closely followed DeepSeek-R1’s token budget.
- 4.3 Analysis: How long does an LLM think before generating code solution?: Extending the reasoning budget from 16k to 32k tokens did not produce substantial accuracy gains on hard problems, and occasional unrecoverable loops consumed available tokens.
- 4.4 Analysis: Reasoning Patterns: Exploration-related reasoning patterns increased with difficulty, while self-evaluation proportions declined.
5 Related Works
Prior work combines reasoning distillation, reinforcement learning, and synthetic code-data generation, while this paper studies scaling reasoning-based data to 736,712 samples.
- Reasoning-oriented LLM research has used reinforcement learning and rule-based reward models to improve mathematical and coding capabilities.
- Synthetic code-data methods have generated solutions, diversified problems, and refined outputs through prompting and related techniques.
- 17k to 114k reasoning-based question-solution datasets have improved fine-tuned coding models, while some results were presented through blog posts.
- This work investigates scaling synthetic reasoning data to 736,712 samples and reports state-of-the-art supervised fine-tuned performance across model sizes.
6 Conclusion
The paper presents OPENCODEREASONING as a large reasoning-based code-generation dataset and evaluates its use in fine-tuning Qwen2.5 models across multiple sizes.
- OPENCODEREASONING is presented as the largest instruction-tuning dataset to date for code generation with reasoning.
- Fine-tuning Qwen2.5 base and instruct models at 7B, 14B, and 32B significantly outperformed DeepSeek-R1-Distill-Qwen models on LiveCodeBench and CodeContests.
- The paper provides ablations and analyses of fine-tuning considerations and states that the dataset will be fully open-sourced.
A Reasoning pattern extraction
The paper extracts reasoning patterns from chain-of-thought using a two-stage segmentation process guided by prompt templates. It then keeps tagged patterns while consolidating verification with reasoning and excluding multiply labeled segments.
- Segmentation: Qwen-32B-Instruct performs initial chain-of-thought segmentation with the Figure 6 prompt, followed by another segmentation round using Figure 7 for unannotated segments.The two prompt templates correspond to the initial and follow-up segmentation stages.
- Pattern extraction: Reasoning patterns are extracted from segments enclosed in <pattern> tags.
- Pattern extraction: Verification and reasoning patterns are merged into a single self-evaluation category.
- Filtering: Segments assigned multiple patterns are excluded because the model is considered unsure about their classification.