Source-linked AI summary
Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems
Zongqian Li, Tengchao Lv, Shaohan Huang, Yixuan Su, Qinzheng Sun, Qiufeng Yin, Ying Xin, Scarlett Li, Lei Cui, Nigel Collier, Furu Wei
TL;DR
Existing coding datasets suffer from difficulty imbalance, format inconsistency, and data-quality problems. The paper introduces a four-stage processing pipeline and predict-calibrate-select difficulty filtering to build MicroCoder, which consistently improves performance over baseline datasets, especially on medium and hard problems.
Problem
Existing coding datasets have imbalanced difficulty, inconsistent formats, noisy content, and limited recent challenging problems.
Method
The paper combines four-stage data processing with LLM-based predict-calibrate-select filtering using five weighted difficulty dimensions.
Results
MicroCoder consistently outperforms DeepCoder across benchmarks and difficulty levels, with larger gains under DAPO than GRPO and strongest improvements on medium and hard problems.
Takeaways & Limitations
Difficulty-aware curation improves code-generation performance on challenging tasks and provides dataset-creation insights.
Abstract
from arXiv · showhide
Training next-generation code generation models requires high-quality datasets, yet existing datasets face difficulty imbalance, format inconsistency, and data quality problems. We address these challenges through systematic data processing and difficulty scaling. We introduce a four-stage Data Processing Framework encompassing collection, processing, filtering, and verification, incorporating Automatic Difficulty Filtering via an LLM-based predict-calibrate-select framework that leverages multi-dimensional difficulty metrics across five weighted dimensions to retain challenging problems while removing simplistic ones. The resulting MicroCoder dataset comprises tens of thousands of curated real competitive programming problems from diverse platforms, emphasizing recency and difficulty. Evaluations on strictly unseen LiveCodeBench demonstrate that MicroCoder achieves 3x larger performance gains within 300 training steps compared to widely-used baseline datasets of comparable size, with consistent advantages under both GRPO and its variant training algorithms. The MicroCoder dataset delivers obvious improvements on medium and hard problems across different model sizes, achieving up to 17.2% relative gains in overall performance where model capabilities are most stretched. These results validate that difficulty-aware data curation improves model performance on challenging tasks, providing multiple insights for dataset creation in code generation.
1 Introduction
Existing coding datasets are limited by difficulty imbalance, inconsistent formats, and noisy or incomplete data. This work addresses these gaps through systematic processing and difficulty-aware curation, producing MicroCoder and reporting stronger gains on challenging problems.
- Web-collected problems introduce incomplete descriptions, incomplete test cases, and irrelevant content, motivating robust quality filtering.
- Existing datasets often contain predominantly simple problems and lack recent challenges that can support capability improvement.
- Mixed function-completion and input/output formats can cause algorithmically correct solutions to use incorrect execution formats.
- The paper introduces a four-stage processing pipeline and LLM-based difficulty-aware filtering across five weighted dimensions.
- 13,300 real competitive programming problems form the MicroCoder dataset, which achieves 3× larger performance gains within 300 training steps than comparable baseline datasets.
2.1 Data Processing Framework
The Data Processing Framework transforms heterogeneous coding-problem sources into standardized, filtered, verified training data. It combines normalization, test-case handling, multi-criteria filtering, and train-test separation.
- The framework uses Collect, Process, Filter, and Verify stages to transform raw sources into a standardized corpus for reinforcement learning.
- Processing translates non-English problems, removes noise, validates completeness, and standardizes prompt formats across problem sources.
- LLM-generated test inputs executed against reference solutions provide accurate outputs for constructing comprehensive test suites.
- Filtering enforces text-only, unique, and unseen requirements while adaptively selecting problems by quality and multidimensional difficulty.
- A 16-gram similarity threshold of 0.22 is applied across datasets to enforce train-test separation.
2.2 Automatic Difficulty Filtering
Automatic Difficulty Filtering predicts problem complexity, calibrates predictions against empirical model performance, and selects problems within target difficulty ranges. The case study shows that calibrated filtering removes simple problems while preserving difficult ones.
- Predict-Calibrate-Select: The predict-calibrate-select framework averages three LLM assessments, maps scores to empirical difficulty, and filters problems outside target ranges.
- Case Study: Ground-truth difficulty is measured by success rates across four model attempts, enabling calibration of predicted scores.
- Multi-dimensional Difficulty Metrics: Five weighted dimensions emphasize algorithmic thinking complexity and implementation difficulty over comprehension and knowledge breadth.
- Case Study: The resulting pipeline retains a dataset composed of real competitive programming problems after difficulty and quality filtering.
- Case Study: 30% filtering removes over 65% of easy problems while preserving difficult problems, reducing the easy-problem ratio from approximately 40% to under 20%.
3 Dataset
MicroCoder is a curated collection of real competitive programming problems from diverse platforms, with controlled statistics, capped test suites, and low similarity to evaluation benchmarks. These properties support diversity and strict train-test separation.
- MicroCoder contains 13,300 curated real competitive programming problems from platforms including AIZU, AtCoder, CodeChef, and Kattis.
- Most problems are 1K–3K characters long, while AIZU problems have the longest average length among the reported sources.
- All datasets are capped at the 15 longest test cases per problem to avoid training pauses from excessively large test suites.
- t-SNE visualization shows platform separation and complementary coverage between privately collected and open-source datasets.
- Cosine similarity between training datasets and AtCoder, LeetCode, and LiveCodeBench ranges from 0.04 to 0.14, with zero overlap reported.
4 Experimental Design
The experiments evaluate MicroCoder against DeepCoder on unseen competitive-programming benchmarks using standardized accuracy and reinforcement-learning training configurations.
- Benchmarks: Evaluation uses AtCoder, LeetCode, and strictly unseen LiveCodeBench v6 problems spanning easy, medium, and hard categories.LiveCodeBench v6 uses its official testing framework and code-execution infrastructure.
- Evaluation: Accuracy is binary per problem and averaged across four independent inference attempts.A solution scores 1 only when it passes all test cases.
- Baselines: MicroCoder is compared with the widely used open-source DeepCoder dataset under identical training and inference configurations.
- Implementation: Experiments use Qwen3-4B-Instruct-2507 with 8K-token responses, batch size 64, learning rate 1e-6, eight samples per query, and binary accuracy rewards.
- Training: Training uses GRPO and DAPO, with DAPO removing KL loss and applying high clipping to encourage diverse solutions.
5 Results
MicroCoder consistently improves unseen benchmark performance over DeepCoder, with the strongest gains on medium and hard problems and under DAPO. Its lower training rewards alongside higher test accuracy indicate that more difficult training problems can support stronger generalization.
- Training Dynamics: MicroCoder maintains higher test accuracy but lower training rewards than DeepCoder under both GRPO and DAPO, indicating greater training difficulty alongside stronger generalization.
- Overall Performance: +2.2, +1.6, and +2.0 points are MicroCoder’s GRPO improvements on AtCoder, LeetCode, and LiveCodeBench overall accuracy, respectively.
- Overall Performance: +3.6, +6.0, and +4.4 points are MicroCoder’s DAPO improvements on AtCoder, LeetCode, and LiveCodeBench, corresponding to 9.3%, 18.7%, and 12.1% relative gains.
- Difficulty-Specific Analysis: +8.6 to +9.7 points are MicroCoder’s DAPO gains on medium problems, including a +40.4% relative improvement on LeetCode Medium.
- Difficulty-Specific Analysis: +20.7% to +22.0% are the largest reported relative gains on hard problems under DAPO, including LiveCodeBench and AtCoder.
- Scaling and Ablations: Across model scales and data sources, MicroCoder’s advantages increase with model size, while DAPO consistently produces larger improvements than GRPO.
6 Conclusions
The paper presents difficulty-aware curation as a systematic approach to constructing competitive-programming datasets and reports broad gains on challenging problems. It also identifies extensions to languages, tasks, difficulty dimensions, and adaptive assessment.
- Conclusions: Difficulty-aware curation is presented as a systematic approach to creating high-quality code-generation datasets.
- Conclusions: Table 1 reports consistent MicroCoder improvements across benchmarks and difficulty levels, including +9.7 points (+40.4%) on LeetCode Medium under DAPO.
- Conclusions: The framework produces 13,300 curated competitive-programming problems through four processing stages and automatic difficulty filtering.
- Future Directions: Future work includes multilingual extension, model-specific difficulty dimensions, other code tasks, and difficulty assessment that adapts to model capabilities.