Source-linked AI summary
Combinatorial Synthesis: Scaling Code RLVR via Atomic Decomposition and Recombination
Jiasheng Zheng, Boxi Cao, Boxi Yu, Yuzhong Zhang, Jialun Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
TL;DR
RLVR scaling is limited by the scarcity of challenging, verifiable code tasks, and heuristic seed expansions do not adequately increase novelty or difficulty. ADR decomposes seed tasks into atomic elements and recombines them under controlled validation and refinement, achieving stronger data quality and RLVR improvements across code domains. Its current evaluation remains limited to specific benchmarks and model scales.
Problem
RLVR scalability is constrained by scarce verifiable code tasks that are sufficiently challenging to target models near their competence edge.
Method
ADR extracts atomic elements from domain-specific seeds, recombines them into tasks, validates them through execution, and refines tests adversarially.
Results
ADR outperforms prior synthetic-data methods across originality, difficulty, diversity, and test quality and consistently improves RLVR performance across code domains and base models.
Takeaways & Limitations
ADR provides a scalable paradigm for synthesizing high-quality code data for RLVR across algorithmic programming, tool usage, and data science.
Takeaways & Limitations
Current evaluation is limited to specific benchmarks and model scales, with broader foundation-model, multilingual, and code-agent settings left for future work.
Abstract
from arXiv · showhide
Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as the cornerstone for shaping the remarkable coding abilities of Large Language Models (LLMs). However, the scalability of RLVR is severely constrained by the scarcity of sufficiently challenging verifiable code tasks that target near the model's edge of competence. Prior studies often rely on heuristic seed expansions for data synthesis, which severely limits both novelty and difficulty. Consequently, the training value of such data fails to scale proportionally with the size of its synthesis. To this end, we propose Atomic Decomposition and Recombination (ADR), a novel framework that generates verifiable code tasks via decomposition into atomic elements and controlled recombination, thereby enabling the generation of genuinely novel and challenging verifiable code tasks. Experiments and analysis demonstrate that ADR achieves superior originality, difficulty, diversity, and test quality over existing baselines, and consistently delivers greater improvements in code ability across RLVR in diverse downstream domains, including algorithmic programming, tool usage, and data science. Our work sheds light on a new paradigm for novel code task synthesis and scalable RLVR training.
1 Introduction
RLVR scaling is constrained by scarce, sufficiently challenging verifiable code tasks, while heuristic seed expansion mainly increases linguistic rather than logical diversity. ADR addresses this bottleneck by decomposing tasks into atomic elements and recombining them to produce novel, challenging tasks, with stronger evaluation and downstream RLVR results.
- RLVR effectiveness depends on large-scale, challenging code tasks with rigorous test cases, but manual construction is difficult to scale.
- Existing heuristic expansions increase linguistic diversity without expanding logical diversity or task difficulty, limiting exploration and causing premature reward saturation.
- ADR extracts atomic elements from domain seeds, forms an element space, and synthesizes, validates, and adversarially refines tasks through controlled recombination.
- ADR evaluates synthetic data across originality, difficulty, diversity, and test quality, then tests RLVR gains across diverse code domains and base models.
- 25.37% (+9.20%) on LCB-v5 for Qwen2.5-Coder-7B exceeds the best baseline’s 22.75%, while ADR delivers a +4.79% Pass@8 improvement.
- ADR shifts verifiable-data synthesis from heuristic seed expansion to atomic decomposition and compositional recombination.
2 Related Work
Related work applies RLVR primarily where correctness is automatically verifiable, especially mathematics and code, while synthetic-code research has largely emphasized pretraining and instruction tuning. Existing generation methods include model-driven expansions but generally pay less attention to code verifiability.
- RLVR elicits complex reasoning through automatically verifiable signals and is concentrated in domains with well-defined correctness criteria, particularly math and code.
- Synthetic code generation is motivated by the high cost and limited scalability of manual annotation, but most methods target pretraining and instruction fine-tuning rather than verifiable code.
3 Method
ADR builds verifiable code data through element extraction, controlled recombination, template-based synthesis, execution-grounded validation, and adversarial test refinement. Information-guided schema optimization and core-element anchoring help balance diversity, correctness, and difficulty.
- ADR’s pipeline comprises element extraction, controlled recombination, template-based problem synthesis, execution-grounded validation, and adversarial solution-space refinement.
- Element Extraction: Element extraction defines task-specific schemas, decomposes high-quality seeds into elements, and iteratively optimizes the schema using information-theoretic signals.
- Element Extraction: ADR estimates element probabilities from embedding-based semantic clusters and uses entropy to guide splitting over-concentrated clusters and merging sparse ones.
- Element Extraction: Conditional mutual information identifies elements with negligible gain for removal and elements that increase task complexity for addition or redefinition.
- Controlled Element Recombination: Controlled recombination anchors generation on a high-information, minimally coupled core element to explore diverse, semantically coherent combinations without contradictions.
- Template-Based Problem Synthesis: Template-based synthesis conditions problem generation on a recombined element set and a template specifying fields such as description, input/output format, and constraints.
- Execution-Grounded Validation: Execution-grounded validation retains only well-defined and solvable tasks after generating reference solutions and test-case generators and executing them in an isolated sandbox.
- Adversarial Solution Space Refinement: Adversarial refinement generates near-miss solutions, measures their erroneous test-passing rate, and iteratively updates the test generator to reduce that rate.
4 Evaluation of Synthetic Data Quality
The evaluation framework assesses synthetic code data across originality, difficulty, diversity, and test quality, then examines validity and test-case effectiveness. ADR shows stronger synthetic-data quality and downstream gains in the reported comparisons.
- The evaluation uses a four-dimensional taxonomy covering originality, difficulty, diversity, and test quality.These dimensions jointly assess novelty, challenge, coverage, and supervision reliability.
- ADR achieves an originality score of 28.91, versus 6.04 for Educational Instruct.
- ADR-only data improves LCB-v5 performance from 16.17 to 20.28, compared with 16.17 to 17.89 for KodCode-only data.The comparison follows 100 RL training steps.
- 90.62% of solutions from 160 valid LCB-v5 samples pass ground-truth tests after one generation round.The samples came from 300 randomly sampled LCB-v5 instances.
- ASSR raises average test cases from 14.75 to 34.78 (+135.8%) and test quality from 72.91 to 81.36 (+11.6%) on 5K ADR tasks.
5 RLVR Experiments
RLVR experiments compare ADR with synthetic-data and real-data baselines across models, domains, and benchmarks. ADR consistently delivers stronger performance and broader exploration than heuristic expansion methods.
- Overall Results: Prior synthetic-data methods fail to surpass original-data performance and can suffer reward saturation, unstable optimization, or degradation below the base model.Educational Instruct averages 20.66% on LCB-v5 and v6 versus 23.23% for TACO, while KodCode reaches 23.16%; on Llama-3.1-8B-Instruct, TACO and Educational Instruct exhibit reward saturation.
- Synthesis Paradigms: In the synthesis case study, heuristic expansion preserves seed structure and alters constraints, whereas ADR produces a significant structural transformation.The comparison concerns examples such as Hamming distance and prefix sums.
- Overall Results: ADR achieves 25.76% overall, surpassing the best baseline’s 23.23% across the reported algorithmic results.On LCB-v5, ADR reaches 25.37% versus 22.75% for the best synthetic-data baseline and 22.60% for the real-data baseline; on LCB-v6, it reaches 26.14% versus 23.86%.
- Sampling and Reasoning: With increased sampling, ADR improves from 28.74% to 33.53% (+4.79%), whereas TACO improves by only +0.60% on LCB-v5 Pass@8.The reported trend is interpreted as evidence that ADR synthesizes data targeting boundary capabilities.
- Generalization: ADR improves performance across base models, reaching gains of 7.57% on Qwen2.5-Coder-7B-Instruct, 7.38% on Llama-3.1-8B-Instruct, and 11.77% on Qwen3-8B.These gains exceed the best baseline improvements of 2.52% and 9.25% on the latter two models, respectively.
- Cross-Domain Generalization: ADR improves tool usage to 41.67% (+3.37%) and data science to 42.44% (+6.16%) over KodCode, extending gains beyond algorithmic programming.The experiments cover algorithms, tool usage, and data science across multiple representative base models.
- RL Training Dynamics: ADR’s cumulative RL improvement is Δ=0.45 versus Δ=0.25 for KodCode, while its Actor KL loss converges near 0.14 versus 0.08.The training dynamics are presented as evidence of a more extensible optimization landscape and deeper policy exploration.
6 Conclusion
The paper concludes that ADR decomposes and recombines atomic code-task elements to move beyond heuristic seed expansion. Its evaluations show improved synthetic-data quality and RLVR performance across algorithmic, tool-usage, and data-science tasks.
- Conclusion: ADR decomposes code tasks into atomic elements and recombines them to generate novel, challenging, and verifiably correct tasks.The framework is presented as a scalable paradigm for synthesizing high-quality code data for future code LLM training.
- Conclusion: ADR improves data diversity and quality while delivering superior RLVR performance across algorithmic programming, tool usage, and data science.The conclusion frames these results as support for scalable synthesis of high-quality code data.
Limitations
ADR’s current evaluation is limited to specific benchmarks and model scales. The authors plan broader validation on larger models, multilingual settings, and code-agent scenarios.
- Limitations: ADR has been evaluated only on specific benchmarks and model scales, leaving robustness at larger scales and in broader settings for future work.Planned extensions include larger foundation models, multilingual environments, and multi-turn code-agent scenarios.
A.1 Evaluation of Synthetic Data Quality
The evaluation taxonomy measures synthetic code-data quality along originality, difficulty, diversity, and test quality. It operationalizes these dimensions using reference-set similarity, model-based difficulty, embedding-space uniformity, and test-case assessment.
- Originality: Originality is measured against 144,169 reference problems using all-MiniLM-L6-v2 embeddings and cosine similarity.The reference set spans Apps, CodeContests, Codeforces, and TACO.
- Difficulty: Difficulty is evaluated with Qwen3-4B, Qwen3-8B, and Qwen3-14B in non-thinking mode.These models form the representative model set for the difficulty dimension.
- Diversity: Diversity uses nearest-neighbor Euclidean distances in embedding space and scores sets by 1 - CV, with higher values indicating more uniform distributions.CV is the standard deviation divided by the mean of nearest-neighbor distances.
- Test Quality: Test quality is included as a separate dimension in the taxonomy for evaluating synthetic code data.The supplied evaluation description identifies test quality as one of the four dimensions.
A.2 ADR-synthesized Data in RLVR Experiments
ADR-synthesized training data is generated separately for algorithmic, tool-use, and data-science tasks through seed selection, controlled recombination, and validation. The experiments retain 5,000 algorithmic examples and 2,000 examples for each of the other two domains.
- Algorithmic tasks: 5,000 valid algorithmic training examples are retained after 8-times controlled recombination and execution-grounded validation.The process starts from 1,710 verified MEDIUM and MEDIUM_HARD TACO problems and filters the synthesized outputs in an isolated sandbox.
- Tool usage: 2,000 tool-use training examples are synthesized from 5,000 randomly sampled Package Instruct examples using one-time controlled recombination and filtering.Tools are filtered to align with the BigCodeBench taxonomy.
- Data science: 2,000 data-science training examples are synthesized from 5,000 randomly sampled Package Instruct examples using one-time controlled recombination and filtering.Retained libraries are aligned with the DS-1000 task taxonomy.
B ADR Prompt Templates
The ADR prompt templates specify staged extraction, recombination, synthesis, and evaluation procedures for generating original algorithmic tasks and assessing their test-case quality. They enforce coherence, originality, format compliance, and coverage across input diversity and edge cases.
- Template organization: ADR documentation maps prompt templates to algorithmic, tool-use, and data-science task stages, including specialized refinement procedures.Figures 6–9 cover algorithmic tasks, Figures 12–15 tool-use tasks, Figures 16–19 data-science tasks, and Figures 10–11 and 20–21 specialized refinement prompts.
- Test quality evaluation: The evaluation prompt generates code that scores test-case coverage from 0 to 1 rather than solving the underlying problem.The evaluator computes exactly three normalized sub-scores, including diversity and non-redundancy, while checking common edge cases such as empty, maximal, boundary, and structurally special inputs.
- Output requirements: Generated solutions and test-case generators must follow fixed output formats and preserve required function signatures.The templates support both stdin/stdout and function-call formats, with self-contained code and a mandatory evaluation-code template.
- Controlled recombination: Generated element sets must preserve coherence among the story background, core algorithm idea, strategy diversity, and difficulty level.The prompt requires constraints to motivate the algorithm, strategy diversity to match its structure, and difficulty to reflect conceptual and technical depth.
- Problem synthesis: The synthesis prompt requires a self-contained, original, challenging algorithm problem that remains faithful to the supplied problem framework.Creative additions are permitted only within the framework’s constraints and core setup.
- Controlled recombination: The recombination prompt treats reference sets as design signals and prohibits exact alignment with any reference along more than one element.This constraint targets novel recombinations rather than direct template reuse.