Source-linked AI summary
X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests
Jie Wu, Haoling Li, Xin Zhang, Jiani Guo, Jane Luo, Steven Liu, Yangyu Huang, Ruihang Chu, Scarlett Li, Yujiu Yang
TL;DR
Competitive programming lacks enough reliable real-world data for scalable Code LLM training, motivating the question of whether fully synthetic data can reach expert-level reasoning. The paper develops domain-specific feature evolution and dual verification, then trains X-Coder with SFT-then-RL, reporting stronger benchmark performance and insights into synthetic-data scaling and code-centric reinforcement learning.
Problem
Finite, heavily used real-world competitive-programming datasets limit scalable reasoning training and raise contamination concerns, while synthetic data quality remains difficult to ensure.
Method
The paper advances feature-based synthesis with domain-specific evolution and dual verification, then trains X-Coder using only synthesized data under SFT-then-RL.
Results
X-Coder-7B outperforms its foundation model by 51.3 points in pass@16 and matches Qwen3-8B with 8× fewer rollouts.
Takeaways & Limitations
Fully synthetic, rigorously verified tasks, solutions, and tests offer a viable alternative to scarce real-world data for training reasoning-focused Code LLMs.
Takeaways & Limitations
Verification introduces significant computational overhead, despite potentially more resource-efficient training from unverified long-CoT data.
Abstract
from arXiv · showhide
Competitive programming poses a significant challenge for Code LLMs. While recent models have shown promise, they heavily rely on finite real-world data, raising concerns about scalability and contamination. In this paper, we investigate a critical question: Can we elevate models to expert-level reasoning performance using fully synthetic data? In response, we first observe that off-the-shelf synthesis methods yield suboptimal results in this domain. To address this, we systematically investigate the key factors governing synthetic data quality. Leveraging these findings, we significantly advance the feature-based synthesis paradigm via domain-specific evolution and a dual-verification strategy, promoting task solvability, solution correctness, and test accuracy. Using this high-quality synthetic data, we train the X-Coder model series under an SFT-then-RL paradigm. X-Coder-7B shows significant performance gains on the challenging LiveCodeBench v5 (62.9% avg@8) and v6 (55.8% avg@8), outperforming larger models trained on real-world data. Extensive analysis distills valuable insights into synthetic data scaling, the necessity of domain-adapted feature evolution, and code-centric reinforcement.
1. Introduction
Competitive programming exposes a data-scarcity bottleneck for Code LLMs because finite real-world tasks are insufficient for scaling and raise contamination concerns. X-Coder addresses this with domain-adapted, dual-verified synthetic data and SFT-then-RL training.
- Competitive programming requires deep algorithmic reasoning and complex problem-solving beyond largely solved basic programming benchmarks.
- Finite real-world competitive programming data is insufficient for reasoning-model scaling and raises concerns about contamination and overfitting.
- Synthetic competitive-programming data must provide solvable, challenging tasks, logically sound solutions, and reliable tests for SFT and RL.
- Domain-specific evolution and dual verification address synthesis quality by establishing a high-quality synthetic foundation for training.
- X-Coder trains only on synthesized data under SFT-then-RL and achieves significant gains on LiveCodeBench v5 and v6, especially medium and hard splits.
- The study distills insights into synthetic-data scaling, domain-specific feature evolution, code-centric reinforcement learning, and fine-grained failure modes.
2. Related Work
Prior synthesis methods address coding-data scarcity but general-purpose approaches perform suboptimally on competitive programming. X-Coder adapts feature-based synthesis with domain-specific evolution and dual verification for code reasoning.
- Prior work evolves coding instructions, composes concepts, or uses coder–reviewer interaction to synthesize instruction–code data.
- General-purpose synthesis techniques yield suboptimal results when applied to competitive programming.
- The proposed domain-tailored strategy generates complex tasks through feature evolution and verifies solution correctness with a dual-verification process.
- Competitive-programming post-training includes purely supervised, purely reinforcement-learning, and hybrid SFT-then-RL paradigms.
3. Synthesis of Competition-Level Coding Data
The synthesis framework generates competitive-programming tasks, tests, and candidate solutions, then uses dual verification to produce trusted assets for SFT and RL. Domain-specific feature evolution and diverse input generation target the domain’s difficulty and coverage requirements.
- Framework overview: The framework evolves domain-specific features into a feature tree, formulates multi-style tasks, synthesizes solutions and tests, and verifies both before SFT and RL.
- Task Generation: Feature-based synthesis is adapted by extracting competition-related features and supporting Codeforces-, LeetCode-, and AtCoder-style task generation.
- Test Input Generation: Test inputs are generated through prompting or the CYaRon tool, with standard, boundary, and edge-case coverage targeted explicitly.
- Candidate Solutions Generation: Multiple candidate solutions are generated and filtered for complete reasoning, Python implementations, and syntactic validity using AST-based analysis.
- Dual-Verification: 94.7% labeling accuracy is achieved with 8 sampled solutions in consensus-based test-case verification on TACO.
- Dual-Verification: Candidate solutions are selected using weighted performance on challenging tests and confirmed on an unseen hold-out set to reduce overfitting.
- Task Validation: Approximately 36.9% of generated tasks are filtered when GPT-5 obtains a zero pass rate on voted tests, supporting task solvability screening.
- Post-training: Verified solution–task pairs support SFT, while verified task–test-suite pairs provide RL data through GRPO.
4. Experiment
The experiments evaluate X-Coder on LiveCodeBench and examine how synthetic-data scaling, SFT quality, and RL affect performance. Results favor broader task diversity, verified synthetic data, and stronger SFT initialization for effective RL.
- Evaluation: 62.9 avg@8: X-Coder further improves after RL over its SFT stage on LiveCodeBench v5.X-Coder-SFT reaches 60.3 avg@8, while X-Coder reaches 62.9; on Qwen3-Base, X-Coder reaches 64.0 avg@8.
- SFT Experiments and Analysis: Task diversity outperforms solution diversity under a fixed computational budget.The performance hierarchy v2 (64k×1) > v5 (16k×4) > v6 (8k×8) favors scaling unique tasks over solutions per task.
- SFT Experiments and Analysis: 6.7 points: fully synthetic data improves over OpenCodeReasoning after SFT using the same number of training tokens.Most gains come from the medium and hard splits, and the paper reports reduced contamination risks relative to real-world datasets.
- RL Experiments and Analysis: 4.6% absolute gain: RL improves the average pass rate when applied to a converged SFT model trained only on code data.The paper describes RL as an optimization stage that can explore policy beyond the initial supervised-data distribution.
- RL Experiments and Analysis: Stronger SFT initializers consistently achieve higher rewards under identical RL settings.The experiments associate stronger initialization with a more promising policy space and a higher performance ceiling.
- RL Experiments and Analysis: RL benefits from synthetic test cases despite imperfect feedback, indicating resilience to noisy supervision.The paper reports that this lowers the barrier to collecting code-RL data at scale.
5. Ablation Study
The ablations identify verification, reasoning length, domain-specific evolution, and rationale-based selection as important factors in synthetic competitive-programming data quality.
- Verification: Verified solutions significantly outperform raw solutions, establishing dual-verification as critical for synthetic-data quality.Verifying 200k samples requires 1.6M CoT trajectories and 24M executions.
- Reasoning Length: 17.2% absolute gain comes from long-CoT solutions, although training converges in 8–10 epochs rather than 2–3.The comparison uses an identical 200k-task set.
- Domain Adaptation: 20.9% improvement over EpiCoder and 4.6% over domain-adapted SelfCodeAlign support domain-specific feature evolution for competitive programming.The proposed strategy extends feature-based synthesis with competition-oriented feature extraction and evolution.
- Data Selection: Tasks inducing longer CoT are treated as more valuable, providing a practical heuristic for pruning data under resource constraints.The selection study compares difficulty-based, rationale-based, and random sampling.
6. Discussion
The discussion links failures to reasoning difficulty, context limits, and execution efficiency, while showing that synthetic training improves test-time scaling and produces distinct post-training behaviors.
- Failure Analysis: Wrong answers are the primary failure type, followed by no-code outputs and time-limit errors.No-code cases exceeded the 32k context window, while TLEs emphasize execution efficiency.
- Difficulty and Reasoning: Higher difficulty is associated with longer reasoning, which is strongly negatively associated with pass rate.The paper summarizes this mediation pattern as higher difficulty → longer reasoning length → lower pass rate.
- Test-Time Scaling: X-Coder-7B exceeds its foundation model by 51.3 points in pass@16 and matches Qwen3-8B with 8× fewer rollouts.Its pass@16 minus pass@1 gap is 19.2, compared with 13.8 for Qwen3-8B.
- Training Behaviors: SFT distills planning and verification, while RL adds performance gains but can produce strategic exploitation of edge cases for partial rewards.Persistent inefficiencies are also observed during later training stages.
7. Conclusion
The paper presents fully synthetic competitive-programming training through verified tasks, solutions, and tests, and reports performance gains from SFT-then-RL training. It also distills findings on synthetic-data scaling, domain-specific evolution, and code-centric reinforcement learning.
- Rigorously verified synthetic tasks, solutions, and tests can train large reasoning models for competitive programming.
- Training and Evaluation: SFT trains models on task–solution pairs and full long-CoT trajectories, covering reasoning steps and final code.
- Training and Evaluation: PPO stabilizes policy optimization by clipping probability-ratio updates, while GRPO removes the critic network and estimates advantages from multiple rollouts.
- Training and Evaluation: The reward emphasizes test-case performance: non-compiling code receives −2, compiling code that passes no tests receives 0, and other outcomes receive 5.0 × #passed/#total.
- Training and Evaluation: Continuous rewards provide denser supervision than all-or-nothing rewards and lead to faster convergence.
A.4. Training Configs and Costs
The training setup uses SFT followed by RL, with substantial compute requirements and a distributed verification infrastructure. Novel task synthesis combines evolved competitive-programming features with multiple task styles.
- Training Configs: RL uses batch size 128, learning rate 7e-5, temperature 1.0, and 8 rollouts to encourage exploration.
- Training Costs: Training large reasoning models is costly because long sequences and multiple RL rollouts increase compute consumption.
- Training Costs: X-Coder training required 128 H20 GPUs for 220 hours during SFT and 32 H200 GPUs for 7 days during RL.
- Verification Infrastructure: The code-verification framework uses an asynchronous API gateway, sandboxed execution workers, and Redis to coordinate distributed validation.
- Verification Infrastructure: The verification system supports high throughput, low latency, horizontal worker scaling, and fault detection through asynchronous Redis-based components.
- Novel Task Synthesis: Feature-based synthesis evolves competitive-programming concepts across breadth and depth to increase diversity and complexity.
- Novel Task Synthesis: The synthesis process supports Codeforces-, LeetCode-, and AtCoder-style tasks, while feature extraction targets algorithms, data structures, implementation logic, and complexity.
B.2.2. FROM FEATURE TO STYLIZED TASK
The task-generation pipeline separates feature selection from task formulation and evaluates styles, difficulty, and diversity. The two-stage design improves task quality, while the generated datasets show greater embedding separation than Evol-Instruct-Code.
- Two-Stage Generation: Separating feature selection from task generation avoids the overly simple problems produced by one-step prompting.
- Two-Stage Generation: The two-stage pipeline selects a feature subtree and integration strategy before formulating a stylized task.
- Two-Stage Generation: A 5.3 gain on LiveCodeBench v5 supports explicit subtree selection and integration over one-step generation.
- Task Styles: The study compares AtCoder-, Codeforces-, and LeetCode-style corpora generated from identical input features, adopting Codeforces-style tasks predominantly.
- Difficulty and Diversity: The difficulty estimator reaches 84% validation accuracy when predicting Codeforces ratings from annotated CodeContests tasks.
- Difficulty and Diversity: Synthetic-task clusters have average centroid distance 0.613 versus 0.507 for Evol-Instruct-Code, indicating greater embedding-space separation.
- Solution Quality: Solutions are filtered for complete reasoning tags, valid Python ASTs, single code blocks, and bounded length.
C.2. SFT Dataset Statistics
The demonstration dataset contains long task–solution examples and uses comprehensive test-case generation prompts. Its token lengths primarily follow a normal distribution with a median of 16k.
- Dataset Statistics: The demonstration dataset’s overall token-length distribution is primarily normal, with a median of 16k.
- Dataset Statistics: The dataset statistics are summarized in Table 11 and Figure 16.
- Test-Case Generation: Test-case generation requests 15 cases spanning edge, small-scale, medium-scale, and large-scale inputs to validate solution correctness.
D.2. Comparison of Test Generation Methods
Tool-based test generation produces more diverse, accurate, challenging, and broadly covering tests than prompting-based generation. The framework labels candidate outputs by voting, weights test cases, and uses weighted selection with hold-out confirmation.
- Test-generation comparison: Tool-based generation supports random, scalable, boundary, and stress tests, providing broader test diversity and coverage.CYaRon supplies data structures and utilities for generating these test types.
- Test-generation comparison: 87.9% vs. 77.4% pass rate on ground-truth solutions shows that tool-based tests are more accurate than prompting-based tests.The tool-based approach also achieves a lower consensus ratio, 78.8% vs. 82.0%, indicating more discriminative tests, albeit with higher computational cost.
- Dual verification: Weighted selection chooses the candidate solution maximizing agreement on weighted golden tests, then confirms it on a randomly held-out validation set.The procedure returns the selected golden solution and test suite only when the hold-out winner matches the weighted-selection winner.
- Test-case weighting: Semantic weighting assigns weights of 1–4 to nominal, complex, boundary, and stress cases, whereas size weighting ranks inputs by file size.The synthesis pipeline adopts size-based weighting, assigning weights 1 through 4 to quartiles of input-file size.
- Dual verification: 94.7% labeling accuracy with 8 sampled solutions produces candidate test outputs before weighting and verification.Majority voting compares sampled outputs for each input, while boundary and edge cases receive special attention through weighting.
E.4.1. ERROR RATE OF GOLDEN SOLUTION
The dual-verification procedure estimates golden-solution error on synthetic and real-world data while using solver performance to assess task solvability. On TACO-verified, it reports a 7.85% error rate at n = 8, with errors decreasing as more solutions are sampled.
- Evaluation design: The evaluation combines synthetic pass-rate distributions with real-world error measurement against ground-truth tests.Synthetic evaluation measures selected-solution pass rates on voted tests, while TACO-verified evaluation measures actual errors on retained ground-truth tests.
- Synthetic verification: 23.66% of selected solutions pass all voted test cases, while 13.39% attain pass rates in the [80, 100) interval.These percentages describe the distribution of golden-solution pass rates on voted tests.
- Task solvability: 66.98% of tasks are solved perfectly in one attempt by GPT-5-High, providing a strong-solver estimate of task solvability.The analysis uses pass@1 on voted test cases from several proprietary LLMs as a proxy for potentially unsolvable problems.
- Real-world verification: 7.85% error rate in selected golden solutions at n = 8 on TACO-verified indicates generally reliable dual verification.The error rate decreases further as the number of rollout solutions increases.
- Generality: 13.4 gains after SFT and 15.3 after RL on Llama-3.1-8B-Instruct show benefits beyond the Qwen model family.The reported progression from 11.8 to 25.2 to 27.1 suggests weaker base models can also benefit from the datasets.
- Scope: The evaluation scope centers on competitive programming, with MBPP+ and HumanEval+ reported additionally for generality.EvoEval, ClassEval, and DS-1000 are outside the study’s primary scope.
G. Data Leakage Analysis
The paper contrasts fully synthetic X-Coder training with real-world-data training and examines reasoning behavior in successful and failed solutions. The analysis identifies leakage-related artifacts and several failure modes that affect code-generation reliability.
- Leakage comparison: X-Coder, trained entirely on fully synthetic data, is presented as having lower data-leakage risk than models trained on real-world data.Real-world-trained models perform strongly on older benchmarks but are less competitive on recent LiveCodeBench versions.
- Successful reasoning: SFT solutions frequently exhibit planning, verification, backtracking, and reflection, suggesting these behaviors can be distilled from the teacher.Successful reasoning examples include defining a recurrence, checking prefixes and boundary cases, and reorganizing computations with divisor-based methods.
- Successful reasoning: The successful solution reduces naive O(m) per-step computation by reorganizing sums over divisors and using Möbius inversion with precomputation.The resulting complexity depends on divisor counts, with average counts around 10–20 and worst-case counts around 128.
- Failure analysis: The analysis identifies three failure modes: premature termination under context exhaustion, retrieval-like translation of memorized code, and incomplete code emission.These behaviors can produce rushed, non-derived, or truncated programs.
- RL behavior: Later RL rollouts sometimes exploit edge cases for partial rewards instead of solving problems robustly.Other rollouts contain irrelevant advertising or editorial-style content, suggesting artifacts inherited from competition-platform pretraining.