Source-linked AI summary
CoDiQ: Test-Time Scaling for Controllable Difficult Question Generation
Zhongyuan Peng, Caijun Xu, Changyi Xiao, Shibo Hong, Eli Zhang, Stephen Huang, Yixin Cao
TL;DR
CoDiQ addresses scarce, costly, and difficult-to-control generation of challenging reasoning questions by combining test-time scaling, difficulty-enhancement strategies, iterative hybrid verification, and reinforcement learning. It produces a 44K-question competition-grade corpus, with high-quality generation and improved downstream reasoning performance, while remaining limited by verifier capacity and English math/code scope.
Problem
High-quality questions that genuinely stress reasoning are scarce, while human-driven construction is expensive and difficult to scale.
Method
CoDiQ combines six difficulty-enhancement strategies, test-time scaling, iterative difficulty and solvability verification, and reinforcement learning to generate progressively challenging valid questions.
Results
44K competition-grade math and coding question sequences were constructed, and CoDiQ-Generator maintained high validity across iterative evolution while improving downstream reasoning performance.
Takeaways & Limitations
CoDiQ supports budget-driven difficulty stratification and curriculum learning with automatically generated questions that surpass zero-shot generation baselines.
Takeaways & Limitations
The pipeline is restricted to English math/code tasks, incurs verification costs, and risks discarding valid problems that exceed the fixed-capacity verifier.
Abstract
from arXiv · showhide
Large Reasoning Models (LRMs) benefit substantially from training on challenging competition-level questions. However, existing automated question synthesis methods lack precise difficulty control, incur high computational costs, and struggle to generate competition-level questions at scale. In this paper, we propose CoDiQ (Controllable Difficult Question Generation), a novel framework enabling fine-grained difficulty control via test-time scaling while ensuring question solvability. Specifically, first, we identify a test-time scaling tendency (extended reasoning token budget boosts difficulty but reduces solvability) and the intrinsic properties defining the upper bound of a model's ability to generate valid, high-difficulty questions. Then, we develop CoDiQ-Generator from Qwen3-8B, which improves the upper bound of difficult question generation, making it particularly well-suited for challenging question construction. Building on the CoDiQ framework, we build CoDiQ-Corpus (44K competition-grade question sequences). Human evaluations show these questions are significantly more challenging than LiveCodeBench/AIME with over 82% solvability. Training LRMs on CoDiQ-Corpus substantially improves reasoning performance, verifying that scaling controlled-difficulty training questions enhances reasoning capabilities. We open-source CoDiQ-Corpus, CoDiQ-Generator, and implementations to support related research.
1 Introduction
CoDiQ addresses the scarcity and expense of scalable, high-difficulty question construction by combining test-time scaling with solvability control and relative difficulty measurement. The resulting 44K-question corpus supports improved reasoning training and is released with the generator and implementations.
- High-quality questions that genuinely stress reasoning are scarce, while expert-driven construction and validation are expensive and difficult to scale.
- Scaling difficult-question generation is constrained by generator capacity, a solvability–complexity trade-off, and the absence of measurable difficulty control.
- CoDiQ introduces test-time scaling to systematically increase question difficulty while targeting well-posed and solvable outputs.
- Six Difficulty-Enhancement Strategies guide high-difficulty synthesis beyond zero-shot generation baselines.
- 44K competition-grade mathematical and coding question sequences form CoDiQ-Corpus, and training on them significantly enhances large reasoning models’ reasoning capabilities.
- CoDiQ-Corpus, CoDiQ-Generator, and all implementations are released to support future research.
2 Related Works
Prior work uses prompt-driven pipelines or trained generators to synthesize difficult questions, but CoDiQ instead makes test-time scaling the central mechanism for controllable difficulty under solvability verification.
- Difficult-question synthesis is motivated by the need for frontier-level supervision and controlled generalization testing beyond scarce human-curated problems.
- Prompt-based and agentic methods bootstrap from seeds or solution structures, then refine candidates using self-critique, filtering, and verification.
- Generator-training methods amortize multi-step synthesis costs and align specialized models toward solvability and difficulty.
- CoDiQ scales difficulty at inference time while enforcing correctness through automated verification rather than relying on filtering.
3 Method
CoDiQ iteratively evolves seed questions into harder variants, estimates relative difficulty, verifies solvability, and stops when difficulty regresses or validity fails. Reinforcement learning further targets the model’s capability boundary and balances validity with progressive difficulty.
- Difficulty-Enhancement Strategies: Six strategies inject structured difficulty through constraints, abstraction, inversion, state explosion, theorem disguise, and edge-case rigor.
- CoDiQ Pipeline: The CoDiQ Pipeline evolves a seed question through up to Tmax = 8 rounds, retaining only valid variants whose difficulty does not decrease.
- Verification and stopping: Difficulty Estimation and Solvability Verification jointly regulate complexity and validity, with invalid or non-monotonic candidates discarded.
- Difficulty control: Relative ranking and normalized scores address saturation in absolute difficulty estimates and support adaptive computation allocation.
- Evaluation: CoDiQ-Bench contains 200 cases sampled across coding and mathematical datasets, intentionally emphasizing relatively simple questions for baseline evaluation.
- Reinforcement learning: CoDiQ-Generator is trained with reinforcement learning on capability-boundary failures, using solvability, difficulty progression, and validity signals.
- Reinforcement learning: The difficulty-aware reward balances validity guarantees with progressive difficulty scaling and treats unsolvable, repetitive, or regressive cases as invalid.
4 Experiments
Experiments evaluate CoDiQ across model backbones, token budgets, generation rounds, solvability, human quality, and downstream training utility. Results show that CoDiQ increases difficulty while preserving solvability more effectively with its RL-trained generator, and that CoDiQ-Corpus improves reasoning performance.
- Prompt effectiveness: CoDiQ Prompt expands reasoning-token usage and improves generated-question difficulty for most baseline models.The prompt induces extended reasoning trajectories that enable more intricate constraints and logic.
- Generator comparison: CoDiQ-Gen-8B outperforms Qwen3-32B on high-complexity question generation.RL alignment optimizes solvability and difficulty progression, allowing more iterative rounds before unsolvability causes termination.
- Scaling with computation: r = 0.8299 and r = 0.8545 show positive correlations between token consumption and DR-LLM and DR-VN difficulty rankings, respectively.Token volume is a reliable difficulty proxy when problem complexity remains within the evaluator’s capability.
- Budget-controlled scaling: CoDiQ-Gen-8B yields higher difficulty scores than baselines across all cumulative token budgets, while CoDiQ Prompt improves Qwen3-8B over direct prompting.The budget includes generation and verification; exceeding it discards the iteration and retains the highest-difficulty valid preceding problem.
- Difficulty and solvability: Across eight rounds, difficulty generally rises, but large models tend to plateau as token use approaches generation-capacity or evaluator limits.Solvability simultaneously degrades with increasing difficulty, with smaller models experiencing validity collapse and CoDiQ-Gen-8B avoiding this pattern through RL.
- Corpus effectiveness: CoDiQ-Corpus is significantly harder than existing competition-level datasets, and curriculum training with it improves reasoning-model performance.The corpus evaluation samples 300 questions per dataset, while downstream evaluation covers MATH-500 and AIME 2024.
5 Conclusion & Limitations
CoDiQ provides a framework for synthesizing verifiable, high-difficulty reasoning problems at scale through test-time scaling and hybrid verification. Its corpus supports curriculum learning, but the approach remains limited by task scope, verification cost, and verifier capacity.
- CoDiQ synthesizes verifiable, high-difficulty reasoning problems at scale.
- Test-time scaling addresses the generator capacity ceiling, while hybrid verification mitigates unsolvable “fake hard” instances.
- CoDiQ-Corpus uses budget-driven difficulty stratification and improves reasoning models through curriculum learning.
- The scope is restricted to English math and coding tasks, and verification cost limits real-time use.
- A fixed-capacity verifier creates an epistemic ceiling that may discard valid problems beyond its capabilities.
Impact Statement
The paper frames scalable difficulty enhancement as a way to build richer reasoning data, illustrated through progressively more constrained subsequence problems. Increasing state complexity raises computational demands and can eventually make generated tasks infeasible or vacuous.
- CoDiQ aims to decouple problem complexity from human curation while maintaining logical validity in synthetic reasoning data.
- The pipeline demonstrates iterative difficulty escalation from an initial easy problem, with solvability verification and difficulty assessment at each step.
- A basic odd-sum subsequence task uses simple parity-tracking dynamic programming in O(n) time.
- Adding parity, length-parity, and modulo-3 conditions expands the dynamic-programming state space to 12 states.
- Solvability Score: 0.90 for the 12-state formulation, with O(12n) time complexity and approximately 8% solution density.
- CRT simplification produces a 120-state formulation with Solvability Score: 0.85 and O(120n) time complexity.
A.4.2 Verification
Verification prevents difficulty escalation from producing computationally infeasible or vacuously trivial problems. In the illustrated case, the pipeline terminates before retaining a candidate whose theoretical difficulty exceeds practical solvability.
- This exceeds practical competitive-programming limits and creates an extremely sparse solution space.
- The probability that a random subsequence satisfies all conditions is approximately 1/18,480, with expected valid subsequences of 2^n/18,480.
- For small to moderate n, the answer is almost always 0, making the problem vacuously trivial.
- The pipeline terminates at Round 3, discards Q3, and outputs {Q0, Q1, Q2} after detecting unsolvability.
B CoDiQ Pipeline: Failure Type Analysis
The failure analysis clusters pipeline failures into validity breaches and difficulty degradation, then characterizes the model and Value Network used to assess generated questions. Unsolvability primarily reflects missing information or logical conflicts, while solvable questions often become too easy.
- Failure Analysis: The analysis uses K-means pre-clustering, TF-IDF keyword extraction, hierarchical clustering, and manual refinement.
- Failure Categories: The two dominant failure dynamics are validity breaches labeled Unsolvable and complexity degradation labeled Difficulty Decreased.
- Unsolvable Scenarios: Most unsolvable cases arise from missing definitions or information combined with constraint and logic conflicts.
- Difficulty Preservation: Difficulty Decreased contains N = 12,916 cases in which problems remain solvable but fail to meet the intended cognitive demand.
- Data Construction: The dataset samples easy and hard classes at a 2:3 ratio and uses an 85:15 train-test split.
- Value Network: Value Network inputs are hidden states sampled from up to 4,096 generated tokens, with higher density near the generation onset.
- Value Network: The Value Network projects 4,096-dimensional hidden states to a scalar correctness score using an MLP trained with weighted binary cross-entropy.
- Evaluation: The configuration and evaluation are documented in Tables 6–9, with class imbalance addressed through positive weighting in the loss.
C.5 Performance Evaluation
CoDiQ-Corpus quality is assessed through solvability, difficulty, and general quality criteria, with expert review used to validate selected problems.
- Selection Criteria: The benchmark applies three selection criteria: solvability, appropriate difficulty level, and overall quality.Quality checks cover clarity, correctness, and relevance, while solvability requires at least one valid solution.
- Selection Criteria: Solvability requires each problem to be well-defined and admit at least one valid solution.
E Statistics of CoDiQ-Corpus
CoDiQ-Corpus transforms eight mathematical and programming datasets into approximately 44,453 progressively difficult question sequences.
- Dataset Statistics: Approximately 44,453 question sequences comprise CoDiQ-Corpus after transformation with CoDiQ-Gen-8B and the CoDiQ Pipeline.The sequences span progressive difficulty from easy to hard.
- Dataset Statistics: Table 10 reports the dataset statistics for CoDiQ-Corpus.
F Scaling Tendency Analysis Details
Across eight rounds and both prompting settings, increased reasoning computation corresponds to greater generated difficulty, while difficulty and solvability trade off. The figure compares normalized difficulty rankings and solvable rates across Long-CoT models.
- Scaling Patterns: Increased reasoning computation correlates with higher problem difficulty across all evaluated Long-CoT models.
- Scaling Patterns: A difficulty–solvability trade-off appears across models, while larger models maintain a better balance between the two metrics.
- Figure 6: Figure 6 reports normalized average difficulty rankings and solvable rates across eight generation rounds under Direct Prompt and CoDiQ Prompt settings.Higher rankings indicate greater difficulty, and higher rates indicate better question quality.
- Figure 6: The scaling analysis evaluates both difficulty and solvability as complementary indicators of generated-question quality.
G.4 Evaluation Protocol
The evaluation protocol combines expert solvability annotation, progressive-difficulty curriculum construction, weighted answer-quality rewards, and structured LLM-based difficulty grouping. CoDiQ’s construction process applies explicit redesign and review steps to produce harder but coherent problems.
- Solvability Evaluation: Three PhD-level experts independently evaluate 300 sampled problems, marking solvability when at least two agree; Fleiss’ κ reaches 0.78.Disagreements are resolved through discussion.
- Curriculum Construction: Curriculum data uses sequences of length |S| ≥3 and assigns initial, intermediate, and final questions to Levels 1, 2, and 3.These levels represent progressively increasing difficulty.
- Curriculum Construction: The curriculum enforces Difficulty(L1) < Difficulty(L2) < Difficulty(L3) with training exposure ratio |L1| : |L2| : |L3| = 2 : 2 : 1.L1 and L2 are duplicated during training to balance exposure.
- Answer Evaluation: The reward function combines problem resolution, reasoning correctness, information completeness, and accuracy using r = wpr · spr + wrc · src + wic · sic + wacc · sacc.Default weights are wpr = 0.20, wrc = 0.35, wic = 0.25, and wacc = 0.20.
- Difficulty Evaluation: LLM difficulty grouping evaluates knowledge complexity, cognitive load, computational complexity, traps, and integration skills, ordering groups from easiest to hardest.The output uses matched result and reason arrays with 0-based question indices.
- Question Construction: The construction protocol analyzes the original problem, selects enhancement categories, redesigns constraints and objectives, reviews standards, and formats the final statement.