Shorter but not Worse: Frugal Reasoning via Easy Samples as Length Regularizers in Math RLVR

1 Introduction

• Filtering easy problems leaves training focused on longer-chain tasks, shifting output lengths upward and encouraging verbosity even when tokens are redundant.The passage links this imbalance to models conflating “thinking longer” with “thinking better.”Source: §1 · p. 1
• Retaining and upweighting moderately easy problems acts as an implicit length regularizer, without explicit reward shaping.The approach exposes the policy to tasks with concise solutions under a limited context window.Source: §1 · pp. 1–2
• With a 16k-token budget, Qwen3-4B-Thinking-2507 preserves baseline pass@1 AIME25 accuracy while nearly halving average solution length.Source: §1 · p. 2
• The findings identify data curation as an important factor in reasoning-model efficiency, alongside reward design and model size.Source: §1 · p. 2

2 Prelude

• RLVR optimizes the expected verifier score, while GRPO estimates reward using groups of responses to the same query and relative advantages.The work uses binary verifiable rewards, with 1 denoting a correct solution.Source: §2 · p. 2
• When all rollouts for a problem are correct or all are incorrect, their advantages are zero, so those groups provide no gradient signal.This is why easy and unsolvable hard problems are typically excluded from RLVR training.Source: §2 · p. 2
• Filtering out easy problems biases training toward medium and hard tasks that require longer chains, and the output length distribution can rise over successive updates.The passage reports that models may produce unnecessarily long solutions even when concise reasoning would suffice.Source: §2 · p. 2
• Conditioning on a longer response prefix cannot increase uncertainty about the final answer, and the section argues this can bias unpenalized policies toward longer completions.The proposed account applies where rewards depend solely on final-answer correctness, particularly on medium or hard instances.Source: §2 · p. 3

Data curation stage 1: emergent brevity

• Retaining problems with p < 1 supplies frequent positive rewards from concise, solvable trajectories, implicitly regularizing output length under a fixed context window.The Stage 1 dataset deliberately contains more moderately easy problems, whose stable rewards contrast with very hard problems that may be truncated or fail verification.Source: §3 · p. 3

Reward shaping

• The verifiable reward is binary: it is 1 when the normalized extracted answer matches the normalized ground truth, and 0 otherwise.The final answer is enclosed in \boxed{} for deterministic parsing and verification.Source: §3 · p. 3

Data curation stage 1: emergent brevity

• At a 16k-token budget, the empirical success-rate distribution has large masses at p = 0 and p = 1 but relatively few intermediate-difficulty problems; some p = 0 instances become solvable with longer budgets and are retained in Stage 1.Figure 1 relates token counts to empirical success rate and isolates the nontrivial, solvable difficulty region; Figure 2 examines 256 p = 0 problems with a 42k-token maximum response length.Source: §3 · p. 4, Figure 1 · p. 4, Figure 2 · p. 4

Data curation stage 2: improvement via curriculum RLVR

• The second stage trains on a filtered DeepMath-103 subset, progressively adding harder instances as policy competence improves while retaining the 16k-token limit.Difficulty filtering leaves 14.5k examples, retaining domain-specific difficulty levels from the first level with less than 75% success.Source: §3 · p. 4
• Curriculum RLVR on the filtered DeepMath-103 subset performs best compared with random shuffling.The subset retains difficulty levels beginning with the first level below 75% success in each math domain.Source: §3 · p. 4

4.1 Experimental Setup

• Training uses GRPO and proceeds through 1,050 Stage 1 optimization steps followed by Stage 2 curriculum RLVR; the 4B model receives 255 Stage 2 steps.All stages share a fixed 16k-token generation limit and identical optimization hyperparameters; the 30B MoE model trains for one Stage 2 epoch and converges with fewer updates.Source: §4.1 · p. 5

4.2 Training Dynamics

• Early Stage 1 generations are often truncated at the 16k-token limit, with response-length clipping above 15%; average length and clipping then decrease as minimum length increases.The training-dynamics analysis focuses on the 4B dense model; the 30B MoE model exhibits qualitatively similar dynamics.Source: §4.2 · p. 5
• Entropy falls early, stabilizes, then rises slightly around steps 400–600 without a return to the initially excessive output lengths.The authors interpret the response-length and entropy pattern as emergent brevity: uncertainty is reduced through more efficient reasoning rather than longer sequences.Source: §4.2 · pp. 5–6
• AIME25 validation accuracy rises from roughly 33% to about 70% during Stage 1 as the policy produces concise outputs.In Stage 2, the minimum response length increases to approximately 1,200 tokens with the increased difficulty.Source: §4.2 · p. 6

5.1 Reasoning Benchmarks

• The evaluation covers mathematical and STEM reasoning as well as instruction following, reporting standard accuracy alongside EAA.The listed benchmarks include AIME25, Omni-MATH-Hard, MATH-500, GSM-Plus, and GPQA-Diamond.Source: §5 · p. 6, §5.1 · p. 6

Instruction Following

• IFEval tests precise adherence to textual instructions using 500 prompts with verifiable outcomes.Source: Instruction Following · p. 6

5.2 Metrics

• EAA combines pass@k or accuracy with mean output length, remains bounded in [0, 1], and penalizes longer outputs more strongly as γ increases.The length term is linearly rescaled to the unit interval.Source: §5.2 · p. 6

5.3 Results

• 68.55% accuracy / 61.22 EAA for 4B Stage 2 beats its base (61.72 / 32.38); 30B-A3B Stage 2 scores 71.43 / 50.11 versus 65.82 / 42.15.Both comparisons are averages under the 42k-token decoding limit.Source: Table 3 · p. 7, §5.3 · p. 7
• On AIME25 and Omni-Hard, 4B Stage 2 uses 55–61% fewer tokens at comparable or higher accuracy, while GSM_PLUS outputs are 5.6% longer with a small EAA regression.Qwen3-4B-Thinking-2507 averages 11,491 tokens per sample, versus 6,270 for Stage 1 and 5,712 for Stage 2.Source: Table 4 · p. 7, §5.3 · pp. 7–8
• At 8k and 16k AIME25 budgets, Frugal 4B models reach accuracy close to or exceeding larger models; baselines improve at 32k and 42k but retain lower EAA.Table 5 reports AIME25 accuracy by context length.Source: §5.3 · p. 8

• The paper attributes verbosity partly to RLVR practices that overemphasize hard problems requiring long reasoning chains.Source: §6 · p. 8
• Retaining and modestly upweighting moderately easy problems exposes the policy to concise, solvable trajectories and implicitly regularizes output length.Source: §6 · p. 8
• On AIME25, the method matches baseline pass@1 accuracy while producing solutions that are nearly twice as short.Source: §6 · p. 8
• The conclusion frames data selection as a practical route to more efficient reasoning models without adding explicit length constraints.Source: §6 · p. 8
• Future work may test the approach in coding or logical reasoning, explore adaptive easy–hard curricula, and combine implicit with explicit regularization.Source: §6 · p. 8

• The study's scope is math reasoning tasks with verifiable rewards, evaluated on dense 4B and MoE 30B models.Source: Limitations · p. 8
• The information-theoretic arguments motivate reduced verbosity, but the emergence of brevity lacks a complete theoretical characterization.Source: Limitations · p. 8