Source-linked AI summary
Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry
TL;DR
Endpoint pass@1 alone cannot distinguish newly reachable solutions from cheaper sampling, surface robustness, or memorisation. The paper compares post-training paths with cross-surface and complementary diagnostics, finding compression on easy problems but ceiling expansion on hard ones.
Problem
Endpoint pass@1 does not identify whether post-training changes solution reachability, sampling cost, surface robustness, or memorisation, though these possibilities imply different evaluation and deployment conclusions.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
Compression is one regime rather than a universal explanation, and reasoning progress requires surface-aware, trajectory-aware, and membership-controlled diagnostics beyond endpoint pass@1.
Source: §7 · p. 7The comparisons are mechanism-relative, not fully causal, because the studied paths differ in historical pipeline and isolate no single algorithmic factor.
Source: Limitations · p. 7
Abstract
from arXiv · showhide
Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed. Gains may reflect newly reachable solutions, cheaper sampling of latent solutions, surface robustness, or memorisation. We compare three post-training paths under a common diagnostic readout: our sufficiently trained off-policy distillation trajectories, released Qwen3 off-policy-plus-on-policy distillation endpoints, and a released DeepSeek-Math endpoint trained with Group Relative Policy Optimisation (GRPO). Our probe uses cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, plus consistency, distribution-shape, and verified supervised-fine-tuning (SFT) membership analyses. We find two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly compresses sample cost. On harder AIME problems, post-training expands the large-K ceiling over the base model: sufficient off-policy distillation already raises this ceiling, Qwen3 released endpoints raise it further, and DeepSeek-Math GRPO does not dominate sufficient off-policy distillation at large K. English-dominant distillation improves non-English reasoning but preserves language-tier gaps. A controlled-overfit audit finds limited sensitivity in current SFT-membership probes. Compression is one regime of post-training, not a universal explanation.
1 Introduction
Endpoint pass@1 cannot reveal whether post-training makes solutions newly reachable, cheaper to sample, more robust to surface changes, or memorised. A shared diagnostic comparison finds that compression explains easy problems, while hard problems show ceiling expansion.
Motivation
- Endpoint pass@1 cannot distinguish newly reachable solutions from cheaper sampling, surface robustness, or memorisation, despite these explanations implying different evaluation and deployment conclusions.Prior evaluations often isolate one axis, making these explanations difficult to separate under a common protocol.Source: §1 · p. 1
Approach
- The study compares trained off-policy trajectories with released Qwen3 distillation and DeepSeek-Math GRPO endpoints using cross-surface pass@K and complementary consistency, distribution-shape, multilingual, and membership probes.Large-K performance pools attempts at the source-problem level across verbatim prompts, paraphrases, numerical isomorphisms, and translations.Source: §1 · pp. 1–2, Figure 1 · p. 2
Main results
- On AIME, sufficient off-policy distillation raises the cross-surface large-K ceiling over the base; Qwen3 endpoints raise it further, while DeepSeek-Math GRPO does not exceed sufficient off-policy distillation.On AMC, large-K ceilings are near saturation, so post-training mainly compresses sample cost.Source: §1 · p. 2
Robustness and distributional diagnostics
- English-dominant distillation improves every evaluated non-English language but preserves language-tier gaps; Qwen3 consolidates solved problems across variants, unlike the GRPO endpoint’s SFT-like consistency.Form sharpening and answer-content sharpening can diverge, and current SFT-membership probes have limited sensitivity.Source: §1 · p. 2
Contributions
- The contribution is a trajectory-aware probe that separates sample-cost compression, ceiling expansion, and surface robustness while comparing distinct post-training paths.The paper treats sufficiently trained off-policy distillation as a post-training model, not a weak SFT baseline.Source: §1 · p. 2
2 Method
The method compares three post-training mechanisms using shared base checkpoints and a common capability probe. It trains off-policy distillation trajectories and evaluates them alongside released endpoints with cross-surface sampling and distributional diagnostics.
2 Method
- The study evaluates trained off-policy trajectories, released Qwen3 distillation endpoints, and a released DeepSeek-Math GRPO endpoint against shared base checkpoints.The model roster distinguishes nine trained off-policy checkpoints from observed released endpoints.Source: §2 · p. 2, Table 1 · p. 3
2.1 Three post-training mechanisms
- Off-policy distillation trains on static teacher-output pairs, on-policy distillation matches teacher logits on student rollouts, and verifier-based reinforcement learning updates from scored student rollouts.The Qwen3 endpoints use off-policy followed by on-policy distillation; the DeepSeek-Math GRPO endpoint starts from an SFT checkpoint.Source: §2.1 · pp. 2–3
2.2 Our off-policy distillation training
- Off-policy trajectories use filtered OpenR1-Math and, for DeepSeek-Math-7B, a 4k-fitted hybrid corpus; runs use full-parameter fine-tuning with epoch checkpoints.The representative sample is stylized; real samples average approximately 3k tokens of think content.Source: §2.2 · p. 3, Figure 2 · p. 3
2.3 Capability probe set (T0–T3)
- The capability set transforms 100 source problems from AIME 2025, AIME 2026, and AMC 2023 into verbatim, paraphrased, numerically isomorphic, and translated prompts.Paraphrases preserve numerical content and answers; numerical isomorphisms change answers while preserving reasoning structure, and translations test cross-lingual generalisation.Source: Table 2 · p. 3, §2.3 · p. 4
2.4 Sampling and strategy-diversity
- Rollouts use fixed sampling settings and boxed-answer scoring; answer-, response-, and cluster-level descriptors distinguish form-collapse from content-collapse.Descriptors include answer entropy and distinct-answer counts, response similarity and length variation, and single-link clustering.Source: §2.4 · p. 4
3 Cross-mechanism analysis
The cross-mechanism analysis uses pooled cross-surface pass@K and complementary consistency and distribution-shape diagnostics to distinguish sample-cost compression from ceiling expansion and surface robustness. Results differ by difficulty and training path: easy AMC ceilings are near saturation, while harder AIME problems show ceiling expansion, with Qwen3 endpoints combining stronger cross-surface consistency and answer consolidation than the DeepSeek-Math GRPO endpoint.
3 Cross-mechanism analysis
- The shared readout evaluates capability and output-distribution signatures, using cross-surface pass@K over paraphrases, numerical isomorphisms, and translations.The full capability summary covers 15 checkpoints and compares AMC with the harder AIME subset and three post-training mechanisms.Source: §3 · p. 4, §3.1 · p. 4
3.1 Compression vs. ceiling expansion
- On AIME, pass@224 rises 47→77 (4B) and 55→83 (8B) with sufficient off-policy distillation, then reaches 90% at both Qwen3 endpoint scales; DSMath GRPO trails off-policy.The Qwen3 and DeepSeek-Math trajectories are shown in Figures 3 and 4, respectively.Source: Figure 3 · p. 4, Figure 4 · p. 5, §3.1 · p. 5
- AMC pass@224 is near saturation, leaving little endpoint-mechanism separation; where headroom remains, SFT closes most of the gap and DSMath RL is slightly below SFT-ep3.The authors caution that with n=40 the one-standard-error scale is about ±3pp.Source: §3.1 · pp. 4–5
- Qwen3 endpoints improve both pass@1 and the large-K ceiling over sufficient off-policy distillation, whereas the headline pass@1 gain is largely compression, so RL and on-policy distillation are not exchangeable.After on-policy distillation, Qwen3-4B and Qwen3-8B nearly coincide on AIME pass@1/pass@224 (55.6/90 vs. 57.2/90) and AMC (87.6/100 vs. 88.2/100).Source: §3.1 · p. 5
3.2 Multi-instance consistency
- On T1 paraphrases, Qwen3 all-3 consistency rises from Base through SFT-ep3 to Endpoint, while DSMath GRPO remains indistinguishable from SFT-ep3.Qwen3 all-3 reaches 84% for 4B and 87% for 8B; DSMath all-3 is 20% for GRPO versus 21% for SFT-ep3.Source: Table 3 · p. 5, §3.2 · p. 5
3.3 Output-distribution signature: form vs. content sharpening
- The distributional view distinguishes mechanisms with similar endpoint gains: Qwen3 combines ceiling expansion, surface consistency, and answer consolidation, while DSMath GRPO mainly prunes SFT answer support.In this setting, DSMath GRPO produces neither a higher large-K ceiling nor stronger surface consistency.Source: §3.3 · pp. 5–6
- Form sharpening and answer-content sharpening can diverge: off-policy distillation sharpens form, while its answer-support effect differs between Qwen3 and DeepSeek-Math.Qwen3 contracts answer support, whereas DeepSeek-Math broadens it; released Qwen3 on-policy endpoints sharpen both axes, while GRPO contracts support without increasing majority-answer share.Source: §3.3 · p. 5, Figure 5 · p. 6, Table 4 · p. 6
4 Cross-lingual generalisation
The multilingual evaluation pools translated attempts across nine languages for the same source problems. English-dominant SFT lifts non-English performance, but language-tier gaps persist and Swahili remains a pronounced floor.
Cross-lingual generalisation
- The evaluation extends T3’s five translations per source (Chinese, Spanish, Arabic, Urdu, Swahili) with Swedish, Icelandic, and Tamil, using the same translation protocol and K=16 per source-language pair.It combines these with 16 T0 samples per source as the English anchor.Source: §4 · p. 6
- Across all 24 non-English cells, SFT-ep3 lifts every language by +4 to +49pp, including Swahili despite no direct supervision.The largest reported lift is Icelandic on Qwen3-8B, 23→72.Source: §4 · p. 6
- The stronger non-English group beats the weaker group in every family-role cell by +3 to +18pp on mean pass@16, with the tier preserved by every mechanism.The groups are defined by Qwen3-4B Base pass@16; the stronger group is {zh, es, sv, ar} and the weaker group is {is, ta, ur, sw}.Source: §4 · p. 6
- Swahili remains the floor across mechanisms, trailing the next-worst language by 24/17/10pp at the Qwen3-4B/8B/DSMath-7B endpoints.On DSMath-7B, Swahili falls 9→4 from SFT-ep3 to the RL endpoint (−56%).Source: §4 · p. 6
5 Memorisation
A controlled-overfit audit finds that current behavioural and forward-pass probes provide limited sensitivity to SFT-specific memorisation. Its weak signals calibrate probe limits, not the presence or absence of memorisation.
Memorisation
- Across three model families, three training epochs, and three probe modalities, none yields a strong, reliable memorisation signal under the audit conditions.The overfit corpus contains 200 training items per family and a 200-item MD5-disjoint held-out set; 10 and 20 epochs provide an order of magnitude more per-item gradient passes than a typical SFT trajectory.Source: §5 · p. 6
- Forward-pass MIA ROC AUC stays within [.51, .65] across the overfit grid, while behavioural deltas are noisy across families and probe types.The behavioural deltas are omitted from quantitative claims.Source: §5 · p. 6
- The audit is a sensitivity ceiling for current methods and cautions against treating modest MIA AUCs as direct evidence of SFT memorisation.The reported MIA range is below the stronger pretraining-data detection signal cited in the passage: Min-K% Prob reaches AUC 0.88 for copyrighted-book detection (Shi et al. 2024).Source: §5 · p. 6
6 Discussion
Post-training shapes mathematical reasoning through multiple effects that vary with problem difficulty. Easy problems mainly show sample-cost compression, while harder problems show ceiling expansion; robustness, distributional changes, language gaps, and probe sensitivity further distinguish mechanisms.
Discussion
- Endpoint pass@1 conflates cheaper sampling, large-K ceiling expansion, surface robustness, and possible memorisation, which can move differently across mechanisms.The paper frames mathematical post-training as distribution shaping over a difficulty-dependent capability landscape.Source: §6 · pp. 6–7
- On easier AMC problems, near-saturated cross-surface large-K ceilings mean post-training mainly improves sample efficiency; on harder AIME problems, post-training expands the reachable ceiling over the base.Sufficient off-policy distillation already expands the harder-problem ceiling; Qwen3 endpoints add further large-K lift, while DeepSeek-Math GRPO sharpens small-K behaviour but does not exceed sufficient off-policy distillation at large K.Source: §6 · p. 7
- Qwen3 endpoints consolidate solved problems across paraphrases and numerical isomorphisms, whereas DeepSeek-Math GRPO remains close to its SFT predecessor in multi-instance consistency.Form sharpening and answer-content sharpening can diverge, so more uniform reasoning traces alone do not establish more reliable reasoning.Source: §6 · p. 7
- English-dominant distillation improves non-English reasoning but preserves stronger-versus-weaker language gaps, suggesting post-training does not erase inherited coverage asymmetries.The passage also reports that a controlled-overfit audit calibrates the sensitivity of current membership probes; weak measured signals bound the probes, not memorisation itself.Source: §6 · p. 7
7 Conclusion
The paper compares three mathematical post-training mechanisms with shared diagnostics to distinguish sample-cost compression, ceiling expansion, robustness, and memorisation sensitivity. Its conclusion is that compression is regime-bound: easy problems mainly expose sampling-cost gains, while sufficient post-training can expand the reachable ceiling on hard problems.
Conclusion
- The shared diagnostic framework compares off-policy distillation, off-policy-plus-on-policy distillation, and off-policy-plus-GRPO RL using cross-surface pass@K, consistency, distribution-shape, multilingual, and verified SFT-membership probes.The probes distinguish sample-cost compression from ceiling expansion, robustness, and memorisation sensitivity.Source: §7 · p. 7
- Compression is one regime, not a universal explanation: easy problems mainly expose sampling-cost compression, while hard problems show sufficient post-training can expand the reachable ceiling over the base.The conclusion identifies sufficient off-policy distillation as already capable of expanding that ceiling.Source: §7 · p. 7
- Evaluating reasoning progress requires surface-aware, trajectory-aware, and membership-controlled diagnostics beyond endpoint pass@1.Source: §7 · p. 7
Limitations
The comparisons are mechanism-relative, not fully causal, and the multilingual and memorisation analyses have defined scope limits.
Limitations
- The studied post-training paths differ in historical pipeline, so the comparisons isolate no single algorithmic factor.The paper frames its goal as mechanism-relative diagnosis rather than a fully matched causal ablation.Source: Limitations · p. 7
- The nine-language evaluation is affected by varying translation quality, cultural specificity, and benchmark familiarity.The memorisation audit calibrates sensitivity on controlled SFT data, not contamination from pretraining or released-endpoint training data.Source: Limitations · p. 7
Ethical Statement
The work uses public competition problems and public training corpora, and provides reproducibility materials. Its formal cross-surface pass@K estimator pools verified rollouts across selected surface transforms at the source-problem level.
Ethical Statement
- The study uses public AIME and AMC problems and public OpenR1-Math and NuminaMath-CoT corpora, with no human subjects, personal data, or sensitive content.The models are openly released, SFT data is curated from public corpora, and the submission archive includes pools, transforms, rollouts, intermediates, and code.Source: Ethical Statement · p. 7
A Formal definition of cross-surface pass@K
- For each source problem, cross-surface pass@K uses the pool of rollouts from selected transforms and counts outputs verified correct by math_verify.The reported source-level mean is over a subset of problems, with K no larger than the rollout pool; the multilingual table and pooled figures use stated sampling conventions.Source: A Formal definition of cross-surface… · p. 8
B Training details
The appendix documents the SFT-run diagnostics and the scope and layout of cross-mechanism and cross-lingual evaluations.
B Training details
- Training and validation losses, learning-rate schedules, and gradient norms are reported for representative SFT runs; the appendix states that no run diverged.The listed cosine schedule was executed as specified, and gradient norms remained bounded without loss explosion.Source: B Training details · pp. 8–9
C Cross-mechanism analysis
- The cross-mechanism appendix reports pass@K for 15 checkpoints and stratified AIME and AMC values, but no family includes all three mechanisms.Off-policy distillation is represented in all three families; on-policy endpoints are available only for Qwen3 and GRPO only for DeepSeek-Math.Source: C Cross-mechanism analysis · p. 9
D Cross-lingual evaluation
- The multilingual appendix reports aggregated results in Table 9 and per-family breakdowns for Qwen3-4B, Qwen3-8B, and DeepSeek-Math-7B.The per-family displays are Figures 13, 14, and 15, respectively.Source: D Cross-lingual evaluation · p. 9
E Memorisation: positive controls
The appendix describes controlled seen-versus-unseen SFT probes and their limits, alongside detailed benchmark, consistency, multilingual, and training diagnostics. The reported evidence includes weak and noisy membership signals, plus distinct AIME and AMC comparisons of GRPO against off-policy distillation.
E Memorisation: positive controls
- The controlled audit fine-tunes each base on 200 seen items for 10 or 20 epochs, with a 200-item MD5-disjoint unseen set as the comparison.The items come from each model’s canonical SFT corpus and are used to deliberately produce an overfit regime.Source: E Memorisation: positive controls · p. 9
- Three probes test distinct memorisation traces: implicit completion, direct answer recall without reasoning, and forward-pass MIA on ground-truth answer tokens.Behavioral probes use one-sided Mann–Whitney tests on 200 items per split; MIA reports ROC AUC against the verified split label.Source: E Memorisation: positive controls · p. 9
- Behavioral probe values are unreliable because verbatim recitation, truncated reasoning, and ordinary problem-solving co-occur within the same model-checkpoint cells.The authors exclude behavioral values from the headline range and rely on deterministic MIA AUC instead.Source: E Memorisation: positive controls · p. 9
- MIA lacks a randomized-label baseline, and DSMath’s approximately 0.65 Loss AUC is present before overfitting and changes little with epoch.Qwen3 Loss AUC responds to overfit, increasing 3–4 percentage points from epoch 0 to epoch 20.Source: E Memorisation: positive controls · p. 9
- On AIME, GRPO does not exceed SFT-ep3 at any K ≥48; the pass@224 point estimate favors SFT-ep3 by 8 percentage points, but every paired-bootstrap interval crosses zero.The AIME subset contains 60 problems.Source: E Memorisation: positive controls · p. 12
- On potentially contaminated AMC, GRPO gains 5.8 percentage points over off-policy distillation at pass@1, the only contrast with a bootstrap interval excluding zero.Beyond pass@1, the point-estimate ordering reverses at pass@48 and pass@224, but those contrasts remain within intervals that include zero.Source: E Memorisation: positive controls · p. 12
- Across the three consistency panels, Qwen3 endpoints lift all three transforms, while the DeepSeek-Math RL endpoint stays within 4 percentage points of SFT-ep3.The figure uses three instances of 16 samples per source problem, with the T0 pool partitioned to match the per-instance budget.Source: Figure 10 · p. 12
- MIA ROC AUCs stay within 0.51–0.65; Qwen3 Loss AUC rises 3–4 percentage points after 20 overfit epochs, while DSMath starts near 0.65 without overfit and stays flat.The DSMath signal therefore reflects baseline corpus statistics rather than memorisation, while the Qwen3 change is overfit-sensitive.Source: Table 10 · p. 18