PaperlayerView exact revision on arXiv

Source-linked AI summary

What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation

Kanghui Tian, Siyuan Liu, Tianxiang Jiang, Shuai Dong, Yizhuo Li, Tian Ding, Yuan Guo, Songze Li, Haowen Hou, Congcong Wang, Yi Wang

arXiv:2609.25623v1cs.LGcs.AI

TL;DR

In on-policy self-distillation, it is unclear how much of a reference solution a self-teacher should see. The paper compares full solutions with abstract hints and answer-only context, finding that the best intermediate context exceeds the full solution at 4B and 8B while using far fewer hint tokens.

  • Problem

    The study asks what level of reference-solution abstraction best guides a self-teacher, since detailed procedures and sparse cues can favor different continuations.

    Source: §1 · p. 1

  • Method

    The paper compares complete solutions with three offline-compiled abstractions and answer-only context, holding the student view and training setup fixed within each scale.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    The best intermediate context exceeds the full solution by 1.4 and 1.6 in-domain peak points at 4B and 8B, respectively, while storing an order of magnitude fewer hint tokens.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The preferred context varies with student scale and task, and initial teacher–student KL does not reliably indicate peak utility.

    Source: §1 · p. 2, §5 · p. 10

    The comparisons do not isolate semantic abstraction because context packages can also differ in content, length, answer access, and bridge wording.

    Source: §3.5 · p. 5

Abstract

from arXiv · show

More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a frozen copy of the base model scores the student's own rollouts under privileged context, conventionally a complete reference solution that bundles the final answer with one particular reasoning path. Holding the student view and training fixed within each scale, we compare that default against three abstractions compiled offline, a named strategy, a method-independent framing, and a problem category, and against an answer-only control that keeps the destination but removes the path. In the primary runs on competition mathematics, the best intermediate contexts improve the in-domain peak mean over the full solution by 1.4 points at 4B and 1.6 at 8B, while storing an order of magnitude fewer hint tokens. Comparisons across three seeds also show positive mean gains for the framing and category contexts at both scales. Answer-only conditioning remains competitive in the primary runs, within 0.2 points of the full solution at these scales. The preferred context varies with student scale and task. Initial teacher-student KL does not order downstream performance. What a self-teacher should see is therefore not everything it could, but the level of abstraction its student can still act on.

1 INTRODUCTION

The paper studies how the abstraction level of privileged context affects on-policy self-distillation, comparing full solutions with shorter semantic contexts and answer-only conditioning. Its findings show that the best context varies by student scale and task, and that initial teacher–student KL does not predict downstream performance.

Introduction

  • The study compares L1 full solutions, three answer-free abstractions, answer-only L5, and a fixed 10% prefix-plus-answer control while holding training fixed within each scale.The abstractions are named strategy (L2), method-independent framing (L3), and problem category (L4); evaluation covers three competition-math and four transfer benchmarks across three Qwen3 scales.Source: §1 · p. 2
  • 1.4 points at 4B and 1.6 at 8B: the best intermediate context beats L1 in in-domain peak mean while using an order of magnitude fewer hint tokens.Across three seeds, L3 and L4 have positive mean gains over L1 at both scales under all four reporting rules; recompiling contexts with Qwen3-8B preserves the peak advantage at larger scales.Source: §1 · p. 2, Figure 1 · p. 2
  • Answer-only conditioning comes within 0.2 points of L1’s in-domain peak mean at 4B and 8B, but trails by 1.76 points at 1.7B.Source: §1 · p. 2
  • The full solution produces the largest initial teacher–student KL at every scale, yet does not yield the highest in-domain peak mean at 4B or 8B.Item-level analysis finds that context redistributes which problems the trained student solves.Source: §1 · p. 2

2 RELATED WORK

Prior work frames on-policy self-distillation as teacher feedback on student-sampled trajectories and privileged context as a training-time signal unavailable at inference. This study examines which parts of a reference solution to expose and in what form.

On-Policy Distillation

  • On-policy distillation gives token-level teacher feedback on student-policy trajectories; this study uses OPSD with a frozen self-teacher conditioned on a complete reference solution as its baseline.Prior work also refines the objective, conditions self-teachers on demonstrations, or combines them with external teachers.Source: §2 · p. 3

Privileged Information and Reasoning Guidance

  • Privileged information gives a same-checkpoint teacher an advantage through training-time-only signals, including feedback, traces, instructions, recoverable cues, and pseudo-solutions.Such guidance can enter through partial references during generation or step-level process supervision; context distillation internalizes prompt-induced behavior into parameters.Source: §2 · p. 3

Reference Exposure

  • Prior reference-exposure work studies prefix exposure, conditioning effects, abstract hints, and selected path structure; this study trains without explicit solution steps across three adjacent abstraction levels.Shrestha & Tessier (2026) compare method-level abstract hints, while Armandpour et al. (2026) find no universally best context using a training-free gradient-alignment proxy rather than trained outcomes.Source: §2 · p. 3

3 PRIVILEGED CONTEXT AS A CONTROLLED INTERVENTION

The section defines context packages as the intervention in on-policy self-distillation and evaluates their utility after training. It specifies how contexts are constructed and tested, and qualifies what the comparisons can establish.

3.1 PROBLEM FORMULATION

  • A context policy supplies the frozen teacher with a prompt built from the problem, a reference-derived representation, and bridge wording, while the student receives only the problem.Utility is defined after training under a chosen benchmark and reporting rule, with the shared initialization also serving as the frozen teacher checkpoint.Source: §3.1 · pp. 3–4

3.2 HOW CONTEXT ENTERS ON-POLICY SELF-DISTILLATION

  • The frozen teacher scores prefixes of student-sampled trajectories under privileged context, changing supervision targets while the student view and primary training setup remain fixed.The rollout procedure is fixed, but trajectories can diverge as students learn under different contexts; context changes can alter both magnitude and direction of the update.Source: §3.2 · p. 4

3.3 CONSTRUCTING MATCHED SEMANTIC CONTEXTS

  • L1 supplies the full solution; L2–L4 provide a method, method-independent framing, or problem category without the answer, while L5 supplies only the final answer.L2–L4 are newly written, staged abstractions rather than truncations of L1, and the primary contexts are compiled offline with Qwen3.5-397B-A17B.Source: §3.3 · pp. 4–5

3.4 UTILITY ESTIMATION AND DIAGNOSTIC TESTS

  • L1, L3, and L4 receive three-seed in-domain comparisons at all three scales, whereas L2, L5, transfer evaluations, and supplementary controls are single runs.The section tests full-solution dominance, discrepancy-based ordering, and context invariance; discrepancy is measured on shared base-model problem–prefix pairs, not as a direct measure of teacher quality.Source: §3.4 · p. 5

3.5 INTERVENTION VALIDITY AND SCOPE

  • Matched training recipes isolate context within each primary comparison, but package differences and changing student recipes across scales limit causal claims about abstraction or scale alone.Packages may differ in content, length, answer access, and bridge wording; across scales, student and training recipe change together, including effective batch size.Source: §3.5 · p. 5
  • L2–L4 hints average 38.9, 25.2, and 17.7 tokens versus 657.3 for L1, a 16.9–37.1× reduction in stored hint content.Including the problem and bridge, prompts average 225.5, 212.7, and 203.3 tokens for L2–L4 versus 835.0 for L1, a 3.7–4.1× reduction.Source: §3.5 · p. 5
  • Answer-equivalent leakage is 1.42% at L2 and 0.33% at L3 for the primary compiler, versus 4.17% and 2.58% for Qwen3-8B; L4 rates are 0.08% and 0.17%.Audits found no empty hints, generation-error sentinels, lexical answer matches, or direct answer copying in the matched 1,200-problem sample.Source: §3.5 · p. 6

4 EXPERIMENTS

In matched scale-specific OPSD runs, intermediate contexts exceed the full solution at 4B and 8B, while the full solution leads at 1.7B. Answer-only context remains competitive, and preferred contexts vary across tasks.

4.1 CONTEXT UTILITY VARIES WITH STUDENT SCALE AND TASK

  • The best intermediate context exceeds the full solution by 1.39 points at 4B and 1.57 at 8B, while the full solution leads by 0.28 at 1.7B.At step 200, L2 and L4 each reach 62.59 against L1’s 60.93 at 4B, while L3 reaches 65.37 against L1’s 65.28 at 8B.Source: §4.1 · p. 6
  • Transfer winners vary by task, and answer-only L5 stays within 0.2 points of L1’s ID peak mean at 4B and 8B.At 1.7B, L3 leads on MT-AIME and GPQA while L1 leads on ZebraLogic; L5 trails L1 by 1.76 points at 1.7B, and on 8B ZebraLogic L3 scores 88.10 against 85.90 for L5.Source: §4.1 · p. 7, Table 2 · p. 7

4.2 ROBUSTNESS AND ALTERNATIVE EXPLANATIONS

  • Across three seeds, L3 and L4 have positive mean paired gains over L1 at 4B and 8B under all four checkpoint rules; compiler replication preserves the scale-dependent ordering.At 1.7B the mean paired peak differences are −0.12 for L3 and 0.00 for L4, while selection-free rules favor L1 on average. The 8B compiler’s L4 leakage is 0.17%.Source: §4.2 · pp. 7–8
  • Prefix truncation peaks below L2–L4, shared bridge wording and doubled rollouts retain intermediate-at-least-L1 peaks, and mixtures or routing never beat the best fixed-level peak.At 4B, the prefix reaches 63.89, with L3 and L4 ahead by 0.93 and 0.74 points; under a shared bridge, intermediate contexts exceed L1 by 1.0–2.3 ID peak points.Source: §4.2 · p. 8

4.3 DISTRIBUTIONAL DISCREPANCY DOES NOT ORDER CONTEXT UTILITY

  • L1 has the largest initial mean KL at every scale, yet intermediate contexts achieve higher 4B and 8B ID peak means with smaller discrepancies, so KL does not order utility.On answer-aligned rollouts, the answer-onward L5−L1 KL difference is −0.100 at 1.7B, +0.043 at 4B, and +0.177 at 8B; this is a localized distributional comparison, not answer correctness.Source: §4.3 · p. 8, Figure 2 · p. 8

4.4 CONTEXT CHANGES WHICH PROBLEMS ARE SOLVED

  • Intermediate contexts recover some L1-unsolved items and improve medium-bucket accuracy, while easy-bucket effects and ZebraLogic net gains vary with scale.With Base-defined difficulty, medium-bucket differences across six MT-AIME and GPQA scale–task pairs are +1.14 to +10.12 points, while easy-bucket differences range from −0.77 to +1.65; ZebraLogic’s net accuracy difference shifts from −0.70 at 1.7B to +1.70 at 8B.Source: §4.4 · p. 9, Figure 3 · p. 9
  • Backtracking and explicit-correction shifts differ by scale, so the pooled behavior does not show invariant changes or support a consistent behavior-based explanation for gains.The item-level and observable-behavior analyses at step 200 do not identify what produces the separately selected peak gains.Source: §4.4 · p. 9, §4.5 · p. 9

4.5 INTERPRETATION AND LIMITS

  • The analyses characterize item-level success and observable reasoning at step 200, but do not establish what produces the separately selected peak gains.Teacher and student capacity covary through the shared base checkpoint, so varying the compiler does not separate teacher elicitation from the student’s ability to learn from its targets.Source: §4.5 · p. 9

5 CONCLUSION · A TRAINING AND EVALUATION DETAILS

With the pipeline otherwise fixed, privileged-context abstraction changes which contexts train the strongest students. The preferred context varies by scale and task, so reference representation is a design choice to evaluate by the students it trains.

5 CONCLUSION

  • Named strategy, method-independent framing, and problem-category contexts change which contexts train the strongest students compared with the complete reference solution.Source: §5 · pp. 9–10
  • By 1.4 points at 4B and 1.6 at 8B, an abstraction leads the full solution in primary-run in-domain peak results.At 1.7B, the full solution keeps the primary-run lead.Source: §5 · p. 10
  • Framing and category contexts retain positive seed-mean differences at 4B and 8B under every reporting rule.Source: §5 · p. 10
  • Answer-only conditioning stays within 0.2 points of the full solution at 4B and 8B.Source: §5 · p. 10
  • Teacher–student divergence is largest for the full solution at every scale, but does not track peak utility.Source: §5 · p. 10
  • The abstraction that leads shifts with student scale and task, making the reference representation a design choice best evaluated by the students it trains.Source: §5 · p. 10

A.1 TRAINING CONFIGURATION

The training configuration defines a clipped, full-vocabulary teacher–student objective and separates rollout, teacher-scoring, and evaluation settings. It also fixes the teacher model and compute setup across runs.

Training objective

  • At valid completion positions, teacher and student logits share scoring temperature γ and are normalized over the full vocabulary; the configured generalized-divergence endpoint is β = 0.Source: §A.1 · p. 13
  • The objective clamps vocabulary components before summing, so it is a clipped teacher–student distribution objective rather than exact forward KL.Prompt and padding positions are masked, and the clamp thresholds follow the original OPSD configuration for each scale while remaining identical across context conditions.Source: §A.1 · p. 13

Training and evaluation settings

  • Rollouts use temperature 1.1, top-p 0.95, and top-k 20; training uses Qwen3’s non-thinking template, teacher scoring its thinking template, and evaluation enables thinking mode.Generation truncation does not change the full-vocabulary loss; the fixed base-model teacher has LoRA disabled, and runs use one node with 8 NVIDIA H200 GPUs.Source: §A.1 · p. 13

A.2 EVALUATION PROTOCOL … B.2 TEACHER INTERFACES

The sections define peak-score and transfer-evaluation protocols and operationalize five teacher-context levels through staged hint generation and level-specific teacher interfaces, while keeping the student prompt fixed.

A.2 EVALUATION PROTOCOL

  • The in-domain table selects each model-condition-benchmark triple’s best score through step 200; this maximum across eight correlated evaluations is upwardly biased versus a single-checkpoint score.Unadapted Base models are evaluated once without checkpoint selection, and Appendix C.3 quantifies the gap between the peak and step-200 score.Source: §A.2 · p. 13
  • Transfer evaluation spans translated AIME problems, executable-verifier arrangement tasks, graduate science questions, and logic-grid puzzles, with benchmark-specific averaging or accuracy metrics.The cited passage specifies equal language weighting and Avg@12 for MT-AIME2024-7Lang, one generation per AutoLogi-EN problem, Avg@10 for GPQA-Diamond, and exact puzzle accuracy as ZebraLogic-grid’s primary metric.Source: §A.2 · p. 14
  • C-Eval is reported separately from the four reasoning-transfer benchmarks because its result depends on completing reasoning within the output budget.It serves as a broad Chinese knowledge and bounded-generation audit.Source: §A.2 · p. 14

B.1 CONTEXT GENERATION AND A WORKED EXAMPLE

  • L1–L4 form progressively abstract roles—from complete solution to category—while L5 retains only the answer; these are operational boundaries, not literal containment.L2 keeps a concrete method, L3 a method-independent framing, and L4 only the problem category.Source: §B.1 · pp. 14–15
  • L2–L4 hints are generated in staged calls from L4 to L3 to L2, each using the problem and a truncated reference solution; matched semantic audits assess both compilers on 1,200 problems.The primary compiler uses Qwen3.5-397B-A17B, while the Qwen3-8B replication changes model and decoding mode together, testing a deployable compiler package rather than isolating parameter count.Source: §B.1 · pp. 15–16
  • L4 classifies what a problem asks without method cues, L3 gives one transferable framing shift, and L2 names a specific method without procedural steps or an answer.The prompts specify separate self-checks and typical lengths of 10–30, 20–60, and 30–90 tokens for L4, L3, and L2, respectively.Source: §B.1 · pp. 15–16

B.2 TEACHER INTERFACES

  • Teacher prompts use a fixed template with a level-specific hint label, delimiters, and transition; L1–L4 openings differ by verb chain, while L5 directs derivation toward a known answer.A shared-bridge control replaces the level-specific elements with one template, and routed mixtures reuse them row by row.Source: §B.2 · p. 16
  • The student-side message is identical across conditions and never includes the hint, separating the student view from the privileged context supplied to the teacher.The student is prompted to reason step by step and place the final answer within a box.Source: §B.1 · p. 14, §B.2 · p. 16

B.3 CONTEXT AUDITS · C FULL RESULTS AND ROBUSTNESS

The context audit measures hint lengths across aligned training rows and checks answer leakage and level-boundary compliance through a stratified, adjudicated sample.

B.3 CONTEXT AUDITS

  • 29,434 aligned, nonempty rows had no error sentinels, and L5 included seven proof-target overrides.Table 6 reports exact Qwen3-token lengths for all rows, but the supplied passage does not include its cell values.Source: §B.3 · p. 17, Table 6 · p. 17
  • The audit distinguishes stored-hint length from the privileged increment, which includes the hint and its level-specific bridge but excludes the problem and shared chat-template tokens.These reductions describe the teacher-scoring interface once a solution exists, not solution acquisition or end-to-end wall-clock cost.Source: §B.3 · p. 17, Table 6 · p. 17
  • Semantic audits sampled 1,200 stratified problem rows shared by both compilers, with 3,600 hints audited per compiler across three levels.Sampling was stratified by answer type and reference-solution length quartile; the deterministic scan covered all 29,434 rows from each compiler.Source: §B.3 · pp. 17–18, Table 7 · p. 18
  • Leakage labels separate direct answers from answer-equivalent conclusions that determine the answer without the intended solution, while useful strategies alone are not leakage.The audit also allowed an uncertain label when evidence was genuinely ambiguous.Source: §B.3 · pp. 18–19
  • Boundary checks require L2 to name a method without executing it, L3 to give one method-independent framing, and L4 to identify the domain and relevant objects without recommending action.Table 7’s “Core role valid” measure checks concrete-method presence for L2, single-framing presence for L3, and domain and object correctness for L4.Source: §B.3 · pp. 18–19, Table 7 · p. 18
  • All 2,400 compiler–problem records were resolved after two independent prompt passes and third-prompt adjudication, with no unresolved cases.All passes used one model, so agreement reflects prompt replication rather than human inter-rater agreement; for primary-compiler L4, κ = 0 was a prevalence artifact despite 99.83% raw agreement.Source: §B.3 · p. 18, Table 8 · p. 18

C.1 FULL EVALUATION RESULTS · C.2 CONTEXT CONTROLS

The section reports transfer, ZebraLogic, and C-Eval evaluations, then describes two controls that vary how privileged contexts are produced while holding the training pipeline fixed. The supplied passages give evaluation scopes and metrics but no result values for comparing conditions.

C.1 FULL EVALUATION RESULTS

  • Table 9 covers step-200 transfer across four benchmarks in single-run comparisons; multi-seed analysis is limited to the in-domain L1, L3, and L4 comparison.The transfer metrics are MT-AIME seven-language macro Avg@12, AutoLogi verifier accuracy, GPQA Avg@10, and ZebraLogic exact puzzle accuracy.Source: Table 9 · p. 20, §C.1 · p. 20
  • The transfer comparisons use a fixed step-200 checkpoint and the primary compiler for L2–L4, while L5 is the answer-only control.The reported transfer scores are percentages.Source: Table 9 · p. 20
  • Table 10 expands ZebraLogic results to exact-grid puzzle accuracy, individual-cell accuracy, and exact-puzzle accuracy on the Hard and XL subsets.It also reports absolute percentage-point change from the same-scale unadapted Base checkpoint.Source: Table 10 · p. 20
  • Table 11 reports a separate C-Eval audit with Macro, Micro, Hard Macro, and extraction-failure metrics under a 32,768-token output budget.Macro averages 52 subjects; extraction failures count as incorrect in Macro and Micro.Source: §C.1 · pp. 20–21, Table 11 · p. 21

C.2 CONTEXT CONTROLS

  • Table 12 regenerates L2–L4 with Qwen3-8B under the same semantic contracts to test whether full-versus-intermediate context ordering holds with a second compiler.The passage describes the control's purpose but provides no outcome.Source: §C.2 · p. 21
  • Table 13 compares semantic rewriting with a fixed 10% reference prefix plus the answer at 4B, shortening context without abstracting it.The comparison differs from L2–L4 in both context length and answer access; no result is supplied here.Source: §C.2 · p. 21

C.3 TRAINING ROBUSTNESS

Robustness checks preserve the advantage of intermediate contexts over L1 at 4B and 8B across additional seeds and a doubled rollout budget, but the preferred intermediate level and fixed-endpoint outcomes vary. Checkpoint performance is nonmonotonic, so peaks can materially exceed step-200 results.

C.3 TRAINING ROBUSTNESS

  • With a doubled rollout budget, peak means change by at most 0.74 points, and every intermediate context again matches or exceeds L1; the nominal leader shifts from L3 to L4.At step 200, L2 falls from 62.59 to 58.61, despite its peak mean increasing from 64.07 to 64.54; these are single-seed 4B runs without transfer evaluation.Source: §C.3 · pp. 21–22, Table 14 · p. 22
  • At 4B and 8B, L4 exceeds its matched L1 peak in all three seed configurations, while L3 does so in four of six scale–configuration cells.Aggregating runs under four checkpoint rules preserves the direction at both scales, including when all checkpoints are averaged.Source: §C.3 · p. 22, Table 15 · p. 23
  • At 1.7B, peak differences range from −1.11 to +0.37 points, and selection-free rules favor L1 by 0.14 to 1.33 points on average.Among L3 and L4, the leader is unstable within each larger scale; the replications cover only L1, L3, and L4, while L2, L5, transfer evaluations, and remaining controls are single runs.Source: §C.3 · pp. 22–23, Table 15 · p. 23
  • Across 45 triples, the mean and median best-checkpoint-to-step-200 gaps are 2.31 and 1.94 points; the largest is 5.83 for 1.7B-L4 on AIME25.That maximum compares 43.06 at step 125 with 37.22 at step 200; benchmark curves are nonmonotonic, and selected peak steps are spread across the schedule.Source: §C.3 · p. 23, Figure 6 · p. 24, Table 17 · p. 25

C.4 SCOPE OF THE CONTROLS · D DIAGNOSTIC ANALYSES · D.1 DISTRIBUTIONAL DISCREPANCY

The controls qualify the scope of the context comparisons, while temporal diagnostics show that the answer-onward L5–L1 contrast changes sign by student scale. The subset and evaluation design impose limits on those findings and motivate broader validation.

C.4 SCOPE OF THE CONTROLS

  • Three-seed replication covers L1, L3, and L4 at all scales, but prefix, bridge, and rollout-budget controls center on 4B, while mixture and routing studies are single runs.The in-domain protocol selects the best of eight checkpoints separately per benchmark; selection-free endpoints and full trajectories are reported in Appendix C.3.Source: §C.4 · p. 25
  • The controls do not fully isolate bridge wording, semantics from length, compiler parameter count, difficulty-selection bias, or judge independence.The prefix control fixes truncation with answer access but matches L2–L4 in neither length nor answer access; Qwen3-8B stays within the Qwen family, and both compilers receive the reference solution. Difficulty stratification still risks selection bias, and one judge supplies semantic audits and trajectory annotations.Source: §C.4 · p. 25
  • Broader validation should test other model families and domains, disentangle semantics from length, and assess compilers that construct useful contexts without reference solutions.The passage also motivates multi-round OPSD with online context selection and stability-aware optimization because in-domain scores do not improve monotonically across checkpoints.Source: §C.4 · p. 25

D.1 DISTRIBUTIONAL DISCREPANCY

  • Each scale’s diagnostic sampled 600 aligned problems and two continuations per problem, then scored each fixed continuation under the student prompt and teacher prompts with L1–L5 packages.The unadapted model generated continuations from the exact OPSD student prompt with thinking disabled; the same base checkpoint scored them under the alternative prompts.Source: §D.1 · p. 25
  • The answer-aligned subset favors completed trajectories: 40.7–43.7% of rollouts were truncated, and most truncated trajectories lacked a boxed answer.There were 610, 636, and 630 valid rollouts at 1.7B, 4B, and 8B, respectively; most truncated trajectories were absent from the subset.Source: §D.1 · p. 26
  • −0.193, +0.030, and +0.188 are the L5 −L1 answer-onward contrasts at 1.7B, 4B, and 8B, respectively, preserving a scale-dependent sign change among apparently correct rollouts.The normalized answer-string check uses 421, 510, and 499 valid rollouts at the respective scales and remains an outcome-composition diagnostic.Source: §D.1 · p. 26

D.2 ITEM-LEVEL AND BEHAVIORAL ANALYSIS

Item-level and behavioral analyses show that ZebraLogic rescue from intermediate contexts increases with scale, while response length does not move uniformly with gains. A contrast case illustrates how correct constraint representation can close a puzzle where L1’s misreading leaves it underdetermined.

Item-level analysis

  • Difficulty-conditioned changes are measured relative to L1, but easy-bucket declines and unsolved-bucket recovery are partly expected from regression to the mean because buckets use L1 success rates.The analysis therefore repeats the grouping using Base success rates.Source: §D.2 · p. 27, Table 18 · p. 27
  • Selected intermediate responses are 4–10% longer than L1 on MT-AIME and GPQA but 3.7% shorter on 8B ZebraLogic, ruling out a uniform length effect.Length may still contribute on individual tasks.Source: §D.2 · p. 27
  • ZebraLogic rescue rates rise from 18% to 35% of L1-unsolved puzzles with scale, turning a net loss at 1.7B into a net gain of +17 puzzles at 8B.Figure 8 compares the strongest intermediate condition with L1 at step 200; regressions shrink as scale increases.Source: Figure 8 · p. 28

Behavioral analysis

  • In the 8B ZebraLogic contrast case, L3 parses the “two houses between” clue as positional distance three and closes the grid, while L1 misreads it and guesses between unresolved alternatives.L1 verifies clues against its misreading and invokes uniqueness despite two unresolved cells; its trajectory is twice as long as L3’s and contains more contradiction mentions.Source: §D.2 · p. 29

E CONTEXT ASSIGNMENT

Mixtures and per-sample policies did not surpass the best fixed context on the in-domain peak mean, although several improved the step-200 endpoint. Shared wording preserved the separation between L1 and intermediate contexts, while transfer rankings depended on the reference.

E CONTEXT ASSIGNMENT

  • No mixture or per-sample policy exceeds the best fixed level on the 4B ID peak mean, including routing by exact per-step likelihood advantage.The comparison covers the mixture and adaptive-policy results; the likelihood-advantage router scores all four hinted contexts plus the hint-free teacher at every step.Source: Table 20 · p. 30, E CONTEXT ASSIGNMENT · p. 30, Table 21 · p. 31
  • Mixtures assign exactly one hint per problem under deterministic L1–L4 assignments, while per-sample policies route each trajectory using a learned classifier, likelihood advantage, or correctness rules.Mixtures use 29,434 rows and are trained once at each scale for 200 steps; routed mixtures reuse those assignments but apply each row’s source-level wrapper, so routing is deterministic prompt construction rather than a learned or test-time router.Source: E CONTEXT ASSIGNMENT · p. 30
  • Offline, the hint advantage Δℓ had a 4.5 median between- to within-level variance ratio and a per-problem argmax stable across rollouts 78% of the time.The absolute student–teacher log-probability gap separated levels weakly at 0.306–0.325 nats, while teacher mass on the student’s top-k support exceeded 0.978 even at k=4; Δℓ’s per-level means were negative throughout and lowest for L1.Source: E CONTEXT ASSIGNMENT · p. 30
  • With shared teacher wording, L2–L4 still exceed L1, but the nominal leader shifts from L3 to L4 and the exact L2–L4 ranking is not stable.The shared bridge changes only the teacher-side wrapper and transition; the student view, contexts, training problems, optimizer settings, checkpoint schedule, and evaluators remain fixed.Source: E CONTEXT ASSIGNMENT · p. 30, Figure 11 · p. 31
  • MidMix reaches 69.52 on MT-AIME, above shared-bridge L2 at 69.40 but below primary-bridge L4 at 69.56; no mixture leads on GPQA.The transfer comparison therefore changes with the reference context.Source: E CONTEXT ASSIGNMENT · p. 30, Table 22 · p. 31
  • At step 200, five of six routed mixtures beat the corresponding fixed-level endpoint by 0.37–0.84 points, but reversed routing lowered peak mean by 1.48 points.The learned and likelihood-advantage routers exceeded fixed L3’s endpoint by 0.93 and 1.11 points, respectively, without changing the best observed peak; these single runs suggest reduced stopping-point sensitivity, not a higher attainable peak.Source: E CONTEXT ASSIGNMENT · p. 31, Figure 11 · p. 31, Table 22 · p. 31

Related papers

Loading 2609.25623v1…