SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation

1 INTRODUCTION

• OPSD provides dense full-vocabulary next-token targets from a frozen solution-conditioned self-teacher on student-generated prefixes, but discards the aligned final hidden states.The paper asks whether hidden supervision helps through privileged-information orientation or merely because an auxiliary gradient regularizes the student.Source: §1 · pp. 1–2
• SCOPE-OPSD projects the aligned final-layer teacher-student residual through a frozen rank-64 factor estimated from privileged-gap covariance and language-model-head Fisher geometry.It reuses OPSD’s aligned forwards and adds no rollout or inference-time component.Source: Figure 1 · p. 2, §1 · p. 2
• The spectrum- and gradient-matched Random control tests data-dependent orientation, while component controls place GapOnly between Random and Structured and favor rank 64 over tested alternatives.The reported component comparisons also include full-state matching.Source: §1 · p. 2
• At step 75, Structured exceeds matched Random by 1.39, 0.37, and 0.93 percentage points on Qwen3-1.7B, 4B, and 8B, and exceeds Pure OPSD by 1.85, 1.67, and 0.83 points, respectively.The cross-fitted offline test also finds 4.40× greater held-out privileged-gap capture than the matched random control.Source: §1 · p. 2

2 BACKGROUND AND PROBLEM FORMULATION

• OPSD samples a student trajectory for a question and minimizes token-averaged forward divergence using a frozen privileged self-teacher that also receives the verified solution.The student predicts from the question and its preceding generated tokens.Source: §2 · p. 3
• The final pre-head hidden states are aligned at each token, but the output projection leaves open whether the privileged residual carries training information beyond next-token matching and which directions merit supervision.The states are viewed through the frozen language-model head for output matching.Source: §2 · p. 3
• Equal scalar coefficients do not ensure equal parameter-update magnitudes because hidden-loss operators can differ in spectrum and alignment with the model Jacobian.The paper therefore calibrates initial auxiliary gradient magnitude separately to test orientation apart from hidden-supervision strength.Source: §2 · p. 3

3.1 A MINIMAL FINAL-LAYER AUXILIARY

• The final-layer auxiliary adds a projected residual loss to OPSD using the same token positions and forwards, with no extra model forward, rollout, reward model, or inference-time state.Only the student LoRA is updated.Source: §3.1 · p. 3

3.2 A GAP- AND SENSITIVITY-CONDITIONED FACTOR

• The frozen rank-64 Structured factor combines privileged-residual covariance with a smooth Fisher-based spectral filter that suppresses output-invisible and most strongly output-sensitive directions.The factor is fitted once from calibration statistics and remains frozen during training.Source: §3.2 · pp. 3–4

3.3 MATCHED ORIENTATION CONTROL

• Matched Random preserves the Structured factor’s rank and nonzero spectrum while using an independent orientation, and each arm’s coefficient is calibrated to the same initial hidden-gradient RMS.Pure OPSD sets the auxiliary coefficient to zero; Structured and Random are alternative orientations rather than additive components.Source: §3.3 · p. 4

3.4 LOCAL INTERPRETATION OF THE STRUCTURED FACTOR

• The offline diagnostic tests held-out privileged-gap capture by the fitted factor, while downstream evaluation separately tests whether its local orientation improves autoregressive reasoning.Source: §3.4 · p. 5

Training data and OPSD recipe

• Training uses the 29,434-example OpenThoughts Math OPSD split and runs Qwen3-1.7B, 4B, and 8B for 100 steps, saving checkpoints at 25, 50, 75, and 100.The student samples non-thinking responses; the frozen step-zero teacher receives the verified solution and the same student prefix.Source: §4 · p. 5

Scale-specific calibration

• At each model scale, the rank-64 factor is calibrated from 128 prompts, with no factor transfer or padding between models.The auxiliary coefficient is also recalibrated per scale while rank, training schedule, and evaluation protocol stay fixed.Source: §4 · p. 5

Compared systems

• Pure OPSD, matched Random, and Structured share the training split, rollout configuration, frozen teacher, adapter, optimizer, and output objective; Random and Structured start with matched hidden-gradient RMS.The comparison adds no rollout samples, optimizer steps, or privileged inputs beyond Pure OPSD.Source: §4 · p. 5
• A 500-step local GRPO run is included as post-training context, not as a control for the Pure–Random–Structured comparison.GRPO changes both objective and schedule.Source: §4 · p. 5

Ablation variants

• The Qwen3-1.7B ablations compare covariance-only GapOnly, full-state FullHidden, and complete Structured factors at ranks 32, 64, and 128.These variants retain the OPSD objective and evaluation protocol and do not retune the frozen cross-scale comparison.Source: §4 · p. 5

Checkpoint policy

• The primary cross-scale comparison uses step 75, chosen from the Qwen3-1.7B trajectory, and all saved checkpoints are also reported to characterize convergence.Every benchmark within a comparison uses the same checkpoint; benchmark-wise checkpoint oracles are not constructed.Source: §4 · p. 5

External context

• External OPSD and PHF results retain their original checkpoint-selection labels, while method deltas are computed only between locally matched methods.The original OPSD paper reports benchmark-wise best values and a separate single step-100 Qwen3-1.7B value.Source: §4 · p. 5

Evaluation

• Evaluation covers AIME 2024, AIME 2025, and HMMT 2025 with 30 problems per benchmark and 12 samples per problem; Macro is the unweighted mean across benchmarks.Each checkpoint covers 1,080 generations, and only complete evaluations with zero failed requests enter the tables after provenance and configuration checks. Table 1 covers the matched step-75 comparison; Table 2 covers Qwen3-1.7B Avg@12 at step 75.Source: §4 · p. 6, Table 1 · p. 6, Table 2 · p. 6

5.1 THE FIXED STEP-75 COMPARISON TRANSFERS ACROSS SCALES

• At step 75, Structured is strongest at all three scales, beating Pure OPSD by 0.83–1.85 points and matched Random at each scale.Even against Pure OPSD’s best saved checkpoint, fixed step-75 Structured is higher by 1.85, 1.21, and 0.65 points at 1.7B, 4B, and 8B.Source: §5.1 · p. 6

5.2 STRUCTURED ORIENTATION IMPROVES OVER MATCHED RANDOM AT 1.7B

• At step 75, Structured beats matched Random by 0.56, 1.81, and 1.81 points on AIME24, AIME25, and HMMT25, yielding a +1.39-point Macro gain.The gain is not driven by a single benchmark; Structured also maintains a higher Macro score at steps 50, 75, and 100.Source: §5.2 · p. 6

5.3 A LOCAL GRPO CURVE ANCHORS THE POST-TRAINING CONTEXT

• Structured scores 43.33 Macro Avg@12, 3.89 points above the local GRPO checkpoint’s 39.44.This is contextual, not a matched causal delta: GRPO uses a different objective and a 500-step schedule, while OPSD arms share a 100-step recipe.Source: §5.3 · p. 7

5.4 COMPONENT AND RANK CONTROLS SUPPORT SELECTIVE STRUCTURE

• The component ordering is Random < GapOnly < Structured (41.94 < 42.41 < 43.33), while rank 64 exceeds tested ranks 32 and 128 by 0.72 and 1.06 points.FullHidden averages 41.82, below Random by 0.12 and Structured by 1.51 points; these controls do not establish a unique causal decomposition or universally optimal rank.Source: §5.4 · p. 7, Figure 2 · p. 7

5.5 COMPLETE TRAJECTORIES AND THE TERMINAL CHECKPOINT

• Structured is never below Pure OPSD across 12 scale–checkpoint comparisons, exceeds it in 11, and exceeds Random in 10; at step 100, Structured–Random gains are +3.10, +0.93, and +1.94 points.Structured peaks at step 75 for 1.7B and 4B, but at step 100 for 8B.Source: §5.5 · p. 7, Figure 3 · p. 8

5.6 THE FITTED FACTOR CAPTURES ITS OFFLINE TARGET

• The structured factor captures 4.40× the held-out privileged-gap score of matched Random; its mean KL reduction is 0.2347 versus 0.0241.These offline diagnostics support that the factor is data-dependent and selects the intended local geometry, but do not replace downstream correctness after autoregressive rollout.Source: §5.6 · p. 8

• Unlike fixed-distribution distillation, this work compares ordinary and solution-conditioned views of one model along its own trajectories.Classical distillation matches teacher and student distributions; representation variants transfer features, attention, relations, or contrastive structure.Source: §6 · p. 8
• OPSD uses a frozen self-teacher with a verified solution to provide dense full-vocabulary supervision on student prefixes; SCOPE-OPSD leaves that objective and rollout routing unchanged.Other on-policy variants alter divergences, token weights, privileged examples, trajectory conditioning, or supervision filtering.Source: §6 · p. 8
• Unlike OPRD, PHF, and LOPD, SCOPE-OPSD tests one final-layer privileged-residual orientation against a rank-, spectrum-, and gradient-matched replacement.It uses output Fisher to construct a frozen hidden factor, rather than an output correction as in FP-OPD.Source: §6 · p. 9

• Experiments cover one model family and three competition-mathematics benchmarks; other architectures, reasoning domains, and longer budgets may behave differently.Avg@12 measures accuracy under a fixed sampling budget rather than large-K solvability.Source: §7 · p. 9
• Mechanism controls are limited to 1.7B and do not establish a unique causal decomposition or a rank optimal beyond the tested set.The matched-random comparison controls rank, spectrum, and initial hidden-gradient scale.Source: §7 · p. 9

8 CONCLUSION

• At step 75, Structured exceeds both Pure OPSD and matched Random across Qwen3-1.7B, 4B, and 8B, while complete checkpoint curves preserve the broader ordering through 100 steps.Component controls place GapOnly between Random and Structured; full-state matching and tested ranks 32 and 128 remain below rank 64.Source: §8 · p. 9
• SCOPE-OPSD adds a frozen Fisher-conditioned privileged subspace using aligned student and teacher states, with no extra rollout or inference-time module.The factor is fitted from privileged-gap covariance and output-Fisher geometry.Source: §8 · p. 9
• A locally trained GRPO curve serves as a post-training reference, not as a matched objective or budget comparison.The results support a compact hidden channel for short-budget on-policy self-distillation.Source: §8 · p. 9

A.1 TRAINING CONFIGURATION

• All three models use vocabulary size 151,936, auxiliary rank 64, and a 100-step training budget.The shared training configuration and scale-specific factor calibration are presented in Tables 3 and 4.Source: §A.1 · p. 12, Table 3 · p. 12, Table 4 · p. 12

A.2 EVALUATION AND STATISTICAL ESTIMANDS

• Equation 15 defines the point estimators, and tables and figures report method-level aggregates under the common evaluation protocol.Every result row has 1,080/1,080 generations and zero failed requests.Source: §A.2 · pp. 12–13

A.3 ANALYTICAL INCREMENTAL COST

• The analytical incremental-cost table describes the added interface rather than end-to-end system measurements.Offline calibration uses 128 prompts, dense d × d moment matrices, and one dense spectral solve per factor fit; the factor is discarded after training.Source: §A.3 · p. 13, Table 6 · p. 13

A.4 DATA PROVENANCE

• The study uses the released OPSD training split without adding AIME24, AIME25, or HMMT25 examples, and preserves the source recipe’s train/evaluation boundary.Evaluation files are immutable 30-problem sets with recorded hashes.Source: §A.4 · p. 13

B.1 QWEN3-1.7B

• Table 7 reports Macro Avg@12 across the complete Qwen3-1.7B checkpoint trajectory.Source: Table 7 · p. 13

B.2 QWEN3-4B

• Table 8 reports the complete Qwen3-4B Avg@12 trajectory.Source: Table 8 · p. 14

B.3 QWEN3-8B

• Table 9 reports the complete Qwen3-8B Avg@12 trajectory.Source: Table 9 · p. 14

C LOCAL GRPO POST-TRAINING CONTEXT

• The source paper reports a GRPO peak within 500 steps of 51.10/38.30/23.70, or 37.70 Macro (Zhao et al., 2026a).Source: C LOCAL GRPO POST-TRAINING CONTEXT · p. 14
• 56.11/38.61/25.56, or 40.09 Macro, results from selecting the best local checkpoint separately for each benchmark; this is a non-deployable diagnostic oracle, not a method result.The local best single checkpoint is step 300 at 39.44.Source: C LOCAL GRPO POST-TRAINING CONTEXT · pp. 14–15, Table 10 · p. 14
• GRPO and OPSD use different objectives and 500- versus 100-step schedules, so the GRPO curve is kept separate from the matched Pure–Random–Structured comparison.Table 10 gives the complete local Qwen3-1.7B GRPO checkpoint curve under the official Think Avg@12 protocol.Source: Table 10 · p. 14, C LOCAL GRPO POST-TRAINING CONTEXT · p. 15

D COMPONENT AND RANK ABLATIONS

• Structured is +1.51 points above FullHidden, while GapOnly is +0.47 above Random and -0.92 below Structured; rank 64 is +0.72 and +1.06 above ranks 32 and 128.FullHidden is +0.34 points above Pure but -0.12 below Random.Source: D COMPONENT AND RANK ABLATIONS · p. 15, Table 11 · p. 15

E OFFLINE FACTOR DIAGNOSTICS

• The cross-fitted offline test partitions prompts into complementary folds, fits a factor on one fold, and evaluates it on the other; its privileged-gap score underlies the reported 4.40× ratio.Source: E OFFLINE FACTOR DIAGNOSTICS · p. 15
• The two cross-fitted rank-64 bases have canonical cosine 0.369, and Structured intervention gain is 0.6544, 0.0023, and 0.0264 in early, middle, and late response buckets.The diagnostics are not a substitute for downstream evaluation.Source: E OFFLINE FACTOR DIAGNOSTICS · p. 16, Table 12 · p. 16