PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention

Luis Zuin, Alexis Huet, Dario Rossi, Zied Ben Houidi

arXiv:2609.39494v1cs.AIcs.LG

TL;DR

Prior self-distillation studies often fix design choices and report conflicting conclusions about acquisition and retention. This paper formalizes and crosses those choices across models and tasks, finding that their effects depend on the task and model.

  • Problem

    Prior studies often fix self-distillation design choices and report conflicting conclusions, leaving their interactions and task dependence uncertain.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    The paper formalizes self-distillation and SFT through rollout source, teacher coupling, and KL direction, then evaluates every combination with a controlled model and LLM experiments.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Teacher coupling changes acquisition on every task, teacher rollouts matter most on contradictory tasks, and the KL-direction retention cost depends on the model.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The preferred setting depends on task and model, so the three design choices are best evaluated together.

    Source: §7 · p. 9

    The controlled model leaves out learning in hidden states, including how the context is encoded.

    Source: §5 · p. 7

Abstract

from arXiv · show

Self-distillation with privileged context adapts a language model from demonstrations by letting the model, once conditioned on a reference response, teach its context-free copy token by token. Our taxonomy reveals existing methods differ along three entangled axes: (i) the rollout source (student or teacher), (ii) the teacher coupling (frozen, or an exponential moving average of the student at some coupling rate) and (iii) the KL direction (reverse or forward), yet these axes are usually studied in fixed combinations and have led to conflicting conclusions. We formalize a unifying framework to encompass all self-distillation methods vs classic supervised fine-tuning: we train every combination of the three axes, on Qwen2.5-7B and Ministral-3-3B across ordinary and contradictory tasks, totaling 1,200 adaptation runs, to systematically investigate the impact of the above axes. We propose a controlled model of the same objective to explain the resulting acquisition-retention trade-offs. We find that (i) the rollout source matters mostly where the task contradicts the pretrained behavior: there teacher rollouts raise acquisition well above what student rollouts achieve, with almost no change in retention; (ii) the teacher coupling changes acquisition most, on every task: acquisition rises with the coupling rate, then falls past a task-specific rate; (iii) switching the KL direction costs retention in one model but not the other so which axis to tune first depends on the model. The controlled model reproduces the three trends.

1 INTRODUCTION

Self-distillation uses a context-conditioned version of a model to teach its context-free copy, but prior studies often fix its design choices and report conflicting conclusions. This paper systematically tests three choices across tasks and models, finding that their effects vary by task and model.

1 INTRODUCTION

  • The paper formalizes self-distillation through three axes: rollout source, teacher coupling, and token-level KL direction.The choices are whether student or teacher generates responses, whether the teacher is frozen or follows the student, and whether the KL is reverse or forward.Source: §1 · p. 1
  • Prior studies often fix the rollout source, teacher coupling, and KL direction, rarely testing multiple choices together.The literature also reports conflicting conclusions about these choices.Source: §1 · pp. 1–2
  • The study crosses the design choices on two instruction-tuned models and five tasks, comparing with SFT and using a controlled model.The evaluation comprises over 1,200 runs; two tasks contradict behavior the pretrained model already follows.Source: §1 · p. 2
  • Teacher coupling affects acquisition on every task, while teacher rollouts matter most on contradictory tasks and KL-direction effects vary by model.Acquisition rises with coupling rate before falling past a task- and model-dependent rate; SFT reaches the highest acquisition on most tasks, while most self-distillation settings retain more at the acquisition they reach.Source: §1 · p. 2

2 RELATED WORK

Contextualized self-distillation lets a model use privileged context to provide token-level teaching signals to its context-free copy. Prior findings conflict, while studies often vary only one axis and give less attention to retention and interactions.

2 RELATED WORK

  • Contextualized self-distillation uses privileged context to elicit a teaching signal that the student learns to reproduce without that context.The context may be demonstrations, solutions, feedback, trajectory-derived skills, or documents.Source: §2 · p. 2
  • Prior studies disagree on whether teacher or student rollouts, moving or frozen teachers, and forward or reverse KL are preferable.Reported findings include both gains and degradation from moving teachers, and conflicting explanations of retention.Source: §2 · p. 2
  • Studies typically vary one axis at a time, and retention is less studied than acquisition, leaving interactions and task dependence largely unknown.Some joint studies examine rollout source with KL direction under an external teacher or rollout source with coupling in Skill-SD.Source: §2 · p. 2

3 TAXONOMY AND UNIFYING FRAMEWORK

The paper represents self-distillation with privileged context and classic SFT in one objective, whose open choices correspond to rollout source, teacher coupling, and KL direction. These choices affect the prefixes, token disagreements, and teacher used during training.

3 TAXONOMY AND UNIFYING FRAMEWORK

  • A single objective formalizes self-distillation with privileged context and classic SFT through rollout source, teacher coupling, and KL direction.The student and teacher start from the same pretrained parameters, but only the teacher sees the privileged context; in this paper that context is the reference response in a fixed template.Source: §3 · p. 3
  • Training samples a response from a rollout distribution, then teacher and student evaluate next-token distributions on the same generated prefix.The student alone receives the gradient; the teacher and sampled response are held fixed.Source: §3 · p. 3
  • The rollout source determines which prefixes receive correction: student rollouts follow the student’s own text, while teacher rollouts use context-conditioned teacher text.Teacher rollouts can cover prefixes where the teacher is informative but the student may not produce them at deployment.Source: §3 · pp. 3–4
  • Teacher coupling ranges from a frozen teacher at α = 0 to a synchronized teacher at α = 1, with EMA coupling between those extremes.An EMA teacher incorporates what the student has learned and also inherits its drift, which it feeds back to the student.Source: §3 · p. 4
  • Forward KL is mode-covering, whereas reverse KL is mode-seeking and concentrates correction on teacher modes the student already approaches.Classic SFT is a special case with the reference response as the sampled response and a one-hot teacher distribution on each reference token.Source: §3 · p. 4
  • Each axis acts on a different training object, and the paper expects no universal ranking given contrasting results in prior work.The axes govern prefixes, token disagreement, and which teacher is used; the task affects all three.Source: §3 · pp. 3–4

4 DESIGN SPACE INVESTIGATION

The study fully crosses rollout source, teacher coupling, and KL direction across five tasks and two instruction-tuned models, with SFT as a comparison. It evaluates acquisition on held-out task data and retention on general-capability benchmarks.

4 DESIGN SPACE INVESTIGATION

  • Five tasks include ordinary tool-use and chemistry tasks, a control task, and two learnable tasks whose rules contradict pretrained conventions.The contradictory tasks rotate compass directions or require arithmetic in base 9 instead of base 10.Source: §4 · p. 4
  • The contradictory task examples show reference answers under ordinary and contradictory conventions in the table’s final two columns.Both models start below 2% accuracy on spatial-contradiction, compared with 20–26% on spatial.Source: Table 2 · p. 5, §4 · p. 5
  • The design crosses two rollout sources, five coupling rates, and two KL directions into 20 configurations, yielding 1,200 runs across tasks and models.The configurations are systematically compared with classic SFT.Source: §4 · p. 4, Table 3 · p. 5
  • The two models are Qwen2.5-7B-Instruct and Ministral 3 3B Instruct, each adapted from its initial checkpoint on each task.SFT trains on the same reference responses that the teacher reads in context, so the reference enters training differently across the two methods.Source: §4 · p. 5
  • Acquisition is held-out task accuracy gain, while retention is mean change across five general-capability benchmarks, with negative values indicating forgetting.Both metrics are computed per run and averaged over two seeds; significance for a binary axis uses matched-pair Wilcoxon signed-rank tests.Source: §4 · p. 5

5 A CONTROLLED MODEL OF SELF-DISTILLATION

The controlled model isolates self-distillation factors in a small language-model analogue where fine-tuning changes a shared output head while hidden states remain fixed. It measures how adaptation to a new task relates to preservation of the old task.

Model mechanics

  • The model generates responses token by token from a finite prefix tree, using one trainable head across fixed hidden states.Because the head is shared, an update at one prefix can change policies at other prefixes.Source: §5 · p. 6
  • The same head defines old- and new-task policies, whose hidden states have cosine similarity ρ_ϕ to model feature overlap.At ρ_ϕ = 1 the tasks share hidden states; at ρ_ϕ = 0 they are orthogonal.Source: §5 · p. 6
  • The new-task target r_new is constructed to favor multiple tokens unrelated to pretrained preferences, while remaining reachable by one head.This lets the model represent both differing KL behaviors and tasks that contradict a confident initial model.Source: §5 · p. 6

Teacher and context

  • Privileged context shifts the teacher’s hidden state, while the student reads the unshifted state; κ sets the context’s initial contribution toward r_new.The shift is fixed before training, with the initial teacher policy defined as a mixture of the initial student policy and r_new.Source: §5 · p. 6

Measures and parameters

  • Acquisition is the change in student similarity to r_new on visited prefixes, while retention is the change in old-task similarity on prefixes visited by r_old.Negative retention corresponds to forgetting; the model separately varies initial-policy concentration λ, context strength κ, and feature overlap ρ_ϕ.Source: §5 · pp. 6–7

6 EXPERIMENTAL RESULTS

Across the experiments, rollout source, teacher coupling, and KL direction affect acquisition and retention differently. The results show that their effects depend on task conditions and, for KL direction, on the model.

6.1 ROLLOUT SOURCE

  • Teacher rollouts add a median 8 acquisition points on contradictory tasks for both models, with under one point median retention difference; ordinary tasks show little acquisition benefit.The advantage increases with initial-model confidence in the controlled model, though student rollouts slightly exceed teacher rollouts at the largest tested λ and κ.Source: §6.1 · pp. 7–8

6.2 TEACHER COUPLING

  • Acquisition rises and then falls with coupling rate α, with a task- and model-dependent peak; on Qwen, moving teachers beat frozen ones through α = 0.10, while Ministral peaks at 0.05 and loses at 0.25.On Qwen the median retention cost is below half a point; in the controlled model, stronger context shifts the peak to slower rates and shrinks the gain over a frozen teacher.Source: §6.2 · p. 8

6.3 KL DIRECTION

  • KL-direction retention costs depend on the model: forward KL costs retention on Qwen but not Ministral, where it can improve acquisition and retention together.On Qwen, tuning coupling under reverse KL gives most of the acquisition gain for a small retention loss; the controlled model reproduces the model dependence.Source: §6.3 · pp. 8–9
  • Most self-distillation families retain more than SFT at matched acquisition—17 of 20 on Qwen and 14 of 20 on Ministral—but SFT usually reaches the highest acquisition.SFT’s highest acquisition comes with 3 to 21 points of retention, except on Qwen tool-alpaca and chemistry.Source: §6.3 · p. 9

7 CONCLUSION

The conclusion finds that self-distillation outcomes depend jointly on task and model, and that a controlled model reproduces the observed trends. The appendices document the study’s disclosures, resources, literature classifications, and task datasets.

7 CONCLUSION

  • The controlled model reproduces the three trends, and the preferred setting depends on the task and model.The study covers five tasks, two model families, and two seeds per configuration, using reference responses as context.Source: §7 · p. 9

AI USE STATEMENT

  • The authors report using generative AI across data generation, modeling, experiments, analysis, literature review, and software, and state that they reviewed AI-assisted work.They also checked literature summaries against original papers and take responsibility for the final content.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • Code, datasets, and results are shared through a GitHub repository, with dataset construction and experimental setup documented in the appendices and main text.The repository includes the new math-contradiction, spatial, and spatial-contradiction datasets, plus controlled-model and LLM experiment materials.Source: REPRODUCIBILITY STATEMENT · p. 10

A PER-ROW JUSTIFICATION FOR TAB. 1

B DATASETS

  • Across tasks, the teacher receives the reference response as privileged context, the student sees only the input, and SFT uses the reference as its target.Evaluation is performed on held-out task examples without privileged context.Source: B DATASETS · p. 13

B.1 TOOLALPACA AND SCIKNOWEVAL CHEMISTRY L-3

  • Tool-alpaca and chemistry use existing datasets and reference responses; tool-alpaca scores structured calls, while chemistry scores an exact-match answer option.Neither task scores the accompanying explanation, so a correct chemistry option does not establish that its reasoning is sound.Source: §B.1 · pp. 13–14

B.2 MATH CONTRADICTION

  • Math-contradiction rewrites numeral literals in base nine while preserving arithmetic operations, then filters examples and audits full solutions for consistency.The final pool contains 5,772 examples; 4,000 are assigned to training and 500 to evaluation, with source identities kept disjoint.Source: §B.2 · pp. 14–15

B.3 SPATIAL TASK, WITH AND WITHOUT CONTRADICTION

This section describes paired spatial tasks that differ in relation semantics and shows how student inputs, reference responses, and teacher prompts are constructed. It also provides examples, including a coordinate calculation, and notes that reference responses may contain reasoning or answer discrepancies.

B.3 SPATIAL TASK, WITH AND WITHOUT CONTRADICTION

  • Spatial and spatial-contradiction use identical inputs and splits but rotate every relation displacement clockwise; scoring accepts only the last complete boxed integer pair matching the task’s reference.The rotation is fixed rather than cumulative, and the generated chains ensure the two reference answers differ for every example.Source: §B.3 · p. 16

C STUDENT PROMPTS, REFERENCE RESPONSES, AND TEACHER TEMPLATES

  • Teacher prompts preserve the student’s role structure and original task input while adding the full reference response as text; SFT trains on that response, whereas self-distillation trains on teacher token distributions.For Math Contradiction and both spatial tasks, the teacher template calls the reference a “verified reference solution,” without asserting every intermediate step is formally proved.Source: C STUDENT PROMPTS, REFERENCE… · p. 16, §C.1 · p. 17

C.2 CHEMISTRY: STUDENT MESSAGES AND REFERENCE RESPONSE

  • The chemistry reference selects option C after calculating 389.52 g/mol, although option C is 387.500 g/mol; the paper retains this discrepancy as an example of incorrect reasoning despite a final label.The passage says the response chooses the closest option after obtaining 389.52 g/mol.Source: §C.2 · p. 18

C.3 TOOL-ALPACA: STUDENT MESSAGE AND REFERENCE RESPONSE

  • The tool-alpaca example asks for a PATCH request to Httpbin and shows the student’s requested tool-call format, including an action name and JSON input.The prompt names sendHttpRequest for the specified URL and data.Source: §C.3 · p. 19

C.4 MATH CONTRADICTION: STUDENT MESSAGE AND REFERENCE RESPONSE

  • The Math Contradiction example presents an arithmetic expression in the student prompt, while its reference response is identified as base nine, a convention not announced in the prompt.The supplied passage identifies the reference as base nine but does not provide its final answer.Source: §C.4 · pp. 19–20

C.5 SPATIAL TASKS: SHARED STUDENT MESSAGE AND BOTH REFERENCE RESPONSES

  • The spatial example uses one shared student message and presents ordinary and rotated reference solutions for the same four-link route, yielding targets (4, 0) and (0, −6), respectively.Both solutions start from anchor (−1, −1); the rotated convention changes the interpreted displacements, not the input wording.Source: §B.3 · p. 16, §C.5 · pp. 20–21
  • (2, -6) + (-2, 0) = (0, -6), Farah’s final coordinates.The passage states the coordinate sum and identifies (0, -6) as Farah’s final coordinates.Source: §C.5 · p. 21

D ADDITIONAL ABLATION STUDIES

The appendix reports matched-pair ablations of rollout source, teacher coupling, and KL direction, and compares best-acquiring self-distillation runs with the SFT Pareto front. It also specifies the controlled model’s experimental setting.

D.1 ROLLOUT SOURCE

  • With reverse KL, rollout-source comparisons preserve the same ordering on contradictory tasks as the forward-KL case.Figure 4 presents the comparison, and Table 5 reports matched-pair tests.Source: §D.1 · p. 21, Table 5 · p. 22, Figure 4 · p. 23

D.2 TEACHER COUPLING

D.3 KL DIRECTION

  • Figure 6 extends the KL-direction comparison to student rollouts, while Table 7 reports matched-pair tests.The figure also shows routes from frozen reverse KL through coupling and direction changes, alongside the SFT grid and its Pareto front.Source: §D.3 · p. 21, Table 7 · p. 23, Figure 6 · p. 24

D ADDITIONAL ABLATION STUDIES

  • Table 8 compares each rollout-source and KL-direction family’s best-acquiring run against the SFT Pareto front.Cells report acquisition and retention; blue marks runs beyond the SFT front, while black marks runs dominated by an SFT run.Source: D ADDITIONAL ABLATION STUDIES · p. 21, Table 8 · p. 23

E EXPERIMENTAL SETTING FOR THE CONTROLLED MODEL

  • The controlled-model appendix tests the three main regimes while varying task controls separately from the self-distillation axes.It specifies the model’s features, teacher, training objective, and evaluation protocol.Source: E EXPERIMENTAL SETTING FOR THE… · p. 25

E.1 FINITE AUTOREGRESSIVE BACKBONE AND TASK GEOMETRY

The controlled model uses fixed old- and new-task feature maps with a shared trainable readout, constructs an attainable adaptation target and contextual teacher, and compares self-distillation axes using exact prefix occupancies.

E.1 FINITE AUTOREGRESSIVE BACKBONE AND TASK GEOMETRY

  • The initial-policy concentration λ sharpens initial preferences without changing token rankings and is not the optimizer step size.Centering removes the action-independent logit direction.Source: §E.1 · p. 25
  • The feature-overlap coefficient ρϕ controls statewise representation reuse, while ρW controls compatibility of the readout used to create adaptation targets.The reference construction uses ρϕ = 0.5 and ρW = 0.Source: §E.1 · pp. 25–26
  • The shared trainable readout can interfere with retained behavior even though the old- and new-task feature maps remain fixed.The model therefore isolates interference to readout adaptation rather than feature learning.Source: §E.1 · p. 25

E.2 ATTAINABLE TARGET AND CONTEXTUAL TEACHER

  • The adaptation target assigns masses 0.30 and 0.15 to the two highest-scoring tokens, then projects centered log probabilities into the shared-readout family.The final reference is attainable by construction, but simultaneous preservation of the old task is not guaranteed.Source: §E.2 · p. 26
  • The context strength κ specifies the fraction of the initial discrepancy from the new-task reference removed by privileged information.At initialization, the contextual teacher is defined as a probability mixture of the initial student and the reference.Source: §E.2 · p. 26
  • The initial teacher is calibrated exactly in the reference dimensions, but subsequent updates process its fixed displacement through the evolving teacher readout.The calibration applies only at initialization; the teacher is not replaced by a fresh probability mixture after each update.Source: §E.2 · p. 26

E.3 EXACT PREFIX WEIGHTING AND SELF-DISTILLATION AXES

  • The objective uses exact prefix occupancies from either student or teacher rollouts and compares teacher predictions with student predictions using forward or reverse KL.Prefix occupancies and teacher predictions are recomputed after updates, while teacher coupling is controlled independently of rollout source and KL direction.Source: §E.3 · p. 27

E.4 REFERENCE PROTOCOL, SELECTION, AND METRICS

The reference protocol pairs method comparisons within regenerated task seeds, specifies local coupling selection and cohort rules, and reports distributional acquisition, retention, and teacher-utility measures with confidence intervals.

E.4 REFERENCE PROTOCOL, SELECTION, AND METRICS

  • Method comparisons are paired within each regenerated controlled-task seed, and cohorts are not pooled across experiments.Main factorial completion and dedicated rollout/coupling sweeps use 96 task seeds; targeted controls specify their own cohort sizes.Source: §E.4 · p. 27
  • Cross-fitted coupling selection chooses the acquisition-maximizing rate on one seed fold and evaluates it on the other; descriptive grid optima use all seeds.Fixed-coupling contrasts do not select α.Source: §E.4 · p. 27
  • Acquisition weights new-task similarity by deployed-student behavior, while target-path fidelity fixes the weighting to the target policy's prefix occupancy.Retention uses the fixed old-reference occupancy, and greater old-task similarity means better retention.Source: §E.4 · pp. 27–28
  • Contextual-teacher utility measures similarity to the new-task reference on states visited by the student.Teacher utility remains on its native [0, 1] scale.Source: §E.4 · p. 28
  • Reported 95% confidence intervals use the normal approximation over task-seed values, with method contrasts formed within seed before aggregation.Similarity changes and forgetting are stored as fractions, while figures labeling differences in points multiply them by 100.Source: §E.4 · p. 28

F CONTROLLED-MODEL RESULTS AND REGIME TESTS

The controlled-model appendix tests the rollout-source, teacher-coupling, and KL-direction result axes, then reports scoped numerical and structural checks under a specified reference configuration.

F CONTROLLED-MODEL RESULTS AND REGIME TESTS

  • The appendix organizes controlled-model experiments around rollout source, teacher coupling, and KL direction, followed by numerical and structural checks.The model construction and metrics are defined in Appendix E.Source: F CONTROLLED-MODEL RESULTS AND REGIME… · p. 28
  • Unless stated otherwise, experiments use the reference task and optimization settings; coupling sweeps use an 18-rate grid unless a control specifies a smaller one.Each experiment states its seed cohort and coupling-selection rule locally.Source: F CONTROLLED-MODEL RESULTS AND REGIME… · p. 28
  • Acquisition and retention are reported in points, teacher utility on [0, 1], and entropy in nats; confidence intervals are pointwise normal 95% intervals.Method-difference intervals use paired within-seed contrasts and are not corrected for multiple comparisons; distinct cohorts are not pooled.Source: F CONTROLLED-MODEL RESULTS AND REGIME… · p. 28

F.1 ROLLOUT SOURCE: TARGET-PATH LEARNING AND DEPLOYMENT TRANSFER

These controlled analyses examine how rollout source, teacher coupling, and KL direction shape acquisition and retention, including how target-path improvements transfer to deployment. The useful coupling rate and KL trade-off depend on the task regime and experimental conditions.

F.1 ROLLOUT SOURCE: TARGET-PATH LEARNING AND DEPLOYMENT TRANSFER

  • At κ = 0.6, the teacher-minus-student acquisition difference peaks at +1.21 points for λ = 2.5, between +0.03 at λ = 0.625 and +0.32 at λ = 5.The selected coupling rates were identical for both rollout sources in this slice, so the contrast was unchanged by imposing a common rate.Source: §F.1 · p. 28
  • At λ = 5 and κ = 0.9, teacher rollouts improve target-path fidelity by +5.86 points but have a deployment-weighted acquisition difference of −0.54 points.Target-path fidelity evaluates matching under fixed target-policy occupancy; the figure contrasts it with acquisition under each final student’s own prefix occupancy.Source: §F.1 · p. 29, Figure 7 · p. 29

F.2.1 ACQUISITION, RETENTION COST, AND TEACHER UTILITY

  • At α = 0.0025, acquisition rises from 24.05 to 28.83 points versus a frozen teacher, while retention loss rises from 24.25 to 28.71 points.At α = 0.02, acquisition falls to 10.45 points as retention loss reaches 36.89 points; at α = 0.16, acquisition is −4.59 points.Source: §F.2.1 · p. 29

F.2.2 PERSISTENCE UNDER REVERSE KL AND STUDENT ROLLOUTS

  • With reverse KL and student rollouts, acquisition again rises then falls with coupling; selected rates for κ = 0.3, 0.6, and 0.9 are 0.005, 0.0025, and 0.000625.Their mean held-out acquisitions are 23.96, 27.65, and 28.34 points, respectively; because both KL direction and rollout source change, this control establishes persistence, not either axis’s isolated effect.Source: §F.2.2 · p. 30, Figure 9 · p. 30

F.2.3 WHAT MOVES THE USEFUL COUPLING TIMESCALE?

  • With relative feature-space noise at scale 1, the preferred coupling rate slows for all three κ values, and freezing is optimal at κ = 0.9; at scale 2, freezing is optimal for all three.At noise scale 0.5, the clean ordering is unchanged.Source: §F.2.3 · p. 31, Table 10 · p. 31
  • Across 100, 200, and 400 updates, all four KL/rollout combinations select α = 0.005, 0.0025, and 0.00125, respectively, giving α⋆N = 0.5.At 800 updates, 0.00125 is again selected but is the smallest positive tested rate, so the experiment does not resolve whether a slower rate is better.Source: §F.2.3 · p. 31, Table 10 · p. 31

F.3.1 ENDPOINTS BEHIND THE MAIN KL COMPARISON

  • At (κ, ρϕ) = (0.6, 0.5), the selected forward endpoint acquires 0.26 points less and retains 3.04 points less than the selected reverse endpoint; at (0.9, 1), it exceeds reverse by 1.23 acquisition and 0.38 retention points.At the reference setting, tuning reverse-KL coupling raises acquisition from 24.00 to 29.43 points while retention changes from −20.55 to −24.81 points.Source: §F.3.1 · pp. 31–32, Table 11 · p. 32

F.3.2 FEATURE SHARING AND READOUT COMPATIBILITY

  • As feature sharing ρϕ rises from 0 to 1, retention cost increases under both KL directions and the reverse-KL retention advantage shrinks from 3.19 points to essentially zero.Across the readout-compatibility sweep, reverse KL retains more throughout, while varying ρW can reverse the small acquisition difference at α = 0.0025.Source: §F.3.2 · p. 32, Table 12 · p. 33

F.3.4 TEACHER UNCERTAINTY AS A COMPLEMENTARY REGIME DIAGNOSTIC

Across the interaction sweep, higher initial teacher entropy is associated with retention ordering shifting toward forward KL, but entropy alone does not determine that ordering. Numerical checks preserve several effects across tested estimators and structural choices, while the acquisition ordering between KL directions remains regime-dependent.

F.3.4 TEACHER UNCERTAINTY AS A COMPLEMENTARY REGIME DIAGNOSTIC

  • Across 147 geometries, higher teacher entropy is associated with retention ordering shifting toward forward KL (Spearman rs = 0.700), including when overlap is fixed.The association remains positive at ρϕ = 0, 0.5, and 1.Source: §F.3.4 · p. 33
  • At λ = 2.5, increasing overlap from ρϕ = 0.5 to 1 changes retention contrast from −2.37 to +2.54 points despite less than 0.001 nats change in entropy.At ρϕ = 0.5, increasing κ from 0.6 to 0.9 raises entropy from 1.871 to 2.028 nats, yet reverse KL retains more at α = 0.0025.Source: §F.3.4 · pp. 33–34
  • Under frozen-teacher rollouts, mean teacher-token entropy is 2.08 nats for Ministral and 0.31 for Qwen, but tokenizer and prefix differences make these values descriptive, not directly calibrated.The paired task-level difference is 1.77 nats (95% CI [1.11, 2.44]).Source: §F.3.4 · p. 34

F.4 CONSTRUCTION AND NUMERICAL CHECKS

  • Across tested occupancy estimators and structural sweeps, rollout-source, coupling, and reverse-KL retention effects remain positive, while forward-minus-reverse acquisition varies from −0.50 to +0.66 points.These checks delimit the mechanism results rather than establish parameter-invariant behavior.Source: §F.4 · p. 35

Related papers

Loading 2609.39494v1…