PaperlayerView exact revision on arXiv

Source-linked AI summary

On-Policy Self-Distillation in Diffusion Models

Wei Zhou, Xiongwei Zhu, Lingdong Kong, Bo Chen, Lei Zhang, Yongyuan Liang, Xiaoxia Hou, Ye Tian, Xian Sun, Yingshuo Wang, Linfeng Li, Shengqiong Wu, Leigang Qu, Feng Li, Wei Liu, Julian McAuley, Tat-Seng Chua

arXiv:2608.24646v1cs.CV

TL;DR

Endpoint rewards do not specify how intermediate diffusion predictions should change, motivating a more actionable form of supervision. DiffusionOPSD converts image-level reward gradients into detached, bounded targets at on-policy queries, fits them, and refreshes the behavior policy. It achieves the best held-out score in 19 of 20 matched settings while reducing training GPU-hours relative to DiffusionNFT, and its diagnostics show that target gains need not equal realized gains.

  • Problem

    Endpoint rewards observe only decoded rollout outcomes, leaving intermediate denoising predictions without explicit actionable supervision.

  • Method

    DiffusionOPSD uses a frozen behavior policy to collect queries and anchors, constructs bounded positive and negative targets from reward gradients, and fits them as detached supervision.

  • Results

    DiffusionOPSD achieves the best final held-out score in 19 of 20 reward-matched settings across two backbones and ten evaluators, with up to 44.0% relative gains over the strongest competing method.

  • Takeaways & Limitations

    Separating target construction from finite realization makes diffusion post-training more efficient and analyzable within the evaluated settings.

  • Takeaways & Limitations

    Finite fitting may produce a displacement different from the loss-preferred displacement, and guidance-scale training introduces inference-scale dependence without improving the best score.

Abstract

from arXiv · show

Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint rewards do not specify how an intermediate denoising prediction should change. We introduce DiffusionOPSD as an on-policy self-distillation framework that converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries. At each outer iteration, a frozen behavior policy generates trajectories and supplies query states and anchors. Reward gradients construct bounded positive and negative targets around each anchor. The trainable policy fits these targets as detached supervision through finite fitting before an exponential moving average update refreshes the behavior policy. This setup lets us measure target construction and finite realization separately. Controlled same-query experiments show that larger target-construction gains do not necessarily translate into larger realized gains after a single fitting update. Across SD 3.5-M and the step-distilled Z-Image-Turbo, our approach achieves the best final held-out scores in 19 of 20 reward-matched settings across two backbones and ten evaluators. It outperforms the strongest competing method by up to 44.0% and reduces training GPU-hours relative to DiffusionNFT by 40% on SD 3.5-M and 63% on Z-Image-Turbo. These results support on-policy self-distillation as an efficient and analyzable approach to diffusion post-training by converting image-level reward guidance into explicit and continually refreshed intermediate supervision, thereby opening a path toward more efficient and diagnosable alignment.

1 Introduction

DiffusionOPSD addresses the mismatch between endpoint rewards and intermediate denoising predictions by constructing explicit targets and separating target construction from finite fitting. It reports strong held-out performance while showing that target-level improvements may not be realized proportionally after fitting.

  • Motivation: Endpoint rewards supervise only decoded rollout outcomes, although diffusion samples depend on interdependent intermediate denoising predictions.The central challenge is translating outcome-level reward into actionable supervision at sampled queries.
  • Analysis: Target construction and finite realization are distinct: fitting can under-realize, rotate, or overshoot a locally beneficial target because predictions are shared across queries.This decomposition diagnoses whether losses arise from poor reward guidance or inadequate model fitting.
  • Results: 19 of 20 matched settings achieve the best held-out score across two backbones and ten evaluators, while training GPU-hours fall by 40% and 63% versus DiffusionNFT.The reported comparison spans SD3.5-M and Z-Image-Turbo.
  • Analysis: Controlled same-query experiments show that larger target-construction gains can yield smaller realized gains after one fitting update.The reversal occurs without cross-query interference, motivating separate evaluation of construction and realization.
  • Method: DiffusionOPSD constructs bounded positive and negative targets from image-level reward gradients, fits detached supervision, and refreshes targets as the behavior policy changes.A frozen behavior policy supplies trajectories, query states, and clean-output anchors before finite fitting.

2 Related Work

Prior diffusion post-training methods expose reward information through likelihood updates, trajectory credit, late-state backpropagation, or teacher-based distillation. DiffusionOPSD instead converts reward gradients into detached intermediate targets and rebuilds them on-policy.

  • Reward-based post-training: Reward-weighted likelihood and preference objectives translate scores or comparisons into likelihood updates but indirectly supervise individual denoising predictions.Policy-gradient methods instead use sampled advantages or group-relative trajectory credit.
  • Reward-based post-training: FlowGRPO and DanceGRPO assign group-relative trajectory credit through reverse-process likelihood ratios, while other methods express improvement through advantage-weighted score updates.These approaches expose reward through trajectory-level policy optimization.
  • Distillation: Diffusion distillation commonly transfers predictions from stronger teachers through progressive, consistency, distribution-matching, or adversarial objectives.Dataset aggregation and on-policy imitation address distribution shift by collecting supervision on learner-induced states.
  • DiffusionOPSD: DiffusionOPSD constructs bounded targets, fits them without retaining the reward graph, and rebuilds them after the behavior policy changes.Its fitting shares the first-order direction of direct reward ascent at the anchor, while its finite optimization procedure differs.

3 Approach

DiffusionOPSD uses on-policy queries and clean-output coordinates to convert local reward gradients into bounded positive and negative targets. Detached finite fitting then updates the trainable policy, while EMA refreshes the behavior policy for the next iteration.

  • 3.1 Clean-Output Prediction: A clean-output prediction yθ(s) is the decoder-evaluable prediction corresponding to the velocity field at a fixed query.The local reward evaluates the decoded clean-output prediction rather than the velocity directly.
  • 3.1 Clean-Output Prediction: Equal clean-output radii correspond to larger velocity displacements at lower σ, so the clean-output coordinate is not geometrically isometric.Local comparisons are defined near the anchor at a fixed query and noise level.
  • 3.2 Query Collection and Anchor: Endpoint rewards determine group-normalized fitting weights, while local decoded rewards construct targets and fixed-suffix rewards measure construction and realized gains at the same query.These reward roles distinguish training weights from diagnostic measurements and held-out policy evaluation.
  • 3.2 Query Collection and Anchor: The frozen behavior policy supplies trajectory queries and clean-output anchors, remains fixed during construction and fitting, and is refreshed by EMA afterward.Replacing rollout queries with matched forward-noised states has little effect in the evaluated low-noise setting.
  • 3.3 Target Construction: Stabilized normalized reward-gradient steps move positive and negative targets around the anchor within a trust-region radius.Projection preserves the radius bound, and the negative target acts as a repulsive reference through its fitting branch.
  • 3.4 Finite Fitting: The local construction result is evaluated separately from finite fitting and end-to-end held-out policy performance.This separation permits direct comparison of target gains with gains realized by the updated model.
  • 3.4 Finite Fitting: Detached target datasets are fitted for a finite update budget, after which the behavior policy determines the next iteration’s queries and anchors.The online loop cycles through trajectory collection, target construction, finite fitting, and behavior-policy refresh.

4 Experiments

DiffusionOPSD delivers strong held-out quality and training efficiency across backbones and reward settings, while its ablations clarify which target-construction choices matter and where finite fitting can diverge from target quality.

  • Main results: 19 of 20 reward-matched settings achieve the best final held-out score with DiffusionOPSD across both backbones and all evaluators.It leads nine of ten SD3.5-M comparisons and all ten Z-Image-Turbo comparisons.
  • Training cost: 40% and 63% lower training GPU-hours than DiffusionNFT are measured on SD3.5-M and Z-Image-Turbo, respectively.The costs are computed from measured per-step wall time on eight GPUs.
  • Optimization dynamics: 98% median terminal position is reached across 71 single-reward runs, exceeding DiffusionNFT and FlowGRPO by 8 and 15 percentage points.This measures within-run native-reward optimization stability rather than held-out quality.
  • Joint training: A jointly trained policy improves PickScore, CLIPScore, and HPSv2.1 over DiffusionNFT while retaining most specialist gains.Its relative improvements are 8.0%, 13.3%, and 14.4%, respectively.
  • Target construction: Reward-gradient targets provide the dominant construction benefit, while replacing rollout query states with forward-noised controls has a much smaller effect.The same-query CLIPScore changes are 0.3122 versus 0.3089 for query provenance, compared with gaps above 0.075 for direction controls.
  • Finite realization: A better target can yield a worse one-step finite update, with the ordering reversing on 62.3% of 512 prompts.The random target realizes 0.000719 more HPSv2.1 reward despite being 0.03496 worse before fitting.

5 Conclusion

DiffusionOPSD converts image-level reward guidance into detached supervision for intermediate denoising predictions and refreshes that supervision through an on-policy behavior-policy loop. Its clean-output parameterization supports target construction while separating target quality from finite model fitting.

  • Conclusion: DiffusionOPSD constructs bounded positive and negative clean-output targets around behavior-policy anchors and fits them as detached supervision.The behavior policy supplies query states and anchors, while the trainable policy fits targets before the behavior policy is updated for the next iteration.
  • Conclusion: Target construction and finite fitting are separately measurable because the method separates reward-gradient target creation from subsequent model updates.The distinction avoids treating endpoint reward as direct supervision for every intermediate prediction.
  • Conclusion: DiffusionOPSD's target construction and fitting objective remain unchanged across schedules after replacing the clean-output map appropriately.The appendix describes the rectified-flow specialization and the general affine-map substitution.
  • Conclusion: At fixed rectified-flow queries, velocity and clean-output coordinates are bijectively related, but the transformation is not geometry- or optimizer-invariant during finite fitting.A clean-output displacement induces a noise-dependent velocity displacement, so algebraic equivalence does not guarantee equivalent optimization behavior.
  • Conclusion: Low but nonzero noise provides semantically meaningful decoded outputs while retaining local controllability for clean-output improvement.Exactly zero noise leaves almost no remaining controllable clean-output change.
  • Conclusion: The affine clean-output inverse becomes ill conditioned when the schedule determinant approaches zero, so low noise alone is insufficient for query selection.Queries should remain in regions where the determinant stays bounded away from zero throughout target construction.

B.2 Endpoint-Weighted Objectives and Reward Weighting

This section distinguishes endpoint-based reward reweighting from DiffusionOPSD’s on-policy target construction. The framework separates query-state selection, reward weighting, and local target construction so their effects can be analyzed independently.

  • DiffusionNFT primarily exploits reward through sample weighting, whereas DiffusionOPSD also changes query-state sourcing and target construction.The latter uses rollout-collected query states and reward-gradient-derived clean-output targets.
  • Endpoint-weighted training can reconstruct old clean-output predictions without performing policy improvement.DiffusionOPSD adds reward-gradient-derived target maps applied to on-policy anchors.
  • DiffusionOPSD targets are reward-ascent and reward-descent clean-output predictions at on-policy states, not merely positive and negative endpoint samples.Reweighting selects anchors for each branch; target construction attaches a locally reward-directed prediction to each selected anchor.
  • The positive and negative branch laws form a soft decomposition of the same on-policy anchor distribution.Their mixture identity holds when the average fitting weight lies strictly between zero and one.
  • Clipped group-normalized weights remain bounded in [0, 1], limiting the influence of outlier rewards on the loss multiplier.Per-prompt centering removes common reward offsets, while the global denominator avoids separate scaling for every group.
  • Positive affine reward transformations preserve normalized advantages and clipped weights when the stabilizer is zero.With a positive stabilizer, translation invariance remains exact while scale invariance is approximate.

B.5 Fitting-Branch Geometry and Ideal Output Improvement

The two-branch fitting objective has a simple clean-output geometry: branch forces combine into a displacement, while target asymmetry, curvature, and finite fitting limit ideal reward improvement. The construction is locally aligned with the reward gradient under symmetric small-step conditions.

  • Fitting-branch geometry: The ideal branch optimum interpolates between attraction to the positive target and repulsion from the negative target when targets are asymmetric.With symmetric targets, the optimum is independent of the positive and negative mixture weights because both branches share the same improvement direction.
  • Ideal output improvement: A gradient-descent step moves the clean output in the direction ad+ − bd−, which aligns approximately with the reward gradient for symmetric targets.When d+ ≈ h̄u_grad and d− ≈ −h̄u_grad, the update direction is approximately h̄(a + b)u_grad.
  • Fitting-branch geometry: The clean-output branch loss is isotropic, with curvature controlled by β and the adaptive branch weights rather than an anisotropic condition number.Any anisotropy in parameter optimization comes from the network Jacobian.
  • Fitting-branch geometry: If weighted positive and negative target forces cancel, the optimum remains at the anchor; if they agree, one displacement can reduce both branch errors.The force norm measures how much of both target requests a single clean-output displacement can satisfy.
  • Target asymmetry: Target asymmetry can arise because multiple target steps evaluate reward gradients at different points, allowing curvature and numerical error to alter the final displacements.This limits the exactness of the symmetric calculation beyond a single shared-gradient step.
  • Finite realization: Actual finite fitting can under-realize, rotate, or overshoot the ideal reward-improving displacement.The ideal-output guarantee concerns the detached branch optimum, not necessarily the output reached under a finite fitting budget.

B.8 Finite Fitting and the Fitting Gap

Finite fitting translates detached reward-improving targets into parameter updates, but the realized reward change can differ from the constructed target gain. The paper therefore analyzes target construction and finite realization separately, including local conditions and practical limits.

  • Mechanism: Finite fitting updates the model toward detached targets without retaining the decoder, reward, or sampling computation graph.Reward differentiation is confined to temporary clean-output variables during target construction.
  • Practical limits: The realized response depends on parameterization, optimizer state, learning rate, precision, batching, and the finite fitting budget rather than representing an exact arg min.Reusing targets for more optimizer passes lowers construction cost but increases staleness and can leave the target-construction neighborhood.
  • Fitting gap: Target construction and finite realization are non-substitutable because fitting can under-realize, rotate, or overshoot the intended reward change.Extending this accounting to end-to-end online performance would require distribution-shift terms that are not estimated.
  • Local explanation: A local preconditioned update applies a kernel-filtered response to the constructed displacement, which can suppress or rotate reward-relevant directions and introduce cross-query interference.This formalization is explanatory and is not claimed as a theorem for finite AdamW.
  • Proposition B.8: Proposition B.8 gives a complete ordering-preservation condition only for one query and one linearization, with symmetric kernels reducing to Kg = λg.For finite candidate sets, the condition remains sufficient but is not necessary; the proposition excludes adaptive AdamW, nonlinear fitting, the full objective, and online training.
  • Online update: The behavior policy is refreshed by exponential moving average after all inner fitting updates, trading fresher on-policy data against a more stable data-collection distribution.An excessively large EMA rate makes the old behavior policy stale and slows improvement incorporation.

C.1 Main Results

The main results evaluate reward-specific and jointly trained checkpoints across two diffusion backbones and multiple evaluators under backbone-specific protocols. The reported setup uses held-out evaluation with checkpoint and optimization details specified for reproducibility.

  • Evaluation scope: The main table reports held-out scores from evaluator-specific checkpoints across ten model-based text-to-image evaluators.Two direct joint-training rows instead evaluate one jointly trained checkpoint under all ten evaluator columns.
  • SD3.5-M protocol: SD3.5-M experiments use 512 × 512 LoRA adaptation from Stable Diffusion 3.5 Medium with Pick-a-Pic prompts and deterministic 10-step DPM-Solver++ 2M trajectories.FlowGRPO instead uses stochastic SDE-flow rollouts because its PPO ratio relies on per-transition log probabilities.
  • Z-Image-Turbo protocol: Z-Image-Turbo experiments use 1024 × 1024 resolution, guidance scale 0.0, and the native deterministic 9-step FlowMatchEuler schedule for DiffusionNFT, ReFL, and DiffusionOPSD.The adapted released checkpoint is used without access to the original teacher or distillation components.
  • Training schedule: Reward-specific rows use 100 optimizer updates, while direct joint PickScore, CLIPScore, and HPSv2.1 rows use 300 updates with a composite reward objective.The Updates column denotes optimizer updates.
  • Metric interpretation: ImageReward and HPSv3 are signed reward scales, so negative unguided SD3.5-M entries indicate low reward-model preference rather than an exceptional evaluation case.This interpretation applies specifically to the negative SD3.5-M without-CFG entries.

C.2 Two-Stage On-Policy Distillation Baselines

Two-stage on-policy distillation first trains reward-specific DiffusionOPSD specialists, then distills them into a shared student for multi-reward performance. The comparison uses matched evaluation and profiling protocols, including explicit efficiency measurements.

  • Two-stage on-policy distillation: The two-stage alternative trains separate PickScore, CLIPScore, and HPSv2.1 DiffusionOPSD specialists before distilling them into a shared student with equal teacher weights.Specialists use 100 optimizer updates, and the shared student uses 300 second-stage updates.
  • Distillation baselines: DanceOPD matches three teachers at one low-noise query, whereas DiffusionOPD matches teacher transition means across all ten denoising steps.All three variants share the SD3.5-M backbone, prompts, resolution, and LoRA configuration.
  • Compute matching: Each second-stage distillation run is approximately 117 GPU-hours, excluding the three preceding specialist-training runs.Rollout batch sizes are calibrated to a measured reference of 175.2 seconds per update on eight GPUs.
  • Evaluation: The shared policies are evaluated on 1,000 images from 200 DrawBench prompts using deterministic 40-step flow sampling and the identical scorer suite.VLM-Pairwise uses fixed, prompt-matched Seedream 5.0 Pro references for every method.
  • Efficiency and quality: DiffusionOPSD takes 28.2 rather than 47.7 GPU-hours per 100 updates on SD3.5-M, while Z-Image-Turbo takes 149.8 rather than ReFL’s 102.1 GPU-hours.The reported comparison also states higher final held-out quality for DiffusionOPSD in all ten reward-matched settings.

C.4 Qualitative Component Ablations

Qualitative component ablations compare rollout query states, forward-noised controls, and large branch-coefficient controls on shared and additional prompts. Query-state source changes little, whereas overly large branch coefficients visibly damage semantics and details.

  • Component ablation: The large branch-coefficient setting can damage prompt semantics under finite fitting, while rollout query-state and forward-noised controls remain closer.The comparison uses a shared visualization pool.
  • Component evolution: Across disjoint additional prompts, changing query-state source has limited effect, whereas an excessively large branch coefficient produces visible semantic drift.All variants use the same update grid.
  • Additional cases: Additional component-evolution cases again show little effect from query-state source, while a large branch coefficient increasingly harms scene and typography details.The qualitative effect extends beyond prompt semantics to visual detail preservation.

C.5 Training Reward Curves

The training-curve inventory defines canonical records and consistent cross-cell normalization for comparing reward trajectories across methods and backbones. The backbone-specific curves show strong and stable DiffusionOPSD gains, while DiffusionNFT exhibits regressions on Z-Image-Turbo.

  • Training-curve inventory: 73 complete runs provide canonical raw per-update reward histories for aggregate training-curve analyses.The inventory includes 70 reward-specific 100-update runs, one additional matched run, and two 300-update joint-training runs.
  • Normalization and aggregation: Cross-cell gains use a joint smoothed best and each run’s own update-0 reward, reducing rollout-sampling offsets.Solid curves report cross-cell medians, bands report interquartile ranges, and terminal positions use each run’s observed native-reward range.
  • SD3.5-M dynamics: DiffusionOPSD achieves the strongest average normalized gain and stable final rewards across ten SD3.5-M objectives.
  • Z-Image-Turbo dynamics: DiffusionOPSD improves consistently across the native Z-Image-Turbo schedule, while DiffusionNFT shows several below-base regressions.
  • Raw records: Raw per-update reward curves preserve the unnormalized histories underlying the aggregate analyses across all 73 runs.Reward-specific and joint-training runs remain on their native scales.

C.6 Fitting Audit

The fitting audit separates endpoint attribution, same-query target construction, and finite realized gain rather than treating them as one causal chain. Controlled audits show that construction and realization can diverge under a fixed optimization protocol.

  • Audit protocols: The audit combines distinct protocols: endpoint attribution, fixed-query construction and realization comparisons, and parameter-restored reversal probes.End-to-end online training efficacy is evaluated separately.
  • Endpoint attribution: The endpoint-attribution evaluation uses 999 images from 200 DrawBench prompts with approximately five shared initial latents per prompt.It evaluates 50-update SD3.5-M CLIPScore variants at 512 × 512 using deterministic 40-step sampling and paired prompt-latent differences.
  • Construction gain: Construction gain is measured before parameter updates using 512 held-out prompts, fixed query noise σq ≃0.278, target radius ρ = 0.10, and a shared suffix sampler.The candidate’s fixed-suffix reward is compared with the anchor, alongside displacement alignment.
  • Realized gain: Every matched realized-gain interface uses one calibrated plain-SGD step at the same query with matched pooled-RMS output displacement.ReFL differentiates through a one-step late-state prediction, whereas detached variants fit explicit targets at that query.
  • Audit finding: The HPSv2.1 reward-gradient target has larger construction gain but smaller realized gain than a matched-radius random-direction target after one fresh-AdamW update.Thus construction gain and realized gain are non-substitutable under the measured protocol; the probe does not explain end-to-end training or isolate EMA’s contribution.
  • Few-step study: The few-step study evaluates ten reward-specific Z-Image-Turbo checkpoints trained for 100 optimizer updates with its native 9-step sampler.Paired base-model significance is available for nine rewards with aligned per-prompt scores.
  • Joint-training stress test: The joint policy trains for 300 updates on PickScore, CLIPScore, and HPSv2.1, while single-reward specialists train for 100 updates.The differing budgets prevent interpreting this stress test as a matched multi-objective leaderboard.

C.8 Ablation Details

The ablation details fix a shared screening protocol and compare target directions, training evolution, human preferences, and classifier-free-guidance formulations. Reward-gradient targets are evaluated against random and no-op controls, while the CFG analysis exposes branch non-identifiability.

  • Shared screening protocol: All ablations use SD3.5-M CLIPScore screening with Pick-a-Pic training prompts and a fixed 20-prompt DrawBench subset producing 100 images per scalar.
  • Training schedule and sampler: Each ablation trains a LoRA for 50 optimizer updates using deterministic 10-step DPM-Solver++ 2M sampling unless explicitly changed.Checkpoint index u equals the optimizer-update count.
  • Canonical OPSD configuration: The canonical configuration uses bounded positive and negative reward-gradient targets at σq = 0.278 and ρ = 0.10 with two ascent and descent steps per branch.
  • Target variants: The target-direction comparison includes reward-gradient, same-radius random, no-op, and rollout-residual variants plus a query-state source check.
  • Qualitative evaluation: Qualitative grids use disjoint prompt sets and are diagnostic rather than inputs to scalar CLIPScore means.
  • Qualitative target-direction ablation: Reward-gradient outputs preserve requested attributes more consistently than random and no-op controls across 40-prompt qualitative comparisons.The figures assess entities, style, text rendering, object identity, color, counting, and spatial relations.
  • Training evolution: Reward-gradient outputs separate early and retain intended actions or attributes through update 50, while controls remain closer to initial behavior.Additional disjoint prompts show stronger preservation of requested vehicles, architecture, objects, and relations.
  • Human preference protocol: Human-preference comparisons use 100 held-out prompts with blinded paired judgments against the base model and three competing methods.Annotators choose the better image overall while allowing ties and assign a primary reason.

Its squared field error is therefore

The squared guided-field error decomposes into the requested correction, train-scale residual, and a guidance-scale mismatch term. This yields a quadratic mismatch penalty and explains why evaluation-scale mismatch can degrade performance when hidden branch components remain.

  • Field-error decomposition: Exact train-scale fitting produces an evaluation correction equal to the target correction plus residual and a scale-difference term.
  • Squared error: The resulting squared error is expanded from the evaluation correction, with the zero-residual case yielding the mismatch component directly.
  • Quadratic mismatch: For fixed training scale, the squared error is quadratic in evaluation scale whenever the hidden branch component is nonzero.
  • No-CFG evaluation: Evaluating a guided-trained SD3.5-M model without CFG corresponds to ζev = 1 and incurs a mismatch penalty under exact train-scale fitting.
  • Practical implication: The mismatch penalty grows quadratically with the distance between training and evaluation guidance scales when the hidden branch component is nonzero.
  • Optimization geometry: The guided operator’s derivatives determine how conditional and unconditional branch changes contribute to optimization.
  • Branch identifiability: Opposite-sign branch coefficients and shared parameters make the guided affine combination more directly controlled than either individual branch.The objective contains no separate regularizer on the branch-difference norm.
  • Comparison with frozen teachers: Frozen-teacher absorption avoids the train–evaluation ambiguity because the guided teacher field is fixed before optimizing a single student field.
Loading 2608.24646v1…