GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales

1 INTRODUCTION

• Diffusion OPD has mostly focused on merging same-scale models, leaving large-to-small distillation underexplored; directly extending the existing recipe causes severe degradation.Reported artifacts include corrupted regions and loss of fine-grained textures.Source: §1 · p. 2
• Fixed-State KL evaluates student–teacher discrepancy on teacher-trajectory states and real-image-derived noise-corrupted states; large-to-small DiffusionOPD shows a substantially larger discrepancy than same-architecture OPD.The analysis also uses spectral energy and high-pass texture to assess coarse- and fine-grained deviations from the teacher.Source: §1 · p. 2
• A smaller student struggles to match a larger teacher, while CFG can accumulate and amplify discrepancies between their conditional and unconditional branches.This amplification leads to greater deviation in the final generation trajectory.Source: §1 · p. 2
• Guidance Folding Distillation reduces the student–teacher gap while avoiding error amplification from CFG composition.The paper presents GFD as enabling effective large-to-small diffusion OPD.Source: §1 · p. 2
• GFD removes cross-scale DiffusionOPD artifacts and outperforms previous baselines in training efficiency and final performance.The experiments combine quantitative and qualitative analysis.Source: §1 · p. 2

2.1 PRELIMINARY: ON-POLICY DISTILLATION IN LANGUAGE MODELS AND DIFFUSION MODELS

• DiffusionOPD extends OPD to diffusion by matching sampler-induced Gaussian transition means, equivalently guided velocities, along student rollouts.The sampler fixes the transition covariance, yielding a closed-form per-step reverse KL.Source: §2.1 · p. 3

2.2 FIXED-STATE KL EVALUATION

• Fixed-State KL evaluates student–teacher discrepancy on shared normal teacher-trajectory and real-image-noise states, avoiding rollout-dependent comparisons.Student rollouts differ across methods and may enter abnormal states where teacher predictions are not guaranteed to be reliable.Source: §2.2 · pp. 3–4, Table 1 · p. 4

2.3 FROM CROSS-SCALE DIFFUSIONOPD TO GFD

• In L→M, guided, conditional, and unconditional KL exceed M→M; limited capacity weakens branch-error cancellation, while separate supervision lowers branch KL but not final guided KL.The authors report that branch discrepancies can be amplified by guidance, and separate supervision does not restore image quality.Source: §2.3 · pp. 4–6
• GFD folds the teacher’s guided policy into the student’s conditional branch, turning two-branch composition into single-branch velocity matching.This design avoids recovering the target through composition of two imperfectly matched branches.Source: §2.3 · p. 6
• GFD yields substantially lower KL than alternatives and restores normal image quality, with outputs closer to the teacher on coarse- and fine-grained metrics.The reported quality comparison spans the shared-artifact examples and the coarse- and fine-grained metrics.Source: Table 1 · p. 4, Figure 4 · p. 5, §2.3 · p. 6

2.4 REWARD EXTRAPOLATION FOR LARGE-TO-SMALL DISTILLATION

• Reward extrapolation replaces the teacher transition kernel with p_tgt ∝ p_T^λ p_R^(1−λ), shifting its mean to v_T +(λ−1)(v_T−v_R) while preserving sampler-fixed variance.The reference kernel is induced by the reference velocity; the extrapolation moves the target away from that reference.Source: §2.4 · pp. 6–7

3.1 EXPERIMENTAL SETUP

• The evaluation uses GenEval, OCR accuracy, and PickScore as task rewards, with additional image-quality probes; methods share prompts, seeds, sampler, and 1024×1024 resolution.Baselines include pretrained SD3.5-Medium and SD3.5-Large, direct Flow-GRPO teachers, DiffusionOPD, PDM, and split-KL.Source: §3.1 · p. 7
• Each single-task GFD-OPD run takes about 10 GPU-hours, versus 15 for DiffusionOPD and 160 for direct Flow-GRPO.The shared setup uses a 10-step reverse-SDE sampler with a 2-step training window; GFD-OPD uses guidance scale w = 2 and extrapolation λ = 1.25.Source: §3.1 · pp. 7–8

3.2 MAIN RESULTS

• GFD-OPD exceeds the SD3.5 teacher on GenEval (0.963 vs. 0.949) and OCR (0.973 vs. 0.969), leads SD3.5 merged rewards and quality, and leads FLUX.2 task rewards.On FLUX.2, it also exceeds the teacher on GenEval and widens its LongText margin at 32B.Source: §3.2 · p. 8
• Among the compared methods, GFD-OPD is closest to the large teacher in layout and fine detail; medium-teacher DiffusionOPD has weaker prompt adherence and fine-grained rendering.Figure 5 compares the base, medium and large teachers, DiffusionOPD→M, and GFDL→M at 1024×1024.Source: §3.2 · p. 8, Figure 5 · p. 8

3.3 ABLATIONS

• Guidance folding accounts for most of the ablation gain and roughly matches the teacher; the extrapolated target adds gains, pushing GenEval and OCR past the teacher.Table 5 compares standard OPD, guidance folding, and the extrapolated-mean target on the SD3.5-Medium student; HPSv3 probes image quality.Source: §3.3 · pp. 8–9, Table 5 · p. 9
• GFD achieves its closest match to the teacher at w = 2, while no tested w brings L→M DiffusionOPD, PDM, or Split-KL close to the teacher.Figure 6 plots student-to-teacher statistic log ratios; the teacher is at the origin, with final-latent energy and decoded-image metrics on its axes.Source: §3.3 · p. 9, Figure 6 · p. 9

On-Policy Distillation

• Language-model OPD aligns students with teacher feedback on student-generated trajectories and serves both compression and specialist-model merging (Agarwal et al., 2024).Yang et al. (2026) interpret OPD as dense KL-constrained RL.Source: §4 · p. 9

On-Policy Distillation for Diffusion Models

• DiffusionOPD derives a closed-form per-step objective for merging task-specific teachers, while PDM separately supervises conditional and unconditional CFG branches (Li et al., 2026b; Li et al., 2026a).Other recent work ports OPD to diffusion and flow-matching models (Ma et al., 2026; Fang et al., 2026; Wei et al., 2026).Source: §4 · p. 9
• Existing diffusion OPD methods target same-family merging, leaving large-to-small distillation largely unexplored.Source: §4 · p. 9

• The existing diffusion OPD recipe does not transfer to the large-to-small regime.Source: §5 · p. 10
• A small student cannot fit the teacher’s guided field exactly, and CFG composition amplifies guidance-increment error by the guidance scale.Source: §5 · p. 10
• GFD-OPD absorbs the teacher’s guided policy into the student’s conditional branch, reducing the student–teacher gap while avoiding CFG-composition error amplification.The authors call for further investigation of how teacher–student distributional discrepancy affects diffusion OPD.Source: §5 · p. 10

A DERIVATION

• ExOPD defines πtgt(y|x) ∝ (π⋆)^λ(πref)^(1−λ)/Z(x), with standard OPD recovered at λ = 1.Here Z(x) normalizes the target distribution.Source: §A.1 · p. 12
• Minimizing the ExOPD objective is equivalent to reverse-KL minimization against πtgt, replacing the teacher in standard OPD.The equivalence follows because Z(x) does not depend on the student parameters.Source: §A.1 · p. 12
• For 0 < λ < 1, πtgt interpolates between reference and teacher; for λ > 1, it extrapolates beyond the teacher along that direction.Source: §A.1 · p. 12
• Diffusion applies the replacement to per-step transition kernels; with equal sampler-fixed variance, the target mean corresponds to vtgt = λvT + (1−λ)vR.The resulting velocity-matching loss uses vtgt in place of the guided target velocity.Source: §A.1 · p. 12

B.1 ANALYSIS OF ERROR CANCELLATION

• The guided objective constrains only δu + wd, so d = −δu/w lies in its null space and permits one branch error to compensate for the other.Table 6 decomposes guided error at w = 4.5 as δg = δu + wd.Source: §B.1 · p. 13, Table 6 · p. 13
• Large-to-small DiffusionOPD has additional error components orthogonal to δu, so its guided-error cancellation is weaker than in same-architecture distillation.Source: §B.1 · p. 13
• PDM and Split-KL reduce the unconditional error δu but make the components nearly orthogonal, leaving substantial final guided error.Source: §B.1 · p. 13

B.2 SHRINKAGE OF THE TRAINED FIELD ALONG THE TEACHER’S GUIDED FIELD

• The projection results average over 500 prompts and over the steps in each window.The projection measures the student’s deployed field along the teacher’s guided field.Source: §B.2 · p. 13
• 0.97–0.98 to 1.00: raising guidance from w = 1 to w = 2 moves average projection over steps 0–37 to 1.00 on both probe sets.Table 7 reports the projection of the deployed student field onto the teacher's guided field.Source: Table 7 · p. 13, §B.2 · p. 14
• At w = 1, the trained field is short along the teacher’s guided direction in every window, especially at high-noise steps 0–9.The student has learned the direction but shrinks the magnitude toward its own prior.Source: §B.2 · p. 13
• In the low-noise regime, the unconditional branch co-evolves with the conditional branch, causing only a marginal CFG-increment increase and avoiding excessive guidance.Source: §B.2 · p. 14

B.3 METRICS OF FIG. 6

• All four quantities compare student and teacher samples from the same prompt and initial noise across 500 prompts; each axis is a log ratio to the teacher.The teacher therefore sits at the origin.Source: §B.3 · p. 14
• Panel (a) measures low- and high-frequency energy in the final latent using frequency bands defined by radial frequency.The bands are Blow = {ρ < 0.25} and Bhigh = {ρ ≥ 0.75}.Source: §B.3 · p. 14
• Panel (b) measures decoded-image tonal amplitude using RMS contrast and mean saturation, and fine-texture amplitude using a 3 × 3 high-pass luminance residual.Its coordinates are mean log ratios across the prompts.Source: §B.3 · p. 14

C.1 GFD WITH UNCONDITONAL BRANCH SUPERVISION

• GFD constrains only the conditional branch, while GFD+uncond also matches the student’s unconditional branch to the teacher.Both variants retain the folded target for the conditional branch.Source: §C.1 · p. 14
• At w = 1, the students use fields trained toward the same target and have very similar KL values and the same final sampling point.The passage locates that shared sampling point in Figure 7.Source: §C.1 · p. 15
• GFD’s conditional and unconditional branches co-evolve, leaving a small CFG increment and avoiding excessive guidance.In GFD+uncond, matching the unconditional branch to the teacher removes that co-evolution; raising guidance scale mainly adds a native-scale texture correction in GFD.Source: §C.1 · p. 15
• GFD+uncond’s CFG increment is doubled, causing larger KL divergence and final samples farther from the teacher distribution.Table 8 reports KL divergence to the teacher for GFD and GFD+uncond; Figure 7 shows final-output statistics as guidance scale varies.Source: §C.1 · p. 15, Table 8 · p. 15, Figure 7 · p. 15
• Figure 7 tracks final-output statistics across guidance scales in coordinates expressed as log ratio to the teacher, whose position is the origin.The coordinates are those of Figure 6.Source: Figure 7 · p. 15

D ADDITIONAL QUALITATIVE RESULTS ON FLUX.2

• The figure compares the FLUX.2-4B base with 9B- and 32B-teacher/GFD-OPD-4B-student pairs; its top four rows use GenEval prompts and bottom four use LongText prompts.The LongText prompts probe long-form visual text rendering.Source: Figure 8 · p. 16