PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

Bingnan Li, Haozhe Wang, Haozhong Xiong, Fangtai Wu, Jinpeng Yu, Yang Shi, Jiaming Liu, Ruihua Huang

arXiv:2607.24731v2cs.CVcs.AIcs.LG

TL;DR

The paper asks when CFG-composed OPD fails to control individual branch errors, especially when teacher-only information remains in the negative branch. It introduces PDM to supervise the positive prediction and CFG conditional direction separately, finding more robust guidance-scale behavior and knowledge transfer in dense-to-sparse video control. The authors caution that PDM’s advantage over IBM is empirical rather than theoretically characterized.

  • Problem

    CFG-composed OPD is under-identified at the branch level because positive- and negative-branch errors can compensate, with harmful dynamics under privileged negative conditioning.

  • Method

    PDM separately constrains the positive prediction and CFG conditional direction, with IBM studied as a branch-aware comparison.

  • Results

    Branch-aware supervision improves robustness across inference guidance scales and enables more effective dense-to-sparse video knowledge transfer, while naive matching is guidance-scale sensitive.

  • Takeaways & Limitations

    CFG-composed matching can be benign with shared negative conditioning but requires branch-aware supervision when privileged negative conditioning causes antagonistic branch-error dynamics.

  • Takeaways & Limitations

    PDM and IBM share the same branch-level zero-loss solution, so PDM’s observed advantage is empirical rather than theoretical.

Abstract

from arXiv · show

On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood. Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is under-identified at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching remains effective under shared negative conditioning, where both branch errors decrease jointly. When the model's native CFG schema retains privileged information in the teacher's negative branch that is unavailable to the student, however, this joint reduction breaks down and the composed objective induces antagonistic branch-error dynamics, reducing the positive-branch error while increasing the negative-branch error. We term this failure mode Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive--Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

1 INTRODUCTION

The paper shows that CFG-composed OPD can hide branch-level error compensation, with harmful antagonistic dynamics emerging under privileged negative conditioning. It introduces branch-aware supervision and demonstrates more robust transfer in dense-to-sparse video control.

  • Motivation: CFG-composed OPD is under-identified because positive- and negative-branch errors can compensate while preserving the guided prediction.The loss constrains only γe+ + (1 −γ)e− at the training guidance scale.
  • Contrasting regimes: Under shared negative conditioning, naive matching remains effective because both branch errors decrease jointly.This regime is observed in text-rendering distillation.
  • Contrasting regimes: Privileged negative conditioning produces Negative Branch Asymmetry (NBA): naive matching reduces positive-branch error while increasing negative-branch error.The teacher’s negative branch can contain information unavailable to the student, breaking joint error reduction.
  • Consequences: NBA can remain hidden at the training guidance scale but becomes visible when inference recomposes branches with different weights.The resulting guidance-scale sensitivity reflects compensation that no longer holds away from γtrain.
  • Proposed remedy: Positive–Direction Matching (PDM) separately supervises the positive prediction and CFG conditional direction to prevent cross-branch error compensation.Independent Branch Matching (IBM) is included as a branch-aware foil.
  • Video control: In dense-to-sparse video control, naive guided matching is highly guidance-scale sensitive, whereas PDM and IBM remain stable and PDM improves control fidelity across modalities.The teacher observes dense per-frame control while the student receives only sparse keyframes.

2 PRELIMINARIES

The preliminaries define OPD as matching teacher and student predictions at student-visited states and explain how CFG composes separate positive and negative branches. They motivate branch-aware analysis when privileged information is retained in the negative branch.

  • On-policy distillation: OPD trains a student on states sampled from its own evolving generation process, matching teacher and student local denoising transitions at those states.Teacher and student conditioning need not be identical when the teacher has privileged information.
  • On-policy distillation: Existing diffusion OPD objectives reduce to matching corresponding teacher and student model predictions, up to timestep-dependent weighting.The paper uses velocity prediction, while the formulation also applies to equivalent noise, score, or flow parameterizations.
  • Classifier-free guidance: CFG evaluates positive and negative branch predictions and composes them as ev = γv+ + (1 −γ)v− for denoising.The guidance scale γ is selected at inference, often after training.
  • Classifier-free guidance: The paper preserves each model’s native CFG schema, which may retain auxiliary reference or control inputs in the negative branch.Thus, changing only the text condition does not guarantee symmetric branch conditioning.
  • Conditioning assumptions: Branch-retaining OPD is commonly studied with shared null-text conditioning, but privileged negative conditioning arises when teacher information is unavailable to the student.This asymmetry motivates analyzing its effect on OPD optimization.

3 NEGATIVE BRANCH ASYMMETRY AND BRANCH-AWARE OPD

Guided-only OPD can hide compensating branch errors, with privileged negative conditioning producing Negative Branch Asymmetry (NBA). Branch-aware supervision via PDM removes this freedom and supports more stable guidance behavior, while early-trajectory supervision reduces video-training cost.

  • Branch ambiguity: Guided matching constrains only the composed discrepancy, allowing infinitely many non-zero positive- and negative-branch errors to compensate.This ambiguity does not necessarily cause failure under shared negative conditioning, where both branch errors may decrease jointly.
  • Negative Branch Asymmetry: Under privileged negative conditioning, naive matching can reduce positive-branch error while increasing negative-branch error, a failure mode termed Negative Branch Asymmetry.The opposing errors may remain hidden at the training guidance scale and emerge when recomposed at another scale.
  • Empirical diagnosis: An inference-scale performance drop alone does not establish NBA because the teacher may intrinsically depend on CFG; the relevant signal is excess degradation from naive distillation.In reference-conditioned examples, PDM follows the teacher across guidance scales, whereas naive matching shows stronger distortion and style drift, especially at γ = 1.
  • Branch-aware supervision: Positive–Direction Matching separately constrains the positive prediction and CFG conditional direction, so zero loss requires both branch errors to vanish.Its objective anchors the positive prediction while preserving v+ −v−, eliminating compensation freedom.
  • Branch-aware supervision: Independent Branch Matching also removes compensation freedom by directly matching both branches, but weights branch supervision differently from PDM.The two branch-aware formulations are compared empirically in Table 2.
  • Efficient video supervision: For dense-to-sparse video control, supervising only the first K student-visited denoising states reduces teacher-evaluation cost while maintaining performance.The student completes the full rollout, while teacher supervision is computed only for those K states; K = N recovers full-trajectory supervision.

4 EXPERIMENTS

The experiments compare branch-error dynamics under shared versus privileged negative conditioning, then evaluate guidance-scale sensitivity. They show that NBA appears under privileged conditioning, while PDM suppresses the resulting sensitivity.

  • Experimental setup: The experiments compare naive matching, PDM, and positive-only training across text-rendering and reference-conditioned distillation.The two settings differ in whether the teacher’s negative branch receives privileged information unavailable to the student.
  • Branch-error dynamics: In text-rendering distillation, positive-only training reduces both branch errors, and naive matching and PDM show the same benign joint reduction.Teacher and student share the same positive prompt and null-text negative condition.
  • Branch-error dynamics: Under privileged negative conditioning, naive matching reduces positive error while increasing negative error, whereas PDM prevents sustained negative-error growth.Positive-only training exhibits the same antagonistic pattern, identifying the optimization signature of NBA.
  • Guidance-scale sensitivity: At γ = 1, text-rendering teacher, naive student, and PDM student obtain OCR rewards of 75.24, 73.87, and 74.48, respectively.All improve similarly as guidance increases, indicating no substantial additional sensitivity from either distillation objective.
  • Guidance-scale sensitivity: In reference-conditioned distillation, naive matching departs from the teacher with distortion and style drift, while PDM closely tracks the teacher across guidance scales.The excess degradation is most pronounced at γ = 1 and, together with branch-error trajectories, indicates NBA-induced sensitivity.

4.2 APPLICATION TO DENSE-TO-SPARSE VIDEO CONTROL

The dense-to-sparse video-control application transfers dense teacher control to a sparse-keyframe student using branch-aware OPD. PDM improves matched-scale control fidelity and remains robust when inference guidance changes, unlike naive matching.

  • Experimental setup: The teacher receives dense control sequences, while the student observes only sparse keyframes in the Wan-VACE application.The benchmark contains 600 OpenHumanVid test clips covering pose, depth, and scribble control.
  • Experimental setup: PDM uses λ = 1 and supervises the first K = 8 student-visited states in the main video experiments.Guidance-scale and hyperparameter studies use pose-only models and are not directly comparable with the jointly trained main protocol.
  • Main results: PDM provides the most consistent control-fidelity improvements across pose, depth, and scribble at the training guidance scale γ = 5.IBM also outperforms naive matching on most control metrics, while PDM’s stronger overall results support its positive–direction parameterization.
  • Main results: At γ = γtrain, PDM improves control fidelity over naive OPD, SFT, and off-policy distillation across all three modalities.This establishes a matched-scale knowledge-transfer benefit independent of guidance-scale shifts.
  • Guidance-scale generalization: PDM remains stable across inference scales, whereas naive matching degrades sharply as the inference scale moves away from γtrain = 5.PDM provides the most consistent control fidelity across the evaluated scales, including the largest naive-matching drop at γ = 1.

4.3 ABLATION STUDIES

Ablations identify λ = 1 and K = 8 as effective defaults for balancing control fidelity with training efficiency in dense-to-sparse video control.

  • Conditional-direction weight: λ = 1 provides the best overall control fidelity, including the lowest keyframe MPJPE and highest PCK@0.1.Larger λ values improve some distributional-quality metrics but weaken control accuracy.
  • Supervision horizon: Training cost rises from 23 seconds per step at K = 1 to 790 seconds at K = 50, while K = 8 gives the strongest evaluated control fidelity short of full-trajectory supervision.The paper therefore uses K = 8 as the main video-experiment default.

5 RELATED WORK

Prior diffusion-distillation methods often bake CFG into a single prediction, while diffusion OPD can retain separate branches. This work studies the underexplored case where branch-retaining OPD matches only CFG-composed predictions under conditioning asymmetry.

  • Few-step distillation methods commonly transfer CFG through a single composed teacher prediction, avoiding separate positive and negative branches at inference.This design targets sampling efficiency and transfers the strongly guided teacher distribution into a compact generator.
  • The paper’s setting retains separate teacher and student CFG branches and matches their final guided predictions, making the objective under-identified at branch level.The ambiguity becomes harmful when privileged information appears in the teacher’s negative branch but is unavailable to the student.
  • Diffusion OPD instead lets the student generate its trajectory while the teacher supplies dense local supervision at student-visited states.Existing configurations either collapse CFG into one student prediction or retain separate branches with shared negative conditioning.
  • Prior OPD configurations did not examine how privileged information in the teacher’s negative branch changes optimization for a branch-retaining student.The work studies this missing regime and distinguishes benign shared-conditioning behavior from harmful asymmetric behavior.

6 CONCLUSIONS AND LIMITATIONS

The paper identifies when CFG-composed OPD’s branch ambiguity becomes harmful and proposes PDM to address it. Its remaining limitation is that the relative optimization behavior of PDM and IBM is not theoretically characterized.

  • CFG-composed OPD is under-identified, but ambiguity is benign under shared negative conditioning and can become harmful with privileged negative conditioning.Privileged negative conditioning can induce antagonistic branch-error dynamics, termed Negative Branch Asymmetry (NBA).
  • PDM separately supervises the positive prediction and CFG conditional direction, while dense-to-sparse video experiments show improved transfer and guidance-scale robustness.Contrasting image-domain studies validate when NBA emerges.
  • PDM and IBM share the same branch-level zero-loss solution under non-degenerate weights, so PDM’s observed advantage is currently empirical rather than theoretical.Future work may characterize their different optimization behavior and extend NBA analysis to other guidance mechanisms and conditioning asymmetries.

7.1 ADDITIONAL EXPERIMENTAL DETAILS

The experiments use adapter-based teacher–student distillation across image and dense-to-sparse video settings, with controlled objectives, guidance scales, datasets, and qualitative evaluation procedures.

  • Text rendering distillation: Stable Diffusion 3.5-Medium uses rank-32, scale-64 LoRA adapters for teacher and student, with the student initialized from the base model and solely updated.Adapters target query, key, value, and output projections in self- and cross-attention blocks; other weights remain frozen.
  • Text rendering distillation: The image-domain comparison keeps rollouts and hyperparameters identical while varying the naive CFG-composed objective, PDM with λ = 2, and a positive-only ablation.Branch errors are per-transition ℓ2 norms in transition-mean space, averaged over supervised timesteps.
  • Text rendering distillation: OCR evaluation uses 19,652 training prompts and 1,018 test prompts, scoring generated text by Levenshtein-based exact-readback quality.A score of 1 means the rendered text is read back exactly, while 0 means no characters match.
  • Image-domain evaluation: Guidance-scale generalization fixes the training configuration at γtrain = 4.5 and evaluates γ ∈ {1, 2, 3, 4, 4.5} with 40 denoising steps.The reference-conditioned image experiment evaluates γ ∈ {1, 1.5, 2, 2.5} using shared initial noise across objectives and scales.
  • Dense-to-sparse video control: The video benchmark uses 81-frame, 832 × 480 clips derived from OpenHumanVid and samples 100 clips from each of six motion-difficulty buckets for a 600-clip test set.The teacher receives dense controls, while the student receives sparse keyframes; all methods are evaluated independently and averaged per clip.
  • Dense-to-sparse video control: Video conditioning reserves frame 0 for the reference image and supplies pose controls at pixel-space keyframes 20, 40, and 60, leaving remaining frames to be synthesized.The same sparse construction is used during on-policy training and inference, with masks constructed in pixel space before compression.

7.2 MORE QUALITATIVE RESULTS

Additional qualitative results consistently favor PDM over naive matching under guidance-scale changes and across dense-to-sparse control modalities. The comparisons emphasize reduced appearance drift, preserved structure, and more faithful sparse-condition following.

  • Reference-conditioned distillation: In reference-conditioned distillation, PDM more closely follows the teacher and preserves reference-supplied style across guidance scales, while naive matching shows appearance drift and artifacts.
  • Dense-to-sparse video control: In dense-to-sparse video control, PDM maintains subject appearance and controlled structure as inference guidance changes, whereas naive matching degrades.Figure 6 highlights severe blurring and structural degradation when naive matching moves away from its training configuration.
  • Control modalities: Across pose, depth, and scribble modalities, PDM follows sparse structural conditions more faithfully while maintaining coherent visual content.The consistency across modalities supports branch-aware supervision under different conditioning asymmetries.
  • Control modalities: Under the default guidance scale γ = 5, PDM produces more consistent structure and visual quality for dense-to-sparse pose and depth control.
  • Control modalities: For dense-to-sparse scribble control at γ = 5, PDM better preserves scribble-guided structure and generates more consistent visual results than other objectives.
Loading 2607.24731v2…