Timing Matters Twice: When to Query and When to Skip in Flow-Matching On-Policy Distillation

1. Introduction

• Existing multi-objective methods combine heterogeneous reward signals, but sparse rewards and competing update directions can cause gradient interference and imbalanced capabilities.Multi-teacher OPD instead trains specialized teachers independently and supervises the student on its own generation trajectories.Source: §1 · pp. 1–2
• OPD does not characterize how each task-specific teacher’s supervision value changes across generation stages, leaving the best query state for each teacher-task pair unresolved.The problem also includes when to stop rollout after queries are complete, since continued steps can add little information and incur computational cost.Source: §1 · p. 2
• PhaseOPD schedules structure supervision during global layout, structure-and-detail supervision during transition, and visual-quality supervision during later refinement.The task-specific teacher determines what is supervised; student trajectory dynamics determine when.Source: §1 · p. 2
• PhaseOPD introduces teacher-specific query timing and dynamic rollout truncation, adding generation dynamics as a dimension orthogonal to task assignment.It identifies useful generation intervals for each task-specific teacher and stops subsequent rollout once required query positions are determined.Source: §1 · p. 2

2. Motivation

• Applying a teacher at an incompatible stage may produce low-value or competing gradients, because local-detail objectives can disrupt structure formation and strong structural supervision can constrain later detail enhancement.Detail-oriented supervision depends on an established global structure as its underlying carrier.Source: §2 · p. 2
• The paper compares timing to course sequencing: learning depends on whether what is taught matches a learner’s current developmental stage.This analogy frames supervision timing as distinct from choosing what objective to teach.Source: §2 · pp. 2–3
• Semantic-structure teachers perform best when queried early, OCR teachers in the middle, and aesthetic-quality teachers in later generation stages.The described differences motivate teacher-query timing that reflects each objective’s denoising-stage preference.Source: §2 · p. 3

3. Method

• PhaseOPD combines adaptive generation-stage identification, dynamic teacher routing, and rollout truncation to reduce cross-stage supervision conflicts and redundant computation.The framework determines the current stage from the student's trajectory, then schedules task-specific teachers according to that stage.Source: §3 · p. 3

3.1. Generative Models and On-Policy Distillation

• Flow matching interpolates clean data and noise while learning a prompt-conditioned velocity field; multi-teacher OPD distills teacher policies on states visited by student trajectories.Each teacher is trained for a different objective, and the student generates its own trajectories for online supervision.Source: §3.1 · p. 3

3.2. Online Generative Phase Perception

• Student clean-latent revisions are decomposed into block-level and local changes; changes in their ratio define slope-based boundaries for structure, local-content, and aesthetic stages.The spatial-scale ratio measures revision composition, not image quality or total revision magnitude.Source: §3.2 · pp. 3–5

3.3. Teacher-Conditioned Phase Routing

• Structure teachers query just before Stage I's left boundary, local-content teachers sample Stage II, and visual-quality teachers query just after Stage III's right boundary.GenEval, OCR, and Aesthetic are representative teacher categories rather than a fixed configuration.Source: §3.3 · p. 5

4. Experiments

• The evaluation uses an SD3.5-Medium student with rank-32 LoRA adapters, 16-step rollouts, three frozen teachers, and Aesthetic, GenEval, and OCR as primary metrics.The experiments also test alternative teachers, other OPD hosts, and BAGEL and FLUX backbones.Source: §4.1 · pp. 5–6, §4 · p. 5

4.3. Compatibility Experiments

• Adding PhaseOPD to DanceOPD, DiffusionOPD, and FlowOPD improves generation quality and training efficiency without changing their distillation objectives.The plug-and-play comparison covers all three underlying OPD methods.Source: §4.2 · p. 6, Figure 4 · p. 6
• With alternative teachers, PhaseOPD achieves the best HPSv3, SigLIP, and independent diagnostic scores among multi-teacher methods while remaining competitive on OCR.The paired examples provide qualitative support across HPSv3, SigLIP, and OCR prompt types.Source: §4.3 · pp. 6–7, Table 2 · p. 7, Figure 6 · p. 7

4.2. Main Results

• PhaseOPD offers a better overall balance across Aesthetic, GenEval, and OCR, reaches competitive quality earlier under the same wall-clock budget, and improves the quality–time trade-off throughout training.The study also reports higher training efficiency and improvements on held-out ImageReward and VQAScore diagnostics.Source: §4.2 · p. 6, Figure 3 · p. 6, Table 1 · p. 7

4.6. Mechanism Analysis

• GenEval queries concentrate early, OCR queries favor the middle, and Aesthetic queries span early and late stages; the preferences persist across OPD objectives.Online boundaries use changes in the spatial-scale revision ratio and its first-order difference, computed from current and past student states.Source: Figure 8 · p. 8, §4.6 · pp. 8–9, Figure 9 · p. 8

4.4. Generalization Experiments

• On BAGEL, PhaseOPD improves text-to-image generation and local editing, remains competitive on global editing, and reduces overall training cost; on FLUX, it beats DanceOPD across generation quality metrics and training efficiency.The BAGEL setting changes both backbone and task formulation, while FLUX retains text-to-image generation and changes the backbone.Source: §4.4 · p. 8, Figure 7 · p. 8, Table 3 · p. 9

4.5. Ablation Studies

• Removing early skipping substantially increases training time with only slight GenEval and OCR decreases, while removing adaptive querying lowers all three quality metrics and raises training time.The full model has the best Aesthetic and OCR scores, second-best GenEval, and highest training efficiency.Source: §4.5 · p. 8, Table 4 · p. 9

4.7. User Preference Study

• In the double-blind study, PhaseOPD receives higher user preference rates than DanceOPD and DiffusionOPD, consistent with perceptible gains beyond metric improvements.Pairs share prompts, seeds, and inference settings and are judged on visual quality, text alignment, and content completeness.Source: Figure 10 · p. 9, §4.7 · p. 9

5. Conclusion

• PhaseOPD links teacher-supervision timing with early termination of student rollouts.Causal signals from predicted-clean latent revisions identify teacher-specific query states, allowing single-query distillation without generating the remaining trajectory.Source: §5 · p. 9
• Experiments report fewer rollout steps, lower training cost, and the strongest GenEval score in the comparison.Source: §5 · p. 9
• PhaseOPD achieves faster first-hour gains on GenEval and OCR.The results support jointly designing where to learn and how far to roll out as a practical route to efficient multi-capability post-training.Source: §5 · p. 9

A.1. Post-training generative models.

• DDPO applies policy gradients to image rewards, Diffusion-DPO learns from pairwise preferences, and Flow-GRPO extends online policy optimization to flow matching.PhaseOPD integrates expert capabilities while reducing rollout computation by identifying supervision states online.Source: §A.1 · p. 11

A.2. On-policy capability transfer.

• PhaseOPD retains task-specific teachers and single-query regression rather than proposing a new regression loss or single-query distillation principle.Prior approaches include DiffusionOPD’s continuous-state formulation, Flow-OPD’s task routing and dense trajectory supervision, and DanceOPD’s low-noise velocity query.Source: §A.2 · p. 11

A.3. Stage specialization and efficient sampling.

• Unlike stage-specialized denoisers or progressive distillation, PhaseOPD changes training supervision timing and removes an unneeded training-trajectory suffix without changing inference.Its early stopping can be studied independently of methods that accelerate completed-image sampling.Source: §A.3 · p. 11

A.4. On-Policy Distillation

• PhaseOPD uses student dynamics to choose teacher-specific query positions and skips the remaining rollout once all query decisions are finalized.This addresses which state supports each teacher’s objective and how much rollout is needed to select it.Source: §A.4 · p. 11

B.1. Detailed Setup

• Training uses SD3.5-Medium at 512 × 512, LoRA with rank r = 32 and scaling factor α = 64, and frozen Aesthetic, GenEval, and OCR teachers.The task-specific training sets contain 25,106, 32,943, and 19,392 prompts, respectively; training uses 16-step rollouts and 3,000 updates on four H20 GPUs.Source: §B.1 · p. 11
• Evaluation measures aesthetic quality with AES on DrawBench, semantic alignment with GenEval, and text-generation accuracy with OCR.HPSv3, PickScore, SigLIP, ImageReward, and VQAScore are auxiliary metrics.Source: §B.1 · p. 11
• Baselines include the base model, three single-teacher models, DanceOPD, and DiffusionOPD; comparisons also apply DanceOPD and Ours to BAGEL and FLUX.Ours denotes DanceOPD augmented with phase-aware querying; applying the strategy to other objectives is labeled DiffusionOPD + Ours or FlowOPD + Ours.Source: §B.1 · p. 11

B.2. Time to Matched Quality

• 16.19, 32.38, and 16.19 minutes are the respective times for Ours to reach Aesthetic ≥5.70, GenEval ≥0.89, and OCR ≥0.90, earlier than DanceOPD and DiffusionOPD.The comparison is reported in Fig. 11 and complements final-checkpoint comparisons.Source: Figure 11 · p. 12, §B.2 · p. 12

• PhaseOPD improves both ImageReward and VQAScore over multi-teacher baselines under both teacher combinations.The seven models per comparison were evaluated on GenAI-Bench-1600 using shared prompts, seeds, and inference settings.Source: §B.3 · p. 12
• ImageReward and VQAScore serve as independent diagnostics beyond the rewards used to train the teachers.Source: §B.3 · p. 12
• The consistent diagnostic improvements support transfer beyond teacher-specific training objectives.Source: §B.3 · p. 12

B.4. Query Intervals and Teacher Routing

• GenEval’s largest relative gain occurs in R1, OCR’s in R2 and then R3, and Aesthetic’s in R3 and then R2.These controlled comparisons vary only the teacher query interval and report each teacher’s target metric relative to Beta.Source: §B.4 · p. 12, Figure 12 · p. 13
• GenEval performs best in R1, OCR in R2, and Aesthetic in R3, while R4 falls below Beta for all three objectives.The absolute scores come from separate single-teacher runs with queries restricted to fixed intervals.Source: Table 5 · p. 12, §B.4 · p. 12
• The Stage-1/2 mixture exceeds Stage-2-only and Stage-2/3 supervision on all three metrics, while balanced routing beats OCR-biased routing across reported scores.Both Ours rows reproduce the full configuration in Table 1.Source: §B.4 · p. 12, Table 6 · p. 13

B.5. Alternative-Teacher Learning Curves

• Under a shared 90-minute budget, the alternative-teacher comparison tracks HPSv3, SigLIP cosine, and OCR over time from a common SD3.5-M Base starting point.The compared methods are DanceOPD, DiffusionOPD, and Ours; the supplied passages describe the trajectory setup but not comparative outcomes.Source: §B.5 · p. 13, Figure 13 · p. 13

B.6. Sensitivity to Training Guidance

• 81.39 vs 161.89 minutes for DanceOPD + Ours, and 205.94 vs 395.57 for DiffusionOPD + Ours, show lower training time at CFG 1.0.At CFG 4.5, DanceOPD + Ours scores higher on all three metrics; for DiffusionOPD + Ours, CFG 1.0 improves GenEval while CFG 4.5 gives higher Aesthetic and OCR scores.Source: §B.6 · p. 13

B.7.1. Text Accuracy

• The shown text examples have fewer character errors and more complete rendered content, though final punctuation remains imperfect in one example.Source: §B.7.1 · p. 14, Figure 14 · p. 14

B.7.2. Text Visual Quality

• The shown examples combine readable lettering with coherent typography, supporting materials, and scene composition.Source: §B.7.2 · p. 15, Figure 15 · p. 15

B.7.3. Aesthetic Quality

• Selected scenes retain coherent illumination and material detail while showing clearer subject contours and visual separation.Source: §B.7.3 · p. 16, Figure 16 · p. 16

B.7.4. Semantic Alignment

• The shown counting results improve over DanceOPD while retaining key attributes and scene relationships.Source: §B.7.4 · p. 17, Figure 17 · p. 17

B.7.5. Joint Text and Scene Quality

• For prompts requiring text and coherent scenes together, the examples show less repeated text or subject occlusion and more complete text-bearing objects.Source: §B.7.5 · p. 18, Figure 18 · p. 18

B.7.6. Alternative Teachers

• With HPSv3, SigLIP, and OCR teachers, the examples retain scene detail and improve count alignment in the macaron example.Source: §B.7.6 · p. 19, Figure 19 · p. 19

B.7.7. Cross-Host Transfer

• Across two OPD hosts, the shown outputs retain readable text and exhibit richer material detail.Source: §B.7.7 · p. 20, Figure 20 · p. 20