Source-linked AI summary
DanceOPD: On-Policy Generative Field Distillation
Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, Tat-Seng Chua
TL;DR
Image generators must combine capabilities whose objectives can interfere, including T2I, local editing, and global editing. DanceOPD composes frozen capability velocity fields by hard-routing samples, querying one low-noise state from the student rollout, and matching velocities with MSE. Experiments report stronger target capabilities while preserving anchor generation quality across composition and field-absorption settings.
Problem
T2I, local editing, and global editing have conflicting objectives, making effective composition in one deployed model a central training challenge.
Method
DanceOPD hard-routes each sample to one capability field, queries it at one low-noise student-induced rollout state, and trains with velocity MSE.
Results
DanceOPD improves multi-capability composition while strengthening target fields and preserving anchor generation quality across T2I/editing, realism-field, and CFG absorption experiments.
Takeaways & Limitations
On-policy generative field distillation is presented as a practical route for composing and absorbing multiple capabilities in flow-matching models.
Takeaways & Limitations
The method assumes compatible velocity fields over a shared generative state space and predefined capability routes, which become weaker when task boundaries are ambiguous or prompts require several capabilities at once.
Abstract
from arXiv · showhide
Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing. However, these capabilities are rarely naturally aligned and often conflict. For instance, editing tends to degrade T2I performance, while global and local editing interfere with each other. Consequently, effectively composing these capabilities has become a central challenge for image generation model training. To tackle this, we introduce DanceOPD, an on-policy generative field distillation framework for flow-matching models that routes each sample to one capability field, queries one low-noise student-induced state, and trains with a simple velocity MSE objective. With each capability source defined as a velocity field over the shared flow state space, the student learns from fields queried on its own rollout states to compose expert capabilities. This formulation also absorbs operator-defined fields such as classifier-free guidance. Comprehensive experiments on T2I, editing, realism-field absorption, and CFG absorption show that our approach improves multi-capability composition, strengthening target capabilities while preserving anchor generation quality. We believe this work establishes a practical route for generative field distillation in flow-matching models.
1 Introduction
DanceOPD addresses interference among T2I, local editing, and global editing by composing capability-specific velocity fields on student-induced states. Its hard-routed, single-query design improves composition while preserving anchor generation quality.
- Hard-routed sample-wise matching assigns each sample to exactly one capability field, preserving its semantic identity.This avoids ambiguous targets created by averaging fields from different capabilities.
- Student-rollout querying aligns supervision with the states the composed model actually visits rather than off-policy states.The routed field is queried on a stop-gradient state from the current student rollout.
- A single semantic-side low-noise query avoids correlated within-trajectory supervision while targeting capability-specific information.The design addresses trajectory-query correlation and concentrates supervision where semantic information is most concentrated.
- 8.1% improvement on GEditBench over the best reproduced OPD baseline accompanies a slight GenEval advantage over the T2I source in T2I and editing composition.The same setting improves over the edit source by 8.5%.
- 16.1% improvement over the best competing composition baseline is reported for local and global edit composition, with GenEval above all compared composition baselines.The model also improves over the local edit source by 7.9%.
- DanceOPD formulates multi-capability generation as on-policy generative field distillation over shared flow states.Each frozen capability source is treated as a velocity field, turning composition into choices about the target field, query state, and number of queried states.
2 Related Work
Related work frames DanceOPD as complementary to existing distillation and capability-composition approaches. The paper combines flow-matching on-policy distillation with multi-capability composition and functional field absorption.
- Multi-capability generation spans T2I, local editing, global editing, and style-specialized generation, whose objectives emphasize different preservation and transformation behaviors.
- Standard distillation methods mainly target compression, sampler or trajectory training, consistency training, or distribution matching rather than this field-composition setting.
- Table 1 positions DanceOPD as combining flow-matching OPD, multi-capability composition, design-space analysis, and functional field absorption.
- Flow-model OPD methods provide related on-policy velocity-matching ideas, while DanceOPD focuses on composing T2I, editing, realism, and guidance capabilities.
- Flow matching represents generation through velocity fields over continuous state space, allowing models and guidance operators to act as local velocity-supervision sources.
3 Approach
DanceOPD trains one student to imitate multiple frozen capability fields through routed queries on its own rollout. The method uses semantic-side single-query supervision and direct velocity regression, extending field matching to guidance and other edits.
- DanceOPD trains a student to imitate frozen capability sources as velocity fields on a shared generative state space.This replaces static parameter interpolation or data-ratio tuning with a field-query formulation.
- Hard-Routed Sample-Wise Field Matching: Each sample is hard-routed to one capability field so its supervision target remains semantically meaningful.Uniform route probabilities are used over active capability buckets unless otherwise stated.
- On-Policy Student-State Querying: The selected field is queried on a stop-gradient state from the current student rollout, addressing mismatch between off-policy training states and student visitation states.
- Semantic-Side Single Query: One semantic-side low-noise query per sample limits correlated supervision from multiple states sharing the same rollout history.Dense trajectory queries can provide non-independent gradients because states share noise, conditioning, dynamics, and path history.
- Objective Design: Plain velocity MSE is used to regress the deterministic target field and is described as the natural local objective under an isotropic Gaussian velocity-likelihood view.
- Classifier-free guidance can be represented as an additional guided velocity field and absorbed into the student with the same MSE objective.
- DanceOPD preserves task-relevant content in material, environment, and lighting edits while applying broad or local visual transformations.
4 Experiments
DanceOPD composes T2I, editing, realism, and CFG capabilities while preserving anchor generation quality. Controlled diagnostics show that hard routing, single low-noise on-policy queries, moderate rollouts, plain velocity MSE, and relevant initialization are important design choices.
- T2I and Edit Composition: 8.1% higher GEditBench average than the best reproduced OPD baseline, while GenEval improves 2.0% over the T2I source.DanceOPD also exceeds the edit source by 8.5% and the strongest composition baseline by 1.6% on GenEval.
- Local and Global Edit Composition: 16.1% higher GEditBench average than the best competing composition baseline when composing local and global editing, with GenEval improving 2.5%.Category gains include 33.5% for background change, 12.9% for style change, and 11.6% for color alteration.
- Realism Absorption: 9.9% higher realism reward than off-policy distillation while closing 85.3% of the student-to-teacher reward gap and keeping T2I within 0.1% of off-policy distillation.The T2I score is also 7.6% above the student anchor.
- CFG Absorption: 7.6% better GEditBench composition than train-only CFG absorption and 1.4% better than eval-only CFG, while excessive guidance composition over-guides the model.Training and inference guidance scales interact, so absorbed and external CFG are not independent.
- Routing Diagnostics: 15.2% higher average under MSE and 10.6% under KL than soft all-teacher mixing, supporting hard-routed sample-level capability identity.Soft mixing averages teacher fields into an ambiguous target rather than preserving the intended capability query.
- Trajectory Queries: The single-query default outperforms weighted dense-query variants by 16.6%, 7.9%, 10.2%, and 12.2% for K=2, 4, 8, and 16.Correlated states from one rollout do not necessarily provide better supervision.
5 Conclusion
DanceOPD composes heterogeneous generative capabilities by treating frozen sources as velocity fields queried on student-visited states. Experiments show stronger target capabilities while preserving anchor quality, supporting on-policy generative field distillation as a practical route for multi-capability image generation.
- DanceOPD treats each frozen capability source as a velocity field over a shared generative state space.Capability composition is formulated as a field-query problem.
- The method uses hard-routed sample-wise field matching and student-visited query states to address target-field ambiguity and state-distribution mismatch.These design choices preserve capability identity and align supervision with the student rollout.
- Experiments cover T2I and editing composition, local and global editing composition, realism-field absorption, and CFG absorption.The evaluation spans multiple capability-composition settings.
- DanceOPD strengthens target capabilities while preserving anchor capabilities across the evaluated composition settings.Ablations support hard routing, student-induced query states, semantic-side single-query supervision, and direct velocity regression.
- The results suggest that on-policy generative field distillation is a practical route toward scalable multi-capability visual generation.
6 Limitations and Discussions
DanceOPD assumes compatible velocity fields and predefined capability routes. These assumptions are well matched to the experiments but weaken when sources or task boundaries are not clearly shared.
- The formulation assumes frozen capability sources expose compatible velocity fields over a shared generative state space.The experiments satisfy this through a shared backbone family, latent representation, scheduler convention, and velocity parameterization.
- Predefined sample-wise hard routing works when capability buckets and task or data identities are known.The implementation separates T2I, local editing, global editing, style, and guidance-field examples.
- Routing becomes weaker when task boundaries are ambiguous or a prompt requires several capabilities at once.The paper identifies verifier- or reward-model-based routing as a natural extension.
7 Theoretical Details
DanceOPD frames capability composition as querying frozen velocity fields at student-visited states, with hard routing and one semantic-side query addressing alignment challenges. Plain velocity MSE provides the default matching objective, while the framework also supports CFG absorption.
- Field-query formulation: Capability composition becomes a field-query problem over a shared flow state space, requiring choices about target field, query state, and rollout-state count.Each frozen capability source defines a velocity field, and the student imitates these fields during its own generation process.
- Objective: Under shared-covariance local Gaussian transitions, KL-style field matching reduces to timestep-weighted velocity MSE, supporting the unweighted MSE default for deterministic velocity fields.The paper reports that unweighted MSE offers the strongest stability and performance trade-off among tested objectives.
- On-policy querying: The teacher field is queried on a stop-gradient state from the current student rollout, aligning supervision with deployment-time visitation rather than fixed off-policy states.The practical estimator avoids differentiating through every solver step while retaining on-policy state coverage.
- Hard routing: Hard routing assigns each sample to one capability field, preserving semantic identity and avoiding within-sample field-conflict bias.Same-step accumulation avoids target bias but still averages gradients from multiple capability buckets in one optimizer update.
- Alignment challenges: Mixing several capability fields within one sample can create a target direction that corresponds to no well-defined capability, causing target-field ambiguity.The mixed target may inject irrelevant directions when fields encode unrelated or conflicting capabilities.
- Field absorption: DanceOPD can absorb classifier-free guidance by treating its guided velocity as another capability field and matching it directly with MSE.In the realizable case, the student matches the guided field on queried states, allowing deployment without recomputing the affine guidance extrapolation.
8 Implementations
The implementation uses on-policy student rollouts, hard-routed teacher supervision, and one queried state for the default method. Reproductions differ mainly in query-state distributions, trajectory density, and optimization procedures.
- Query density: DanceOPD uses one semantic-side query state per routed sample, whereas DiffusionOPD supervises all 16 rollout states.The shared rollout makes on-policy methods more expensive than off-policy distillation, while dense supervision multiplies teacher-query and backward costs.
- Computational complexity: Flow-OPD also uses dense trajectory supervision and requires two micro-batches for its group size, approximately doubling its wall-clock cost relative to the DiffusionOPD reproduction.The additional overhead includes SDE sampling, cached log-probabilities, and PPO clipping.
- Off-policy baseline: The off-policy reproduction changes only the query-state distribution, replacing student-visited states with fixed endpoint-forward-noised states.It retains hard-routed teacher supervision and a plain velocity-regression objective.
- DanceOPD configuration: DanceOPD generates a 16-step stop-gradient student rollout during training, but this rollout length does not constrain the deployed sampler.Training uses the rollout to generate query states for per-state velocity matching, while benchmark inference uses its own sampling configuration.
9 Additional Qualitative Results
The qualitative examples show DanceOPD combining substantial edits with preservation of source structure and identity. It handles global transformations, local changes, and diverse edits applied to one shared object.
- Global edits: DanceOPD performs large scene and style transformations while preserving geometry, foreground structure, face, pose, and portrait composition.Examples include converting a warm canal scene into a snowy blue night and applying a cyberpunk portrait style.
- Local edits: For local edits, DanceOPD changes material, color, tidiness, or staging while retaining the subject pose, background, or room identity.The examples contrast targeted gown changes and room restaging with baselines that introduce broader style or layout drift.
- Shared-object edits: Across six instructions on one water bottle, DanceOPD supports surface, decorative, contextual, structural, and material transformations.The bottle identity, cap shape, upright composition, and central placement are generally preserved across these edits.
10 Additional Quantitative Results
Additional diagnostics isolate guidance composition, rollout-step choice, and qualitative edit behavior rather than replacing the main composition results. The reported controls clarify how training queries and external guidance should be interpreted.
- CFG diagnostics: The CFG diagnostics separate absorbed training guidance α from external inference guidance β, whose effective strength is approximately αβ.Overly large combined guidance produces the expected over-guidance failure mode.
- Rollout diagnostics: Longer training rollouts do not automatically improve GEditBench or GenEval because the rollout generates field-matching query states rather than compressing a trajectory.The benchmark evaluation sampler remains fixed across rollout-step comparisons.
- Global-edit comparison: DanceOPD better follows large global transformations while preserving image structure than off-policy distillation, joint training, DiffusionOPD, and Flow-OPD.The comparison is reported for global scene and style edits.
- Local and global composition: DanceOPD preserves content under local edits and produces stronger global transformations than the competing baselines.The figure compares local and global edit composition in a shared evaluation block.
- Object-level composition: DanceOPD supports material, style, scene, structure, and transparency transformations on the same source object.The figure evaluates diverse object-level transformations using one shared object.