Source-linked AI summary
Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models
Siming Fu, Haojun Xu, Ruizhe He, Zheming Fu, Hualiang Wang, Jie Huang, Xiaoxiao Ma, Mingchen Zhong, Weihu Huang, Xiaoxuan He, Linjiang Huang, Si Liu
TL;DR
Text-to-image specialists often have complementary strengths, but incompatible autoencoders and noise schedules hinder consolidating them. Poly-OPD uses heterogeneous on-policy distillation to produce a switchable compact student, raising GenEval from 67.3 to 73.3 and DrawBench HPSv3 from 9.34 to 11.35.
Problem
Text-to-image specialists rarely combine preference-aligned aesthetics and compositional instruction following, while heterogeneous latent spaces prevent direct distillation into one selectable student.
Method
Poly-OPD bridges teacher latent spaces through student-generated pixels, semantic-space supervision, compatibility-based adapter sharing, and gap-aware capability allocation.
Results
GenEval rises from 67.3 to 73.3, surpassing both larger teachers, while DrawBench preference metrics also improve, including HPSv3 from 9.341 to 11.354.
Takeaways & Limitations
A single 2.5B student can consolidate complementary teacher capabilities while retaining inference-time capability selection through adapter switching.
Abstract
from arXiv · showhide
Leading open text-to-image models often carry complementary strengths: one may lead on preference-aligned aesthetics while another follows compositional instructions more faithfully. However, differences in their autoencoders and noise schedules make it difficult to transfer these strengths across models. In this paper, we present Poly-OPD, a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student. To bridge the incompatible latent spaces of different teachers, Poly-OPD performs on-policy distillation through a pixel bridge. Each student-generated image is re-encoded by a selected teacher's encoder and refined from a noise level matched by magnitude under the teacher's noise schedule. The resulting target is further matched to the student in frozen DINOv2 space, enabling supervision across incompatible latent spaces. To retain complementary capabilities without cross-teacher interference, Poly-OPD uses a gradient compatibility diagnostic to organize its adapters: attention LoRA modules are shared across teachers, whereas feed-forward adapters remain teacher-specific. During distillation, a gap-aware curriculum devotes more training to compositional categories where the student still falls short of the teacher. As each gap narrows, training shifts toward categories with larger remaining gaps. By distilling FLUX.1-dev and Z-Image into a 2.5B SD3.5-Medium student, Poly-OPD improves GenEval from 67.3 to 73.3, surpassing both larger teachers, and raises DrawBench HPSv3 from 9.34 to 11.35, consolidating both strengths within a switchable model.
1 Introduction
Poly-OPD addresses multi-capability consolidation by distilling architecturally heterogeneous, latent-incompatible teachers into one flow-matching student with selectable capabilities. It bridges teacher latent spaces through pixels and combines on-policy supervision, compatibility-based adapter organization, and gap-aware sampling.
- Motivation: Specialist text-to-image models divide strengths between preference-aligned visual quality and compositional instruction following, forcing applications needing both to deploy multiple large models.This incurs the memory, latency, and maintenance cost of each model.
- Problem: Multi-capability consolidation distills frozen, architecturally heterogeneous teachers into one flow-matching student whose capabilities match teachers, avoid mutual degradation, and remain selectable at inference.The setting breaks the central assumption of existing distillation because teacher architectures and latent spaces differ.
- Method: Poly-OPD decodes each student sample to pixels, re-encodes it into the active teacher’s latent space, and refines it from a noise level matched by magnitude.The refined image provides an on-policy teacher correction of the student’s actual output.
- Contribution: The framework makes inference capability selectable while consolidating heterogeneous teacher expertise in a single student, where existing latent-space distillation is inapplicable by construction.The contribution targets architecturally heterogeneous, latent-incompatible teachers within one flow-matching model.
- Method: Poly-OPD uses semantic-space supervision, gradient-compatibility measurements to determine adapter sharing or isolation, and gap-aware sampling to allocate compositional training budget.These mechanisms address incompatible supervision, conflicting capabilities, and uneven remaining gaps across compositional categories.
2 Related Work
Prior work accelerates diffusion and flow-matching generation through several distillation objectives, while recent on-policy methods supervise student-visited states. Multi-teacher distillation and knowledge amalgamation transfer complementary capabilities, including across heterogeneous architectures, but existing on-policy approaches assume compatible teacher and student dynamics.
- Diffusion distillation: Diffusion distillation accelerates generation through trajectory matching, consistency training, adversarial objectives, and distribution matching.Diffusion and flow-matching models generate through iterative denoising or continuous transport from noise to data.
- Diffusion distillation: Recent on-policy methods supervise states visited by the student, using transition-level objectives or dense velocity supervision for task-specialized flow teachers.CollectionLoRA also consolidates multiple customized LoRA teachers and few-step generation into a shared adapter.
- Diffusion distillation: Existing on-policy approaches assume compatible transition kernels, vector fields, or model-derived supervision.
- Multi-teacher adaptation: Multi-teacher knowledge distillation transfers complementary knowledge from several teachers into a compact student.Knowledge amalgamation extends this setting to teachers trained for different tasks or built with heterogeneous architectures through feature transformation or selective teacher supervision.
3 Method
Poly-OPD distills heterogeneous teachers through a pixel bridge and frozen DINOv2 supervision, aligning teacher refinement with the student’s noise schedule. It preserves complementary capabilities by sharing attention LoRA, isolating FFN adapters, and sampling compositional categories according to remaining teacher–student gaps.
- On-policy distillation: Student-generated images are decoded into pixels, re-encoded in the selected teacher’s latent space, refined from a matched noise level, and compared in frozen DINOv2 space.Pixel space is the shared coordinate system, while DINOv2 CLS features transfer semantic content rather than model-specific texture statistics.
- Gap-aware curriculum: Compositional sampling reallocates training toward categories with the largest remaining teacher–student gaps instead of categories with the lowest absolute teacher scores.This avoids spending equal budget on saturated and unsolved categories and avoids treating raw difficulty as a proxy for useful supervision.
- On-policy distillation: The refinement depth r controls how much the teacher overrides the student, ranging from an off-policy teacher target at r = Ne to near-student behavior at r = 1.Intermediate refinement depths interpolate between the two stages, and the same control also affects training cost.
- Capability-selectable adapters: Attention gradients remain aligned across modes, whereas FFN gradients become negative in several blocks, motivating shared attention LoRA and teacher-specific FFN adapters.Only the FFN path is duplicated, so capability switching requires an adapter swap rather than a second model.
4 Experiments
Experiments show that Poly-OPD consolidates preference and composition capabilities in a single 2.5B student, outperforming teacher references on several metrics. Ablations identify adapter selection, warm start, gap-aware sampling, refinement range, and DINOv2 supervision as important components.
- Models and training: SD3.5-Medium is distilled from FLUX.1-dev for preference mode and Z-Image for composition mode using 500 warm-start steps followed by 800 OPD steps.Training uses a 20-step student Euler sampler, DINOv2 CLS supervision, refinement depths r ∈[15, 20], and gap-aware sampling over five composition categories.
- Overall comparison: A single 2.5B student surpasses both larger teachers on GenEval and exceeds FLUX.1-dev on ImageReward and UR-Alignment.The evaluation compares Poly-OPD with the original student and both teacher references using matched adapters and benchmark-specific sampling protocols.
- Preference mode: 1.168 ImageReward and 11.354 HPSv3 improve from 0.922 and 9.341, respectively, while PickScore remains essentially unchanged at 0.869 versus 0.866.Against FLUX.1-dev, the student also surpasses ImageReward and UR-Alignment and recovers about 78% of the HPSv3 margin.
- Composition mode: 73.30 GenEval accuracy rises from 67.30, exceeding Z-Image by 3.90 points and FLUX.1-dev by 8.10, with the largest gains in two-object, attribute, and position categories.Two-object composition improves by 14.93 points, attribute binding by 8.25, and position by 7.20; DPG-Bench rises from 84.51 to 85.80.
- Architecture and sampling: 11.0 GenEval points are lost without capability-specific adapter selection, while removing warm start degrades GenEval by 24.1 points and DPG-Bench by 4.80.Shared adapters average conflicting FFN updates, whereas direct OPD from the base student receives unreliable teacher refinement before its distribution approaches the teachers’.
5 Conclusion
Poly-OPD enables on-policy distillation from heterogeneous teachers despite incompatible latent spaces and noise schedules. Its pixel bridge, gradient-measured adapter sharing, and gap-aware sampling consolidate capabilities in one frozen backbone, with selection requiring only an adapter swap.
- Conclusion: Poly-OPD distills from heterogeneous teachers whose latent spaces and noise schedules are incompatible with the student.The framework uses on-policy distillation to address these incompatibilities.
- Conclusion: Its pixel bridge converts incompatible teachers into sources of on-policy supervision.The bridge is the mechanism that enables supervision across the differing teacher and student representations.
- Conclusion: Gradient measurement determines which adapters are shared or isolated across teacher capabilities.This sets the sharing–isolation boundary for the adapters.
- Conclusion: Gap-aware sampling anneals as teacher–student residuals close, concentrating training according to remaining gaps.The sampling strategy changes as the residuals diminish.
- Conclusion: One frozen backbone carries every capability, and selection costs an adapter swap.Capability selection therefore occurs through changing the active adapter rather than the backbone.