CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation

1 Introduction

• Conventional effect-LoRA pipelines incur storage overhead, routing latency and errors, and conflicts with acceleration LoRAs that cause concept bleeding and style degradation.CollectionLoRA instead consolidates target concepts and the acceleration prior in one module through multi-teacher distillation.Source: §1 · pp. 2–3, Figure 2 · p. 3
• PDSR regularizes with unlabeled general-domain data, AOP separates teacher and student prompts using orthogonal trigger words, and C2F-DO combines trajectory anchoring with distribution matching to stabilize optimization and restore details.AOP assigns original prompts to teachers and VLM-rewritten prompts to the student; C2F-DO uses Target Simulation to restore realistic fine details.Source: §1 · pp. 2–3
• CollectionLoRA distills 50 visual effects and few-step generation into one LoRA, surpasses independent single-task teachers in concept fidelity, and scales to 180 effects at 0.5% of conventional deployment overhead.It also enables zero-shot composition of multiple effects at inference without additional training.Source: §1 · p. 3

2.1 Customized Image Generation

• Personalization has evolved from token learning and subject fine-tuning to adapters and in-context models, but zero-shot adapters can fail on out-of-distribution effects and LoRA-accelerator composition causes interference.The paper motivates distilling multiple customized effects into a single few-step LoRA to avoid conflicts inherent in multi-module composition.Source: §2.1 · pp. 3–4

2.2 Few-Step Generation

• Consistency Models and DMD enable few-step generation, but DMD methods largely target single-teacher settings and can destabilize or collapse when fitting multiple targets or bridging large teacher-student gaps.Decoupled-DMD [26] enhances fine details, while DMDR [19] and Flash-DMD [2] incorporate external preference rewards.Source: §2.2 · p. 4

2.3 On-Policy Distillation

• CollectionLoRA extends on-policy distillation to consolidate 50 to 180 diverse visual effects alongside few-step generation in one unified module.The authors group DMD under the OPD taxonomy because both correct student-explored states using teacher signals.Source: §2.3 · pp. 4–5

3 Preliminaries: Distribution Matching Distillation

• DMD trains an efficient student generator Gθ to make its generated distribution pfake approximate the teacher-defined real distribution preal.The generator is optimized by matching the score functions of the real and fake distributions.Source: §3 · p. 5
• DMD2 simulates inference from pure noise through iterative denoising and re-noising to a selected timestep, producing a simulated clean image.That image captures cumulative multi-step sampling characteristics and replaces conventional real data as the training target.Source: §3 · p. 5
• The generator update uses the fake-minus-real score difference multiplied by the generated sample’s parameter Jacobian; the real score comes from a frozen teacher and the fake score from a critic.The critic is updated using standard denoising loss.Source: §3 · p. 5

4.1 Problem Formulation

• Distilling multiple effect teachers into one student LoRA eliminates runtime retrieval and composition overhead as well as conflicts between effect and acceleration modules.The student fits target distributions from the effect teachers, replacing dynamic loading and composition at inference.Source: §4.1 · p. 7

4.2 Probabilistic Dual-Stream Routing

• PDSR randomly routes each training step to a general stream or an effect stream according to a preset switching probability.The general stream uses unlabeled general-domain images and the frozen base model as teacher; the effect stream loads an effect LoRA teacher and applies C2F-DO.Source: §4.2 · p. 7

4.3 Asymmetric Orthogonal Prompting

• AOP isolates concepts by retaining each teacher’s original prompt while giving the student a VLM-generated caption and a unique orthogonal trigger word.The captioning process is automatic, avoiding manual prompt engineering.Source: §4.3 · p. 8

4.4 Coarse-to-Fine Distillation Objective

• C2F-DO combines trajectory anchoring and distribution matching to stabilize distillation when teacher and student distributions differ substantially early in training.TA-FM stabilizes early optimization but can smooth high-frequency textures; target simulation restores detail, while generator and critic timestep bounds preserve teacher priors and support score guidance. Backward-simulation matching regularizes the teacher’s global style distribution.Source: §4.4 · pp. 8–9

4.5 Overall Objective

• The overall objective applies backward-simulation DMD on general-stream steps and C2F-DO on effect-stream steps, with routing states mutually exclusive.The training process naturally equips the model with few-step generation capabilities.Source: §4.5 · p. 9

5.1 Experimental Setup

• EffectBench evaluates animal and portrait edits using 5,000 instructions per model; training uses 50 effects with 20 image pairs each and 20K general source-image instruction pairs.The effect pairs are animal or portrait images; general-data pairs use MLLM-generated instructions and require no target images.Source: §5.1 · p. 10

5.2 Quantitative Evaluation

• At NFE=8, CollectionLoRA reaches CLIP 0.727, DreamSim 0.425, EditReward 1.052, BCR 0.087, and VSA 4.380, outperforming independent teachers (NFE=80) and FM+Lightning overall.Its DINO score is 0.600 versus 0.611 for the teacher; BCR is 0.087 versus 0.217 for the baseline and 0.141 for the teacher.Source: §5.2 · p. 11, Table 1 · p. 11
• For 10–50 LoRAs, CollectionLoRA bypasses routing with 0s latency, 100% accuracy, and 2.2GB storage; at 100–150, it uses VLM routing with 2% of baseline storage.At 100–150 LoRAs, it makes 136 model switches versus 200 and achieves 82% accuracy versus 76%.Source: Table 2 · p. 11, §5.2 · p. 12

5.3 Qualitative Evaluation

• Qualitatively, CollectionLoRA restores texture and realism, reduces style bleeding, and preserves OOD structure; chained prompts compose two effects zero-shot while retaining subject identity.The composition is performed in one forward pass without additional fine-tuning, and the general-domain stream supports structural fidelity for rare subjects.Source: §5.3 · pp. 12–13, Figure 6 · p. 12, Figure 7 · p. 13

5.4 Ablation Study

• AOP lowers BCR from 0.378 to 0.207; TS yields CLIP 0.736 and DreamSim 0.420; TA-FM gives VSA 4.380 and BCR 0.087; PDSR restores EditReward to 1.052.Qualitative ablations associate AOP with accurate task triggering, TS with texture restoration, TA-FM with structural consistency, and PDSR with reduced background blending.Source: §5.4 · pp. 14–15, Table 3 · p. 14, Figure 8 · p. 14
• Across 10–180 effects, CollectionLoRA outperforms FM+Lightning; at 10–50 effects it surpasses all baselines, while at 100–180 it remains competitive with Base+Lightning.The authors report a moderate performance drop at 100–180 effects, attributed to the increased optimization difficulty of extreme concept compression.Source: §5.4 · p. 15, Table 4 · p. 15
• Adding effects 51st–54th to the 50-effect model with 100 generator fine-tuning steps yields CLIP scores of 0.725–0.728 and outperforms Base+Lightning across settings.The reported extension does not show catastrophic forgetting and requires no retraining from scratch.Source: §5.4 · p. 15, Table 5 · p. 15
• In training dynamics, TS accelerates early convergence, while TA-FM smooths optimization oscillations and yields the most stable convergence curve.The baseline fluctuates under complex multi-task distributions, and TS alone remains volatile.Source: §5.4 · p. 15, Figure 10 · p. 16

6 Conclusion

• CollectionLoRA surpasses single-task teachers in concept fidelity, feature isolation, and high-quality generation while retaining few-step generation.Extensive experiments support these outcomes.Source: §6 · p. 16
• The unified framework integrates diverse customized visual effects and few-step inference into one module, eliminating multi-LoRA storage overhead and concept interference such as semantic drift.Source: §6 · p. 16
• Probabilistic Dual-Stream Routing provides structural regularization, Asymmetric Orthogonal Prompting isolates latent concepts, and Coarse-to-Fine Distillation stabilizes optimization and restores high-frequency details.Source: §6 · p. 16

• The supplementary document provides comprehensive technical details, in-depth theoretical analyses, and extensive visual results to support the main manuscript’s claims.Source: Overview of Supplementary Material · p. 21
• Appendices 7 and 8 cover dataset and evaluation protocols, followed by extended empirical analyses.Source: Overview of Supplementary Material · p. 21
• Appendix 9 presents implementation details.Source: Overview of Supplementary Material · p. 21
• Appendix 10 presents the user study and additional qualitative results.Source: Overview of Supplementary Material · p. 21

7.1 Dataset Details

• The dataset contains 180 distinct effects, each with approximately 20 training pairs for LoRA training.Source: §7.1 · p. 21

7.2 Prompt Design for Evaluation

• Qwen-VL-Max-Latest evaluates all test samples, using Figure 11 prompts for BCR and bad-case detection and Figure 12 for VSA consistency scoring.Source: §7.2 · p. 21

8.1 Analysis of the Vanishing Gradient Problem

• In heterogeneous distillation, student domain deviation can make DMD teacher predictions nearly identical to fake outputs, driving update gradients close to zero; Target Simulation provides distinct representations and informative gradients.The problem arises when students acquire novel capabilities and Backward Simulation produces samples too far outside the teacher’s domain.Source: §8.1 · p. 21, Figure 13 · p. 24

8.2 Ablation Study on Timestep-Constrained Target Simulation

• Target Simulation avoids absolute gradient vanishing without a timestep constraint, but that setting yields smaller gradient updates than timestep-constrained Target Simulation.The timestep constraint amplifies the discrepancy between real and fake predictions, facilitating more effective student updates.Source: §8.2 · p. 24, Figure 14 · p. 24

9 Implementation Details

• Qwen-VL-Max-Latest generates refined student prompts from two randomly sampled training pairs and the corresponding teacher prompt, automating prompt engineering and mitigating conflicts in all-in-one effect scenarios.The prompt template and illustrative examples are shown in Figure 15 and Table 6.Source: §9 · p. 24

10 User Study and Additional Qualitative Results

• Ours received the highest preferences for visual quality (49.9%), consistency (66.2%), and style alignment (53.9%).The study compared Base, Base+Lightning, 50 in 1 (FM) + Lightning, and Ours; the reported preference distributions are shown in Figure 16.Source: §10 · p. 27
• Ten professional evaluators, blind to the methods, chose the best of four candidates across 50 randomly sampled test sets.They judged image quality, consistency, and style alignment using each set’s original reference image.Source: §10 · p. 27
• Lightning variants received minimal votes across all metrics, suggesting they compromise subjective visual quality and stylistic fidelity despite potential inference-speed advantages.Base was the second-most preferred for visual quality (35.0%) and style alignment (41.4%).Source: §10 · p. 27