Source-linked AI summary
Perceptual Flow Matching for Few-Step Generative Modeling
Chuyang Zhao, Yifei Song, Hongfa Wang, Jianlong Yuan, Yuan Zhang, Siming Fu, Zhineng Chen, Huilin Deng, Haoyang Huang, Nan Duan
TL;DR
Flow-matching models often need dozens of sampling steps, limiting practical deployment. PFM instead supervises flow matching in pretrained perceptual feature space, enabling high-quality generation in 4–8 steps across image, editing, and video tasks.
Problem
Flow-matching models typically require 35–50 sampling steps for high-quality outputs, creating substantial inference latency and computational overhead.
Method
PFM replaces VAE-latent velocity regression with perceptual regression on decoded predictions using pretrained perceptual features.
Results
PFM enables high-quality sampling in 4–8 steps across image generation, image editing, and video generation, outperforming few-step baselines and sometimes matching multi-step models.
Takeaways & Limitations
The supervision space shapes prediction geometry, with perceptual supervision favoring sharper, more visually coherent on-manifold predictions for coarse integration.
Takeaways & Limitations
PFM performance degrades below two sampling steps, and the optimal perceptual space remains unknown.
Abstract
from arXiv · showhide
We propose Perceptual Flow Matching (PFM), a simple yet effective framework for few-step generation in flow-matching models. Rather than performing velocity regression in the conventional VAE latent space, PFM supervises flow matching in a perceptual feature space using pretrained perceptual models. This simple change substantially improves the few-step generation capability of flow-matching models, reducing the number of sampling steps from 35-50 to 4-8 while preserving generation quality. Unlike existing acceleration and distillation approaches, PFM requires neither teacher models nor auxiliary score networks and can be integrated into standard flow-matching training pipelines with minimal modifications. Extensive experiments on image generation, video generation, and image editing tasks demonstrate that PFM consistently produces high-quality results while producing fewer artifacts than existing distillation-based methods. We further show that perceptual supervision shifts the regression minimizer from mean-seeking to mode-seeking, biasing predictions toward on-manifold modes that remain accurate under coarse few-step integration. Our results reveal that standard flow-matching training can naturally yield high-quality few-step generators when supervised in an appropriate representation space. We hope this insight inspires future research into representation-aware objectives for efficient generative modeling.
1 Introduction
Flow matching delivers strong generation quality but remains costly at inference, motivating few-step methods that often add auxiliary models or substantially alter training. PFM instead applies perceptual regression in decoded-sample feature space, yielding mode-seeking predictions and strong few-step performance across image, editing, and video tasks.
- Motivation: Flow matching achieves strong results across image generation, image editing, and video generation, but its high inference cost limits practical deployment.The introduction identifies inference computation as the central deployment challenge.
- Motivation: Prior few-step approaches use distillation, consistency training, or continuous-time flow-field modeling, often requiring auxiliary models or major training modifications.These requirements increase computational cost and reduce compatibility with the original training framework.
- Perceptual Flow Matching: PFM replaces VAE-latent velocity regression with perceptual regression on decoded samples using features from pretrained perceptual models.Classifier-free guidance can also be incorporated naturally during training.
- Perceptual Flow Matching: Perceptual supervision shifts flow-matching predictions from mean-seeking to mode-seeking, favoring on-manifold modes that remain easier to integrate accurately in few steps.Euclidean regression can place predictions between multimodal data modes, producing off-manifold and blurry estimates at high noise levels.
- Experimental Validation: 33.93 FID, 31.70 CLIP score, and 11.42 HPSv3 score are achieved by PFM on text-to-image COCO 2014 val among 8-step generation methods.Experiments also report substantial improvements over few-step baselines on image editing and video generation, with comparable results in some comparisons.
2 Related Work
Related work on efficient generative modeling includes several paradigms for reducing diffusion and flow-model sampling costs, while perceptual losses have primarily targeted perceptual quality and reconstruction. This work instead studies perceptual supervision as a mechanism for few-step generation.
- Sampling Acceleration: Sampling-cost reduction for diffusion and flow-based generative models has become a central research topic, with existing methods organized into four broad paradigms.The passage identifies Progressive Distillation and Consistency Models/Distillation among these approaches.
- Sampling Acceleration: Progressive Distillation repeatedly trains a student to emulate multiple sampling steps of a pretrained teacher, progressively reducing inference steps.This approach accelerates generation through repeated teacher-student training.
- Perceptual Supervision: Perceptual losses have been widely used in image synthesis, restoration, and representation learning to improve perceptual quality.Prior works primarily apply perceptual supervision to reconstruction fidelity or visual realism.
- Perceptual Supervision: In contrast, this work investigates perceptual supervision as a mechanism for enabling few-step generation.The authors distinguish their objective from prior uses focused primarily on reconstruction fidelity or visual realism.
3 Method
Perceptual Flow Matching (PFM) replaces conventional velocity supervision in VAE latent space with perceptual supervision on decoded clean-sample predictions. This changes the regression geometry toward feature-space barycenters that favor on-manifold modes, while optionally baking classifier-free guidance into training.
- Perceptual Flow Matching: PFM decodes predicted and ground-truth clean samples into pixel space, then minimizes their distance in a pretrained perceptual feature space.The model recovers x̂0 from the predicted velocity before applying perceptual supervision.
- Inference: PFM models can use standard consistency sampling with an adjustable number of steps without retraining, trading generation quality against speed.Each step predicts x̂0 from xt and renoises it for the next timestep.
- Classifier-Free Guidance Baking: PFM supports prediction-side and target-side classifier-free guidance baking, requiring only an additional unconditional forward pass during training.Prediction-side baking uses stop-gradient on the unconditional branch, while target-side baking interpolates toward a CFG-enhanced target with α ∈ [0, 1].
- Perceptual Geometry: Perceptual supervision replaces the Euclidean posterior-mean minimizer with a feature-space barycenter that favors coherent data-manifold modes over blurry inter-mode averages.At high noise, Euclidean averaging can produce off-manifold predictions that are difficult to correct with few sampling updates.
- Perceptual Geometry: Rϕ > 1 indicates that a perceptual representation expands midpoint-to-manifold distance, penalizing blurry averages and pushing the barycenter toward a valid mode.By contrast, isometric or locally linear feature spaces yield Rϕ ≈ 1 and treat off-manifold midpoints as cheap interpolations.
4 Experiments
PFM achieves strong few-step generation across text-to-image, image editing, and text-to-video tasks, outperforming relevant baselines while avoiding classifier-free guidance at inference in several settings. Ablations show that perceptual supervision, pretrained representations, and deeper feature layers are central to sharp, high-quality outputs.
- Text-to-image generation: In 8 steps, PFM achieves state-of-the-art CLIP score, HPSv3 score, and FID against LCM and DMD2 on text-to-image generation.All methods use SD3-Medium, the same data, and 1,000 optimization steps.
- Image editing: PFM significantly outperforms the 8-step image-editing baseline and surpasses the 40-step original on CLIP-I, DINO, L1, and L2.Its CLIP-T score is comparable to the 40-step model, dropping by 0.01, without classifier-free guidance at inference.
- Supervision on other spaces: Perceptual supervision produces sharp, high-quality 8-step images, whereas VAE-latent, pixel-space, and Huber-loss alternatives yield noticeably blurrier outputs.The comparison uses SD3-Medium models evaluated on PartiPrompts with 8-step sampling.
- Perceptual models: All pretrained perceptual models produce sharp, coherent outputs, while randomly initialized RandViT produces noticeably blurrier results.Combining perceptual models yields better performance, and results are closely related to measured off-manifold distance.
- Classifier-free guidance and layer depth: PFM trained only with perceptual supervision generates high-quality samples without classifier-free guidance, while shallow DINOv2 layers produce blurrier outputs than deeper layers.The model remains compatible with standard inference-time CFG, and the deepest layers tend to collapse to fixed modes.
- Inference steps: At 4–8 sampling steps, PFM achieves strong visual quality with most fine details preserved, while 1–2 steps remain recognizable but noticeably blurry.Generation quality improves consistently as the number of sampling steps increases.
5 Conclusion
PFM enables sharp, visually coherent few-step generation by changing only flow-matching supervision from VAE-latent velocity regression to decoded predictions with pretrained perceptual features. Its performance degrades below two sampling steps, and the optimal perceptual space remains unknown.
- Contribution: PFM changes only the supervision space, using pretrained perceptual features on decoded clean predictions instead of VAE-latent velocity regression.It preserves the standard flow-matching framework without teacher models, auxiliary score networks, or specialized distillation.
- Few-step performance: PFM encourages sharper and more visually coherent predictions under large sampling steps while retaining the standard flow-matching framework.Figure 9 reports high-fidelity results at 4 and 8 NFE and sharp results even at 1 NFE.
- Limitations: PFM performance degrades in extremely low-step regimes, especially with fewer than two sampling steps.The optimal perceptual space for few-step prediction remains unknown, leaving stronger one-step generation and deeper characterization for future work.