Source-linked AI summary
Guided Flows for Generative Modeling and Decision Making
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, Ricky T. Q. Chen
TL;DR
Conditional guidance has mainly been used with diffusion models, leaving its value for flow-based conditional generation unclear. This paper introduces Guided Flows by adapting classifier-free guidance to Flow Matching. It reports state-of-the-art generative results and offline-RL planning with a 10x speedup over diffusion models and comparable performance.
Problem
Conditional guidance has primarily been used in diffusion models, and its integration into flow-based models and effect on their performance remain unclear.
Source: §1 · p. 1Method
Guided Flows adapt classifier-free guidance to Flow Matching by modifying the velocity vector fields of flow models.
Source: §1 · p. 1Results
Takeaways & Limitations
The results highlight the potential of flow matching for model-based planning, especially when computational efficiency is important.
Source: §6 · p. 9The guidance analysis guarantees sampling from the target distribution at time t = 1 only if the guided probability path is close to the marginal probability path, and how well that assumption holds in practice is unclear.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Classifier-free guidance is a key component for enhancing the performance of conditional generative models across diverse tasks. While it has previously demonstrated remarkable improvements for the sample quality, it has only been exclusively employed for diffusion models. In this paper, we integrate classifier-free guidance into Flow Matching (FM) models, an alternative simulation-free approach that trains Continuous Normalizing Flows (CNFs) based on regressing vector fields. We explore the usage of \emph{Guided Flows} for a variety of downstream applications. We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text-to-speech synthesis, boasting state-of-the-art performance. Notably, we are the first to apply flow models for plan generation in the offline reinforcement learning setting, showcasing a 10x speedup in computation compared to diffusion models while maintaining comparable performance.
1 Introduction
The introduction presents Guided Flows as an adaptation of classifier-free guidance to Flow Matching, addressing whether guidance can help flow models in conditional generation. It reports improved generative quality and offline-RL results, including faster computation than diffusion models.
1 Introduction
- The introduction situates conditional generative modeling across image generation, text-to-speech, and decision making, with diffusion and flow-based models among the approaches used.The cited examples include Dhariwal and Nichol (2021), Rombach et al. (2022), Wang et al. (2023), Le et al. (2023), Chen et al. (2021), and Janner et al. (2021, 2022).Source: §1 · p. 1
- Table 1.1 catalogs the data points and conditioning variables for three application settings.Source: Table 1.1 · p. 1
- Guided Flows adapt classifier-free guidance to Flow Matching by modifying velocity vector fields, including optimal-transport and cosine-scheduling flows.The paper studies whether conditional guidance can be applied to flow models and improve their performance.Source: §1 · p. 1
- Guided Flows improve image and zero-shot speech sampling quality to state-of-the-art levels, while offline-RL returns exceed unguided flows and computation is 10x faster than diffusion models.For offline RL, performance is also described as state-of-the-art.Source: §1 · pp. 1–2
- Flow-based models are applied to return-conditioned plan generation in offline reinforcement learning for the first time.The introduction describes the plans as conditioned on the current state and a target return.Source: §1 · pp. 1–2
2 Related Work
Related work centers conditional guidance on diffusion models, leaving its use in other generative models an open question. The section also situates Guided Flows among concurrent flow-guidance methods and generative approaches to offline reinforcement learning.
Diffusion and Flow Generative Models
- Conditional guidance methods have primarily been developed for diffusion models, and whether they can be integrated into other generative models remains open.The cited guidance approaches include Dhariwal and Nichol (2021) and Ho and Salimans (2022).Source: §2 · p. 2
- Dao et al. (2023) offer a similar approach motivated heuristically for conditional optimal transport, while Hu et al. (2023) add an offset to the learned vector field without theoretical guarantees on the sample distribution.The paper distinguishes its broader range of domains, including offline RL, from those concurrent works.Source: §2 · p. 2
Conditional Generative Modeling in Offline Reinforcement Learning
- Offline RL learns policies from static datasets of previously collected experiences; conditional generative methods also frame it as sequence modeling, with imagined trajectories used as plans given a current state and target output [Janner et al., 2021, 2022; Ajay et al., 2022].These methods maximize the likelihood of offline-dataset sequences rather than optimizing expected return directly.Source: §2 · p. 2
- Generative models in offline RL also serve as autoregressive action policies or model multimodal action distributions; Akimov et al. (2022) use flows as conservative action encoders and decoders.Source: §2 · p. 2
3 Guided Flow Matching
The section adapts classifier-free guidance to Flow Matching by guiding velocity fields, then trains with a conditional flow-matching loss and samples by solving an ODE. In a toy mixture, larger guidance weights move samples away from the unconditional marginal.
Guided Flow Matching
- Flow Matching trains a vector field to map noise samples to data along marginal probability paths that interpolate between noise and the conditional data distribution.The target field generates the marginal path, and the conditional flow-matching loss has that target as its global minimizer.Source: §3 · pp. 2–3
- Guided Flows adapt classifier-free guidance to conditional velocity fields, with a geometric-weighted interpretation of the guided probability path.The analysis justifies this form through a relation between velocity fields and score functions, while noting its guarantee depends on the guided path being close to the relevant marginal path.Source: §3 · p. 3
- Training replaces the diffusion training loss used in classifier-free guidance with the conditional flow-matching loss.The training procedure samples a null condition with Bernoulli probability, interpolates noise and data using the scheduler, and regresses the target velocity.Source: §3 · p. 4
- Sampling approximates the ODE solution from the initial noise sample at time 0 through time 1 using the guided velocity field.The procedure takes a condition, guidance parameter, and number of ODE steps as inputs.Source: §3 · pp. 2, 4
- In the toy Gaussian-mixture example, guidance weight 1.0 gives unconditional samples, while increasing the weight moves samples away from the unconditional marginal.The example conditions on the cluster identity, with all three clusters shown on the same plots.Source: §3 · p. 4, Figure 3.1 · p. 4
4 Conditional Generative Modeling
The section evaluates Guided Flows on conditional image generation and zero-shot text-to-speech, testing sample quality across image and speech modalities.
4.1 Conditional Image Generation
- Guidance reduced Flow Matching FID from 2.54 to 1.68 with a midpoint solver at 200 NFE; optimal guidance weight varied with compute cost, and FM-OT was slightly more efficient than other considered models.Source: §4.1 · p. 4
4.2 Zero-shot Text-to-Speech Synthesis
- The zero-shot TTS experiments used a model trained on 60K hours of English audiobooks and assessed continuation and text-only speech generation using WER.Continuation supplied the first three seconds of an utterance; text-only generation supplied only text, with WER measured using ASR models.Source: §4.2 · pp. 4–5
- Guidance marginally improved continuation TTS and yielded a much larger gain for text-only TTS, which the authors attribute likely to its more diverse distribution.Table 4.1 reports WER and indicates improvement with guidance weights greater than 1.0.Source: §4.2 · p. 5, Table 4.1 · p. 5
5 Planning for Offline RL
The section frames offline RL planning as conditional generation of future state sequences from the current state and target return, with actions inferred from successive states. Experiments report comparable performance to diffusion models with substantially lower sampling cost, and better returns for guided than unguided flows.
5.2 Our Setup
- The planning setup predicts future state sequences conditioned on the current state and target return, then uses an inverse dynamics model to infer the action from consecutive states.The model is trained with return-to-go conditioning and the current state is retained as general conditioning.Source: §5.2 · p. 6
5.3 Experiments
- Experiments use hopper, walker, and halfcheetah D4RL datasets, with five training seeds and 20 evaluation episodes per instance.Decision Diffuser is the diffusion baseline and models state sequences under the same paradigm.Source: §5.3 · p. 6
5.3.1 Plan Generation (Q1)
- On hopper-medium, a guided flow conditioned on the first state and RTG 0.8 produces a state sequence almost identical to the ground truth, although 0.8 exceeds the training dataset’s maximum RTG of 0.61.The authors describe this as suggesting possible robustness to out-of-distribution RTG values.Source: §5.3.1 · p. 7, Figure 5.3 · p. 8
5.3.2 Benchmark (Q2)
- Across the evaluated tasks, flow matching and diffusion performance are comparable, with flow matching marginally better.The comparison uses 200 sampling steps and reports the best result across saved checkpoints.Source: §5.3.2 · p. 8, Table 5.1 · p. 8
- 10 ODE steps let flows match the return from 200 ODE steps, while DDPM and DDIM require 100 diffusion steps; flows use 10% of diffusion’s computing time for the same downstream performance.At matched internal-step counts, CPU time is roughly the same for flows and diffusion models and scales linearly with step count.Source: §5.3.2 · p. 8
5.3.3 Influence of Guidance (Q3)
- Guided flows outperform unguided flows on all three hopper and halfcheetah datasets tested with varying guidance weights.Guidance weight 1.0 is the unguided-flow condition.Source: §5.3.3 · p. 8
6 Conclusion
The conclusion reports that conditional guidance improves flow-based models across diverse domains, including image and speech generation and offline RL planning. In offline RL, guided flows achieve a 10x speedup while performing on par with diffusion models.
6 Conclusion
- The experiments validate conditional generative capabilities of flow-based models trained with simulation-free algorithms across diverse domains.The authors report that conditional guidance can lead to better results for flow-based models.Source: §6 · pp. 8–9
- Guided flows achieve a 10x speedup in offline RL while maintaining performance on par with diffusion models.The conclusion identifies computationally demanding applications such as online planning as a potential setting for flow matching.Source: §6 · p. 9
- Guided flows achieve state-of-the-art performance in image synthesis and speech generation.The conclusion describes their efficacy and efficiency in model-based planning as well.Source: §6 · p. 9
A.1 Proof of Lemma 1
The appendix relates the Gaussian-path generating velocity to the score function, shows that classifier-free guidance yields the paper’s guided velocity field, and describes an offline-RL sampling adjustment for known states.
A.1 Proof of Lemma 1
- The proof relates a Gaussian probability path’s generating velocity field to its score function under the Gaussian-path assumption.The supplied proof passages state the relationship but do not include the complete derivation.Source: §A.1 · p. 13
B Probability Flow ODE for Scheduler (αt, σt)
- For paths defined by a scheduler, classifier-free guidance’s probability-flow ODE has a velocity field that coincides with the paper’s guided velocity field in equation (9).The guided distribution is given as ˜q(x|y) ∝ q(x)^(1−ω)q(x|y)^ω.Source: B Probability Flow ODE for Scheduler… · pp. 13–14
C Flow Matching Sampling with Guidance for Offline RL
- Offline-RL sequence sampling conditions on the current state by zeroing its corresponding vector field; goal-conditioned sampling can be done similarly.The algorithm initializes from a Gaussian and applies ODE steps using the guided field with the known-state dimension zeroed.Source: C Flow Matching Sampling with Guidance… · p. 14
D Image Generation Experiment Details
The ImageNet-64 experiments train DDPM and two Flow Matching variants, with implementation and guidance settings documented alongside a hyperparameter table.
D Image Generation Experiment Details
- The ImageNet-64 comparison trains DDPM with noise prediction and trains FM-CS and FM-OT using the loss in equation (11).The FM-CS and FM-OT training setup is described separately from DDPM’s noise-prediction objective.Source: D Image Generation Experiment Details · p. 14
- For Flow Matching, time is sampled uniformly, and FM-CS uses a cosine scheduler while FM-OT’s scheduler is specified separately.The passage also defines the noise, data samples, and interpolated state used in the training setup.Source: D Image Generation Experiment Details · p. 14
- The implementation samples time uniformly and uses a VP scheduler with c1 = 20 and c2 = 0.1.The passage identifies these as implementation settings.Source: D Image Generation Experiment Details · pp. 14–15
- All three models share a U-Net architecture; guidance settings and model hyperparameters are reported with a sweep range of 1.0–2.0 and grid size 0.05.The passage separately reports DDPM, DDIM, and FM-CS guidance weight 0.2, and FM-OT guidance weight 0.15; Table D.1 is captioned as listing training hyperparameters.Source: D Image Generation Experiment Details · p. 15, Table D.1 · p. 15
E Offline RL Experiment Details
The offline RL experiments specify sequence-model settings, distinct architectures for guided flows and diffusion, and a separately trained inverse dynamics model.
E Offline RL Experiment Details
- Experiments use a PyTorch implementation heavily influenced by the Decision Diffuser codebase.Sequence models train for 2 × 10^6 iterations, with EMA checkpoints saved every 200k iterations.Source: E Offline RL Experiment Details · p. 15
- State sequences use horizon 64, null-conditioning probability 0.25, batch size 64, and task-specific RTG reward scales.The scales are 400 for hopper, 550 for walker, and 1200 for halfcheetah; the final model uses an EMA updated every 10 iterations with decay 0.995.Source: E Offline RL Experiment Details · p. 15
Guided Flows
- The guided-flow velocity field uses a temporal U-net with six residual blocks, each containing two temporal convolutions, group normalization, and Mish activation.Time and RTG are embedded with separate MLPs; learning rates vary by dataset from 10^-4 to 3 × 10^-4.Source: Guided Flows · p. 15
Guided Diffusion Models
- Guided diffusion models use a cosine noise schedule and a temporal U-net matching the guided-flow architecture.The diffusion model uses Adam with learning rate 2 × 10^-4 across datasets.Source: Guided Diffusion Models · p. 15
Inverse Dynamics Model
- The inverse dynamics model is a two-hidden-layer MLP with 1024 units per layer and 10% dropout, trained for 100k iterations.Ten percent of offline trajectories form the validation set, and the checkpoint with best validation performance is used.Source: Inverse Dynamics Model · pp. 15–16
F Additional Experiments
The additional experiments show generated state sequences under a specified conditioning setup and compare CPU generation time for flows and diffusion.
F.1 Flow Generated State Sequences
- Flow-generated hopper-medium state sequences are shown alongside true sequences, conditioned on the first state and a large out-of-distribution RTG.The guidance weight is 3.0.Source: Figure F.1 · p. 16
F.2 Computational Speed Comparison
- For 704-dimensional batches, flow and diffusion CPU generation times are roughly the same across the tested numbers of internal steps.The comparison uses sequence length 64 and state dimension 11.Source: Figure F.2 · p. 17