PaperlayerView exact revision on arXiv

Source-linked AI summary

A Long-Short Flow-Map Perspective for Drifting Models

Zhiqi Li, Bo Zhu

arXiv:2602.20463v1cs.LG

TL;DR

The paper addresses the missing connection between the one-step Drifting Model and trajectory-consistent flow maps, along with the challenge of obtaining dataset-level supervision for long-range maps. It introduces a semigroup-consistent long-short factorization with a closed-form terminal velocity, derives the drifting formulation in the zero-step limit, and develops likelihood learning and feature-space analyses. The framework is supported by theoretical analysis and empirical evaluations, while feature-space optimization and conditional generation remain open issues.

  • Problem

    The Drifting Model’s relation to transport and flow-map structures, and the origin of dataset-level supervision without multi-step distillation or optimization, had not been explicitly characterized.

  • Method

    The paper factorizes transport into a long flow map and a short terminal flow map, uses closed-form terminal velocity supervision, and aligns likelihood learning with density evolution under transport.

  • Results

    The zero-terminal-interval limit exactly recovers the drifting formulation, with attraction from the first-order expansion and a conservative impulse correction from the second-order expansion.

  • Takeaways & Limitations

    The long-short flow-map perspective provides a structured interpretation of the Drifting Model’s attraction, impulse, and feature representations while supporting benchmark validation.

  • Takeaways & Limitations

    Feature-space optimization remains unresolved because its kernel weights are still defined in the original high-dimensional space, and faithful lower-feature representations are not yet understood.

Abstract

from arXiv · show

This paper provides a reinterpretation of the Drifting Model~\cite{deng2026generative} through a semigroup-consistent long-short flow-map factorization. We show that a global transport process can be decomposed into a long-horizon flow map followed by a short-time terminal flow map admitting a closed-form optimal velocity representation, and that taking the terminal interval length to zero recovers exactly the drifting field together with a conservative impulse term required for flow-map consistency. Based on this perspective, we propose a new likelihood learning formulation that aligns the long-short flow-map decomposition with density evolution under transport. We validate the framework through both theoretical analysis and empirical evaluations on benchmark tests, and further provide a theoretical interpretation of the feature-space optimization while highlighting several open problems for future study.

1 Introduction

The paper connects the Drifting Model to transport and flow-map theory through a semigroup-consistent long-short factorization. This perspective supplies dataset-level supervision, recovers drifting dynamics in the infinitesimal limit, and motivates likelihood learning and feature-space analysis.

  • Motivation: The Drifting Model is a deterministic one-step generator whose connection to established transport and flow-map frameworks had not been explicitly characterized.It learns a mapping f that pushes a noise distribution to the data distribution for one-step sampling.
  • Long-Short Flow Map: The global transport map is factorized into a long flow map followed by a short terminal flow map using the semigroup property.The terminal component is evaluated with a closed-form optimal velocity near the final time, while the long component preserves trajectory consistency.
  • Further Formulation: The framework introduces a likelihood-learning formulation aligned with density evolution under transport and proposes a squared-kernel modification for theoretical consistency with the noise form.The paper also uses this perspective to interpret attraction, impulse, and multiple feature representations.
  • Evaluation and Open Problems: The paper validates the framework on illustrative and benchmark tests while identifying open questions in feature-space optimization.The feature-space discussion is presented as part of the framework’s theoretical interpretation.

2 Background

Flow Matching provides data-supervised velocity learning but can require many numerical sampling steps, while flow-map methods enable one-step generation yet lack direct dataset supervision. The background therefore motivates methods that combine tractable terminal velocity estimation with trajectory-consistent long-horizon transport.

  • Flow Matching: Flow Matching learns a time-dependent velocity field transporting a Gaussian base distribution to the data distribution along intermediate distributions.Its learned dynamics generate samples by numerically integrating an ordinary differential equation.
  • Flow Matching: The standard Flow Matching objective is difficult to optimize directly because the marginal velocity and distribution are hard to calculate from the dataset.Conditional velocities and flows provide data-based surrogate supervision through marginalization.
  • Closed-Form Computation: A closed-form optimal velocity is computationally impractical at earlier timesteps because diffuse kernels require aggregating contributions from much of the data distribution.Near t = 1, the kernel concentrates and computation becomes effectively local.
  • Flow Maps: Flow-map methods reduce sampling to one or a few map applications, but learning them requires trajectory consistency through the semigroup property.The marginal path is represented as a pushforward of the base distribution, and composition preserves intermediate trajectories.
  • Flow-Map Supervision: The central flow-map limitation is the lack of an easily computable conditional flow map from data pairs, preventing a simple dataset-supervised objective.Progressive-extension methods propagate indirect short-horizon signals, while continuous-based methods inject instantaneous velocity supervision into long-horizon maps.

3 Long-Short Flow-Map Perspective

The paper derives the Drifting Model from a semigroup-consistent decomposition of global transport into a long map and a short terminal map. Closed-form terminal velocities provide dataset-tied supervision, while the terminal approximation order explains the model’s drift-field variants and associated losses.

  • Long-Short Decomposition: The full transport map is factorized as a long map followed by a short terminal map, ψ0→1 = ψ1−∆t→1 ◦ ψ0→1−∆t.The short interval is near the endpoint, while the long map performs the main transport from noise toward data.
  • Long-Short Decomposition: Closed-form Flow Matching velocities on the short terminal interval provide dataset-tied supervision for learning the remaining long map.The formulation uses a forward Euler or trapezoidal approximation before applying the closed-form endpoint solution.
  • Endpoint Velocity: The endpoint velocity requires the closed-form left-limit solution u1−(x) ≜ limt→1−u∗t(x), rather than simply setting u1−(x) = x.The latter choice does not cancel the ∆t terms or represent the discrepancy between intermediate and target distributions.
  • Approximation Order: The trapezoidal objective compares generated and target distributions, so its residual becomes exactly zero when the learned flow map is optimal.This differs from the Euler objective’s point-to-distribution regression interpretation.
  • Approximation Order: The first-order Euler approximation recovers the Drifting Model’s attraction-only field V+, whereas the second-order trapezoidal rule yields the balanced correction V+ − V−.The symmetric correction is reported to outperform V+ and imbalanced alternatives such as 3V+ − V−.
  • Kernel Selection: The kernel is coupled to the initial noise distribution: Gaussian noise gives a squared-exponential kernel, while Laplace noise recovers the Drifting Model’s Laplacian kernel.The paper leaves systematic investigation of noise-dependent kernel selection open.

4 Application: Likelihood Learning

The paper derives a likelihood-learning objective for the Drifting Model from Eulerian density evolution and long-short flow-map consistency. It addresses intractable Jacobian evaluation and unknown intermediate distributions with divergence formulas, normalization, and importance sampling.

  • Likelihood formulation: The framework learns the Eulerian log-likelihood change G_t→r(x) and combines it with the flow map for likelihood estimation.G_t→r satisfies an evolution equation and a consistency relation across intermediate times.
  • Long-short derivation: The likelihood change is decomposed at 1−Δt, with the terminal interval approximated by a trapezoidal rule using endpoint velocities and divergences.The resulting approximation requires divergence terms for the closed-form velocity field.
  • Training objective: The terminal limit Δt→0 yields a loss for learning the likelihood change, while unknown p_1−Δt samples are replaced by samples from the model-induced distribution.The loss uses stop-gradient structure and expectations estimated from generated or model-induced samples.
  • Normalization: A normalization loss is added to enforce the likelihood normalization constraint, producing the final objective L(θ)+λL_norm(θ).The expectations are approximated with mini-batch Monte Carlo estimates.
  • Sampling: Importance sampling makes otherwise inaccessible expectations tractable by using a reference distribution with an available density, ideally matching the model distribution’s support.In practice, the reference is constructed as a multimodal Gaussian mixture around mini-batch samples.

5 Open Problems in Feature-Space Optimization

The paper examines feature-space optimization as a response to high-dimensional kernel weighting, but identifies unresolved mismatches caused by original-space kernels and nonlinear feature maps. It derives a feature-space velocity relation and explains why many representations can help, while leaving objective design open.

  • Motivation: High-dimensional Drifting Model training suffers from weak and unstable pairwise-distance-based kernel weights, motivating computation in feature spaces with more meaningful neighborhood structure.Lower-dimensional or anisotropic representations may better reflect semantic similarity.
  • Open problems: The proposed feature-space objective still evaluates kernels in the original space, so it does not realize the expected reduction in effective dimensionality.The neighborhood structure used for weighting remains high-dimensional despite the feature-space formulation.
  • Feature-space derivation: Feature-space Flow Matching uses Z_t=tZ_1+(1−t)Z_0 and yields an optimal velocity whose kernel is evaluated in feature space.The construction applies Flow Matching between Gaussian reference and data distributions after feature mapping.
  • Velocity relation: The relation between feature-space and data-space velocities is approximate because nonlinear mappings introduce Taylor-expansion discrepancies.The approximation becomes exact only under special conditions, such as a linear feature map with an induced pushed-forward path.
  • Limitations: The feature-space derivation typically assumes linearity of ϕ_j, whereas practical mappings such as MAE encoders are nonlinear; using many features only mitigates this approximation.The paper leaves better feature-map design or objective modification as an open question.
  • Multiple representations: Using only a small number of information-preserving features produces weaker optimization performance, supporting the use of many feature representations.The paper explains this as forming a more faithful surrogate direction for optimizing the flow map.

6 Experiment

The experiments validate likelihood learning on 2D distributions and evaluate image generation with feature-space optimization on CelebA-HQ. Results also examine feature count, kernel choice, and batch-size practicality.

  • Image Generation: On CelebA-HQ, feature-space optimization reaches an FID of 14.71 after 100K steps, compared with MeanFlow’s FID of 12.4 after 400K steps.The comparison is reported for unconditional latent generation using a pretrained MAE feature extractor.
  • 2D Examples for Likelihood Learning: Likelihood learning recovers the data distribution well on Spiral, Checkerboard, and Two Moons in both Eulerian and Lagrangian views.The experiments use 50K samples per dataset and report results with an MLP trained under dataset-specific schedules.
  • Image Generation: Feature-space optimization with thousands of features produces better samples than using only four features, while removing feature-space optimization causes training to fail completely.The four-feature setting still learns but introduces visible distortions despite retaining the full latent information collectively.
  • Image Generation: A Gaussian kernel remains effective on CelebA-HQ and achieves results comparable to, and in some settings better than, the original Laplacian kernel.The Gaussian form follows from the initial Gaussian noise in the flow-map derivation.
  • Image Generation: With appropriate learning-rate tuning, the method produces strong CelebA-HQ samples using batch size 64.This contrasts with prior reported ImageNet experiments using batch sizes of at least 4096.

7 Related Work

The related work situates the paper among flow-map, closed-form velocity, and distribution moment-matching methods. These lines of work address efficient generation, transport consistency, analytic velocities, and kernel-based distribution alignment.

  • Flow Map Methods: Flow-map methods learn long-range transport mappings, often using consistency constraints or progressive distillation from instantaneous velocity fields.The semigroup property provides the trajectory-consistency perspective underlying these approaches.
  • Closed-Form: Closed-form velocity analyses provide analytic solutions but are mainly used for theory because kernel-weighted evaluation over the dataset is expensive.The cited applications include stability, memorization, and generalization analyses.
  • Distribution Moment Matching: Distribution moment-matching methods align model and data distributions through kernel discrepancies, including Gaussian, Laplace, and Coulomb-style formulations.MMD-based methods and related GAN variants provide the main comparison points.

8 Conclusion

The paper presents a long-short flow-map interpretation of the Drifting Model, derives likelihood-based learning, and analyzes feature-space optimization. It identifies unresolved questions around conditioning, feature design, broader applications, and kernel-noise connections.

  • Conclusion: The long-short flow-map perspective explains key design choices in the Drifting Model.The conclusion presents this interpretation as the paper’s central contribution.
  • Conclusion: The paper derives a likelihood-based learning formulation and discusses feature-space optimization in depth.These developments build on the flow-map interpretation.
  • Conclusion: Open problems include principled classifier-free guidance, better feature encoders and objectives, extensions beyond image generation, and links between kernel design and noise optimization.The conclusion frames these as questions for future study rather than settled results.

A.1 Proof of Theorem 2.1

The proof establishes a closed-form optimal solution for the Flow Matching regression objective and then presents a non-existence result for conditional flow maps satisfying both marginal consistency and conditional-velocity consistency.

  • Proof of Theorem 2.1: The Flow Matching objective has an optimal solution because it is an l2 regression whose minimizer is a conditional expectation.The optimal velocity is expressed through conditional averaging over the target variable.
  • Proof of Theorem 2.1: The optimal velocity admits a kernel-weighted closed-form expression involving the target distribution.The derivation rewrites the conditional expectation using the kernel representation.
  • Proof of Theorem 2.1: No conditional flow map can simultaneously follow the conditional velocity and equal the conditional expectation of marginal flow maps.The theorem concludes that a self-consistent conditional flow map does not exist.
  • Proof of Theorem 2.1: The contradiction arises because the resulting marginal transport would collapse the target distribution to a Dirac mass rather than reproduce pdata.The proof identifies this degenerate pushforward as incompatible with the actual data distribution.

A.3 Proof of Theorem 3.1

The section derives closed-form endpoint velocity and divergence expressions for flow matching, then relates data-space and feature-space optimal velocities through conditional distributions and approximation results.

  • Endpoint velocity: Theorem 3.1 states that, under mild smoothness assumptions, the endpoint velocity has a closed-form representation as t approaches 1.The proof uses the conditional path X_t = tX_1 + (1 − t)X_0 and conditional expectations at the endpoint.
  • Endpoint velocity: The proof shows E[X_t|X_1 = x] = tx by using independence of X_0 and X_1 and the zero mean of the reference distribution.This conditional-expectation identity supplies the endpoint behavior used in the velocity derivation.
  • Velocity divergence: Theorem 4.1 gives closed-form expressions for the divergences of the optimal and endpoint velocity fields with the mollified kernel.The divergence formulas are expressed through expectations involving kernel-weighted data samples and the intermediate distribution.
  • Feature-space relation: The feature-data relation defines separate optimal velocities for data and feature spaces and gives approximate relations when features are induced by a deterministic map.The approximation follows from a first-order Taylor expansion; an exact relation is available in the special case where the feature path equals the mapped data path.
  • Feature-space relation: The feature-space velocity can be computed with a first-order approximation, with conditional distributions accounting for the inverse-image relation between feature and data spaces.The derivation introduces conditional densities over feature and data variables to connect the two velocity formulations.

B.1 Training Details

The training setup uses DiT-B/2 image generation with specified batch-size, optimization, precision, VAE, and MAE-feature configurations, alongside 2D likelihood experiments and image-generation evaluations.

  • Image generation: Image generation uses DiT-B/2 with batch sizes 64, 256, and 1024, 100K training steps, EMA, Adam, learning-rate sweeps, and a 5K-step warmup.The standard settings use EMA decay 0.9999, Adam parameters (0.9, 0.95), and weight decay 0.
  • Image generation: The Figure 6 small-batch experiment trains for 400K steps at batch size 64 using a staged learning-rate schedule.The schedule is 5×10−5 for steps 0–200K, 2.5×10−5 for 200K–300K, and 1.25×10−5 for 300K–400K.
  • Feature extraction: The image-generation pipeline uses a pretrained Stable Diffusion VAE and a four-resolution MAE encoder whose extracted features total several thousand channels.The MAE is trained separately, and features are taken at each resolution rather than every two blocks.
  • 2D likelihood experiments: The 2D experiments report generated samples and learned likelihoods for Spiral, Checkerboard, and Two Moons under Eulerian and Lagrangian views.The experiments use Figure 4 to compare ground-truth distributions, generated samples, and learned likelihoods.

C Discussion on Classifier-Free Guidance

The discussion examines classifier-free guidance choices and empirical design comparisons involving kernels, batch sizes, feature spaces, and training budgets.

  • Guidance choices: Classifier-free guidance can modify the conditional velocity used to estimate the terminal short flow-map segment.The proposed interpolation combines conditional and null-condition velocities using a guidance scale w.
  • Empirical comparisons: The Gaussian kernel performs comparably to the Laplacian kernel and can be better in some regimes, while using only four features or the original space fails to produce high-quality results.The comparison uses FID trajectories and an ablation with batch size 64 and learning rate 5 × 10−5.
  • Empirical comparisons: Strong generative performance is achieved with batch size 64 under a tuned learning rate, without extremely large batches.Figure 6 reports FID curves for the Long-Short Flow-Map method and Drifting Model under this setting.
  • Guidance choices: Classifier-free guidance can alternatively modify the target distribution or be applied to the impulse term rather than only the attraction component.The discussion identifies target-distribution guidance and impulse-term guidance as additional directions.
  • Empirical comparisons: Table 1 compares batch size, training steps, and FID for the proposed method and prior baselines using DiT-B/2 for one-step latent-space generation.The table organizes the comparison by training budget rather than by classifier-free guidance configuration.
Loading 2602.20463v1…