PaperlayerView exact revision on arXiv

Source-linked AI summary

Asymmetric Flow Models

Hansheng Chen, Jan Ackermann, Minseo Kim, Gordon Wetzstein, Leonidas Guibas

arXiv:2605.12964v2cs.CV

TL;DR

High-dimensional pixel-space flow generation is hindered by the burden of predicting noise, while latent models limit direct control over fine detail. AsymFlow restricts noise prediction to a low-rank subspace while retaining full-dimensional data prediction, achieving 1.57 FID on ImageNet 256×256 and state-of-the-art pixel-space text-to-image generation.

  • Problem

    High-dimensional pixel-space flow generation is difficult because predicting full-dimensional noise burdens transformer representations, while latent generation delegates fine detail to a fixed decoder.

  • Method

    AsymFlow predicts full-dimensional data and low-rank noise components, then analytically recovers full-dimensional velocity without changing architecture, training, or sampling.

  • Results

    AsymFlow achieves 1.57 FID on ImageNet 256×256 and state-of-the-art pixel-space text-to-image generation, outperforming prior pixel models and its latent base across reported benchmarks.

  • Takeaways & Limitations

    AsymFlow provides a practical route to scalable pixel generators and latent-to-pixel finetuning while preserving pretrained latent models’ high-level semantics and structure.

  • Takeaways & Limitations

    Latent-to-pixel finetuning assumes a good patch-level linear lift and may perform poorly when pretrained latent spaces do not preserve pixel structure, such as RAE models.

Abstract

from arXiv · show

Flow-based generation in high-dimensional spaces is difficult because velocity prediction requires modeling high-dimensional noise, even when data has strong low-rank structure. We present Asymmetric Flow Modeling (AsymFlow), a rank-asymmetric velocity parameterization that restricts noise prediction to a low-rank subspace while keeping data prediction full-dimensional. From this asymmetric prediction, AsymFlow analytically recovers the full-dimensional velocity without changing the network architecture or training/sampling procedures. On ImageNet 256$\times$256, AsymFlow achieves a leading 1.57 FID, outperforming prior DiT/JiT-like pixel diffusion models by a large margin. AsymFlow also provides the first-ever route for finetuning pretrained latent flow models into pixel-space models: aligning the low-rank pixel subspace to the latent space gives a seamless initialization that preserves the latent model's high-level semantics and structure, so finetuning mainly improves low-level mismatches rather than relearning pixel generation. We show that the pixel AsymFlow model finetuned from FLUX.2 klein 9B establishes a new state of the art for pixel-space text-to-image generation, beating its latent base on HPSv3, DPG-Bench, and GenEval while qualitatively showing substantially improved visual realism.

1 Introduction

AsymFlow addresses the difficulty of high-dimensional velocity prediction by combining full-dimensional data prediction with low-rank noise prediction and analytically recovering the full velocity. It also enables latent-to-pixel finetuning, achieving leading ImageNet results and state-of-the-art pixel-space text-to-image performance.

  • AsymFlow: AsymFlow restricts noise prediction to a low-rank subspace while keeping data prediction full-dimensional, then analytically recovers the full-dimensional velocity.Standard flow-matching training and sampling remain unchanged.
  • Latent-to-pixel finetuning: AsymFlow enables finetuning pretrained latent flow models into pixel models through a principled latent-to-pixel lift without architectural modifications.The lifted pixel model inherits the latent generator’s semantics and structure, so finetuning focuses on low-level mismatches.
  • ImageNet results: 1.57 FID is achieved on ImageNet 256×256 with an additional REPA loss, outperforming prior DiT/JiT-like pixel diffusion models by a large margin.With the JiT-H/16 network alone, AsymFlow reaches 1.76 FID.
  • Text-to-image results: The pixel AsymFlow model finetuned from FLUX.2 klein 9B sets a new state of the art in pixel-space text-to-image generation.It beats its latent base on HPSv3, DPG-Bench, and GenEval while qualitatively exhibiting substantially improved visual realism.

2 Related Work

Recent work addresses the high-dimensional bottleneck either by modifying architectures to improve noisy-input access or by changing prediction parameterizations to avoid high-dimensional noise prediction. Hierarchical designs remain effective but complicate plain transformer scaling, while recent pixel methods increasingly favor clean-data x0 prediction.

  • Overview: Recent approaches target the high-dimensional bottleneck through architectural changes or prediction parameterizations that avoid high-dimensional noise prediction.These are the two main directions identified in the related-work discussion.
  • Hierarchical architectures: Hierarchical U-Nets, U-ViT-like transformers, and DDT-like decoder designs use multi-scale bypasses or decoder pathways to make noise or velocity prediction feasible.Examples include RAE, PixNerd, PixelDiT, DiP, and DeCo among decoder-based designs.
  • Hierarchical architectures: These hierarchical designs are effective but complicate the plain transformer recipe that has scaled successfully in large image and video generators.The contrast motivates approaches that preserve simpler transformer architectures.
  • Prediction parameterizations: Early diffusion models favored hierarchical architectures for practical ϵ-prediction, whereas x0-prediction was often disfavored because of low-noise numerical issues.The discussion frames prediction choice as historically linked to the underlying architecture.
  • Prediction parameterizations: JiT argues that pixel diffusion should predict clean data x0 rather than noise or velocity, and follow-up pixel methods adopt the same x0-prediction backbone.The follow-up methods additionally use perceptual or representation-based components, as indicated in the passage.

3 Preliminaries

This section introduces flow matching, where noisy samples interpolate between data and Gaussian noise and generation follows a reverse-time SDE or ODE. It also contrasts direct velocity prediction with x0-prediction, motivating alternatives for high-dimensional representations.

  • Flow matching: Flow matching constructs xt = α_t x0 + σ_t ε from data x0 and Gaussian noise ε, using α_t = 1 − t and σ_t = t.Here, t ∈ (0, 1] is diffusion time and x0 ∈ R^D.
  • Flow matching: Generation transports Gaussian noise to data by solving a reverse-time stochastic differential equation or ordinary differential equation.The ODE velocity is the posterior mean of the sample velocity.
  • Velocity prediction: A flow model trains a neural network to estimate the posterior-mean velocity using the flow matching loss.The direct parameterization maps (xt, t) to a predicted velocity ˆu = Gθ(xt, t).
  • u-prediction vs. x0-prediction: Direct u-prediction is common in compressed latent flow models but becomes challenging in pixels because u = ε − x0 includes high-dimensional noise alongside structured data.The passage introduces x0-prediction as an alternative parameterization, but its description is truncated here.

4 Asymmetric Flow Modeling

AsymFlow keeps data prediction full-dimensional while restricting noise prediction to a low-rank subspace, making the target more predictable without changing the network architecture. It analytically recovers full-rank velocity for training and sampling, with rank and subspace choices supporting patch-wise modeling and latent-to-pixel initialization.

  • Asymmetric parameterization: AsymFlow restricts only the stochastic noise term to a low-rank subspace while preserving the full-dimensional data term, reducing high-dimensional noise burden without architectural changes.Image data concentrates near a low-dimensional manifold, making the asymmetric target more predictable for neural networks.
  • Velocity recovery: The model predicts asymmetric velocity ˆuA, then converts it to full-rank velocity ˆu for flow-matching loss calculation and denoising sampling.The low-rank component is retained directly, while the orthogonal x0-style component is converted using the x0-to-u relation.
  • Implementation: Patch-wise projection applies a shared projector independently to each image patch, reducing noise-prediction dimension while preserving the full set of image tokens.For patch dimension D, the subspace basis has rank r < D.
  • Implementation: For pretrained latent-model adaptation, Procrustes alignment between latent variables and corresponding pixel patches selects the subspace and enables seamless latent-to-pixel initialization.When training from scratch, the basis can instead be obtained from data-dependent patch PCA.
  • Rank-controlled decomposition: AsymFlow behaves like u-prediction in Im(P) and x0-prediction in Im(I − P), interpolating between full x0-prediction at r = 0 and full u-prediction at r = D.The parameterization family is controlled by the rank r of the low-rank subspace.

5 Finetuning Latent Flow into Pixel AsymFlow

AsymFlow converts pretrained latent flow models into pixel-space models by lifting latent representations into a low-rank pixel subspace with exact trajectory coupling. Finetuning then corrects primarily low-level discrepancies, aided by variance reduction and perceptual correction.

  • Latent-to-pixel initialization: Trajectory coupling preserves the latent ODE up to an analytically determined orthogonal noise component, producing low-rank pixel samples aligned with decoded latent samples.The remaining gap is mainly low-level and is therefore targeted during finetuning.
  • Latent-to-pixel initialization: A patch-wise Procrustes lift reinterprets a d-dimensional latent u-prediction model as an exact rank-d pixel flow through input and output conversions.The projections A^T and A are fused into the network’s learnable input and output layers to initialize the pixel AsymFlow model.
  • Latent-to-pixel initialization: Scale calibration introduces a factor s because orthonormal Procrustes alignment matches directions but not the scale of real pixels.The correction is folded into model input, output, and internal timestep calibration.
  • Finetuning objective: The paired low-rank target enables a control-variate flow-matching objective that reduces variance when the latent-to-pixel approximation gap is small.A frozen initialized low-rank model predicts the control variate, while an adaptive patch-wise weight minimizes the loss gradient norm; empirically, LVR improves fine-grained details.
  • Perceptual correction: Because the variance-reduction approximation can introduce bounded low-rank-subspace error and excessive noise, training fades toward a λ-gated LPIPS perceptual loss over diffusion time.The perceptual loss is applied between real pixels x0 and the finetuned prediction x̂0.

6 Experiments

Experiments show that AsymFlow improves ImageNet pixel-generation quality, convergence, and robustness by using a meaningful low-rank PCA subspace. Finetuning a latent FLUX.2 klein model into pixel space also yields state-of-the-art text-to-image results and improves human-aligned visual quality.

  • ImageNet pixel models: AsymFlow (r = 8) improves over JiT in both FID and IS and is less sensitive to σmin clamping during sampling.The comparison uses the optimal σmin = 0.04 for both methods after 600 epochs.
  • ImageNet pixel models: The best patch rank is r = 8; higher ranks mildly degrade guided FID, while random subspaces perform close to JiT and far worse than PCA.These results support using a meaningful low-rank PCA subspace rather than reducing rank alone.
  • ImageNet pixel models: AsymFlow (r = 8) reaches comparable FID roughly 40% faster than JiT with the same architecture and training recipe.The rank-asymmetric target improves both final quality and optimization efficiency.
  • ImageNet pixel models: 1.57 FID establishes AsymFlow with REPA as state of the art among practical ImageNet 256×256 pixel diffusion models.It outperforms prior plain-transformer models by a large margin: FID 1.57 vs. 1.81*.
  • Text-to-image finetuning: Finetuned AsymFLUX.2 klein improves over its FLUX.2 klein latent base on HPSv3, DPG-Bench, and GenEval, then surpasses PixelDiT-T2I across all metrics.The model is finetuned from the pretrained FLUX.2 klein Base 9B latent flow model into pixel space and establishes a new state of the art for pixel-space text-to-image generation.
  • Text-to-image finetuning: Variance reduction boosts all metrics except pFID, while LPIPS perceptual correction improves pFID and HPS scores and produces more natural, realistic texture.Variance reduction can introduce excessive noise from low-noise approximation error; LPIPS suppresses this artifact while preserving sharpness.

7 Conclusion

AsymFlow enables high-dimensional pixel-space generation with plain diffusion transformers and achieves a leading 1.57 FID among ImageNet pixel diffusion models. Its latent-to-pixel finetuning route improves visual fidelity but depends on a good patch-level linear lift.

  • Conclusion: AsymFlow enables high-dimensional pixel-space generation with plain diffusion transformers through a rank-asymmetric flow velocity parameterization.The parameterization is presented as a single approach for pixel-space generation.
  • Conclusion: 1.57 FID is achieved when AsymFlow is trained from scratch, leading among ImageNet pixel diffusion models.This result concerns the from-scratch setting and the ImageNet pixel diffusion-model comparison.
  • Limitations: Latent-to-pixel finetuning assumes a good patch-level linear lift and may fail when pretrained latent spaces do not preserve pixel structure, such as RAE models.The limitation specifically concerns latent representations that lack sufficient pixel structure.

A Method Details … B Experiment Details

The method constructs an orthonormal low-rank pixel subspace, calibrates latent-to-pixel scale and timesteps, and analytically converts asymmetric predictions into full-rank velocities. Finetuning further reduces variance with adaptive patch-wise weighting and blends this correction with LPIPS near the low-noise end.

  • A.1 Low-Rank Subspace Construction: AsymFlow requires a patch-wise low-rank subspace, using PCA directions for training from scratch and an orthogonally aligned Procrustes lift for latent-to-pixel finetuning.Both constructions require orthonormal columns so projected Gaussian noise remains Gaussian in low-rank coordinates.
  • A.1 Low-Rank Subspace Construction: The PCA construction retains dominant patch variations, while the Procrustes objective maximizes alignment between lifted latent tokens and pixel patches under orthonormality.Procrustes aligns directions but does not determine the correct pixel scale.
  • A.2 Scale and Timestep Calibration: Scale calibration introduces s so the calibrated latent lift and projected pixels match in Frobenius norm, correcting the Procrustes lift’s magnitude mismatch.The calibrated lift is sAz0 with ATA = Id and P = AAT.
  • A.2 Scale and Timestep Calibration: Noisy-input calibration matches the projected signal-to-noise ratio and noise magnitude to the pretrained latent trajectory by selecting latent time τ and an input rescaling.The projected signal coefficient is sαt and the noise coefficient is σt.
  • A.2 Scale and Timestep Calibration: The calibrated network prediction is converted back to the original full-rank pixel velocity, with the generalized formula reducing to the uncalibrated case when s = 1 and k = 1.The converted velocity is used in flow-matching loss and denoising sampling.
  • A.3 Adaptive Weighting for Variance Reduction: Adaptive variance reduction chooses each patch’s coefficient by one-dimensional least squares, subtracting the deviation component aligned with the frozen low-rank prediction.In practice, the coefficient is clamped as λ = min(max(λ⋆, 0), 1).
  • A.4 Perceptual Correction: Because the variance-reduction term can introduce low-rank approximation error for t < 1, the method reduces reliance on it near the low-noise end.The condition needed for the approximation generally does not hold away from t = 1.

B.1 ImageNet Experiments

ImageNet 256×256 experiments use JiT-H/16’s architecture and training setup, with AsymFlow evaluated under JiT-aligned inference settings and tuned guidance configurations. Training costs approximately 1750 NVIDIA H100 GPU hours for 600 epochs.

  • Experimental setup: ImageNet 256×256 training uses JiT-H/16’s architecture, optimizer, and other hyperparameters for 600 epochs.Training costs approximately 1750 NVIDIA H100 GPU hours; the REPA variant applies its loss after the 8th transformer block with weight 0.5.
  • Inference settings: At inference, AsymFlow uses σmin = 0.04, a 50-step Heun ODE solver, class-balanced sampling, BF16 inference, and attention upcasting.The σmin = 0.04 clamp performs better than JiT’s default σmin = 0.05 for both JiT and AsymFlow.
  • Guidance configuration: CFG results use grid searches with 0.1 scale steps and 0.02 guidance-interval steps.The final Table 1 AsymFlow result uses CFG scale 2.3 and interval [0, 0.88], while the Table 2 REPA result uses scale 2.2 and the same interval.
  • Guidance configuration: Table 5 reports guidance settings selected by grid-searching guided FID for each patch rank.These settings are used for the ImageNet patch-rank sweep.

B.2 Text-to-Image Experiments · C Mathematical Derivations · C.1 AsymFlow Decomposition and Recovery

The text-to-image setup aligns a low-rank pixel subspace with FLUX.2’s latent dimension while preserving the original token count and speed. Mathematically, AsymFlow decomposes into velocity-like and clean-data components, then exactly recovers standard full-rank velocity across ranks.

  • B.2 Text-to-Image Experiments: Pixel experiments use normalized Oklab images with 16×16 patches, giving D = 768 and AsymFlow rank r = d = 128.The rank matches the original FLUX.2 latent dimension.
  • B.2 Text-to-Image Experiments: Finetuning uses 3M filtered LAION-Aesthetics images, one-megapixel resolution, Qwen2.5-VL captions, frozen base weights, projection updates, and rank-256 LoRA adapters.The setup is designed to reduce overfitting and preserve the pretrained model.
  • B.2 Text-to-Image Experiments: AsymFLUX.2 klein preserves the original latent model’s token count and per-step runtime, while avoiding VAE processing makes overall generation marginally faster.The comparison baseline is the original FLUX.2 klein latent model.
  • C.1 AsymFlow Decomposition and Recovery: With orthonormal A, P = AA^T is an orthogonal projector, and AsymFlow applies projection only to the Gaussian-noise term while retaining the full data term.The projector satisfies A^TA = I_r, P^2 = P, and (I − P)P = 0.
  • C.1 AsymFlow Decomposition and Recovery: AsymFlow is velocity-like in Im(P) and x0-like in Im(I − P), separating low-rank velocity information from the orthogonal clean-data component.Projecting onto Im(P) retains low-rank noise minus low-rank data; the complementary projection removes noise and yields negative clean data.
  • C.1 AsymFlow Decomposition and Recovery: The asymmetric target exactly reconstructs the standard full-rank velocity by retaining the low-rank branch and converting the orthogonal clean-data branch through the usual x0-to-u relation.The orthogonal clean-data component is sign-corrected before conversion to velocity.
  • C.1 AsymFlow Decomposition and Recovery: At rank r = 0, AsymFlow becomes signed full x0-prediction; at full rank r = D, it becomes standard velocity prediction.These endpoint cases correspond to P = O and P = I, respectively.

C.2 Latent–Pixel Flow Coupling at Initialization · C.3 Details on Variance-Reduced Loss

C.2 establishes exact latent-to-pixel trajectory coupling at initialization: lifting the latent process and adding analytically determined orthogonal noise reproduces the low-rank pixel flow throughout sampling. C.3 shows that the variance-reduced objective preserves the conditional mean while lowering sampling variance, with approximation limits addressed by perceptual correction.

  • C.2 Latent–Pixel Flow Coupling at Initialization: AsymFlow uses a rank-matched lift A from latent tokens to pixel patches, with orthonormal columns and projection P = AA^T onto the latent-induced pixel subspace.The construction sets the pixel low-rank rank to r = d and defines the lifted target as xL_0 = Az0.
  • C.2 Latent–Pixel Flow Coupling at Initialization: Projecting the lifted pixel process through A^T exactly recovers the pretrained latent process, so the lifted pixel model evaluates the latent network at the paired noisy latent state.The pixel process uses full-rank pixel noise, while A^T maps it to the corresponding latent noise.
  • C.2 Latent–Pixel Flow Coupling at Initialization: At initialization, the low-rank branch is the lifted latent velocity, while the orthogonal branch is recovered from the noisy pixel state; this simplification does not apply after finetuning.The initialization-specific simplification relies on the lifted latent prediction lying in the low-rank subspace.
  • C.2 Latent–Pixel Flow Coupling at Initialization: The coupled trajectories satisfy xL_t = Az_t + σ_t(I − P)ϵ for all t ∈ (0, 1], preserving the lifting relation through the final sample.The same relation holds under Euler discretization with a shared time grid, making the initialized lifted latent model an exact low-rank pixel flow model.
  • C.3 Details on Variance-Reduced Loss: The variance-reduced loss acts as a control variate: a paired low-rank target reduces pixel-residual variance without changing the conditional mean target.Before approximation, the objective remains equivalent to standard flow matching in x0 format.
  • C.3 Details on Variance-Reduced Loss: Because E[xL_0|x_t] is unavailable, the practical loss approximates it with the frozen low-rank model prediction from the paired noisy low-rank sample.Substituting this approximation yields the practical variance-reduced loss in Eq. (7).
  • C.3 Details on Variance-Reduced Loss: The approximation is exact when the full and paired low-rank noisy inputs differ only in the orthogonal complement, because their low-rank components then match.This condition ensures the frozen low-rank model receives identical low-rank information.
  • C.3 Details on Variance-Reduced Loss: The sufficient condition generally fails because of VAE encoder nonlinearity, so approximation error can enter the low-rank subspace and is addressed with perceptual correction in the low-noise regime.The correction replaces variance reduction in that regime, as detailed in Sec. A.4.

D Additional Qualitative Results · E Impact Statement

The paper provides additional qualitative text-to-image comparisons and highlights both the creative benefits and societal risks of enhanced photorealism. It emphasizes high-fidelity creation alongside risks from disinformation, non-consensual media, and persuasive dataset biases.

  • D Additional Qualitative Results: Additional qualitative text-to-image comparisons are presented in part A.Figure 9 documents these comparisons.
  • D Additional Qualitative Results: Additional qualitative text-to-image comparisons are also presented in part B.Figure 10 documents these comparisons.
  • E Impact Statement: Enhanced photorealism significantly benefits creative industries through high-fidelity prototyping and asset creation.The impact statement identifies these applications as a major benefit of the method.
  • E Impact Statement: More realistic imagery creates a dual-use risk by facilitating convincing disinformation.The statement links this capability to increased potential for societal harm.
  • E Impact Statement: More realistic imagery also facilitates non-consensual media, increasing the potential for societal harm.The impact statement explicitly identifies non-consensual media as a harmful use.
  • E Impact Statement: Higher visual quality requires renewed scrutiny of dataset biases because those biases will be rendered more persuasively.The statement warns that improved realism can amplify the persuasive presentation of biased data.
Loading 2605.12964v2…