PaperlayerView exact revision on arXiv

Source-linked AI summary

Geometry-Aware Image Flow Matching

Junho Lee, Kwanseok Kim, Joonseok Lee

arXiv:2605.25294v1cs.CV

TL;DR

Natural-image generation largely assumes Euclidean geometry despite potentially meaningful intrinsic structure. This paper models images on a hypersphere with spherical flow-matching methods, which consistently outperform Euclidean baselines across CIFAR-10 and ImageNet-256, RGB and latent spaces.

  • Problem

    Natural-image generation relies on Euclidean assumptions despite limited understanding and use of the data’s intrinsic geometric structure.

  • Method

    The paper models natural images on a hypersphere and introduces SOT-CFM with angular distance and SFM with geodesic dynamics.

  • Results

    Spherical methods consistently outperform Euclidean baselines across CIFAR-10 and ImageNet-256, with SFM achieving the best performance among evaluated variants.

  • Takeaways & Limitations

    Natural images can be effectively modeled through intrinsic spherical geometry because semantic information is primarily directional and norms can be approximated by dataset averages.

  • Takeaways & Limitations

    The approach assumes that image norms can be approximated by a global dataset average, including in high-dimensional latent spaces.

Abstract

from arXiv · show

Recent advances in generative models highlight the power of geometry-aware modeling in manifold-constrained settings. Yet, for natural images, the field remains confined to Euclidean assumptions, failing to exploit the potential of intrinsic geometric structures within the data. In this work, we investigate the geometry of natural images and observe that semantic information is predominantly encoded in directional components, while norm components can be approximated by the global average. This property holds across both RGB and latent spaces, suggesting that natural images can be effectively modeled on a hypersphere. Building on this finding, we introduce Spherical Optimal Transport Flow Matching (SOT-CFM), which utilizes angular distance, and Spherical Flow Matching (SFM), which constrains dynamics directly on the manifold. Our experiments demonstrate that these geometry-aware methods achieve superior performance against Euclidean baselines. Ultimately, this work provides a novel perspective that bridges the gap between Riemannian manifold-based modeling and natural image generation.

1. Introduction

The paper argues that natural-image semantics are primarily encoded in direction rather than magnitude, motivating a hyperspherical view of both RGB and latent representations. It introduces spherical flow-matching methods that exploit this geometry and reports improved generation quality over Euclidean baselines.

  • Existing image-generation methods fundamentally model images in high-dimensional Euclidean space, potentially missing their intrinsic geometric structure.
  • Semantic information is predominantly encoded in directional components, while norms contribute minimally and can be approximated by the dataset-wide average, including in reconstruction-oriented latent spaces.
  • Hyperspherical projection preserves semantic and visual integrity despite substantial L2-norm changes, supporting a fixed-radius hyperspherical model for RGB and latent images.
  • The proposed geometry-aware flow matching uses hyperspherical projection or directional metrics, letting models focus exclusively on learning direction rather than magnitude.
  • SOT-CFM and SFM are evaluated against Euclidean I-CFM and OT-CFM on CIFAR-10 and ImageNet-256, with spherical projection improving generation quality by reducing magnitude-modeling difficulty.

2. Preliminary

This section introduces Flow Matching as ODE-based transport from a source distribution to data, then presents conditional, optimal-transport, and manifold-aware extensions. These methods replace intractable marginal velocities or inefficient Euclidean paths with conditional fields, optimized couplings, or geodesic dynamics.

  • Flow Matching: Flow Matching transports a source distribution p0, typically standard Gaussian, to a target distribution p1 through a time-dependent velocity field and an ODE.It learns a neural approximation vϕ to the marginal velocity field and solves the resulting ODE during sampling.
  • Conditional Flow Matching: Conditional Flow Matching avoids directly computing the intractable marginal velocity by constructing a conditional probability path and matching its conditional velocity field.The path uses a coupling π between source and target distributions, with linear interpolation as the standard choice.
  • Conditional Flow Matching: When σ(t) ≡0, the conditional path is deterministic with velocity ut = ˙α(t)(x1 −x0), including the linear schedule α(t) = t used for Optimal Transport paths.More general interpolation schedules can be used to improve sample quality.
  • Optimal Transport CFM: OT-CFM replaces independent source-target coupling with optimal pairings based on a cost function, while mini-batch approximation yields straighter, more stable, and faster flows.The typical cost is ∥x0 −x1∥2, and exact coupling over the entire dataset is approximated on finite mini-batches.
  • Riemannian Flow Matching: Riemannian Flow Matching extends flow matching from Euclidean space to a manifold M with metric tensor g, using tangent-space vector fields and geodesic conditional paths.Its training objective measures errors with the Riemannian norm induced by g.

3. Flow matching on Spherical Geometry

The section shows that natural-image semantics are concentrated in directional components, making spherical geometry a useful approximation across RGB and latent spaces. Building on this structure, it introduces SOT-CFM with angular transport costs and SFM with geodesic dynamics constrained to a hypersphere.

  • Geometric structure: Natural images can be effectively approximated by spherical geometry because meaningful information lies mainly in direction while norm is well approximated by a global average.This structure enables projecting data onto a single hypersphere and focusing model capacity on semantically important directional variations.
  • Geometric structure: Projection experiments show robust reconstruction quality across hypersphere radii in RGB and SD3-VAE latent spaces, with rFID near zero and LPIPS consistently low in RGB space.The robustness also extends across multiple autoencoder latent spaces and datasets including CIFAR-10, ImageNet, COCO-2014, and CelebA-HQ.
  • SOT-CFM: SOT-CFM replaces Euclidean transport costs with angular distance on directional components, making couplings invariant to magnitude differences and better aligned with semantic similarity.Euclidean OT can assign high costs to semantically similar pairs when their magnitudes differ, despite similar angular separation.
  • SFM: SFM constrains source and target distributions to the hypersphere and defines flow paths as manifold geodesics, so the entire generative process operates within spherical geometry.Projected vectors share radius r, and the path geometry is determined by the angle θ between them.
  • SFM: SFM establishes a practical application of manifold-based generative modeling to natural images and demonstrates the viability of geometry-aware frameworks for real-world image generation.The approach constrains generation to the hypersphere where crucial semantic information resides.

4. Experiment

Experiments on CIFAR-10 and ImageNet-256 evaluate spherical projection, angular transport, and fully Riemannian flow matching against Euclidean baselines. The results show consistent benefits from spherical geometry, with SFM achieving the strongest overall performance and remaining robust across radius choices.

  • Experimental Setup: Experiments use CIFAR-10 and class-conditional ImageNet-256, evaluating 50,000 generated samples with gFID, sFID, IS, precision, and recall.CIFAR-10 generation is unconditional, while ImageNet-256 uses classifier-free guidance and Euler sampling.
  • Baselines: The comparison covers Euclidean I-CFM and OT-CFM, their spherical adaptations, and fully Riemannian SFM to separate projection and intrinsic-geometry benefits.Spherical projection targets training efficiency, whereas intrinsic geometric modeling targets performance improvements.
  • Utilizing Spherical Geometry: Spherical projection consistently improves CIFAR-10 and ImageNet-256 without changing architecture, sampling strategy, or training objective.Removing norm variability focuses learning on directional semantic structure and provides a common scale across samples.
  • Utilizing Spherical Geometry: SOT-CFM reaches gFID 4.11 versus OT-CFM’s 4.30 on CIFAR-10 and improves ImageNet-256 gFID from 5.22 to 5.15.The gains are reliable across datasets, while combining projection with SOT-CFM further improves ImageNet-256.
  • Spherical Flow Matching: SFM attains the best CIFAR-10 gFID of 3.79 and achieves the best ImageNet-256 results across all metrics except precision.SFM operates source, target, and flow paths directly on the hyperspherical manifold, outperforming Euclidean methods even with spherical projections.
  • Radius Ablation: SFM performance peaks at r = 120 with gFID: 4.62, remains stable across a broad radius range, and slightly degrades at r = 130.Increasing radius amplifies tangent-vector magnitudes for a given angular displacement, strengthening the regression target without changing directional structure.

5. Related Work

This section situates the work within advances in Flow Matching, geometric generative modeling, Optimal Transport, and latent diffusion. It highlights progress toward scalable, manifold-aware, and modality-diverse generation while identifying limitations of Euclidean metrics for image data.

  • Flow Matching: Flow Matching replaced costly Continuous Normalizing Flow trace computations with simulation-free objectives that regress directly onto conditional vector fields, improving scalability.Early CNFs also faced training instability, whereas Flow Matching enabled practical deployment.
  • Flow Matching: Recent Flow Matching extensions support discrete states, simplex-constrained distributions, alpha-blended trajectories, and metric-aware paths.These developments expand Flow Matching beyond linear interpolation to accommodate diverse probability paths.
  • Applications: Flow Matching has enabled advances in text-to-image synthesis, controllable generation, audio and music generation, and hierarchical video synthesis.The cited applications span visual and temporal modalities.
  • Geometric Generative Modeling: Manifold-aware generative models include sphere- and torus-specific normalizing flows, Riemannian Continuous Normalizing Flows, and Riemannian score-based and diffusion models.Riemannian Continuous Normalizing Flows integrate differential geometry through manifold-aware ODEs respecting local curvature.
  • Optimal Transport: Optimal Transport supplies principled source-target pairings for Flow Matching, while Euclidean metrics may miss directional or semantic similarity in high-dimensional images.Entropic-regularized Sinkhorn iterations and progressive mini-batch solvers improve OT scalability for large-scale generation.
  • Latent Diffusion: Latent diffusion compresses images into semantically meaningful lower-dimensional spaces using autoencoders, with later systems adding higher-resolution training and rectified flows.The passage identifies Autoencoder-KL, SDXL, and Stable Diffusion v3 as successive developments.

6. Conclusion

The paper finds that natural images are effectively modeled on a hypersphere, with semantics primarily encoded in directions and scalar norms approximated by dataset averages. It introduces SOT-CFM and SFM to produce geometrically consistent generative paths using angular metrics and geodesic dynamics.

  • Geometric structure: Natural images are effectively modeled on a hypersphere, with semantic information primarily encoded in directions and scalar norms approximated by dataset averages.This conclusion follows from the paper’s directional decomposition analysis.
  • Proposed methods: The work introduces SOT-CFM and SFM as geometry-aware methods for image generation.These methods build on the hyperspherical modeling insight.
  • Proposed methods: SOT-CFM and SFM leverage angular metrics and geodesic dynamics to ensure geometrically consistent generative paths.The methods incorporate geometry through angular metrics and manifold-consistent dynamics.

Impact Statement · Appendix

The paper advances geometry-aware image generation through Riemannian manifold theory, with foundational contributions rather than new application domains. It reports no immediate negative societal consequences specific to the proposed techniques, while acknowledging broader misuse concerns shared across generative modeling.

  • Impact Statement: The work advances geometry-aware image generation through Riemannian manifold theory.Its contributions are described as foundational in nature.
  • Impact Statement: The paper’s contributions are foundational rather than focused on introducing new application domains.The impact statement explicitly distinguishes the work from application-domain expansion.
  • Impact Statement: Potential misuse of generative models is identified as a broader societal implication shared across the field.The statement does not characterize misuse as specific to this work.
  • Impact Statement: The authors do not anticipate any immediate negative societal consequences from the proposed techniques.This assessment is specific to the techniques introduced in the paper.
  • Impact Statement: The societal-impact discussion centers on the general context of generative-model misuse rather than a newly introduced application domain.The paper states that it introduces no new application domains and that misuse concerns are field-wide.
  • Appendix: The appendix impact statement frames the work as foundational and reports no immediate work-specific negative societal consequences.No separate appendix-specific passage is supplied beyond this impact-statement paragraph.

A. Geodesic Interpolation and Tangent Vector Derivation · B. Implementation Details · B.1. Model Architecture

The method derives spherical geodesic interpolation and its tangent vector field for SFM, ensuring trajectories remain on the hypersphere and velocities remain tangent. Implementation uses task-specific architectures, with U-Net for CIFAR-10 and DC-AE plus DiT-XL/2 for ImageNet-256.

  • A. Geodesic Interpolation and Tangent Vector Derivation: Geodesics between hypersphere points are represented by great-circle arcs connecting the endpoints.The construction assumes points have equal norm r and uses the shortest spherical path.
  • A. Geodesic Interpolation and Tangent Vector Derivation: The derivation focuses on θ ∈(0, π), treating coincident and antipodal endpoints as negligible measure-zero cases.This range guarantees sin θ ≠ 0.
  • A. Geodesic Interpolation and Tangent Vector Derivation: Spherical linear interpolation satisfies γ(0) = ˜x0, γ(1) = ˜x1, and preserves radius r for every t.Thus, the entire interpolation remains on the hypersphere.
  • A. Geodesic Interpolation and Tangent Vector Derivation: Differentiating the interpolation yields the geodesic tangent vector, used as the conditional SFM vector field.The field is defined as ut(˜xt | ˜x0, ˜x1) = ˙γ(t), with ˜xt = γ(t).
  • A. Geodesic Interpolation and Tangent Vector Derivation: The velocity is orthogonal to the current point, so SFM dynamics remain in the tangent space T˜xtSd−1.The tangent-space inner product and norm use the Euclidean metric restricted to that space.
  • A. Geodesic Interpolation and Tangent Vector Derivation: As θ →0, first-order trigonometric approximations reduce spherical interpolation and velocity to Euclidean linear counterparts.This establishes that SFM smoothly generalizes standard Euclidean flow matching as curvature approaches zero.
  • B. Implementation Details: Architectures are optimized separately for each task to accommodate differences in image resolution and dataset scale.The implementation details therefore vary between CIFAR-10 and ImageNet-256.
  • B.1. Model Architecture: CIFAR-10 uses a standard U-Net with 128 base channels, four resolution levels, 16×16 self-attention, residual blocks, GroupNorm, SiLU, and sinusoidal time embeddings.The channel multipliers are [1, 2, 2, 2], with two residual blocks per resolution level and a 2-layer MLP for time embeddings.

B.2. Training Configuration … C.1. Norm Refinement Network

The paper specifies separate training and optimal-transport configurations for CIFAR-10 and ImageNet-256, then evaluates whether norm refinement improves hyperspherical representations. Norm refinement predicts corrections from unit vectors for image- and latent-space models, with expected gains in reconstruction fidelity and negligible inference overhead.

  • B.2. Training Configuration: CIFAR-10 models train for 200,000 iterations with batch size 512 using Adam at learning rate 2 × 10−4 and EMA decay 0.9999 on one NVIDIA A6000 GPU.The optimizer betas are (0.9, 0.999).
  • B.2. Training Configuration: ImageNet-256 models train for 140,000 iterations with batch size 1,024 using AdamW at learning rate 2 × 10−4, weight decay 0, and β = (0.9, 0.95) on 2 NVIDIA A100 40GB GPUs.Training also uses gradient checkpointing and fp16 mixed precision.
  • B.3. Optimal Transport Computation: Mini-batch optimal transport uses batch size 128 for CIFAR-10 and 256 for ImageNet-256, with POT, entropy regularization ϵ = 0.1, and Sinkhorn iterations.For spherical OT, the cost matrix uses geodesic distance c(x0, x1) = arccos(⟨ˆx0, ˆx1⟩).
  • B.4. Implementation Framework: The implementation builds on OT-CFM for CIFAR-10 and LightningDiT for ImageNet-256, using PyTorch and reproducible seeds.These choices define the implementation framework for the reported experiments.
  • C. Norm Prediction for Adjustment: Hyperspherical projection using 1 N i=1 ∥xi∥2 preserves essential information, although minor degradation remains observable.This motivates testing whether a lightweight Norm Refinement Network can recover lost magnitude information.
  • C.1. Norm Refinement Network: ResNet50 and MobileNetV2 encoders take the unit direction vector ˆx and predict a norm correction term ∆sϕ approximating ∆s = s −¯s.The refined vector is constructed from the predicted correction and the unit direction.
  • C.1. Norm Refinement Network: The refinement network uses pixel-level MSE in image space and applies the same process to unit latent vectors, with decoded losses and an additional latent consistency term.For latent models, the prediction is zpred = (¯s+∆sϕ)·ˆz.
  • C.1. Norm Refinement Network: The refinement strategy is expected to improve reconstruction fidelity for representations approximated by the global average while adding negligible inference-time computational overhead.The claim concerns refinement of already well-approximated representations.

C.2. Quantitative and Qualitative Analysis · C.3. Computational Cost · D. Additional Analysis

Norm refinement produces only marginal quality gains, while generated samples remain visually indistinguishable across refinement conditions. Its computational overhead is negligible, supporting hyperspherical projection with a global average norm as the simpler, equally effective choice.

  • C.2. Quantitative and Qualitative Analysis: Norm prediction networks consistently improve gFID across methods and architectures, but the gains are marginal.This supports using hyperspherical projection with the average norm to preserve essential information.
  • C.2. Quantitative and Qualitative Analysis: Explicit norm modeling yields negligible improvement, indicating that directional components dominate semantic encoding.The result supports the theoretical and practical sufficiency of the spherical approximation.
  • C.2. Quantitative and Qualitative Analysis: Generated samples appear almost identical with no refinement, ResNet50, or MobileNetV2 across diverse ImageNet categories.The inspected categories included iguana scales, jewelry, volcano, hummingbird, and pug.
  • C.3. Computational Cost: ResNet50 and MobileNetV2 norm-refinement networks have millisecond-level inference latency and very small computational footprints (< 0.03 GFLOPs).These costs are effectively negligible relative to the diffusion model’s per-step cost.
  • C.3. Computational Cost: Training the MobileNetV2 regressor required 0.044 seconds per iteration, totaling roughly 3 hours for 10,000 iterations.All measurements used the same benchmarking protocol as the main generation model on an NVIDIA A100-40GB GPU.
  • C.3. Computational Cost: Norm refinement is lightweight and inexpensive to deploy, but its marginal benefits offer little justification for adding it.A more sophisticated refinement module could be explored because the current design uses only a minimal architecture.
  • C.3. Computational Cost: Hyperspherical projection using a single global average norm remains a simpler and equally effective choice for high-quality generation.This conclusion follows from the limited quality gains and negligible visual differences associated with norm refinement.

D.1. Robustness of Projection Across Datasets · D.2. Extreme Norm Cases

Mean-norm spherical projection preserves reconstruction quality and semantic content across diverse datasets, but extreme norm cases can cause visible artifacts. Because such outliers are rare, the approach remains viable for most samples, while specialized datasets may require individual treatment.

  • D.1. Robustness of Projection Across Datasets: Mean-norm projection causes minimal rFID increases across CIFAR-10, COCO-2014, and CelebA-HQ, preserving semantic information across tested datasets.CIFAR-10 latent results are omitted because SD3-VAE targets higher-resolution inputs.
  • D.1. Robustness of Projection Across Datasets: Both RGB and latent spaces maintain good reconstruction quality near each dataset’s mean norm across diverse image domains.Figure II evaluates reconstruction quality near the mean norm for each dataset.
  • D.1. Robustness of Projection Across Datasets: COCO-2014 retains excellent reconstruction quality at ImageNet’s mean radius because its RGB and latent mean norms closely match ImageNet’s.The similarity occurs in both representations.
  • D.1. Robustness of Projection Across Datasets: Despite CelebA-HQ’s norm mismatch with ImageNet, RGB reconstruction remains good while latent reconstruction degrades, suggesting greater magnitude-based semantic encoding in latent representations.CelebA-HQ is an extreme, highly homogeneous face-image dataset that may require individual treatment.
  • D.2. Extreme Norm Cases: Extreme minimum- and maximum-norm ImageNet-256 samples show visible artifacts after projection, especially color casts in RGB and brightness shifts for minimum-norm samples.These artifacts occur despite preserved semantic content.
  • D.2. Extreme Norm Cases: Directional components preserve most semantic information at distribution boundaries, but extreme cases show magnitude has greater perceptual impact than in typical samples.The finding motivates examining extreme norm cases after average-norm projection.
  • D.2. Extreme Norm Cases: Less than 1% of samples are extreme boundary outliers, limiting projection degradation and supporting hyperspherical projection for the vast majority of data.RGB norms are broadly symmetric around 250, whereas latent norms are sharper and left-skewed, peaking near 80.

D.3. Class-Specific Norm Analysis. · D.4. Downstream Classification of Reconstructed and Generated Images

Class-specific norm patterns are more pronounced in latent space but can be approximated by a global average without significant perceptual loss. Downstream classification shows that hyperspherical projection preserves reconstruction semantics, while geometry-aware generation substantially improves class discriminability.

  • D.3. Class-Specific Norm Analysis.: Different ImageNet classes exhibit distinct norm characteristics, especially in latent space, with simple objects tending toward lower norms than complex natural scenes.Examples include “packet” and “ring binder” versus “accordion” and “bib.”
  • D.3. Class-Specific Norm Analysis.: Latent norm distributions are sharper and left-skewed near 80 with a long tail, whereas RGB norms are broad and symmetric around 250.These differing distributions explain why extreme cases behave differently across spaces.
  • D.3. Class-Specific Norm Analysis.: Class-specific magnitude variations may encode visual complexity or texture richness, but overlapping distributions and successful projection support global-average norm approximation without significant perceptual loss.The result indicates that magnitude information is not critical for reconstruction.
  • D.4. Downstream Classification of Reconstructed and Generated Images: The ResNet-50 evaluation measures semantic preservation in reconstructions and generation quality beyond rFID and LPIPS using class discriminability.Reconstruction rows use the original validation set, while generation rows use 50,000 generated samples.
  • D.4. Downstream Classification of Reconstructed and Generated Images: 1.01%p is the Top-1 accuracy drop from Original Images to DC-AE Proj Recon, decreasing from 77.30% to 76.29%.This marginal loss indicates that hyperspherical projection preserves class-discriminative features beyond the autoencoder’s reconstruction loss.
  • D.4. Downstream Classification of Reconstructed and Generated Images: 61.83% is the Top-1 accuracy achieved by I-CFM, showing substantially degraded class discriminability despite matching the other methods’ architecture and training data.The comparison concerns generated samples evaluated by the pretrained ResNet-50 classifier.
Loading 2605.25294v1…