PaperlayerView exact revision on arXiv

Source-linked AI summary

Simplex-to-Euclidean Bijections for Categorical Flow Matching

Bernardo Williams, Victor M. Yeom-Song, Marcelo Hartmann, Arto Klami

arXiv:2510.27480v2cs.LG

TL;DR

Simplex-based categorical modeling must account for the simplex’s geometry and boundary, where discrete observations lie. The paper maps the interior to Euclidean space and uses Dirichlet interpolation for categorical data; it reports competitive performance, including an advantage over discrete-state models in low-dimensional problems.

  • Problem

    Simplex-based models must address the simplex’s non-Euclidean geometry and its boundary, where categorical observations lie.

    Source: §1 · p. 1

  • Method

    The method maps the simplex interior to Euclidean space with Aitchison-geometry-based bijections and uses Dirichlet interpolation to represent categorical observations as interior points.

    Source: §1 · p. 1, §3 · p. 3, §3.1 · p. 4

  • Results

    The method is highly competitive among continuous relaxations and outperforms discrete-state models on low-dimensional problems.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The approach provides a bridge from Euclidean continuous generation to discrete data and can be demonstrated with Flow Matching.

    Source: §6 · p. 8

    The model defines a continuous density on the simplex interior, so it cannot directly evaluate categorical probabilities.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

We propose a method for learning and sampling from probability distributions supported on the simplex. Our approach maps the open simplex to Euclidean space via smooth bijections, leveraging the Aitchison geometry to define the mappings, and supports modeling categorical data by a Dirichlet interpolation that dequantizes discrete observations into continuous ones. This enables density modeling in Euclidean space through the bijection while still allowing exact recovery of the original discrete distribution. Compared to previous methods that operate on the simplex using Riemannian geometry or custom noise processes, our approach works in Euclidean space while respecting the Aitchison geometry, and achieves competitive performance on both synthetic and real-world data sets.

1 INTRODUCTION

The paper addresses how simplex-based models can handle both the simplex’s non-Euclidean geometry and categorical observations at its boundary. It maps the simplex interior to Euclidean space and uses Dirichlet interpolation to represent categorical data continuously for standard generative models.

1 INTRODUCTION

  • The method addresses two challenges for simplex-based categorical modeling: the boundary, where discrete data lie, and the simplex’s non-Euclidean geometry.The introduction distinguishes simplex-based models from continuous relaxations that work in the ambient space.Source: §1 · p. 1
  • The approach maps the simplex interior to Euclidean space with a smooth bijection based on compositional data analysis and Aitchison geometry.The method uses Flow Matching as its continuous generative model and proposes stick-breaking and isometric logratio bijections.Source: §1 · pp. 1–2
  • Dirichlet interpolation maps categorical observations to interior points for training, while applying the inverse bijection and arg max recovers discrete samples.The Dirichlet interpolation is described as a stochastic generalization of a recent dequantization approach by Chereddy and Femiani (2025).Source: §1 · p. 2, Figure 1 · p. 2
  • The method achieves highly competitive performance among continuous relaxations and outperforms discrete-state models on low-dimensional problems.The paper demonstrates its approach with Flow Matching and two alternative bijections: stick-breaking and isometric logratio.Source: §1 · p. 2

2 BACKGROUND

The background establishes the simplex and geometric setting, then reviews Conditional Flow Matching as the Euclidean model used in the paper. It also describes Statistical Flow Matching, which maps the simplex to a sphere but retains Riemannian machinery and cannot evaluate likelihoods on the boundary.

2 BACKGROUND

  • For K classes, the simplex has dimension D = K − 1; its open simplex contains points with all coordinates positive, while categorical data lie on its boundary.The boundary consists of points with at least one zero coordinate.Source: §2 · p. 2

2.1 Conditional Flow Matching

  • Conditional Flow Matching learns a vector field that transports samples from a simple base distribution to the target distribution by regressing toward conditional velocities along interpolating paths.A commonly used path is linear interpolation between the endpoints.Source: §2.1 · p. 2

2.2 Riemannian Flow Matching on the Simplex

  • Statistical Flow Matching maps the simplex to the positive orthant of a sphere, but its volume term is singular at the boundary, so likelihood evaluation is limited to the open simplex.Training also requires a Riemannian variant of Flow Matching and its associated geometric machinery.Source: §2.2 · p. 3

3 METHOD

Simplex-to-Euclidean Flow Matching maps the open simplex to Euclidean space and uses Dirichlet interpolation to handle categorical observations on the boundary. Its components support continuous-model training and recovery of discrete categories.

3 METHOD

  • FM-∆ maps the open simplex bijectively to Euclidean space, with information-preserving mappings between the simplex interior and discrete categories.The method combines an open-simplex bijection with Dirichlet interpolation for boundary observations.Source: §3 · p. 3

3.1 Simplex-to-Euclidean bijections

  • ILR is order-invariant and isometric between the simplex with Aitchison geometry and Euclidean space, so Flow Matching paths follow Aitchison geodesics.The isometry connects Euclidean computations to the simplex's compositional geometry.Source: §3.1 · p. 4
  • The shifted stick-breaking transform is order-dependent but centers the mapping so the Euclidean zero vector maps to the simplex midpoint.The paper also considers ILR as an order-invariant alternative.Source: §3.1 · p. 4

3.2 Handling discrete data

  • Dirichlet interpolation trains on a continuous mixture, while λ > 1/2 gives almost-sure argmax recovery and vanishing mixture-density error recovers categorical probabilities in total variation.The training procedure samples Dirichlet noise for categorical observations; the recovery and convergence claims apply to the interpolated distribution.Source: §3.2 · pp. 4–5
  • The method recommends the smallest valid λ and α = 100; under ILR, the transformed Dirichlet covariance is independent of category count K.For stick-breaking, the same α choice is used without the corresponding theoretical covariance result.Source: §3.2 · p. 5

3.3 Training, sampling and evaluation

  • For categorical data, training uses interpolation and sampling uses argmax after solving an ODE; compositional data skips interpolation.The paper names Euler and Dormand–Prince as example ODE solvers.Source: §3.3 · p. 5
  • The model does not directly evaluate categorical probabilities because it defines only a continuous density on the simplex interior.The paper says these probabilities are not needed for learning or sampling, though estimating them is useful for evaluation.Source: §3.3 · pp. 5–6
  • On Checkerboard, FM-∆ aligns more closely with the true distribution than LinearFM and SFM, whose invalid-sample counts are each more than twice as high.Figure 3 marks samples outside the true density in red and zooms in on the top region.Source: Figure 3 · p. 6

4 EXPERIMENTS

The experiments report competitive results across compositional data, image, DNA, and text tasks, and compare categorical-probability divergence across category counts. Performance varies by benchmark: the method leads among continuous relaxations on Text8, while discrete-space models lead in NLL.

Experimental setup

  • We evaluate our approach on five tasks against continuous and discrete baselines, report results with and without minibatch OT, and fix λ = 1.LinearFM denotes a baseline using Euclidean geometry directly on the simplex.Source: §4 · p. 6

Binarized MNIST

  • FM-˚∆ has the lowest NLL and FID on Binarized MNIST, with similar performance across its four variants.Some baselines—D3PM, DFM, and DirichletFM—perform substantially worse.Source: Table 1 · p. 7, §4 · p. 7

Compositional data

  • For compositional checkerboards, FM-˚∆ aligns more closely with the true distribution than LinearFM and SFM, which produce more poor samples near vertices.Both LinearFM and SFM have more than twice as many invalid samples as this method.Source: §4 · pp. 6–7

DNA sequence generation

  • On DNA sequence generation, the method is best under SP-MSE evaluation using a pretrained Sei model.The evaluation compares generated and test sequences through their embeddings.Source: Table 2 · p. 7, §4 · p. 7

Text8

  • On Text8, discrete-space models have the best NLL, while this method is best among continuous relaxations; entropy is similar across methods.Evaluation uses GPT-J-6B, with generated samples tokenized to its vocabulary.Source: §4 · p. 7, Table 3 · p. 8

Scalability

  • Across category counts, the method generally outperforms SFM and LinearFM, especially at medium dimensionalities, and is comparable to SEDD through K = 27.Figure 4 reports divergence between ground-truth and estimated categorical probabilities.Source: §4 · p. 7, Figure 4 · p. 8

5 DISCUSSION

The discussion situates the approach among recent simplex and Euclidean methods for categorical generation. It emphasizes the method’s generality and its use of simplex geometry with established continuous-generation tools.

Related methods

  • Diederen and Zamboni (2025) use ILR for compositional data without discrete observations, while Potapczynski et al. (2020) target a VAE-specific bijection.The latter approach recovers categories with arg max but has no obvious generalization to other generative models.Source: §5 · p. 8
  • Chereddy and Femiani (2025) use an additive logratio transform and deterministic interpolation, while Cheng et al. (2025) define a different geometry on logit-simplex space.The discussion notes that the former’s mass stays close to the borders and its solution is geared to order-dependent ALR.Source: §5 · p. 8
  • Mahabadi et al. (2024) and Tae et al. (2025) place tokens near simplex vertices without formally connecting them to vertices and well-defined interior points.Euclidean-noise models such as Hoogeboom et al. (2020, 2021) use arg max but do not connect to the simplex’s natural geometry.Source: §5 · p. 8

Discussion

  • The authors position their method as a general way to use continuous-generation tools for discrete data while accounting for simplex geometry without complex tools.They characterize its results as competitive and its approach as conceptually and computationally elegant.Source: §5 · p. 8

6 CONCLUSION

The conclusion presents a bridge between Euclidean continuous generation and discrete generation on the simplex, demonstrated with flow matching and promising results.

Conclusion

  • The method bridges Euclidean continuous generation and discrete generation on the simplex, enabling a broad range of Euclidean generative models for categorical data.The paper demonstrates the approach with flow matching and identifies other generative models as future work.Source: §6 · p. 8

Checklist

The checklist marks documentation of model assumptions, theoretical results, experiments, and existing assets as included, while several asset and human-subject items are not applicable.

Checklist

  • The checklist marks model assumptions, algorithm properties and complexity, and dependency-specified source code as included.Anonymized source code is listed as optional.Source: Checklist · p. 11
  • The checklist marks full assumptions, complete proofs, and explanations of assumptions for theoretical claims as included.Source: Checklist · p. 11
  • For empirical results, the checklist marks reproducibility materials, training details, metrics and error bars, and computing infrastructure as included.Source: Checklist · p. 11
  • The checklist marks citations and licenses for existing assets as included, while new-asset release and human-subject items are not applicable.Source: Checklist · p. 11

A.1 Proof of proposition 1

Proposition 1 connects accuracy in modeling the Dirichlet-interpolated mixture to accuracy in the induced categorical distribution: the total-variation error is bounded by the mixture-density error, and convergence recovers the true law.

A.1 Proof of proposition 1

  • The interpolated mixture has almost surely disjoint component supports, with component k contained in the strict arg max region for category k.Source: §A.1 · p. 12
  • The total-variation error between generated and true categorical probabilities is bounded by half the L1 distance between the modeled density and the interpolated mixture.The bound applies componentwise to each category probability.Source: §A.1 · p. 12
  • If the modeled density converges to the interpolated mixture in L1, arg max discretization recovers the true categorical distribution in total variation.Source: §A.1 · p. 12

A.2 Proof of proposition 2

Proposition 2 shows that Dirichlet interpolation preserves the original category under arg max when the vertex weight is at least one half, with the boundary case holding almost surely.

A.2 Proof of proposition 2

  • The interpolation combines the category vertex with a Dirichlet sample as x = λc + (1 − λ)ε.Source: §A.2 · p. 12
  • For λ ≥ 1/2, arg max of the interpolated point returns the original category; at λ = 1/2, this holds almost surely under the Dirichlet noise.Source: §A.2 · p. 12
  • At λ = 1/2, equality between the category coordinate and another coordinate can occur only on a probability-zero Dirichlet boundary event.Source: §A.2 · p. 12

A.3 Proof of proposition 3

These appendices establish properties of the ILR covariance and isometry, relate stick-breaking to the unit-simplex transform, and derive implementation and density-estimation details for the bijections.

A.3 Proof of proposition 3

  • For a symmetric Dirichlet input, ILR coordinates are uncorrelated with variance ψ′(α), independent of the number of categories K.Source: §A.3 · p. 13

A.4 Proof of Theorem 1

  • The ILR map is an isometry between the simplex with Aitchison geometry and Euclidean space.Source: §A.4 · p. 13

A.5 Stick-breaking transform

  • The stick-breaking inverse transform equals the unit-simplex inverse transform, and its lower-triangular Jacobian determinant is the product of its diagonal terms.Source: §A.5 · p. 14

A.6 Isometric logratio transform

  • The ILR transform is z = H log x, and its matrix-vector product can be computed in O(K) time without explicitly forming H.Source: §A.6 · pp. 14–15

A.7 Simplex to sphere transform +

  • The sphere transform maps x to √x, and its volume element is derived using a rank-one-update determinant calculation.Source: §A.7 · pp. 15–16

A.8 Categorical probabilities estimation

  • The interpolated component density is supported on the truncated simplex where x_k ≥ λ, and the categorical-probability estimator combines model and component densities.Source: §A.8 · pp. 16–17

A.9 Standardize the data

  • Mean compositions move logarithmically farther from the origin as category count increases, while preliminary experiments found no notable benefit from norm-based normalization.Source: §A.9 · p. 17

B EXPERIMENTAL DETAILS

The experimental appendix documents task-specific evaluation setups and reports categorical-probability and parameter-effect findings, alongside the design of runtime comparisons.

B EXPERIMENTAL DETAILS

  • The experiments use task-specific solvers and architectures, including Dopri5 for compositional data and Euler-based sampling for BMNIST, promoter design, and Text8.Source: B EXPERIMENTAL DETAILS · pp. 17–18

C.1 Estimation of the categorical probabilities

  • Up to 25 categories, the estimator closely matches true probabilities in KL divergence and estimates p1 = 0.5 accurately, while SFM's lower bound deviates and can overflow at high category counts.Source: §C.1 · pp. 18, 20, Figure 6 · p. 20

C.2 Effect of the parameters

  • Scaling improves accuracy with 2 categories but degrades it above 25; α = 1 is slightly more accurate than λ = 0.99 for λ ∈ {0.5, 0.75}.Sensitivity to α and λ increases with K; the authors did not fine-tune them, aiming for an easy-to-use pipeline.Source: §C.2 · p. 20

C.3 Run Times

C.4 Numerical results for Additive Logratio and Multiplicative Logratio bijections

The section evaluates ALR and MLR mappings on Checkerboard, binarized MNIST, and DNA, finding that the proposed mappings retain dataset-specific advantages while ALR and MLR also work relatively well.

C.4 Numerical results for Additive Logratio and Multiplicative Logratio bijections

  • The experiments address the open question of how well the previously defined ALR and MLR bijections work.The section introduces their numerical evaluation after defining ALR and MLR as candidate bijections.Source: §C.4 · p. 20
  • SB remains best on DNA and Checkerboard, while ILR remains best on binarized MNIST.The numerical comparisons are reported in Table 7.Source: §C.4 · p. 20, Table 7 · p. 21
  • Figure 7 compares KL divergence between true and estimated categorical probabilities across α ∈ {1, 10, 100, ∞}, λ ∈ {0.5, 0.75, 0.99}, and with or without scaling.Bands indicate two standard deviations from the mean; the caption does not state comparative outcomes.Source: Figure 7 · p. 21
  • ALR and MLR also work relatively well, and SB and MLR are highly similar, exactly matching for BMNIST with 2 classes.The text attributes SB's main distinction from MLR to stabilizing computation by centering.Source: §C.4 · p. 21

Related papers

Loading 2510.27480v2…