PaperlayerView exact revision on arXiv

Source-linked AI summary

A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds

Alessio Marta, Paola Causin

arXiv:2608.23798v1cs.LG

TL;DR

The paper develops an intrinsic theory of speciation in diffusion models on compact Riemannian manifolds, treating it as bifurcations of density critical points rather than only symmetric pitchforks. It proves that generic mixture events are one-dimensional folds, while topology constrains score equilibria and exceptional symmetries produce pitchforks or multidirectional transitions.

  • Problem

    Existing descriptions often identify speciation with symmetric pitchfork bifurcations in large-dimensional spaces, motivating an intrinsic manifold-based theory.

  • Method

    The paper analyzes density critical-point bifurcations using heat-kernel representations, topology, Morse theory, normal forms, perturbation stability, and intrinsic chart-based score learning.

  • Results

    Generic heat-kernel mixture speciation events have one-dimensional critical kernels and A2 fold normal forms, whereas pitchforks and multidirectional transitions occur in nongeneric symmetric configurations.

  • Takeaways & Limitations

    Speciation is generally a geometrically organized evolution of the score landscape, with topology producing geometrical modes alongside data modes.

  • Takeaways & Limitations

    Future work must address curvature effects, realistic multidirectional bifurcations, generic boundaries, and latent diffusion models.

Abstract

from arXiv · show

Speciation in generative diffusion models denotes the emergence of distinct stable branches during denoising, through which initially undifferentiated trajectories progressively commit to different data classes. In this work we develop an intrinsic theory of speciation for diffusion models supported on compact Riemannian manifolds: the aim is to go beyond existing theoretical descriptions, which usually identify speciation with a symmetric pitchfork bifurcation and assume to work in a large-dimensional space. We characterize speciation by bifurcations of the critical points of the evolving probability density. A spectral heat-kernel representation makes explicit the role of the manifold geometry, while Poincaré-Hopf and Morse theory impose global constraints on the number and type of score equilibria and reveal topologically-imposed geometrical modes. For mixtures of heat kernels, we prove that generic speciation events have a one-dimensional critical kernel and admit an A2 fold normal form; pitchforks and simultaneous multidirectional transitions arise from nongeneric symmetric configurations. We derive geometry-dependent estimates of speciation times for bimodal mixtures and Riemannian regular simplices. We further establish structural stability of nondegenerate folds under score perturbations and show that the first-order time shift is determined solely by the component of the score error along the critical direction. The theory is illustrated on the sphere using mixtures of von Mises-Fisher distributions, where pitchfork and saddle-node bifurcations, topological modes, and hierarchical multiple speciations are observed. Finally, a chart-based intrinsic score-learning scheme based on neural networks contrasts the theoretically predicted transitions on prototypal and more complex datasets.

1. Introduction

The paper develops an intrinsic theory of speciation as bifurcations of score equilibria on compact Riemannian manifolds, extending symmetry-breaking accounts beyond Euclidean settings. It combines geometric, topological, singularity-theoretic, stability, and coordinate-based learning perspectives.

  • Motivation: Diffusion trajectories progressively commit to semantic classes during reverse diffusion at characteristic speciation times.Earlier work relates these transitions to pitchfork-like bifurcations and spontaneous symmetry breaking.
  • Intrinsic formulation: The framework models diffusion intrinsically on compact Riemannian manifolds, with heat-kernel spectral expansions exposing the roles of metric and spectrum.The formulation uses Brownian motion, Laplace–Beltrami heat flow, and the Riemannian score.
  • Topology: Poincaré–Hopf and Morse theory constrain score critical points and distinguish data modes from topology-forced geometrical modes.Geometrical modes need not correspond to data classes but arise within topologically admissible critical-point configurations.
  • Local classification: For sufficiently rich heat-kernel mixtures, generic speciation has a one-dimensional critical kernel and an A2 fold normal form.Pitchforks and simultaneous multidirectional transitions instead arise from nongeneric symmetric configurations.
  • Speciation times: The theory derives geometry-dependent speciation-time estimates for bimodal mixtures and symmetric mixtures on Riemannian regular simplices.For equal weights in bimodal mixtures, the estimate depends explicitly on geodesic distance between the modes.
  • Stability and computation: Nondegenerate A2 speciations persist under small smooth score perturbations, while first-order time shifts depend only on score error along the critical direction.The paper also develops chart-based intrinsic score learning and numerical experiments testing theoretical predictions.

2. Noising and denoising processes

This section formulates forward and reverse diffusion through stochastic processes, Fokker–Planck equations, and Riemannian differential operators. It then introduces probability-flow dynamics and spectral density evolution for studying generative trajectories and multimodal distributions.

  • Euclidean processes: Euclidean diffusion models are described by SDEs whose evolving densities satisfy Fokker–Planck equations.The reverse process uses the score of the time-marginal density, approximated in practice by a neural network trained through denoising score matching.
  • Riemannian processes: On a compact Riemannian manifold, Brownian motion and density evolution are defined using local coordinates, the metric, Riemannian volume, and the Laplace–Beltrami operator.Coordinate expressions involve the metric tensor, Christoffel symbols, and local orthonormal frames.
  • Reverse dynamics: The Riemannian score is the central ingredient in the reverse diffusion process.The reverse dynamics can be represented in Stratonovich form or converted to a local-coordinate Itô SDE.
  • Probability flow: A probability-flow ODE rewrites density evolution as a continuity equation and permits trajectories to be integrated backward from the noisy distribution.This deterministic formulation makes the generative process amenable to dynamical-systems analysis.
  • Spectral evolution: On compact manifolds, the density admits a discrete spectral expansion whose nonconstant modes decay exponentially toward the uniform steady density.The slowest decay is governed by the first spectral gap, while higher frequencies decay earlier and the score norm tends to zero.
  • Mixture models: The analysis specializes to multimodal mixtures whose evolved density and score follow from linearity of the forward diffusion equation.These mixtures provide the distributions used to analyze critical points and speciation.

3. General theory of speciation

The paper frames speciation as bifurcation-driven branch formation in the critical points of evolving densities, extending pitchfork-based descriptions to intrinsic, generally nonsymmetric settings. For heat-kernel mixtures on compact manifolds, generic events are folds with one-dimensional critical kernels, while topology and geodesic geometry constrain equilibria and transition locations.

  • Speciation as critical-point bifurcation: The one-dimensional model shows that exact symmetry yields a supercritical pitchfork, whereas symmetry breaking unfolds it into a saddle-node transition.The offset parameter shifts the critical time to tc = t∗ + a^2, while a generic field leaves one primary branch plus one fold.
  • Speciation as critical-point bifurcation: Speciation occurs when new stable branches of score zeros appear through bifurcations of the evolving probability density.The score-zero set contains attractors, repellers, and saddles, and persistent branches correspond to commitment to data centers.
  • Topological constraints and geometrical modes: Poincaré-Hopf and Morse theory constrain the number and types of score equilibria, forcing geometrical modes in addition to data modes.On the sphere, a data-mode maximum is accompanied by an antipodal geometrical minimum, unlike the corresponding Euclidean strip problem.
  • Genericity and local classification: For sufficiently rich heat-kernel mixtures, generic bifurcations have a one-dimensional critical kernel and an A2 fold normal form.Under real-analyticity, the one-dimensional-kernel property holds for a full-measure set of mixture parameters.
  • Genericity and local classification: Pitchforks and simultaneous multidirectional speciations arise from exceptional symmetric configurations rather than generic parameter choices.The exceptional cases have dimkerHess_x∗ p_t∗ > 1 and form a null-measure parameter set in the analytic setting.
  • Geometry-dependent speciation estimates: Immediately after speciation, new branches depart from the critical point with square-root scaling in the time offset.The local behavior is x1 ∝ ±√(t∗ − t).
  • Geometry-dependent speciation estimates: For bimodal mixtures, critical points lie on the minimizing geodesic between centers, and equal-weight bifurcations occur uniquely at its midpoint.The reduction enables geometry-dependent position and time estimates expressed through the geodesic distance and heat-kernel expansion.

4. Stability of the generic bifurcation

Small smooth score perturbations preserve generic A2 speciation events, while the first-order timing shift depends only on score error along the one-dimensional critical direction.

  • Generic fold assumptions: A generic speciation point has a one-dimensional Hessian kernel satisfying fold nondegeneracy and time-transversality conditions.The kernel is kerH = Rv, with nonzero third derivative along v and nonzero time derivative in that direction.
  • Persistence under perturbations: Under these assumptions, the perturbed system retains a smoothly displaced A2 fold for sufficiently small perturbations.The critical point and speciation time become smooth functions x∗(ε) and t∗(ε), and the fold persists for |ε| < ε0.
  • Persistence under perturbations: The perturbed speciation location shifts by order ε in the direction of ker(H|x∗).The displacement is controlled by the critical direction rather than by arbitrary tangent directions.
  • Score approximation error: The first-order speciation-time shift is determined by the projection of the score error onto the critical direction.For a learned score bSt = St + E, the shift is expressed through the component ⟨v,E(x∗,t∗)⟩.
  • Score approximation error: Score errors orthogonal to the critical direction produce no first-order time shift, with δt∗ = o(ε).Small score errors still preserve the event with bounded shifts in time and location, although the formula is not quantitatively usable without the exact score.

5. An illustrative case: data from vMF distributions on the S2 manifold

On S2, analytic vMF mixtures exhibit geometry-dependent speciation through pitchfork and saddle-node bifurcations, with persistent geometrical modes and hierarchical multimodal transitions.

  • Bimodal distribution: For bimodal vMF mixtures, the density evolves along the great circle φ = π connecting the two centers, where data modes emerge as maxima during reverse diffusion.The equatorial critical point changes from a maximum to a separating minimum as time decreases, while θ = 0 and θ = π remain geometrical minima.
  • Bimodal distribution: 0.89 and 0.21 are the observed speciation times for α = π/6 and α = π/3, respectively, agreeing with theoretical estimates 0.99 and 0.18.The closer-center configuration transitions nearer the initial time, and the estimates are obtained from the geometry-dependent formula.
  • Bimodal distribution: At α = π/6, the equatorial attractor becomes a saddle while new maxima at θ = π/2 ± π/3 become stable score equilibria after the pitchfork.The minima at θ = π/2, φ = 0 and φ = 2π remain geometrical modes.
  • Bimodal distribution: At the bimodal critical point, one Hessian eigenvalue vanishes while the other remains negative, giving a one-dimensional critical kernel.This matches the generic fold characterization even though the symmetric example displays a pitchfork.
  • Multimodal distribution: In the trimodal mixture, the first speciation occurs near t∗1 ≈ 0.6 through a saddle-node, while the second near t∗2 ≈ 0.12 is a pitchfork separating C1 and C2.The first event makes C3 an attractor; the later event makes C1 and C2 attractors.
  • Multimodal distribution: The attractor associated with the unresolved C1-C2 cluster is a geometrical mode forced by the Poincaré–Hopf equality after C3 speciation.A saddle-node pair emerges while the original attractor persists, illustrating hierarchical multiple speciation.

6. Experiments with score approximated via neural networks

The experiments approximate Riemannian scores with neural networks trained in intrinsic charts and compare learned dynamics with analytical and theoretical bifurcation predictions. Numerical results reproduce pitchforks, their saddle-node unfolding, and hierarchical speciation on realistic fire data.

  • Numerical integration: The numerical scheme simulates manifold diffusion in local charts using Euler–Maruyama updates, periodic boundaries, or chart changes when trajectories leave a chart.The metric can be computed analytically or from an embedding map via automatic differentiation.
  • Score learning: The learned score is trained by intrinsic implicit score matching on noisy samples generated by the forward diffusion.The score network approximates S(x,t)=∇x log pt(x) without requiring explicit transition-density gradients.
  • Numerical stability: Finite-precision errors can break metric symmetry and make pullback metrics ill-conditioned because the condition number satisfies cond(g)=cond(JF)^2.Symmetrization and Tikhonov regularization address these instabilities by restoring symmetry and shifting eigenvalues away from zero.
  • Bimodal vMF distribution: For the bimodal vMF experiment, the learned score shows a speciation time near t*=0.89, after which the equatorial equilibrium becomes unstable and trajectories are attracted to the two centers.The observed transition agrees with a pitchfork bifurcation in the symmetric configuration.
  • Bifurcation perturbations: Score approximation breaks the perfect pitchfork into a saddle-node pair while preserving the original stable branch, consistent with the perturbation theory.For the second center configuration, the predicted learned transition is t*learned≈0.58 with theoretical discrepancy δt*≈0.019.

7. Conclusion

The paper formulates speciation intrinsically as bifurcation of density critical points on compact Riemannian manifolds, incorporating geometric and topological constraints. It proves generic fold behavior, perturbation stability, and illustrates these phenomena through spherical and realistic-data experiments.

  • Framework: Speciation is characterized by bifurcations of density critical points, equivalently zeros of the Riemannian score field, rather than only by symmetry breaking.This formulation retains information about the manifold’s geometry and topology.
  • Topology and geometry: Poincaré–Hopf and Morse theory constrain score critical points, distinguishing data modes from geometrical modes forced by topologically admissible configurations.Geometrical modes may be attractors or repellers and can participate in hierarchical class sifting.
  • Generic bifurcations: Generic heat-kernel mixture speciations have a one-dimensional critical kernel and an A2 fold normal form, while pitchforks and multidirectional bifurcations are nongeneric symmetric cases.The theory also derives geometry-dependent speciation-time estimates for bimodal mixtures and Riemannian regular simplices.
  • Perturbation stability: Nondegenerate folds persist under small score perturbations, with first-order speciation-time shifts determined only by score error along the critical direction.This supports applying the theory when the exact score is replaced by a learned approximation.
  • Experiments and conclusion: The experiments display folds, pitchforks, symmetry-breaking unfoldings, geometrical modes, and hierarchical multiple speciations.The authors conclude that speciation generally reflects geometrically organized score-landscape evolution rather than symmetry breaking alone.
  • Future work: Future work includes curvature effects, guidance improvements, multidirectional kernels, reflected-boundary diffusions, and computationally cheaper latent diffusion models.These directions mark the current scope of the framework.

A.2. Analytic solution of the Fokker-Planck equation on S2 for data from the vMF distribution

The appendix solves the spherical Fokker–Planck equation for vMF data using spherical harmonics and Legendre polynomials. Rotational symmetry reduces the solution to geodesic-angle dependence, while all nonconstant modes decay toward the uniform distribution.

  • Spherical coordinates: On S2, spherical coordinates have metric g=dθ^2+sin^2θ dφ^2 and volume form sinθ dθ dφ, with periodic treatment of longitude.The coordinate Fokker–Planck equation includes a chart-dependent geometric term within the Laplace–Beltrami diffusion.
  • Spectral solution: The Fokker–Planck solution expands in spherical harmonics with eigenvalues λℓ=ℓ(ℓ+1), whose first spectral gap is 2.For an axially symmetric vMF initial condition, only the m=0 harmonics contribute.
  • Rotational symmetry: Rotational invariance makes pt depend only on the geodesic angle between x and the vMF mean direction, so solving the north-pole case loses no generality.The general orientation is recovered by replacing the axial coordinate with the rotationally invariant inner product.
  • Long-time behavior: Every nonconstant spectral mode decays exponentially, and pt converges uniformly to 1/(4π) as t→∞.The explicit score is obtained by differentiating the spectral density, with the numerator beginning at ℓ=1.
  • Numerical evaluation: The numerical evaluation of the analytic density and score is delicate at small times and uses 19 terms plus the constant unless otherwise specified.High spectral truncation is needed to achieve sufficient precision in this regime.

A.3. Large κ asymptotics of vMF distribution and comparison with the heat kernel

The appendix relates concentrated vMF distributions to Gaussian approximations in geodesic coordinates. This relation connects the vMF concentration parameter to the variance used in heat-kernel speciation-time estimates.

  • Gaussian asymptotics: For large κ, a highly concentrated vMF distribution resembles a Gaussian in geodesic distance from its mean direction.The comparison is made by expanding the vMF exponential in the squared geodesic distance.

B. Proofs of propositions of Section 3.4

The proof framework uses jet bundles and singularity theory to classify degenerate critical points of evolving densities. Corank-one germs are reduced via splitting to one-dimensional normal forms, with A2 folds characterized by a nonvanishing cubic term.

  • Generic classification: Every generic heat-kernel-mixture bifurcation is an A2 fold, established using jet transversality.The fold classification is the section’s main conclusion.
  • Jet bundles: A k-jet records a function’s derivatives through order k, providing a coordinate-free replacement for Taylor expansion.The jet bundle collects these equivalence classes over the manifold.
  • Jet bundles: For Riemannian manifolds, the Levi-Civita connection supplies covariant-derivative coordinates for higher-order jets.The first jet splits canonically into function value and cotangent data, unlike higher-order jets without additional structure.
  • Singularity theory: An A_k singularity is classified by a normal form whose Hessian has a one-dimensional kernel and whose reduced direction determines the singularity type.The classical cases include A1 Morse, A2 fold, A3 cusp, and A4 swallowtail.
  • Singularity theory: Thom’s splitting lemma separates nondegenerate quadratic variables from a single reduced variable, while time unfolds the resulting spatial singularity.For an A2 fold, nondegeneracy requires a nonzero mixed time-spatial derivative.

B.2. Ampleness of heat-kernel jets

This section proves that heat-kernel mixtures generate jet data richly enough to support transversality arguments. Analyticity strengthens the conclusion from nonempty openness to full-measure genericity for suitable center configurations.

  • Jet ampleness: Heat-kernel mixtures generate every jet-bundle fiberwise at fixed positive time and manifold point.The result holds for every jet order.
  • Spectral argument: The heat-kernel eigenfunction expansion converges smoothly because elliptic estimates and Weyl’s law are dominated by exponential spectral decay.This permits continuous differential functionals to be applied term by term.
  • Spectral argument: Smooth functions are recovered in C∞ from rapidly decaying spectral coefficients, allowing the jet-spanning claim to extend from eigenfunctions to arbitrary smooth functions.The argument uses convergence of partial spectral sums and continuity of the relevant functional.
  • Center configurations: For sufficiently many centers, their second jets form a basis on a nonempty open set of configurations.The determinant-like function W is nonzero for some center choices and remains nonzero nearby.
  • Center configurations: With a real-analytic metric, the basis-forming center configurations are open, dense, and full measure because the exceptional set is a proper analytic subset.This follows from analyticity of the heat kernel and the identity theorem.

B.3. Submersion via unnormalized weights

The section converts fiberwise jet spanning into a submersion property for heat-kernel mixtures with unnormalized weights. It then formulates bifurcations through critical points and Hessian degeneracy of the evolving mixture.

  • Submersion construction: The parameter-to-jet map is a submersion when component second jets span the second-jet space.Linearity in the weights makes the differential constant and onto.
  • Weight parametrization: Working with positive unconstrained weights avoids the simplex tangent constraint, after which scale invariance transfers the conclusions to probability mixtures.Scaling preserves critical points, Hessian kernels, coranks, and A_k germ types.
  • Mixture dynamics: The evolving mixture advances all component scales simultaneously, with each component using heat-kernel scale τ_i+t.Bifurcations occur where the mixture gradient vanishes and its Hessian becomes degenerate.
  • Jet-space geometry: The critical and Hessian-degeneracy conditions intersect transversely in the jet space because they constrain independent coordinate blocks.Their intersection is therefore a smooth submanifold with the corresponding codimension.
  • Jet-space geometry: The dimension count shows that the corank-one stratum has isolated preimages in the relevant bifurcation locus.Higher-corank strata have larger codimension and do not generically contribute points there.

B.4.3. Transversality argument

The transversality argument constructs parameter sets with uniformly spanning jets and applies genericity to the bifurcation strata. It yields corank-one, discrete bifurcations that are generically A2 folds, with full-measure conclusions under analytic metrics.

  • Uniform spanning: For smooth metrics, a nonempty open parameter set provides uniform second-jet spanning across compact space-time slabs.Compactness reduces pointwise spanning to finitely many pooled center sets.
  • Transversality: Uniform spanning makes the total evaluation map a submersion, enabling parametric transversality for almost every parameter in the good set.The resulting fibers are transverse to the critical and Hessian-degeneracy strata.
  • Analytic genericity: For real-analytic metrics, the failure set has lower dimension and measure zero, so the good parameter set is open, dense, and full measure.The argument uses analytic incidence varieties and proper projection.
  • Bifurcation structure: Every generic bifurcation has a one-dimensional Hessian kernel, and no corank-two-or-higher degeneracy occurs.The corank-one proposition follows from transversality to the relevant Hessian strata.
  • Bifurcation structure: Bifurcations are discrete in space-time, with finitely many events on every compact time interval.The bifurcation locus is closed and discrete, hence compact on compact time slabs.
  • Fold classification: At corank-one points, generic nonvanishing of the reduced cubic term yields the A2 fold normal form.The conclusion follows from a transversality count in the third jet.
  • Fold classification: Generically, the reduced cubic does not vanish at bifurcations, excluding higher-order degeneracies in the reduced direction.The preimage of the additional vanishing condition is empty.

B.5.3. The surviving germ is exactly A2

For generic heat-kernel mixtures, every bifurcation reduces to a corank-one cubic singularity and is therefore exactly an A2 fold. The fold normal form classifies whether a critical-point pair is born or annihilated.

  • A2 reduction: A corank-one critical point with nonzero third derivative is smoothly equivalent to the cubic germ ξ^3, defining the A2 fold.Smooth coordinate changes reduce the surviving one-variable germ to ξ^3.
  • Genericity: For sufficiently rich heat-kernel mixtures, generic parameters make every bifurcation an A2 fold with a one-dimensional Hessian kernel and nonzero reduced cubic.The parameter set is open and positive-measure for smooth metrics, and full-measure for real-analytic metrics.
  • Bifurcation outcomes: For ε1 = +1 a pair of critical points annihilates as time increases, whereas ε1 = −1 creates a pair; when the remaining signs are negative, the mode count drops by one in the death case.The merging pair is then a local maximum and an index-(n−1) saddle.
  • Reduction: The reduction first eliminates the nondegenerate Hessian directions through the implicit function theorem, leaving a scalar bifurcation equation on the critical kernel.A smooth splitting isolates the kernel coordinate while the complementary Hessian block remains invertible.
  • Unfolding: Transversality makes the time-dependent scalar germ a versal one-parameter unfolding, which is equivalent to the fold normal form after reparametrizing time.The unfolding parameter varies nontrivially in the one-dimensional normal space.
Loading 2608.23798v1…