Beyond Unimodal Bases: Pullback Geometry for Multimodal Data

• Single-Gaussian latent models can require substantial deformation to represent multiple modes, compromising the induced geometry.A fixed mixture adds flexibility but does not by itself ensure that representations or geodesics reflect data structure.Source: §1 · p. 2
• The paper proposes a normalizing flow with an adaptive Gaussian-mixture latent distribution for data supported on mixtures of manifolds.Source: §1 · p. 2
• The proposed responsibility-weighted expected-precision metric is smooth, positive definite, tractable, and recovers the Gaussian construction in the single-component limit.For structured shared-mean mixtures, the paper also establishes conditions relating geodesics to high-likelihood regions and derives local curvature relations.Source: §1 · p. 2
• The paper jointly learns the flow and mixture, allowing the number of active components to emerge during training.Responsibilities also support component-wise reconstruction and local effective-dimension estimation.Source: §1 · p. 2
• Experiments evaluate transport distortion, path recovery, and interpolation quality on synthetic geometric data, a controlled multi-view image dataset, and MNIST.Source: §1 · p. 2

• When a flow is approximately locally isometric on the data, latent-space geometric computations can transfer back to data space with controlled distortion.Source: §2 · p. 3
• A single Gaussian latent model places the burden of representing multimodal structure on the flow, creating tension between expressiveness and approximate local isometry.Source: §2 · p. 3
• Geodesic log-concavity means log-density does not fall below the linear interpolation of its endpoint values along a geodesic, giving a precise test of high-likelihood path support.Source: §2 · p. 3
• This condition cannot hold globally for every multimodal density, particularly when modes are strongly separated, so the paper analyzes structured overlapping mixtures.Source: §2 · p. 3

3 Multimodal Pullback Geometry

• The proposed mixture metric uses the square of responsibility-weighted expected component precision, omitting a covariance term that can make the score-based metric degenerate.The resulting metric is positive definite and recovers the single-Gaussian metric in the single-component limit.Source: §3 · p. 3
• For separated components with identical covariance, the metric reduces to the prior Gaussian metric and satisfies the desiderata locally within each component.The paper notes that geodesic log-concavity cannot hold everywhere when paths between high-density regions must cross low-density regions.Source: §3 · p. 3

3.1 Star-shaped densities

• The theoretical analysis restricts mixtures to a shared origin mean and component precisions with a common eigenbasis; learned experimental mixtures need not satisfy these assumptions.Source: §3.1 · p. 4
• For Gaussian stars, the theorem gives a pointwise condition for geodesic log-concavity; nonpositive cross-coordinate precision covariance yields global log-concavity, and coordinate-axis geodesics are log-concave.Source: §3.1 · p. 4
• The same precision covariance controls geodesic log-concavity and sectional curvature at the origin.The local quadratic correction separates spatial variation of expected precision from a curvature contribution generated by cross-coordinate precision covariance.Source: §3.1 · p. 4
• For a two-dimensional, three-component low-central-covariance mixture, a large region satisfies geodesic log-concavity and a geodesic follows the arms.Source: Figure 2 · p. 4
• The star–Ricci quantity is a post-training structured curvature summary, not the exact Ricci curvature for learned mixtures with untied means.In the diagonal case, it can be computed in O(Kd) time.Source: §3.1 · pp. 4–5

4.1 Adaptive flow–mixture learning

• Alternating flow and overcomplete-mixture updates use slower, damped mixture steps and conservative merge–prune operations so active components emerge during training.A symmetric Dirichlet prior regularizes weights; low-weight components are pruned, and merges are accepted only when they improve the penalized mixture objective.Source: §4 · p. 5, §4.1 · p. 5

4.2 Training and geometric computation

• The flow objective combines likelihood and isometry regularization, with metric-smoothness regularization additionally used for MNIST and BlueWhale.A separate mean-shrinkage update encourages component overlap in the three-dimensional synthetic experiment, MNIST, and BlueWhale; it is not part of the stated objective.Source: §4.2 · p. 5
• For Wavy-X and MNIST, fixed-endpoint paths minimize a midpoint approximation of metric energy and are decoded through the inverse flow; BlueWhale instead uses a support-graph path.The continuous paths use no density, smoothness, spacing, arclength, or log-concavity penalty.Source: §4.2 · p. 5

4.3 Component-wise local reconstruction

• Responsibility-weighted data-space covariances yield component-wise linear reconstructions and local effective-dimension estimates from leading eigenvectors retained at a variance threshold.The fitted subspaces are in data space, not the latent GMM covariance spaces.Source: §4.3 · p. 6

5.1 Experimental Setup

• Wavy-X tests transport distortion and path support, BlueWhale supplies a metadata-defined reference trajectory, and MNIST assesses realism without a ground-truth path.Models share data splits, a RealNVP architecture, and training budget; fixed-m controls separate mixture-base effects from adaptive component selection.Source: §5 · p. 6, §5.1 · p. 6

5.2 Wavy-X: Transport Distortion and Path Support

• On Wavy-X, the adaptive model has the lowest NLL, isometry deviation, and upper-tail Jacobian conditioning, and best support for Path 1; fixed m = 1 best supports Path 2.It reduces 20 candidate components to two and better preserves the intersecting structures with less transport deformation; support deviation alone does not penalize folding or distinguish equally supported routes.Source: §5.2 · pp. 6–7, Table 1 · p. 7

5.3 BlueWhale: Recovery of a Data-Supported Reference Trajectory

• The support-graph path has lowest MSE, MAE, and worst-step MSE and highest mean SSIM, while Figure 3 shows alignment with the reference and blurred decoded alternatives.Because the adaptive route uses observed samples, this evaluates reference-trajectory recovery rather than continuous geodesic solvers.Source: §5.3 · pp. 7–8, Figure 3 · p. 7, Table 2 · p. 8

5.4 MNIST: Interpolation Realism and Perceptual Variation

• On MNIST, the adaptive model improves classifier-feature Fréchet distance and recognizable intermediates, at the cost of longer paths and greater perceptual step-size variation.Its component-wise analysis retains 13 active components with effective dimensions from 17 to 56.Source: §5.4 · pp. 8–9, Figure 4 · p. 8, Table 3 · p. 8

5 Experiments

• Across the experiments, the adaptive model reduces transport distortion, offers competitive path support, closely recovers the BlueWhale reference trajectory, and improves MNIST interpolation realism.The experiments use dataset-appropriate path procedures to assess complementary aspects of the geometry.Source: §5.4 · p. 9

6 Conclusion

• The adaptive Gaussian-mixture latent distribution and smooth expected-precision metric extend pullback geometry to mixtures of manifolds while recovering the single-Gaussian construction.For shared-mean Gaussian stars, the paper relates geodesic log-concavity to precision covariance and local curvature.Source: §6 · p. 9
• Across Wavy-X, BlueWhale, and MNIST, results show reduced transport distortion, competitive path support, close reference-trajectory recovery, and improved interpolation realism.The star–Ricci quantity is a post-training diagnostic; studying it as a regularizer on more complex data and using more scalable flow architectures remain future directions.Source: §6 · p. 9

AI Use Statement

• The authors used Claude for text revision, proofreading, and code documentation, and say they assume responsibility for the content.The code for experimental results is stated to be forthcoming; measurements use a fixed seed.Source: AI Use Statement · p. 9, Reproducibility statement · p. 9

A Related Work

• Prior work connects learned geometry to interpolation and manifold tasks, while emphasizing that unsupported directions can yield misleading paths and representations.Scalable pullback methods remain unimodal, while multimodal generative models do not by themselves specify a Riemannian metric.Source: A Related Work · p. 11
• The paper distinguishes its approach from methods that modify distances or mappings: it derives a positive-definite metric from a Gaussian-mixture base and analyzes density compatibility for a structured class.It retains one invertible flow rather than using multiple local maps.Source: A Related Work · p. 11

B Notation and Geometric Background

• The geometric background defines pullback metrics, curves, geodesics, curvature, and the differential notation used in later derivations.The conventions follow [5], extended to include curvature quantities.Source: B Notation and Geometric Background · pp. 11–12

C Geometric Derivations

• The derivations establish notation for mixture responsibilities and component scores as foundations for the Gaussian-mixture score-Hessian analysis.Expectations, variances, and covariances indexed by γ(z) use the discrete component distribution.Source: B Notation and Geometric Background · p. 13, C Geometric Derivations · p. 13, §C.1 · p. 13

C.2 Validity of the expected-precision metric

• For positive mixture weights and positive-definite component covariances, the expected-precision matrix square defines a smooth Riemannian metric.Its positive definiteness follows from the positive-definite responsibility-weighted expected precision.Source: §C.2 · p. 14
• With one component, the metric reduces to the single-Gaussian score-based pullback metric.The single-component expected precision is the component precision, so its matrix square gives the stated construction.Source: §C.2 · p. 14

C.3.1 Geodesic log-concavity

• For diagonalized Gaussian stars, nonpositive off-diagonal precision covariances ensure log-concavity along every geodesic, and nonconstant coordinate-axis geodesics are strictly log-concave.The coordinate axes are invariant under the geodesic flow.Source: §C.3.1 · p. 15

C.3.2 Sectional curvature; C.3.3 Local expansion of geodesic log-density

• At the centre of a Gaussian star, pairwise precision covariance determines coordinate sectional curvature and also governs the geodesic log-concavity correction.The local expansion separates expected-precision variation from a curvature contribution generated by cross-coordinate precision covariance.Source: §C.3.2 · p. 15, §C.3.3 · p. 16

C.3.4 Ricci contraction

• For shared-mean mixtures without a shared precision eigenbasis, the paper derives a Ricci contraction; diagonalized precisions allow all diagonal Ricci entries to be accumulated in O(Kd) time.The derived quantities provide a computationally tractable local curvature diagnostic.Source: §C.3.4 · p. 16

C.4 Chain-shaped densities

• For shared-covariance mixtures, the metric is constant, but geodesic log-concavity depends on the density as well as the metric.In the chain-shaped case, the direction of changing component means is the first direction where log-concavity fails as their responsibility-weighted spread increases.Source: §C.4 · p. 17

D.1 Adaptive Flow–Mixture Training

• Training alternates flow updates with mixture-structure updates, and proposed component merges are accepted only when their score improves beyond a threshold.The merge score is evaluated with the flow fixed.Source: D Training and Path-Optimization Details · p. 17, §D.1 · p. 17

D.2 Geometric Regularizers

• The stochastic local-isometry regularizer uses flow Jacobians, while a weak metric-smoothness regularizer discourages abrupt local changes in expected precision.The metric-smoothness term is used for MNIST and BlueWhale and weights nearby latent-code pairs more heavily.Source: §D.2 · p. 17

D.3 Mean-Shrinkage Update

• For selected experiments, mean shrinkage moves selected component means toward their target-weighted centre as part of the mixture update, not the flow-training objective.The selected set is either all active components or the two with the largest mixture weights.Source: §D.3 · p. 18

D.4 Metric-Based Geodesic Computation

• The latent geodesic solver minimizes only the discrete energy of the proposed metric and adds no density or auxiliary path penalties.The constant factor omitted from the energy does not change the minimizing path.Source: §D.4 · p. 19

D.5 BlueWhale Support-Graph Path Computation

• For BlueWhale, the reported density-aware, metric-weighted support-graph path stays on sampled observations and is not an exact continuous geodesic.Its graph edge costs combine proposed-metric length with a negative-log-likelihood penalty.Source: §D.5 · pp. 19–20

E.1.1 Wavy-X; E.1.2 Tetrahedral Star

• Wavy-X comprises two noisy sinusoidally perturbed strokes intersecting at 90 degrees, while the tetrahedral-star dataset has four noisy arms in three dimensions.Each dataset contains 8,000 observations.Source: §E.1.1 · p. 20, §E.1.2 · p. 20

E.1.3 BlueWhale

• BlueWhale images are rendered from a three-arm viewpoint family whose metadata define a reference trajectory through the common origin.The dataset contains 5,000 rendered 28 × 28 grayscale images.Source: §E.1.3 · p. 20

E.1.4 MNIST

• MNIST digit labels are withheld from generative-model training and used for endpoint selection, evaluation-classifier training, and visualization organization.The adaptive model starts with 30 candidate components, and paths use 16 segments.Source: §E.1.4 · p. 20

E.2 Evaluation Metrics

• The evaluation defines Wavy-X transport distortion and flow conditioning, BlueWhale reference-path errors, and MNIST feature realism and perceptual path measures.Feature Fréchet distance is the primary interior-sample realism measure; PPL and relative PDV measure perceptual length and step-size uniformity.Source: §E.2 · p. 21

E.3 Component-Wise Linear Reconstruction

• Component-wise effective dimension counts the leading data-space directions needed to reach a variance-retention threshold, using responsibility-weighted component subspaces.The oracle reconstruction diagnostic selects the component with the lowest reconstruction error for each observation and is not an inference-time assignment rule.Source: §E.3 · p. 22

F Additional Synthetic Results

• The synthetic analyses treat component-wise reconstruction as a local effective-dimension estimate, not a formal intrinsic-dimension result or evidence that all Wavy-X data are globally linear.The reported reconstruction analysis is post-training and does not optimize the flow.Source: F Additional Synthetic Results · p. 22, §F.1.1 · p. 22

F.1.1 Component-Wise Reconstruction

• On Wavy-X, one principal direction explains 99.41% and 99.29% of the weighted data-space variance for the two retained components.The components' dominant directions approximately recover the crossing arm pairs; oracle best-subspace selection gives mean MSE 1.04 × 10−2 and Q95 MSE 4.17 × 10−2.Source: Figure 5 · p. 22, §F.1.1 · p. 22

F.1.2 Layer-Wise Transport

• Figure 6 traces matched Wavy-X samples from input through intermediate flow layers to latent representations for SBPRG and three Gaussian-mixture variants.It complements numerical distortion measures by visualizing the different transport deformations.Source: §F.1.2 · p. 22, Figure 6 · p. 23

F.2 Three-Dimensional Tetrahedral-Star Experiment

• Adaptive metric paths preserve the tetrahedral star’s multi-arm organization and remain close to observed support for both selected endpoint pairs.The same endpoint pairs are re-encoded under each model before computing paths.Source: §F.2 · p. 23, Figure 7 · p. 23
• The adaptive model obtains the best NLL, isometry distortion, and upper-tail Jacobian condition number, with the lowest support deviation for Path 2; SBPRG has the lowest for Path 1.Table 4 compares the tetrahedral-star metrics, for which lower values are better.Source: §F.2 · p. 23, Table 4 · p. 24

G Additional Image-Data Analysis

• The additional analyses summarize component-wise linear structure for BlueWhale and MNIST using subspaces fitted in data space from latent responsibilities, without changing the learned flow.Figure 8 also visualizes component-wise reconstructions and layer-wise flow trajectories.Source: G Additional Image-Data Analysis · p. 24, Figure 8 · p. 24

G.1 BlueWhale

• The learned mixture retains three active components with weights 0.2760, 0.3182, and 0.4058; under the 90% weighted-variance criterion, dimensions {2, 3, 2} explain 96.65%, 91.16%, and 90.70% of respective weighted data-space variance.Although each generating arm is parameterized by one scalar, its rendered image trajectory is nonlinear in pixel space and may require more than one linear direction.Source: §G.1 · p. 24
• Oracle best-subspace reconstruction yields mean MSE 5.66 × 10−4 and Q95 MSE 1.46 × 10−3.Oracle selection chooses the lowest-error component reconstruction per sample and is a post-training diagnostic, not an inference-time assignment rule.Source: §G.1 · p. 24

G.2 MNIST

• The model retains 13 active components, with component-wise effective dimensions from 17 to 56 and a mean of 36.Figure 9 shows representative samples and oracle best-subspace reconstructions; digit labels are not used during training or subspace fitting.Source: §G.2 · p. 24, Figure 9 · p. 25
• Digit 1 has the lowest reported reconstruction MSE, 4.1 × 10−3, while digit 8 has the highest, 1.78 × 10−2.The difference is consistent with greater linear complexity of two-loop shapes, but reconstruction errors alone do not identify topology.Source: §G.2 · p. 24
• Residual blur is expected because each reconstruction uses a linear approximation in pixel space.Digit labels organize the visualization but are not used to fit the component models.Source: §G.2 · p. 25, Figure 9 · p. 25