PaperlayerView exact revision on arXiv

Source-linked AI summary

Matérn Gaussian processes on Riemannian manifolds

Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, Marc Peter Deisenroth

arXiv:2006.10160v6stat.MLcs.LG

TL;DR

Manifold Gaussian processes need kernels that remain valid beyond Euclidean spaces and methods that avoid cumbersome numerical SPDE solvers. The paper constructs Matérn and squared exponential kernels on compact Riemannian manifolds through Laplace–Beltrami spectral theory, enabling standard scalable GP methods. Its examples demonstrate the approach for dynamical-system learning and geometry-aware uncertainty visualization.

  • Problem

    Geodesic-distance kernel substitutions are not generally valid, while SPDE implementations can require hand-tuned numerical solvers and obstruct scalable or non-conjugate GP methods.

  • Method

    The paper expresses Matérn and squared exponential GP kernels and spectral objects using Laplace–Beltrami eigenvalues and eigenfunctions on compact Riemannian manifolds.

  • Results

    The resulting constructive framework computes kernels, spectral measures, and Fourier-feature approximations and supports sparse GP techniques, with demonstrations on an ideal pendulum and a dragon-shaped manifold.

  • Takeaways & Limitations

    The techniques allow practitioners to apply familiar Fourier-feature and sparse variational GP methods on compact Riemannian manifolds.

  • Takeaways & Limitations

    The framework is scoped to compact Riemannian manifolds without boundary, and kernels depend on the Riemannian metric rather than merely the manifold's diffeomorphism class.

Abstract

from arXiv · show

Gaussian processes are an effective model class for learning unknown functions, particularly in settings where accurately representing predictive uncertainty is of key importance. Motivated by applications in the physical sciences, the widely-used Matérn class of Gaussian processes has recently been generalized to model functions whose domains are Riemannian manifolds, by re-expressing said processes as solutions of stochastic partial differential equations. In this work, we propose techniques for computing the kernels of these processes on compact Riemannian manifolds via spectral theory of the Laplace-Beltrami operator in a fully constructive manner, thereby allowing them to be trained via standard scalable techniques such as inducing point methods. We also extend the generalization from the Matérn to the widely-used squared exponential Gaussian process. By allowing Riemannian Matérn Gaussian processes to be trained using well-understood techniques, our work enables their use in mini-batch, online, and non-conjugate settings, and makes them more accessible to machine learning practitioners.

1 Introduction

Gaussian processes provide uncertainty-aware models, but extending Matérn kernels to Riemannian manifolds is difficult because geodesic-distance substitutions can fail and SPDE formulations are cumbersome computationally. This work develops constructive spectral techniques that make manifold Matérn GPs compatible with standard scalable methods.

  • Gaussian processes are useful for learning unknown functions when calibrated uncertainty matters, including Bayesian optimization and reinforcement learning.
  • The Matérn family controls Gaussian-process sample-path smoothness through differentiability, while the squared exponential kernel represents infinite smoothness.
  • Generalizing Gaussian processes to manifolds supports data on spheres, cylinders, and tori but requires positive semi-definite kernels on those spaces.
  • Replacing Euclidean distance with Riemannian geodesic distance can produce ill-defined kernels in many cases.
  • SPDE-based manifold Matérn GPs are cumbersome because numerical solvers can require involved finite-element setup, limiting sparse inducing-point, mini-batch, online, and non-Gaussian-likelihood methods.
  • The paper develops constructive manifold GP tools, deriving torus kernels and spectral measures before extending kernel computation to compact Riemannian manifolds using Laplace–Beltrami harmonic analysis.

2 Gaussian processes

Gaussian processes model unknown functions with uncertainty through covariance kernels, but extending Matérn kernels from Euclidean spaces to compact Riemannian manifolds requires alternatives to geodesic-distance substitution. The paper develops constructive kernel and spectral-measure computations using periodic summation and Laplace–Beltrami eigenfunctions, enabling scalable GP methods.

  • Gaussian processes: A Gaussian process is specified by a mean function and covariance kernel, with the paper adopting zero prior mean.Posterior inference remains Gaussian under noisy observations, with mean and covariance determined by kernel matrices.
  • Matérn kernels: Matérn parameters control GP variability, dependence between nearby points, and mean-square differentiability; the squared exponential kernel is the infinite-smoothness limit.The paper treats the squared exponential kernel as a Matérn kernel with infinite smoothness.
  • Manifold kernels: Replacing Euclidean distance with Riemannian geodesic distance does not generally produce a valid positive semi-definite kernel on compact manifolds.If the geodesic squared exponential kernel is positive semi-definite for every length scale, the manifold must be isometric to Euclidean space, which excludes compact manifolds without boundary.
  • SPDE formulation: The SPDE formulation extends naturally by replacing the Euclidean Laplacian with the Beltrami Laplacian and white noise with canonical Riemannian white noise.However, its non-constructive form typically requires PDE discretization and can demand substantial accuracy tuning.
  • Constructive framework: The paper constructs Riemannian Matérn kernels and generalized spectral measures through torus periodic summation and Laplace–Beltrami eigenfunctions, supporting sparse GP training.The framework is also positioned for Fourier-feature methods and scalable training on compact Riemannian manifolds without boundary.

3 A first example: the d-dimensional torus

The torus case is handled by viewing functions as periodic functions on Euclidean space and using periodic summation to construct valid Matérn and squared exponential kernels. The resulting kernels have discrete spectral measures and Fourier-feature representations.

  • Periodic construction: The d-dimensional torus is the product manifold T^d = S^1 × ... × S^1, with each circle represented by a periodic coordinate.Periodic summation transforms functions on R^d into functions on the torus.
  • Periodic construction: Periodic summation preserves positive-definiteness, providing a direct way to construct valid kernels on T^d.The preservation follows from positivity of the Fourier transform and Bochner’s theorem.
  • Kernel validity: The torus Matérn and squared exponential kernels are covariance kernels of the corresponding Whittle Gaussian processes.The construction also establishes their connection to the SPDE definitions.
  • Geometric interpretation: The geodesic-distance generalization is only the first term of the torus periodic summation, whereas the full kernel accounts for global loops and topology.This explains why the naive construction can be locally accurate while failing globally.
  • Explicit examples: On the circle, explicit kernel and spectral-measure formulas are available, and the spectral measures are discrete because the Laplace–Beltrami spectrum is discrete.Fourier-feature approximations are given for the circle case.

4 Compact Riemannian manifolds

For compact boundaryless Riemannian manifolds, the paper replaces Euclidean harmonic analysis with Laplace–Beltrami spectral theory to compute Matérn and squared exponential kernels constructively. Eigenpairs yield finite kernel approximations and Fourier features that can be evaluated and differentiated in standard computational frameworks.

  • Spectral framework: The general-manifold framework uses Laplace–Beltrami eigenfunctions and eigenvalues to replace the group structure available in the torus case.Compactness supplies a countable spectrum and an orthonormal eigenfunction basis.
  • Spectral framework: Functional calculus applies functions to the Laplace–Beltrami operator by replacing each eigenvalue λ_n with the corresponding transformed value.This provides the operator formalism needed to define the SPDE components spectrally.
  • Kernel derivation: The SPDE solutions are Gaussian processes whose kernels equal reproducing kernels of manifold Sobolev or diffusion spaces.The resulting kernels are expressed through Laplace–Beltrami eigenvalues and eigenfunctions.
  • Computational representation: The manifold spectral measure is discrete, and top eigenvalues provide a regular finite Fourier-feature approximation.Analogous expressions simplify on manifolds such as spheres when eigenpairs are known.
  • Computational representation: Kernel computation requires obtaining eigenpairs once, then truncating the infinite spectral sums to form a pointwise-evaluable approximation.For finite smoothness, truncation convergence is polynomial; for infinite smoothness, it is exponential.

5 Illustrated Examples

The examples demonstrate geometry-aware Gaussian processes in dynamical-system prediction and posterior visualization on a complex dragon manifold. Fourier features support differentiable posterior sampling for trajectory simulation, while the dragon example shows uncertainty following geodesic rather than Euclidean structure.

  • Dynamical system prediction: The dynamical-system example places a squared exponential GP prior on the Hamiltonian of an ideal pendulum with circular configuration space.The phase space is the cotangent bundle of the circle, represented as a cylinder.
  • Dynamical system prediction: Fourier-feature posterior samples are differentiated and inserted into Hamilton’s equations to generate trajectories numerically.The basis-function representation makes the sampled posterior straightforward to differentiate.
  • Sample path visualization: The dragon experiment uses a 202490-triangle mesh and 500 numerically computed Laplace–Beltrami eigenpairs to represent a complex manifold.The model is trained from 52 observations of a function based on geodesic distance.
  • Sample path visualization: Posterior samples are defined over the entire dragon mesh using the path-wise sampling expression.The visualization includes true values, posterior mean, marginal variance, and a posterior sample.
  • Sample path visualization: Uncertainty differs substantially across the dragon’s snout despite close Euclidean proximity, reflecting the manifold’s geodesic geometry.Posterior mean and uncertainty follow the manifold shape and generally decay with geodesic distance.

6 Conclusion

The paper develops constructive spectral techniques for Matérn and squared exponential Gaussian processes on compact Riemannian manifolds and demonstrates them on pendulum dynamics and a dragon manifold. The approach supports standard scalable GP methods, including variational inference in mini-batch, online, and non-conjugate settings.

  • Contributions: The method computes kernels, spectral measures, and Fourier-feature approximations from Laplace–Beltrami eigenvalues and eigenfunctions.It constructively extends standard Gaussian-process techniques to compact Riemannian manifolds.
  • Demonstrations: The theory is demonstrated through ideal-pendulum dynamics and posterior sample visualization on a dragon manifold.The examples cover both analytically known and numerically computed eigenfunctions.
  • Practical scope: The resulting framework supports Matérn and squared exponential GPs in mini-batch, online, and non-conjugate settings using variational inference.These capabilities address limitations of previous approaches.

Broader Impact

The paper’s technical tools are intended to make Riemannian Matérn Gaussian processes easier to use for practitioners and relevant physical-science applications. Proposed applications include robotic-arm dynamics and climate-related environmental modeling.

  • The work aims to help non-experts in stochastic partial differential equations model data on spaces such as spheres and tori.The paper describes itself as purely theoretical and focuses on technical tools for Riemannian Matérn Gaussian processes.
  • The authors expect applications in model-based reinforcement learning because robotic-arm state spaces are Riemannian manifolds.They suggest the ideas could make geometric prior information easier to incorporate into models.
  • The authors also expect applications to climate science, including Gaussian-process modeling of phenomena such as sea surface temperatures.They suggest this could facilitate more accurate climate models compared with current methods.
  • The paper encourages practitioners in other disciplines to consider the effects of incorporating geometry into their models.

Sample path visualization

The dragon experiment visualizes a Matérn posterior and constructs it through Laplace–Beltrami spectral computations on a triangular mesh. Standard GP training, Fourier approximations, inducing points, and numerical trajectory tools are used across the illustrated settings.

  • Spectral construction: The dragon-manifold Matérn kernel requires Laplace–Beltrami eigenvalues and eigenfunctions computed with Galerkin FEM on a triangular mesh with K = 100179 vertices.The resulting Helmholtz problem is deterministic, second-order, and solved once rather than once per sample.
  • Spectral construction: The finite-dimensional eigenproblem is solved using standard numerical approaches after representing the eigenfunctions and basis functions as K-dimensional vectors.The discrete Laplacian has at most K eigenvalues, and the experiments use Firedrake with an Arnoldi shift-invert spectral transform.
  • Training: The spectral formula yields a Fourier approximation of the prior, after which standard GP training computes the posterior using path-wise sampling.The experiments set ν = 3/2 and Gaussian noise variance to 10^-15, while optimizing σ2 and κ by marginal-likelihood gradient descent.
  • Visualization: Figure 6 displays true values, posterior mean, marginal posterior standard deviation, and nine random posterior draws on the dragon manifold.Black dots mark training locations, while color represents function values; the palette is adjusted to cover the sample range.
  • Dynamical-system experiment: The pendulum experiment models angle and angular momentum on the cylinder S1 × R1 and observes partial derivatives of the Hamiltonian at 1000 random locations.The parameters use m = 1, g = 9.8, and l = 2; the full model uses the gradient of a GP on the Hamiltonian.
  • Dynamical-system experiment: For the cylinder kernel, an importance-sampling reparameterization makes the optimization objective differentiable with respect to κpθ.The issue arises because the spectral measure combines a discrete angle component with an absolutely continuous momentum component.
  • Dynamical-system experiment: The dynamical-system model uses 35 vector-valued inducing points, 128 random Fourier features, mini-batches of 128, and ADAM optimization until convergence.Training minimizes KL divergence between the inducing-point GP and the true posterior GP while optimizing inducing variables and model hyperparameters.
  • Dynamical-system experiment: Posterior basis-function draws are inserted into Hamilton’s equations and integrated numerically with a Störmer–Verlet method to generate trajectories.Trajectory error bars are constructed by projecting trajectories onto a tangent-plane direction and calculating 95% intervals.

Sphere

For the d-dimensional sphere, the paper derives Matérn and squared exponential kernels from Laplace–Beltrami spectral structure using spherical harmonics. Truncated series provide practical approximations despite the absence of closed forms.

  • Setup: The sphere section assumes d > 1 and treats M = S^d, while noting that the one-dimensional case requires additional care.
  • Spectral structure: Sphere Laplace–Beltrami eigenvalues are λn = n(n + d −1), and each eigenspace Hn has dimension dn and consists of degree-n spherical harmonics.
  • Kernel results: The paper derives explicit spectral formulas for both the Matérn and squared exponential kernels on S^d.These formulas are summarized in the sphere example and corresponding kernel expressions.
  • Spectral approximation: Because each eigenspace has multiple eigenfunctions, the sphere’s Fourier feature approximation uses spherical harmonics directly rather than a Gegenbauer-polynomial simplification.The functions fn,k form an orthonormal basis of Hn.
  • Approximation boundary: No closed-form expressions are known for kν and k∞, so the paper approximates their defining series by truncation with controllable practical error.Larger ν produces faster series convergence and more accurate approximations.

C Proof of Proposition 2

The proof establishes that the periodic-summation formulas on the torus coincide with the Whittle Matérn and squared exponential kernels. It does so by identifying the torus Laplace–Beltrami basis and applying Fourier and Poisson-summation identities.

  • Proposition: Proposition 2 states that the Matérn and squared exponential kernels in the torus formulas are covariance kernels of the corresponding Gaussian processes.
  • Conclusion: The proof concludes that the periodic-summation kernels equal the Whittle kernels on the torus.
  • Torus basis: For the flat torus T^d, Laplace–Beltrami eigenfunctions are periodic sines, cosines, and complex exponentials inherited from the Euclidean Laplacian.The proof retains only 1-periodic Euclidean eigenfunctions.
  • Torus basis: The torus eigenvalues satisfy λτ = 4π2|τ|2, with τ and −τ generating the same eigenfunction pair up to sign.Together with the constant eigenfunction, these functions form an orthonormal basis of L2(T^d).
  • Proof mechanism: The proof rewrites periodic Fourier terms using cosine and sine identities before applying the generalized Poisson summation formula.The resulting spectral density is that of the Matérn kernel on R^d.
  • Proof mechanism: The same argument is applied to the squared exponential case, completing the proof for both kernel families.

D Theory: compact Riemannian manifolds without boundary

The theory formulates manifold Gaussian processes through generalized Gaussian fields and SPDEs, then uses Laplace–Beltrami spectral theory to construct their covariance kernels. Sobolev and diffusion spaces provide the functional setting linking SPDE solutions to reproducing kernels and Gaussian processes.

  • Spectral construction: The Laplace–Beltrami operator on a compact boundaryless manifold has an orthonormal eigenbasis, enabling spectral diagonalization and functional calculus.Its eigenvalues satisfy 0 = λ_0 < λ_1 ≤ ... and diverge to infinity.
  • Generalized Gaussian fields: Generalized Gaussian fields are defined over Hilbert spaces through Gaussian linear functionals with covariance operators, with white noise corresponding to the identity covariance operator.White noise can also be viewed as an isometric embedding into square-integrable random variables.
  • SPDE formulation: For an invertible operator L, the SPDE solution is F = L^-1W, whose covariance operator is L^-2 and whose integral kernel becomes the Gaussian process covariance function.This identifies the SPDE formulation with a reproducing-kernel construction.
  • Function spaces: Sobolev and diffusion spaces are chosen so the relevant operators are bounded and invertible, allowing SPDE solutions to be constructed as generalized Gaussian fields.The Sobolev spaces serve the Matérn-type operator, while diffusion spaces serve the squared exponential-type operator.
  • From fields to processes: Because the relevant Sobolev spaces are reproducing kernel Hilbert spaces, evaluating the generalized field at kernel sections yields an ordinary Gaussian process with that reproducing kernel as covariance.The Sobolev construction requires s > d/2, with s = ν + d/2 and ν > 0.
  • Kernel normalization: The resulting kernels can be normalized to average variance σ2, while pointwise variance is constant in cases such as homogeneous manifolds.Whether homogeneous manifolds are the only manifolds with constant k(x,x) is left as an open question.
Loading 2006.10160v6…