PaperlayerView exact revision on arXiv

Source-linked AI summary

Gaussian Process Morphable Models

Marcel Lüthi, Christoph Jud, Thomas Gerig, Thomas Vetter

arXiv:1603.07254v1cs.CV

TL;DR

Statistical shape models restrict variation to patterns learned from example data, motivating a more flexible and continuous shape-prior framework. The paper introduces GPMMs, approximates Gaussian processes with leading Karhunen–Loève components, and applies them to registration and segmentation. Experiments show that changing the GPMM prior adapts one registration method to varied applications, while accuracy depends on effective low-rank approximation.

  • Problem

    Statistical shape models restrict shape variation to the span of example data, while registration requires priors that can express broader deformation patterns.

  • Method

    GPMMs model deformations as Gaussian processes, represent them with leading Karhunen–Loève components computed using a Nyström approximation, and fit them to surfaces or images.

  • Results

    GPMMs support varied registration applications by changing the prior model while retaining the same registration method, including surface registration, image registration, and Active Shape Model fitting.

  • Takeaways & Limitations

    Separating deformation modeling from optimization allows learned, analytical, combined, multi-scale, spatially varying, and landmark-informed priors to be used within one framework.

  • Takeaways & Limitations

    The approach becomes inefficient and inaccurate when very fine deformations must be modeled over a large domain, and it depends on accurate low-rank approximations.

Abstract

from arXiv · show

Statistical shape models (SSMs) represent a class of shapes as a normal distribution of point variations, whose parameters are estimated from example shapes. Principal component analysis (PCA) is applied to obtain a low-dimensional representation of the shape variation in terms of the leading principal components. In this paper, we propose a generalization of SSMs, called Gaussian Process Morphable Models (GPMMs). We model the shape variations with a Gaussian process, which we represent using the leading components of its Karhunen-Loeve expansion. To compute the expansion, we make use of an approximation scheme based on the Nystrom method. The resulting model can be seen as a continuous analogon of an SSM. However, while for SSMs the shape variation is restricted to the span of the example data, with GPMMs we can define the shape variation using any Gaussian process. For example, we can build shape models that correspond to classical spline models, and thus do not require any example data. Furthermore, Gaussian processes make it possible to combine different models. For example, an SSM can be extended with a spline model, to obtain a model that incorporates learned shape characteristics, but is flexible enough to explain shapes that cannot be represented by the SSM. We introduce a simple algorithm for fitting a GPMM to a surface or image. This results in a non-rigid registration approach, whose regularization properties are defined by a GPMM. We show how we can obtain different registration schemes,including methods for multi-scale, spatially-varying or hybrid registration, by constructing an appropriate GPMM. As our approach strictly separates modelling from the fitting process, this is all achieved without changes to the fitting algorithm. We show the applicability and versatility of GPMMs on a clinical use case, where the goal is the model-based segmentation of 3D forearm images.

1 Introduction

The paper introduces Gaussian Process Morphable Models as a continuous generalization of statistical shape models, using Gaussian processes to define flexible shape variation and registration priors. It develops a low-dimensional approximation and demonstrates applications across shape modeling, registration, and segmentation.

  • Introduction: Gaussian Process Morphable Models generalize statistical shape models by representing shapes as deformations from a reference shape.The deformation is modeled as a Gaussian process over a domain containing the reference shape.
  • Introduction: The model uses leading Karhunen–Loève basis functions to obtain a practical low-dimensional parametric representation.Smooth deformations can often be approximated with only a few components, while the Nyström method computes the leading eigenfunction/eigenvalue pairs.
  • Introduction: Flexible covariance functions let GPMMs define spline, radial-basis, multi-scale, spatially varying, landmark-constrained, and hybrid shape priors.Different models can be combined without changing the fitting algorithm because modeling is separated from fitting.
  • Introduction: GPMMs unify learned shape modeling and registration by extending statistical shape models while providing Gaussian-process priors for surface and image registration.The paper positions this approach alongside related work on flexible shape models and Gaussian-process registration.
  • Introduction: The paper extends earlier conference work with improved numerical presentation, approximation analysis, kernel combinations, and broader validation.The validation includes surface and image registration and Active Shape Model fitting on forearm data.

2 Gaussian Process Morphable Models

GPMMs generalize PCA-based statistical shape models by defining shape deformations as Gaussian processes and approximating them with a low-dimensional Karhunen–Loève representation. Unlike SSMs, they permit arbitrary covariance functions, while Nyström-based computation introduces approximation error.

  • PCA-based statistical shape models: PCA-based SSMs learn normally distributed shape variation from corresponding landmark vectors and obtain a compact representation through covariance-matrix PCA.The covariance has rank at most equal to the number of example datasets, enabling an eigenvector-based parameterization.
  • Gaussian Process Morphable Models: GPMMs model deformations directly as Gaussian processes over a reference domain, making the representation independent of the shape discretization.A sampled deformation warps the reference shape into a new shape.
  • Gaussian Process Morphable Models: The Karhunen–Loève expansion yields a finite-dimensional model by retaining leading basis functions whose eigenvalues explain the deformation variance.When eigenvalues decay quickly, few components can provide a practical approximation; the expected truncation error is the eigenvalue tail sum.
  • Gaussian Process Morphable Models: Any valid positive-definite covariance function can define a GPMM, enabling expressive priors even with little or no example data.This removes the restriction that shape variation must lie in the span of training examples.
  • Numerical approximation: The Nyström method numerically estimates eigenfunction/eigenvalue pairs from sampled points, with random SVD used to scale the computation.Approximation quality improves with more sampled points, while random-SVD error is reported as negligible relative to low-rank and Nyström errors.
  • Accuracy of the low-rank approximation: Low-rank GPMM accuracy depends on both how well leading basis functions represent the process and how accurately Nyström computes its eigenpairs.Strong pointwise correlation supports compact representations, whereas independently moving points may require one basis function per point.

3 Modeling with kernels

Gaussian-process kernels provide flexible priors for shape variation and registration. Smoothness, locality, and anatomical plausibility can be controlled through generic or data-learned covariance models.

  • Modeling with kernels: Gaussian-process modeling offers a broad language for constructing deformation priors for model-based image analysis and surface or image registration.The section focuses on kernels useful for these registration applications.
  • Visualization: Sampled deformations are visualized on a 3D face surface to assess how different models affect anatomical validity.The face is the average face from the Basel Face Model.
  • Models of smooth deformations: A zero-mean Gaussian process assumes the reference surface is representative of the modeled shape class and uses a Gaussian kernel to enforce smooth deformations.The zero-mean assumption is typical in registration tasks.
  • Models of smooth deformations: Larger Gaussian-kernel bandwidth σ produces smoother, more global deformations, while smaller σ produces more local deformations.The scale factor s controls deformation variance, and the identity matrix makes vector components independent.
  • Models of smooth deformations: Spline, radial-basis, and Matérn kernels provide alternative smoothness priors, with Matérn kernels allowing explicit control of differentiability.Examples include Elastic-Body Splines, B-Splines, and Thin Plate Splines.
  • Statistical shape models: A sample covariance kernel learned from 200 training faces produces random samples that represent anatomically plausible faces.This contrasts with generic smoothness kernels, which do not encode prior shape knowledge.

3.2 Statistical shape models

Statistical shape models can be expressed as Gaussian processes whose covariance is learned from example deformations. The resulting sample covariance model is a continuous analogue of a PCA-based shape model and preserves anatomical plausibility in sampled faces.

  • Statistical shape models: Sample covariance kernels estimate characteristic deformation variation from corresponding training surfaces and define a Gaussian process over those deformations.The deformations map points on a reference surface to corresponding points on example surfaces.
  • Statistical shape models: A sample covariance kernel learned from 200 Basel Face Model surfaces generates random samples that represent anatomically plausible faces.The resulting model is described as a continuous analogue of a PCA-based shape model.

3.3 Combining kernels

Combining positive-definite kernels lets GPMMs encode multiscale, localized, spatially varying, and hybrid deformation priors. These combinations extend statistical shape models while retaining flexible control over correlations.

  • Multiscale models: Positive-definite kernels can be linearly combined to model deformations across multiple spatial scales.Summing smooth kernels with more local kernels provides a simple multiscale construction.
  • Multiscale models: A multiscale kernel produces samples with both large overall-shape deformations and local detailed variations.The construction uses Gaussian kernels with decreasing scale and bandwidth.
  • Bias models: An explicit spatially correlated Gaussian bias model can extend a sample covariance model to address errors caused by limited training examples.The bias model assumes neighboring points are likely to share errors when the statistical model cannot explain local structure.
  • Localized models: Multiplying kernels reduces long-range correlations, yielding localized models that are more flexible than the original statistical shape model.The localized samples retain locally valid face variations but lose global correlations and may not remain anatomically valid globally.
  • Spatially varying models: Summing localized models over partitioned regions produces non-stationary kernels for spatially varying deformation priors.Different regions can be assigned kernels with desired characteristics and blended using spatial weight functions.
  • Spatially varying models: A spatially varying face model can combine a statistical kernel in the upper face with arbitrary smooth deformations below the nose.This demonstrates region-specific modeling through different kernels.

3.4 Posterior models

Known landmark correspondences can be incorporated into a GPMM through Gaussian process regression. The resulting posterior process is a data-dependent GPMM that enforces the specified landmark deformations.

  • Landmark conditioning: Landmark pairs on reference and target shapes provide known deformation values at corresponding points.These constraints are used as observations for Gaussian process regression.
  • Posterior construction: Gaussian process regression transforms a prior GP(µ, k) into a posterior model GP(µp, kp) under noisy deformation observations.The posterior mean and covariance are available in closed form.
  • Posterior construction: The posterior remains a Gaussian process and therefore constitutes a data-dependent GPMM usable wherever a standard GPMM is used.Random samples can preserve specified landmark constraints by fixing the corresponding deformations.

4 Registration using Gaussian Process Morphable Models

GPMMs provide deformation priors for fitting reference surfaces or images to targets through a MAP-based registration formulation. Separating model construction from fitting enables multiple registration variants without changing the fitting algorithm.

  • Registration formulation: Registration seeks a deformation field that maps a reference object to a corresponding target object.The framework assumes corresponding points can be identified through an admissible deformation.
  • Registration formulation: The method formulates registration as a MAP estimate combining a Gaussian process deformation prior with an object-similarity likelihood.The likelihood uses a similarity metric D and weighting parameter η.
  • Registration formulation: The low-rank GPMM representation converts the registration problem into a finite-dimensional parametric optimization over Gaussian-process basis functions.The eigenfunctions determine how the prior assumptions enter the final registration.
  • Model–fitting separation: Modeling, approximation, and fitting are separated, allowing prior assumptions to be designed independently of the registration search algorithm.The paper uses the same simple fitting approach for surface and image fitting.
  • Surface and image fitting: Surface registration uses a reference-to-target closest-point distance, while image registration can use an intensity-based similarity measure.Surface Nyström sampling need only cover the reference surface; image models require sampling the full image domain.
  • Extensions: GPMMs can support PCA-based shape-model algorithms and hybrid registration by discretizing basis functions or conditioning models on landmarks.The hybrid scheme combines landmark information with shape or intensity information.

5 Results

The forearm experiments build application-specific GPMMs from analytically defined kernels and evaluate them for surface registration, Active Shape Model fitting, and image registration. Model flexibility improves fitting, while anisotropic and application-targeted models provide robustness advantages over simpler or generic alternatives.

  • Experimental setup: The study uses 36 segmented right-forearm images, extracts ulna surfaces, and defines four proximal and distal landmarks for rigid alignment.Twenty-seven images also include the original CT data.
  • Building prior models: Three ulna priors model smooth deformation using isotropic, multiscale, and anisotropic covariance functions.The anisotropic model amplifies deformation along the bone’s principal length axis.
  • Building prior models: The anisotropic model is more specific and compact than the other models, indicating a better-targeted prior for practical applications.Specificity and compactness are evaluated using both the first component and the full model.
  • Surface registration: Multiscale and anisotropic models achieve similar generalization, and both outperform the simple Gaussian model.The anisotropic model reaches comparable accuracy with substantially greater compactness than the multiscale model.
  • Generalized Active Shape Model fitting: Extended and localized GPMM priors improve Active Shape Model fitting over the standard model, but added flexibility increases outliers without landmark constraints.A posterior model incorporating proximal and distal landmarks prevents excessive movement and produces clearly improved results.
  • Image to image registration: Application-targeted GPMMs are more robust but less accurate on average than multiresolution B-spline registration, which uses 37,926 versus 500 parameters.The comparison covers single-level and multilevel B-spline registration against the best GPMM model.

6 Conclusion

The paper presents GPMMs as a continuous, kernel-defined generalization of PCA-based statistical shape models. Experiments and implementation demonstrate their use for registration and segmentation, while the method’s main scope boundary is the efficiency of low-rank approximation for fine deformations over large domains.

  • Conclusion: GPMMs generalize PCA-based statistical shape models through continuous Gaussian-process representations and arbitrary positive definite kernels.They can be discretized into the mathematical form required by standard shape-model algorithms.
  • Conclusion: GPMMs can encode learned shape characteristics, spline-like priors, and other complex shape models even when few training examples are available.The model’s covariance structure is specified through the chosen kernel rather than only estimated from example datasets.
  • Conclusion: The approach is implemented in the open-source statismo and scalismo frameworks for use with standard shape-model algorithms.The discretized GPMM can be brought into the same mathematical form as an SSM.
  • Conclusion: The experiments support GPMMs for prior shape modeling in registration, with the same fitting methods usable across different deformation priors.The approach separates prior modeling from optimization and supports direct landmark integration.
  • Conclusion: GPMMs become limited when very fine deformations must be modeled across a large domain because the approximation becomes inefficient and inaccurate.The authors suggest hierarchical multiresolution approximation over smaller subdomains as future work.

A Accuracy of the low-rank approximation

The low-rank approximation depends on truncating the Gaussian process to leading eigenfunctions and on accurately computing those eigenfunctions with the Nyström method.

  • Accuracy of the low-rank approximation: The method’s success depends on how well a truncated Karhunen–Loève expansion approximates the full Gaussian process.The approximation uses only a moderate number of leading basis functions.
  • Accuracy of the low-rank approximation: Approximation error comes from both retaining only the leading eigenfunctions and computing those eigenfunctions with finite accuracy using Nyström approximation.Both sources contribute to the final approximation error.

A.1 An analytic example

The Gaussian-kernel example shows how kernel bandwidth and domain support shape the eigenfunctions and eigenvalue decay, determining the number of basis functions needed for approximation.

  • Larger Gaussian-kernel bandwidth σ produces lower-frequency leading eigenfunctions and faster eigenvalue decay.
  • Larger domain support, represented by variance s2, also accelerates eigenvalue decay.
  • The ratio between domain support and kernel smoothness determines how many basis functions are needed for an accurate approximation.
  • The leading eigenfunctions are global even when the Gaussian kernels are highly localized.
  • For smoothness-enforcing kernels, stronger smoothness lets a few basis functions capture more of the variance because domain points vary together.

A.2 Approximation accuracy of the eigenfunction computations

The Nyström approximation provides convergence guarantees for eigenvalue and eigenfunction estimates, while kernel-induced smoothness and eigenvalue decay govern approximation quality.

  • Analytic eigenfunctions are generally unavailable, motivating numerical procedures such as the Nyström approximation.
  • Nyström approximation bounds hold for arbitrary sampling distributions, so convergence is guaranteed when sufficiently many points are used.
  • The eigenvalue error bound is independent of kernel smoothness and domain support for a fixed number of samples.
  • Projection error is mainly influenced by the decay of successive eigenvalues, with smoother kernels requiring fewer approximation points.
  • With n = 200, the eigenvalues are extremely well approximated in the tested cases σ = 1 and σ = 0.2.

A.3 Choosing the approximation parameters

The proposed parameter-selection procedure chooses the model, retained variance, basis size, and Nyström sampling density in sequence, then validates the low-rank approximation empirically.

  • The procedure begins by defining a Gaussian process GP(µ, k) that represents prior knowledge.
  • The desired retained fraction of total variance determines how many leading eigenfunctions r are needed.
  • The Nyström sample count is chosen so the eigenfunction at the smallest eigenvalue gap has a stable approximation.
  • Total variance can be estimated by numerical integration, or directly from k(x, x) for stationary kernels.
  • Low-rank accuracy is tested by sampling from N(µ, K) at discrete points and measuring reconstruction with GP(µ, k̃).
Loading 1603.07254v1…