Source-linked AI summary
Geodesic convolutional neural networks on Riemannian manifolds
Jonathan Masci, Davide Boscaini, Michael M. Bronstein, Pierre Vandergheynst
TL;DR
Shape analysis needs descriptors that support correspondence, retrieval, and segmentation while handling non-Euclidean geometry. GCNN addresses this by applying learned convolutional layers to local geodesic patches, and the paper reports state-of-the-art performance across shape description, retrieval, and correspondence.
Problem
Intrinsic CNNs are difficult to formulate for deformable shapes because Riemannian manifolds lack ordinary shift invariance.
Source: §1 · p. 1Method
GCNN extracts local patches with geodesic polar coordinates and processes them through learned convolutional, linear, and nonlinear layers.
Source: §1 · p. 1, §4.1 · p. 3Results
GCNN achieves state-of-the-art performance across shape description, retrieval, and correspondence.
Source: §1 · p. 2, §7.1 · p. 6, §7.2 · p. 6Takeaways & Limitations
GCNN provides a generalizable, local, anisotropic extension of CNNs for learning invariant shape features on non-Euclidean manifolds.
Source: §1 · p. 2, §5 · p. 4The geodesic-disc construction requires a sufficiently small radius relative to the manifold’s local convexity radius.
Source: §4.1 · p. 3
Abstract
from arXiv · showhide
Feature descriptors play a crucial role in a wide range of geometry analysis and processing applications, including shape correspondence, retrieval, and segmentation. In this paper, we introduce Geodesic Convolutional Neural Networks (GCNN), a generalization of the convolutional networks (CNN) paradigm to non-Euclidean manifolds. Our construction is based on a local geodesic system of polar coordinates to extract "patches", which are then passed through a cascade of filters and linear and non-linear operators. The coefficients of the filters and linear combination weights are optimization variables that are learned to minimize a task-specific cost function. We use GCNN to learn invariant shape features, allowing to achieve state-of-the-art performance in problems such as shape description, retrieval, and correspondence.
1. Introduction
Shape descriptors support correspondence, retrieval, and segmentation, but deformable shapes make intrinsic CNNs difficult because manifolds lack ordinary shift invariance. The paper proposes GCNN, using local geodesic coordinates as image-like patches, and reports state-of-the-art performance across shape description, retrieval, and correspondence.
Motivation
- Shape descriptors are expected to be discriminative, robust, compact, and computationally efficient for application-dependent analysis tasks.Source: §1 · p. 1
- Intrinsic CNNs are difficult to formulate because Riemannian manifolds lack the shift invariance available in Euclidean image grids.Source: §1 · p. 1
Contribution
- GCNN extends CNNs to non-Euclidean manifolds by extracting local geodesic-coordinate patches and learning task-specific features.The construction addresses the difficulty of formulating intrinsic CNNs on deformable shapes.Source: §1 · p. 1
- GCNN generalizes previous descriptors because HKS, WKS, optimal spectral descriptors, and intrinsic shape context arise as particular configurations.Source: §1 · p. 2
Applications
- The model is designed to learn invariant shape features for shape description, retrieval, and correspondence.Source: §1 · p. 2
2. Background
The paper models shapes as Riemannian surfaces and uses the intrinsic Laplace–Beltrami operator to obtain geometry-aware spectral representations. Its discrete implementation samples surfaces as triangular meshes and computes eigenfunctions through a generalized eigendecomposition.
Manifold model
- A 3D shape is modeled as a connected smooth compact two-dimensional manifold with a Riemannian metric on each tangent plane.Source: §2 · p. 2
Laplace–Beltrami operator
- The intrinsic Laplace–Beltrami operator is invariant to isometric deformations and has eigenfunctions forming an orthonormal basis analogous to the Fourier basis.Source: §2 · p. 2
Heat diffusion
- Heat diffusion uses the manifold’s Laplace–Beltrami spectrum, with heat-kernel coefficients e^-tλ acting as a low-pass transfer function.Source: §2 · p. 2
Discretization
3. Spectral descriptors
Spectral descriptors apply banks of transfer functions to Laplace–Beltrami eigenvalues, producing dense intrinsic features from a compact spectral representation. HKS, WKS, and learned optimal spectral descriptors instantiate this framework with different filters and trade-offs.
Generic spectral descriptors
- A generic spectral descriptor applies a bank of transfer functions to Laplace–Beltrami eigenvalues, yielding dense intrinsic features efficiently from a small number of eigenpairs.Source: §3 · p. 2
HKS
- HKS uses low-pass filters and has an autodiffusion interpretation, but its low-pass construction provides poor spatial localization.Source: §3 · p. 2
WKS
- WKS uses band-pass filters with better localization, but its matches tend to be noisier.Source: §3 · p. 3
Optimal spectral descriptors
- Optimal spectral descriptors parameterize transfer functions in a B-spline basis and learn their coefficients by minimizing a task-specific loss.Source: §3 · p. 3
4. Convolutional neural networks on manifolds
GCNN extends convolution to manifolds by extracting local geodesic-polar patches and processing them with learned and fixed neural-network layers. Its architecture addresses angular-coordinate ambiguity through rotated filters and rotation-invariant pooling or Fourier magnitudes.
4.1. Geodesic convolution
- Local geodesic polar coordinates extract manifold patches using radial distance levels and angular geodesics, with sufficiently small discs producing valid topological patches.The angular origin is arbitrary, and boundary points use half-discs.Source: §4.1 · p. 3
- On triangular meshes, the discrete patch operator partitions each vertex’s 1-ring into Nθ angular bins, unfolds rays across triangles, and forms radial bins from fast-marching geodesic distances.Source: §4.1 · p. 3, Figure 2 · p. 4
4.2. Geodesic Convolutional Neural Networks
- GCNN stacks geodesic-convolution, linear, nonlinear, pooling, Fourier-magnitude, and covariance layers to transform pointwise inputs into learned descriptors or global shape representations.Linear layers adjust dimensions, while covariance aggregates pointwise descriptors for retrieval.Source: §4.2 · p. 4
- Geodesic-convolution layers evaluate filters over all Nθ rotations, while angular max-pooling retains the maximum response across rotations.Fourier-transform magnitude provides another route for removing rotational phase ambiguity.Source: §4.2 · p. 4
5. Comparison to previous approaches
GCNN uses local geodesic patches to generalize CNNs across manifolds while capturing anisotropic structure. It also subsumes several established spectral and intrinsic descriptors as special configurations.
5. Comparison to previous approaches
- Local geodesic patches capture anisotropic structures, and the method generalizes across shapes without relying on approximate cross-shape invariance of Laplacian eigenfunctions.This contrasts with spectral formulations whose coefficients may be domain-dependent or sensitive to non-isometric deformation.Source: §5 · pp. 4–5
- HKS, WKS, OSD, and intrinsic shape context arise as fixed or learnable GCNN configurations using linear and Fourier-transform-magnitude layers.Source: §5 · p. 4
- GCNN1 applies one convolutional layer to M = 150-dimensional geometry vectors and produces a Q = 16-dimensional feature descriptor.Source: Figure 3 · p. 5
6. Applications
GCNN is modeled as a nonlinear hierarchical parametric function whose parameters are learned with task-specific losses. The paper applies this framework to invariant descriptors, correspondence, and retrieval using positive/negative or labeled training data.
6. Applications
- GCNN maps input features at all mesh points through a nonlinear hierarchical function ψΘ(F), with parameters learned by minimizing application-specific losses.Inputs may include HKS, WKS, geometry vectors, or other features.Source: §6 · p. 5
- For invariant descriptors, a siamese GCNN makes corresponding points similar and non-corresponding points dissimilar using positive and negative vertex pairs.Source: §6 · p. 5
- For shape correspondence, GCNN produces a soft probability distribution over reference-shape vertices and trains it with known correspondences using multinomial regression.Source: §6 · p. 5
- For shape retrieval, a covariance layer aggregates local features into a global descriptor trained to distinguish shapes from the same class from shapes in different classes.Source: §6 · p. 6
7. Results
GCNN descriptors were evaluated for intrinsic description, correspondence, and pose-invariant retrieval on FAUST, with transfer testing on TOSCA. Across these tasks, GCNN showed strong robustness, generalization, and performance relative to standard descriptors and prior methods.
7. Results
- The results used FAUST and TOSCA meshes with geodesic patches at 1% of geodesic diameter, 150-dimensional inputs, and up to 2.5K optimization updates.GCNN1 and GCNN2 trained in approximately 30 and 50 minutes, while descriptor computation reached 75K and 45K vertices per second, respectively.Source: §7 · p. 6
7.1. Intrinsic shape descriptors
- GCNN descriptors provided better localization than HKS, fewer spurious minima than WKS and OSD, and robustness to deformation, noise, resampling, and missing parts.These properties were assessed through descriptor-space distance maps over transformed shapes.Source: §7.1 · p. 6, Figure 4 · p. 7
- GCNN significantly outperformed other descriptors on FAUST across CMC, ROC, and Princeton evaluations, while GCNN2 further improved performance.The evaluation used disjoint training and test sets, and the more complex GCNN2 configuration boosted results over GCNN1.Source: §7.1 · p. 6
- The FAUST-trained GCNN transferred well to the TOSCA dataset, demonstrating learned-descriptor generalization beyond the training dataset.The transfer experiment applied models learned on FAUST to TOSCA shapes with near-isometric deformations.Source: §7.1 · p. 6
7.2. Shape correspondence
- On FAUST correspondence, GCNN significantly outperformed previous methods under the Princeton benchmark.GCNN3 replaced the random forest in the reproduced experiment and generated soft correspondences converted by maximum response.Source: §7.2 · p. 6
7.3. Shape retrieval
- For pose-invariant FAUST retrieval, GCNN significantly outperformed HKS, WKS, and OSD aggregated into global covariance descriptors.The experiment used precision-recall evaluation with five training poses per subject and leave-one-out testing on the remaining shapes.Source: §7.3 · p. 7
8. Conclusions
The paper presents GCNN as a flexible generalization of CNNs for learning hierarchical, task-specific features on non-Euclidean manifolds. The framework supports correspondence and retrieval and can extend to other shape representations and manifold functions.
8. Conclusions
- GCNN generalizes CNNs to learn hierarchical task-specific features on non-Euclidean manifolds for applications including shape correspondence and retrieval.The architecture can be made arbitrarily complex by stacking multiple layers.Source: §8 · p. 7
- GCNN can be adapted to point clouds by modifying local geodesic charting and can operate on any function defined on the manifold, including textured-surface data.The paper used intrinsic spectral properties as inputs but describes the framework more generally.Source: §8 · p. 7