Learning shape correspondence with anisotropic convolutional neural networks

Davide Boscaini; Jonathan Masci; Emanuele Rodolà; Michael M. Bronstein

Source-linked AI summary

Learning shape correspondence with anisotropic convolutional neural networks

Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M. Bronstein

arXiv:1605.06437v1May 20, 2016cs.CV

TL;DR

Shape correspondence remains difficult for non-rigid shapes with missing parts and topological noise, while Euclidean representations are not deformation-invariant. ACNN generalizes convolution to non-Euclidean domains using oriented anisotropic diffusion kernels and learns intrinsic dense correspondences, outperforming prior intrinsic CNNs and other state-of-the-art methods on challenging benchmarks.

Problem
Dense correspondence is difficult for non-rigid shapes with missing parts and topological noise, while Euclidean representations can lose geometry and are not deformation-invariant.
Method
ACNN generalizes convolution to non-Euclidean domains using anisotropic heat kernels that create local intrinsic representations, with network parameters learned through task-specific optimization.
Results
ACNN outperforms previously proposed intrinsic CNN models and additional state-of-the-art methods on challenging shape-correspondence benchmarks.
Takeaways & Limitations
The generic ACNN framework supports intrinsic dense correspondence learning and can be used for other applications on geometric data.
Takeaways & Limitations
LSCNN requires explicitly producing each spectral window, resulting in high memory and computational requirements, while SCNN filters lack cross-domain generalizability and guaranteed spatial localization.

Abstract

from arXiv · show

Establishing correspondence between shapes is a fundamental problem in geometry processing, arising in a wide variety of applications. The problem is especially difficult in the setting of non-isometric deformations, as well as in the presence of topological noise and missing parts, mainly due to the limited capability to model such deformations axiomatically. Several recent works showed that invariance to complex shape transformations can be learned from examples. In this paper, we introduce an intrinsic convolutional neural network architecture based on anisotropic diffusion kernels, which we term Anisotropic Convolutional Neural Network (ACNN). In our construction, we generalize convolutions to non-Euclidean domains by constructing a set of oriented anisotropic diffusion kernels, creating in this way a local intrinsic polar representation of the data (`patch'), which is then correlated with a filter. Several cascades of such filters, linear, and non-linear operators are stacked to form a deep neural network whose parameters are learned by minimizing a task-specific cost. We use ACNNs to effectively learn intrinsic dense correspondences between deformable shapes in very challenging settings, achieving state-of-the-art results on some of the most difficult recent correspondence benchmarks.

1. Introduction

Shape correspondence is a fundamental geometry-processing problem made difficult by non-rigid transformations, acquisition artifacts, and limitations of existing extrinsic and intrinsic learning approaches. The paper introduces ACNN as a generic intrinsic CNN framework for challenging dense correspondence.

Dense intrinsic correspondence targets non-rigid shapes affected by missing parts, geometric and topological noise, and differing representations such as meshes and point clouds.
Intrinsic correspondence supports applications across geometry processing, computer graphics, and vision, including texture mapping and animation.
Extrinsic deep learning: Extrinsic approaches treat geometric data as Euclidean structures, which can lose fine details, break topology, and vary under pose or deformation.Volumetric CNNs used a 64×64×64 cube, allowing only a coarse representation of 3D geometry.
Intrinsic deep learning: Intrinsic learning generalizes convolutional ingredients to non-Euclidean domains, while prior intrinsic methods expose representation, topology, orientation, or computational limitations.GCNN is limited to meshes and lacks guaranteed topological meaning; LSCNN lacks oriented structures and requires explicitly producing each window.
ACNN: ACNN generalizes CNNs to non-Euclidean domains and combines properties of prior approaches while avoiding their stated drawbacks.The framework uses anisotropic heat kernels for local intrinsic representations and is presented as applicable beyond shape correspondence.
ACNN: The paper reports that ACNN outperforms prior intrinsic CNN models and other state-of-the-art methods on challenging shape-correspondence benchmarks.

2. Background

The section develops intrinsic spectral tools for representing functions and diffusion on Riemannian surfaces, including anisotropic operators that encode direction-dependent heat flow.

Manifolds: A 3D shape is modeled as a two-dimensional compact Riemannian manifold, and intrinsic quantities depend only on its metric, remaining invariant under isometric deformations.
Differential operators on manifolds: The intrinsic gradient gives the tangent direction of maximal function change, while divergence and the Laplace-Beltrami operator provide differential operators on the surface.
Spectral analysis: Laplacian eigenfunctions form an orthonormal basis analogous to the Fourier basis, with eigenvalues acting as non-Euclidean frequencies.
Heat diffusion: Heat diffusion applies the heat operator to an initial distribution, with the heat kernel acting as a non-shift-invariant convolution whose spectral factors are low-pass filters.
Anisotropic diffusion: Anisotropic diffusion uses a thermal conductivity tensor to model position- and direction-dependent heat flow; isotropic diffusion is the special case D(x) = I.
Anisotropic diffusion: Rotated conductivity tensors align diffusion with principal curvature directions, while α controls anisotropy and α = 1 recovers the isotropic case.
Anisotropic heat kernels: Anisotropic heat kernels generalize spectral descriptors by using diagonal kernel values and learned task-specific filters instead of fixed low-pass filters.

3. Intrinsic deep learning

Intrinsic deep learning adapts CNN components to non-Euclidean domains, addressing the lack of shift-invariance through spectral or spatial constructions. Existing approaches trade off cross-shape generalizability, topology, orientation, and computational cost.

Motivation: CNNs on non-Euclidean domains face a fundamental lack of shift-invariance, preventing the direct interpretation of convolution as correlating a fixed template across locations.
Intrinsic learning: Intrinsic CNNs accept surface functions such as texture, geometric descriptors, or motion fields, unlike descriptor-learning methods that learn spectral kernels.
Geodesic CNN: GCNN constructs geodesic polar patches on triangular meshes, but its charting can fail on irregular meshes and requires patch radii smaller than the injectivity radius.
Spectral CNN: SCNN replaces spatial convolutions with learned frequency-domain filters, but basis-dependent coefficients make results differ across shapes and provide weak spatial localization guarantees.
Localized Spectral CNN: LSCNN combines spatial patch ideas with spectral filters through the windowed Fourier transform, representing local function values in the frequency domain.
Localized Spectral CNN: LSCNN applies to meshes and point clouds with suitable Laplace-Beltrami discretization and avoids topological patch issues, but lacks oriented structures and requires explicitly producing each window.

4. Anisotropic convolutional neural networks

ACNN constructs intrinsic patches from oriented anisotropic heat kernels, then stacks convolutional and nonlinear layers to learn soft shape correspondences. Its refinement procedure converts confident predictions into functional maps and pointwise correspondences.

Intrinsic convolution: ACNN uses anisotropic heat kernels as local weighting functions to map nearby function values into polar-like coordinates.The coordinates use orientation θ and heat-kernel scale t.
Intrinsic convolution: The intrinsic convolution correlates the patch representation with a learnable filter over orientation and scale.
Intrinsic convolution: ACNN patches apply to arbitrary shape representations, have spatial geometric meaning, capture directional patterns, and do not depend on injectivity radius.These properties combine spectral applicability with spatially interpretable, direction-aware patches.
ACNN architecture: The architecture stacks fully connected, intrinsic convolution, nonlinear, dropout, batch-normalization, and softmax layers into a learned mapping fΘ(x).The task-specific parameters are optimized for shape correspondence, and the softmax output can be interpreted as a probability distribution.
Correspondence learning: For correspondence, ACNN outputs an m-dimensional probability distribution over reference-shape vertices for each query point and trains it using multinomial regression loss.The loss represents the Kullback-Leibler divergence from the predicted distribution to the ground-truth distribution.
Correspondence refinement: Correspondence refinement selects predictions with confidence c(x)>τth, fits a functional map using an overdetermined system, and produces a new pointwise correspondence.The same procedure is adapted to partial correspondence using a partial functional map.

5. Numerical implementation

The numerical implementation discretizes anisotropic Laplace–Beltrami operators on triangular meshes using curvature-aligned frames, weighted inner products, and sparse matrices. Rotated frames handle general orientations, while heat kernels are computed spectrally from a truncated eigendecomposition.

Anisotropic Laplacian discretization: Each mesh triangle receives an orthonormal frame whose tangent directions are principal-curvature directions.The normal completes the frame, and the curvature directions are computed using the cited method.
Anisotropic Laplacian discretization: Anisotropic edge interactions use an H-weighted inner product, which reduces to the standard inner product when α=1.In the isotropic case, H=I and the weights reduce to the classical cotangent construction.
Anisotropic Laplacian discretization: The discretized anisotropic Laplacian is the sparse matrix L=−S^-1W, combining a diagonal area-based mass matrix with stiffness weights.The mass entries are assembled from incident triangle areas.
Anisotropic Laplacian discretization: For θ≠0, the method rotates each triangle’s basis around its normal and replaces H with the rotated matrix Hθ.The same angle θ is used for all triangles.
Heat-kernel computation: Heat kernels are computed in the frequency domain from the first k eigenfunctions and eigenvalues obtained through generalized eigendecomposition.The heat operator is represented as Φe^-tΛΦ^T.

6. Results

ACNN is evaluated against state-of-the-art correspondence methods on full FAUST meshes and partial SHREC’16 meshes, including cuts and holes. It outperforms the compared approaches across these challenging settings, with especially low distortion and strong performance on partial shapes.

Implementation and timing: Forward propagation takes approximately 0.5 sec to produce dense soft correspondence for all vertices of a FAUST shape with 6.9K vertices.Laplacian computation and eigendecomposition took 1 sec and 4 seconds per angle, respectively.
Full mesh correspondence: ACNN clearly outperforms all compared approaches on FAUST under both symmetric and asymmetric Princeton protocols and perfectly distinguishes symmetric points.The comparison includes RF, BIM, LSCNN, and ADD.
Full mesh correspondence: Over 60% of FAUST matches are exact, while only a few points have geodesic error larger than 10% of the geodesic diameter.ACNN shows dramatically smaller pointwise distortions than the other methods.
Partial correspondence: ACNN outperforms other approaches with a significant margin on SHREC’16 Partial cuts meshes.The benchmark contains nearly-isometrically deformed shapes with parts removed, and evaluation follows the Princeton-style protocol.
Partial correspondence: ACNN produces high-quality correspondences on SHREC’16 Partial cuts despite the challenging partial-matching setting.The paper illustrates partial correspondence and pointwise geodesic error on horse shapes.
Partial correspondence: ACNN also outperforms other approaches with a significant margin on SHREC’16 Partial holes meshes.This experiment evaluates shapes with many small parts removed.

7. Conclusions

The paper presents Anisotropic CNN as a generic framework for extending CNNs to non-Euclidean geometric data. Experiments show that it outperforms prior intrinsic CNNs and other state-of-the-art methods for challenging shape correspondence tasks.

Conclusions: Anisotropic CNN generalizes convolutional neural networks to non-Euclidean domains for deep learning on geometric data.The framework is presented as a generic intrinsic CNN model.
Conclusions: ACNN outperforms previously proposed intrinsic CNN models and additional state-of-the-art methods for shape correspondence in challenging settings.The authors state that the generic model can be used for many other applications.