Source-linked AI summary
Unsupervised Deep Learning for Structured Shape Matching
Jean-Michel Roufosse, Abhishek Sharma, Maks Ovsjanikov
TL;DR
SURFMNet addresses the limited applicability of supervised shape-matching methods that require ground-truth maps. It uses bi-level optimization with functional maps to learn descriptors while enforcing structural properties of inferred maps, achieving strong unsupervised accuracy and substantially faster runtime without geodesic computation.
Problem
Supervised shape-matching methods require high-quality ground-truth maps, limiting their use on new shape classes and making some training objectives computationally expensive.
Method
SURFMNet uses bi-level optimization with FMNet and functional maps, optimizing descriptors while enforcing structural properties such as bijectivity, approximate isometry, and pointwise-map consistency.
Results
SURFMNet achieves state-of-the-art accuracy among unsupervised methods, comparable or sometimes superior performance to supervised techniques, and runtime over an order of magnitude faster than FMNet.
Takeaways & Limitations
The method enables shape correspondence without ground-truth maps while operating entirely in the spectral domain and avoiding geodesic-distance computation.
Takeaways & Limitations
The authors identify extension to partial and non-isometric shapes and to other modalities such as images or point clouds as future work.
Abstract
from arXiv · showhide
We present a novel method for computing correspondences across 3D shapes using unsupervised learning. Our method computes a non-linear transformation of given descriptor functions, while optimizing for global structural properties of the resulting maps, such as their bijectivity or approximate isometry. To this end, we use the functional maps framework, and build upon the recent FMNet architecture for descriptor learning. Unlike that approach, however, we show that learning can be done in a purely \emph{unsupervised setting}, without having access to any ground truth correspondences. This results in a very general shape matching method that we call SURFMNet for Spectral Unsupervised FMNet, and which can be used to establish correspondences within 3D shape collections without any prior information. We demonstrate on a wide range of challenging benchmarks, that our approach leads to state-of-the-art results compared to the existing unsupervised methods and achieves results that are comparable even to the supervised learning techniques. Moreover, our framework is an order of magnitude faster, and does not rely on geodesic distance computation or expensive post-processing.
1. Introduction
Shape correspondence is important but difficult for non-rigid shapes, and existing learning-based methods depend on ground-truth maps. SURFMNet removes that requirement by learning descriptors while enforcing global structural properties of the resulting maps.
- Non-rigid shape matching is challenging because shapes may undergo arbitrary deformations that are difficult to characterize axiomatically.
- Learning-based correspondence methods often require high-quality ground-truth maps across enough training examples, limiting use on new shape classes.
- SURFMNet uses a purely unsupervised bi-level strategy that optimizes shape descriptors while penalizing structural properties of the inferred entire map.
- The method enforces properties such as bijectivity or approximate isometry instead of matching inferred maps to ground-truth correspondences.
- SURFMNet achieves state-of-the-art accuracy among unsupervised methods and performance comparable to supervised techniques.
2. Related Work
Prior work combines descriptors with global map objectives, but most learning-based approaches require supervision or lack global consistency. SURFMNet instead optimizes structural map properties without ground truth or geodesic distances.
- Functional Maps: Functional maps represent correspondences as small matrices whose objectives can encode properties such as area preservation, Laplacian commutativity, pointwise products, and partial-map structure.
- Functional Maps: Prior regularizers include bidirectional relations, adjoints, cycle consistency, continuity, and kernel-based techniques for extracting information from descriptors.
- Related Methods: BCICP combines many functional constraints with sophisticated postprocessing and descriptor selection for map optimization.
- Learning-based Methods: Deep learning methods commonly learn point labels or local transformations, but may lack global consistency and require postprocessing to remove artifacts.
- Contribution: SURFMNet transforms descriptors while optimizing structural map properties without ground-truth correspondences or geodesic distances.
- Contribution: The proposed approach improves over standard unsupervised methods and reaches performance comparable to supervised approaches.
3. Background & Motivation
The functional-map pipeline converts descriptor functions into compact map representations and then pointwise correspondences, but its accuracy depends on the descriptors. FMNet learns descriptor transformations supervised by ground-truth maps, motivating a spectral unsupervised alternative.
- Basic Pipeline: The functional-map pipeline computes basis functions, descriptor coefficients, an optimal functional map, and finally a point-to-point map.
- Basic Pipeline: Descriptor coefficients are stored in basis matrices A1 and A2 after computing functions expected to be preserved by the unknown map.
- Basic Pipeline: The map objective combines descriptor preservation with structural regularization, including Laplace-Beltrami commutativity.
- Basic Pipeline: Functional-map estimation remains efficient because its least-squares formulation has k1k2 unknowns, independent of the number of shape points.
- Deep Functional Maps: SURFMNet optimizes transformed descriptors so their functional maps satisfy structural penalties rather than ground-truth map losses.
- Deep Functional Maps: FMNet learns a neural transformation T of descriptors so the resulting soft map approximates known ground-truth correspondences while evaluating the entire map.
- Motivation: FMNet requires ground-truth maps and geodesic distances, motivating a method that learns without training data and operates purely in the spectral domain.
4. SURFMNet
SURFMNet learns nonlinear descriptor transformations without ground-truth correspondences by optimizing structural properties of bidirectional functional maps. Its penalties target bijectivity, orthogonality, intrinsic isometry, and pointwise-map consistency while preserving efficiency in the spectral domain.
- Overview: SURFMNet replaces ground-truth map supervision with penalties on structural properties of inferred functional maps.The method optimizes transformed descriptors rather than deviation from known correspondences.
- Overview: The method uses functional maps in both directions and optimizes a neural transformation of input descriptor functions.The bidirectional maps are computed from the transformed descriptors and jointly used in the objective.
- Bijectivity: The bijectivity penalty drives compositions of the two functional maps toward the identity map.This encodes the requirement that maps in opposite directions act as approximate inverses.
- Structural penalties: Orthogonality encodes local area preservation, while Laplacian commutativity encodes intrinsic isometry.The latter penalty measures whether the functional map commutes with the Laplace-Beltrami operator.
- Descriptor preservation: Descriptor-preservation penalties encourage functional maps induced by point-to-point maps rather than general soft correspondences.The construction uses preservation of pointwise products through multiplicative operators built from optimized descriptors.
- Optimization: Only neural-network parameters are unknown; the functional maps are determined by optimized descriptors through linear systems.Although formulated as optimization problems, the map computations reduce to solving linear systems.
5. Implementation & Parameters
The implementation adapts FMNet in TensorFlow, using independent point subsampling, spectral nearest-neighbor conversion, and fixed SHOT descriptor parameters. An ablation on FAUST finds that combining all four penalties performs best, while Laplacian commutativity is strongest individually.
- Implementation: The neural network uses seven fully connected residual layers with exponential linear units and no dimensionality reduction.The implementation reuses the FMNet descriptor-transformation architecture.
- Implementation: Training independently subsamples 1500 points on each shape, preserving vertex-permutation invariance because the network is connected across descriptor dimensions.The implementation also subsamples 20% of optimized descriptors for the penalty computation.
- Ablation study: The FAUST ablation study reports that combining all four penalties significantly outperforms every tested subset.Among individual penalties, Laplacian commutativity gives the best result.
- Implementation: Functional maps are converted to pointwise maps using nearest-neighbor search in the spectral domain.This follows the original functional-map approach.
- Parameters: The experiments use 352-dimensional SHOT descriptors and fixed scalar penalty weights w1 = 103, w2 = 103, w3 = 1, and w4 = 105.The weights were selected by examining relative penalty values during optimization on a small shape set.
6. Results
Across challenging shape-matching benchmarks, SURFMNet achieves strong correspondence accuracy, including results comparable to supervised methods, while retaining an efficient spectral-domain pipeline.
- Evaluation setting: The remeshed FAUST and SCAPE datasets are more difficult because shapes lack identical meshes and one-to-one vertex correspondence.The authors regard remeshed datasets as more realistic tests of accuracy and generalization.
- Ablation study: Combining all four penalties significantly outperforms every tested subset, while Laplacian commutativity performs best among individual penalties.The ablation evaluates average correspondence geodesic error against ground-truth maps.
- Accuracy comparison: SURFMNet optimized on all shapes achieves the best results against unsupervised methods and remains comparable to supervised GCNN and FMNet.The comparison uses the accuracy evaluation protocol associated with Figures 3 and 4.
- Accuracy comparison: On remeshed datasets, SURFMNet obtains the lowest average error; on remeshed SCAPE, it leads across all measures despite being unsupervised.On remeshed FAUST, supervised GCNN produces a similarly large fraction of low-error correspondences.
- Postprocessing: ICP refinement can improve challenging subset-trained cases without geodesic distances or equal point counts, preserving pipeline flexibility.The refinement is reported for cases such as the remeshed SCAPE dataset.
- Runtime and qualitative comparison: SURFMNet operates entirely in the spectral domain and is over an order of magnitude faster than FMNet and significantly faster than BCICP.Runtime comparisons average performance over 190 shape pairs; BCICP is roughly seven times slower than SURFMNet in the SCAPE texture-transfer comparison.
7. Conclusion & Future Work
The paper presents an unsupervised functional-maps method for shape correspondence and reports performance comparable to, and sometimes better than, supervised techniques. Future work targets additional map penalties, cycle consistency, partial and non-isometric shapes, and other modalities.
- Conclusion: The method uses bi-level optimization to learn descriptors while promoting structural properties of the resulting functional map.The formulation optimizes descriptor functions and structural properties of the entire map obtained through the functional maps framework.
- Conclusion: The unsupervised approach achieves similar, and sometimes superior, performance compared with supervised correspondence techniques.
- Future Work: Future work includes new functional-map penalties, orientation preservation, and cycle-consistency constraints.
- Future Work: The authors also plan to extend the method to partial and non-isometric shapes and to modalities such as images and point clouds.
8. Supplement
The supplement reports a strong correlation between the proposed loss and correspondence quality measured by average geodesic error.
- The proposed loss strongly correlates with average geodesic error, supporting its use as an indicator of correspondence quality.
B. Detailed Tabular Quantitative Comparison
The supplement provides detailed quantitative statistics and notes that learned descriptors perform competitively across all datasets. An additional figure visualizes correlation between the proposed loss and ground-truth geodesic loss.
- Table 3 reports detailed statistics beyond the average geodesic errors shown in Figures 3 and 4.
- The learned-descriptor variant Fmap Ours Opt performs competitively across all datasets, demonstrating the utility of descriptor learning.Fmap Ours Opt is equivalent to Fmap Basic but uses learned rather than original descriptors.
- Figure 7 visualizes correlation between the proposed loss and average geodesic loss computed from ground-truth correspondences.
C. Sensitivity to number of basis functions
Sensitivity experiments vary the eigenfunction basis from 20 to 150 on SCAPE. Performance is best at 120 eigenfunctions, while large bases can overfit under cross-dataset training conditions.
- Figure 8 evaluates SURFMNet on SCAPE remeshed while varying the number of eigenfunctions from 20 to 150.
- Best results on SCAPE remeshed are obtained using 120 eigenfunctions.
- Training on one dataset and testing on another can produce overfitting with a large eigen-basis.The authors attribute this to SHOT initialization being local and not robust to strong mesh variability.
- Overfitting is minimal when SCAPE and FAUST subsets are trained jointly and tested on different shape subsets with a smaller eigen-basis.
- The supplement also includes quantitative benchmark and ablation tables, plus a texture-transfer comparison with supervised methods.
D. More Qualitative Comparison
SURFMNet is compared with supervised and unsupervised methods on remeshed FAUST and SCAPE datasets using qualitative and quantitative evaluations. Across these comparisons, the method produces the highest-quality maps.
- Overall Result: Across the reported qualitative comparisons, SURFMNet produces the highest-quality maps.
- SCAPE: The figures include qualitative comparisons for texture transfer on the SCAPE remeshed dataset.
- FAUST: The figures include qualitative comparisons for texture transfer on the FAUST remeshed dataset, including a comparison with BCICP.BCICP is reported as roughly 7 times slower than SURFMNet, and its matching shortcomings are marked with red circles.
- Quantitative Evaluation: Quantitative evaluations compare pointwise correspondences produced by SURFMNet with supervised and unsupervised methods.