Source-linked AI summary
A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes
Isaak Lim, Alexander Dielen, Marcel Campen, Leif Kobbelt
TL;DR
Representing unstructured surface meshes for machine learning commonly requires resampling, which can add computation and reduce data fidelity. The paper introduces a direct, resampling-free neighborhood encoding and evaluates it for non-rigid shape correspondence, achieving competitive results while substantially reducing preprocessing.
Problem
Existing surface-mesh learning methods commonly resample neighborhoods into structured representations, adding computational effort and potentially reducing data fidelity.
Method
The paper directly serializes local per-vertex neighborhood features and processes them with fully connected layers or LSTMs instead of resampling and CNN-based regularization.
Results
The approach achieves competitive correspondence results, outperforms several current methods, and reduces preprocessing without sacrificing precision.
Takeaways & Limitations
Direct raw neighborhood encoding can provide an efficient alternative to resampling-based learning on manifold 3D meshes for shape correspondence estimation.
Takeaways & Limitations
The method remains affected by mesh tessellation, and high-resolution meshes require longer sequences and potentially more parameters and memory.
Abstract
from arXiv · showhide
The question of representation of 3D geometry is of vital importance when it comes to leveraging the recent advances in the field of machine learning for geometry processing tasks. For common unstructured surface meshes state-of-the-art methods rely on patch-based or mapping-based techniques that introduce resampling operations in order to encode neighborhood information in a structured and regular manner. We investigate whether such resampling can be avoided, and propose a simple and direct encoding approach. It does not only increase processing efficiency due to its simplicity - its direct nature also avoids any loss in data fidelity. To evaluate the proposed method, we perform a number of experiments in the challenging domain of intrinsic, non-rigid shape correspondence estimation. In comparisons to current methods we observe that our approach is able to achieve highly competitive results.
1 Introduction
The paper asks whether mesh-neighborhood resampling can be avoided and proposes a direct encoding for learning on unstructured surface meshes. Evaluated on non-rigid shape correspondence, the approach reduces preprocessing while maintaining or sometimes improving precision.
- 1 Introduction: Surface-mesh representations are difficult because conversion to point clouds, voxels, or range images can lose fidelity, accuracy, or conciseness.Existing graph- and patch-based methods address direct mesh input differently, while patch-based methods rely on regularizing resampling.
- 1 Introduction: The proposed approach targets the efficiency cost of typically nontrivial preprocessing and the data-fidelity loss that can accompany resampling.The paper motivates these as potential benefits requiring experimental evaluation rather than assuming them in advance.
- 1 Introduction: The method directly encodes local neighborhoods in manifold 3D meshes and uses RNNs or fully connected networks for their non-uniform structure.Its raw encoding is sufficiently rich to theoretically emulate common patch-resampling operators.
- 1 Introduction: Resampling-free encoding reduces preprocessing effort without sacrificing precision in non-rigid shape correspondence, with precision sometimes increasing over resampling-based techniques.The method achieves competitive results and outperforms several current methods in correspondence prediction.
2 Related Work
Prior correspondence methods use functional maps, structured patch resampling, global resampling, or graph-based neighborhood aggregation. The proposed approach instead uses direct per-vertex descriptors with little preprocessing and no resampling.
- 2 Related Work: Functional-map methods formulate correspondence through function correspondences, but point-to-point inference still requires computing a functional map.Later work improves the functional-map framework, including learned input descriptors.
- 2 Related Work: Generalized mesh CNNs resample geodesic patches into fixed-size, fixed-pattern samples before applying convolution operations.The fixed sampling pattern enables weighted sums over predefined sample positions.
- 2 Related Work: Other methods perform global structured surface resampling for standard CNN inputs, while Kostrikov et al. aggregate neighborhoods through Laplace or Dirac operators.The proposed method processes neighborhood information natively rather than through reduced blended forms.
- 2 Related Work: Unlike prior approaches, the method requires very little preprocessing, no heavy online computation, and no resampling while exploiting per-vertex descriptors directly.This distinguishes it from methods that use blended neighborhood information or resampled inputs.
3 Resampling-free Neighborhood Encoding
The method serializes local mesh neighborhoods directly by ordering vertices in a spiral, then encodes their features with LSTMs or fully connected layers instead of resampling patches. This preserves raw neighborhood information while accommodating variable or fixed-length sequences, though the encoding remains tessellation-dependent.
- 3.1 Spiral Operator: Randomizing the spiral’s starting direction during training teaches robustness to rotational ambiguity without requiring pooling over multiple orientations or extrinsic information.The paper assumes sufficient network capacity for the model to learn this robustness.
- 3.1 Spiral Operator: Spiral ordering serializes a center vertex and its surrounding k-rings, or truncates the sequence after N vertices for fixed-length input.The ordering starts with the 1-ring and induces subsequent ring orders from shared neighbors; clockwise orientation is fixed while the starting direction is randomized during training.
- 3.2 Learning: The approach feeds unaltered serialized vertex features into recurrent or fully connected networks, avoiding neighborhood resampling and blended sample computation.LSTMs process variable-length sequences, while fully connected layers handle fixed-length concatenations.
- 3.2 Learning: The raw serialization is not independent of input tessellation, but appending metric information such as distances and angles may let the network learn greater independence.The implementation concatenates each vertex’s distance to the center and the angle formed with the preceding vertex.
- 3.3 Architecture Details: LSTM-NET replaces GCNN3 convolution layers with LSTM layers, producing per-vertex neighborhood representations, while FCS-NET applies fully connected layers to fixed-length sequences.The architectures are designed for variable-length and fixed-length serialization respectively and are compared with parameter-matched baselines.
4 Experiments
Experiments on FAUST and independently remeshed FAUST evaluate correspondence accuracy, sequence-length effects, tessellation dependence, and robustness to spiral orientation.
- Experimental setup: The evaluation uses FAUST with 80 training shapes, 10 validation shapes, and 20 test shapes, optimizing all networks with Adam.
- Experimental setup: Fixed-length sequences prevent LSTM-NET from exploiting FAUST's shared connectivity and valence distribution as prediction information.
- Evaluation protocol: Correspondence accuracy is measured as the percentage of predictions within geodesic radii, without symmetry information, against prior methods and a SHOT-based GCNN3 implementation.
- Tessellation dependence: The independently remeshed FAUST benchmark tests tessellation dependence; relative angles and distances are concatenated to SHOT descriptors as additional information.
- Tessellation dependence: On remeshed FAUST, the proposed networks achieve competitive results, with additional relative information enabling less tessellation-dependent neighborhood representations.
- Rotation robustness: Across 100 inference runs with random spiral rotations on remeshed FAUST, the networks remain highly robust, with separate prediction curves indistinguishable.
5 Conclusion
The paper concludes that resampling-free neighborhood encoding achieves competitive shape-correspondence results while reducing preprocessing. It remains affected by tessellation, and high-resolution meshes and RNN-based training impose practical costs.
- 5 Conclusion: Competitive correspondence results are achieved with a simple, efficient resampling-free encoding strategy for local 3D mesh neighborhoods.The approach avoids resampling costs while retaining competitive performance on shape correspondence estimation.
- 5 Conclusion: Tessellation still affects the method, so truly tessellation-oblivious encoding remains an open challenge.A tessellation-oblivious strategy would remove the need for training to learn tessellation independence for optimal performance.
- 5 Conclusion: High-resolution meshes require longer neighborhood sequences, increasing FCS-NET parameters and potentially causing memory issues.Sub-sampled but not resampled serialization is proposed as a future direction.
- 5 Conclusion: RNN training tends to be slower than CNN training, motivating investigation of 1D convolutions as an alternative.The paper also proposes studying feature learning directly from raw mesh information instead of descriptors such as SHOT.