PaperlayerView exact revision on arXiv

Source-linked AI summary

DiffusionNet: Discretization Agnostic Learning on Surfaces

Nicholas Sharp, Souhaib Attaiki, Keenan Crane, Maks Ovsjanikov

arXiv:2012.00888v3cs.CVcs.CGcs.LG

TL;DR

Surface-learning methods often struggle with representation changes, mesh sensitivity, and scalable processing of real-world geometry. DiffusionNet addresses these issues with learned diffusion and spatial gradients, achieving robust, efficient, representation-agnostic performance across surface-learning tasks while retaining stated limitations around topology, local discrete structure, and transfer gaps.

  • Problem

    Existing surface-learning networks are tied to particular representations or resolutions, sensitive to mesh structure, and challenged by robustness and scalability requirements.

  • Method

    DiffusionNet combines learned diffusion for spatial communication with spatial gradients for directional information, discretized using discrete differential geometry.

  • Results

    DiffusionNet achieves state-of-the-art results across surface segmentation, classification, and non-rigid shape matching while supporting meshes, point clouds, and cross-representation evaluation.

  • Takeaways & Limitations

    The same efficient architecture can generalize across sampling changes and representations, including training on one representation and evaluating on another.

  • Takeaways & Limitations

    DiffusionNet is not automatically robust to topological errors or outliers, cannot communicate across disconnected components, and may retain a small cross-representation transfer gap.

Abstract

from arXiv · show

We introduce a new general-purpose approach to deep learning on 3D surfaces, based on the insight that a simple diffusion layer is highly effective for spatial communication. The resulting networks are automatically robust to changes in resolution and sampling of a surface -- a basic property which is crucial for practical applications. Our networks can be discretized on various geometric representations such as triangle meshes or point clouds, and can even be trained on one representation then applied to another. We optimize the spatial support of diffusion as a continuous network parameter ranging from purely local to totally global, removing the burden of manually choosing neighborhood sizes. The only other ingredients in the method are a multi-layer perceptron applied independently at each point, and spatial gradient features to support directional filters. The resulting networks are simple, robust, and efficient. Here, we focus primarily on triangle mesh surfaces, and demonstrate state-of-the-art results for a variety of tasks including surface classification, segmentation, and non-rigid correspondence.

1 INTRODUCTION

Surface learning methods must handle varied geometric representations and discretizations, but existing approaches can be sensitive to mesh structure and difficult to scale. DiffusionNet addresses these challenges with learned diffusion and spatial gradients, supporting robust, scalable learning across representations and tasks.

  • 1 INTRODUCTION: Surface learning methods must generalize across representations and resolutions, yet existing networks are often tied to particular geometric representations or discretization resolutions.Converting data to a common representation can alias thin features and lose informative details.
  • 1 INTRODUCTION: Defining convolution and pooling on curved surfaces is difficult because there is no universal canonical surface convolution.Existing solutions include canonical mappings, local parameterizations, and mesh-edge convolutions.
  • 1 INTRODUCTION: Advanced geometric operations such as geodesics and parallel transport can reduce robustness and scalability, while mesh-sensitive methods struggle with changing samplings and triangulations.These limitations make real-world variability, complexity, and scale difficult to handle.
  • 1 INTRODUCTION: DiffusionNet replaces expensive geometric operations with learned diffusion for information propagation and spatial gradients for capturing anisotropy.Discrete differential geometry provides the discretization, preserving simplicity while supporting robustness and scalability.
  • 1 INTRODUCTION: The architecture achieves state-of-the-art results across deformable segmentation, classification, and supervised and unsupervised non-rigid shape matching.The same architecture supports meshes and point clouds, including training on one representation and evaluation on another.
  • 1 INTRODUCTION: DiffusionNet combines learned diffusion, spatial gradient features, and discretization robustness into an architecture for surface learning.The stated contributions include spatial data sharing, directional filtering, and benchmark performance.

2 RELATED WORK

Surface-learning research spans point-based, mesh-based, spectral, and pooling approaches, but many methods face computational or transfer limitations. DiffusionNet uses transferable learned diffusion and gradient features to support efficient learning across surface representations without downsampling.

  • 2 RELATED WORK: Volumetric methods can be computationally expensive and difficult to apply to detailed deformable shapes.They represent 3D shapes with voxel grids and include efficient extensions.
  • 2 RELATED WORK: Deep learning on 3D shapes commonly uses point-cloud or triangle-mesh representations.These categories organize the related approaches discussed in the paper.
  • 2 RELATED WORK: Mesh-based methods can be costly on densely sampled shapes and often fail to remain robust under substantial changes in mesh structure.These approaches leverage mesh connectivity, discrete operators, or edge-based random walks.
  • 2 RELATED WORK: Spectral filters are difficult to transfer across shapes because Laplacian eigenfunctions depend on each shape.DiffusionNet instead learns a diffusion process directly transferable across shapes and uses spectral operations only for acceleration.
  • 2.2 Spectral Methods: DiffusionNet replaces complex polynomial filters with simple learned diffusion and uses gradient features to inject orientation information.The paper reports improved performance and robustness from this combination.
  • 2.2 Spectral Methods: Unlike related diffusion methods, DiffusionNet learns a separate diffusion time per feature, incorporates a learned gradient operation, and applies diffusion directly to mesh surfaces.Learned per-feature diffusion time significantly improves performance according to the cited discussion.
  • 2.1 Learning on Surfaces: DiffusionNet supports global spatial support without downsampling, simplifying implementation and avoiding pooling operations.Its efficiency also enables direct learning on full-resolution mesh data rather than dramatically simplified inputs.

3 METHOD

DiffusionNet combines pointwise MLPs, learned diffusion, and spatial gradient features in a representation-agnostic surface architecture. Learned diffusion provides differentiable spatial support from local to global, while gradient features extend filters beyond radial symmetry.

  • Architecture: DiffusionNet uses pointwise MLPs, learned diffusion, and local spatial gradient features as its three main building blocks.MLPs transform features independently at vertices; diffusion propagates information; gradients expand the filter space.
  • Learned Diffusion: The heat operator produces increasingly global smoothing as diffusion time grows, from the identity at t = 0 toward the domain average as t → ∞.The continuous process is modeled by the heat equation and represented as H_t(u_0) = exp(tΔ)u_0.
  • Computing Diffusion: Discrete diffusion replaces the Laplacian with weak Laplace and mass matrices whose sparsity scales as O(V), supporting large inputs.The method uses M and L to approximate diffusion on sampled surfaces.
  • Learned Diffusion: Learned diffusion propagates each feature channel with its own trainable time, automatically tuning support from purely local to totally global.This avoids manually selecting convolution support radii or pooling sizes.
  • Spatial Gradient Features: Gradient features add directional filters through inner products of transformed local gradients while remaining invariant to the choice of tangent basis.Diffusion followed by an MLP supports radially symmetric filters; gradient features expand this space beyond radial symmetry.
  • Computing Diffusion: Spectral acceleration evaluates diffusion by scaling Laplacian-basis coefficients elementwise, while low-frequency truncation introduces little observed effect on accuracy.The eigenbasis requires precomputation, but evaluation uses dense arithmetic and networks achieve similar accuracy to implicit timestepping.

4 DIFFUSIONNET ARCHITECTURE

DiffusionNet combines learned diffusion, spatial gradients, and pointwise MLPs in repeated blocks, providing directional filtering without explicit surface convolutions or pooling hierarchies.

  • DiffusionNet blocks: Each DiffusionNet block diffuses features, constructs spatial gradient features, and feeds the result to a spatially shared pointwise MLP.The architecture uses a fixed scalar channel width throughout, with residual connections and linear input/output layers.
  • Directional filtering: Spatial gradient features support local directional filters, complementing diffusion’s spatial information propagation.The method uses these geometric operations instead of explicit surface convolutions.
  • Architecture properties: DiffusionNet avoids explicit spatial convolutions and pooling hierarchies, keeping the network simple and robust.The same architecture can operate at mesh vertices or points in a point cloud.
  • Invariance: Rigid-motion invariance follows from the intrinsic geometric nature of diffusion and spatial gradients when input features remain unchanged.Overall invariance still depends on the selected input features.
  • Spatial communication: Learned diffusion provides spatial communication while its time scale controls support from local to global.Diffusion time is differentiable, allowing spatial support to be optimized as a network parameter.

5 EXPERIMENTS AND ANALYSIS

Experiments evaluate DiffusionNet across classification, segmentation, correspondence, representation transfer, efficiency, scaling, and robustness, reporting strong performance and stability across inputs.

  • Classification: DiffusionNet achieves nearly-perfect accuracy on 30-class SHREC11 while training on only 10 samples per class.Results are averaged over 10 random training splits; the table includes rows trained and tested on simplified models.
  • Segmentation: DiffusionNet achieves state-of-the-art results for RNA and human segmentation, including direct application to meshes and sampled point clouds.RNA segmentation uses raw meshes and point clouds, while the human segmentation benchmark compares xyz and hks inputs.
  • Functional correspondence: DiffusionNet yields state-of-the-art non-rigid correspondences in supervised and weakly supervised settings, especially when transferring between datasets.Using HKS features with KPConv still leaves significant improvements for DiffusionNet, and only DiffusionNet obtains high-quality correspondence on the cited cross-dataset example.
  • Discretization agnostic learning: Under remeshing and resampling, other mesh-based approaches degrade rapidly, whereas DiffusionNet’s correspondences remain largely stable; point-based methods do not match its overall accuracy.The FAUST evaluation includes original, isotropically remeshed, locally refined, simplified, and point-cloud test inputs.
  • Transfer across representations: The same trained network can be applied directly across discrete representations without special treatment or fine-tuning.The required Laplacian, mass, and spatial gradient matrices can be constructed for meshes and point clouds, preserving the meaning of fitted weights.
  • Efficiency: DiffusionNet trains on 14k-vertex RNA meshes in 38ms per input using 2.2GB of GPU memory, with 5.4sec CPU preprocessing per input.The reported setup uses spectral acceleration and stores 12MB of preprocessing data per RNA input.
  • Scaling: DiffusionNet runs at full resolution on meshes with up to 13k vertices and is demonstrated on a 184k-vertex raw FAUST scan without special treatment.This contrasts with approaches that simplify or downsample comparable meshes to roughly 1k vertices for training.
  • Robustness: DiffusionNet is demonstrated on a low-quality, nonmanifold raw scan mesh without issues, avoiding several complex geometry-processing operations.The paper attributes robustness partly to stable diffusion and a gradient matrix obtained by least-squares fitting.

6 CONCLUSION

DiffusionNet uses learned diffusion and spatial gradients to provide efficient, sampling-robust surface learning across representations. Its scope is bounded by sensitivity to topological errors and local discrete structure.

  • DiffusionNet combines learned diffusion with spatial gradient features for efficient surface learning robust to sampling changes and transferable across representations.The method also achieves state-of-the-art results across several tasks.
  • Disconnected surface components cannot communicate through diffusion, causing nonsensical outputs when topological errors create spurious components.The authors suggest combining diffusion with global pooling or latent nearest-neighbor edge convolutions as possible mitigation.
  • DiffusionNet may not suit tasks that learn properties of local discrete structure, including denoising or mesh modification.
  • The framework can extend to surface representations with constructible Laplacians and spatial gradients, including occupancy grids and subdivision surfaces.The authors also suggest adaptation beyond explicit surfaces to other geometric domains.

A AN ARGUMENT FOR GENERALITY

The analysis argues that diffusion followed by pointwise functions can represent radially symmetric convolutions. It provides evidence for expressivity while noting that the formal treatment remains limited in scope.

  • Diffusion at learned timescales followed by a learned pointwise function forms the essential architecture components analyzed for generality.
  • Radially symmetric convolution values can be recovered from diffused field values through an inverse-transform argument.The construction uses the relationship between spherical integrals and the Laplace transform.
  • The function space defined by diffusion followed by a pointwise map contains the space of radially symmetric convolutions.
  • The formal analysis treats the continuous Euclidean setting rather than curved manifolds or finite-dimensional discrete MLP and diffusion implementations.The authors identify formal stability analysis as valuable future work.

B ANALYSIS

The analysis finds that each DiffusionNet component contributes to human-segmentation accuracy, while spectral diffusion requires a sufficiently large basis but gains little beyond 128 eigenvectors.

  • Ablation: Omitting any DiffusionNet component causes a noticeable accuracy drop on the full-resolution human-segmentation task.The ablations remove diffusion, gradient features, learned gradient transformations, or learned diffusion times.
  • Ablation: Manually fixing diffusion times at t=0.1 and t=0.5 yields significantly worse accuracy than learned diffusion times.The optimization tunes diffusion time separately for each feature channel.
  • Spectral basis size: Performance degrades significantly with fewer than 64 eigenvectors on the FAUST vertex-labeling correspondence task.
  • Spectral basis size: Using more than 128 eigenvectors offers negligible benefit, so the experiments use k=128.Larger spectral bases resolve diffusion more accurately but increase computation.

C EXPERIMENT DETAILS

The experiments evaluate orientation, human segmentation, and discretization robustness across mesh and point-based settings. DiffusionNet performs accurately across task variants and uniquely maintains correspondence quality after remeshing and resampling.

  • Orientation: The orientation experiment segments left versus right on FAUST using an intrinsic DiffusionNet with HKS input to test sensitivity to asymmetric mesh biases.
  • Human segmentation: Human-segmentation evaluations span original full-resolution meshes, subsampled vertices, and simplified edge-labeled meshes with differing ground-truth conventions.DiffusionNet is also applied to the simplified variant by averaging per-vertex predictions to edges.
  • Human segmentation: DiffusionNet achieves highly accurate performance across segmentation variants and can train directly on original meshes without special treatment.Other methods may require preprocessing or postprocessing schemes for full-resolution evaluation.
  • Discretization agnostic learning: The discretization study compares DiffusionNet with mesh-based and point-based methods on remeshed FAUST data.The evaluated methods include SplineCNN, ACSCNN, HSN, PointNet, and DGCNN, with both vertex-set and sampled-point-cloud settings considered.
  • Discretization agnostic learning: Only DiffusionNet learns accurate correspondences robust to both remeshing and resampling.ACSCNN performs well on original templates but produces essentially random noise after remeshing, while point-based methods are less accurate.
Loading 2012.00888v3…