PaperlayerView exact revision on arXiv

Source-linked AI summary

Subdivision-Based Mesh Convolution Networks

Shi-Min Hu, Zheng-Ning Liu, Meng-Hao Guo, Jun-Xiong Cai, Jiahui Huang, Tai-Jiang Mu, Ralph R. Martin

arXiv:2106.02285v2cs.CVcs.GRcs.LG

TL;DR

Irregular mesh connectivity limits direct use of mature 2D CNN operations and multiresolution structures. SubdivNet introduces face-based convolution and subdivision-based hierarchy for triangle meshes, achieving state-of-the-art results across several 3D shape-analysis tasks. Its applicability remains bounded by remeshing requirements and input-resolution trade-offs.

  • Problem

    Irregular mesh connectivity makes standard convolutions with variable kernel size, stride, and dilation, as well as simple pooling hierarchies, difficult to define.

  • Method

    SubdivNet uses Loop subdivision sequence connectivity, face-neighborhood convolution, uniform pooling, and upsampling to adapt 2D CNN architectures to meshes.

  • Results

    SubdivNet achieves state-of-the-art performance on mesh classification, segmentation, and shape correspondence.

  • Takeaways & Limitations

    SubdivNet provides a general mesh-learning framework for closed 2-manifold triangle meshes by combining subdivision connectivity with flexible convolution.

  • Takeaways & Limitations

    Applying SubdivNet to arbitrary meshes requires remeshing, whose approaches trade off mesh quality and base mesh size and remain limited for imperfect meshes, borders, and large-scale flawed scenes.

Abstract

from arXiv · show

Convolutional neural networks (CNNs) have made great breakthroughs in 2D computer vision. However, their irregular structure makes it hard to harness the potential of CNNs directly on meshes. A subdivision surface provides a hierarchical multi-resolution structure, in which each face in a closed 2-manifold triangle mesh is exactly adjacent to three faces. Motivated by these two observations, this paper presents SubdivNet, an innovative and versatile CNN framework for 3D triangle meshes with Loop subdivision sequence connectivity. Making an analogy between mesh faces and pixels in a 2D image allows us to present a mesh convolution operator to aggregate local features from nearby faces. By exploiting face neighborhoods, this convolution can support standard 2D convolutional network concepts, e.g. variable kernel size, stride, and dilation. Based on the multi-resolution hierarchy, we make use of pooling layers which uniformly merge four faces into one and an upsampling method which splits one face into four. Thereby, many popular 2D CNN architectures can be easily adapted to process 3D meshes. Meshes with arbitrary connectivity can be remeshed to have Loop subdivision sequence connectivity via self-parameterization, making SubdivNet a general approach. Extensive evaluation and various applications demonstrate SubdivNet's effectiveness and efficiency.

1 INTRODUCTION

SubdivNet addresses the difficulty of applying image-based CNN structures to irregular meshes by combining face-based convolution with subdivision hierarchies. Its design supports established CNN operations and achieves strong results across 3D shape-analysis tasks.

  • Motivation: Irregular mesh connectivity makes variable-size, stride, and dilated convolutions difficult, while also preventing simple fine-to-coarse pooling hierarchies.Mesh simplification does not provide well-defined mappings between levels or an intuitive pooling operator.
  • Motivation: Subdivision surfaces provide a well-defined mesh pyramid, and each closed 2-manifold triangle face has exactly three neighboring faces for regular local aggregation.Loop subdivision splits each triangle into four and preserves a fine-to-coarse correspondence across suitable connectivity sequences.
  • Method: SubdivNet defines face convolution supporting variable kernel size, stride, and dilation, enabling large receptive fields and adaptations of VGG, ResNet, and DeepLabv3+ to meshes.The operation uses neighboring faces in a structure analogous to pixels in images.
  • Method: Self-parameterization remeshes closed 2-manifold triangle meshes with arbitrary genus into Loop subdivision sequence connectivity, broadening SubdivNet’s applicable input class.The resulting connectivity supports the framework as a general feature extractor for closed 2-manifold triangle meshes.
  • Results: SubdivNet achieves state-of-the-art performance on mesh classification, segmentation, and shape correspondence, with ablations supporting its convolution, pooling, and network designs.The reported evaluation covers multiple 3D shape-analysis tasks and tests key architectural components.

2 RELATED WORK

Related mesh-learning methods use projections, local parameterizations, graph structures, or hierarchical designs to process geometric data. SubdivNet instead uses subdivision connectivity for uniform multiresolution processing and larger receptive fields.

  • Indirect methods: Projection-based methods apply 2D CNNs to views or geometry images, but their additional view-dependent projection step makes them pose-sensitive.These methods transform 3D shapes into images before learning.
  • Local methods: Mesh-learning approaches commonly encode sampled-point neighborhoods or geodesic patches into regular domains before applying convolutional operations.Examples include tangent-plane parameterizations and related local-domain constructions.
  • Hierarchical methods: Hierarchical alternatives use spectral kernels, mesh simplification, geodesic or Euclidean neighborhoods, edge contraction, or variable-step walks to aggregate multiscale information.These approaches differ in how they construct neighborhoods and resolution changes.
  • SubdivNet: Unlike several prior hierarchical methods, SubdivNet provides regular uniform downsampling and supports stride and large dilation to capture long-range face features.Its subdivision-based hierarchy expands the receptive field through consistent fine-to-coarse mappings.
  • Subdivision surfaces: Loop subdivision refines triangle meshes by splitting faces and updating vertex positions, while SubdivNet uses the resulting connectivity for mesh analysis.Arbitrary meshes can be remeshed to subdivision sequence connectivity through a multiresolution process.

3 SUBDIVNET

SubdivNet builds CNNs for meshes with Loop subdivision sequence connectivity, using face neighborhoods for convolution and a mesh pyramid for multiscale feature processing. Its operators support variable receptive fields, pooling, upsampling, and adaptation of established 2D CNN architectures.

  • Mesh hierarchy: SubdivNet assumes Loop subdivision sequence connectivity, where each refinement splits faces into four and meshes form a hierarchical sequence.The mesh has subdivision depth L, base mesh M0, and |F| = 4^L|F0| faces.
  • Mesh hierarchy: Arbitrary closed 2-manifold triangle meshes can be remeshed through self-parameterization to obtain the required connectivity.The remeshing procedure targets a specified base size and subdivision depth.
  • Mesh convolution: The face convolution uses local neighborhoods and supports variable kernel size, dilation, and stride, enabling larger receptive fields analogous to image convolutions.A kernel pattern with size k contains 3 × (2^k−1) faces before accounting for duplicate accesses.
  • Mesh convolution: For kernel sizes above 3, SubdivNet preserves duplicated face accesses rather than redesigning the convolution pattern.When k = 3, no duplication occurs; larger kernels can access some faces repeatedly.
  • Mesh convolution: The zig-zag strategy defines dilated face neighborhoods with uniform spatial distribution, while the alternative scheme can select triangles too close to the center.The proposed dilation preserves the basic pattern’s number of elements and reduces duplicated faces.
  • Multiscale operations: Pooling uniformly merges four finer faces into one parent face, and upsampling reverses this hierarchy by splitting one face into four.The convolution uses order-invariant intermediate features to remove ambiguity in the starting point of ring ordering; asymmetric dilation may require multiple dilated convolutions to cover all directions.

4 REMESHING FOR SUBDIVISION CONNECTIVITY

SubdivNet requires subdivision sequence connectivity, so inputs must be remeshed before processing. Self-parameterization provides this connectivity, but remeshing assumes closed manifold meshes and may introduce distortion.

  • SubdivNet requires subdivision sequence connectivity, while most available meshes lack this property and must be remeshed beforehand.
  • Self-parameterization maps an input mesh to a simplified base mesh, then subdivides and back-projects it to create subdivision connectivity.
  • Remeshing methods are selected by task: MAPS favors global feature aggregation, while Liu et al.’s approach preserves local details with a larger base size.
  • Both remeshing algorithms require closed manifold inputs; more general meshes need conversion to watertight manifolds before processing.

5 EXPERIMENTS

Experiments evaluate SubdivNet across classification, segmentation, and correspondence, using remeshing, augmentation, and task-specific preprocessing. The method achieves strong results across datasets, while input resolution and remeshing distortion define practical trade-offs.

  • Experiments: SubdivNet is evaluated on mesh classification, segmentation, shape correspondence, and qualitative mesh retrieval, with ablations of its key components.
  • Data preprocessing and augmentation: Training and test meshes are remeshed to subdivision connectivity, with multiple remeshed variants and majority voting used to reduce remeshing variance.
  • Classification: 100% accuracy is achieved on SHREC11 with voting, while accuracy without voting is around 95% in training.
  • Classification: SubdivNet makes no mistakes on Cube Engraving, becoming the first method reported to correctly classify all test meshes.
  • Classification: SubdivNet outperforms mesh-based methods on Manifold40, although the Transformer-based PCT is more robust to distortion than hierarchical networks.
  • Segmentation: Human-body segmentation results outperform other methods, with more accurate parts and more consistent boundaries than MeshCNN and PD-MeshNet.
  • Segmentation: SubdivNet significantly improves over MeshCNN and PD-MeshNet on COSEG segmentation, reaching 97.0% ± 0.6% on vases and 95.1% ± 1.5% on chairs.
  • Ablation and robustness: Large kernels and dilation are effective in segmentation ablations, while performance remains close across input sizes and tolerates some remeshing distortion.

Segmentation

SubdivNet is evaluated across segmentation, retrieval, and computational efficiency, with ablations examining architectural and input choices. The reported results show improved segmentation quality, strong retrieval recall, and favorable efficiency relative to mesh and 2D baselines.

  • Segmentation: Table 9 indicates that both shape and pose are necessary for mesh learning.
  • Segmentation: Learning on raw meshes slightly improves segmentation quality, but the additional layers increase computing time by 20%.The comparison uses a feature propagation layer and additional convolution layers on raw meshes.
  • Efficiency: SubdivNet is more than 20 times faster than an edge-based approach, uses less than one-third of its GPU memory, and has comparable performance to an optimized 2D CNN.Measurements include forward and backward propagation duration and GPU memory consumption.
  • Shape retrieval: 76.8% top1, 83.3% top5, and 88.0% top10 recall are achieved for synthetic point-cloud shape retrieval.The evaluation also includes synthetic point clouds and real-world depth scans.

6 LIMITATIONS AND FUTURE WORK

SubdivNet depends on subdivision connectivity and remeshing, which introduce quality–base-size trade-offs and limit supported inputs. The authors identify imperfect meshes and large-scale scenes as areas requiring further development.

  • 6.2 Subdivision Connectivity: Applying SubdivNet to arbitrary meshes requires remeshing, whose approaches trade mesh quality against base mesh size.The authors note that differences between remeshed shapes and raw meshes affect results to some extent.
  • 6.2 Subdivision Connectivity: Processing imperfect meshes, polygon soups, objects with borders, and flawed large-scale scenes requires further work.The paper suggests adaptive remeshing may be more helpful than the current uniform remeshing.
  • 6.3 Applications: The current network cannot be directly applied to large-scale scenes because of limitations in the remeshing technique.The stated scope demonstrated in the paper is single-shape analysis.

7 CONCLUSIONS

The paper presents SubdivNet as a mesh CNN framework that combines Loop subdivision connectivity, flexible mesh convolution, and subdivision-based pooling and upsampling. Evaluations indicate improved accuracy and efficiency over existing mesh-learning approaches.

  • 7 CONCLUSIONS: SubdivNet remeshes inputs into Loop subdivision sequence connectivity to construct a uniform mesh hierarchy for feature aggregation.Self-parameterization is used before building the mesh pyramid.
  • 7 CONCLUSIONS: Its mesh convolutions support user-specified kernel size, stride, and dilation, while subdivision connectivity supports pooling and upsampling.These properties enable direct application of well-known 2D image CNNs to mesh learning.
  • 7 CONCLUSIONS: SubdivNet surpasses existing mesh learning approaches in both accuracy and efficiency.

A NETWORK IMPLEMENTATION

The implementation integrates SubdivNet operations into classification and dense-prediction architectures through neighborhood indexing, hierarchical network blocks, and raw-mesh feature propagation. Design choices balance computational efficiency, resolution, and end-to-end processing.

  • Convolution Neighborhood Indexing: Neighborhoods for kernels larger than three are found by depth-first search and rearranged by in-order traversal of a binary tree.Neighbor indexing can be performed in parallel.
  • Classification Network: The classification network uses VGG-like convolution, batch normalization, and ReLU blocks at each resolution, with max-pooling for downsampling.The authors report that this simple architecture provides sufficient performance.
  • DeepLabv3+: DeepLabv3+ uses ResNet50 as a feature extractor, reduced kernel size and stride, and three downsampling layers to fit raw mesh resolution.These choices are described as improving efficiency and avoiding overfitting.
  • DeepLabv3+ with Feature Propagation: Feature propagation and convolutions on raw meshes create a complete end-to-end pipeline, increasing computation by about 20%.
  • Input shape: Classification uses base mesh size 48 and subdivision depth 4, while dense prediction uses base size 256 and depth 3.The configurations target comparable face counts and balance prediction quality with computational efficiency.

B FURTHER REMESHING DETAILS

The remeshing pipeline constructs subdivision connectivity through MAPS-style decimation and parameterization, while addressing distortion, base-size, and UV-flip problems.

  • Remeshing pipeline: MAPS establishes a bijective map between the raw mesh and a decimated mesh by iteratively removing vertices, retriangulating neighborhoods, and parameterizing removed vertices.A modified method instead uses edge-collapse decimation to improve decimation quality.
  • Remeshing limitations: Vertex-removal order can undersample small important details, causing severe reconstruction distortion; edge-collapse constraints improve quality but restrict the minimum reachable base size.The paper illustrates this issue using limb and hoof details in horse meshes.
  • UV-flip handling: UV flips prevent original triangle faces from mapping to triangular parameter-domain regions and become more probable as the base mesh size decreases.The method avoids flips by splitting problematic triangles during decimation and parameterization.
  • UV-flip handling: Figure 13 illustrates UV-flip avoidance by splitting a problematic triangle into three smaller triangles along the simplified mesh triangulation.The blue triangle crosses an edge of the current simplified mesh before splitting.
  • Examples: Figure 14 provides examples from the Manifold40 dataset.

C MANIFOLD40

Manifold40 preprocessing reconstructs and cleans meshes, adapts remeshing to complicated topology, and accommodates variable mesh sizes during training.

  • Preprocessing: ModelNet40 meshes were made watertight through octree-based isosurface reconstruction, vertex projection, component cleaning, and splitting of occasional non-manifold vertices.
  • Topology and base size: Manifold40 shapes with genus above 20 could rarely reach base size 48, so most meshes used enlarged base sizes between 96 and 192.Eleven training samples were discarded, and network depth was reduced from 4 to 3 to limit computational demands.
  • Evaluation: Table 12 reports mesh segmentation accuracy using MeshCNN’s metric, with different train-test splits for vases and chairs.
  • Training setup: Variable base sizes produce variable input sizes, which are handled by padding meshes with empty faces within each mini-batch.Global pooling permits training and evaluation with variable numbers of mesh faces.

D EVALUATION METRICS FOR MESH SEGMENTATION

The segmentation evaluation uses metrics defined on original meshes, simplified-mesh edges, and simplified-mesh faces to accommodate differing inputs and label representations.

  • Evaluation setup: Compared methods use different inputs and evaluation metrics, while the datasets provide per-face labels on meshes with substantially different face counts.The original human-body meshes contain up to 30k faces, whereas COSEG meshes range from hundreds to thousands.
  • Original-mesh metric: Per-face Accuracy measures overall accuracy on faces of the original meshes before simplification.For other methods, predictions are mapped or projected back to original mesh faces using nearest-point or nearest-face procedures.
  • Simplified-mesh metrics: Per-edge Soft Accuracy on Simplified Meshes counts an edge prediction as correct when it matches any neighboring ground-truth label.This metric originates from MeshCNN and is computed on its simplified meshes.
  • Simplified-mesh metrics: Per-face Hard Accuracy on Simplified Meshes measures overall accuracy on faces of the simplified meshes after projecting SubdivNet predictions onto them.
Loading 2106.02285v2…