Source-linked AI summary
MeshNet: Mesh Neural Network for 3D Shape Representation
Yutong Feng, Yifan Feng, Haoxuan You, Xibin Zhao, Yue Gao
TL;DR
Recent 3D shape representation has relied heavily on volumetric grids, multi-view data, and point clouds, while mesh data remains less explored because of its complexity and irregularity. MeshNet learns directly from mesh faces using face-unit processing, spatial and structural feature splitting, and neighboring-feature aggregation. On ModelNet40 classification and retrieval, it improves over traditional mesh methods and performs comparably to recent methods using other data types.
Problem
Mesh-based 3D shape representation is comparatively underexplored because mesh data is complex and irregular, despite its strong shape-description ability.
Method
MeshNet learns directly from mesh data by treating faces as units, splitting spatial and structural features, and aggregating neighboring information with designed network blocks.
Results
On ModelNet40 classification and retrieval, MeshNet significantly improves over traditional mesh-based methods and is comparable to recent methods using other data types.
Takeaways & Limitations
The experiments demonstrate MeshNet's effectiveness for 3D shape representation from mesh data across classification and retrieval tasks.
Abstract
from arXiv · showhide
Mesh is an important and powerful type of data for 3D shapes and widely studied in the field of computer vision and computer graphics. Regarding the task of 3D shape representation, there have been extensive research efforts concentrating on how to represent 3D shapes well using volumetric grid, multi-view and point cloud. However, there is little effort on using mesh data in recent years, due to the complexity and irregularity of mesh data. In this paper, we propose a mesh neural network, named MeshNet, to learn 3D shape representation from mesh data. In this method, face-unit and feature splitting are introduced, and a general architecture with available and effective blocks are proposed. In this way, MeshNet is able to solve the complexity and irregularity problem of mesh and conduct 3D shape representation well. We have applied the proposed MeshNet method in the applications of 3D shape classification and retrieval. Experimental results and comparisons with the state-of-the-art methods demonstrate that the proposed MeshNet can achieve satisfying 3D shape classification and retrieval performance, which indicates the effectiveness of the proposed method on 3D shape representation.
Introduction
MeshNet addresses the limited use of mesh data for 3D shape representation by organizing faces and splitting their features within a dedicated neural architecture. On ModelNet40, it improves mesh-based classification and retrieval while remaining comparable to methods using other data types.
- Motivation: Mesh data offers strong 3D shape description but remains difficult to process because its elements, connections, counts, and permutations are complex and irregular.Recent work has focused more on volumetric grids, multi-view data, and point clouds, while mesh methods have largely used handcrafted features.
- Method: MeshNet treats polygon faces as units, defines connections between faces sharing edges, and uses per-face processing with a symmetry function.These design choices target mesh complexity and irregularity directly.
- Method: MeshNet splits face features into spatial and structural components and aggregates neighboring information through mesh convolution blocks.Spatial and structural descriptors learn initial features before mesh convolution combines information from neighboring faces.
- Results: MeshNet is evaluated for 3D shape classification and retrieval on ModelNet40, with significant improvement over traditional mesh-based methods and comparable performance to recent methods using other data types.The reported comparison covers both target tasks and multiple representation types.
Related Work
Prior 3D shape representation methods are organized by input data type, including handcrafted mesh descriptors, volumetric grids, multi-view images, point clouds, and feature-fusion approaches. Each category uses different strategies to handle irregularity, local structure, sparsity, or computational cost.
- Mesh descriptors: Earlier mesh approaches extracted handcrafted features from tetrahedra, triangles, surfaces, or shape functions, including rotation-invariant and multiresolution descriptors.Examples include moments, triangle-based functions, multiresolution surface features, SPH, Gaussian filtering, and intrinsic shape context.
- Voxel-based methods: Voxel-based methods learn from regular volumetric grids, but sparsity introduces extra computation and limits scalability as inputs become larger.FPNN, Vote3D, and OCNN address sparsity, while the passage still describes restrictions with larger inputs.
- View-based methods: View-based methods aggregate multiple 2D images of a 3D shape, with hierarchical view grouping used to produce more discriminative descriptors.The passage also reports higher computation cost and difficulty applying these methods to larger scenes.
- Point-based methods: Point-based methods process unordered point clouds per point, then add neighbor aggregation to recover local information ignored by PointNet.Subsequent methods develop additional point-cloud structures and descriptors.
- Fusion methods: Fusion methods combine features from multiple data types, such as volumetric grids with multi-view data or point clouds with multi-view data.FusionNet and PVNet are representative examples of this category.
Method
MeshNet organizes irregular mesh data around faces, separates spatial and structural information, and processes neighboring faces with shared, permutation-robust operations. Its descriptors and mesh convolution blocks produce per-face features for 3D shape representation.
- Mesh Representation: MeshNet treats each triangular face as the basic unit and connects faces sharing an edge, making neighborhood relationships regular and bounded.A triangular face connects with no more than three faces.
- Mesh Representation: The method splits each face’s features into spatial information and structural information to capture position, shape, and local geometry explicitly.Initial face values include center, corner vectors, and normal information, while neighbor indices encode connectivity.
- Descriptors: Spatial and structural descriptor blocks generate initial per-face features from face centers, corner vectors, and neighboring face normals.The spatial descriptor applies a shared MLP to face centers, while structural descriptors process corner geometry or normal-based neighborhoods.
- Descriptors: Face rotate convolution applies shared functions to successive pairs of corner vectors, averages the results, and uses an MLP to obtain structural features.Rotating the operation across corner pairs removes dependence on the order in which corners are processed.
- Descriptors: Face kernel correlation compares each face and its neighbors’ normals with learnable reference kernels to encode local surface structure.More similar normal-kernel pairs receive higher values, producing one feature per learnable kernel.
- Mesh Convolution: Mesh convolution combines spatial and structural features while aggregating neighboring structural features to expand each face’s receptive field.The final aggregation uses concatenated face-neighbor pairs, a shared MLP, max pooling, and an additional MLP for feature fusion.
Experiments
Experiments evaluate MeshNet on classification and retrieval, analyze its architectural blocks, test robustness to face counts and computational complexity, and visualize structural features.
- Experiments: Experiments evaluate MeshNet for 3D shape classification and retrieval before conducting architectural ablations and robustness, complexity, and feature-visualization analyses.The experiments examine block effectiveness, sensitivity to the number of faces, time and space complexity, and structural-feature behavior.
3D Shape Classification and Retrieval
On ModelNet40, MeshNet represents shapes directly from mesh faces and achieves satisfying classification and retrieval performance. Its design combines spatial and structural features with neighborhood aggregation, while experiments examine component effectiveness, face-count robustness, complexity, and feature behavior.
- Experimental Setup: MeshNet is trained and evaluated on ModelNet40, using 9,843 models for training and 2,468 for testing after simplifying each mesh to at most 1,024 faces.Each model is centered, normalized to a unit sphere, and represented with face normals and connected-face indexes.
- Classification and Retrieval: MeshNet achieves satisfying classification and retrieval performance, improves substantially over traditional mesh-based methods, and is comparable with recent methods using other data types.The comparison is reported on ModelNet40 for both classification and retrieval.
- Network Design: Spatial and structural features separately capture spatial distribution and local shape structure, while mesh convolution aggregates neighboring face features.Face-unit and per-face processing address mesh complexity and irregularity; the two feature types support complementary shape information.
- Ablation Studies: The adopted concatenation method performs better than alternative aggregation methods for combining neighboring features.This comparison is conducted in the mesh convolution block.
- Robustness to Face Count: MeshNet’s classification accuracy shows no downtrend as the number of faces decreases; on nine models with fewer than 50 faces, accuracy reaches 100%.The minimum face count among those models is 10.
- Complexity: MeshNet has linear theoretical complexity in the number of faces and comparable time and space complexity to point-cloud methods.The paper reports this comparison using parameter counts and floating-point operations per sample.
- Structural Feature Visualization: Face rotate convolution produces similar features for faces with similar appearances, whereas face kernel correlation groups faces by surrounding-area types such as flat surfaces and steep slopes.The visualizations concern different structural descriptors and their channel activations.
Conclusions
The paper concludes that MeshNet learns 3D shape representations directly from mesh data by processing polygon faces and splitting their features into spatial and structural components. Experiments on classification and retrieval demonstrate the method’s effectiveness, with future development targeting additional computer-vision tasks.
- Conclusions: MeshNet learns 3D shape representations directly from mesh data by treating polygon faces as units and splitting their features into spatial and structural components.The network includes blocks for capturing and aggregating face features.
- Conclusions: Experiments and comparisons with state-of-the-art methods demonstrate MeshNet’s effectiveness on 3D shape classification and retrieval.The paper identifies further development for additional computer-vision tasks as future work.