Source-linked AI summary
MeshCNN: A Network with an Edge
Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, Daniel Cohen-Or
TL;DR
Irregular, non-uniform meshes preserve 3D surface and topology information but complicate CNN-based analysis. MeshCNN addresses this with edge-based convolutions and learned edge-collapse pooling, and reports effective, semantically adaptive behavior across mesh learning tasks. Its generalization nevertheless remains vulnerable to adversarial remeshing.
Problem
Irregular and non-uniform mesh structure makes it difficult to apply convolution and pooling directly while retaining mesh-specific information.
Method
MeshCNN analyzes triangular meshes directly using edge-based convolutions and task-driven edge-collapse pooling that selects which regions to simplify.
Results
MeshCNN demonstrates superior results on shape classification and segmentation tasks, with pooling that converges toward similar representations across different initial triangulations.
Takeaways & Limitations
Mesh-specific convolution and pooling enable direct analysis of meshes in their native, non-uniform form while exposing semantically important features.
Takeaways & Limitations
MeshCNN remains vulnerable to adversarial remeshing attacks that may hinder performance despite robustness to different triangulations.
Abstract
from arXiv · showhide
Polygonal meshes provide an efficient representation for 3D shapes. They explicitly capture both shape surface and topology, and leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This non-uniformity and irregularity, however, inhibits mesh analysis efforts using neural networks that combine convolution and pooling operations. In this paper, we utilize the unique properties of the mesh for a direct analysis of 3D shapes using MeshCNN, a convolutional neural network designed specifically for triangular meshes. Analogous to classic CNNs, MeshCNN combines specialized convolution and pooling layers that operate on the mesh edges, by leveraging their intrinsic geodesic connections. Convolutions are applied on edges and the four edges of their incident triangles, and pooling is applied via an edge collapse operation that retains surface topology, thereby, generating new mesh connectivity for the subsequent convolutions. MeshCNN learns which edges to collapse, thus forming a task-driven process where the network exposes and expands the important features while discarding the redundant ones. We demonstrate the effectiveness of our task-driven pooling on various learning tasks applied to 3D meshes.
1 INTRODUCTION
MeshCNN addresses the difficulty of applying CNN operations to irregular triangular meshes by operating directly on mesh edges and using task-driven mesh pooling. The method preserves mesh-specific structure while adapting simplification to the learning objective and achieves strong results across shape tasks.
- Polygonal meshes efficiently represent large flat regions and intricate features while explicitly preserving surface connectivity.
- Irregular mesh structure makes standard convolution and pooling operations nontrivial, unlike image CNNs operating on regular grids.
- MeshCNN applies convolution directly to mesh edges, using each edge’s four incident triangle edges as a fixed-size neighborhood.
- Mesh pooling uses edge collapse, with the network selecting task-relevant edges instead of minimizing geometric distortion as conventional simplification does.
- The method supports different input mesh sizes and triangulations, targets a predetermined edge count, and demonstrates strong classification and segmentation results.
2 RELATED WORKS
Prior 3D learning approaches use projections, voxels, point clouds, graphs, or manifold parameterizations, but often lose mesh topology, incur computational costs, or do not exploit triangular structure directly. MeshCNN instead defines mesh-specific edge convolutions and learned pooling that selects task-relevant regions.
- Multi-view 2D projections and Volumetric: Multi-view and volumetric methods reuse grid-based CNNs, but indirect representations can require large memory and redundant computation.
- Graph and Manifold: Graph and manifold methods generalize neural processing to irregular data through spectral, spatial, or parameterized neighborhoods.
- MeshCNN: The paper presents learned mesh pooling that selects simplification regions according to the target task, unlike fixed or topology-agnostic alternatives.
- Point clouds: Point-cloud methods provide compact inputs but do not inherently encode local surface connectivity and neighborhood structure.
- MeshCNN: MeshCNN differs by using mesh edges for non-uniform geodesic neighborhoods, consistent convolution support, invariant features, and topology-aware edge collapse.
3 OVERVIEW: APPLYING CNN ON MESHES
MeshCNN applies CNN-style operations directly to triangular meshes, using edge-based neighborhoods and task-adaptive pooling to preserve and emphasize meaningful structure.
- Overview: Triangular meshes encode non-uniform geometry and surface topology, but their irregular connectivity makes standard CNN operations difficult to apply directly.Large flat regions can use few polygons, while intricate regions use more, and topology distinguishes nearby surfaces.
- Mesh convolution: MeshCNN addresses ordering ambiguity by using relative geometric edge features and symmetric aggregation of ambiguous neighbor pairs.Its five-dimensional edge descriptor contains a dihedral angle, inner angles, and edge-length ratios, all made similarity-invariant through sorting and relative measurements.
- Mesh pooling: Mesh pooling repeatedly collapses edges, allowing the network to select which regions to simplify while retaining control over the target edge count.The collapse operation transforms five edges into two and prioritizes edges using feature magnitude.
- Mesh pooling and unpooling: Pooling and unpooling aggregate and restore mesh features and connectivity, supporting interpretable intermediate representations and recovery of mesh resolution.Unpooling reinstates topology stored before pooling and computes unpooled edge features from pooled features using recorded adjacencies.
4 METHOD
The method separates mesh connectivity from edge features, defines invariant edge convolutions, and generalizes pooling through topology-aware edge collapses and reversible unpooling.
- Mesh representation: MeshCNN represents a mesh with vertices, faces, and edges, while edge features begin as invariant geometric descriptors and become more abstract through network layers.Vertices provide positions, faces define triangular connectivity, and edges carry the network’s features.
- Mesh representation: The mesh supplies convolutional connectivity and initial geometric features; vertex positions after edge collapse are used only for visualization.New vertex locations do not affect classification or segmentation in this formulation.
- Mesh convolution: Mesh convolution uses each edge’s four incident neighbors, resolves their ordering ambiguity with symmetric functions, and produces ordering-invariant outputs.The implementation unwraps local neighborhoods into a matrix so optimized batched operators and GEMM can perform the convolution.
- Mesh pooling: Generalized pooling defines a region, merges its features, and redefines adjacency; mesh pooling instantiates these steps through topology-based edge collapses.Each collapse converts five edges into two, and a target-edge hyperparameter controls the pooled mesh resolution.
- Mesh pooling: Feature magnitude determines collapse priority, enabling task-aware non-uniform simplification while aggregation forms new feature vectors for the merged edges.The half-edge structure is updated after collapse, and invalid collapses that create non-manifold faces are prohibited.
- Mesh unpooling: Unpooling partially reverses pooling by restoring stored topology and computing higher-resolution edge features from recorded pre- and post-pooling adjacencies.Unpooling has no learnable parameters and is paired with convolutions to recover the resolution lost during pooling.
5 EXPERIMENTS
MeshCNN is evaluated on classification, segmentation, and robustness tasks using mesh-specific processing and task-driven pooling. Across these experiments, the method preserves task-relevant structure while simplifying meshes and achieves strong or superior results in the reported comparisons.
- Experimental setup: MeshCNN provides a plug-and-play framework for classification and segmentation experiments on triangular meshes.The experiments use different network configurations built from MeshCNN blocks.
- Data processing: Classification experiments use simplified meshes, typically 750 edges, while segmentation uses higher-resolution meshes with about 2250 edges.The paper uses lower resolution for global shape classification and higher resolution for segmentation.
- Mesh classification: On engraved cubes, MeshCNN removes edges from flat surfaces while preserving edges within and around icon engravings relevant to classification.The dataset contains 23 classes generated by placing extruded MPEG-7 icons on randomly selected cube faces and locations.
- Mesh segmentation: On COSEG segmentation, MeshCNN outperforms PointNet, PointNet++, and PointCNN, while replacing learned pooling with random pooling reduces performance.The evaluation covers aliens, vases, and chairs using 85%/15% train/test splits.
- Additional evaluations: MeshCNN remains resilient to tessellation variation, with only minor performance differences after remeshing or randomly perturbing 30% of vertex positions.Additional experiments also report better stretched-shape generalization for geometric features than for absolute coordinates.
6 DISCUSSION AND FUTURE WORK
MeshCNN directly analyzes irregular triangular meshes with operations tailored to their structure, especially task-driven edge-collapse pooling. The discussion highlights interpretable, semantically consistent pooling, broader application opportunities, implementation constraints, and vulnerability to adversarial remeshing.
- Discussion: MeshCNN defines convolution and pooling operations tailored to irregular, non-uniform triangular meshes for direct shape analysis in their native representation.Its edge-based operations exploit the mesh representation's surface-manifold structure.
- Invariant Convolutions: Choosing mesh edges provides fixed-sized convolution neighborhoods, while triangular-mesh symmetries support invariance to transformations.Vertex positions are used only to display evolving meshes and do not affect the task.
- Spatially Adaptive Pooling: Mesh pooling collapses edges selected from learned edge features, making pooling task-driven through guidance from the network loss.Unlike geometric simplification, the network removes edges whose features contribute least to the objective.
- Spatially Adaptive Pooling: Differential features provide invariance to similarity transformations and inhibit overfitting relative to absolute Cartesian coordinates.Semantically similar pooling across different objects further demonstrates the network's generalization and is associated with better results.
- Future Applications: Spatially adaptive pooling simplifies uniform regions while preserving complex ones, motivating analogous irregular pooling for high-resolution image segmentation.The proposed image application would represent large uniform regions with fewer triangles.
- Implementation: The current implementation performs sequential edge collapses, although parallel GPU processing is proposed with non-adjacent simultaneous collapses.The resulting pooled features might differ from those produced sequentially.
- Limitations: MeshCNN remains vulnerable to adversarial remeshing attacks despite robustness to different triangulations, making attack robustness a future-work direction.Successful generalization also relies on good training data.
- Future Work: Future directions include reversing edge collapses for generative mesh modification and extending edge-collapse pooling and unpooling to general graphs.General-graph convolution would require an alternative suited to graph irregularity, potentially using attention over edges.
A TRAINING CONFIGURATIONS
The paper uses specified architectures and training settings for classification and segmentation experiments across four datasets. Classification networks begin with 750 edges, whereas segmentation networks begin with 2250 edges.
- Classification: Classification uses the same network architecture for the SHREC and Cube engraving datasets.The configurations and learning parameters are provided in Table 6.
- Segmentation: Segmentation on COSEG and human body datasets uses a Unet-type network.Table 7 details the segmentation network, whose up part is symmetric to the presented down part.
- Classification: 750 edges initialize both classification networks, which use Adam with lr = 0.0002, group norm with д = 16, 5% edge flips, and 20% slide vertices.These settings are specified in Table 6.
- Segmentation: 2250 edges initialize the segmentation networks, and Table 7 presents only their down part because the up part is symmetric.This configuration applies to both COSEG and human body datasets.