Source-linked AI summary
VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapes
Zongji Wang, Feng Lu
TL;DR
Fine-grained 3D part segmentation needs detailed structural information from voxelized shapes, yet the paper targets feature extraction at limited resolution. VoxSegNet combines spatial dense extraction with attention-based feature aggregation; experiments on a large-scale dataset demonstrate its effectiveness for 3D shape part segmentation.
Problem
3D shape part segmentation partitions shapes into meaningful parts, and the paper targets extracting detailed structural information from voxelized data at limited resolution.
Source: §1 · pp. 1–2Method
VoxSegNet combines an SDE module that preserves spatial resolution with an AFA module that adaptively selects informative features across abstraction levels.
Source: §1 · p. 2Results
Experiments on a large-scale dataset demonstrate VoxSegNet’s effectiveness in 3D shape part segmentation.
Source: §1 · p. 2Takeaways & Limitations
The proposed modules produce dense predictions with semantic consistency and high accuracy of details.
Source: §1 · p. 2The reported performance upper bound rises with resolution, but is harder to improve at higher resolutions and remains separated from state-of-the-art performance.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Voxel is an important format to represent geometric data, which has been widely used for 3D deep learning in shape analysis due to its generalization ability and regular data format. However, fine-grained tasks like part segmentation require detailed structural information, which increases voxel resolution and thus causes other issues such as the exhaustion of computational resources. In this paper, we propose a novel volumetric convolutional neural network, which could extract discriminative features encoding detailed information from voxelized 3D data under a limited resolution. To this purpose, a spatial dense extraction (SDE) module is designed to preserve the spatial resolution during the feature extraction procedure, alleviating the loss of detail caused by sub-sampling operations such as max-pooling. An attention feature aggregation (AFA) module is also introduced to adaptively select informative features from different abstraction scales, leading to segmentation with both semantic consistency and high accuracy of details. Experiment results on the large-scale dataset demonstrate the effectiveness of our method in 3D shape part segmentation.
1 INTRODUCTION
VoxSegNet addresses the challenge of extracting detailed, discriminative features for 3D part segmentation from voxelized shapes at limited resolution. Its SDE and AFA modules preserve spatial detail and aggregate informative features across abstraction levels.
1 INTRODUCTION
- The growth of online 3D shapes motivates analysis methods, while semantic segmentation partitions shapes into meaningful parts for applications such as modeling, editing, and classification.Directly applying CNNs to 3D shapes faces obstacles including irregular data formats.Source: §1 · p. 1
- Voxel, multi-view, mesh, and point-set representations support different 3D CNN or feature-extraction approaches, including non-Euclidean mesh convolution and PointNet’s shared point processing.The paper describes volumetric data as gridded occupancy, multi-view data as rendered 2D images, and PointNet as aggregating point features with max pooling.Source: §1 · p. 1
- Volumetric representation is chosen for its explicit spatial relationships, suitability for 3D convolution, and ease of conversion from other 3D formats.The paper contrasts this with mesh methods requiring smooth manifolds, point clouds lacking neighborhood structure, and multi-view methods sensitive to viewpoint and lighting.Source: §1 · p. 1
- Part segmentation needs detailed structural information, but larger voxel resolutions raise computational and memory costs, and subsampling still loses detail.Prior approaches use more complex representations or carefully designed convolutions to manage the cost of higher resolutions.Source: §1 · pp. 1–2
- SDE preserves spatial resolution to alleviate detail loss from subsampling, while AFA selects informative features across abstraction levels for dense prediction.The authors describe the resulting prediction as having semantic consistency and high accuracy of details.Source: §1 · p. 2
2 RELATED WORK
Prior work spans hand-engineered mesh features, volumetric CNNs, and direct point-cloud methods. The section highlights voxel-based part segmentation while noting the detail lost through feature-extraction subsampling.
2 RELATED WORK
- Mesh-segmentation work moved from individual geometric features to learning-based combinations, while still relying on hand-designed features and smooth manifold meshes.The cited learning-based methods include Kalogerakis et al. and Guo et al..Source: §2 · p. 2
- Volumetric CNNs learned discriminative shape features for identification or classification, while sparse CNNs applied operations to active voxels and octree representations enabled high-resolution inputs and improved computation efficiency.Source: §2 · p. 2
- Voxel-based part segmentation includes octree U-shaped networks and sparse-convolution FCN and U-Net methods, but subsampling still loses detailed information.Wang et al. report state-of-the-art results using 64^3 resolution of leaf octant.Source: §2 · p. 2
- Direct point-cloud segmentation methods include PointNet, PointNet++, Kd-Networks, and RSNet, using global aggregation, local dependencies, spatial partitioning, or recurrent slices.RSNet processes sliced points with bidirectional RNNs that interact with neighboring points.Source: §2 · p. 2
3 METHOD
VoxSegNet assigns part labels to occupied voxels using multi-scale features extracted without reducing spatial resolution. Its SDE and AFA modules address sparse volumetric feature extraction and the selective combination of low- and high-stage features.
3.1 Problem statement
- Each occupied voxel in a volumetric 3D object receives one of K part-category labels.The segmentation framework models a function mapping the voxel volume to part labels.Source: §3.1 · p. 3
3.2 Network Architecture
- SDE units extract multi-scale features, and AFA units combine features across stages before 1×1×1 convolutions predict each voxel’s part label.SDE units use stacked Atrous Residual Blocks; the network applies Softmax for prediction.Source: §3.2 · p. 3, Figure 1 · p. 3
3.3 Atrous Convolution in 3D
- Atrous convolution enlarges receptive fields without down-sampling, but sparse volumetric inputs can leave only the kernel center active and blur distinctions between local patterns.The resulting identical responses at different boundary locations can reduce feature robustness and discrimination.Source: §3.3 · p. 4, Figure 2 · p. 4
3.4 Spatial Dense Extraction
- SDE stacks atrous convolutions with dilation rates chosen so the final receptive field covers a cubic region without holes, extracting multi-scale features from sparse volumes.An example uses three residual blocks with dilation rates [1, 2, 3].Source: §3.4 · pp. 4–5
3.5 Attention Feature Aggregation
- AFA uses high-stage semantic features to weight low-stage features before adding them to high-stage features, selectively aggregating detail and semantics.Feature maps from adjacent stages are concatenated to compute channel-wise attention, and aggregation units are applied progressively between SDE stages.Source: §3.5 · p. 5, Figure 3 · p. 5
4 EXPERIMENTS AND DISCUSSION
VoxSegNet was evaluated on a 16-category, 16,881-model shape-part dataset at voxel size 48. Experiments report strong benchmark performance, examine feature responses and module ablations, and apply part-based features to chair clustering.
4 EXPERIMENTS AND DISCUSSION
- Evaluation used 16,881 models across 16 categories and 50 parts, voxelized at size 48; overall average IoU is a weighted average of per-category IoU.The per-category IoU averages shape-level part-class IoUs, and the overall weights are category shape counts.Source: §4 · p. 5, §4.1 · p. 6
4.1 Comparison to state-of-the-arts
- VoxSegNet achieved the best overall average IoU, ranking first in 9 of 16 categories, second in 3, and third in the remaining 4.The comparison used object-part segmentation results measured by averaged IoU (%).Source: §4.1 · p. 6, Table 1 · p. 7
4.2 Visualization and analysis
- First-stage filters captured geometric patterns such as planar wings and sharp points, while second-stage filters distinguished semantic regions such as airplane wings, fuselage, and engines.For a chair, second-stage filters also distinguished the leg from the back despite their similar bar-like low-level geometry.Source: §4.2 · p. 7, Figure 5 · p. 7
4.3 Ablation study
- The Atrous 3DCNN did not outperform U-Net: first-block filters activated broadly, and the authors suggest sparse inputs can leave atrous kernels without local information.The authors report that less effective early feature maps are concatenated with higher-layer maps, decreasing feature quality; later ARBs show more discriminative responses.Source: §4.3 · p. 8, Figure 6 · p. 8
- SDE+concat achieved 84.34% average IoU, a +0.37% improvement over the U-Net baseline.SDE uses two units with dilation rates and, then concatenates features from different levels.Source: §4.3 · p. 8
- SDE+AFA achieved 86.24% average IoU, +1.90% versus SDE+concat, and helped separate a chair back from a leg with similar low-level patterns.AFA re-weights low-stage feature maps using high-stage features before combining them.Source: Figure 7 · p. 8, §4.3 · p. 9
- Adding SDE and AFA improved their corresponding networks, and three SDE units outperformed two while providing a 43-voxel receptive field at input resolution 48.The study adds SDE and AFA to the baseline network gradually.Source: §4.3 · p. 9, Table 2 · p. 9
4.4 Application: fine-grained shape clustering
- Part-based features clustered chairs with arms separately from those without arms and placed chairs with similar geometric structures near one another, despite ambiguity within part categories.For chairs, the feature concatenates descriptors for back, seat, leg, and arm; the features were not specifically designed for object classification.Source: §4.4 · pp. 9–10
5 CONCLUTION
VoxSegNet targets detailed part segmentation under constrained voxel resolution with SDE and AFA; experiments and ablations support its effectiveness, while the conclusion identifies remaining headroom.
5 CONCLUTION
- Experiments on the large-scale shape part annotation dataset and ablations demonstrate the proposed method’s effectiveness.Source: §5 · p. 10
- SDE alleviates detail loss from sub-sampling and data sparsity, while AFA fuses SDE multi-scale information through attention.Source: §5 · p. 10
- Higher voxel resolution raises mIoU but makes the performance upper bound harder to improve, and a gap from state-of-the-art remains.The authors identify finer feature extraction at constrained resolution as a promising direction.Source: §5 · p. 10