Source-linked AI summary
O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, Xin Tong
TL;DR
3D shape analysis needs CNNs that handle irregular shapes without the full-volume costs of voxel representations. O-CNN uses sparse octrees with surface normals and GPU-oriented operations, achieving comparable or better performance at lower computational and memory cost across three tasks.
Problem
Adapting CNNs to irregular triangle meshes or point clouds while avoiding the high memory and computation costs of dense 3D volumes remains challenging.
Method
O-CNN represents shapes with octrees, uses averaged normals in finest leaf octants, and performs CNN operations only on surface-occupied octants using GPU-resident data structures.
Results
O-CNN achieves comparable or better accuracy with much lower computational and memory costs than existing 3D CNN solutions across classification, retrieval, and segmentation.
Takeaways & Limitations
Sparse octree computation provides an efficient CNN solution for analyzing high-resolution 3D shapes across multiple representations and CNN structures.
Takeaways & Limitations
The paper leaves rigorous theoretical analysis of restricted information propagation and generalization to other lattice structures for future studies.
Abstract
from arXiv · showhide
We present O-CNN, an Octree-based Convolutional Neural Network (CNN) for 3D shape analysis. Built upon the octree representation of 3D shapes, our method takes the average normal vectors of a 3D model sampled in the finest leaf octants as input and performs 3D CNN operations on the octants occupied by the 3D shape surface. We design a novel octree data structure to efficiently store the octant information and CNN features into the graphics memory and execute the entire O-CNN training and evaluation on the GPU. O-CNN supports various CNN structures and works for 3D shapes in different representations. By restraining the computations on the octants occupied by 3D surfaces, the memory and computational costs of the O-CNN grow quadratically as the depth of the octree increases, which makes the 3D CNN feasible for high-resolution 3D models. We compare the performance of the O-CNN with other existing 3D CNN solutions and demonstrate the efficiency and efficacy of O-CNN in three shape analysis tasks, including object classification, shape retrieval, and shape segmentation.
1 INTRODUCTION
O-CNN adapts CNN-based 3D shape analysis to sparse octree representations, avoiding computation over empty space while preserving surface orientation information. Its GPU-oriented structure supports efficient analysis across multiple tasks and high-resolution shapes.
- Method: O-CNN represents 3D shapes with octrees and applies 3D CNN operations only to sparse octants occupied by boundary surfaces.Average normal vectors sampled in finest leaf octants provide the input signal.
- Method: The method uses a novel octree structure that stores sparse-octant features and information for CNN operations in graphics memory.A label buffer links features across octree levels for convolution and pooling.
- Efficiency: O-CNN restricts computation to occupied octants, reducing memory and computation growth to quadratic with octree depth and enabling high-resolution 3D analysis.The complete training and evaluation process runs on the GPU.
- Evaluation: Across object classification, shape retrieval, and shape segmentation, O-CNN achieves comparable or better accuracy with substantially lower computational and memory costs than existing 3D CNN solutions.The paper also evaluates different octree depths for object classification.
2 RELATED WORK
Prior 3D CNN approaches use voxels, manifolds, multiview images, or other representations, each with resolution, coverage, or irregularity challenges. O-CNN instead organizes sparse surface information in an octree and implements CNN operations efficiently over that structure.
- Voxel-based methods: Full-voxel CNNs are limited to low resolutions because their memory and computational costs scale with the entire 3D volume.The cited methods are described as limited to resolutions such as 30^3.
- Voxel-based methods: Sparse voxel CNNs reduce computation by activating occupied and neighboring voxels, but deep networks and large kernels can still incur high costs.The passage also notes limitations in efficiency as pooling layers become less frequent.
- Manifold-based methods: Manifold-based methods apply CNNs to geometric features on meshes, using surface parameterizations or graph structures for irregular triangle meshes.These approaches operate directly on mesh-derived geometric representations.
- Multiview-based methods: Multiview methods feed rendered image stacks to 2D CNNs, but view placement must cover the shape while avoiding self-occlusions.The passage identifies view-number and view-distribution choices as unresolved concerns.
- Octree-based method: O-CNN constructs a hierarchical octree from non-empty surface octants and packs octant properties and features into depth-ordered vectors for GPU CNN operations.Shuffle keys order octants, while labels support correspondence between levels.
- Octree-based method: Averaged normals in finest leaf octants are sparse on surfaces and encode local shape orientation more faithfully than binary voxel indicators.The representation is illustrated against a voxelized version of the same shape.
- CNN operations: Octree pooling selects maxima from contiguous sibling entries, while labels reconstruct coarser feature maps and support efficient GPU execution.Unpooling reverses pooling using stored switch variables.
- CNN operations: Restricting computation to occupied octants yields O(n^2) memory and computation costs versus O(n^3) for full-voxel solutions.Here n denotes the finest-level voxel resolution in each dimension.
4 NETWORK STRUCTURE
The paper builds O-CNN by repeatedly applying convolution and pooling across octree levels, then adapts the network for classification, retrieval, and segmentation. The designs include a simple LeNet-inspired classifier and a DeconvNet-style decoder for dense part predictions.
- O-CNN: The network uses a simple LeNet-inspired structure to isolate the benefits of the octree-based representation.The authors choose a basic network because varying network architectures can obscure the source of performance gains.
- O-CNN: O-CNN repeatedly applies convolution and pooling to octree data from bottom to top.A basic unit consists of convolution, batch normalization, ReLU activation, and pooling.
- O-CNN for shape analysis: O-CNN is applied to object classification, shape retrieval, and shape part segmentation.For retrieval, the object-classification output serves as the key for searching similar shapes.
- Object classification: The classification model adds fully connected layers, softmax, and dropout after O-CNN(d).The fully connected layers use 128 neurons, and dropout is used to avoid overfitting.
- Shape part segmentation: The segmentation model cascades a DeconvNet-style deconvolution network after O-CNN(d) for dense predictions.The decoder mirrors the convolution network, replacing convolution and pooling with deconvolution and unpooling.
5 EXPERIMENTS AND DISCUSSION
Experiments evaluate O-CNN on classification, retrieval, segmentation, efficiency, and feature behavior, showing strong accuracy, compact computation, and informative multilevel responses.
- 5.1 Object classification: O-CNN classification improves over VoxNet and FPNN, with O-CNN(5) surpassing all compared methods except voting-based VRN.O-CNN(4) at 16^3 already exceeds FPNN at 64^3, while O-CNN(5) wins without voting among the compared methods.
- 5.1 Object classification: Above 85% accuracy is achieved by O-CNN(3), indicating that coarse octree resolutions with normal information remain informative for recognition.The authors relate this result to recognizing shape categories from relatively far away.
- 5.1 Object classification: Normal signals outperform binary signals in both octree and full-voxel representations, while octree normals achieve the highest accuracy overall.Full-voxel normals do not improve on octree normals despite computing everywhere; the authors attribute binary-octree degradation to discarded interior information.
- 5.1 Object classification: O-CNN runs faster at all tested resolutions and uses less memory than the compared solution above 16^3 resolution.The measurements include CPU-GPU communication over 1000 forward-backward iterations.
- 5.1 Object classification: Filter responses progress from low-level geometric features at U5 to high-level, category-sensitive features at U4 and U3.Examples include planar and round-region detectors at U5 and category-sensitive filters at U3.
- 5.2 Shape retrieval: O-CNN(6) achieves the best retrieval results among tested methods and the best NDCG despite discarding subcategory information during training.O-CNN(5) is comparable to state-of-the-art results, while the learned octree features distinguish similar shapes effectively.
- 5.3 Object part segmentation: The segmentation pipeline uses a unary energy matching neural-network label probabilities and a pairwise energy incorporating neighbor information.The dense CRF refinement minimizes the combined energy over predicted point labels.
- 5.3 Object part segmentation: Segmentation evaluation compares methods using category-averaged part-class IoU, with O-CNN(5) and O-CNN(6) reported in Table 6.The supplied passage identifies the metric and compared methods but does not include the numerical outcomes.
6 CONCLUSION
O-CNN exploits sparse octree representations and local shape orientation for compact storage and fast computation, with better or comparable performance across three shape-analysis tasks. The authors identify adaptive octrees, alternative lattices, and advanced network structures as future directions.
- 6 CONCLUSION: O-CNN uses octree sparsity and local shape orientation to enable compact storage and fast computation while achieving better or comparable performance to existing work.Experiments cover object classification, shape retrieval, and shape segmentation.
- 6 CONCLUSION: The authors plan to apply O-CNN to additional fine-grained tasks requiring high resolutions, including denoising, correspondence, generation, and scene analysis.These applications are identified as future work rather than evaluated contributions in the supplied passage.
- 6 CONCLUSION: Adaptive octree construction based on local geometry could further improve O-CNN computation and memory efficiency by avoiding subdivision of nearly flat regions.The current octree construction does not consider geometry change.
- 6 CONCLUSION: The octree hierarchy could be generalized to other lattices, including tetrahedral and permutohedral lattices, but the authors leave this extension for future studies.The permutohedral lattice is noted as a possible basis for higher-dimension CNNs.
- 6 CONCLUSION: The authors have not experimented with advanced CNN structures such as deep residual or recurrent networks and propose integrating them with O-CNN in future work.They connect this extension to more challenging tasks.