Source-linked AI summary
FPNN: Field Probing Neural Networks for 3D Data
Yangyan Li, Soeren Pirk, Hao Su, Charles R. Qi, Leonidas J. Guibas
TL;DR
3D CNNs struggle with cubic computational growth and increasingly sparse voxel occupancy. FPNN represents shapes as volumetric fields and learns both probing-point weights and locations, achieving strong classification performance with lower computation. Its current scope favors robust global descriptions over finer internal structures, and independent filters may create too many parameters for small training sets.
Problem
3D CNNs incur cubic computational complexity as voxel resolution grows, while boundary-based geometry produces increasingly sparse occupied regions and wasted computation.
Method
FPNN represents 3D data as fields and uses filters of trainable probing points whose locations and weights adaptively sample the input.
Results
FPNN matches 3DCNN performance with much less computation on ModelNet classification, while learned field-probing layers improve top-1 accuracy from 79.1% to 85.0%.
Takeaways & Limitations
The resolution-agnostic computation and long-range interactions support efficient global 3D shape descriptions from a small number of probing filters.
Takeaways & Limitations
FPNN currently favors robust global descriptions but lacks finer-structure understanding, while independent filters may create too many parameters for small training sets.
Abstract
from arXiv · showhide
Building discriminative representations for 3D data has been an important task in computer graphics and computer vision research. Convolutional Neural Networks (CNNs) have shown to operate on 2D images with great success for a variety of tasks. Lifting convolution operators to 3D (3DCNNs) seems like a plausible and promising next step. Unfortunately, the computational complexity of 3D CNNs grows cubically with respect to voxel resolution. Moreover, since most 3D geometry representations are boundary based, occupied regions do not increase proportionately with the size of the discretization, resulting in wasted computation. In this work, we represent 3D spaces as volumetric fields, and propose a novel design that employs field probing filters to efficiently extract features from them. Each field probing filter is a set of probing points --- sensors that perceive the space. Our learning algorithm optimizes not only the weights associated with the probing points, but also their locations, which deforms the shape of the probing filters and adaptively distributes them in 3D space. The optimized probing points sense the 3D space "intelligently", rather than operating blindly over the entire domain. We show that field probing is significantly more efficient than 3DCNNs, while providing state-of-the-art performance, on classification tasks for 3D object recognition benchmark datasets.
1 Introduction
3D shape understanding requires representations that are both discriminative and efficient, because voxel-based 3D CNNs waste computation on sparse occupancy grids and scale cubically with resolution. FPNN addresses this with trainable field-probing sensors that adapt their locations and weights to extract features efficiently while retaining strong classification performance.
- 3D semantic understanding remains challenging because effective schemes for converting 3D input into informative machine-learning features are difficult to find.
- 3D occupancy grids become increasingly sparse as surface-approximation fidelity and voxel resolution increase, leaving much of the computational domain empty.
- 3D convolutions have cubical complexity in spatial resolution, and conventional filters waste computation on largely empty voxel blocks.
- FPNN represents 3D data as fields and samples them with probing filters whose trainable point locations and weights adapt to sparsity and long-range effects.
- FPNN computation depends on the number of filters and sampled points rather than input resolution, allowing a small filter set to capture sufficient information.
- On ModelNet, FPNN matches 3DCNN performance while requiring much less computation.
2 Related Work
Prior work includes handcrafted and learned descriptors for 3D shapes, CNNs for 2D or depth data, and volumetric or multi-view approaches for 3D recognition. FPNN follows data-driven feature learning while specifically exploiting 3D sparsity and offering an end-to-end alternative to conventional sparse coding.
- 3D Shape Descriptors: 3D shape descriptors have used rendered 2D images, statistical properties, spherical harmonics, and heat diffusion to describe shape geometry.
- Convolutional Neural Networks: CNNs learn general-purpose image descriptors from data, motivating FPNN’s data-driven feature-learning paradigm for 3D inputs.
- CNNs on Depth and 3D Data: Volumetric 3D CNNs face cubic-resolution complexity, while multi-view CNNs render 3D shapes into images and can outperform 3D CNNs on classification tasks.
- FPNN is related to sparse coding but is more computationally tractable and supports an end-to-end learning system.
3 Field Probing Neural Network
FPNN represents 3D shapes as differentiable fields and uses probing filters whose locations and weights are learned. This design focuses computation on informative regions while retaining deep-network processing for shape classification.
- 3.1 Input 3D Fields: Meshes and point clouds are converted to occupancy grids, then distance fields sampled at arbitrary locations by trilinear interpolation.The distance field is indexed on a resolution-R 3D array; occupied cells form the zero level set before distance transformation.
- 3.2 Field Probing Layers: The Sensor layer samples T-channel fields at C filters with N points each, producing a C × N × T data chunk.The DotProduct layer then combines sampled signals with the filters’ point-associated weights into one output per filter.
- 3.2 Field Probing Layers: Field probing filters contain trainable probing points and weights, allowing the filters to adapt both their locations and shapes.Unlike conventional CNNs with fixed filter shapes, FPNN optimizes probing-point locations through back-propagation.
- 3.2 Field Probing Layers: A Gaussian transform emphasizes samples near the object surface because directly using distant distance-field values does not converge.The transform assigns larger weights to regions approaching the zero surface and reduces the influence of distant regions.
- 3.2 Field Probing Layers: Field probing layers have complexity O(C × N × T), compared with O(K^3 × C × S^3) for 3D convolutional layers.With the reported settings, the field-probing computation is approximately 1.83% of the compared convolutional-layer computation.
- 3.2 Field Probing Layers: FPNN feeds field-probing outputs into further inference layers, including fully connected layers, to minimize task-specific losses.Long-range connections support using a small number of probing filters, and 3D object classification is used to evaluate the design.
4 Results and Discussions
Experiments show that field probing layers provide efficient, flexible 3D feature extraction with competitive classification performance and robustness to spatial variation. The method benefits from high-resolution fields, long-range sensor connections, and multiple input fields, while remaining subject to scope limitations for fine structures and parameter sharing.
- Efficiency: Field probing layers have computation cost largely independent of input resolution, unlike convolutional layers whose runtime is compared across resolutions.The slight runtime increase at higher resolutions is attributed to GPU memory latency from larger 3D fields.
- Classification performance: 5.9% accuracy gain, from 79.1% to 85.0%, occurs when field probing parameters are optimized in the 1-FC ModelNet40 setting.The resulting 85.0% exceeds reported 3DCNN accuracies of 83.0% and 83.8%.
- Classification performance: The 4-FC setting reaches 87.5% versus 86.6% without field probing optimization, adding a 0.9% gain over a stronger baseline.The smaller gap is attributed to the additional fully connected layers' learning capability.
- Network design: FPNNs remain reasonable without optimizing probing layers, supporting the benefit of long-range connections among sensors.Distant sensor information is directly coupled because the field probing output is low-dimensional enough to feed into fully connected layers.
- Robustness and representations: Higher-resolution input fields improve performance without increasing field-probing computation costs, while multiple fields and spatial perturbation tests provide additional supported benefits.The experiments include distance and normal fields, and evaluate random rotations, translations, and scaling applied during training and testing.
- Limitations and future work: The current framework is limited for finer internal structures, independent filters may create too many parameters on small training sets, and higher-dimensional extensions require careful storage design.The authors suggest hierarchical filters and partially shared probing-network parameters as possible directions.
5 Conclusions
The paper proposes a novel design for extracting features from 3D data whose computation cost is agnostic to representation resolution. It also couples long-range interactions and aims to stimulate further 3D feature-learning research.
- The proposed design extracts features from 3D data with computation cost agnostic to representation resolution.
- The design easily couples long-range interaction in 3D feature extraction.
- The authors release their code to encourage future developments in feature learning from 3D data.