Source-linked AI summary
Sparse 3D convolutional neural networks
Ben Graham
TL;DR
The paper addresses the challenge of applying CNNs efficiently to sparse three-dimensional data. It implements sparse CNNs using smaller filters and triangular or tetrahedral lattices, reporting results across several 2D, 3D, and space-time tasks. The experiments show useful accuracy–cost trade-offs, while the authors note that the reported results are not state of the art.
Problem
Conventional 3D CNNs face rapidly growing computation and memory demands, motivating efficient methods for sparse three-dimensional data.
Method
The paper combines sparse CNN computation with smaller filters and triangular or tetrahedral lattice structures for processing 3D and related data.
Results
The experiments report 88.0% accuracy for RHA and 60.4% single-test accuracy for UCF101, rising to 67.8% with 12-fold testing.
Takeaways & Limitations
Sparse 3D CNNs can be implemented efficiently and produce interesting results across varied 3D data types, with potential applications in biochemistry and robotics.
Takeaways & Limitations
The human-action-recognition results are not state of the art, and the authors did not optimize them or explore alternative optical-flow encodings.
Abstract
from arXiv · showhide
We have implemented a convolutional neural network designed for processing sparse three-dimensional input data. The world we live in is three dimensional so there are a large number of potential applications including 3D object recognition and analysis of space-time objects. In the quest for efficiency, we experiment with CNNs on the 2D triangular-lattice and 3D tetrahedral-lattice.
1 Convolutional neural networks
The paper extends sparse CNNs to three-dimensional data, using sparse inputs and alternative lattices to address the computational and memory costs of conventional 3D CNNs.
- Applications: The framework targets spatially structured applications including knot classification, robotics, biochemistry, motion analysis, and depth-based object recognition.A traced 3D string is sparse because the string visits only a small part of the surrounding volume.
- Sparse 3D CNNs: Sparse 3D CNNs extend sparse 2D CNNs by computing hidden units only where input activity can influence them.Input sites are active when their vectors are non-zero; hidden sites are active when they receive input from an active lower-layer site.
- Motivation: 3D inputs are costly because an N × N × N grid contains many more points than an N × N grid, while conventional 3D filters and memory requirements grow rapidly.Extending a 2D architecture to 3D would require 6.1 trillion operations in the example described.
- Efficiency strategy: The method combines smaller filters with sparse inputs, using 2 × 2 × 2 cubic filters covering 8 sites or size-2 tetrahedral filters covering 4 sites.Smaller filters also help keep hidden layers sparser.
- Alternative lattices: Tetrahedral and triangular lattices reduce the smallest filter size from 2^d sites on hypercubic grids to d + 1 sites in d dimensions.The paper considers triangular grids in 2D and tetrahedral grids in 3D, with filters moving across the corresponding graphs.
- Sparse operations: The sparse algorithm represents each layer on a lattice graph, assigns active locations, and pre-computes a shared ground-state vector for non-active hidden locations.The ground state for the input layer is the zero vector.
2 Experiments
The experiments evaluate triangular and sparse 3D CNNs across image, object, handwriting, and video data, emphasizing accuracy–compute trade-offs. Results show lower computational cost for triangular or tetrahedral designs, while sparse space-time models improve recognition in selected tasks.
- 2.1 Square versus triangular 2D convolutions: TriangLeNet costs 26% less than SquareNet, with no real difference in test errors.Both networks were evaluated on CIFAR-10 using small convolutional layers and max-pooling.
- 2 Experiments: The experiments use 6-fold cross-validation for 3D object recognition and favor smaller networks when computational cost and accuracy trade off.The study covers multiple data types because standard 3D CNN benchmarks were not yet available.
- 2.2 Object recognition: Tetrahedral CNNs are substantially cheaper computationally but less accurate at the smallest object scale.FMP pooling gives the highest small-scale accuracy but is more expensive; small-network throughput is I/O-bound.
- 2.3 2D space + 1D time = 3D space-time: A 3D handwriting CNN requires 118 million operations and achieves a 4.93% test error using stroke order as a time dimension.The input spatial size was held constant so improvements would be attributable to introducing time.
- 2.4 Human action recognition: On RHA, the sparse video approach reaches 88.0% accuracy at 1.1 billion operations per test case, versus 71.7% reported previously.Using a 12% threshold left 1.7% of input pixels active.
- 2.4 Human action recognition: On UCF101, accuracy rises from 60.4% with single testing to 67.8% with 12-fold testing, at a cost of 2.7 billion operations.A 13% threshold resulted in 3.1% of input pixels being active.
3 Conclusion
The paper concludes that sparse 3D CNNs can be implemented efficiently and produce results across varied 3D data. It identifies potential applications in molecular-structure analysis and robotics.
- 3 Conclusion: Sparse 3D CNNs are implemented efficiently and produce interesting results for a variety of 3D data.The conclusion also notes that potential applications remain untested.
- 3 Conclusion: Potential applications include analyzing 3D molecular structures and robotics models built from images combined with depth data.The paper connects these applications to proteins with similar shapes and to 3D robotic models.