Source-linked AI summary
SBNet: Sparse Blocks Network for Fast Inference
Mengye Ren, Andrei Pokrovsky, Bin Yang, Raquel Urtasun
TL;DR
Deep CNNs apply costly computation uniformly even when only sparse spatial regions matter, and prior sparse methods often lacked practical speedups against optimized dense convolution. This paper proposes SBNet, a tiled sparse convolution approach using structured computation masks, and reports substantial wall-clock gains on LiDAR 3D detection without noticeable accuracy loss.
Problem
Uniform spatial computation makes deep CNNs expensive for real-time applications, while prior sparse activation methods often improved theoretical FLOPs without practical speedups over optimized dense convolution.
Method
SBNet uses structured computation masks to decompose convolution into uniform blocks, gathering overlapping tiles for batched dense convolution and scattering the results.
Results
Over 2.6× speed-up is achieved on the KITTI benchmark with almost no loss of accuracy, including the mask-network cost.
Takeaways & Limitations
SBNet provides practical sparse inference acceleration for high-resolution detection while retaining input resolution and model capacity.
Takeaways & Limitations
The approach assumes structured computation masks and requires end-to-end training because batch-normalization statistics differ between full-scale and dense-only activations.
Abstract
from arXiv · showhide
Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications. For many problems such as object detection and semantic segmentation, we are able to obtain a low-cost computation mask, either from a priori problem knowledge, or from a low-resolution segmentation network. We show that such computation masks can be used to reduce computation in the high-resolution main network. Variants of sparse activation CNNs have previously been explored on small-scale tasks and showed no degradation in terms of object classification accuracy, but often measured gains in terms of theoretical FLOPs without realizing a practical speed-up when compared to highly optimized dense convolution implementations. In this work, we leverage the sparsity structure of computation masks and propose a novel tiling-based sparse convolution algorithm. We verified the effectiveness of our sparse CNN on LiDAR-based 3D object detection, and we report significant wall-clock speed-ups compared to dense convolution without noticeable loss of accuracy.
1. Introduction
Deep CNNs spend expensive computation uniformly across spatial locations, despite many vision tasks having sparse regions of interest. SBNet uses computation masks and tiled sparse convolution to reduce inference cost while preserving resolution and model capacity.
- Over a hundred convolution layers can make deep CNN computation prohibitively expensive for real-time applications such as autonomous driving.
- Spatial sparsity can be represented by binary computation masks whose ones mark active locations requiring more computation.Masks may come from problem knowledge or be predicted by a relatively cheap network.
- SBNet computes convolution on a blockwise decomposition of structured computation masks using GPU sparse convolution kernels.The method is designed to retain efficient dense convolution operations through tiled processing.
- The method works with ResNet architectures and can provide further speed-up when integrated within a residual unit.
- SBNet targets sparse-region applications without sacrificing input resolution or model capacity, and achieves significant inference speedup without noticeable accuracy loss on LiDAR 3D object detection.
2. Related work
Prior work explores sparsity in weights, activations, dynamic computation masks, and attention, but practical inference acceleration remains a central concern. SBNet applies sparse convolution operators jointly with available computation masks to target speedups on full-resolution inputs.
- Weight sparsity: Weight sparsity methods reduce model size through pruning or low-rank decomposition, but sparse filters do not trivially deliver large speedups without accuracy loss.Structured approaches include pruning entire filters, while individual filter channels are rarely near zero everywhere.
- Activation sparsity: Activation sparsity exploits zeros from ReLU, sparse gating modules, or structured sparse inputs to reduce computation on hardware or algorithmically.Prior applications include sparse input patterns in 3D object classification.
- Dynamic computation masks: Dynamic computation methods infer masks or heat maps during inference to skip layers, reweight pixels, or identify objectness across deep networks.These masks provide the basis for applying sparse operators to full-resolution processing.
- Sparse inference: Sparse inference can focus computation on useful activation patterns, with sparse batch normalization reported as invariant to input sparsity and superior to regular BN in optical flow.The paper incorporates sparse BN within sparse residual units.
- Attention and sparse convolution: Attention mechanisms improve interpretability and accuracy but generally use dense attention weights across receptive fields, so they do not reduce inference cost in the same way.SBNet instead uses computation masks and sparse convolution operators to pursue inference speedups.
- Sparse convolution algorithms: Existing sparse convolution algorithms commonly use im2col matrix lowering, which gathers sliding windows shaped kH ×kW ×C for active windows.SBNet contrasts its approach with these matrix-lowering methods.
3. SBNet: Sparse Blocks Network
SBNet exploits structured sparsity in computation masks by decomposing active regions into uniform blocks that can be processed with dense convolutions. Sparse gather/scatter operations, custom GPU kernels, and grouped residual units are designed to achieve practical inference speedups.
- SBNet overview: Block sparsity reduces convolutional computation by skipping spatial clumps of activations that are inactive in structured masks.The approach targets structured sparsity because neighboring background pixels often form spatial blocks.
- SBNet overview: SBNet accepts a dense binary mask and reduces it to active block indices before extracting and recombining sparse regions.The mask-to-index operation is followed by sparse gather and scatter operations.
- Tiling-based sparse convolution: Uniform rectangular blocks are used so gathered regions can be batched and processed by one dense convolution operation.Uniform sizing avoids the load-balancing difficulty of rectangles with different shapes.
- Tiling-based sparse convolution: The tiling algorithm gathers overlapping blocks, then scatters results into non-overlapping regions to avoid atomic locking during writes.Mask pooling and thresholding identify the spatial block locations from which patches are extracted.
- Sparse gather/scatter: Sparse gather and scatter kernels expand indices into neighborhood windows, enabling adjacent block results to stitch together locally like dense convolution.The gather kernel produces a B×h×w×C tensor, while scatter reverses the transformation after convolution.
- Sparse residual units: A residual unit can group nine layers under one gather/scatter pair because its receptive-field increase matches that of one 3 × 3 convolution.Sparse batch normalization is also used to normalize across non-sparse elements and avoid invalid data in the statistics.
- Implementation: Custom CUDA kernels fuse indexing, transpose, gather, scatter, and residual addition to reduce intermediate data movement and support wall-clock speedups.The implementation is intended to improve practical runtime rather than only theoretical FLOP counts.
4. Experiments
The experiments evaluate SBNet on LiDAR bird’s-eye-view detection using static road-map and predicted masks, comparing practical speed-ups with dense and theoretical baselines. SBNet achieves substantial inference acceleration while retaining competitive detection accuracy.
- Experimental setup: The evaluation uses TOR4D and KITTI LiDAR bird’s-eye-view detection benchmarks, with road-map and predicted foreground masks providing sparse computation regions.TOR4D supplies offline road maps, while PSPNet predicts foreground masks; KITTI provides comparison with published methods.
- Experimental design: The layerwise study compares SBNet with theoretical speed-up and released sub-manifold sparse convolution implementations across synthetic, road-map, and predicted masks.Synthetic masks estimate an upper bound, road maps come from TOR4D, and predicted masks come from PSPNet.
- Layerwise results: Over 2 × speed-up is achieved for single convolutions at 90% sparsity and large resolutions, while residual units gain more from sharing sparse gather and scatter costs across layers.The sparse residual block groups convolutions, batch normalization, and ReLU operations into one block.
- Layerwise results: SBNet outperforms the released sub-manifold implementation in practical speed, especially at large resolutions, while narrowing the gap to the theoretical maximum.The sub-manifold implementation is slower than dense convolution on most tested activation sizes and sparsity values.
- Mask comparison: Predicted masks average 85 - 90% sparsity versus 70 - 80% for road masks, increasing speed-up across sparse layers despite overhead from irregular mask shapes.Road masks are typically least sparse at 50 - 60% in the dataset.
- Detection results: On TOR4D, road-mask SBNet provides 1.8× speedup and a 0.3% AP gain, while predicted masks provide 2.7× speedup with accuracy slightly below the dense baseline.On KITTI, SBNet exceeds 2.6× speed-up with almost no accuracy loss; including PSPNet, it is reported as the fastest among top-performing methods.
- Detection results: The PSPNet mask network consumes less than 4% of a full dense detector pass, and the combined system gains 26.6% relative speed over the road-map counterpart.Predicted masks also support regions without offline maps.
5. Conclusion and Future Work
SBNet introduces fast convolution for computation masks with structured sparsity and reports wall-clock speed-ups in LiDAR 3D detection. Future work includes combining sparse blocks with pruning and quantization and extending mask representations.
- Contribution: SBNet performs fast convolution from structured computation masks using a sparse-block network.The method decomposes masks blockwise and implements sparse convolution kernels on GPUs.
- Evaluation: The evaluation includes SBNet speed-up and detection accuracy on TOR4D at 70% IoU and benchmark results on KITTI BEV 2017.
- Results: Significant wall-clock speed-ups are reported against state-of-the-art dense convolution implementations.
- Results: LiDAR 3D detection experiments show speed-up and improved detection accuracy with road-map masks, while predicted masks provide higher speed-up with a small accuracy trade-off.
- Future Work: Future work proposes combining sparse blocks with weight pruning and model quantization, and using varied rectangle shapes for finer-grained masks and multi-scale reasoning.