Source-linked AI summary
Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
Weijing Shi, Ragunathan, Rajkumar
TL;DR
3D object detection from sparse, irregular LiDAR point clouds is difficult for grid-based and repeatedly sampled representations. Point-GNN encodes the cloud as a fixed-radius graph, adds auto-registration and box merging, and achieves leading KITTI accuracy using point clouds alone, even surpassing sensor-fusion approaches.
Problem
Grid representations can lose information or waste computation on irregular point clouds, while point-set approaches repeatedly sample and group points at computational cost.
Method
Point-GNN uses point clouds as graph vertices connected to fixed-radius neighbors, then predicts object categories and bounding boxes with auto-registration and box merging.
Results
Point-GNN achieves leading 3D and Bird's Eye View detection accuracy on KITTI using point clouds alone and surpasses sensor-fusion approaches.
Takeaways & Limitations
The experiments demonstrate the potential of graph neural networks as an approach to accurate 3D object detection from point clouds.
Takeaways & Limitations
The box-merging procedure can select an inaccurate bounding box when relying on classification score alone.
Abstract
from arXiv · showhide
In this paper, we propose a graph neural network to detect objects from a LiDAR point cloud. Towards this end, we encode the point cloud efficiently in a fixed radius near-neighbors graph. We design a graph neural network, named Point-GNN, to predict the category and shape of the object that each vertex in the graph belongs to. In Point-GNN, we propose an auto-registration mechanism to reduce translation variance, and also design a box merging and scoring operation to combine detections from multiple vertices accurately. Our experiments on the KITTI benchmark show the proposed approach achieves leading accuracy using the point cloud alone and can even surpass fusion-based algorithms. Our results demonstrate the potential of using the graph neural network as a new approach for 3D object detection. The code is available https://github.com/WeijingShi/Point-GNN.
1. Introduction
Point-GNN addresses the difficulty of applying grid-based and repeatedly sampled point-set processing to sparse, irregular point clouds by representing them as graphs. It combines graph-based detection with auto-registration and box merging, achieving leading KITTI accuracy.
- Grid-based convolutions can lose information in crowded cells or waste computation in empty cells because point clouds are sparse and irregular.
- Point-set methods avoid regular grids but repeatedly sample and group large point clouds, which can be computationally costly.
- Point-GNN represents points as graph vertices and connects neighbors within a fixed radius, allowing features to flow over the point cloud's native structure.
- The model predicts each vertex's object category and bounding box, enabling one-stage detection of multiple objects in a single shot.
- Auto-registration reduces translation variance, while box merging and scoring combines detections from multiple vertices accurately.
- Point-GNN achieves state-of-the-art 3D object detection accuracy on KITTI using point clouds alone and surpasses sensor-fusion approaches.
2. Related Work
Prior 3D point-cloud detection methods commonly use grids or point sets, each introducing representation or computation costs. Point-GNN instead preserves point-cloud irregularity with a graph and performs single-stage detection on one repeatedly updated graph.
- Grid-based methods convert point clouds into regular representations such as BEV or FV images and can incur quantization error from limited image resolution.
- Point-set methods avoid grid mapping but incur additional computational costs from large-scale point sampling and grouping.
- Point-GNN preserves point-cloud irregularity with a graph, constructs that graph once, and iteratively updates vertex features on it.
- Unlike many prior approaches, Point-GNN is a single-stage detector without a second-stage refinement network.
3. Point-GNN for 3D Object Detection in a Point Cloud
Point-GNN represents a point cloud as a fixed-radius graph, updates vertex features with a graph neural network, and predicts object categories and bounding boxes. It adds auto-registration and box merging with scoring to improve translation handling and combine multiple detections.
- The method uses graph construction, a T-iteration graph neural network, and bounding box merging and scoring.The graph is built from a point cloud before detection and post-processing.
- Graph construction: Vertices are downsampled point-cloud points, and edges connect neighbors within a fixed radius r.A cell list solves the fixed-radius neighbor search in O(cN), while voxelization reduces graph-construction density without replacing the graph representation.
- Graph neural network: The graph neural network aggregates edge features to iteratively update vertex states before predicting each vertex’s object category and class-specific bounding box.The network uses MLPs for edge and vertex updates, a Max aggregation function, and residual connections.
- Graph neural network: Auto-registration aligns neighbor coordinates using structural features to reduce translation variance within local neighborhoods.The alignment offset is predicted from the center vertex state and can be disabled by setting the offset function to zero.
- Prediction and loss: The detector predicts 7-degree-of-freedom bounding boxes and trains localization with a Huber loss, while classification uses average cross-entropy.The box parameters encode center position, length, height, width, and yaw; L1 regularization is added to each MLP.
- Box merging and scoring: Box merging replaces score-only NMS by aggregating overlapping boxes with median position and size and a confidence score weighted by IoU and occlusion.The modified NMS returns merged boxes and their confidence scores.
4. Experiments
Point-GNN is evaluated on KITTI using point clouds alone, with benchmark comparisons, ablations, qualitative results, runtime analysis, and robustness tests under LiDAR sparsity. The experiments show strong detection performance and clarify the effects of its components and graph iterations.
- Benchmark evaluation: Point-GNN is evaluated on KITTI using only point clouds, measuring Average Precision for Cars, Pedestrians, and Cyclists in 3D and BEV detection.The benchmark includes Easy, Moderate, and Hard difficulty levels.
- Benchmark results: Point-GNN achieves leading KITTI results, including Easy BEV Car AP 3.45 above the previous state of the art and better performance than fusion methods except for Pedestrians.It leads Car detection on Easy and Moderate 3D levels and Cyclist detection on Moderate and Hard levels.
- Ablation study: Box merging and scoring improve detection accuracy compared with standard NMS in the ablation study.The comparison replaces median box merging or classification-based scoring with highest-score selection to form controlled baselines.
- Ablation study: Auto-registration improves every 3D detection category over the baseline and most BEV categories, while combining all modules does not outperform box merging and scoring alone.The authors hypothesize that regularization requires tuning after adding the auto-registration branch.
- Ablation study: Offset visualization shows vertices moving toward vehicle centers across GNN iterations, reducing sensitivity to vertex translation.The offsets make neighbor coordinates depend less on the center vertex position and more on point-cloud properties.
- Efficiency: The implementation processes one validation sample in 643ms, with GNN inference taking 363ms and graph construction taking 121ms.Reading the dataset and calibration accounts for 70ms.
- Robustness: The experiments also investigate performance on less dense point clouds by downsampling KITTI’s 64-scanning-line LiDAR data.The study motivates sparsity testing because high-density LiDAR can impose high cost.
5. Conclusion
The paper presents Point-GNN as a graph-based approach for 3D object detection from point clouds. Its experiments report leading KITTI 3D and BEV accuracy, while identifying auto-registration and box merging and scoring as effective components.
- 5. Conclusion: Point-GNN detects 3D objects from a compact graph representation of the point cloud without grid mapping or repeated sampling and grouping.The graph representation uses points as vertices and connects nearby points.
- 5. Conclusion: Point-GNN achieves leading accuracy on both 3D and BEV object detection in the KITTI benchmark.The evaluation uses the proposed graph neural network for point-cloud-based detection.
- 5. Conclusion: The experiments report that auto-registration reduces translation variance and box merging and scoring improve detection accuracy.The conclusion identifies both mechanisms as effective components of Point-GNN.
- 5. Conclusion: Future work will optimize inference speed and fuse inputs from other sensors.These directions are stated as planned extensions.