Source-linked AI summary
Towards Real-Time and Adaptable LiDAR Scene Completion
Azhar Hussian, Martin Vossiek, Vasileios Belagiannis
TL;DR
LiDAR scene completion must reconstruct missing geometry from sparse and occluded measurements while meeting real-time demands. RapidLiDAR learns spatially adaptive initialization and multi-scale reconstruction using voxel and BEV features, avoiding manual noise tuning and point-neighborhood operators. On SemanticKITTI and KITTI-360, it matches state-of-the-art completion performance and completes scenes in 0.1 seconds, 2.3 times faster than the fastest prior method.
Problem
Sparse and incomplete LiDAR measurements leave missing geometry, while existing generative and fixed-noise initialization strategies are slow or limited in coverage and sensor generalization.
Method
RapidLiDAR learns spatially varying displacements for adaptive initialization, then refines the scene with multi-scale voxel and BEV features in a single forward pass.
Results
0.1 seconds: RapidLiDAR completes full scenes on SemanticKITTI and KITTI-360, 2.3 times faster than the fastest prior method, with completion performance on par with the state of the art.
Takeaways & Limitations
The method matches state-of-the-art completion performance at the 10 Hz acquisition rate of typical automotive LiDAR sensors.
Takeaways & Limitations
The authors plan to evaluate robustness across domain shifts in sensor beam geometries and generalize across different sensor configurations.
Abstract
from arXiv · showhide
LiDAR scene completion is a key component of 3D perception in autonomous driving, where the scene must be completed in real time to be usable in downstream tasks. Existing approaches typically follow an initialize-and-refine paradigm, in which a coarse initialization of the scene is first constructed, then refined into complete 3D geometry. Generative models are slower because they iteratively refine random Gaussian noise into the scene, while non-generative methods perturb the partial scene with a fixed noise scale, which limits coverage of large gaps and occluded regions and requires manual recalibration for each new sensor configuration. We present RapidLiDAR, a LiDAR scene completion method that treats the initialization itself as a learned, data-driven component. We propose an adaptive initialization module that predicts a spatially varying displacement for each partial input point, expanding the partial observations into a coarse scene initialization adapted to the local geometry, without requiring manual noise tuning. To refine this coarse initialization into a complete and coherent scene, we additionally propose a multi-scale reconstruction module that further refines point positions by querying multi-scale 3D voxel and 2D BEV feature maps constructed from the input scan. By replacing point-neighborhood operators such as farthest point sampling and $k$-nearest neighbor search with voxel- and BEV-based feature extraction, our architecture is faster and can handle different input resolutions by design. Experiments on SemanticKITTI and KITTI-360 show that our method achieves completion performance on par with the state of the art while completing a full scene in 0.1 seconds, which is 2.3 times faster than the fastest prior method. This matches the 10 Hz acquisition rate of typical automotive LiDAR sensors, taking a step toward real-time LiDAR scene completion.
1 Introduction
LiDAR scene completion must infer missing geometry from sparse, occluded measurements while remaining fast enough for autonomous-driving perception. RapidLiDAR learns adaptive initialization and multi-scale reconstruction to improve coverage and efficiency without manual noise tuning.
- Sparse and incomplete LiDAR measurements leave missing geometry that downstream perception systems need completed for holistic scene understanding.
- Existing methods initialize a coarse scene before refining it, using either iterative Gaussian-noise denoising or fixed-noise perturbations.
- Fixed-noise initialization requires manual retuning across sensor configurations and cannot cover large gaps or occluded regions.
- RapidLiDAR predicts spatially varying learned displacements to adaptively expand partial observations into a coarse initialization, then refines points using multi-scale voxel and BEV features.
- 0.1 seconds: RapidLiDAR completes a full scene on SemanticKITTI and KITTI-360, 2.3× faster than the fastest prior method while achieving performance on par with the state of the art.
2 Related Work
Prior LiDAR scene completion methods include iterative generative approaches and single-pass coarse-to-fine methods. RapidLiDAR retains the single-pass paradigm while learning structure-conditioned initialization and replacing point-neighborhood operators with multi-scale BEV feature aggregation.
- Generative approaches: Generative LiDAR completion treats the task as point-level denoising, but iterative sampling requires hundreds of network evaluations and tens of seconds per scan.
- Single-pass approaches: Single-pass methods commonly encode a partial cloud, decode a coarse point set, and refine it in a coarse-to-fine manner.
- Single-pass approaches: LiNeXt extends single-pass completion to scene scale by replicating observed points and perturbing them to construct an initial point set.
- BEV feature aggregation: BEV and deformable-attention methods aggregate scene features for perception tasks, but typically predict labels at fixed object or grid-query locations.
- RapidLiDAR: RapidLiDAR uses output points as queries, initializes them to cover unobserved regions, and refines them with deformable cross-attention over multi-scale BEV features.
3 Method
RapidLiDAR completes an incomplete LiDAR scene in two learned stages: adaptive initialization followed by multi-scale reconstruction. It combines sparse voxel and dense BEV features to adaptively redistribute points and refine them into coherent geometry without point-neighborhood operations.
- Overall architecture: RapidLiDAR maps an incomplete point cloud X ∈ R^M×3 to a complete scene P ∈ R^N×3 in a single forward pass.The network is trained end-to-end against complete ground-truth scenes.
- Multi-Scale Feature Extraction: The feature extractor voxelizes X into multi-scale 3D voxel features and produces a dense 2D BEV map for subsequent completion stages.The hybrid representation combines sparse volumetric geometry with dense scene context.
- Adaptive Initialization Module: The adaptive initialization module repeats and lightly perturbs observed points, then predicts spatially varying displacements that redistribute them toward sparse and occluded regions.Unlike fixed global noise, the learned displacement reflects local scene structure and provides a better-informed starting point for reconstruction.
- Adaptive Initialization Module: Per-point features are formed by interpolating multi-scale voxel and BEV features, then concatenating the sampled vectors for each expanded point set.Trilinear interpolation is used on voxel grids and bilinear interpolation on the BEV grid to preserve smooth gradient flow.
- Multi-Scale Reconstruction Module: A final MLP predicts residual 3D displacements that align initialized points with target surfaces to produce the completed scene.The reconstruction stage refines the coarse initialization into complete and coherent geometry.
- Multi-Scale Reconstruction Module: The reconstruction module uses initialized points as queries over multi-scale BEV context, avoiding full attention whose cost scales as O(N · Kctx).Multi-scale deformable attention supplies long-range context without farthest point sampling or k-nearest neighbor operations.
4 Experiments
Experiments evaluate RapidLiDAR’s completion quality, efficiency, cross-dataset behavior, architectural components, displacement bound, voxel resolution, and qualitative reconstructions. The method achieves state-of-the-art or comparable quality while completing scenes substantially faster than prior approaches.
- Evaluation setup: The evaluation uses SemanticKITTI for training and validation, KITTI-360 for zero-shot testing, and Chamfer Distance plus 3D and BEV Jensen–Shannon divergence as geometric metrics.The KITTI-360 model is evaluated without fine-tuning, while training follows standard point-count protocols.
- Scene completion: RapidLiDAR achieves the best results across all reported SemanticKITTI metrics and state-of-the-art performance on KITTI-360 without fine-tuning.The reported metrics are Chamfer Distance, 3D JSD, and BEV JSD.
- Computational efficiency: 0.1 s per scan is RapidLiDAR’s inference time, making it 2.3× faster than LiNeXt and matching the 10 Hz acquisition rate of typical automotive LiDAR sensors.The comparison reports a lower Chamfer Distance for RapidLiDAR than LiNeXt and uses standard library operations rather than specialized CUDA kernels.
- Ablation studies: Removing either the Adaptive Initialization Module or Multi-Scale Reconstruction Module consistently reduces all three reported metrics on the SemanticKITTI validation set.The ablations replace adaptive displacement prediction with fixed perturbation or bypass deformable cross-attention.
- Ablation studies: Performance is largely insensitive to Smax values of 50, 70, and 100, supporting Smax = 50 as a robust default displacement bound.The comparison uses Chamfer Distance on a downsampled validation set with 180,000 output points.
- Ablation studies: Finer voxel resolutions consistently improve Chamfer Distance, while parameter count and inference time increase moderately; η = 0.3 is selected as a quality-efficiency trade-off.The architecture permits voxel-resolution adjustment for different speed or quality requirements.
- Qualitative results: Qualitative comparisons show that diffusion methods can deviate from ground-truth layouts in large occluded regions, whereas LiNeXt is often closer but can leave parts incomplete.Figure 4 compares reconstructions of occluded regions and large-scale scene geometry on SemanticKITTI.
5 Conclusion
RapidLiDAR learns scene initialization from data, adaptively expanding partial observations before multi-scale reconstruction produces complete geometry. It reaches state-of-the-art-comparable performance at 0.1 seconds per scene, while generalization across sensor configurations remains future work.
- 5 Conclusion: RapidLiDAR learns the initialization itself directly from data rather than relying on manually tuned noise.Its adaptive initialization module predicts spatially varying displacements that adapt partial observations to local geometry.
- 5 Conclusion: A multi-scale reconstruction module refines the coarse initialization using voxel and bird’s-eye-view features from the partial scan.It predicts residual displacements to produce complete and coherent scenes.
- 5 Conclusion: Replacing farthest point sampling and k-nearest neighbor search with voxel- and BEV-based extraction makes the architecture faster and resolution-flexible.
- 5 Conclusion: 0.1 seconds per scene is 2.3 times faster than the fastest prior method and matches the 10 Hz acquisition rate of typical automotive LiDAR sensors.Experiments on SemanticKITTI and KITTI-360 report completion performance on par with the state of the art.
- 5 Conclusion: Generalization across different sensor configurations and robustness to domain shifts in sensor beam geometries remain future research directions.