Source-linked AI summary
Ghost-FWL: A Large-Scale Full-Waveform LiDAR Dataset for Ghost Detection and Removal
Kazuma Ikeda, Ryosei Hara, Rokuto Nagata, Ozora Sako. Zihao Ding, Takahiro Kado, Ibuki Fujioka, Taro Beppu, Mariko Isogawa, Kentaro Yoshioka
TL;DR
Ghost removal remains difficult for sparse mobile LiDAR because prior methods rely on dense geometric cues. The paper introduces Ghost-FWL and FWL-MAE to exploit full-waveform signals, and reports improved ghost detection plus downstream gains in SLAM and 3D object detection.
Problem
Ghost removal lacks a publicly available mobile FWL dataset with peak-level annotations, while prior geometric methods do not generalize to sparse dynamic LiDAR.
Method
Ghost-FWL provides annotated mobile FWL data, while FWL-MAE learns representations by reconstructing masked temporal regions and peak position, amplitude, and width.
Results
66% trajectory error reduction and 50× fewer ghost-induced false positives were achieved in downstream SLAM and 3D object detection.
Takeaways & Limitations
Ghost-FWL enables learning-based ghost detection and removal at the waveform level using temporal and peak-level physical information.
Takeaways & Limitations
Peak-level annotations focus on static multi-viewpoint captures, while mobile trajectories remain unlabeled and coverage emphasizes glass-induced ghosts in clear weather.
Abstract
from arXiv · showhide
LiDAR has become an essential sensing modality in autonomous driving, robotics, and smart-city applications. However, ghost points (or ghosts), which are false reflections caused by multi-path laser returns from glass and reflective surfaces, severely degrade 3D mapping and localization accuracy. Prior ghost removal relies on geometric consistency in dense point clouds, failing on mobile LiDAR's sparse, dynamic data. We address this by exploiting full-waveform LiDAR (FWL), which captures complete temporal intensity profiles rather than just peak distances, providing crucial cues for distinguishing ghosts from genuine reflections in mobile scenarios. As this is a new task, we present Ghost-FWL, the first and largest annotated mobile FWL dataset for ghost detection and removal. Ghost-FWL comprises 24K frames across 10 diverse scenes with 7.5 billion peak-level annotations, which is 100x larger than existing annotated FWL datasets. Benefiting from this large-scale dataset, we establish a FWL-based baseline model for ghost detection and propose FWL-MAE, a masked autoencoder for efficient self-supervised representation learning on FWL data. Experiments show that our baseline outperforms existing methods in ghost removal accuracy, and our ghost removal further enhances downstream tasks such as LiDAR-based SLAM (66% trajectory error reduction) and 3D object detection (50x false positive reduction). The dataset and code is publicly available and can be accessed via the project page: https://keio-csg.github.io/Ghost-FWL
1. Introduction
Ghosts are false LiDAR detections caused by multi-path reflections, undermining mapping, localization, and object detection in mobile environments. Ghost-FWL addresses the lack of suitable mobile full-waveform data with a large annotated dataset, an FWL baseline, and FWL-MAE.
- Ghosts arise from reflections on transparent or reflective surfaces and can create false positives, incorrect maps, and SLAM localization collapse.
- Prior geometric-consistency methods assume static, dense scans and do not generalize reliably to sparse, dynamic mobile LiDAR.
- FWL records complete temporal intensity profiles and indirect returns, providing cues for distinguishing ghosts from genuine reflections beyond peak distances.
- Ghost-FWL contains 24K frames, 7.5B peak-level labels, and 10 diverse real-world scenarios, making it more than 100 times larger than previous annotated FWL datasets.
- FWL-MAE performs self-supervised pre-training by reconstructing masked temporal regions and modeling peak position, amplitude, and width.
- 66% trajectory error reduction and 50× fewer ghost-induced false positives were achieved in downstream SLAM and 3D object detection.
2. Related Work
Existing ghost-removal and FWL datasets provide insufficient support for sparse mobile ghost detection. Ghost-FWL instead uses waveform-level physical cues and specialized self-supervised representation learning.
- Geometric and learning-based ghost-removal methods struggle when mobile LiDAR produces sparse data and geometric consistency cannot be reliably established.
- Unlike ranging-focused FWL studies, this work classifies peak origins by physical cause: Object, Glass, or Ghost.
- Conventional LiDAR datasets contain point clouds without full-waveform data or ghost annotations, while stationary ghost datasets are unsuitable for mobile platforms.
- FWL captures complete temporal intensity profiles and multi-path returns, but existing public or mobile datasets lack ghost-specific peak-level annotations or were not released publicly.
- The paper uses real-world data rather than synthetic data because simulating multi-path ghosts is computationally expensive and physically inaccurate.
- FWL-MAE extends masked reconstruction by explicitly modeling temporal continuity and histogram peak information.
3. Ghost-FWL Dataset
Ghost-FWL is a large real-world mobile FWL dataset with peak-level physical-origin labels collected across diverse indoor and outdoor scenes. Its acquisition and annotation pipeline supports supervised ghost detection while preserving unlabeled mobile trajectories for self-supervised learning.
- 24,412 annotated frames across 10 scenes contain 7.5B peak-level labels for object, glass, ghost, and noise reflections.
- The custom acquisition system accesses the LiDAR FPGA to capture raw full-waveform signals rather than only processed point clouds.
- Scenes include four indoor and six outdoor environments collected with the sensor mounted on a mobile platform.
- Static multi-viewpoint captures vary incident angle and sensor orientation, producing the 24,412 annotated frames used for supervised detection.
- Continuous mobile trajectories total 8,933 frames and remain unlabeled for self-supervised pre-training.
- Each reflection peak above threshold is assigned to Object, Glass, Ghost, or Noise using 3D-map-based rules and expert review.
4. FWL-based Ghost Removal Framework
The framework classifies full-waveform LiDAR peaks into reflection categories and removes predicted ghost peaks. FWL-MAE provides self-supervised representations that model waveform reconstruction and peak properties.
- 4.1. Full Waveform LiDAR Masked Autoencoder: FWL-MAE is a self-supervised pretraining method designed to learn latent representations from full-waveform LiDAR data.
- 4.1. Full Waveform LiDAR Masked Autoencoder: The encoder masks temporal bins within sampled spatial patches and reconstructs waveform regions while estimating peak position, amplitude, and width.These peak attributes are modeled alongside voxel reconstruction to capture physically meaningful waveform structure.
- 4.1. Full Waveform LiDAR Masked Autoencoder: The FWL-MAE objective combines mean squared reconstruction loss with weighted peak-property losses controlled by λp, λa, and λw.
- 4.2. Ghost Detection and Removal: The ghost-removal framework classifies FWL data into Glass, Ghost, Object, and Noise categories, then removes 3D points corresponding to predicted Ghost peaks.
- 4.2. Ghost Detection and Removal: Focal loss is used for the multi-class classification task because Ghost is a minority class alongside the majority Noise class.
5. Experiments and Results
Experiments evaluate ghost classification and removal, then assess its effects on SLAM and 3D object detection. The method qualitatively removes diverse ghost artifacts and reduces ghost-related downstream errors.
- 5. Experiments and Results: The evaluation covers ghost classification and removal, followed by downstream tests in SLAM and 3D object detection.
- 5.1. Ghost Detection and Removal: Ghost removal eliminates horizontally, vertically, and inter-building ghost artifacts while producing accurate class classification.
- 5.2.1. Evaluation on SLAM: 66% trajectory error reduction is reported for SLAM after ghost removal, whose trajectory more closely follows the ground-truth path than Multi-Peak processing.
- 5.2.2. Evaluation on Object Detection: The object-detection evaluation uses 102 frames and 239 object instances collected in indoor and outdoor environments containing glass surfaces.
- 5.2.2. Evaluation on Object Detection: Ghost False Positive Rate counts pedestrian detections whose bounding boxes overlap annotated ghost regions.
- 5.2.2. Evaluation on Object Detection: 50× reduction in Ghost FP Rate is achieved, from 67.9% to 1.34%, compared with baseline LiDAR processing.
6. Conclusions and Future Work
Ghost-FWL is presented as a large-scale annotated FWL dataset supporting learning-based ghost removal and improved LiDAR perception. The authors identify annotation coverage and environmental diversity as future expansion areas.
- 6. Conclusions and Future Work: Ghost-FWL contains over 24,000 annotated frames and 7.5 billion peak-level labels across 10 diverse scenes, making it 100× larger than prior work.
- 6. Conclusions and Future Work: The dataset provides reflection-origin annotations for Ghost, Glass, and Object while retaining temporal intensity profiles for learning-based removal.
- 6. Conclusions and Future Work: Models trained on Ghost-FWL improve LiDAR-based SLAM and 3D object detection in ghost-existing environments.
- Limitations and Future Work: Peak-level annotations focus on static multi-viewpoint captures, while continuous mobile sequences remain unlabeled and are used for self-supervised pre-training.
- Limitations and Future Work: Future work includes annotating continuous sequences and studying reflective materials beyond glass under adverse conditions such as rain and fog.
A. Overview of Supplementary Material
The supplementary material adds experimental details and results that complement the main paper, with an accompanying video presenting SLAM comparisons.
- A. Overview of Supplementary Material: The supplementary material provides additional details and experimental results complementing the main paper.
- A. Overview of Supplementary Material: A supplementary video presents SLAM results for comparative methods and the proposed method.
B. Ghost-FWL Dataset
This section introduces the Ghost-FWL dataset and summarizes its scene coverage and organization.
- Ghost-FWL includes indoor and outdoor scenes, with per-scene frame counts summarized in Table 5.
C. Fundamentals of LiDAR and Full-Waveform Signals
LiDAR provides compact 3D point clouds from return times, whereas full-waveform LiDAR preserves temporal intensity information useful for identifying reflections and ghosts. The annotation pipeline combines accumulated waveforms with high-precision maps to produce peak-level labels.
- Standard LiDAR outputs dominant return peaks as compact 3D point clouds, reducing downstream data bandwidth.
- Full-waveform LiDAR records each beam’s complete temporal intensity profile, preserving cues from materials, geometry, incidence angle, and multipath reflections.
- Ghost annotation is difficult because virtual reflections lack direct physical ground truth and raw waveforms contain substantial noise.
- Annotation Pipeline: The pipeline accumulates multiple frames into high-SNR waveforms, converts detected peaks into point clouds, and transfers point-cloud labels back to waveform peaks.
- Annotation Pipeline: Ghost-FWL labels reflections as Glass, Object, Ghost, or Noise using manually defined glass and reflection regions aligned with a high-precision map.
- Training annotations are expanded around each peak by its full width at half maximum to reduce label loss during temporal down-sampling.
D. Implementation Details of the FWL-based Ghost Removal Framework
The framework pretrains a masked autoencoder on full-waveform data, then uses its representations for four-class peak classification and ghost removal. Training and inference use fixed waveform preprocessing and sliding-window processing.
- FWL-MAE: FWL-MAE creates patch embeddings, masks temporal bins within sampled spatial patches, and reconstructs the masked waveform regions with Transformer encoder-decoder blocks.
- FWL-MAE: The Peak Head estimates the position, amplitude, and width of up to K = 4 peaks to model physically meaningful waveform properties.
- FWL-MAE: The model uses 70% masking with input size (128, 128, 256), patch size (16, 16, 256), and encoder and decoder dimensions 768 and 384.
- Ghost Detection and Removal: A frozen FWL-MAE encoder feeds a two-layer classification head that predicts Glass, Ghost, Object, and Noise probabilities for waveform coordinates.
- Ghost Detection and Removal: Focal loss addresses severe class imbalance between minority Ghost samples and majority Noise samples.
- Inference: Inference removes ceiling, floor, and internal-reflection bins, then processes sequential nonoverlapping sliding windows whose outputs are merged and up-sampled.
- Evaluation Metrics: Ghost detection is evaluated with peak-level recall, while Ghost Removal Rate measures successfully removed ghost points after conversion to 3D points.
E.1.1. Ghost Classification Evaluation
The proposed waveform-based approach improves ghost classification and downstream SLAM performance relative to competing models and point-cloud denoising strategies. FWL-MAE is especially beneficial with less training data, while real-time full-frame inference remains challenging.
- Ghost Classification: Transformer-based models pretrained with MARMOT or FWL-MAE classify waveform peaks more accurately than the evaluated 3D convolutional models.
- Ghost Classification: Noise peaks are sometimes classified as Ghost, although this has limited downstream impact and leaves classification accuracy open to improvement.
- Ghost Classification: Recall is 0.751, 0.702, and 0.622 at detection thresholds 0.5, 0.6, and 0.7, respectively; threshold 0.5 is used for the main paper.
- FWL-MAE Ablation: FWL-MAE preserves higher recall as training data decreases, widening its performance gap over training without self-supervised pretraining.
- Computational Cost: The Transformer approach achieves higher computational efficiency than 3D CNN methods by processing spatial patches with the temporal dimension as a single tube.
- Computational Cost: Real-time performance remains challenging because full-frame inference requires iterative processing under memory and computational constraints.
- Additional Qualitative Results: The model also detects ghosts from water and metal surfaces despite being primarily trained on glass-induced ghosts.
- SLAM Ablation: Waveform-based removal reduces ATE by 54–76% and RTE by 53–78% versus Statistical Outlier Filter combinations, and yields the highest SLAM accuracy.