Source-linked AI summary
Mapless Online Detection of Dynamic Objects in 3D Lidar
David J. Yoon, Tim Y. Tang, Timothy D. Barfoot
TL;DR
Live-data lidar detection avoids reliance on object classes or maps, but is limited to objects moving in the current scene. This paper uses motion-compensated pointcloud and freespace processing to label points dynamic or static, and reports benchmark and real-data evaluations alongside a public motion-distorted dataset.
Problem
Live-data methods avoid object and setting priors but cannot detect objects that are stationary in the current scene, while point-level ground-truth data are difficult to obtain.
Source: §I · p. 1, A. Simulated Lidar Benchmark (CARLA) · p. 5Method
The method uses live lidar data, compensates for motion distortion in pointclouds and freespace querying, and labels points dynamic or static.
Source: §I · p. 1, §III · p. 2Results
The pipeline works well in scenarios with consistent motion, and the real-data examples show 22 different dynamic objects.
Source: C. Experiments on Real Data · p. 6Takeaways & Limitations
The paper establishes a public CARLA benchmark with motion-distorted lidar data and point-level ground truth on moving objects.
Source: §I · p. 1, §V · p. 6The method cannot distinguish maximum-range measurements from faulty returns, making objects without lidar-perceivable geometry behind them, such as flying objects, an issue.
Source: C. Experiments on Real Data · p. 6
Abstract
from arXiv · showhide
This paper presents a model-free, setting-independent method for online detection of dynamic objects in 3D lidar data. We explicitly compensate for the moving-while-scanning operation (motion distortion) of present-day 3D spinning lidar sensors. Our detection method uses a motion-compensated freespace querying algorithm and classifies between dynamic (currently moving) and static (currently stationary) labels at the point level. For a quantitative analysis, we establish a benchmark with motion-distorted lidar data using CARLA, an open-source simulator for autonomous driving research. We also provide a qualitative analysis with real data using a Velodyne HDL-64E in driving scenarios. Compared to existing 3D lidar methods that are model-free, our method is unique because of its setting independence and compensation for pointcloud motion distortion.
I. INTRODUCTION
The paper targets live-data lidar detection that does not depend on object classes or maps, while addressing motion distortion in spinning-lidar scans. It introduces point-level dynamic/static labels, a motion-compensated freespace method, and a simulated benchmark with point-level ground truth.
I. INTRODUCTION
- Live-data methods avoid object priors and maps but cannot detect objects that are stationary in the current scene, even if they may later move.The paper gives stationary cars in traffic as an example.Source: §I · p. 1, §II · p. 2
II. RELATED WORK
- The paper distinguishes class-specific detectors, map-based methods, and live-data methods, noting that class-specific detectors do not detect objects outside their training classes.The categories are complementary, and combinations can be effective.Source: §I · p. 1, §II · p. 2
I. INTRODUCTION
- The method labels each point dynamic or static using live data, with motion compensation for pointclouds and freespace querying, and is designed to be setting-independent.It is model-free and isotropic, and does not use maps or training data; the authors say it can be combined with strategies from other categories when setting independence is not needed.Source: §I · p. 1, §II · p. 2
- The CARLA benchmark provides motion-distorted lidar data with point-level ground truth on moving objects, addressing the difficulty of obtaining such labels.The authors believe it is the first benchmark to provide these data; they also qualitatively evaluate real driving data collected with a Velodyne HDL-64E.Source: §I · p. 1
II. RELATED WORK
- Map-based methods compare observations with a stationary-world map, but freespace querying by Pomerleau et al. assumes pointclouds are not motion distorted.The paper notes that compensation for occupancy-voxel-grid methods is straightforward with a continuous-time trajectory.Source: §II · p. 2
- Prior live-data methods using pointcloud comparisons face occlusion and sparsity challenges, while cited approaches may rely on ground removal or unpublished manual labels.Dewan et al. remove ground points; Postica et al. use short manually annotated KITTI sequences that are not public and ignore measurements beyond 30 m.Source: §II · p. 2
III. METHODOLOGY
The pipeline labels a query scan point by point, treating a scan as both pointcloud and freespace information. It aligns scans, compares pointclouds, checks candidate dynamic points against freespace, then filters and clusters them.
III. METHODOLOGY
- A scan is a full lidar revolution and includes both pointcloud and freespace information; the pipeline labels every point in a query scan.The laser ray paths that form freespace are not equivalent to endpoint azimuth and elevation computations in a local reference frame.Source: §III · p. 2
- The methodology section presents the pipeline as sequential steps and develops each step’s role and contribution in turn.The steps are odometry, pointcloud comparison, freespace check, box filtering, and region growth.Source: §III · p. 2
- The pipeline aligns the latest scan, compares pointclouds to mark discrepancies dynamic, and checks those points against another scan’s freespace.Points not in freespace are changed back to static; a sliding box filter then rejects outliers, and region growth clusters dynamic points and adds nearby points that meet same-object conditions.Source: §III · p. 2
A. Odometry
Lidar odometry provides a continuous-time sensor trajectory that aligns measurements using their exact timestamps. The pointcloud comparison then uses motion-compensated scans to label points as dynamic or static, with scan gap and error threshold shaping which objects can be detected.
A. Odometry
- Continuous-time lidar odometry uses exact measurement timestamps to produce a sensor trajectory and query pose or velocity at any timestamp.The trajectory aligns the latest scan.Source: A. Odometry · p. 2, Figure 2 · p. 3
B. Pointcloud Comparison
- Pointcloud comparison motion-compensates query and reference scans, then labels points with error above a constant threshold dynamic and the rest static.It uses point-to-plane error when surface normals can be computed and point-to-point error otherwise; viewpoint occlusions and new surface observations can also produce high error.Source: B. Pointcloud Comparison · p. 3
- The scan gap, error threshold, and lidar scan rate together define the minimum speed of detectable objects.Because object speed and size are unknown, the method aims to partially label objects and recover the rest later in the pipeline.Source: B. Pointcloud Comparison · p. 3
C. Freespace Check
The freespace check tests dynamic query points against reference-scan laser rays to refine labels from pointcloud comparison. It uses motion-compensated ray geometry and a backward-then-conditional-forward checking strategy.
C. Freespace Check
- The check tests dynamic query points, potentially up to half the scan, against another scan's freespace; inside points are consistent with dynamic labels, while border or outside points may be mislabelled.Static points are not checked because pointcloud comparison is equivalent to a freespace border check; the three ray cases distinguish inside, border, and outside freespace.Source: C. Freespace Check · pp. 3–4
- Rather than represent all freespace with costly occupancy voxels, the method selects the reference ray nearest each query point using point-to-line distance.The query uses the lidar's approximately outward-radiating laser arrangement and elevation order to speed the search.Source: C. Freespace Check · pp. 3–4
- The nearest-ray strategy compensates for sensor-platform motion with a continuous-time trajectory, unlike the ideal-snapshot assumption described for Pomerleau et al..The formulation assumes lasers rotate together at constant speed and have unique poses relative to the sensor hub.Source: C. Freespace Check · p. 4
- The search iteratively minimizes point-to-line distance by selecting a laser and solving for time, then checking elevation-neighbouring lasers while error decreases.It initializes time and laser using Pomerleau et al.'s ideal pointcloud method, which the authors report experimentally always converges.Source: C. Freespace Check · p. 4
- For a selected ray, the method checks its intersection with the query point's surface plane to determine whether the point is on the freespace border.It checks backward against a previous scan, then checks forward only for points outside backward freespace; forward checking handles objects moving away from the sensor.Source: C. Freespace Check · p. 4
D. Box Filter
The box filter removes outlier dynamic labels from the scan's image representation using a pixelwise XNOR score and a lidar-resolution-dependent threshold.
D. Box Filter
- The filter represents each laser as an image row and consecutive measurements as columns, then slides a box filter to remove sparse dynamic mislabels.Dynamic and static labels are encoded as 1 and 0, respectively.Source: D. Box Filter · p. 4
- Pixelwise XNOR operations produce a score, and scores above a constant threshold are treated as outliers; the threshold depends on lidar resolution.The filter is intended to remove mislabelled dynamic points while retaining correctly labelled ones.Source: D. Box Filter · p. 4
E. Region Growth
Region growth addresses objects that have not fully displaced from their previous position during the scan gap. The method clusters dynamic points and grows clusters using pointcloud segmentation.
E. Region Growth
- When the scan gap is too small for an object to fully displace, the pipeline uses region growth.Source: E. Region Growth · p. 4
- Dynamic query points are clustered with a radially bounded nearest-neighbour strategy, then grown by testing neighbouring points for parallelism or convexity.Growth continues until no qualifying neighbours are found.Source: E. Region Growth · p. 5
A. Simulated Lidar Benchmark (CARLA)
The benchmark addresses the difficulty of obtaining public lidar data with accurate point-level ground truth by constructing motion-distorted simulated sequences in CARLA. It defines dynamic points by speed and evaluates detection with total and average precision and recall, including under varied noise and range limits.
A. Simulated Lidar Benchmark (CARLA)
- The benchmark fills a data gap: KITTI lacks point-level labels and raw lidar, while Paris-Lille-3D provides motion-compensated data and removes points beyond 20 m.Comparable publications either omit quantitative results or use short manually labelled sequences that are not public.Source: A. Simulated Lidar Benchmark (CARLA) · p. 5
- CARLA sequences model a Velodyne HDL-64E with 128,000 measurements per scan, a 120 m maximum range, and a 10 Hz frequency.The benchmark modifies CARLA to match the sensor's laser positions and orientations and produce motion distortion; sequences cover 2.9 km of roads in Town 1 and 1.9 km in Town 2.Source: A. Simulated Lidar Benchmark (CARLA) · p. 5
- Points moving faster than 0.2 m/s are labelled dynamic for ground truth; other points are static.True positives are correctly labelled dynamic points, while false positives and false negatives are incorrectly labelled dynamic and static, respectively.Source: A. Simulated Lidar Benchmark (CARLA) · p. 5
- Precision and recall are computed both as totals across scans and as averages over valid scans; scans with zero denominators are excluded.The valid scan counts for precision and recall are distinguished because their denominators can be zero on different scans.Source: A. Simulated Lidar Benchmark (CARLA) · p. 5
- Figure 6 shows total and average precision-recall plots for two simulated sequences as the standard deviation of range-measurement noise varies; a Velodyne HDL-64E is rated below 0.02 m.Source: Figure 6 · p. 5
- Figure 7 compares total and average recall using ground-truth labels across two sequences as the limited range varies.The total recall is high at low range limits because nearby objects contribute more points, downplaying missed distant objects.Source: Figure 7 · p. 5
B. Benchmark Results
The authors report that their total precision-recall values on longer simulated sequences are comparable to Dewan et al.'s real-data results, while noting that a fair comparison is not possible. The results also show sensitivity to noise, scan weighting, occlusion, and detection latency.
B. Benchmark Results
- Figure 8 shows 22 real-data dynamic objects, including occasional false detections, occlusion misses, and region-growth errors linked to inaccurate surface normals.False detections rarely persist into the next scan; faulty returns also make one vehicle barely visible.Source: Figure 8 · p. 6
- Greater range noise reduces performance, while averaging downplays scans with more mistakes, including scans containing very close objects.Figure 6 varies range-measurement noise; total curves are more affected because point counts on nearby objects can dominate.Source: B. Benchmark Results · pp. 5–6
- Town 2 performs worse with more partial occlusions, and the scan gap delays detection of objects accelerating from stationary, reducing recall.The authors identify occlusions such as fences as a difficulty.Source: B. Benchmark Results · p. 6
- The authors judge total precision-recall values on longer simulated sequences comparable to Dewan et al.'s real-data results, but say a fair comparison is impossible.Dewan et al. used manually labelled real sequences and a different, unclear precision-recall computation.Source: B. Benchmark Results · p. 6
C. Experiments on Real Data
Real-data experiments use a vehicle-mounted Velodyne HDL-64E with lidar odometry and show examples of 22 dynamic objects in scenarios with consistent motion. The pipeline has limitations involving faulty returns, occlusion, surface normals, and processing speed.
C. Experiments on Real Data
- The system cannot distinguish maximum-range measurements from faulty returns, limiting freespace computation for objects without geometry behind them, such as flying objects.The authors say this is not detrimental for ground-based applications because Velodyne lasers are angled downward toward the ground.Source: C. Experiments on Real Data · p. 6
- The real-data examples cover 22 dynamic objects, and the pipeline works well in scenarios with consistent motion such as no traffic slow-downs.The sensor was mounted on a vehicle driven through Richmond Hill, Ontario, with lidar odometry estimating motion.Source: C. Experiments on Real Data · p. 6
- Occlusion and inaccurate surface normals impair detections, causing missed objects, incomplete region growth, or growth into static points.A vehicle is also barely visible in one example because of many faulty returns.Source: C. Experiments on Real Data · p. 6
- On a single-thread Intel Core i7-6820HQ laptop, processing averages 3 Hz, below the Velodyne's 10 Hz scan rate.The paper reports this as the pipeline's current processing speed.Source: C. Experiments on Real Data · p. 6
V. CONCLUSIONS AND FUTURE WORK
The paper presents online, point-level dynamic/static labeling for 3D lidar that compensates for motion distortion and uses only recent scans without maps or training data. It also establishes a simulated benchmark, while noting difficulty detecting objects without geometry behind them.
V. CONCLUSIONS AND FUTURE WORK
- The method labels 3D lidar points as dynamic or static online while compensating for motion distortion and relying only on recent scans.It is model-free and setting-independent.Source: §V · p. 6
- The paper establishes and makes public a benchmark of simulated motion-distorted lidar data with point-level ground truth for dynamic objects.Source: §V · p. 6
- Objects without geometry behind them, such as flying objects, remain difficult to detect because maximum-range and faulty returns cannot be distinguished.Source: §V · p. 6