Source-linked AI summary
Exploring Simple 3D Multi-Object Tracking for Autonomous Driving
Chenxu Luo, Xiaodong Yang, Alan Yuille
TL;DR
LiDAR multi-object tracking commonly depends on heuristic association and manually tuned track-life rules. SimTrack jointly detects and tracks by reading identities from first-appear locations and updating positions with estimated motion. Experiments report that it compares favorably with existing methods while avoiding heuristic matching.
Problem
LiDAR tracking-by-detection methods require heuristic association and track-life rules that are difficult to tune and can be data- and model-dependent.
Source: §1 · p. 1Method
SimTrack predicts objects’ first-appear locations in a hybrid-time centerness map, reads off identities, and updates tracked locations with estimated motion.
Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 4Results
Takeaways & Limitations
The authors present SimTrack as a first step toward simpler and robust tracking systems for autonomous driving.
Source: §5 · p. 8
Abstract
from arXiv · showhide
3D multi-object tracking in LiDAR point clouds is a key ingredient for self-driving vehicles. Existing methods are predominantly based on the tracking-by-detection pipeline and inevitably require a heuristic matching step for the detection association. In this paper, we present SimTrack to simplify the hand-crafted tracking paradigm by proposing an end-to-end trainable model for joint detection and tracking from raw point clouds. Our key design is to predict the first-appear location of each object in a given snippet to get the tracking identity and then update the location based on motion estimation. In the inference, the heuristic matching step can be completely waived by a simple read-off operation. SimTrack integrates the tracked object association, newborn object detection, and dead track killing in a single unified model. We conduct extensive evaluations on two large-scale datasets: nuScenes and Waymo Open Dataset. Experimental results reveal that our simple approach compares favorably with the state-of-the-art methods while ruling out the heuristic matching rules.
1. Introduction
LiDAR tracking-by-detection relies on heuristic association and track-life rules that are difficult to tune and generalize. SimTrack replaces these steps with a unified, trainable approach for association, new-track detection, and dead-track removal.
1. Introduction
- 3D multi-object tracking in LiDAR point clouds supports autonomous-driving modules including perception, prediction, and planning.The paper identifies LiDAR as the most commonly used sensor for self-driving vehicles to perceive their surroundings.Source: §1 · p. 1
- Existing LiDAR tracking-by-detection methods require hand-crafted association and track-life rules whose hyperparameters demand substantial tuning and are data- and model-dependent.Track-life management includes delaying initialization to filter false detections and retaining disappeared objects to handle occlusion.Source: §1 · pp. 1–2
- Some methods predict object motion or locations across frames, but use forecasts only to bridge matching and do not model association confidence.They can remain prone to spurious detections and require manually chosen retention durations; other methods re-detect current-frame objects and still need matching to identify newborns.Source: §1 · p. 2
- SimTrack jointly links tracked objects, detects newborns, and removes dead tracks in one forward pass without heuristic matching.It predicts first-appear locations on a hybrid-time centerness map and uses estimated motion to update object locations.Source: Figure 1 · p. 1, §1 · p. 2
- The authors present SimTrack as the first learning paradigm for LiDAR point-cloud tracking that removes heuristic matching and simplifies the tracking system.The model is end-to-end trainable for joint detection and tracking, and the authors report that it compares favorably with existing methods.Source: §1 · p. 2
2. Related Work
Prior multi-object tracking work commonly detects objects frame by frame and then associates them, while newer joint approaches use learned queries or regress object locations. LiDAR methods chiefly rely on motion and commonly retain a matching step.
2. Related Work
- Image-sequence tracking-by-detection detects objects in individual frames and associates detections over time, commonly using motion or appearance cues.Motion-based methods can use temporal modeling and Kalman filtering; appearance-based methods use re-identification features.Source: §2 · p. 2
- Trackor regresses past object locations to current positions, but relies on an additional detector for newborns and heuristic rules to separate them from tracked objects.SimTrack instead produces tracked and newborn objects in a single forward pass without heuristic post-processing.Source: §2 · p. 2
- Transformer-based methods use learned track and object queries to follow objects and handle newborns or associate current-frame detections.TrackFormer follows locations autoregressively; TransTrack uses a query-key mechanism.Source: §2 · p. 2
- Most LiDAR tracking methods use tracking-by-detection and require bipartite matching, such as Hungarian or greedy matching, for final tracking output.Methods described include Kalman-filter-based 3D motion estimation, velocity regression, prediction, and learned affinity matrices.Source: §2 · pp. 2–3
3. Method
SimTrack combines detection and tracking in an end-to-end model, using first-appear locations and motion updates to avoid heuristic matching. Its unified design supports tracked-object association, new-object detection, and dead-object removal.
3. Method
- SimTrack unifies tracked-object linking, new-object detection, and dead-object removal in one end-to-end trainable model.It combines a hybrid-time centerness map with a motion-updating branch to avoid heuristic matching.Source: §3 · p. 3, §3.3 · p. 4
3.1. Preliminary
- Objects are represented by centers on a centerness map, with a Gaussian heatmap around each center as the detection target.The point-cloud pipeline voxelizes pillars or voxels, extracts features, and computes BEV features before the detection head predicts centers, offsets, size, and heading.Source: §3.1 · p. 3
3.2. Overview
- The model uses a point-cloud snippet to predict each object's first-appear location and motion, then reads its identity from the previous centerness map and updates its location.Past sweeps are transformed into the current coordinate system using ego-motion compensation; points also carry relative timestamps.Source: §3.2 · p. 4
3.3. Joint Detection and Tracking
- The hybrid-time map assigns tracked objects to their first-appear centers, treats objects disappearing after the first frame as negatives, and detects objects appearing in the second frame.The same map supports association, dead-object removal by confidence thresholding, and new-object detection; its confidence can represent both detection and association confidence.Source: §3.3 · p. 4
- SimTrack also regresses object height, box size, and heading alongside motion.Heading is represented as (sin θ, cos θ), where θ is the bounding-box yaw angle.Source: §3.3 · p. 4
- At inference, SimTrack reuses an identity at a matching center, initializes tracks for remaining centers, removes low-confidence objects, and uses predicted motion to update locations.The motion branch regresses the offset from an object's first-frame center to its current center; unlike CenterPoint, SimTrack uses motion to produce tracking output rather than as a matching bridge.Source: §3.3 · pp. 4–5
4. Experiments
Experiments compare SimTrack with tracking methods on nuScenes and Waymo, assess design choices, and report results on validation and test sets.
4.1. Datasets; 4.2. Evaluation Metrics
- The evaluation uses nuScenes and Waymo with their official tracking metrics: AMOTA averaged across recalls for nuScenes and MOTA for Waymo.nuScenes evaluates seven moving classes using a 2 m BEV center-distance threshold; Waymo evaluates vehicles at 3D IoU 0.7 and reports LEVEL 1 and LEVEL 2.Source: §4.2 · p. 5, §4.1 · p. 5
4.4. Results on nuScenes
- SimTrack exceeds CenterPoint and Kalman filter by 3.5% and 8.8% with a pillar backbone, and 2.4% and 7.1% with a voxel backbone on nuScenes validation.The gains remain around 2% when the baselines use SimTrack's detection results, indicating that the tracking design also contributes.Source: Table 1 · p. 6, §4.4 · p. 6
- On the nuScenes test set, SimTrack without test-time augmentation outperforms enhanced CenterPoint; for cars, IDS falls from 315 to 214 and FRAGS from 296 to 186.The compared CenterPoint uses deformable convolutions and test-time augmentation.Source: Table 2 · p. 6, §4.4 · p. 7
4.3. Implementation Details
- Models train for 20 epochs on nuScenes and 12 on Waymo, using AdamW and one-cycle learning-rate scheduling, with global rotation/scaling, axis flips, and 3D object cut-and-paste augmentation.Training uses PyTorch on 8 TITAN RTX GPUs, with batch sizes of 8 and 4 per GPU for nuScenes and Waymo, respectively.Source: §4.3 · p. 6
4.5. Ablation Studies
- A unified map outperforms separated tracking and newborn-detection maps, and averaging current and previous updated centerness maps improves results across classes.The authors hypothesize that newborn objects' small share of a scene makes training separate maps difficult; averaging is described as effective temporal fusion, particularly for occlusion.Source: Table 3 · p. 7, §4.5 · p. 7
- SimTrack retains an occluded car using prior-map confidence and motion updating, while CenterPoint loses its identity; the qualitative example also contrasts their handling of inaccurate pedestrian velocity.The figure colors encode object identity over time; the orange car is unannotated in ground truth because of occlusion.Source: §4.5 · p. 7, Figure 3 · p. 8
4.6. Results on Waymo
- On Waymo vehicle tracking, SimTrack is better than or on par with CenterPoint across metrics and improves consistently across the two datasets without heuristic matching or complex track-life management.The reported Waymo results use a pillar backbone and are shown for LEVEL 1 / LEVEL 2.Source: Table 4 · p. 8, §4.6 · p. 8
4.5. Ablation Studies
- Higher-resolution maps improve pedestrian and motorcycle tracking, while pillar SimTrack reduces mAVE by 33% overall and 61% for motorcycles versus CenterPoint.The resolution increase changes decoder downsampling from 4 to 2 while keeping the encoder intact.Source: §4.5 · p. 8
5. Conclusion
SimTrack combines hybrid-time centerness and motion updating in an end-to-end model that integrates association, newborn detection, and dead-object removal. Experiments demonstrate its efficacy for LiDAR 3D multi-object tracking.
5. Conclusion
- SimTrack combines a hybrid-time centerness map with a motion-updating branch to integrate tracked-object association, newborn detection, and dead-object removal.The paper presents this as a single unified model for 3D multi-object tracking in LiDAR point clouds.Source: §5 · p. 8
- The authors frame SimTrack as a first step toward simplifying tracking pipelines that use complex heuristic matching and manual track-life management.Source: §5 · p. 8
- Extensive experimental results demonstrate the efficacy of the approach.Source: §5 · p. 8
Appendix
The appendix outlines further analysis of heuristic matching, CenterPoint comparisons, inference latency, and results on nuScenes and Waymo.
Appendix
- The appendix covers matching heuristics and their hyper-parameters, further CenterPoint comparisons, inference latency, and additional nuScenes and Waymo results.Source: Appendix · p. 9
A. Heuristic Matching and Hyper-Parameters
The appendix examines how heuristic matching settings affect tracking performance, including sensitivity to covariance and CenterPoint matching parameters. It contrasts those tuning requirements with SimTrack’s heuristic-free approach.
A. Heuristic Matching and Hyper-Parameters
- Matching thresholds, the number of frames to retain dead tracks, and the minimum frames before initializing new tracks are examples of heuristic-matching hyper-parameters.Source: A. Heuristic Matching and… · p. 9
- AMOTA on nuScenes validation rises from 37.1 with the default Kalman covariance matrix to 51.2 after tuning it using prediction-error statistics [6].Source: A. Heuristic Matching and… · p. 9
- CenterPoint’s maximum-age and maximum-distance settings significantly affect tracking performance, while SimTrack is heuristic-free and achieves better performance in the comparison.Table 5 reports validation results for nuScenes cars using the pillar-based backbone; the passage does not provide its cell values.Source: A. Heuristic Matching and… · p. 9, Table 5 · p. 9
B. More Comparisons with CenterPoint
Comparisons with CenterPoint show that SimTrack has fewer identity switches at high recall rates and superior MOTA at high recall rates across the reported classes.
B. More Comparisons with CenterPoint
- At high recall rates, SimTrack has much less identity switching and superior MOTA than CenterPoint across car, pedestrian, and motorcycle results.The authors attribute the lower identity-switch count to CenterPoint’s vulnerability to false-positive detections and SimTrack’s joint detection-and-tracking design.Source: B. More Comparisons with CenterPoint · p. 9
C. Inference Latency
SimTrack can be added to a 3D object detection network with only a small inference-latency increase. The comparison uses pillar- and voxel-based backbones and different centerness-map resolutions.
C. Inference Latency
- 1–2 ms: SimTrack increases detection-only model inference latency by only 1–2 ms.Latency was measured on a single TITAN RTX GPU.Source: C. Inference Latency · p. 9
- SimTrack’s joint detection and tracking design can be incorporated into a 3D object detection network.The comparison covers pillar- and voxel-based backbones and different centerness-map resolutions.Source: C. Inference Latency · p. 9
- Inference latency was measured on a single TITAN RTX GPU.Source: C. Inference Latency · p. 9
D. More Results on nuScenes and Waymo
The section reports improved nuScenes detection performance for SimTrack relative to CenterPoint and presents Waymo tracking results using dynamic voxelization. It also compares IDS and MOTA across recall rates and centerness-map resolutions.
D. More Results on nuScenes and Waymo
- SimTrack significantly improves detection performance over CenterPoint on nuScenes, with mAP and NDS reported for all classes on the validation and test sets.The test-set CenterPoint result uses its enhanced version.Source: D. More Results on nuScenes and Waymo · p. 9, Table 7 · p. 9
- Figure 4 compares IDS and MOTA for SimTrack and CenterPoint across recall rates, with centerness-map resolutions of 0.4 m and 0.8 m.Ours-0.8 is the default resolution.Source: Figure 4 · p. 10
- On Waymo’s validation set, the tracking results use a pillar-based backbone with dynamic voxelization replacing the hard voxelization used in the other experiments.The table reports LEVEL 1 / LEVEL 2 results.Source: D. More Results on nuScenes and Waymo · p. 10, Table 8 · p. 10