Source-linked AI summary
SAVTrack: Selective Vote Aggregation for Reliability-Aware Point Cloud Tracking
Sifan Zhou, Linyue Tan, Qiwei Wang, Ziyu Zhao, Xiaobo Lu
TL;DR
Sparse LiDAR observations create point-to-center votes with uneven reliability, which existing point-based trackers generally do not model before aggregation. SAVTrack uses motion-conditioned Selective Vote Aggregation to filter low-confidence votes before clustering, achieving competitive KITTI and nuScenes tracking at 82 FPS. Its fixed center-relative partition and seed-independent reliability estimates remain scope boundaries for future refinement.
Problem
Sparse and incomplete LiDAR observations make point-to-center hypotheses unevenly reliable, while existing pipelines largely aggregate them without explicitly modeling reliability.
Method
SAVTrack estimates center-relative sub-region posteriors from joint point-wise and inter-frame motion representations, then hard-gates low-confidence votes before proposal clustering.
Results
SAVTrack consistently improves its corresponding voting baseline on KITTI and nuScenes while maintaining real-time inference at 82 FPS.
Takeaways & Limitations
Pre-aggregation reliability filtering prevents unreliable votes from affecting local neighborhood construction and proposal formation while retaining fewer than one-sixth of dense aggregation’s votes.
Takeaways & Limitations
SAV uses a fixed uniform center-relative partition and estimates reliability independently for individual seeds, without explicit set-level geometric or temporal consistency.
Abstract
from arXiv · showhide
3D single object tracking (SOT) in LiDAR point clouds is essential for autonomous systems, but remains challenging under sparse and incomplete observations. In such cases, different target points provide highly uneven constraints on the object center, causing some point-to-center votes to be substantially less reliable than others. Existing point-based trackers typically aggregate these hypotheses without explicitly modeling their reliability, allowing inaccurate votes to contaminate proposal clustering and degrade localization accuracy. To address this issue, we propose \textbf{SAVTrack}, a motion-aware tracking framework with \textbf{Selective Vote Aggregation (SAV)}. SAVTrack estimates the reliability of each candidate vote from both local seed features and inter-frame motion context, and removes low-confidence hypotheses before proposal clustering. This pre-aggregation gating prevents unreliable hypotheses from affecting cluster formation while introducing only modest computational overhead. SAVTrack achieves competitive performance on KITTI and nuScenes, reaching 68.4/87.4 and 58.44/69.82 Success/Precision, respectively, while running at 82 FPS. It retains fewer than one-sixth of the candidate votes used by dense aggregation and remains particularly effective under sparse target observations.
1. Introduction
3D LiDAR tracking faces uneven vote reliability under sparse observations, yet conventional point-based pipelines aggregate hypotheses indiscriminately. SAVTrack estimates reliability from local and temporal motion cues, gates low-confidence votes before clustering, and reports competitive real-time tracking.
- Point-to-center hypotheses vary in reliability because planar or ambiguous regions provide weaker geometric constraints than structurally distinctive regions.
- Existing point-based tracking pipelines largely overlook individual vote reliability, allowing inaccurate hypotheses to affect proposal aggregation.
- SAV predicts center-relative sub-region posteriors from joint point-wise and inter-frame motion representations, using candidate probabilities as confidence scores.
- Hard pre-aggregation gating removes low-confidence votes before clustering, preventing them from affecting local neighborhood construction and proposal formation.
- SAVTrack achieves competitive KITTI and nuScenes tracking at 82 FPS while retaining fewer than one-sixth of dense aggregation’s candidate votes.
2. Related Work
Related 3D SOT work includes Siamese appearance matching and motion-based tracking, while point-based methods preserve local evidence through voting. SAVTrack’s overview combines motion context with selective vote aggregation before proposal formation.
- Siamese-based 3D Single Object Tracking: Siamese-based trackers match template and search features, but sparse or incomplete observations can make appearance matching vulnerable to distractors.
- SAVTrack’s architecture uses motion-aware seed features, candidate vote generation, confidence gating, and vote clustering for end-to-end target-box prediction.
- Motion-based 3D Single Object Tracking: Motion-based trackers estimate target state from inter-frame motion rather than relying primarily on template–search appearance matching.
3. Method
SAVTrack combines motion-aware point features with Selective Vote Aggregation to estimate candidate-vote reliability and filter unreliable hypotheses before proposal clustering. The retained votes are then clustered and refined into target proposals.
- 3.1. 3D SOT Task Definition: SAVTrack encodes current and previous point clouds with shared PointNet++ features, fuses them into inter-frame motion context, and broadcasts that context to sampled current-frame seeds.The resulting motion-aware seed representation preserves local evidence while incorporating temporal target dynamics.
- 3.3. Selective Vote Aggregation: SAV retains candidate votes whose posterior confidence exceeds a threshold, using that confidence as a proxy for vote reliability before aggregation.This hard pre-aggregation gating changes the candidate set and prevents discarded votes from affecting neighborhood construction during proposal clustering.
- 3.3. Selective Vote Aggregation: Each seed predicts a posterior over predefined center-relative sub-regions and regresses candidate center offsets for those regions.The classifier is supervised by ground-truth sub-region labels, while vote regression uses the correct-region offset with an L1 loss.
- 3.4. Proposal Generation and Training: Filtered high-confidence votes are clustered by spatial neighborhood, then a proposal head predicts each cluster’s target state and objectness score.Proposal states contain the target center and heading angle, while target dimensions are inherited from the initialized box and remain fixed.
- 3.4. Proposal Generation and Training: The inference pipeline samples seeds with FPS, applies posterior thresholding with an empty-vote fallback, clusters retained votes, and returns the refined bounding-box prediction.All components are optimized jointly in an end-to-end manner.
4. Experiments
Experiments on KITTI and nuScenes show that SAVTrack combines competitive accuracy with efficient selective voting, while confidence filtering improves localization and remains effective under sparse observations.
- Comparison on KITTI: 68.4/87.4 mean Success/Precision on KITTI at 82 FPS demonstrates competitive accuracy with a lightweight point-based representation.SAVTrack improves over P2P-point from 66.2/85.4 to 68.4/87.4 while retaining point-level seed hypotheses.
- Comparison on nuScenes: 58.44% mean Success and 69.82% mean Precision on nuScenes improve Success over MBPTrack while maintaining nearly identical Precision.SAVTrack also achieves the best Success/Precision on Truck, Trailer, and Bus and the second-best results on Car.
- Ablation Study: 68.4/87.4 is achieved by SAV while retaining approximately 250 high-confidence votes, outperforming dense, soft-weighted, and Top-1 aggregation.SAV improves over Top-1 by 1.6/1.1 on Car and 1.6/1.8 on Pedestrian, showing that selective removal preserves informative candidates.
- Component Analysis: Confidence filtering raises mean performance from 67.2/87.0 to 68.4/87.4 and reduces retained votes from approximately 1,536 to 250.The gains are observed on both Car and Pedestrian, supporting suppression before proposal clustering rather than multi-hypothesis regression alone.
- Confidence Threshold Sensitivity: The default threshold τ=0.3 achieves 68.4/87.4 with approximately 250 votes, whereas τ=0.5 slightly degrades performance after further reducing the vote set.This indicates that overly aggressive filtering can discard informative hypotheses.
- Confidence and Geometry Analysis: Higher posterior confidence corresponds to lower mean center error, declining from 0.72 m to 0.13 m, and a higher successful-vote ratio, rising from 28.3% to 89.4%.Confidence is also associated with distinctive geometry: planarity correlates negatively with confidence and positively with vote error, while curvature, normal variation, and entropy show opposite trends.
- Sparsity Analysis: Point-count-stratified evaluation isolates observation sparsity by grouping frames into six foreground-point bins and comparing tracking performance across them.The analysis is designed to separate sparsity effects from category-specific size, shape, and occlusion factors.
- Motion-Conditioned Reliability Estimation: Inter-frame motion cues improve static-only reliability estimation from 69.1/82.0 to 71.1/83.9 on Car and from 62.8/89.4 to 65.1/91.8 on Pedestrian.With the same motion-aware representation, hard gating also outperforms soft weighting by 1.9/1.7 on Car and 2.0/2.3 on Pedestrian.
5. Conclusion
SAVTrack addresses heterogeneous vote reliability by filtering unreliable hypotheses before proposal clustering, improving voting-based 3D tracking while retaining real-time inference. Its current fixed partition and seed-independent reliability estimates leave adaptive and consistency-aware extensions open.
- SAVTrack uses Selective Vote Aggregation to predict vote confidence from point-wise and inter-frame motion representations, filtering unreliable votes before clustering.The method is lightweight and applies hard pre-aggregation gating to prevent unreliable hypotheses from affecting proposal formation.
- 82 FPS real-time inference is maintained while SAVTrack consistently improves its corresponding voting baseline on KITTI and nuScenes.Analyses report stronger gains under sparse target observations and show that confidence predicts vote localization quality and correlates with local geometry.
- SAV currently uses a fixed uniform center-relative partition that may miss anisotropic or object-dependent center distributions.Adaptive or hierarchical partitions are proposed as potential improvements to hypothesis modeling.
- Independent per-seed reliability estimates do not explicitly model geometric consensus among retained hypotheses, motivating set-level or temporal consistency checks.The paper identifies these extensions as relevant to robustness under severe sparsity and occlusion.
CrediT authorship contribution statement
The CrediT statement assigns authors distinct contributions spanning investigation, methodology, software, validation, analysis, writing, visualization, project administration, funding, and related activities.
- Sifan Zhou led investigation, methodology, software, validation, writing, project administration, and related editorial work, while the other authors contributed across methods, validation, analysis, visualization, writing, funding, and administration.The statement attributes funding acquisition to Xiaobo Lu and visualization to Linyue Tan.