PaperlayerView exact revision on arXiv

Source-linked AI summary

TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking

Zhaofeng Hu, Sifan Zhou, Jiahao Nie, Ziyu Zhao, Weizi Li, Ci-jyun Liang

arXiv:2609.07738v1cs.CVcs.AI

TL;DR

LiDAR-based 3D SOT must track objects across sparse point-cloud frames, but existing approaches use costly dual inputs or complex motion models. TFTrack instead tracks from the current frame using prior box geometry, and reports competitive benchmark accuracy with lower computation and high speed.

  • Problem

    Existing 3D SOT methods rely on costly dual-input processing or complex multi-frame motion modeling, limiting efficiency for resource-constrained robotic platforms.

    Source: §I · p. 1

  • Method

    TFTrack uses only the current point cloud, guided by the prior bounding-box center and size, and instantiates the framework with point, voxel, and pillar representations.

    Source: §I · p. 2, 4) Box-Conditioned Feature Encoding · p. 4

  • Results

    TFTrack reports competitive accuracy on KITTI and nuScenes while trimming inference FLOPs by roughly 50% and reaching approximately 120 FPS.

    Source: §I · p. 2, 1) Results on KITTI dataset · p. 5, 2) Results on nuScenes dataset · p. 5

  • Takeaways & Limitations

    The authors conclude that the single-frame design's competitive accuracy and efficiency make TFTrack well-suited for onboard deployment in embedded systems.

    Source: §I · p. 2, §V · p. 7

Abstract

from arXiv · show

LiDAR-based 3D Single Object Tracking (3D SOT) is critical for robotic perception and navigation and aims to localize dynamic objects across frames in sparse point clouds. Existing methods, rooted in the Siamese tracking paradigm from 2D vision, rely on costly dual-input designs and excessive motion modeling guided by template priors, hindering their efficiency. Our in-depth analysis reveals: (i) the template paradigm is redundant, as the previous bounding box center encodes sufficient historical context; (ii) complex motion modeling is unnecessary, as geometric alignment provides adequate motion priors. Based on the above findings, we propose the first Template-Free Tracking framework (TFTrack). The novel framework eliminates the need for template-search pairings and operates directly on the current frame guided solely by the prior bounding box center and size. We instantiate this paradigm into three variants: TFTrack-Voxel, TFTrack-Pillar, and TFTrack-Point, to explore different 3D representations under a unified framework, ensuring flexibility across sparse and dense scenes. Extensive experiments on KITTI and nuScenes benchmarks show that TFTrack is competitive with leading template-based trackers, while reducing FLOPs by approximately 50% and running at approximately 120 FPS. By simplifying overcomplicated motion-centric designs, TFTrack establishes a new minimalist paradigm for efficient 3D point cloud tracking, paving the way for real-time and resource-efficient deployment in embedded robotic systems, such as autonomous vehicles. The code is available at https://github.com/tftrack-anonymous/TFTrack/tree/main.

I. INTRODUCTION

Existing 3D SOT trackers rely on costly dual-input processing or complex motion modeling, while experiments suggest dense template point clouds contribute little in the evaluated setting. TFTrack replaces these designs with a template-free, single-frame framework that uses a prior box center and offers three representations.

I. INTRODUCTION

  • Dual-input Siamese trackers duplicate feature extraction, while motion-centric methods can overcomplicate modeling, limiting efficiency for resource-constrained platforms.LiDAR sparsity and occlusion also weaken appearance representations and robustness.Source: §I · p. 1
  • Average success drops by less than 1% until the template mask ratio reaches 80%, indicating substantial template redundancy in the evaluated motion-centric tracker.The ablation feeds M2Track the current frame while zeroing out template input at different mask ratios.Source: §I · p. 2, Table I · p. 2, Figure 2 · p. 2
  • TFTrack processes only the current point cloud, using the prior bounding-box center as a lightweight motion prior and eliminating the template branch and cross-frame matching head.The framework recasts tracking as a lightweight geometric transformation around the prior box center.Source: §I · p. 2
  • TFTrack trims inference FLOPs by roughly 50% while maintaining competitive accuracy and reaches approximately 120 FPS across its three variants.The variants are TFTrack-Point, TFTrack-Voxel, and TFTrack-Pillar.Source: §I · p. 2
  • The framework is evaluated on KITTI and nuScenes, with its speed and accuracy presented as suitable for resource-constrained robotic platforms.Source: §I · p. 2

II. RELATED WORK

Prior 3D SOT work follows appearance-matching or motion-modeling approaches, each with efficiency or robustness challenges. The task is to predict an object's box in each subsequent frame, typically by regressing its four pose parameters.

II. RELATED WORK

  • Appearance-based trackers use Siamese feature matching but incur computational costs and can struggle with sparse or occluded point clouds.Examples include SC3D, P2B, and 3D-SiamRPN.Source: §II · p. 2
  • Motion-centric methods use inter-frame geometric consistency to reduce reliance on appearance, but retain complex multi-frame pipelines and computational redundancy.M2Track introduced this motion-centric approach; M2Track++ [20] extended it to a semi-supervised setting.Source: §II · p. 2

A. Overview

  • The three compared paradigms are Siamese template-search matching, template-guided motion modeling, and TFTrack's template-free, Siamese-free one-shot design.Source: Figure 3 · p. 3
  • Because target size is assumed to remain approximately unchanged across frames, 3D SOT regresses center position and yaw as four pose degrees of freedom.The tracked box also includes dimensions and vertical-axis rotation.Source: A. Overview · p. 3

B. Revisit the Template-based Tracking Paradigm

The section distinguishes appearance-based template-search tracking from template-guided inter-frame motion tracking. Both process paired frames, bringing computational redundancy and complexity that motivate a single-frame alternative.

B. Revisit the Template-based Tracking Paradigm

  • Appearance-based tracking crops a first-frame template and a search region around the previous prediction, then matches their features to regress the target pose.Source: B. Revisit the Template-based Tracking… · p. 3
  • Motion-centric tracking segments foreground points from two frames, estimates relative motion, and applies a rigid-body transformation to the previous prediction.Source: B. Revisit the Template-based Tracking… · p. 3
  • Both approaches double input size and add cross-frame computation; motion-centric methods also complicate modeling despite geometric alignment from prior boxes being sufficient.The appearance module is denoted Fapp; the motion pipeline uses segmentation, motion inference, and refinement modules.Source: B. Revisit the Template-based Tracking… · p. 3

C. Unifying Template-Free Framework for 3D SOT

TFTrack recasts 3D SOT as template-free motion prediction around the prior bounding-box center, removing dual-frame inputs and cross-frame matching.

  • TFTrack pioneers a template-free 3D tracker that treats tracking as a lightweight geometric transformation around the prior bounding-box center.The design is motivated by ablation results showing minimal performance loss without templates.Source: C. Unifying Template-Free Framework… · p. 3
  • The framework processes the current point cloud and uses the prior bounding-box center as a lightweight motion prior, eliminating template points and cross-frame matching.It also removes multi-frame motion modeling, reducing computation and memory.Source: C. Unifying Template-Free Framework… · p. 3
  • TFTrack unifies point-, voxel-, and pillar-based variants, each with a lightweight backbone, Box-Conditioned Feature Encoding, and a prediction head.The head regresses the final position.Source: C. Unifying Template-Free Framework… · p. 3

1) Target-Centric Transformation:

TFTrack uses the previous box center to align the current point cloud, then supports voxel, point, and pillar representations within the framework.

1) Target-Centric Transformation:

  • Subtracting the previous box center provides a location prior and reformulates tracking as predicting a 4-DoF rigid transformation to the target’s true location.Centering embeds historical motion context in spatial coordinates and preserves geometric continuity without explicit template matching.Source: 1) Target-Centric Transformation · pp. 3–4

2) Different Point Cloud Views:

  • The framework implements voxel, point, and pillar views: sparse 3D voxels, an unordered matrix of point coordinates, and top-down xy-grid pillars.Sparse formats are adopted for voxels and non-empty pillars.Source: 2) Different Point Cloud Views · p. 4
  • 50% fewer FLOPs: all variants process only the current frame, compared with Siamese or motion-centric backbone designs.The prior center provides lightweight geometric alignment.Source: 2) Different Point Cloud Views · p. 4

3) Siamese-free Backbone:

A siamese-free backbone extracts features from the centered current frame, and Box-Conditioned Feature Encoding conditions them on the prior box size.

3) Siamese-free Backbone:

  • The siamese-free backbone extracts features from the current, target-centered point cloud in a single forward pass, without dual-frame inputs or cross-frame matching.The backbone can use PointTransformerV3, VoxelNet, or PillarNet for point, voxel, or pillar representations, respectively.Source: 3) Siamese-free Backbone · p. 4

4) Box-Conditioned Feature Encoding:

  • Box-Conditioned Feature Encoding appends an embedding of the previous box’s width, height, and length to each aligned feature token.This supplies an object-aware geometric prior for interpreting features relative to expected target scale and shape.Source: 4) Box-Conditioned Feature Encoding · p. 4
  • The BCFE encoder uses point-wise feature-mixing layers for point features and BEV convolutional blocks for voxel and pillar features.A domain-specific readout produces the representation for offset regression.Source: 4) Box-Conditioned Feature Encoding · p. 4

5) Prediction Head and Training Loss:

The prediction head estimates a relative 4-DoF motion offset, while training uses a residual log-likelihood loss and test-time box recovery uses a rigid transformation.

  • A two-layer MLP predicts the relative 4-DoF offset.Source: 5) Prediction Head and Training Loss · p. 4
  • At test time, the predicted bounding box is recovered through a rigid body transformation.Source: 5) Prediction Head and Training Loss · p. 4
  • The KITTI comparison table evaluates Success and Precision, separating dual-frame trackers in its upper part from single-frame trackers in its lower part.Reported means are frame-weighted.Source: Table II · p. 5
  • Training uses residual log-likelihood estimation loss, which models the motion residual distribution with a flow-based density estimator.The density is parameterized by a conditional normalizing flow; the passage identifies ground-truth and predicted motion vectors.Source: 5) Prediction Head and Training Loss · p. 5

IV. EXPERIMENTS

TFTrack is evaluated on KITTI and nuScenes using standard 3D SOT protocols. The setup accounts for different object scales and the greater complexity of nuScenes.

IV. EXPERIMENTS

  • Experiments use KITTI and nuScenes with Success and Precision under the One-Pass Evaluation setting.Evaluation follows standard LiDAR-based 3D SOT protocols.Source: A. Implementation Details · p. 5
  • The defined spatial ranges differ for cars and pedestrians, and the processed spatial resolution is downsampled to 16×16×128.The ranges are [(−4.8,4.8),(−4.8,4.8),(−1.5,1.5)] for cars and [(−1.92,1.92),(−1.92,1.92),(−1.5,1.5)] for pedestrians.Source: A. Implementation Details · p. 5
  • The dataset-statistics comparison covers KITTI and nuScenes.Source: Table III · p. 5

1) Results on KITTI dataset:

TFTrack variants are competitive on nuScenes and report distinct accuracy and speed results on KITTI. In a voxel comparison, the template-free model approaches the dual-frame model's Success/Precision while using roughly half the FLOPs.

1) Results on KITTI dataset:

  • 64.0/81.6, 64.4/83.7, and 60.8/79.0 mean Success/Precision are reported for TFTrack-Voxel, TFTrack-Point, and TFTrack-Pillar on KITTI, respectively.Their reported speeds are 92, 121, and 95 FPS, respectively; BEVTrack achieves higher KITTI accuracy.Source: 1) Results on KITTI dataset · p. 5

2) Results on nuScenes dataset:

3) Template-Free vs. Template-Based:

4) Backbone Analysis:

Backbone comparisons show TFTrack can use different sparse backbones, while ablations identify re-centering and size conditioning as complementary. Template masking and tracklet diagnostics further characterize its robustness.

4) Backbone Analysis:

  • VoxelNet improves by +3.7/+3.2 and PillarNet by +3.3/+3.2 over their compared backbones; PTv3 yields a small gain over Point-MAE.Reported Success/Precision pairs are 59.0/71.4 versus 55.3/68.2 for VoxelNet/FocalsConv, 58.0/71.1 versus 54.7/67.9 for PillarNet/PointPillars, and 57.5/69.1 versus 57.2/69.0 for PTv3/Point-MAE.Source: 4) Backbone Analysis · p. 6

5) Ablation of Proposed Module:

  • Re-centering plus BCFE reaches 59.0/71.4 mean Success/Precision, versus 50.2/62.5 without either module.Re-centering alone reaches 55.1/67.6; BCFE alone reaches 50.7/62.5, and the passage attributes complementary object-scale conditioning to BCFE.Source: 5) Ablation of Proposed Module · p. 6

6) Template Masking:

  • Masking up to 80% of the template causes only minor drops in the dual-frame setup, indicating implicit geometric priors capture most tracking cues.Source: 6) Template Masking · p. 6

7) Robustness Analysis:

  • TFTrack has the best Precision in all reported nuScenes Car tracklet settings and the best Success on overall, late-stage, and fast-start subsets.The diagnostics cover overall tracklet mean, the last 25% of tracklets, sparse-initial cases, and fast-start cases.Source: 7) Robustness Analysis · p. 6

8) Visualization:

Visualizations show TFTrack variants tracking multiple nuScenes object classes near ground truth. The conclusion presents TFTrack as a template-free, single-frame approach and notes limits under difficult observations and deformation.

8) Visualization:

  • Across Car, Pedestrian, Truck, Trailer, and Bus, the visualized TFTrack variants align closely with ground truth without template point-cloud input.The comparison includes M2Track and ground truth.Source: 8) Visualization · pp. 6–7

V. CONCLUSION

  • TFTrack may degrade under extremely sparse LiDAR observations, severe occlusion, or adverse weather, and its 4-DoF rigid-motion prediction assumes constant object size.The conclusion identifies multimodal sensing, lightweight recovery mechanisms, and richer motion representations as future work.Source: §V · p. 7
  • The conclusion describes TFTrack as using the previous box center and size to estimate motion from the current frame, with point, pillar, and voxel variants.It reports competitive accuracy and high inference efficiency, without giving a specific value in this passage.Source: §V · p. 7

Related papers

Loading 2609.07738v1…