Source-linked AI summary
GlobalTrack: A Simple and Strong Baseline for Long-term Tracking
Lianghua Huang, Xin Zhao, Kaiqi Huang
TL;DR
Tracking systems commonly use general-purpose proposals, although tracking requires candidates for specific targets. GlobalTrack performs query-guided global instance search with two-stage detectors and achieves strong results on long-term tracking benchmarks, including gains over prior approaches on LaSOT and TLP.
Problem
General-purpose RPNs generate class-agnostic candidates, whereas tracking needs candidates specific to the queried target.
Method
GlobalTrack uses a query-guided RPN and query-guided RCNN, with correlation-based query encoding and cross-query loss over co-existing instances.
Results
GlobalTrack shows strong benchmark performance, including 2.2% and 3.6% precision gains over ATOM and SiamRPN++ on LaSOT and an approximately 11.1% SR0.5 gain over SPLT on TLP.
Takeaways & Limitations
Because GlobalTrack imposes no temporal-consistency constraint, previous tracking failures do not affect performance in later frames, supporting its use for long-term tracking.
Abstract
from arXiv · showhide
A key capability of a long-term tracker is to search for targets in very large areas (typically the entire image) to handle possible target absences or tracking failures. However, currently there is a lack of such a strong baseline for global instance search. In this work, we aim to bridge this gap. Specifically, we propose GlobalTrack, a pure global instance search based tracker that makes no assumption on the temporal consistency of the target's positions and scales. GlobalTrack is developed based on two-stage object detectors, and it is able to perform full-image and multi-scale search of arbitrary instances with only a single query as the guide. We further propose a cross-query loss to improve the robustness of our approach against distractors. With no online learning, no punishment on position or scale changes, no scale smoothing and no trajectory refinement, our pure global instance search based tracker achieves comparable, sometimes much better performance on four large-scale tracking benchmarks (i.e., 52.1% AUC on LaSOT, 63.8% success rate on TLP, 60.3% MaxGM on OxUvA and 75.4% normalized precision on TrackingNet), compared to state-of-the-art approaches that typically require complex post-processing. More importantly, our tracker runs without cumulative errors, i.e., any type of temporary tracking failures will not affect its performance on future frames, making it ideal for long-term tracking. We hope this work will be a strong baseline for long-term tracking and will stimulate future works in this area. Code is available at https://github.com/huanglianghua/GlobalTrack.
Related Work
Long-term trackers must recover targets after disappearance or failures, but existing approaches differ in when and how they search globally. GlobalTrack performs full-image search in every frame without relying on locality assumptions or online learning.
- Long-term tracking continuously locates arbitrary targets in long videos where targets may temporarily disappear.Recovering a target after absence or tracking failure is identified as a key challenge.
- TLD, SPL, LCT, and EBT support full-image search, while TLD, SPL, and LCT invoke it after detecting failures.EBT performs full-image search continuously, whereas DaSiam LT expands its search area when failures are detected.
- GlobalTrack also performs full-image search, but differs from these trackers by using no online learning.
- GlobalTrack searches globally in all frames without relying on a locality assumption, unlike ATOM and SiamRPN.
Our Approach
GlobalTrack combines query-guided proposal and classification stages to perform query-specific global instance search. Its cross-query loss improves discrimination against distractors, while training and tracking use simple frame-pair and top-1 prediction procedures.
- Architecture: GlobalTrack uses a Query-Guided RPN to generate query-specific proposals and a Query-Guided RCNN to classify and refine them.Both components encode correlations between query and search-image features through feature modulation.
- Query-guided RPN: QG-RPN encodes query information in backbone features before reusing RPN processes for proposal classification, localization, and filtering.The query ROI features and search-image features are correlated to produce feature maps retaining the search-image size.
- Query-guided RCNN: QG-RCNN modulates each proposal with the query features, then applies standard RCNN classification and localization to produce final predictions.The proposal and query ROI features are projected and combined before prediction refinement.
- Cross-query Loss: The cross-query loss averages QG-RPN and QG-RCNN losses across multiple co-existing instances to improve discrimination against instance-level distractors.Backbone feature extraction is shared across query-search pairs, making the loss calculation efficient.
- Offline Training: Training samples frame pairs and optimizes the averaged query losses with stochastic gradient descent, while tracking fixes the first-frame query and selects QG-RCNN’s top-1 prediction without post-processing.The query remains unchanged throughout tracking.
- Offline Training: QG-RPN achieves consistently higher AR@k than general-purpose RPN and GA-RPN, including high recall with few proposals.The comparison is reported in Figure 4 and Table 1.
Experiments
GlobalTrack is evaluated on four large-scale tracking benchmarks, including long-term datasets with thousands of frames per video. The experiments report comparisons with existing trackers and component analyses.
- GlobalTrack is evaluated on LaSOT, TrackingNet, TLP, and OxUvA, whose test sets contain 280, 511, 180, and 166 videos, respectively.
- LaSOT, TLP, and OxUvA are long-term benchmarks with average video lengths of 2500, 13529, and 4260 frames, respectively.
- Table 2 compares QG-RPN and QG-RCNN using AR@k, showing different accuracy and recall behavior as the number of candidates increases.
- Table 3 compares GlobalTrack trained with cross-query loss against single-query loss on the LaSOT test set across three metrics.
Implementation Details
The implementation uses a Faster-RCNN-based configuration, training data from three datasets, and standard optimization with stochastic gradient descent. The supplied implementation passages also include Figure 5’s QG-RPN/QG-RCNN comparison.
- The model uses Faster-RCNN with a ResNet-50 backbone, 256-channel features, 7×7 ROI features, and depth-wise query processing.
- Training combines COCO, GOT-10k, and LaSOT with sampling probabilities of 0.4, 0.4, and 0.2, respectively.
- The model is trained with stochastic gradient descent using batches of 4 frame pairs, momentum 0.9, and weight decay 1 × 10^-4.
- Figure 5 contrasts QG-RPN and QG-RCNN, with QG-RCNN stronger at AR@1 while QG-RPN surpasses it in average recall as candidates increase.
- Training runs for 12 epochs on COCO and another 12 epochs on the combined COCO, GOT-10k, and LaSOT data, with learning-rate decay at epochs 8 and 11.
Ablation Study
The analyses examine proposal quality, tracker comparisons, the roles of QG-RPN and QG-RCNN, and the effect of cross-query loss. Results show complementary stage behavior and improvements from query-aware components and training.
- Analysis of QG-RPN: QG-RPN achieves 67.1% recall with top-1 prediction, 91.9% with top-10 proposals, and 99.2% with top-2000 proposals.
- Analysis of QG-RPN: QG-RPN consistently outperforms general-purpose RPN and GA-RPN in recall rate and efficiency.
- Tracker comparison: GlobalTrack gains 2.2% in precision and 0.7% in success over ATOM on LaSOT, while also outperforming SPLT and DaSiam LT.
- Benchmark metrics: Table 4 reports TLP comparisons using success rate, success score, and precision score, while Table 5 reports TrackingNet precision, normalized precision, and success AUC.
- Comparison of QG-RPN and QG-RCNN: QG-RCNN exceeds QG-RPN in top-1 accuracy by up to 9.5%, whereas QG-RPN provides higher recall once more candidates are introduced.
- Impact of Loss Function: Cross-query loss improves all three LaSOT metrics over single-query loss by 2.6%–4.2%.
Comparison with State-of-the-art
GlobalTrack is compared with state-of-the-art trackers across four large-scale benchmarks and achieves strong results, including substantial gains on long-term tracking datasets.
- The evaluation covers comparisons with state-of-the-art trackers using LaSOT, TrackingNet, TLP, and OxUvA benchmarks.
- LaSOT: GlobalTrack achieves absolute gains of 2.2% and 3.6% in precision and 0.7% and 2.5% in success over ATOM and SiamRPN++ on LaSOT, respectively.
- TLP: GlobalTrack outperforms all other trackers on TLP by an approximately 11.1% absolute gain in SR0.5 over SPLT.
- TrackingNet: GlobalTrack achieves a 70.4% success score on TrackingNet, comparable with the best trackers SiamRPN++ and ATOM.
- OxUvA: GlobalTrack achieves absolute MaxGM gains of 14.9% on the OxUvA test set and 24.2% on its development set over SiamFC-R.
Conclusions and Future Works
GlobalTrack is a pure global instance-search tracker without temporal-consistency constraints, using query-guided proposal generation and classification. Experiments on four large-scale benchmarks verify its strong tracking performance.
- GlobalTrack imposes no assumption or constraint on temporal consistency, so performance in one frame is not affected by previous tracking failures.
- The tracker uses a query-guided RPN to generate instance candidates and a query-guided RCNN to classify candidates and produce final predictions.
- Experiments on four large-scale tracking benchmarks verify the strong performance of the proposed approach.