Source-linked AI summary
High-Performance Long-Term Tracking with Meta-Updater
Kenan Dai, Yunhua Zhang, Dong Wang, Jianhua Li, Huchuan Lu, Xiaoyun Yang
TL;DR
Long-term tracking needs reliable re-detection and must manage the noise introduced by online updates. This paper proposes a sequential multi-cue meta-updater and embeds it in a long-term framework, which outperforms state-of-the-art trackers across five benchmarks. The meta-updater also generalizes across online-updated trackers, with only a slight speed decrease.
Problem
Long-term trackers largely use offline-trained Siamese architectures, while directly introducing online updating is risky because long-term uncertain and noisy observations can pollute the tracker.
Method
An offline-trained meta-updater integrates geometric, discriminative, and appearance cues sequentially and produces a binary decision for whether to update the tracker.
Results
The proposed tracker outperforms state-of-the-art methods across the VOT2018LT, VOT2019LT, OxUvALT, TLP, and LaSOT benchmarks.
Takeaways & Limitations
The meta-updater reduces the risk of online updating, generalizes across different online-learning trackers, and slightly decreases tracking speed when added.
Takeaways & Limitations
Training uses sequences initialized with ground truth in the first frame and follows online single-object tracking settings.
Abstract
from arXiv · showhide
Long-term visual tracking has drawn increasing attention because it is much closer to practical applications than short-term tracking. Most top-ranked long-term trackers adopt the offline-trained Siamese architectures, thus, they cannot benefit from great progress of short-term trackers with online update. However, it is quite risky to straightforwardly introduce online-update-based trackers to solve the long-term problem, due to long-term uncertain and noisy observations. In this work, we propose a novel offline-trained Meta-Updater to address an important but unsolved problem: Is the tracker ready for updating in the current frame? The proposed meta-updater can effectively integrate geometric, discriminative, and appearance cues in a sequential manner, and then mine the sequential information with a designed cascaded LSTM module. Our meta-updater learns a binary output to guide the tracker's update and can be easily embedded into different trackers. This work also introduces a long-term tracking framework consisting of an online local tracker, an online verifier, a SiamRPN-based re-detector, and our meta-updater. Numerous experimental results on the VOT2018LT, VOT2019LT, OxUvALT, TLP, and LaSOT benchmarks show that our tracker performs remarkably better than other competing algorithms. Our project is available on the website: https://github.com/Daikenan/LTMU.
1. Introduction
Long-term tracking is increasingly relevant to practical applications because it involves much longer sequences and requires handling target disappearance and reappearance. The paper addresses the risks of online updating under uncertain observations with a meta-updater and an online-updated long-term framework.
- Long-term tracking better matches practical applications than short-term tracking because its benchmark sequences typically last at least minutes rather than seconds.
- Long-term tracking additionally requires frequent target disappearance and reappearance, creating a need for strong re-detection capability.
- Online-updated trackers can be more accurate but are risky in long-term tracking because noisy observations can pollute the model.
- The proposed framework combines an online local tracker, an online verifier, a SiamRPN-based re-detector, and an offline-trained meta-updater.
- Experimental results on five long-term benchmarks show that the proposed method outperforms state-of-the-art trackers by a large margin.
2. Related Work
Prior long-term trackers combine local tracking with global re-detection, while online updating introduces a trade-off between adapting to appearance changes and accumulating noise. The paper proposes a sequential multi-cue meta-updater that decides when online updating is appropriate.
- 2.1. Long-term Visual Tracking: Long-term tracking methods commonly combine a local tracker with a global re-detector, sometimes using the same powerful model for both components.
- 2.2. Online Update for Visual Tracking: Online update captures target and background appearance variation but can accumulate errors, use inappropriate samples, and cause tracking drift during disappearance.
- 2.2. Online Update for Visual Tracking: Existing reliability criteria include confidence score, MAX response, PSR, average peak-to-correlation energy, and MAX-PSR.
- 2.2. Online Update for Visual Tracking: The meta-updater integrates multiple cues sequentially and outputs a binary score indicating whether the tracker should update in the current frame.
3. Long-term Tracking with Meta-Updater
The framework combines online local tracking, verification, re-detection, and a meta-updater that decides whether the tracker should update. It integrates geometric, discriminative, and appearance cues over time, trains on IoU-based labels, and processes them with cascaded LSTMs.
- Framework: The framework combines an online local tracker, verifier, detector-based re-detection, and meta-updater to control long-term tracking.The verifier permits continued local tracking above a threshold and triggers candidate detection otherwise.
- Implementation: The improved ATOM* local tracker uses ATOM classification for localization, SiamMask for scale estimation, and RTMDNet as the verifier.The implementation notes that SiamMask provides more accurate scale estimation in practice than the original ATOM setup.
- Meta-Updater: The meta-updater addresses whether the tracker is ready to update by integrating geometric, discriminative, and appearance cues from historical tracking results.It is designed to generalize across trackers that provide response maps or classification-based confidence scores.
- Cascaded LSTM: Three cascaded LSTMs mine sequential information with progressively shorter time steps, emphasizing recent frames before fully connected layers produce a binary update decision.The model uses the cascaded LSTM outputs to indicate whether the tracker should be updated.
- Meta-Updater Training: Training samples are collected by running the local tracker on annotated sequences and labeling time slices using current-frame localization success.Samples with IoU between 0 and 0.5 are excluded, while labels distinguish IoU above 0.5 from unsuccessful localization.
4. Experiments
Experiments across five long-term tracking benchmarks evaluate accuracy, ablations, generalization, and speed. The tracker achieves leading results, while the meta-updater benefits multiple trackers with only a slight speed decrease.
- Quantitative Evaluation: The tracker outperforms all other methods by a very large margin on VOT2018LT.Trackers are ranked using tracking F-score on a dataset with frequent target disappearances.
- Quantitative Evaluation: The tracker achieves first place on the VOT2019LT challenge and performs best in MaxGM and TPR on OxUvALT.On OxUvALT, it maintains a very competitive TNR value.
- Quantitative Evaluation: The tracker achieves the best results among competing methods on both LaSOT and TLP one-pass evaluations.LaSOT uses success and precision plots, while TLP evaluates success and precision on 50 high-definition videos.
- Ablation Study: The best meta-updater performance occurs with a time step of 20, balancing historical information and current observations.The ablation study also evaluates bounding box, confidence, response-map, and appearance inputs.
- Ablation Study: Removing any meta-updater input reduces its contribution, and increasing cascaded-LSTM iterations gradually improves performance.The tested inputs are bounding box, confidence score, response map, and appearance score.
- Discussions: The meta-updater consistently improves accuracy across four trackers while slightly decreasing speed.The authors conclude that it improves accuracy almost without sacrificing efficiency.
5. Conclusions
The work presents a long-term tracking framework centered on a meta-updater that guides online tracker updates using sequential geometric, discriminative, and appearance cues. Experiments on five recent long-term benchmarks show significantly better performance than state-of-the-art methods and indicate good generalization.
- The framework uses an online-update-based tracker for local tracking and a meta-updater to determine whether updating is appropriate.The meta-updater integrates geometric, discriminative, and appearance cues sequentially to guide updates.
- The meta-updater substantially reduces the risk of online updating in long-term tracking while guiding updates effectively and efficiently.
- Five recent long-term benchmarks show significantly better performance than other state-of-the-art methods.
- The experimental results indicate that the meta-updater has good generalization ability.