Source-linked AI summary
'Skimming-Perusal' Tracking: A Framework for Real-Time and Robust Long-term Tracking
Bin Yan, Haojie Zhao, Dong Wang, Huchuan Lu, Xiaoyun Yang
TL;DR
Long-term tracking must handle targets that disappear and reappear, yet existing work lacked a robust, real-time framework. The paper introduces a Skimming-Perusal tracker that combines local regression and verification with efficient global re-detection, and reports the best performance with real-time speed on two long-term benchmarks.
Problem
Existing trackers and datasets largely focus on short-term tracking, while long-term tracking requires handling target disappearance and reappearance; prior methods lacked a robust, real-time framework.
Method
The framework uses perusal for local bounding-box regression and verification, then uses skimming to select likely regions during image-wide global search.
Results
The tracker achieves the best performance with real-time speed on the VOT2018LT and OxUvA long-term benchmarks.
Takeaways & Limitations
The Skimming-Perusal framework is presented as a simple, effective real-time long-term tracker and a possible baseline for further research.
Abstract
from arXiv · showhide
Compared with traditional short-term tracking, long-term tracking poses more challenges and is much closer to realistic applications. However, few works have been done and their performance have also been limited. In this work, we present a novel robust and real-time long-term tracking framework based on the proposed skimming and perusal modules. The perusal module consists of an effective bounding box regressor to generate a series of candidate proposals and a robust target verifier to infer the optimal candidate with its confidence score. Based on this score, our tracker determines whether the tracked object being present or absent, and then chooses the tracking strategies of local search or global search respectively in the next frame. To speed up the image-wide global search, a novel skimming module is designed to efficiently choose the most possible regions from a large number of sliding windows. Numerous experimental results on the VOT-2018 long-term and OxUvA long-term benchmarks demonstrate that the proposed method achieves the best performance and runs in real-time. The source codes are available at https://github.com/iiau-tracker/SPLT.
1. Introduction
Long-term tracking addresses realistic videos where targets disappear and reappear, but existing methods have lacked a robust, real-time framework. The proposed Skimming-Perusal framework combines precise local tracking with efficient global re-detection.
- Motivation: Most trackers and datasets focus on short-term tracking, leaving a gap from realistic applications requiring long-term target capture.Long-term benchmarks include VOT2018LT and OxUvA.
- Motivation: Long-term tracking must handle longer videos and frequent target disappearance and reappearance, creating greater challenges than short-term tracking.Average sequence lengths are 4196 frames for VOT2018LT and 4235 for OxUvA, versus 590 for OTB2015 and 350 for VOT2018.
- Motivation: Existing hand-crafted and deep-learning trackers improved performance but still lacked a robust, real-time framework for long-term tracking.Hand-crafted methods performed poorly on recent benchmarks, while newer deep methods improved tracking performance without resolving the framework gap.
- Proposed framework: The Skimming-Perusal framework uses a perusal module for precise local capture and a skimming module for efficient image-wide re-detection.Both modules are based on deep networks and are used directly during tracking after offline training.
- Evaluation: Experiments on VOT2018LT and OxUvA report the best accuracies with real-time performance.The framework is presented as simple yet effective and as a possible new baseline for long-term tracking.
2. Related Work
Prior long-term trackers used hand-crafted features, keypoint matching, global proposals, or deep models, but important robustness and speed limitations remained. The related work motivates a deep-learning framework targeting high accuracy with real-time performance.
- Traditional Long-term Tracking: Traditional trackers combined local tracking with detection or matching, using optical flow, correlation filters, weak classifiers, keypoints, or global proposals.Examples include TLD, LCT, FCLT, CMT, MUSTer, and EBT.
- Traditional Long-term Tracking: Keypoint extractors and descriptors were often unstable in complicated scenes, while hand-crafted low-level features produced unsatisfactory performance.These limitations affected keypoint-based and other traditional long-term methods.
- Deep Long-term Tracking: The related work identifies high accuracy with real-time performance as the goal for a deep-learning-based long-term tracking framework.This goal follows the speed and feature limitations described for earlier approaches.
- Deep Long-term Tracking: A deep framework combined offline bounding-box regression with online verification to switch dynamically between local and global search.Its global search used image-wide sliding windows, but the method reported only 2.7fps and achieved 4.4fps in the authors' setting.
3. ‘Skimming-Perusal’ Tracking Framework
The framework combines a local perusal module for proposal generation and verification with a skimming module that accelerates global re-detection. Confidence-based switching selects local or global search, while skimming filters sliding-window candidates before detailed verification.
- Framework overview: The framework uses perusal for local regression and verification, and skimming to select likely regions during global search.Perusal operates in a local search region, while skimming selects candidates from many sliding windows.
- Local perusal: SiameseRPN generates bounding-box proposals and similarity scores, but its highest-scoring proposal can drift toward distractors.The verifier is therefore used to determine the optimal candidate rather than relying directly on the highest SiameseRPN score.
- Local perusal: The verification model computes candidate confidence scores and switches between local and global search using threshold θ = 0.65.Scores above the threshold keep the tracker in local search; otherwise, global search is invoked in the next frame.
- Global search: The skimming design targets the high cost of image-wide re-detection, which can make deep-learning-based global search non-real-time.The paper contrasts this challenge with MBMD, reported to run at less than 5 fps, and reports reduced running time when the target disappears and reappears.
- Global search: The skimming module applies a learned classifier to dense sliding-window regions, retains the top-K candidates, and sends them to perusal.K is set to 3, reducing the number of regions requiring the more detailed SiameseRPN-plus-verifier processing.
- Implementation: The modules are trained offline, using MobileNet-based regression and skimming models and a ResNet50 verification backbone.Training uses ImageNet DET and VID data with augmentation, while the regression, verification, and skimming modules are trained independently.
4. Experiments
Experiments evaluate SPLT on VOT2018LT and OxUvA, including benchmark comparisons, attribute analyses, ablations, and parameter studies. SPLT achieves strong long-term tracking performance while maintaining real-time speed.
- Experimental setup: 25.7 fps: SPLT runs in real time on the reported test platform.The implementation uses an Intel i7 CPU, 32G RAM, and NVIDIA GTX1080Ti GPU.
- VOT2018LT results: SPLT achieves the best VOT2018LT F-score and recall while maintaining the highest re-detection success rate.MBMD and DaSiam LT also achieve top-ranked performance, but SPLT is reported as strongest overall on these criteria.
- VOT2018LT results: 25.7 fps vs 4.4 fps: SPLT runs much faster than MBMD while achieving better accuracy and the same re-detection capability.The paper attributes the speed advantage mainly to the skimming and offline-trained verification modules.
- VOT2018LT results: 100% re-detection success: SPLT passes the re-detection experiment, whereas DaSiam LT’s success rate is almost zero.The comparison also reports that SPLT uses substantially less training data than DaSiam LT.
- Ablation study: All components improve long-term tracking: the verifier substantially improves performance but reduces speed, while skimming restores speed and slightly improves performance.Skimming selects a few likely regions from many sliding windows, filters distractors, and reduces regressor inputs during image-wide re-detection.
- OxUvA results: On OxUvA, SPLT ranks first in MaxGM, TPR, and TNR, with relative MaxGM gains of 37.0% over MBMD and 14.3% over SiamFC+R.MaxGM is the most important OxUvA metric according to the reported comparison.
5. Conclusion
The paper presents Skimming-Perusal as a simple, effective real-time framework for long-term visual tracking. It combines perusal-based local localization and verification with skimming-based global search acceleration.
- The proposed Skimming-Perusal model is presented as a simple, effective real-time framework and a potential baseline for long-term tracking.The authors describe it as suitable for further research.
- The perusal module uses offline-trained regression and verification networks to precisely locate the tracked object in a local search region.Its confidence score determines whether the object is present or absent and whether local tracking or image-wide re-detection follows.
- The skimming module selects likely regions from densely sampled sliding windows, speeding up the global search process.This supports image-wide re-detection when the tracked object is absent.