Source-linked AI summary
Beyond Local Search: Tracking Objects Everywhere with Instance-Specific Proposals
Gao Zhu, Fatih Porikli, Hongdong Li
TL;DR
Local search can miss fast, irregularly moving objects and becomes vulnerable to background distractors when widened. The paper uses instance-specific, edge-based proposals to search the full frame and update an existing object model with hard negatives. It reports best results on common tracking benchmarks and robustness for fast motion and extremely low frame rates.
Problem
Local search windows can fail under deformation, occlusion, fast object-camera motion, and expanded background clutter that complicates classifier updates.
Method
The tracker generates a small set of high-quality, instance-specific whole-frame proposals, evaluates them with an existing object model, and uses them for model updating and proposal re-ranking.
Results
The tracker reports the best results on common benchmarks including OTB, TB50, VOT2014, and ALOV300, including top rankings against state-of-the-art trackers.
Takeaways & Limitations
The method supports tracking objects moving randomly or very fast and tracking under extremely low frame rates.
Takeaways & Limitations
The method imposes a smooth size-change constraint on bounding boxes between consecutive frames.
Abstract
from arXiv · showhide
Most tracking-by-detection methods employ a local search window around the predicted object location in the current frame assuming the previous location is accurate, the trajectory is smooth, and the computational capacity permits a search radius that can accommodate the maximum speed yet small enough to reduce mismatches. These, however, may not be valid always, in particular for fast and irregularly moving objects. Here, we present an object tracker that is not limited to a local search window and has ability to probe efficiently the entire frame. Our method generates a small number of "high-quality" proposals by a novel instance-specific objectness measure and evaluates them against the object model that can be adopted from an existing tracking-by-detection approach as a core tracker. During the tracking process, we update the object model concentrating on hard false-positives supplied by the proposals, which help suppressing distractors caused by difficult background clutters, and learn how to re-rank proposals according to the object model. Since we reduce significantly the number of hypotheses the core tracker evaluates, we can use richer object descriptors and stronger detector. Our method outperforms most recent state-of-the-art trackers on popular tracking benchmarks, and provides improved robustness for fast moving objects as well as for ultra low-frame-rate videos.
1. Introduction
Existing trackers often restrict candidate search to a local window, which can fail under deformation, occlusion, or rapid and irregular motion. The proposed method instead searches the entire frame with instance-specific proposals and uses them to improve model updates and tracking robustness.
- Local search windows assume accurate previous tracking and manageable motion, but can fail during deformation, occlusion, or large object-camera displacement.
- Wider search is difficult because expanded background regions create distracting negatives that complicate discriminative classifier updates.
- The method efficiently generates a small, high-quality set of whole-frame proposals using bottom-up edge-based features and instance-specific objectness.
- An existing object model can evaluate proposals, while an online classifier learns to re-rank them according to similarity to the tracked object.
- Instance-specific proposals provide harder negatives for model updates, reducing drift and producing better object-model adaptation than generic proposals.
- The tracker was evaluated with NCC and Struck object models and reported top performance on VOT2014, OTB, and TB50 against contemporary trackers.
2. Related Work
Related tracking methods differ in how they select candidate windows, balancing local or sampled search against computational cost and proposal quality. Object proposals can improve detection efficiency and accuracy, while this work adapts objectness to the specific tracked object.
- Search Schemes: Single-window and particle-based searches explore candidates near prior estimates or through sampling, while uniform sampling over larger regions becomes computationally intractable.
- Object Proposals: Object proposals improve detection speed and accuracy by restricting evaluation to a subset of high-quality candidates and reducing false positives.
- Objectness: Edge Box proposes candidates using the number of contours enclosed by a bounding box as an object-likelihood signal, while emphasizing speed and recall.
- Objectness: Unlike methods that linearly combine tracking confidence with generic objectness, the proposed approach adapts objectness to the specific tracked object using a separate classifier.
3. Global Tracking with Proposals
The tracker replaces a small local search window with a compact set of instance-specific proposals generated across the entire frame. These proposals use objectness and object-model information to improve sample selection, reduce distractors, and support robust tracking beyond smooth motion assumptions.
- Limitations of Local Search: Traditional tracking-by-detection methods generate candidates near the previous object location, which can miss objects after occlusion or motion beyond the search radius.The candidate set is typically produced by a motion model and may use a heuristic radius such as 30 pixels.
- Instance-Specific Proposals: The proposed motion model treats candidate boxes unequally according to their object-like appearance and does not constrain search to a small local window.This retains image information that local sampling would discard.
- Instance-Specific Proposals: Edge-based processing generates a small set of candidate boxes across the frame, while a smooth size-change constraint is imposed between consecutive frames.The proposals are denoted as BE_t and are built on an object proposal algorithm using edge information.
- Proposal Selection: Using only object proposals produced the best reported results, reducing spurious false positives and preventing drift toward textureless background regions.The paper reports that combining global proposals with local candidates was inferior to using the proposals alone for tracking.
- Model Update: The proposal tracker updates the object model using selected proposal sets, including options that combine global proposals with local candidates for negative support vectors.The best combination is reported to use both sets during model update, even though proposal-only candidates perform best for tracking selection.
- Proposal Ranking: Instance-specific proposals are selected and re-ranked using objectness features and an online linear SVM trained from proposal properties.The ranking feature is 10-dimensional and concatenates EdgeBox scores from Haar wavelet-like partitions of each candidate box.
- Candidate Classification: The candidate decision function combines the object-model score with a motion-smoothness term that softly penalizes distance from the previous location.The smoothness term uses bounding-box centers, with ws = 0.1 and σ equal to the diagonal length of the initialized bounding box.
4. Proposed Trackers
The proposed tracker integrates instance-specific proposals with established object models, including SSVM and NCC, so proposal generation can support both sophisticated classifiers and simple template matching. Richer histogram descriptors are used to exploit the reduced proposal count while maintaining similar speed.
- Core Object Models: Two core trackers are implemented: EBT combines the proposal method with an SSVM, while NCCEB combines it with normalized cross-correlation template matching.NCCEB is included to measure the additional improvement provided by instance-specific proposals over a simpler object model.
- EBT Tracker: The SSVM classifier represents support vectors with weighted affinities and is updated online using an intersection kernel and a budget.Its parameters are set according to the referenced SSVM tracker.
- Object Descriptors: The tracker uses a 2640-D spatial-pyramid histogram feature, compared with the 480-D feature in the SSVM reference method, at a similar speed.The feature concatenates 16-bin intensity histograms over five pyramid levels and separate RGB channels.
- NCCEB Tracker: The NCC model scores candidates with normalized cross-correlation between the candidate patch and the object template, accelerated using the FFT trick.The paper compares NCCEB with local-exhaustive-search NCC and IMPNCC baselines using fixed-template or online-update variants.
5. Experiments
Experiments evaluate the tracker on OTB, TB50, VOT2014, fast-motion categories, and an ultra-low-frame-rate dataset. The method consistently outperforms state-of-the-art trackers while retaining comparable computational speed and benefiting from instance-specific proposals.
- Benchmark Results: EBT ranks first on VOT2014 and achieves the best overall performance across the evaluated datasets, substantially improving over Struck.The authors report consistent gains over state-of-the-art trackers, including MEEM, KCF, and Struck.
- Benchmark Results: NCCEB significantly improves the simple NCC tracker and outperforms several other trackers, including IMPNCC.This result is attributed in the discussion to incorporating objectness into tracking.
- Tracking Fast Objects: The method outperforms other trackers in fast-motion categories, including fast moving, randomly moving, and moving-camera scenarios.The motivation is global image search, which can recover objects that leave traditional local search windows.
- Tracking under Ultra-Low-Frame-Rate: On VOT2014+, created by sampling every 20 frames, the tracker retains very high performance while KCF and MEEM rapidly fail.The resulting videos contain objects moving 20× faster than in the original sequences.
- Ablation Analysis: Using hypothesis proposals for model updates and proposed locations for testing performs best among the evaluated proposal combinations.The analysis reports that this combination reduces spurious false positives and improves tracking accuracy.
- Ablation Analysis: Instance-specific proposal re-ranking outperforms directly applying EdgeBox proposals, while a lower-dimensional feature variant performs worse than the 2640-D version.Removing the smoothness term also reduces success rate because tracking motion is not completely random.
- Computational Speed: The proposal component takes less than 100 milliseconds, and overall computational speed is reported as comparable to state-of-the-art trackers.This speed is achieved despite searching over the entire image.
6. Conclusion
The paper concludes that its tracker can locate randomly and rapidly moving objects, including under extremely low frame rates. It reports the best results on the common OTB, TB50, VOT2014, and ALOV300 benchmarks.
- Conclusion: The tracker locates objects moving randomly and very fast and tracks them under extremely low frame rates.These capabilities are the paper’s stated central conclusion.
- Conclusion: The authors report the best results on OTB, TB50, VOT2014, and ALOV300 among the common benchmark datasets.