Source-linked AI summary
Discriminative Scale Space Tracking
Martin Danelljan, Gustav Häger, Fahad Shahbaz Khan, Michael Felsberg
TL;DR
Accurate scale estimation is difficult because exhaustive scale search is computationally expensive and can struggle with large scale variations. The paper proposes separate online discriminative filters for translation and scale, together with strategies that reduce computational cost. Experiments on OTB and VOT 2014 report improved tracking performance, faster operation, and top benchmark rankings, while the current method uses only intensity-based image representations.
Problem
Accurate and robust target-scale estimation remains challenging in visual tracking, while exhaustive scale search is computationally demanding for real-time applications.
Method
DSST learns separate discriminative correlation filters for translation and scale from target appearances sampled at different scales, with additional strategies to reduce computational cost.
Results
The method outperforms 19 state-of-the-art trackers on OTB and 37 on VOT 2014, obtaining the top combined ranking score on VOT 2014.
Takeaways & Limitations
Reducing computational cost allows a larger target search space while preserving real-time performance and improving tracking performance.
Takeaways & Limitations
The method uses only intensity-based image representations; color fusion remains future work, especially for target deformations.
Abstract
from arXiv · showhide
Accurate scale estimation of a target is a challenging research problem in visual object tracking. Most state-of-the-art methods employ an exhaustive scale search to estimate the target size. The exhaustive search strategy is computationally expensive and struggles when encountered with large scale variations. This paper investigates the problem of accurate and robust scale estimation in a tracking-by-detection framework. We propose a novel scale adaptive tracking approach by learning separate discriminative correlation filters for translation and scale estimation. The explicit scale filter is learned online using the target appearance sampled at a set of different scales. Contrary to standard approaches, our method directly learns the appearance change induced by variations in the target scale. Additionally, we investigate strategies to reduce the computational cost of our approach. Extensive experiments are performed on the OTB and the VOT2014 datasets. Compared to the standard exhaustive scale search, our approach achieves a gain of 2.5% in average overlap precision on the OTB dataset. Additionally, our method is computationally efficient, operating at a 50% higher frame rate compared to the exhaustive scale search. Our method obtains the top rank in performance by outperforming 19 state-of-the-art trackers on OTB and 37 state-of-the-art trackers on VOT2014.
1 INTRODUCTION
Accurate scale estimation is difficult because exhaustive scale search is computationally demanding, while real-time tracking requires both robustness and efficiency. DSST addresses this by learning separate translation and scale filters, with reduced-cost variants improving speed and robustness.
- Scale variations, occlusions, fast motion, and illumination changes make accurate target-size estimation challenging in visual tracking.
- Exhaustive scale search evaluates the appearance model at multiple resolutions but is computationally demanding for real-time applications.
- DSST learns separate discriminative correlation filters for translation and scale estimation from target samples at different scales.The scale filter explicitly learns appearance changes induced by target-size variation and reduces the search space.
- The fast DSST reduces computational cost without sacrificing robustness or accuracy and achieves a twofold speed gain over DSST.The reduced cost also permits a larger target search area.
- DSST improves accuracy and speed over baseline DCF exhaustive scale search methods in evaluations on OTB and VOT 2014.
2 RELATED WORK
Visual tracking estimates a target trajectory from its initial location, using generative or discriminative appearance models. DCF trackers localize accurately and efficiently but are limited by scale changes, motivating DSST and its computational improvements.
- Generic visual tracking estimates the trajectory of an initially specified target across an image sequence.Applications include robotics, surveillance, and road-scene understanding.
- Generative methods model target appearance, whereas discriminative methods learn to distinguish target appearance from the surrounding background.Examples of discriminative learning approaches include SVMs and boosting.
- DCF trackers provide strong benchmark performance at real-time speeds by learning filters for target localization.
- Standard DCF trackers estimate translation but perform poorly under significant target-scale variations.
- This paper extends DSST with computational-cost reductions, larger search areas, superior performance, and a twofold speedup.The evaluation covers all 50 OTB videos, 19 state-of-the-art trackers, and the VOT 2014 dataset.
3 MULTI-CHANNEL DISCRIMINATIVE CORRELATION FILTERS
The paper formulates multi-channel DCF learning as regularized correlation-filter optimization solved efficiently in the Fourier domain. The framework supports filters for translation and scale estimation and updates them online from multiple samples.
- DCF learning uses target appearance samples and cyclic shifts to learn a correlation filter for locating the target in a new frame.
- The framework supports one-dimensional scale filters, two-dimensional translation filters, and higher-dimensional filters by adapting feature extraction.
- A multi-channel target sample contains a d-dimensional feature vector at each location, with one filter per feature channel.
- The objective minimizes squared correlation error against a desired output while applying regularization weighted by λ.The desired output is typically a Gaussian function.
- The least-squares problem is solved efficiently in the Fourier domain, with pointwise multiplications and divisions in the resulting filter expression.
- Robust filters incorporate multiple time-instance samples, while online updates use a learning-rate parameter η.
- At detection, correlation scores are computed for a test sample and the current target state is selected at the maximum score.
4 CORRELATION FILTERS FOR TRANSLATION AND SCALE ESTIMATION
The paper compares DCF-based strategies for estimating translation and scale, including exhaustive searches and a joint scale-space filter. Joint estimation can suffer from location-induced shearing, while iterative detection reduces this distortion at added computational cost.
- Standard DCF Tracker: The standard DCF baseline learns a 2-dimensional multi-channel translation filter from patches centered on the target.
- Multi-Resolution Translation Filter: Multi-resolution translation filtering applies the translation filter to patches sampled at different resolutions around the previous target location.
- Joint Scale Space Filter: A 3-dimensional scale-space filter jointly estimates translation and scale by maximizing correlation scores over a feature-pyramid region.
- Joint Scale Space Filter: The joint filter is trained on an M × N × S feature-pyramid cuboid centered at the target location and scale, using a 3-dimensional Gaussian output.
- Iterative Joint Scale Space Filter: Location errors in the predicted center introduce shearing between the test sample and actual-target pyramid, biasing the translation estimate.
- Iterative Joint Scale Space Filter: Iterating detection around the updated target estimate reduces shearing distortion, but increases computational time.
5 OUR APPROACH
The proposed DSST separates translation and scale estimation with an explicit one-dimensional scale filter trained on multi-scale target samples. Computational reductions enable larger search areas and produce fDSST, which improves robustness while operating at twice DSST’s speed.
- Motivation: Scale-adaptive tracking is needed because scale extensions to standard DCF tracking significantly increase computational cost.
- Scope: The scale estimation approach is generic and can be used in tracking frameworks without an existing scale-estimation component.
- Discriminative Scale Space Tracking: DSST learns a separate 1-dimensional scale correlation filter whose scores estimate target scale at a selected image location.
- Discriminative Scale Space Tracking: The scale filter is trained from feature descriptors extracted from target patches sampled at different scales.
- Discriminative Scale Space Tracking: DSST first estimates translation with a standard filter, then applies the scale filter at the new target location to estimate size.
- DSST Algorithm: Each DSST iteration estimates translation, estimates scale at the updated position, and updates both translation and scale models.
- Computational Reduction: Sub-grid interpolation and PCA reduce the FFT costs of training and detection, allowing computationally efficient scale tracking.
- Compressed Scale Filter: For the scale filter, feature dimensionality can be compressed to the number of sampled scales, approximately d̃ = S, without information loss.
6 EXPERIMENTS
The experiments evaluate DSST and fDSST on OTB and VOT 2014 using standardized parameters, features, dimensionality reductions, and tracking metrics.
- PCA-HOG with grayscale intensity represents the target, while scale-filter patches are resized and described using HOG features.fDSST reduces the translation representation to 18 dimensions and the scale representation to S = 17 dimensions.
- Experiments use the 50-video OTB dataset and the VOT 2014 benchmark, with quantitative and qualitative evaluations.
- OTB results report overlap precision (OP), distance precision (DP), and tracking speed in frames per second (FPS).OP is reported at an overlap threshold of 0.5, while DP uses a 20-pixel centroid-distance threshold.
- Success plots summarize mean OP across overlap thresholds from 0 to 1, with each tracker’s area-under-the-curve (AUC) score reported.
6.3 Experiment 1: DCF-based Scale Estimation
DSST is compared with translation-only and exhaustive scale-search DCF trackers on OTB and VOT 2014, improving accuracy and robustness while reducing computational burden.
- On OTB, DSST provides a 10.0% mean OP gain over the baseline translation tracker and improves mean DP over other scale-adaptive trackers.The baseline translation tracker obtains 57.7% mean OP; DSST is also 17 times faster than the joint scale space filter.
- DSST improves the baseline translation tracker by 6.6% in AUC on the OTB success plot.The baseline translation filter has an AUC of 49.5%, while the joint scale space filter reaches 52.6%.
- On VOT 2014, DSST achieves the best final rank among the compared DCF approaches, with a final rank of 2.80.Its average failure rate is 1.16 while maintaining accuracy.
- The experiments conclude that DSST improves the standard translation tracker and provides superior performance and frame-rate relative to other scale-adaptive DCF trackers.
6.4 Experiment 2: Fast Discriminative Scale Space Tracker
fDSST accelerates DSST through computational reduction and a larger search space, while improving tracking performance on OTB.
- fDSST improves mean OP by 7.0% and mean DP by 4.4% over DSST on OTB.
- The performance gains occur while fDSST operates at over twice DSST’s mean FPS.
- fDSST tracking performance remains consistent as translation-filter dimensionality decreases from 32 dimensions, then degrades rapidly at about 6 dimensions.The experiments use 18 PCA dimensions for stable results.
6.5 Experiment 3: State-of-the-Art Comparison
fDSST is compared with state-of-the-art trackers on OTB and VOT 2014, achieving strong accuracy, robustness, and real-time speed across benchmark and attribute analyses.
- On OTB, fDSST outperforms SAMF by 4.6% in mean OP while operating at 54.3 mean FPS.SAMF obtains 69.7% mean OP among existing methods; fDSST also provides superior mean DP.
- fDSST achieves a 60.3% AUC on the OTB success plot, exceeding KCF by 8.5% and SAMF by 2.6%.
- fDSST provides favorable results on 7 of 11 OTB attributes, including scale variation, rotation, clutter, illumination variation, blur, and fast motion.On scale-variation sequences, it outperforms SAMF by 5.1% in AUC.
- In temporal and spatial robustness evaluations, fDSST performs favorably compared with five top-performing trackers.
- Qualitative examples show favorable tracking in scale variation, out-of-plane rotation, illumination variation, background clutter, and occlusion scenarios.
- On VOT 2014, fDSST achieves the best final score among participating methods.The evaluation reports accuracy and robustness rankings, average overlap, and tracking failures.
6.6 Experiment 4: VOT Challenge 2014
On VOT 2014, fDSST achieved the top average rank among 38 trackers, while comparisons also examined overlap, failures, attributes, and region-noise robustness.
- VOT Challenge 2014: The VOT evaluation included baseline and region-noise experiments, with average overlap and failure rate reported for the baseline.
- VOT Challenge 2014: fDSST achieved the top average rank among all 38 trackers on VOT 2014.The evaluation covered 25 videos and reported baseline and region-noise results.
- VOT Challenge 2014: Compared with SAMF, fDSST obtained a reduced failure rate for four annotated attributes.The attributes were camera motion, illumination change, occlusion, size change, and motion change.
- VOT Challenge 2014: PLT-13 and PLT-14 showed better robustness across all attributes than fDSST, SAMF, and KCF.The PLT approaches use online structural SVM detectors and feature selection.
- VOT Challenge 2014: Correlation trackers such as fDSST were more accurate than PLT, likely because they compute dense classification scores in a limited search region.
- VOT Challenge 2014: fDSST used the same parameter settings on both datasets.
7 CONCLUSION
The paper presents scale-adaptive tracking with separate discriminative filters for translation and scale, alongside cost-reduction strategies for real-time operation. Experiments on OTB and VOT 2014 report broad state-of-the-art performance, while the method uses only intensity-based image representation.
- 7 CONCLUSION: The method learns separate discriminative correlation filters for translation and scale estimation.Its explicit scale filter is learned from appearance changes induced by scale variations.
- 7 CONCLUSION: Computational-cost reductions enable a larger target search space without sacrificing real-time performance.The resulting tracker provides a twofold gain in speed.
- 7 CONCLUSION: The approach outperformed 19 state-of-the-art trackers on OTB and 37 on VOT 2014, obtaining the top combined ranking score on VOT.
- 7 CONCLUSION: The experiments used only intensity-based image representation; future work proposes efficient fusion of intensity and color information.The paper expects color information could improve performance, especially with target deformations.
APPENDIX
The appendix derives the correlation-filter solution in the Fourier domain by decomposing the objective into independent frequency-wise quadratic problems and solving their normal equations analytically.
- APPENDIX: Parseval’s formula and the DFT correlation property transform the filter-loss derivation into the frequency domain.
- APPENDIX: F(n) and H(n) are d-dimensional complex vectors, and F(n)∗H(n) denotes their inner product.
- APPENDIX: Each frequency term can be minimized independently because it depends only on the filter coefficients at that frequency.
- APPENDIX: The resulting normal equations are solved analytically using the inverse formula for a rank-1 matrix adjustment.
- APPENDIX: Using A = λI_d and x = y = F(n) in the rank-1 inverse formula yields an expression equivalent to solution (2).