Source-linked AI summary
Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking
Martin Danelljan, Gustav Häger, Fahad Shahbaz Khan, Michael Felsberg
TL;DR
Tracking-by-detection depends on training samples that the tracker labels itself, making the model vulnerable to corrupted data from misalignment, occlusion, and other perturbations. The paper jointly learns the appearance model and continuous sample-quality weights in a unified loss, rather than using a separate management component. It reports state-of-the-art tracking performance on three benchmarks, including a 3.8% mean overlap precision gain over baseline on OTB-2015.
Problem
Tracking-by-detection models rely on training samples labeled by the tracker, which can introduce corrupted samples from misalignment, occlusion, and other perturbations.
Method
The paper jointly learns the appearance model and training-sample weights by minimizing a single loss, using an alternating optimization strategy.
Results
State-of-the-art tracking performance is reported on OTB-2015, VOT-2015, and Temple-Color, including a 3.8% mean overlap precision gain over baseline on OTB-2015.
Takeaways & Limitations
The unified formulation down-weights corrupted samples and increases the importance of accurate samples without an explicit sample-management component.
Takeaways & Limitations
The weight regularization is necessary to prevent overfitting to samples from a single frame; as µ approaches zero, weights approach their prior values for fixed model parameters.
Abstract
from arXiv · showhide
Tracking-by-detection methods have demonstrated competitive performance in recent years. In these approaches, the tracking model heavily relies on the quality of the training set. Due to the limited amount of labeled training data, additional samples need to be extracted and labeled by the tracker itself. This often leads to the inclusion of corrupted training samples, due to occlusions, misalignments and other perturbations. Existing tracking-by-detection methods either ignore this problem, or employ a separate component for managing the training set. We propose a novel generic approach for alleviating the problem of corrupted training samples in tracking-by-detection frameworks. Our approach dynamically manages the training set by estimating the quality of the samples. Contrary to existing approaches, we propose a unified formulation by minimizing a single loss over both the target appearance model and the sample quality weights. The joint formulation enables corrupted samples to be down-weighted while increasing the impact of correct ones. Experiments are performed on three benchmarks: OTB-2015 with 100 videos, VOT-2015 with 60 videos, and Temple-Color with 128 videos. On the OTB-2015, our unified formulation significantly improves the baseline, with a gain of 3.8% in mean overlap precision. Finally, our method achieves state-of-the-art results on all three datasets. Code and supplementary material are available at http://www.cvl.isy.liu.se/research/objrec/visualtracking/decontrack/index.html .
1. Introduction
Generic visual tracking estimates a target’s trajectory from its initial location, but tracking-by-detection models are vulnerable to corrupted self-labeled training samples. The paper proposes jointly learning sample quality weights and the appearance model to reduce corruption without a separate sample-management component.
- Generic visual tracking estimates a target’s trajectory in an image sequence given only its initial location.
- Tracking-by-detection trains an appearance model from target and background samples, so model quality depends directly on the training set.
- Misaligned, occluded, and cluttered samples can contaminate the training set, reducing discriminative power and causing drift or tracking failure.
- Existing methods either ignore corrupted samples or manage them through separate components, often relying on heuristics.
- The proposed unified formulation jointly learns the appearance model and sample weights, down-weighting corrupted samples while increasing the impact of correct ones.
- 3.8% gain in mean overlap precision is reported on OTB-2015 versus the baseline, with state-of-the-art results on OTB-2015, VOT-2015, and Temple-Color.
2. Discriminative Tracking Methods
Discriminative tracking-by-detection learns appearance models from temporally collected positive and negative samples. Sample weights control their influence, but existing strategies mainly age, prune, reject, or otherwise manage samples using fixed rules or separate mechanisms.
- Discriminative tracking formulates appearance-model learning as supervised optimization over labeled feature vectors and their corresponding labels.
- SVMs and discriminative correlation filters are examples of learning approaches covered by the general loss formulation.
- The training set is updated with samples from successive frames, including positive and negative examples extracted around the estimated target location.
- Frame weights control sample impact: increasing a frame’s weight gives its samples greater influence during learning.
- Existing trackers reduce older-sample influence through learning-rate reweighting, age-based rejection, or support-vector removal.
- Corrupted samples arise because tracking algorithms, rather than human annotators, label data during tracking; rotations, deformations, occlusions, and blur contribute to contamination.
3. Our Approach
The approach jointly learns the appearance model and nonnegative sample-impact weights through one loss, using prior weights to regularize their adaptiveness. Alternating convex optimization updates both components and can correct corrupted-sample weights over time.
- Problem Formulation: The formulation jointly estimates appearance-model parameters θ and training-sample weights α by minimizing a single loss.It introduces a joint optimization framework for determining sample quality.
- Problem Formulation: The weights αk are nonnegative, sum to one, and are regularized toward prior weights ρk with flexibility controlled by µ.Larger µ permits more adaptive weights, while µ approaching zero recovers the standard weighted-loss formulation.
- Optimization: The biconvex objective is minimized by alternating convex searches that update θ and α in separate subproblems.Given one variable, the optimization over the other is convex; the α subproblem is solved with quadratic programming.
- Prior Weights Selection: Recent samples receive larger prior weights through a temporal decay rule, while samples older than K frames retain equal prior importance.The strategy is intended to address rapid appearance changes while preserving influence from older samples.
- Prior Weights Selection: In the Skating example, corrupted samples are later down-weighted while accurate samples remain highly weighted after weights are recomputed each frame.Samples 82 and 93 are corrupted by an occluding skater; samples 1 and 58 are accurate.
4. The Tracking Framework
The framework integrates the unified learning formulation into an SRDCF tracking-by-detection baseline. Each frame adds a training sample, then alternates model and weight updates while retaining a bounded training set.
- Baseline Tracker: The baseline appearance model is an SRDCF discriminative convolution filter trained on feature maps with Gaussian confidence-map labels.The filter estimates target location by maximizing a confidence map around the predicted location.
- Baseline Tracker: The per-sample loss is the total squared error between the desired confidence map and the filter’s confidence scores.The model also uses spatial regularization to penalize filter coefficients outside the target region.
- Unified Tracker: Instead of exponentially decaying fixed weights, the unified tracker jointly estimates θ and α in every frame.This formulation is applied directly to the baseline tracker’s weighted learning objective.
- Tracking Procedure: Each frame estimates the target, augments the training set, updates prior weights, and alternates “Update θ” with “Update α”.The new sample starts with its prior weight, while earlier weights are initialized from the previous frame.
- Tracking Procedure: When the training set exceeds T samples, the tracker removes the sample with the smallest impact weight among eligible samples.This bounds memory consumption while using the learned weights for sample selection.
5. Experiments
The method is evaluated on three standard visual-tracking benchmarks: OTB-2015, VOT-2015, and Temple-Color.
- Evaluation: Experiments use OTB-2015, VOT-2015, and Temple-Color benchmark datasets.The paper evaluates the approach across three tracking benchmarks.
5.1. Parameter Settings
The experiments use fixed benchmark-specific settings and compare the method with the baseline using mean overlap precision. Table 1 reports the OTB-2015 comparison and a 3.8% gain over baseline.
- Settings: The prior-weight settings use K = 50 and η = 0.035, while the flexibility parameter is µ = 5 in the reported experiments.The chosen µ enables a large degree of adaptiveness in the sample weights.
- OTB-2015: 3.8% gain in mean OP is achieved over the baseline tracker on OTB-2015.The comparison includes the baseline and versions incorporating entropy and PSR strategies.
- Implementation: The implementation starts joint learning at t = 10 frames and uses a single ACS iteration per frame.This is intended to refine estimates while minimally increasing computation relative to the original learning approach.
- Comparison Protocol: The same parameter settings are used for the proposed tracker and baseline within each comparison, with features selected by dataset.OTB-2015 uses HOG, while VOT-2015 and Temple-Color use HOG with Color Names.
5.2. Baseline Experiments
Baseline experiments compare the proposed approach with SRDCF and existing sample-management strategies using mean overlap precision on OTB-2015 and an SVM-based tracker. Additional comparisons examine AUC performance, qualitative re-detection, and initialization robustness.
- State-of-the-art comparison: On OTB-2015 and Temple-Color, the approach improves mean OP over the best existing trackers by 3.8% and 3.6%, respectively.The comparison uses mean OP (%) across both datasets.
- Success plots: Figure 3 reports AUC gains of 2.9% on OTB-2015 and 2.5% on Temple-Color over the best previous method.The figure presents success plots for both datasets.
- Mean OP comparison: 3.8% gain in mean OP over the SRDCF baseline demonstrates improved performance from decontaminating the training sample set.SRDCF obtains 72.9% mean OP, while the proposed approach provides the reported gain.
- SVM generality: The proposed formulation transfers to an SVM-based discriminative model, raising mean OP from 58.2% to 61.4%.This corresponds to a significant gain of 3.2% over the baseline SVM tracker.
- Qualitative comparison: The approach accurately re-detects the target in the Girl video after down-weighting corrupted training samples.The qualitative comparison covers the Box and Girl videos.
- Initialization robustness: Spatial and temporal initialization robustness both show consistent performance improvements compared with previous approaches.The evaluation uses SRE and TRE success plots on OTB-2015.
5.3. OTB-2015 Dataset
On OTB-2015, the approach is compared with recent trackers using mean OP, success-plot AUC, initialization robustness, and attribute-based analyses. It achieves the best reported performance and consistent gains in challenging corruption-related scenarios.
- State-of-the-art comparison: 76.7% mean OP is achieved on OTB-2015, significantly outperforming the best compared tracker.MEEM and HCF obtain 63.4% and 65.5% mean OP, respectively.
- Success plots: 63.4% AUC is achieved on the OTB-2015 success plot, exceeding SRDCF, LCT, and HCF at 60.5%, 56.7%, and 56.6%.The success plot displays AUC scores for the compared trackers.
- Robustness to initialization: The tracker provides consistent performance gains under both spatial robustness (SRE) and temporal robustness (TRE) evaluations.SRE perturbs the initial location, while TRE initializes tracking at 20 different frames.
- Attribute-based analysis: The tracker outperforms previous approaches on all 11 OTB-2015 attributes.Four attributes with commonly corrupted training samples receive focused success-plot analysis.
- Attribute-based analysis: Joint learning reduces the impact of misaligned and background-contaminated samples in scale variation, rotation, clutter, and occlusion scenarios.The passage links these corrupted samples to drift and tracking failure and describes per-frame re-determination of sample weights.
5.4. VOT-2015 Dataset
The VOT-2015 evaluation uses expected average overlap on 60 challenging videos to compare the proposed approach with state-of-the-art trackers. The reported comparison shows improved performance over the best existing tracker.
- Dataset and metric: VOT-2015 contains 60 challenging videos for tracker evaluation.The dataset evaluation is described in terms of expected average overlap.
- Dataset and metric: Expected average overlap is estimated from average overlap as a function of sequence length and the typical-sequence-length distribution.Both functions are cut off at a threshold where the mass is 0.5.
- Comparison: The proposed approach provides improved performance compared to the best existing tracker on VOT-2015.The comparison is reported using expected average overlap, with the table presenting AEO results.
5.5. Temple-Color Dataset
On Temple-Color, the proposed tracker outperforms the compared state-of-the-art methods in both mean overlap precision and AUC.
- 65.8% mean OP surpasses the 62.2% achieved by both MEEM and SRDCF on Temple-Color.
- 54.1% AUC exceeds MEEM’s 50.6% and SRDCF’s 51.6% on the 128-video dataset.
6. Conclusions
The paper concludes that unified learning addresses corrupted training samples by adjusting their influence within discriminative tracking. The formulation is generic, and experiments report state-of-the-art tracking performance.
- The unified formulation down-weights corrupted samples and up-weights accurate samples in tracking-by-detection.
- The approach can be integrated into other discriminative tracking frameworks.
- Experiments demonstrate state-of-the-art tracking performance.