Source-linked AI summary
Correlation Tracking via Joint Discrimination and Reliability Learning
Chong Sun, Dong Wang, Huchuan Lu, Ming-Hsuan Yang
TL;DR
Existing CF trackers emphasize discrimination but can be dominated by salient yet unreliable feature regions. The paper introduces DRT, which jointly learns a discrimination base filter and reliability term with local response consistency, and reports favorable performance across OTB-2013, OTB-2015, and VOT-2016. The method remains vulnerable to cluttered backgrounds and rapid large deformations or rotations.
Problem
Existing CF tracking methods often focus on discrimination while paying less attention to reliability, allowing unreliable salient feature regions to dominate learned filters.
Method
DRT represents the filter as the element-wise product of a base filter and reliability weight map, adds local response consistency, and jointly optimizes them using alternating direction methods.
Results
DRT achieves the best reported results across OTB-2013, OTB-2015, and VOT-2016, including an EAO of 0.442 versus ECO's 0.374 on VOT-2016.
Takeaways & Limitations
Jointly modeling discrimination and reliability makes the tracker less susceptible to non-uniform feature-map responses and better able to suppress background regions.
Takeaways & Limitations
Cluttered backgrounds and rapid large deformations or rotations can still cause tracking failures; optical-flow information is identified as future work.
Abstract
from arXiv · showhide
For visual tracking, an ideal filter learned by the correlation filter (CF) method should take both discrimination and reliability information. However, existing attempts usually focus on the former one while pay less attention to reliability learning. This may make the learned filter be dominated by the unexpected salient regions on the feature map, thereby resulting in model degradation. To address this issue, we propose a novel CF-based optimization problem to jointly model the discrimination and reliability information. First, we treat the filter as the element-wise product of a base filter and a reliability term. The base filter is aimed to learn the discrimination information between the target and backgrounds, and the reliability term encourages the final filter to focus on more reliable regions. Second, we introduce a local response consistency regular term to emphasize equal contributions of different regions and avoid the tracker being dominated by unreliable regions. The proposed optimization problem can be solved using the alternating direction method and speeded up in the Fourier domain. We conduct extensive experiments on the OTB-2013, OTB-2015 and VOT-2016 datasets to evaluate the proposed tracker. Experimental results show that our tracker performs favorably against other state-of-the-art trackers.
1. Introduction
Visual tracking remains challenging under deformations, rotations, and other changes, while CF trackers can be influenced by background regions and unreliable high-response features. DRT addresses this by jointly learning discrimination and reliability through a factorized filter, local response consistency, and reliability weighting.
- Visual tracking remains challenging because targets may undergo large deformations, rotations, and other changes.
- CF trackers exploit many cyclically shifted samples efficiently but can learn filters influenced by background regions.
- DRT reports comparable or better example tracking results than ECO on the OTB dataset.
- CNN feature maps are often sparse and non-uniform, causing filters to emphasize high-response regions that are not always reliable.
- DRT factorizes the filter into a discrimination-focused base filter and a jointly learned reliability term.
- The local response consistency constraint gives different base-filter sub-regions similar importance, while the reliability map models their relative reliability.
2. Relate Work
Related work develops correlation-filter trackers from efficient single-channel and kernelized formulations toward patch-based methods that combine or jointly learn sub-patch information.
- Correlation filters are widely used in visual tracking because their learning process is efficient.
- Early CF trackers use single feature channels, with MOSSE optimizing squared error and KCF extending kernelized Fourier-domain formulations.
- Patch-based CF methods combine part trackers or jointly learn coefficients for multiple patches under assumptions about sub-patch motion.
3. Proposed Method
The proposed tracker jointly learns a discriminative base filter and a reliability map, with local response consistency encouraging balanced contributions across target regions. The resulting non-convex problem is optimized by alternating updates and accelerated through Fourier-domain computations.
- Joint discrimination and reliability modeling: The filter weight for each feature channel is modeled as the element-wise product of a base filter and a reliability map.The base filter captures discrimination, while the reliability map assigns weights to target regions and zeros to non-target regions.
- Joint discrimination and reliability modeling: The target is divided into M patches, with a shared coefficient β_m representing each patch’s reliability across channels.Patch masks crop the filter region associated with each reliability coefficient.
- Joint discrimination and reliability modeling: The reliability map is nonzero only on the target region, enabling real positive and negative samples to be generated by circularly shifting the input image.The target-region mask defines the support of the reliability map used in the filter.
- Joint discrimination and reliability modeling: The objective combines classification loss, local response consistency, and squared ℓ2 regularization while constraining reliability coefficients to avoid concentrating on a small target region.The consistency term promotes similar importance among local regions, and the coefficient constraints encourage an accurate reliability map.
- Joint discrimination and reliability learning: The joint optimization is non-convex in the base filter and reliability vector but becomes convex when either variable is fixed, enabling alternating-direction optimization.The base-filter subproblem uses conjugate gradient descent, while related computations exploit circulant structure and Fourier-domain operations.
- Model extension: The method extends to multiple frames by weighting each frame’s sample matrix with a temporal coefficient, while retaining the previous solution derivations.The reported computational complexity for associated Fourier-domain updates is O(DK log K).
- Model extension: Examples show lower reliability values for frequently occluded regions, fast-moving legs, and background regions.These learned weights are intended to support subsequent tracking by downweighting unreliable patches.
4. Model Update
The tracker uses sparse model updates to reduce computation, while Figure 4 illustrates learned reliability maps for training samples.
- Model update: Sparse model updates reduce the computation required by correlation-filter trackers.The tracker follows ECO's sparse update mechanism.
- Reliability maps: Figure 4 shows three example training samples followed by their learned reliable weight maps.
- Model update: During update frames, the tracker updates the base-filter coefficients with conjugate gradient descent and reliability weights by quadratic programming.
5. Target Localization
The tracker jointly localizes the target and estimates its scale using multi-scale response maps, while evaluations compare precision and success across datasets and attributes.
- Multi-scale search: The tracker extracts multi-scale ROI regions centered at the previous estimate and computes a response map for each scale.
- Joint estimation: Target location and scale are jointly determined by selecting the maximum across the response maps.
- Evaluation: OTB evaluations compare top trackers using distance precision at 20 pixels and area-under-curve scores across datasets and benchmark attributes.
- Joint estimation: This joint estimation strategy performs better than methods that estimate position first and refine scale afterward.
6. Experimental Results
DRT is evaluated on OTB-2013, OTB-2015, and VOT-2016 against state-of-the-art trackers, with component ablations and failure cases also examined. It achieves leading benchmark results, while failures remain under cluttered backgrounds and rapid target deformation or rotation.
- OTB-2013: DRT obtains the best OTB-2013 performance, reaching 95.3% distance precision at 20 pixels and 72.0% AUC.
- OTB-2013: DRT achieves the best results across eight OTB-2013 attributes and improves over ECO by 1.4%–2.9% on deformation, background clutter, illumination variation, and scale variation.
- OTB-2015: DRT provides the best OTB-2015 result, with 92.3% distance precision and 69.9% AUC, improving over ECO, C-COT, and DeepSRDCF.
- VOT-2016: On VOT-2016, DRT reaches 0.442 EAO versus ECO’s 0.374, a relative performance gain of 18.2%, and has the fewest failures among compared methods.
- Ablation Studies: The joint learning formula improves the baseline by 3.5% distance precision and 2.3% AUC on OTB-2015, and by 6.9% EAO on VOT-2016.
- Failure Cases: Failure cases arise from distractors in cluttered backgrounds and from large deformations and rotations occurring within a short time span.The authors identify optical-flow information as future work for partially addressing these failures.
7. Conclusion
The conclusion presents DRT as a correlation-filter tracker that jointly learns discrimination and reliability through a base filter, reliability map, and local response consistency constraint. Across three benchmarks, the reported experiments show performance superior to state-of-the-art algorithms.
- DRT rewrites the filter weight as the element-wise product of a base filter and a reliability weight map.
- A local response consistency constraint gives target sub-regions similar importance while separating reliability information from the base filter.
- The joint learning problem is solved with Fourier-domain acceleration, and experiments on OTB-2013, OTB-2015, and VOT-2016 report superiority over state-of-the-art algorithms.