Source-linked AI summary
Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object Tracking
Tianyang Xu, Zhen-Hua Feng, Xiao-Jun Wu, Josef Kittler
TL;DR
Visual tracking must handle redundant, irrelevant, and temporally varying multi-channel representations while learning discriminative filters. GFS-DCF jointly selects spatial and channel feature groups and constrains filters with low-rank temporal smoothness. Across multiple tracking benchmarks, the authors report effective and robust performance, including superiority over state-of-the-art trackers.
Problem
The structural relevance of high-dimensional multi-channel features is insufficiently investigated, leaving redundancy and irrelevance in DCF filter learning.
Method
GFS-DCF jointly learns discriminative filters with group feature selection across spatial and channel dimensions and low-rank temporal smoothness.
Results
GFS-DCF demonstrates effectiveness and robustness on OTB2013, OTB2015, VOT2017, VOT2018, and TrackingNet, with reported superiority over state-of-the-art trackers.
Takeaways & Limitations
The learned filters use adaptive temporal-spatial-channel configurations with enhanced discrimination and interpretability while reducing feature redundancy.
Abstract
from arXiv · showhide
We propose a new Group Feature Selection method for Discriminative Correlation Filters (GFS-DCF) based visual object tracking. The key innovation of the proposed method is to perform group feature selection across both channel and spatial dimensions, thus to pinpoint the structural relevance of multi-channel features to the filtering system. In contrast to the widely used spatial regularisation or feature selection methods, to the best of our knowledge, this is the first time that channel selection has been advocated for DCF-based tracking. We demonstrate that our GFS-DCF method is able to significantly improve the performance of a DCF tracker equipped with deep neural network features. In addition, our GFS-DCF enables joint feature selection and filter learning, achieving enhanced discrimination and interpretability of the learned filters. To further improve the performance, we adaptively integrate historical information by constraining filters to be smooth across temporal frames, using an efficient low-rank approximation. By design, specific temporal-spatial-channel configurations are dynamically learned in the tracking process, highlighting the relevant features, and alleviating the performance degrading impact of less discriminative representations and reducing information redundancy. The experimental results obtained on OTB2013, OTB2015, VOT2017, VOT2018 and TrackingNet demonstrate the merits of our GFS-DCF and its superiority over the state-of-the-art trackers. The code is publicly available at https://github.com/XU-TIANYANG/GFS-DCF.
1. Introduction
GFS-DCF addresses the challenge of tracking arbitrary objects under appearance variation by selecting features jointly across spatial and channel dimensions while enforcing temporal smoothness. The method learns adaptive filter configurations and is evaluated across established tracking benchmarks.
- Visual object tracking must localise arbitrary objects consistently despite broad appearance variations in unconstrained videos.
- DCF trackers benefit from spatial regularisation, temporal smoothness, and robust image features, especially deep CNN features.
- High-dimensional deep feature channels may be structurally redundant or irrelevant because tracking has limited training samples and often relies on pretrained networks.
- GFS-DCF jointly selects groups across spatial and channel dimensions to reduce feature redundancy and irrelevance while learning discriminative filters.
- A low-rank temporal smoothness constraint encourages correlated filters across successive frames and consistent temporal-spatial-channel configurations.
- GFS-DCF is evaluated on OTB13/15, VOT17/18, and TrackingNet, with results reported as demonstrating merits and superiority over state-of-the-art trackers.
2. Related Work
Related work covers major DCF-based tracking developments and identifies unaddressed structural relevance in multi-channel features. GFS-DCF combines spatial and channel group selection with low-rank temporal smoothing to address redundancy, interference, and temporal variation.
- DCF-based tracking is emphasized because of its strong performance in recent visual tracking competitions.
- Early DCF developments introduced discriminative filter learning, circulant matrices for additional background samples, spatial-temporal context, and kernel methods.
- High-performance tracking remains difficult because of spatial boundary effects, limited feature representation capacity, and temporal filter degeneration.
- Spatial regularisation methods address boundary effects through weighting, pruning, masks, or output-region constraints.
- Existing DCF trackers use hand-crafted and deep feature combinations, but structural relevance, redundancy, and interference across multi-channel features remain insufficiently addressed.
- GFS-DCF performs joint spatial-channel group feature selection and uses an efficient low-rank approximation to smooth filters across frames under temporal appearance variation.
3. DCF-based Visual Object Tracking
DCF-based tracking learns multi-channel correlation filters from feature tensors and desired response maps, then updates and applies them across frames. Frequency-domain computation and response-map maximisation support efficient localisation.
- DCF localises an object in successive frames by learning multi-channel filters from features extracted in the current frame.
- The training data comprise a feature tensor X_t and desired response map Y, and filter learning is formulated as a regularised least-squares problem.
- A closed-form solution to the DCF optimisation can be obtained efficiently in the frequency domain.
- Filters learned from the first frame are used for the second frame and updated in subsequent frames using a predefined update rate.
- For a new search window, the learned filters estimate a response map efficiently in the frequency domain.
- The predicted target location is the position of the maximum value in the response map after inverse discrete Fourier transformation.
4. Group Feature Selection for DCF
GFS-DCF addresses redundancy and noise in multi-channel DCF features by jointly selecting spatial locations and feature channels, while adding temporal smoothness through a low-rank constraint. The resulting scheme adaptively learns compact spatial-channel filters and improves tracking performance.
- Motivation: Spatial feature selection is insufficient for high-dimensional deep CNN representations, whose hundreds or thousands of channels may contain redundancy and interference.Existing approaches mainly reduce redundancy in the spatial dimension, leaving channel-wise selection underexplored.
- Joint spatial-channel selection: GFS-DCF incorporates group feature selection across spatial and channel dimensions within the DCF optimisation problem.Grouping assigns variables according to spatial location and channel attributes, enabling structured sparsity and compact target representations.
- Spatial selection: Spatial regularisation groups feature elements across channels at each spatial location, using an ℓ2-based grouping attribute followed by an implicit ℓ1 sparsity penalty.This structured sparsity selects spatial groups according to their joint contribution across channels.
- Evaluation: Figure 2 compares spatial and channel group selection on OTB2015 for hand-crafted and deep CNN features across selection ratios.The 100% selection ratio represents the original DCF tracker without feature selection.
- Channel selection: Channel regularisation groups each feature channel using its Frobenius-norm attribute and an implicit ℓ1 constraint, then removes channels with the lowest grouping attributes.Selection is applied separately to each individual feature type using a predefined proportion of retained spatial units or channels.
- Evaluation: Group spatial-channel regularisation enables dimensionality reduction while improving tracking performance, with benefits depending on the feature representation.The reported results attribute performance gains to reducing redundancy and ambiguity in the representations.
- Temporal smoothness: A low-rank temporal constraint promotes coherence among filters across frames and improves robustness to appearance variation.Because direct rank computation is inefficient for long videos, the method uses a sufficient substitute based on previous learned filters and their mean.
- Optimisation: The resulting optimisation is convex and is solved with an augmented Lagrange formulation using a slack variable and alternating direction method of multipliers.The Lagrange multiplier and penalty parameter enforce equality between the slack and filter variables during iterative optimisation.
5. Evaluation
The evaluation studies GFS-DCF through component ablations, filter visualisation, parameter sensitivity, and comparisons across OTB, VOT, and TrackingNet benchmarks. GFS-DCF consistently improves tracking performance over classical and state-of-the-art trackers while retaining favourable speed.
- Ablation Study: The ablation evaluates spatial selection, channel selection, and low-rank temporal smoothness against a matched classical DCF baseline on OTB2013.The study constructs BaseLine, BaseLine SS, BaseLine CS, BaseLine LR, and BaseLine ALL trackers.
- Ablation Study: 10.1%/6.1% and 10.6%/6.7% improvements in DP/AUC are obtained by channel/spatial selection over classical DCF.The reported gains correspond to BaseLine CS and BaseLine SS, respectively.
- Ablation Study: The low-rank temporal smoothness term improves tracking and makes learned filters more invariant to appearance variations.The authors report that the simplified regularisation achieves a low-rank property using only the filter model, avoiding the increased complexity and storage of some historical-frame methods.
- Ablation Study: GFS-DCF produces sparsity by adaptively shrinking channel and spatial groups rather than using a predefined spatial mask or weighting.The filter visualisation shows that elements can be shrunk even within the centre region.
- Ablation Study: Performance remains stable for λ1, λ2 ∈[0.01, 100] and λ3 ∈[10, 100], with selection ratios identified as the most essential parameters.Threshold pruning fixes the selected spatial-unit and channel ratios, supporting robustness to regularisation parameters.
- Comparison with the State-of-the-art: GFS-DCF achieves 1.5%/1.2% and 1.4%/0.2% gains in DP/AUC over the second-best tracker on OTB2013 and OTB2015, respectively.The tracker also achieves the best reported EAO score of 0.397 on VOT2017/VOT2018, with Accuracy 0.511 and Robustness 0.143 within the top three.
6. Conclusion
The paper presents an effective appearance model using spatial-channel group-sparse discriminative correlation filters with low-rank temporal constraints. Experiments support its effectiveness and robustness, while feature-diversity analysis indicates that selection strategies should vary across feature categories.
- The method learns spatial-channel group-sparse discriminative correlation filters constrained by low-rank approximation across successive frames.This produces adaptive temporal-spatial-channel filter learning on a low-dimensional manifold with enhanced interpretability.
- Extensive visual-tracking benchmark experiments demonstrate the method’s effectiveness and robustness compared with state-of-the-art trackers.
- Analysis of hand-crafted and deep features shows that different feature categories require different spatial and channel selection strategies.