Source-linked AI summary
Target-Aware Deep Tracking
Xin Li, Chao Ma, Baoyuan Wu, Zhenyu He, Ming-Hsuan Yang
TL;DR
Visual tracking needs features that distinguish arbitrary targets from background despite appearance and scale changes, while generic pre-trained features can be target-agnostic and high-dimensional. The paper learns target-aware features with regression and ranking losses, selects filters using gradients, and integrates them into a Siamese tracker. Across five benchmark datasets, the resulting tracker performs favorably against state-of-the-art methods in accuracy and speed.
Problem
Generic pre-trained deep features are less effective for tracking arbitrary targets because they are target-agnostic, insensitive to intra-class variation, and high-dimensional.
Method
The method learns target-aware features with regression and ranking losses, selecting convolutional filters via back-propagated gradients before integrating them into a Siamese matching network.
Results
The proposed tracker performs favorably against state-of-the-art methods in accuracy and speed across five benchmark datasets.
Takeaways & Limitations
Selecting target-active and scale-sensitive filters provides a compact feature representation for Siamese visual tracking.
Abstract
from arXiv · showhide
Existing deep trackers mainly use convolutional neural networks pre-trained for generic object recognition task for representations. Despite demonstrated successes for numerous vision tasks, the contributions of using pre-trained deep features for visual tracking are not as significant as that for object recognition. The key issue is that in visual tracking the targets of interest can be arbitrary object class with arbitrary forms. As such, pre-trained deep features are less effective in modeling these targets of arbitrary forms for distinguishing them from the background. In this paper, we propose a novel scheme to learn target-aware features, which can better recognize the targets undergoing significant appearance variations than pre-trained deep features. To this end, we develop a regression loss and a ranking loss to guide the generation of target-active and scale-sensitive features. We identify the importance of each convolutional filter according to the back-propagated gradients and select the target-aware features based on activations for representing the targets. The target-aware features are integrated with a Siamese matching network for visual tracking. Extensive experimental results show that the proposed algorithm performs favorably against the state-of-the-art methods in terms of accuracy and speed.
1. Introduction
Visual tracking must represent arbitrary targets through appearance changes, but generic pre-trained features are poorly aligned with target-specific discrimination, localization, scale estimation, and efficiency. The paper proposes target-aware features, integrates them into a Siamese tracker, and reports favorable accuracy and speed across benchmarks.
- Visual tracking locates a bounding-box-specified target in later frames despite significant appearance changes and temporary disappearance from view.
- Generic pre-trained features receive less attention in tracking and are less effective because targets may have arbitrary forms, including unseen objects or specific object parts.
- Pre-trained features are agnostic to target objectness and semantics, retain insufficient localization and scale information, and impose high computational costs through high dimensionality.
- The method learns target-aware features with regression and ranking losses, using back-propagated gradients to select effective convolutional filters.
- The target-aware features are integrated with a Siamese matching network and evaluated on five benchmark datasets for accuracy and tracking speed.
- In t-SNE visualizations, target-aware features better expose within-class differences across videos and separate different object categories than pre-trained features.
2. Related Work
Related work includes deep trackers built on pre-trained classification features, Siamese matching frameworks, attention mechanisms, and gradient-based channel-importance models. This paper distinguishes its approach by designing tracking-specific regression and ranking losses.
- Deep trackers: Deep trackers commonly use pre-trained classification models, sometimes combining shallow and deep features or features from multiple layers.
- Deep trackers: Siamese trackers formulate tracking as matching between an initial target template and current-frame search regions, with later work adding attention mechanisms.
- Gradient-based deep models: Gradient-based deep models estimate the importance of CNN feature channels for describing specific object classes, often using classification losses.
- Gradient-based deep models: Unlike classification-oriented gradient models, this paper uses regression and ranking losses to identify filters active for targets and sensitive to scale changes.
3. Target-Aware Features
The paper learns target-aware features by using regression and ranking losses to select convolutional filters that are active on the target and sensitive to scale changes. These features are integrated into a Siamese matching tracker to improve target representation and tracking efficiency.
- Features of pre-trained CNNs: Pre-trained CNN features are less effective for tracking arbitrary targets because they lack target-specific objectness information and may retain insufficient detail for localization and scale estimation.The paper identifies arbitrary or unseen targets, high-level last-layer representations, and computational load as key concerns.
- Features of pre-trained CNNs: Gradient-based channel importance selects a subspace of convolutional features using losses designed specifically for visual tracking.The importance of channel i is computed by globally average pooling the gradient of the designed loss with respect to that filter's output feature.
- Framework: The framework combines a CNN backbone, target-aware model, and correlation matching module, whose maximum score-map response gives the target position.The target-aware model uses Ridge and ranking losses, while the matching module compares the template with the search region.
- Target-Active Features via Regression: Ridge regression identifies target-active filters by fitting samples around the target center to a Gaussian label map and back-propagating the regression loss.The resulting filters are active on the target while inactive on the background, producing discriminative features for arbitrary or unseen targets.
- Feature visualization: Visualizations show that target-aware features separate targets from background more effectively than the original pre-trained features.The visualization averages all channels and compares Conv4-1 and Conv4-3 features learned with or without the proposed losses.
- Scale-Sensitive Features via Ranking: A ranking loss identifies filters sensitive to scale changes by ranking paired training samples according to which sample has size closer to the target.The ranking-loss gradients are combined with regression-loss gradients to select filters that are both target-active and scale-sensitive.
4. Tracking Process
The tracker combines a pre-trained extractor, target-aware feature module, and Siamese matching module. It detects targets by comparing template and search-region features, while evaluating multiple search-region scales to select the strongest response.
- Online detection: The Siamese framework compares target and search-region features through a convolution operation to produce tracking responses.Its concise matching design makes tracking efficient and emphasizes the contribution of the feature model.
- Tracker initialization: The framework comprises a pre-trained feature extractor, a target-aware feature module, and a Siamese matching module.The extractor is trained offline for classification, while the target-aware part is trained only in the first frame.
- Online detection: During inference, response-map values indicate confidence that corresponding positions contain the target.The predicted position is obtained from the response map for the current search region.
- Scale evaluation: Scale evaluation compares the fixed template with smaller, larger, and unchanged search-region feature maps.The selected scale is the one whose score map contains the highest response.
5. Experimental Results
The tracker is evaluated on five benchmarks, with experiments covering implementation, overall comparisons, and ablations of feature-selection losses. Results report competitive accuracy and real-time speed, while ablations support the contribution of regression- and ranking-guided features.
- Evaluation setup: The evaluation covers OTB-2013, OTB-2015, VOT-2015, VOT-2016, and Temple Color-128, with comparisons against correlation-filter and deep trackers.The study also includes ablation experiments analyzing different losses and feature layers.
- OTB results: 0.680 and 0.660 AUC scores are achieved on OTB-2013 and OTB-2015, respectively, with the best performance among real-time trackers.The proposed method also performs best on OTB-2015 among Siamese trackers with offline training.
- OTB results: 33.7 FPS is achieved while maintaining favorable performance against state-of-the-art real-time trackers on the OTB benchmarks.Figure 5 reports success and precision plots, while Table 1 reports AUC scores and running speeds across tracker categories.
- VOT results: 0.327 EAO, 0.59 accuracy, and 1.09 robustness are reported on VOT-2015, giving the tracker the second-best EAO and best accuracy.FlowTrack obtains higher EAO at 0.341 but runs at 12 FPS compared with the proposed tracker’s 33.7 FPS.
- VOT results: 0.55 accuracy and 0.299 EAO are obtained on VOT-2016, while the tracker runs at 33.7 FPS versus C-COT’s 0.3 FPS.The tracker is reported to perform well in accuracy, robustness, and running speed without online updating or offline training.
6. Conclusions
The paper learns target-aware features by selecting effective filters from pre-trained CNN layers with regression and ranking losses, then integrates them into a Siamese tracker. Experiments on five public datasets demonstrate favorable effectiveness and efficiency against state-of-the-art trackers.
- Target-aware features are learned to narrow the gap between pre-trained classification models and tracking targets of arbitrary forms.
- Regression and ranking losses use gradients to select effective filters from pre-trained CNN layers for target representation.
- The selected features are integrated with a Siamese tracking framework and evaluated on five public datasets.
- The proposed algorithm performs favorably against state-of-the-art trackers in effectiveness and efficiency.