Source-linked AI summary
GradNet: Gradient-Guided Network for Visual Object Tracking
Peixia Li, Boyu Chen, Wanli Ouyang, Dong Wang, Xiaoyun Yang, Huchuan Lu
TL;DR
Siamese trackers achieve real-time speed with fixed templates but struggle with target appearance variation and background clutter. GradNet uses gradient-guided feed-forward and backward operations for template adaptation, supported by template-generalization training to reduce overfitting. Experiments report improved tracking over real-time trackers on four benchmarks, with 80fps runtime.
Problem
Siamese trackers rely on a fixed initial template, limiting adaptation to target appearance variations and background clutter while increasing tracking-drift risk.
Method
GradNet uses discriminative gradients in two feed-forward and one backward calculation to update the siamese template, with template-generalization training.
Results
Experiments on four benchmarks show significantly improved tracking performance compared with other real-time trackers, at 80fps.
Takeaways & Limitations
Gradient-guided template updating provides accurate tracking while preserving high-speed inference in the reported experiments.
Takeaways & Limitations
The tracker is trained only on ILSVRC2014 VID, and the whole network is fixed during inference.
Abstract
from arXiv · showhide
The fully-convolutional siamese network based on template matching has shown great potentials in visual tracking. During testing, the template is fixed with the initial target feature and the performance totally relies on the general matching ability of the siamese network. However, this manner cannot capture the temporal variations of targets or background clutter. In this work, we propose a novel gradient-guided network to exploit the discriminative information in gradients and update the template in the siamese network through feed-forward and backward operations. Our algorithm performs feed-forward and backward operations to exploit the discriminative informaiton in gradients and capture the core attention of the target. To be specific, the algorithm can utilize the information from the gradient to update the template in the current frame. In addition, a template generalization training method is proposed to better use gradient information and avoid overfitting. To our knowledge, this work is the first attempt to exploit the information in the gradient for template update in siamese-based trackers. Extensive experiments on recent benchmarks demonstrate that our method achieves better performance than other state-of-the-art trackers.
1. Introduction
Siamese trackers run in real time but fixed initial templates cannot adapt well to target appearance changes or background clutter. GradNet addresses this gap by using gradients for one-step template adaptation and template-generalization training.
- Motivation: Gradient values can reflect both target variations and background clutter, providing discriminative information beyond template appearance.Figure 1 visualizes target patches alongside absolute gradient values, with red regions indicating large gradients.
- Motivation: Siamese-based trackers use a fixed initial target feature during testing, enabling real-time speed but limiting adaptation to appearance variations.Their limited online adaptability increases the risk of tracking drift.
- Gradient-guided adaptation: One backward update is difficult because no learning rate makes SiameseFC converge in a single iteration under nonlinear gradient-based optimization.Table 1 reports that there is no proper step for one-iteration convergence.
- Gradient-guided adaptation: GradNet simulates nonlinear gradient-based optimization with two feed-forward calculations and one backward calculation to adapt the template.The design reduces the number of online optimization iterations while retaining gradient-guided updating.
- Template generalization: Template-generalization training is introduced because GradNet can otherwise rely on template appearance and overfit instead of using gradients effectively.Normal training reaches low training error quickly but produces less promising test accuracy.
- Results: Experiments on four popular benchmarks report promising real-time tracking at 80fps.The contribution statement identifies both benchmark evaluation and runtime as headline outcomes.
2. Related Work
Related tracking methods update models through template combination, gradient descent, or correlation-based optimization, each differing in information used and computational demands. GradNet instead targets one-iteration gradient-guided template updating within a siamese tracker.
- Siamese Network based Tracking: SiameseFC and related siamese trackers match an initial target representation to candidates without important online updating.Their fixed models can be disturbed by similar instances or background noise when appearance changes.
- Template Combination: Template-combination methods merge target features from previous frames but ignore background clutter.GradNet is contrasted with these methods because it uses discriminative information in backward gradients.
- Gradient-descent based approaches: Gradient-descent trackers use backward gradients to update models through many iterations, which reduces tracking speed.The cited approaches require hundreds or many iterations to capture target appearance variations.
- Correlation based Tracking: Correlation-based trackers update classifiers by solving closed-form optimization problems, while deep networks mainly provide robust features.GradNet instead seeks to update the template through a deep network.
- Gradient Exploiting: Offline deep-network training commonly uses hand-designed gradient optimizers such as momentum, Adagrad, and ADAM, often requiring expensive computation and large datasets.This motivates interest in accelerating gradient-based optimization.
- Meta Learning: Compared with optimization-based meta-learning, GradNet updates only the template, uses one iteration, and includes second-order gradients during optimizer training.These differences are specifically described for the visual-tracking update task.
3. Proposed Algorithm
GradNet updates SiameseFC templates using gradient-guided feed-forward and backward operations, then trains template generalization to improve adaptation and reduce overfitting. The pipeline generates discriminative templates from target features, search-region information, and gradients for online tracking.
- Template Generation: GradNet uses two branches: one extracts search-region features, while an update branch generates templates through shared-parameter subnetworks and forward/backward propagation.The search branch processes X; the update branch performs template generation.
- Template Generation: The update branch is trained with gradient information because one-step gradient adaptation cannot properly update a SiameseFC template through ordinary nonlinear optimization.The method targets online updating while reducing the number of training iterations.
- Template Generation: The initial target feature is transformed into a template, whose score map and loss produce gradients used to obtain an updated target feature and optimal template.The updated template then produces the final score map used to estimate the target position.
- Template Generalization: Without template generalization, training tends to emphasize template appearance over gradients and cannot avoid overfitting.The authors report a lower gradient weight ratio for training without template generalization.
- Template Generalization: Template generalization uses one template across search regions from different videos, encouraging gradient-based adaptation and a versatile template rather than reliance on initial appearance.Training batches contain multiple image pairs, while the same template is used to search all selected regions.
- Online Tracking: During online testing, the fixed update branch initializes the template from the first frame and updates it with one reliable training sample through one iteration.Reliable samples are saved from tracking results and used with the current template for subsequent updates.
4. Experiments
Experiments across four tracking benchmarks evaluate GradNet against real-time trackers and ablate its components. The results show strong benchmark performance, real-time speed, and benefits from template generalization and gradient-guided updating.
- Experimental Setup: 80fps: the tracker runs in real time on four popular benchmarks using a CPU and GPU implementation.The experiments cover OTB-2015, TC-128, VOT-2017, and LaSOT.
- Experimental Setup: The evaluation compares GradNet with real-time deep and traditional trackers using precision, success, accuracy, robustness, and EAO measures.OTB-2015 uses success and precision plots; VOT2017 ranks trackers by EAO.
- OTB-2015: On OTB-2015, GradNet improves over SiameseFC by almost 8% in precision and 6% in success.It performs slightly below ECO-HC in success while handling challenging factors consistently.
- VOT2017: On VOT2017, GradNet achieves the best EAO while maintaining competitive accuracy and robustness.Its EAO is 3.5% higher than the real-time challenge winner CSRDCF++ and exceeds SiamRPN despite using less training data.
- Ablation Analysis: Across ablations, all components improve tracking accuracy, while the full method raises Ours-baseline precision by about 9% and IOU by 5%.The comparison with Ours w/o M supports the contribution of template generalization training.
- Training Analysis: Template generalization produces noisy initial score maps that become target-focused after gradient-based updating.Without template generalization, the model tends to produce target-localized initial maps directly; with it, the two stages learn general embedding and gradient-based template updating.
5. Conclusions
GradNet updates siamese templates using discriminative gradients through feed-forward and backward operations, while template generalization supports versatile templates and reduces overfitting. Experiments on four benchmarks report improved tracking performance at high speed.
- Conclusions: GradNet exploits discriminative gradients to update the template and accelerate a hand-designed optimization process.The method uses two sub-networks with feed-forward and backward operations.
- Conclusions: Template generalization during offline training helps the update branch focus on gradients and avoid overfitting.The stated purpose is to make fuller use of gradients and obtain versatile templates.
- Conclusions: Experiments on four benchmarks show significantly improved tracking performance compared with other real-time trackers.The conclusion also characterizes the tracker as accurate and high speed.