Source-linked AI summary
DCFNet: Discriminant Correlation Filters Network for Visual Tracking
Qiang Wang, Jin Gao, Junliang Xing, Mengdan Zhang, Weiming Hu
TL;DR
DCF tracking methods often use hand-crafted or independently pretrained features, leaving feature extraction separate from correlation filtering. DCFNet jointly learns lightweight features and correlation tracking by embedding DCF as a Siamese-network layer with Fourier-domain backpropagation. It runs above 60 FPS and reports competitive accuracy, including gains over HoG-based KCF and DSST on OTB2015.
Problem
DCF trackers commonly rely on hand-crafted features or convolutional features pretrained independently for other tasks, rather than features learned specifically for DCF tracking.
Method
DCFNet adds a DCF correlation-filter layer to a Siamese network and jointly learns lightweight convolutional features through derived backpropagation.
Results
DCFNet runs at more than 60 FPS and achieves 10% and 6.2% success-plot gains over KCF and DSST using HOG features, respectively, on OTB2015.
Takeaways & Limitations
DCFNet provides a more compact and faster alternative to several state-of-the-art deep trackers while improving accuracy over HoG-based DCF tracking.
Abstract
from arXiv · showhide
Discriminant Correlation Filters (DCF) based methods now become a kind of dominant approach to online object tracking. The features used in these methods, however, are either based on hand-crafted features like HoGs, or convolutional features trained independently from other tasks like image classification. In this work, we present an end-to-end lightweight network architecture, namely DCFNet, to learn the convolutional features and perform the correlation tracking process simultaneously. Specifically, we treat DCF as a special correlation filter layer added in a Siamese network, and carefully derive the backpropagation through it by defining the network output as the probability heatmap of object location. Since the derivation is still carried out in Fourier frequency domain, the efficiency property of DCF is preserved. This enables our tracker to run at more than 60 FPS during test time, while achieving a significant accuracy gain compared with KCF using HoGs. Extensive evaluations on OTB-2013, OTB-2015, and VOT2015 benchmarks demonstrate that the proposed DCFNet tracker is competitive with several state-of-the-art trackers, while being more compact and much faster.
1. INTRODUCTION
DCF trackers balance accuracy and speed, but commonly rely on hand-crafted or independently pretrained features. DCFNet addresses this by jointly learning lightweight convolutional features and performing correlation tracking end to end.
- Problem: Object tracking estimates an initialized target’s trajectory despite deformations, appearance variations, occlusions, and real-time constraints.The target is initialized with a bounding box in the first frame without assuming its category beforehand.
- Motivation: DCF trackers use hand-crafted or independently pretrained features, which can be heavyweight and require separate feature extraction and correlation filtering.The paper identifies HoGs and features pretrained for image classification or object detection as common choices.
- Approach: DCFNet treats DCF as a special correlation filter layer within a Siamese network and derives backpropagation through it.This integrates feature learning and correlation tracking in one trainable architecture.
- Approach: The network uses lightweight convolutional layers to encode tracking knowledge offline and produces a probability heatmap of object location.The correlation filter layer handles online learning and tracking behind the convolutional feature extractor.
- Efficiency: Fourier-domain computation preserves DCF efficiency, enabling test-time speed above 60 FPS while maintaining competitive tracking accuracy.The paper positions this speed against heavyweight and slow state-of-the-art trackers.
2. RELATED WORK
DCF tracking has progressed from single-channel gray features to multi-channel hand-crafted and pretrained deep features. Related methods increasingly integrate deep features, but feature choices remain tied to existing extractors.
- DCF feature evolution: MOSSE introduced DCF tracking with single-channel gray features and reported speeds above 600 FPS.CSK and KCF subsequently generalized correlation filters to multi-channel features.
- DCF feature evolution: CN added color-names features to improve the performance of CSK.This followed the multi-channel feature developments represented by CSK and KCF.
- Deep-feature integration: Later DCF trackers integrated pretrained multi-layer deep features into correlation-filter tracking.The passage places these methods after earlier hand-crafted and color-based feature approaches.
3. THE PROPOSED NETWORK
The proposed network combines a feature extractor with a discriminant correlation filter layer, training features and tracking jointly while preserving Fourier-domain efficiency. Its online tracker updates filters incrementally, using a recurrent interpretation and a small sample footprint.
- Discriminant correlation filters: The standard DCF learns a filter by fitting target-patch features to a Gaussian response peaked at the object center with ridge regularization.The filter operates across feature channels using circular correlation.
- Network architecture: DCFNet cascades a feature extractor with a DCF module that produces an object-location response from search-patch features.The desired response is high at the real object location.
- Backpropagation: The DCF layer is trained end-to-end by backpropagating through Fourier-domain Hadamard operations and division.After gradients reach real-valued feature maps, conventional CNN optimization completes training.
- Online model update: During online tracking, DCFNet updates filters incrementally over time rather than maintaining a large sample set.The incremental process requires only a small footprint and can be viewed as an RNN.
- Online model update: The online tracking process recurrently forward-propagates and updates the numerator and denominator of the Fourier-domain filter.These quantities are updated according to Eq. (13).
4. EXPERIMENTS
Experiments evaluate DCFNet through ablations and comparisons on OTB2013, OTB2015, and VOT2015, emphasizing the balance between tracking accuracy and speed. The reported results show gains over HOG-based correlation-filter trackers and competitive performance against CNN-based trackers.
- Implementation: The lightweight network contains only 75KB of convolutional layers and is evaluated using 166,643 training frames from non-overlapping videos.Implementation uses conv1 from VGG with pooling removed and the output forced to 32 channels.
- Ablation study: The 3-layers design provides a balance between performance and tracking speed across tested scale-level settings.The ablation tests S = 1, 5, and 7.
- Comparison on OTB: 10% and 6.2% gains in success plots on OTB2015 are reported over KCF and DSST using HOG features, respectively.The comparison concerns DCFNet's simple feature training against HOG-based correlation-filter trackers.
- Comparison on OTB: DCFNet achieves competitive performance with much faster speed than the shallower network baselines cited in the comparison.The text also reports slightly better tracking than SiamFC with a more robust online update strategy.
- Comparison on VOT2015: On VOT2015, DCFNet is reported to achieve a balance between performance and speed compared with KCF and MUSTer.The comparison is made against the VOT2015 state-of-the-art bound shown in Fig. 4.
5. CONCLUSION
The conclusion presents DCFNet as a lightweight end-to-end approach that learns features suited to DCF tracking while improving accuracy over HOG-based methods. It also reports compactness and speed advantages over several deep-learning trackers, while leaving deeper architectures and larger training sets for future work.
- Conclusion: DCFNet learns convolutional features end-to-end for DCF tracking and can substitute feature extraction in common DCF trackers.The conclusion describes the learned features as fitting DCF-based tracking.
- Conclusion: DCFNet achieves a significant accuracy gain compared with DCF trackers using HoGs.The conclusion attributes the gain to the trained convolutional features.
- Conclusion: Evaluations on several benchmarks show DCFNet is more compact and much faster than several state-of-the-art deep-learning trackers.The conclusion reports this as a benchmark-level comparison.
- Future work: Using deeper architectures and larger training sets is identified as future work for obtaining potentially more robust feature extractors.This is stated as a future direction rather than a reported result of the current system.