Source-linked AI summary
Context-aware Deep Feature Compression for High-speed Visual Tracking
Jongwon Choi, Hyung Jin Chang, Tobias Fischer, Sangdoo Yun, Kyuewang Lee, Jiyeoup Jeong, Yiannis Demiris, Jin Young Choi
TL;DR
Real-time tracking must reconcile strong performance with the computational cost of online adaptation and high-dimensional deep features. The paper proposes context-aware compression with specialized auto-encoders, denoising, and orthogonality-aware fine-tuning; TRACA maintains competitive performance while running at over 100 fps. Its self-comparisons also show that the context selection, adaptation, denoising, and multi-stage design materially affect performance.
Problem
Online deep trackers can be accurate but too slow, while correlation filters using raw deep features remain too high-dimensional for real-time tracking.
Method
TRACA clusters training samples into contexts, trains one expert auto-encoder per cluster, selects an expert for each target, and fine-tunes it using denoising and orthogonality-aware objectives.
Results
Over 100 fps: TRACA achieves competitive or comparable performance to state-of-the-art trackers, including non-real-time trackers, across the reported comparisons.
Takeaways & Limitations
Context-aware deep feature compression enables correlation-filter tracking that combines reduced computational cost with competitive tracking performance.
Abstract
from arXiv · showhide
We propose a new context-aware correlation filter based tracking framework to achieve both high computational speed and state-of-the-art performance among real-time trackers. The major contribution to the high computational speed lies in the proposed deep feature compression that is achieved by a context-aware scheme utilizing multiple expert auto-encoders; a context in our framework refers to the coarse category of the tracking target according to appearance patterns. In the pre-training phase, one expert auto-encoder is trained per category. In the tracking phase, the best expert auto-encoder is selected for a given target, and only this auto-encoder is used. To achieve high tracking performance with the compressed feature map, we introduce extrinsic denoising processes and a new orthogonality loss term for pre-training and fine-tuning of the expert auto-encoders. We validate the proposed context-aware framework through a number of experiments, where our method achieves a comparable performance to state-of-the-art trackers which cannot run in real-time, while running at a significantly fast speed of over 100 fps.
1. Introduction
Deep tracking methods face a speed–representation challenge: online fine-tuning can be accurate but slow, while raw deep features are too high-dimensional for real-time correlation filtering. TRACA addresses this with context-aware compression using specialized auto-encoders and achieves over 100 fps with competitive tracking performance.
- Online deep trackers achieve high accuracy and robustness, but their continuous fine-tuning is too slow for real-time tracking.
- Over 100 fps: TRACA delivers comparable performance to the best-performing non-real-time trackers on the CVPR2013 dataset.
- Raw deep convolutional features are high-dimensional, so correlation-filter trackers using them also fail to meet real-time requirements.
- TRACA compresses raw deep features with multiple expert auto-encoders specialized by object category and selected contextually for each target.Categories are found by clustering training samples according to contextual information.
- The selected auto-encoder is fine-tuned with an orthogonality-aware loss and denoising processes before correlation filters operate on the compressed feature map.The compressed map is designed to reduce redundancy and sparsity while supporting tracking accuracy and computational efficiency.
2. Related Works
Related tracking work spans online deep-learning methods, correlation filters, and context-specialized networks. The cited approaches improve accuracy or efficiency in different ways, but raw deep features remain difficult to process in real time.
- Online deep learning based trackers: Online deep-learning trackers use continuous network fine-tuning or learned similarity and classification mechanisms to model changing target appearance.
- Correlation filter based trackers: Correlation-filter trackers have improved through multi-channel inputs, kernel training, scale modeling, long-term memory, and attentional mechanisms.
- Correlation filter based trackers: Deep convolutional features give correlation-filter trackers state-of-the-art performance, including through regularized, continuous-domain, and multi-resolution formulations.
- Correlation filter based trackers: Raw deep features still contain too many channels for real-time processing, despite efforts to reduce the feature space with linear combinations.
- Multiple-context deep learning frameworks: Multiple-context frameworks divide samples by contextual properties and train specialized networks, motivating context-specialized tracking models.
3. Methodology
TRACA combines context-selected expert auto-encoders with VGG-Net features and correlation filters to compress target representations for tracking. Its training pipeline uses denoising, multi-stage reconstruction, clustering, expert fine-tuning, and a context-aware selector.
- Architecture: TRACA uses multiple expert auto-encoders, a context-aware network, and correlation filters to compress and track VGG-Net features.Each expert is trained for a different context, and the selector chooses the expert suited to the tracking target.
- Auto-encoder architecture: Each encoding layer halves the feature channels while preserving spatial resolution, producing a compact latent feature map that the decoder can reconstruct.The compressed representation is Z, formed after the stacked encoding layers, while decoding restores the original channel dimension.
- Pre-training: Pre-training first learns a context-independent base auto-encoder, clusters its compressed training features, and fine-tunes one expert auto-encoder per resulting cluster.A two-step clustering procedure removes the smallest clusters before reclustering to obtain clusters with enough samples for expert training.
- Training objectives: The base auto-encoder is trained on VGG-Net convolutional features, while a multi-stage distance loss addresses overfitting and unstable convergence beyond final-output reconstruction.Partial auto-encoders compare intermediate outputs with the input, providing the multi-stage training objective.
- Denoising: Two extrinsic denoising criteria corrupt randomly selected channels and exchange spatial feature vectors to improve robustness to appearance changes and occlusions.Channel corruption sets selected channels to zero, while vector exchange mimics spatial region interchange, including background-target occlusion effects.
- Context-aware network: The context-aware network predicts a training sample’s cluster index and is trained by minimizing a multiclass loss while keeping early VGG-M layers fixed.The network uses image/cluster-index pairs; the weights of fc5 and fc6 are trained while conv1 through fc4 remain fixed.
4. Experimental Result
Experiments evaluate TRACA’s implementation, datasets, metrics, ablations, comparisons, and contextual clustering. TRACA achieves state-of-the-art performance among real-time trackers on two datasets while running above 100 fps.
- Implementation: 101.3 FPS was measured on the CVPR2013 dataset using an NVIDIA GTX1080 GPU.The implementation used MATLAB and MatConvNet on an Intel i7-2700K CPU, 16GB RAM, and GTX1080 GPU.
- Datasets: CVPR2013 and TPAMI2015 provide 51 targets across 50 videos and 100 targets across 98 videos, respectively.Both datasets include ground-truth bounding boxes for every frame and varied environments.
- Evaluation metrics: Average precision at a 20-pixel location threshold and success-plot area under the curve were used as representative tracking scores.The average precision curve was computed by averaging sequence precision curves under location-error and overlap thresholds.
- Self-comparison: 14.3% performance reduction occurred without extrinsic denoising, while simple-TRACA, single-TRACA, and label-TRACA reductions were 13.4%, 13.6%, and 13.2%.The comparisons indicate contributions from denoising, multi-stage distance, multiple contexts, and feature-space clustering.
- Self-comparison: 6.6% performance reduction followed removal of initial expert adaptation, and omitting orthogonality loss caused a further 1% decrease relative to pretrain-TRACA.Random and l2 expert selection also reduced performance, reflecting the importance of the context-aware network.
- Comparison with state-of-the-art trackers: TRACA achieved state-of-the-art performance among real-time trackers on CVPR2013 and TPAMI2015 while running at over 100 fps.Figure 4 reports average precision at a 20-pixel centre-error threshold and success-plot area under the curve.
- Further analysis: Contextual clusters categorized Caltech256 samples according to appearance patterns, using five highest-scoring samples per category.The context denotes a coarse category of compressed feature maps encoding target-object appearance.
5. Conclusion
The paper concludes that context-aware deep feature compression with multiple auto-encoders enables high-speed tracking while maintaining competitive state-of-the-art performance. It also identifies broader applications and joint expert training as future directions.
- Conclusion: TRACA combines context-specialized expert auto-encoders with a context-aware network that selects the best expert for each tracking target.The framework applies context-aware compression to correlation-filter tracking with deep convolutional features.
- Conclusion: Over 100 fps was achieved while maintaining competitive performance compared with state-of-the-art trackers.The conclusion presents high-speed tracking and competitive performance as the main experimental finding.
- Future applications: The compression scheme is expected to integrate with other raw-deep-feature trackers and may address overfitting in few-sample tasks such as k-shot learning and domain adaptation.These are proposed applications rather than experimentally established results in the supplied conclusion passage.
- Future work: Joint training of the expert auto-encoders is identified as future work.