Source-linked AI summary
AutoTrack: Towards High-Performance Visual Tracking for UAV with Automatic Spatio-Temporal Regularization
Yiming Li, Changhong Fu, Fangqiang Ding, Ziyuan Huang, Geng Lu
TL;DR
DCF trackers commonly rely on predefined regularization parameters that require tuning and may not adapt to new UAV tracking situations. AutoTrack learns spatio-temporal regularization online from local and global response variations, achieving state-of-the-art tracking performance at ~60 fps on a single CPU and supporting UAV localization.
Problem
Predefined DCF regularization parameters require substantial tuning and may fail to adapt to unforeseen situations, while fixed STRCF penalties are not general across conditions.
Method
AutoTrack automatically and adaptively learns spatio-temporal regularization by using local response variation for spatial regularization and global response variation for temporal updating.
Results
AutoTrack achieves state-of-the-art UAV tracking performance across 278 difficult sequences at ~60 fps on a single CPU and is effective for UAV localization in practical scenarios.
Takeaways & Limitations
The framework provides an efficient adaptive tracker and extends visual tracking to a generally applicable UAV localization system.
Takeaways & Limitations
The localization system requires known non-symmetric four-object configurations and intrinsic UAV-mounted camera parameters.
Abstract
from arXiv · showhide
Most existing trackers based on discriminative correlation filters (DCF) try to introduce predefined regularization term to improve the learning of target objects, e.g., by suppressing background learning or by restricting change rate of correlation filters. However, predefined parameters introduce much effort in tuning them and they still fail to adapt to new situations that the designer did not think of. In this work, a novel approach is proposed to online automatically and adaptively learn spatio-temporal regularization term. Spatially local response map variation is introduced as spatial regularization to make DCF focus on the learning of trust-worthy parts of the object, and global response map variation determines the updating rate of the filter. Extensive experiments on four UAV benchmarks have proven the superiority of our method compared to the state-of-the-art CPU- and GPU-based trackers, with a speed of ~60 frames per second running on a single CPU. Our tracker is additionally proposed to be applied in UAV localization. Considerable tests in the indoor practical scenarios have proven the effectiveness and versatility of our localization method. The code is available at https://github.com/vision4robotics/AutoTrack.
1. Introduction
AutoTrack addresses the difficulty of adapting DCF regularization to changing UAV tracking situations by learning spatio-temporal regularization from local and global response variations. The tracker reports favorable state-of-the-art performance and extends visual tracking to UAV localization.
- DCF methods are selected for UAV tracking because their Fourier-domain computation is efficient under onboard power and computational constraints.
- Existing improvements use regularization to suppress boundary effects or constrain learning, but predefined parameters require tuning and may not adapt to unforeseen situations.
- Local response variation indicates credibility at different object locations, enabling AutoTrack to avoid learning unreliable appearances caused by illumination changes or partial occlusion.
- ~60 frames per second on a single CPU and evaluations on 278 difficult UAV sequences support favorable performance against current CPU- and GPU-based trackers.
- AutoTrack jointly exploits local and global response-map information through automatic spatio-temporal regularization.
- Visual tracking is additionally applied to UAV localization, with practical-scenario tests reported to validate its effectiveness and generality.
2. Related Works
Related work emphasizes the efficiency advantages of DCF trackers for UAVs while identifying boundary effects, filter degradation, and limited generality in existing tracking and localization methods.
- Traditional DCF trackers offer exceptional efficiency, while deep trackers may fail to meet mobile-robot real-time requirements even with high-end GPUs.
- Constant spatial penalization cannot adapt to varied object changes, motivating more generic spatial regularization for aerial tracking.
- Non-adaptive temporal regularization is prone to tracking drift once the filter is corrupted.
- Deep-learning tracking research includes deep features, reinforcement learning, residual learning, adversarial learning, and siamese networks.
- Prior UAV localization systems face low-light degradation or restricted infrared-spectrum applicability, whereas this work builds localization on visual tracking.
3. Revisit STRCF
STRCF learns filters with spatial and temporal regularization, but its fixed spatial penalty and unchanged temporal penalty strength limit adaptation to varied aerial appearances.
- STRCF learns the optimal filter by minimizing an objective involving extracted features, a Gaussian-shaped desired response, and previous-frame filters.
- Its bowl-shaped spatial regularization parameter is borrowed from SRDCF to reduce boundary effects.
- Temporal regularization restricts filter variation by penalizing the difference between current and previous filters.
- STRCF uses fixed spatial regularization that fails to address unforeseeable appearance variation in aerial tracking.
- Its unchanged temporal penalty strength θ, set as 15, is not general across all situations.
4. Automatic Spatio-Temporal Regularization
AutoTrack uses local and global response-map variations to learn spatial and temporal regularization automatically and adaptively. Its optimization jointly updates the filter and temporal regularization parameter for subsequent-frame detection.
- 4. Automatic Spatio-Temporal Regularization: AutoTrack exploits local and global response variations to achieve automatic, adaptive spatio-temporal regularization.The method introduces local variation for spatial regularization and global variation for temporal regularization.
- 4.1. Response Variation: Local response variation measures pixel credibility after aligning response-map peaks to remove motion influence.The resulting variation vector is used to prepare spatial regularization.
- 4.1. Response Variation: Low-credibility locations are restricted from learning new appearance through a spatial penalty based on local variation.The spatial parameter also retains a boundary-effect mitigation term inherited from STRCF.
- 4.1. Response Variation: Global response variation adaptively determines temporal regularization: aberrant maps stop filter learning, while substantial benign variation permits faster adaptation.A threshold φ distinguishes aberrant response maps, and the reference temporal parameter decreases as variation grows below that threshold.
- 4.2. Objective Optimization: The objective combines response reconstruction, spatially regularized filters, temporal filter-change penalties, and auxiliary optimization variables.AutoTrack solves the resulting problem in the frequency domain using ADMM and iteratively updates four subproblems.
- 4.3. Object Localization: After iterative optimization, the learned filter is used to detect the object in the next frame by selecting the maximum response-map value.The optimized temporal parameter and filter are obtained for frame t before detection in frame t + 1.
5. Localization by Tracking
The paper applies AutoTrack to UAV self-localization by tracking four objects simultaneously in RGB images. Their tracked image locations are matched with known world configurations to optimize the UAV’s 6D pose.
- 5. Localization by Tracking: AutoTrack is introduced into UAV localization to track four objects simultaneously instead of segmenting infrared LEDs.The system is intended for dynamic and uncertain environments.
- 5. Localization by Tracking: The localization system requires four non-symmetric object positions in world coordinates and intrinsic parameters of the UAV-mounted camera.World-coordinate observations are obtained from a motion-capture system.
- 5. Localization by Tracking: After manual assignment, AutoTrack independently tracks the objects and outputs their RGB-image locations for use in versatile environments.This differs from the referenced system, which is applicable only in the infrared spectrum.
- 5. Localization by Tracking: The system establishes world-image correspondences and fine-tunes reprojection error to optimize the final 6D pose.The pose optimization follows correspondence search between tracked configurations and image-frame results.
6. Experiments
Experiments evaluate AutoTrack on four UAV tracking benchmarks and an indoor UAV localization task. AutoTrack achieves strong CPU-based tracking performance while its automatic regularization supports robust tracking and centimeter-level localization.
- Tracking performance: AutoTrack is evaluated against state-of-the-art trackers on four UAV benchmarks containing 119,830 frames.The benchmarks are DTB70, UAVDT, UAV123@10fps, and VisDrone2018-test-dev.
- Overall comparison: AutoTrack ranks first in precision and success rate against CPU-based real-time trackers on DTB70, UAVDT, and UAV123@10fps.On VisDrone2018-test-dev, it achieves comparable precision and success rate to MCCT-H and ECO-HC.
- Overall comparison: 59.2fps and 4.8% higher precision than ECO-HC summarize AutoTrack’s average CPU-based performance across the four benchmarks.It is second fastest behind ECO-HC at 69.5fps, while exceeding STRCF by 7.9% in precision and 108.5% in speed.
- Robustness: 5.8% higher precision than ECO-HC on UAV123@10fps demonstrates robustness against the dataset’s larger inter-frame object motion.AutoTrack is also reported as 3.1 times faster than ARCF-HC while using local-global response information for automatic regularization.
- Attribute-based evaluation: Automatic regularization improves success rates over STRCF by 15.9%, 15.5%, and 4.6% for deformation, in-plane rotation, and viewpoint change.For illumination variation and large occlusion, the reported improvements are 7.0% and 15.7%, respectively.
- Ablation study: Adding automatic spatial and temporal regularization modules progressively improves STRCF, while temporal regularization can increase speed by avoiding training on contaminated samples.The response maps are reported as more reliable than those of the baseline.
7. Conclusion
AutoTrack proposes automatic spatio-temporal regularization for UAV tracking and applies visual tracking to UAV localization, with tests supporting the method’s effectiveness and generality.
- AutoTrack uses local response variation to restrict local correlation-filter learning and global variation to control whole-object filter updates.
- AutoTrack is reported as the best CPU-based tracker at approximately 60 fps and outperforms some state-of-the-art deep trackers on two UAV datasets.
- The framework is applied to UAV localization to connect visual-tracking theory with real-world practice.
- Considerable tests support the effectiveness and generality of the localization method.