Source-linked AI summary
GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021
Yunhao Du, Junfeng Wan, Yanyun Zhao, Binyu Zhang, Zhihang Tong, Junhao Dong
TL;DR
Drone-video MOT remains challenging because small objects, camera movement, view changes, and other scene factors complicate tracking. GIAOTracker combines online tracking, global linking, and post-processing with global-information and adaptive association strategies. On VisDrone, it achieves state-of-the-art results, including 54.18 mAP after result fusion and a 2nd-place challenge ranking.
Problem
Drone-video MOT is challenging because aerial footage contains small objects, camera movements, view changes, irregular motion, and occlusion.
Method
GIAOTracker uses online tracking, global link, and post-processing, with EMA Bank, NSA Kalman, GIModel, and four refinement methods.
Results
54.18 mAP was achieved on the VisDrone MOT test-challenge dataset after fusing two tracking results, earning 2nd place in the VisDrone2021 MOT Challenge.
Takeaways & Limitations
The three-stage framework achieves state-of-the-art results on the VisDrone MOT dataset and supports refined tracking through global information and post-processing.
Abstract
from arXiv · showhide
In recent years, algorithms for multiple object tracking tasks have benefited from great progresses in deep models and video quality. However, in challenging scenarios like drone videos, they still suffer from problems, such as small objects, camera movements and view changes. In this paper, we propose a new multiple object tracker, which employs Global Information And some Optimizing strategies, named GIAOTracker. It consists of three stages, i.e., online tracking, global link and post-processing. Given detections in every frame, the first stage generates reliable tracklets using information of camera motion, object motion and object appearance. Then they are associated into trajectories by exploiting global clues and refined through four post-processing methods. With the effectiveness of the three stages, GIAOTracker achieves state-of-the-art performance on the VisDrone MOT dataset and wins the 3rd place in the VisDrone2021 MOT Challenge.
1. Introduction
GIAOTracker addresses the challenges of multi-class multi-object tracking in drone videos with a three-stage framework combining global information and optimizing strategies. Its components target association robustness, trajectory linking, and post-processing refinement, achieving strong VisDrone results.
- Drone-video MOT is difficult because aerial footage contains many small objects, irregular motion, camera movement, view variation, and occlusion.
- EMA Bank and NSA Kalman are introduced to support more accurate and robust association by modeling feature states and adapting motion noise to detection quality.
- GIModel extracts global and part spatial features and fuses them with self-attention temporal modeling to improve tracklet appearance representations.
- Four post-processing strategies address redundant trajectories, missing detections, trajectory scoring, and fusion of tracking results.
- 54.18 mAP was achieved on the VisDrone MOT test-challenge dataset after fusing two tracking results, earning 2nd place in the VisDrone2021 MOT Challenge.
- GIAOTracker consists of online tracking, global link, and post-processing stages for multi-class multi-object tracking in drone videos.
2. Related works
Related MOT work is organized around detection–embedding design and whether association uses future information. For challenging drone videos, GIAOTracker follows an SDE-based hierarchical framework that combines online tracking, global linking, and post-processing.
- 2.1. SDE and JDE: SDE separates detection from embedding, whereas JDE integrates both in one framework; SDE is more flexible for complex scenarios but typically slower at inference.
- 2.1. SDE and JDE: GIAOTracker follows SDE because independently training the detector can produce more accurate localization and classification than JDE.
- 2.2. Online and Offline Tracking: Online MOT associates detections without future frames, while offline methods can use future information and generally achieve better tracking quality.
- 2.2. Online and Offline Tracking: GIAOTracker uses three hierarchical stages—online tracking, global link, and post-processing—balancing accuracy with deployment flexibility.
- 2.3. MOT in Drone: Drone-video MOT is difficult because aerial footage contains small objects, camera movements, variant views, occlusions, and irregular motion.
- 2.3. MOT in Drone: Prior drone trackers often underuse camera motion, global information, or post-processing, whereas GIAOTracker combines these elements through ORB/RANSAC, GIModel, and four refinement methods.
3. Method
GIAOTracker uses a three-stage hierarchical framework: online tracking generates tracklets, global linking forms trajectories, and post-processing refines results. Its online stage improves motion and appearance modeling, while global association combines appearance and spatio-temporal costs before TrackNMS fusion.
- Framework: GIAOTracker comprises online tracking, global link, and post-processing stages, balancing online flexibility with the stronger performance of the full pipeline.The online stage generates reliable tracklets; later stages associate them into trajectories and refine the results.
- Online Tracking: ORB and RANSAC align inter-frame images, while OSNet-based appearance features and EMA Bank improve association under camera movement and detection noise.EMA Bank combines multi-frame feature states with inter-frame feature changes.
- Online Tracking: NSA Kalman adaptively scales measurement noise using detection confidence, replacing the constant noise scale used by standard Kalman filtering.The proposed covariance is defined as (1 - c_k)R_k, so lower-confidence detections receive less update weight.
- Online Tracking: Rough2Fine first performs coarse tracking and then uses confidence-weighted soft voting to assign fine classes, improving robustness to classification errors.The method is designed for visually similar categories such as car and van.
- Global Link: GIModel combines global and part-level spatial features with Transformer-based temporal modeling to represent tracklets more robustly.The model extends ResNet50-TP with part features and self-attention temporal modeling.
- Global Link and Post-processing: The global link stage uses Hungarian matching with appearance, time, and space costs subject to thresholds, while TrackNMS fuses different tracking results.TrackNMS is based on SoftNMS and is reported to work well in experiments.
4. Experiment
Experiments evaluate GIAOTracker’s components, post-processing, detection settings, and challenge performance on VisDrone MOT. Results show that the framework benefits from motion compensation, stronger appearance and motion modeling, staged refinement, and improved detections.
- Experimental setup: VisDrone MOT evaluation uses 96 sequences totaling 39,988 frames and ranks trajectories by confidence-weighted mAP under the challenge protocol.
- Global link: GIModel experiments compare ImageNet pretraining, VisDrone training, self-attention temporal modeling, and part-level features for tracklet representation.
- Online Tracking: Online-tracking ablations find that ORB image matching, OSNet features, EMA Bank, NSA Kalman, and UKF improve tracking over the baseline.
- Ablation and post-processing: The complete tracker adds online tracking, global linking, and post-processing, while TrackNMS fusion reaches 44.46 mAP on test-dev.
5. Conclusion
GIAOTracker is a three-stage framework for multi-class multi-object tracking in drone videos. It combines online tracklet generation, global trajectory association, and four post-processing methods, achieving state-of-the-art VisDrone results and 2nd place in the 2021 challenge.
- GIAOTracker targets multi-class multi-object tracking in drone videos with a comprehensive three-stage framework.
- Online tracking generates reliable tracklets using camera movement, object appearance, and object motion information.
- Global linking associates tracklets into trajectories using tracklet features and spatio-temporal distances, followed by four post-processing methods.
- GIAOTracker achieves state-of-the-art results on VisDrone MOT and ranks 2nd in the VisDrone2021 MOT Challenge.