Source-linked AI summary
Edge-Aware Thermal Infrared UAV Swarm Tracking
Yu-Hsi Chen
TL;DR
Real-time TIR UAV swarm tracking must balance robustness with computational efficiency on resource-constrained edge devices. The paper proposes an AKKF-centered online pipeline with false-positive suppression and predictive coasting; on the BSB benchmark, the full AKKF achieves the best overall tracking performance while reducing FPS from 132.03 to 94.92.
Problem
Real-time multi-object tracking for TIR UAV swarms remains challenging because resource-constrained edge devices require a balance between tracking accuracy and computational efficiency.
Method
The paper proposes an edge-aware online pipeline centered on an Adaptive Kinematic Kalman Filter with state-dependent motion modeling, false-positive suppression, and predictive coasting.
Results
The full AKKF achieves the best overall tracking performance, improving all metrics over the baseline while reducing FPS from 132.03 to 94.92.
Takeaways & Limitations
Experiments on the BSB benchmark provide a practical reference for jointly evaluating tracking robustness and computational efficiency in edge-aware TIR UAV tracking.
Takeaways & Limitations
Future work is needed on model compression and validation of the pipeline on dedicated edge devices.
Abstract
from arXiv · showhide
Thermal infrared (TIR) imaging is essential for UAV swarm operations in visually degraded environments. However, tracking tiny UAVs remains challenging due to limited appearance cues, frequent occlusions, and rapid maneuvers. Despite significant progress driven by benchmarks such as the Anti-UAV challenge, existing methods primarily prioritize accuracy while overlooking the computational constraints of real-time edge deployment. The standard Kalman Filter (KF) offers the efficiency required for edge devices, yet its constant-velocity assumption often breaks down under highly dynamic UAV motion and thermal sensor jitter. More sophisticated nonlinear estimators can improve robustness but often introduce additional computational costs. To address this gap, we propose an edge-aware online tracking pipeline centered on the Adaptive Kinematic Kalman Filter (AKKF), which augments the linear KF with state-dependent kinematic modeling while preserving real-time efficiency. Combined with transient false-positive suppression and kinematics-driven predictive coasting, the presented pipeline improves trajectory continuity under challenging TIR conditions. Experiments on the Beyond Strong Baseline (BSB) benchmark provide a starting point for edge-aware UAV tracking by jointly evaluating tracking performance and computational efficiency, offering insights toward future real-time deployment.
1. Introduction
Real-time TIR multi-object tracking for cooperative UAV swarms must balance robust perception under degraded conditions with the computational limits of edge devices. The paper addresses this challenge with an Adaptive Kinematic Kalman Filter and an edge-aware pipeline evaluated on the BSB benchmark.
- Motivation: TIR imaging supports UAV swarm perception at night and in adverse weather, but real-time multi-object tracking remains challenging on resource-constrained edge devices.The introduction frames the problem around balancing tracking capability with edge-computing constraints.
- Related Work: Tracking-by-detection pipelines commonly combine object detectors with association methods such as ByteTrack and BoT-SORT, while the standard KF remains dominant for online tracking.The KF is favored for simplicity and efficiency, but its fixed kinematic assumptions limit robustness under dynamic UAV motion.
- Contributions: The Adaptive Kinematic Kalman Filter augments the linear KF with state-dependent kinematic modeling for robust, real-time UAV swarm tracking.This contribution preserves the KF-centered tracking formulation while adapting its motion model to UAV dynamics.
- Contributions: The edge-aware online tracking pipeline integrates TFPS and kinematics-driven predictive coasting for reliable, continuous TIR UAV tracking.These components target transient false positives and trajectory continuity during challenging tracking conditions.
- Evaluation: Experiments on the BSB benchmark characterize the accuracy-efficiency trade-off among tracking pipelines and provide a practical reference for edge-aware UAV tracking.The evaluation is designed to inform real-time deployment considerations.
2. Related Work
Related work spans diverse UAV perception datasets and tracking strategies, with motion-based methods commonly built around the efficient discrete-time Kalman Filter. Although nonlinear variants offer greater modeling flexibility, their computational cost helps explain continued reliance on linear constant-velocity KF models in multi-target and TIR UAV tracking.
- UAV perception datasets: UAV perception datasets cover ground-to-air and air-to-air detection and segmentation, UAV swarm monitoring, and TIR UAV detection and tracking.These datasets reflect diverse sensing modalities and viewpoints.
- Motion-centric tracking: Data-driven state estimators model complex motion dynamics, while ByteTrack and OCSORT use bounding-box kinematics and observation history for efficient association.StrongSORT and BoT-SORT further incorporate ReID modules as hybrid tracking approaches.
- Kalman filtering: The discrete-time KF provides an efficient framework for state estimation under linear dynamics in motion-based tracking.The KF is formulated for target states and noisy observations evolving over discrete time steps.
- Kalman filtering: KF prediction propagates prior estimates through kinematic evolution with Gaussian process and measurement noises, while the Kalman gain fuses predictions with incoming measurements according to uncertainty.The gain balances motion prediction and detector measurements according to their uncertainties.
- Nonlinear filtering: EKF and UKF provide greater modeling flexibility for complex dynamics but are rarely used in bounding-box MOT because their computational cost increases for multiple targets.Consequently, SOTA and TIR UAV tracking systems continue using standard KF models with linear constant-velocity assumptions and predefined noise models.
3. Methodology
The methodology extends the linear Kalman Filter into an Adaptive Kinematic Kalman Filter with state-dependent uncertainty and motion modeling for efficient TIR UAV tracking. It further adds damping, braking, measurement-noise adaptation, false-positive suppression, and predictive coasting to improve trajectory continuity and reliability.
- Adaptive Kinematic Kalman Filter: AKKF extends the linear KF with state-dependent kinematic modeling while adaptively updating transition and measurement uncertainty for real-time efficiency.The formulation follows BoT-SORT while preserving the computational efficiency of a linear Kalman Filter.
- Motion Modeling: Area-Adaptive Motion Damping dynamically decays positional velocity during occlusions based on bounding-box area, reducing trajectory drift while maintaining continuity.Velocity retention terms are replaced by (1 −Dxy), with Abox = max(1, w × h), α controlling maximum damping strength, and β controlling saturation behavior.
- Motion Modeling: Perspective-Aware Velocity Braking suppresses negative scale velocities as target size changes, reducing excessive scale variation and preventing scale collapse during missed detections.The braking strength is controlled by γ, while ϵ prevents numerical instability.
- Measurement Modeling: Dynamic Measurement Noise Estimation enlarges Rk when measured and predicted aspect ratios are geometrically inconsistent, reducing unreliable detections’ Kalman-gain contribution.The anomaly sensitivity is controlled by κ, allowing prediction to dominate in TIR clutter.
- Online Tracking Enhancements: The online pipeline adds TFPS, which outputs only temporally confirmed tracklets, and predictive coasting, which skips measurement updates and adopts predictions during missed associations.Temporary misses may arise from thermal sensor jitter, motion blur, or mutual occlusions; both standard KF and AKKF perform best with a single coasting frame under RF-DETR Nano at 640 resolution.
4. Experiment Results
Experiments evaluate advanced detectors paired with BoT-SORT and AKKF, emphasizing HOTA alongside training time, inference speed, memory, detection accuracy, and tracking performance. RF-DETR Nano at 1280 resolution achieves the highest HOTA, while ablations show that full AKKF improves tracking metrics with reduced FPS.
- Overall results: RF-DETR Nano with 1280 resolution achieves the highest HOTA score of 0.8437.RF-DETR Nano at 640 resolution and YOLO26n at 1280 and 640 resolutions are selected as edge-aware candidates for further evaluation.
- Overall results: YOLO26n at 640 resolution achieves a superior accuracy-throughput trade-off over YOLOv12n at the same resolution.The visualization comparison shows gradually decreasing tracking quality across the three edge-aware candidates.
- AKKF ablation: The full AKKF achieves the best overall tracking performance and improves all metrics over the baseline.The complete AKKF reduces FPS from 132.03 to 94.92, illustrating a performance-efficiency trade-off.
- AKKF ablation: Higher detection accuracy does not necessarily translate into better tracking performance, revealing a detection-tracking paradox.For tiny-object tracking, increasing input resolution consistently improves performance.
5. Conclusion
The paper presents an edge-aware TIR UAV swarm tracking pipeline that jointly targets robustness and computational efficiency. It combines AKKF, transient false-positive suppression, and predictive coasting, while identifying compression, temporal information, and edge-device validation as future directions.
- Conclusion: The proposed pipeline jointly considers tracking robustness and computational efficiency for TIR UAV swarms.It is designed for robust tracking under challenging thermal infrared conditions.
- Conclusion: AKKF augments the standard KF with state-dependent motion damping, velocity constraints, and measurement uncertainty estimation.Combined with TFPS and predictive coasting, it supports tracking during temporary observation gaps.
- Conclusion: Future work will explore model compression, temporal information utilization, and validation on dedicated edge devices.These directions extend the pipeline toward practical edge deployment.