PaperlayerView exact revision on arXiv

Source-linked AI summary

A Topology-Aware Spatiotemporal Handover Framework for Continuous Multi-UAV Tracking

Jianlin Ye, Christos Kyrkou, Panayiotis Kolios

arXiv:2605.15779v1cs.ROcs.AI

TL;DR

Trajectory fragmentation across multi-UAV fields of view breaks global vehicle identity and limits network-level traffic analysis. The paper introduces a topology-based spatiotemporal handover system with deterministic queue matching, achieving 99.8% HOSR while supporting real-time edge deployment. It also identifies congestion handling and manual handover-zone calibration as remaining limitations.

  • Problem

    Trajectory fragmentation across sequential UAV fields of view creates disconnected tracklets and prevents network-level analysis such as Origin-Destination estimation.

  • Method

    The system uses topology-based spatiotemporal handover with geometric overlap, virtual lane discretization, FIFO queues, and kinematic state estimation instead of visual Re-ID.

  • Results

    99.8% HOSR was achieved with real-time performance, compared with 74.1% HOSR for FastReID.

  • Takeaways & Limitations

    Roadway directionality and sequential flow support high-precision identity persistence in complex aerial traffic, including identical vehicle platoons.

  • Takeaways & Limitations

    Congested stops can exceed the static TTL threshold, and handover zones currently require manual annotation.

Abstract

from arXiv · show

The integration of Unmanned Aerial Vehicles(UAVs) into Intelligent Transportation Systems (ITS) offers synoptic visibility for traffic monitoring, yet scalable deployment is hindered by trajectory fragmentation, where vehicle identity persistence is lost across multi-UAV Fields of View (FOV). While state-of-the-art frameworks excel in optimizing local trajectory extraction and stability for single-drone imagery, they often function as isolated data silos that generate disjointed trajectories, thereby precluding network-level analysis such as Origin-Destination estimation. This paper presents a real-time Multi-Camera Multi-Vehicle Tracking (MCMT) system designed to handle global identity persistence. Addressing the visual ambiguity and computational cost of appearance-based Re-Identification (Re-ID) in nadir views, we introduce a lightweight Topology-Based Spatiotemporal Handover mechanism. We implement a high-throughput parallel pipeline leveraging YOLO11 and ByteTrack to process concurrent 4K streams. Our core contribution is a deterministic queue-based matching algorithm that utilizes geometric overlaps and virtual lane discretization to predictively manage identity handover via FIFO queues. Experimental results on complex urban environments, including intersections and merging traffic, demonstrate a Handover Success Rate (HOSR) of 99.8% in continuous traffic flows, significantly outperforming Re-ID baselines (74.1%) while validating edge deployment feasibility. The source code is available at https://github.com/JYe9/multi-camera-multi-vehicle-tracking-system.

I. INTRODUCTION

UAVs provide broad traffic visibility, but restricted FOVs and isolated local processing fragment vehicle identities across sequential views. The proposed framework uses topology-aware handover and calibrated kinematic estimation to produce continuous global trajectories for network-level analysis.

  • UAVs offer synoptic traffic visibility and can monitor congestion without permanent infrastructure.
  • Restricted FOVs require multiple UAVs arranged along corridors, creating trajectory fragmentation across sequential views.
  • Fragmented tracklets prevent reliable derivation of macroscopic measures such as average travel time, long-distance lane-changing frequency, and Origin-Destination estimation.
  • Appearance-based Re-ID is vulnerable to nadir-view ambiguity and costly visual-embedding extraction for real-time onboard execution.
  • The framework replaces fragile visual matching with topology-based handover that treats vehicle exits and entries as causally linked events in a connected UAV pipeline.
  • The deterministic queue matcher uses overlapping polygons and virtual lanes to stitch tracklets without visual features, while a kinematic estimator produces calibrated velocity and heading profiles.
  • The system is validated in intersections and merging traffic as a scalable, computationally efficient alternative for continuous wide-area tracking.

A. Deep Learning in Aerial Object Detection

Aerial tracking must handle small objects, occlusion, unstable confidence, and weak visual distinctiveness. The paper situates its approach within detector and tracker advances while formulating MCMT as global identity association in a sensor network.

  • Deep Learning in Aerial Object Detection: Aerial imagery presents extreme scale variation and dynamic occlusion because vehicles occupy minimal frame area and may be obscured by vegetation.
  • Deep Learning in Aerial Object Detection: ByteTrack and OC-SORT reduce track fragmentation by associating detections beyond simple Kalman filtering, including low-confidence bounding boxes.
  • Deep Learning in Aerial Object Detection: Vehicle confidence scores fluctuate with changing viewing angles and environmental shadows in aerial surveillance.
  • Deep Learning in Aerial Object Detection: Appearance-based Re-ID is limited in aerial highway monitoring because nadir views hide distinctive vehicle features and increase interclass visual similarity.
  • Deep Learning in Aerial Object Detection: Topology-based methods reduce association search to physically plausible camera transitions, while probabilistic arrival predictions remain sensitive to nonlinear speed variation.
  • Deep Learning in Aerial Object Detection: The MCMT task is formulated as global identity association governed by network topology, vehicle state, and spatiotemporal handover constraints.

A. Network Topology and Field of View

The sensor network models UAV FOVs as geometric regions connected when they overlap, enabling sequential vehicle transitions. Each local vehicle tracklet is a time-ordered sequence of states with a local identifier.

  • Network Topology and Field of View: Each aerial sensor node has a ground-projected FOV, and directed graph edges represent spatial overlap permitting sequential vehicle transitions.
  • Network Topology and Field of View: Deterministic handover requires a non-empty intersection region between connected UAV FOVs.
  • Network Topology and Field of View: Within the shared overlap domain, a physical target can theoretically be observed simultaneously by both sensors, subject to synchronization latency and perspective distortion.
  • Network Topology and Field of View: A vehicle state contains centroid coordinates, velocity components, and heading angle in the metric domain.
  • Network Topology and Field of View: A local tracklet is a temporal sequence of vehicle states associated with a unique local identifier.
  • Network Topology and Field of View: Standard local processing produces multiple independently identified tracklets for one vehicle traversing different network nodes.

C. The Spatiotemporal Handover Problem

The handover problem assigns an upstream terminating tracklet to a downstream initiating tracklet using spatiotemporal consistency. The proposed solution enforces this assignment through temporal windows and spatial metadata rather than traditional probabilistic matching.

  • The Spatiotemporal Handover Problem: The system seeks a local minimum-cost assignment linking a terminating tracklet in upstream node C_i to an initiating tracklet in downstream node C_i+1.
  • The Spatiotemporal Handover Problem: Exiting tracklets are defined at the upstream overlap boundary, while entering tracklets are those arriving in the corresponding downstream region.
  • The Spatiotemporal Handover Problem: The handover objective selects an association pair that minimizes the spatiotemporal cost J.
  • The Spatiotemporal Handover Problem: Candidate associations are subject to consistency constraints involving position and heading alignment.
  • The Spatiotemporal Handover Problem: A lateral-aware spatiotemporal buffering mechanism enforces the constraints through temporal search windows and spatial metadata matching.

IV. PROPOSED APPROACH

The framework decomposes global MCMT association into local tracking, kinematic estimation, and topology-aware handover. Geometric overlap polygons and directional zones constrain deterministic identity transfer across UAV nodes.

  • The pipeline couples local tracklet generation, kinematic state estimation, and topology-aware global handover.These three stages jointly address global identity association.
  • YOLO11 and ByteTrack generate continuous local tracklets from 4K aerial imagery despite small objects and occlusion.Inference uses 1280-pixel resolution and τconf = 0.25; ByteTrack’s two-stage association preserves low-confidence detections.
  • The handover module converts spatiotemporal constraints into deterministic buffering using calibrated network topology and coordinate metadata.The mechanism resolves complex interactions, including parallel overtaking maneuvers.
  • 1) Geometric Overlap Calibration: Non-convex overlap polygons calibrated to road geometry trigger handovers when a tracklet centroid enters the overlap region.The calibration accounts for UAV yaw and camera perspective projection.
  • 2) Directional Spatial Partitioning: A roadway partitioning function maps vehicles into directional zones based on a road-aligned split, isolating opposing traffic streams.The zones prevent Eastbound exits from matching Westbound entries regardless of visual similarity.

3) Lateral-Aware Spatial Matching:

Lateral-aware matching extends directional buffers with coordinate metadata and temporal search windows. The mechanism supports simultaneous handovers, rejects implausible lateral displacements, and uses velocity context for identity management.

  • Directional buffers support coordinate-metadata matching for simultaneous handovers, including parallel driving and overtaking.The system avoids relying solely on temporal FIFO order within a zone.
  • Push Operation (Exit Event): Exit events push full metadata {gID, texit, yrel} into the corresponding directional buffer.Here, yrel denotes lateral position normalized to road width.
  • Match & Pop Operation (Entry Event): Entry events query the upstream buffer within a temporal window ∆t and select candidates by minimizing lateral spatial displacement.The search accommodates synchronization latency and physical gaps between FOVs.
  • The temporal bound enables soft handover, while lateral gating rejects implausible matches and stale entries are managed with a static TTL.Rejected entries receive new global identities when displacement exceeds ϵlat.
  • Kinematic State Estimation: Metric velocity, calibrated from pixel displacement and smoothed over a sliding window, supplies stopped-or-moving context for global identity management.The conversion uses a GSD-based coefficient λ.

D. Temporal Synchronization and Scalable Architecture

The architecture synchronizes distributed UAV observations before executing topology-aware queue operations while parallelizing local perception. This design addresses temporal causality and scales perception linearly with the number of nodes.

  • Asynchronous frame arrivals can violate handover causality by making downstream appearances precede upstream departures.The global snapshot must represent a temporally aligned network observation.
  • Algorithm 1 initializes directional metadata buffers, processes streams in parallel, estimates kinematics and zones, performs entry and exit logic, and checks timeouts.The algorithm updates global state after each tracklet operation.
  • Local perception runs independently across streams, making detection complexity scale linearly O(N) with UAV-node count.This parallelization is intended to prevent processing bottlenecks as swarm size increases.
  • A synchronization barrier buffers local tracklets until a unified timestamp before running global handover logic.Push and Pop operations therefore use a synchronized traffic snapshot.

V. EXPERIMENTAL RESULTS AND ANALYSIS

The evaluation tests topology-aware tracking against appearance-based Re-ID and assesses computational feasibility for edge deployment. The supplied result highlights YOLO11s as the selected operating point on Jetson Orin NX.

  • The evaluation protocol targets topology-aware superiority over appearance-based Re-ID and computational efficiency for edge deployment.The claims are tested in the paper’s evaluation design.
  • YOLO11s is selected as the optimal Jetson Orin NX operating point in the 15W latency-accuracy analysis.The supplied figure caption identifies the selected model but does not provide its numerical performance.

A. Experimental Setup

The evaluation uses a custom nadir-view UAV dataset covering a synchronized three-camera urban corridor with varied traffic regimes and manually annotated identities. Comparative testing benchmarks topology-aware tracking against motion and appearance-based multi-camera paradigms.

  • Dataset: The custom dataset contains three synchronized 4K streams covering a 500m urban corridor with an intersection and merging traffic.The UAV2–UAV3 segment includes a university campus entrance, producing merge and diverge behaviors.
  • Traffic regimes: The footage is divided into Free-Flow, Congestion, and Overtaking regimes spanning low density, stop-and-go waves, and frequent lane changes.Set A has density below 10 veh/km and speeds above 50 km/h, while Set B exceeds 50 veh/km.
  • Metrics: HOSR measures correct global associations in overlap zones, IDF1 measures global identity stability, and FPS measures end-to-end throughput.The evaluation combines standard MOT metrics with handover-specific indicators.
  • Platforms and baselines: The benchmark includes Kalman filter integration with appearance embeddings, a ResNet50-based Re-ID model, and a pure motion-based tracker without topological constraints.These systems represent appearance-based and motion-only alternatives to the proposed approach.

2) FastReID (Re-ID):

The comparison shows that appearance-based Re-ID is poorly suited to nadir-view aerial tracking, while topology-aware matching maintains high handover accuracy under geometric noise and complex traffic.

  • FastReID comparison: 99.8% HOSR is achieved by the topology-aware approach versus 74.1% HOSR for FastReID.FastReID also reduces server-side throughput to 8.4 FPS, whereas the proposed approach reaches 62.1 FPS.
  • FastReID comparison: The topology-aware approach maintains 62.1 FPS, comparable to the lightweight ByteTrack baseline.The method uses deterministic topological constraints rather than visual embeddings for handover.
  • Robustness analysis: HOSR remains above 98% when UAV drift causes overlap misalignments of up to 15–20 meters.The metadata matching window compensates for geometric errors equivalent to approximately 30% of the overlap length.
  • Robustness analysis: 98.6% of parallel handover events are resolved in Overtaking scenarios using lateral-aware spatial matching.Disabling lateral-aware matching reduces Set C HOSR to 74.5%; congestion can instead trigger identity resets when stops exceed the static TTL threshold.

D. Edge Deployment Feasibility

Edge deployment transfers the pipeline to an NVIDIA Jetson Orin NX and identifies YOLO11s as the practical detector configuration for real-time per-stream processing. The results support decentralized multi-UAV tracking while acknowledging calibration and measurement-noise boundaries.

  • Edge deployment: YOLO11s achieves 25.6 FPS in 15W mode on the Jetson Orin NX, with approximately 0.58 J/frame.YOLO11n reaches 43 FPS but has lower small-object mAP, while YOLO11m exceeds the real-time latency budget.
  • Edge deployment: The reported 25.6 FPS is dedicated throughput for one video stream, enabling each UAV to sustain real-time perception with its own compute unit.This decentralized arrangement avoids transmitting multiple high-resolution feeds to a centralized edge server.
  • Operational validation: The operational dashboard renders three simultaneous 4K feeds and visualizes overlap polygons used for topological handover.The dashboard provides qualitative validation of concurrent multi-stream processing and geometric constraints.
  • Conclusion: The framework reports 99.8% HOSR and real-time global association on resource-constrained edge hardware.The conclusion links high-precision identity persistence to roadway directionality and sequential flow.
  • Limitations and future work: The current pixel-domain implementation assumes a strictly nadir view and remains sensitive to measurement noise beyond the soft handover window.Future iterations target homography calibration and probabilistic multimodal association.
  • Limitations and future work: Future work targets adaptive temporal gating, automated handover-zone calibration, and decentralized UAV-to-UAV communication.These directions address congestion-related resets, manual calibration, and dependence on high-bandwidth central transmission.
Loading 2605.15779v1…