PaperlayerView exact revision on arXiv

Source-linked AI summary

KinoUAV: A Benchmark for Safe and Stable UAV Vision-Language Navigation with Kinodynamic Modeling

Chenyao Yang, Peijin Wang, Yingchao Feng, Wenhui Diao

arXiv:2610.30609v1cs.CV

TL;DR

Existing UAV navigation benchmarks and methods do not explicitly incorporate kinodynamic modeling or safety-distance supervision into trajectory generation, leaving gaps in safe, stable navigation support. KinoUAV combines constrained acceleration-driven trajectory generation, DFD supervision, and a Cruise-Seek task. KinoUAV-Agent achieves the highest reported success rate while obtaining lower collision and yaw rates.

  • Problem

    Existing UAV VLN datasets often lack kinodynamic trajectory modeling and effective obstacle-avoidance supervision, while current task guidance can hinder autonomous exploration.

    Source: §I · pp. 1–2

  • Method

    KinoUAV combines hierarchical kinodynamic trajectory generation, DFD supervision for safe action decisions, and a Cruise-Seek task that shifts from region guidance to target search.

    Source: §I · p. 2

  • Results

    KinoUAV-Agent achieves the highest success rate of 89.3% while obtaining lower collision rate and yaw rate.

    Source: §I · p. 2

  • Takeaways & Limitations

    Tests across seen environments, unseen target objects, and a held-out map show higher navigation success and fewer collisions than the evaluated baselines.

    Source: §V · p. 15

    The benchmark uses simulated scenes and a simplified dynamics model; future work will address realistic disturbances, moving obstacles, and transfer to real UAVs.

    Source: §V · p. 15

Abstract

from arXiv · show

Unmanned aerial vehicles (UAVs) have recently emerged as important remote sensing platforms, and UAV navigation has shown great potential in remote sensing applications. However, existing methods do not explicitly incorporate kinodynamic modeling or safety distances into trajectory generation, resulting in high absolute yaw rates, frequent collisions, and low navigation success rates, which limit safety and motion stability. To address this challenge, we introduce KinoUAV, a kinodynamic benchmark comprising a dataset and KinoUAV-Agent. First, we develop a hierarchical kinodynamic trajectory generation pipeline that drives UAV through acceleration actions. It uses waypoint chains from global planning and generates trajectories segment by segment under velocity, acceleration, and jerk constraints, reducing oscillations and improving trajectory stability. Furthermore, we introduce directional free-space distance (DFD) labels into the constructed dataset to describe the available clearance for safe motion decisions. The proposed KinoUAV-Agent predicts DFDs and uses them to generate safe actions, decoupling perception from action decision-making. This design enables autonomous obstacle avoidance and improves navigation safety. The benchmark incorporates the proposed Cruise-Seek task, in which the UAV first enters the target region and then searches for the target, reducing the difficulty of target search and improving navigation success. Experiments on KinoUAV show that KinoUAV-Agent outperforms existing methods, achieving the highest success rate while maintaining lower absolute yaw rates and collision rates. These results demonstrate the effectiveness of kinodynamic modeling for safe and stable UAV navigation.

I. INTRODUCTION

KinoUAV addresses gaps in UAV vision-language navigation data and evaluation by combining kinodynamic trajectory generation, DFD supervision, and a staged Cruise-Seek task. KinoUAV-Agent achieves the highest reported success rate while also obtaining lower collision and yaw rates.

I. INTRODUCTION

  • Existing UAV VLN datasets often omit kinodynamic modeling and effective obstacle-avoidance supervision, while their instruction formats can hinder autonomous exploration.Path instructions may weaken exploration, whereas target descriptions alone can make search inefficient and leave navigation-stage guidance misaligned.Source: §I · pp. 1–2
  • KinoUAV generates trajectories with acceleration actions under velocity, acceleration, and jerk constraints, and uses forward and downward DFDs as intermediate supervision for safe decisions.The DFD supervision decouples environmental risk perception from action prediction.Source: §I · p. 2
  • Cruise-Seek provides target-region guidance during Cruise, then target descriptions for localization during Seek after the UAV enters the region.This task design balances navigation guidance and autonomous exploration.Source: §I · p. 2
  • 89.3% success rate: KinoUAV-Agent achieves the highest rate while obtaining lower collision and yaw rates.KinoUAV also adds absolute yaw rate as a motion-stability metric beyond existing VLN metrics.Source: §I · p. 2

A. Kinodynamic Motion Planning

  • Kinodynamic planning research builds executable motion into search, extending geometric planning with motion constraints and feasible motion primitives [18]–.The cited work includes aerial kinodynamic search and approaches combining motion primitives with trajectory optimization.Source: A. Kinodynamic Motion Planning · p. 2

B. VLN Datasets

  • VLN benchmarks span ground and aerial settings, but existing aerial datasets vary in scene realism, trajectory construction, and language guidance –[13],.KinoUAV adds kinodynamic modeling and motion-state records to support safe and stable aerial navigation.Source: B. VLN Datasets · pp. 2–3

C. VLN Methods

  • VLN methods increasingly combine semantic reasoning with executable motion, while KinoUAV connects these aspects through kinodynamically generated trajectories and acceleration-driven control, [15]–, –[52].Related approaches include hierarchical decoding, motion-state supervision, and model-predictive control.Source: C. VLN Methods · p. 3

A. Trajectory Generation Pipeline

The trajectory-generation pipeline builds waypoint chains, then uses hierarchical kinodynamic A* to plan constrained reference trajectories and AirSim flight control to collect observations and motion states. The Main and Auxiliary Subsets use different waypoint-chain generation strategies but share the subsequent kinodynamic planner.

A. Trajectory Generation Pipeline

  • The pipeline first generates an unconstrained waypoint chain, applies hierarchical kinodynamic A* to produce discrete reference points, then uses AirSim control to generate a continuous trajectory.Trajectory cleanup occurs before the reference trajectory is tracked for data collection.Source: A. Trajectory Generation Pipeline · p. 3, Figure 2 · p. 4

1) Waypoint Chain Generation:

  • For the Main Subset, geometric A* plans from a specified start to goal, sampling candidate displacements at a fixed 3 m step to form an ordered waypoint chain.The chain's final waypoint corresponds to the goal.Source: 1) Waypoint Chain Generation · pp. 3–4
  • For the Auxiliary Subset, waypoints are randomly sampled relative to the previous waypoint until a ten-waypoint chain is formed; occupied-voxel samples are rejected.Both subsets pass their waypoint chains to the same hierarchical kinodynamic planner.Source: 1) Waypoint Chain Generation · p. 4
  • The kinodynamic planner searches position, velocity, previous acceleration, and heading states using acceleration controls, while accounting for velocity-dependent drag.The simplified model propagates reference states at fixed intervals; the selected control acceleration and drag acceleration determine the modeled motion.Source: 2) Hierarchical Kinodynamic A∗planning · pp. 4–5
  • Candidate transitions are rejected when they violate speed or jerk limits or collision checks, and cost-threshold pruning guides the search toward each waypoint.Each completed segment's terminal position, velocity, and acceleration initialize the next segment; cleanup removes oscillatory steering and loops.Source: 2) Hierarchical Kinodynamic A∗planning · p. 5

B. Dataset Construction

KinoUAV uses an automated kinodynamic data-collection pipeline that records UAV motion, images, and directional free-space distances at each valid control step. Its annotations support both waypoint planning and low-level motion control.

B. Dataset Construction

  • Hierarchical kinodynamic A* is integrated into automated data collection using UE4 assets from OpenUAV.The resulting reference trajectory is executed progressively through the flight-control API.Source: B. Dataset Construction · pp. 5–6
  • At each valid control step, the dataset records UAV pose, velocity, reference control acceleration, and front- and bottom-view images.It also computes front- and bottom-view DFDs from the occupancy map; each DFD measures distance to the first occupied voxel along camera-observed rays, capped at 100 m.Source: B. Dataset Construction · p. 6
  • High-level samples pair observations and instructions with the next waypoint, while low-level samples pair state and observations with acceleration and free-space annotations.The waypoint-planning sample is formed at each high-level planning event.Source: B. Dataset Construction · p. 6

1) Auxiliary Subset:

The Auxiliary Subset supplies low-level waypoint-following samples generated from randomly created waypoint chains, complementing Main Subset navigation trajectories.

1) Auxiliary Subset:

  • Random waypoint chains and hierarchical kinodynamic A* generate trajectories for low-level waypoint-following behavior.Each valid control step is paired with its active waypoint, which changes as the UAV advances through the chain.Source: 1) Auxiliary Subset · p. 6
  • Waypoint labels supervise successive local navigation objectives rather than the endpoint of the entire waypoint chain.This label choice is used for every valid control step in the subset.Source: 1) Auxiliary Subset · p. 6
  • The subset contains only low-level samples and requires no language-instruction annotations.It complements Main Subset trajectories with varied waypoint transitions for training the low-level controller.Source: 1) Auxiliary Subset · p. 6

2) Main Subset:

Main Subset provides complete target-oriented navigation data for Cruise and Seek, with trajectory, language, visual, motion, and free-space annotations. Its dataset statistics and action patterns characterize the constructed data.

2) Main Subset:

  • Main Subset trajectories support high-level waypoint planning and low-level acceleration control across Cruise and Seek stages.Cruise uses spatial guidance toward the target region; Seek uses the target description to visually locate the object.Source: 2) Main Subset · p. 6
  • Cruise instructions encode the target’s relative direction and distance, using a trajectory displacement scaled to 10 m when it extends beyond that range.If the trajectory endpoint is less than 10 m away, its body-frame displacement is used without scaling.Source: 2) Main Subset · p. 6
  • Target descriptions are generated from four views and incorporate one of nine application scenarios, such as medical rescue or logistics transport.The views include top-down images at 5 m and 8 m and frontal images at 2 m height along circular paths of 5 m and 8 m radius.Source: 2) Main Subset · p. 6
  • Seek samples use randomly selected points above the target, filtered so top-down DFD is at least the point’s distance to the target.Waypoint labels are target coordinates in the vehicle’s local frame, scaled to 10 m when their magnitude exceeds that distance.Source: 2) Main Subset · p. 6
  • Urban scenarios account for 62.9% of samples; Fig. 3 also shows the description vocabulary and Main and Auxiliary Subset trajectory-length distributions.Its panels show scenario categories, a word cloud, and trajectory lengths for the two subsets.Source: 2) Main Subset · p. 6, Figure 3 · p. 7
  • KinoUAV automates trajectory collection and instruction annotation, and adds kinodynamic modeling beyond OpenFly to account for UAV motion constraints.The dataset comparison covers trajectory and instruction lengths, action counts, and vocabulary size; KinoUAV statistics exclude Auxiliary Subset.Source: 2) Main Subset · p. 7, Table I · p. 7
  • The forward action becomes less likely as forward DFD decreases, and downward actions show the same pattern with bottom DFD.Actions are grouped by direction regardless of magnitude; the infrequent stay action is excluded.Source: 2) Main Subset · p. 7

C. KinoUAV-Agent

KinoUAV-Agent combines a high-level waypoint planner with a low-level acceleration controller in a hierarchical navigation framework. The UAV progresses through takeoff, Cruise, and Seek while replanning from observations and obstacle responses.

C. KinoUAV-Agent

  • The high-level planner generates a waypoint at planning events, while the low-level controller selects an acceleration action at every control step.The two modules therefore operate at different frequencies.Source: C. KinoUAV-Agent · p. 7
  • After takeoff, Cruise guides the UAV toward the target vicinity and Seek uses descriptions and visual observations to locate and approach it.Obstacle responses use front and bottom views; waypoint arrival or a stage transition can trigger replanning before the current waypoint is completed.Source: C. KinoUAV-Agent · pp. 7–8
  • KinoUAV-Agent hierarchically combines a high-level waypoint planner, DFD estimation, and acceleration-based motion control.The high-level planner generates waypoints, while the low-level controller uses estimated DFDs for obstacle avoidance and predicts acceleration actions for waypoint tracking.Source: Figure 5 · p. 8

1) High-Level Waypoint Planner:

The high-level planner generates successive waypoints using stage-dependent cues: coarse direction and distance during Cruise, then the target description during Seek. A Cruise reference point above the target marks the stage-transition vicinity, but its exact coordinate is not planner input.

1) High-Level Waypoint Planner:

  • The Cruise reference point lies vertically above the ground-level target and defines the stage-transition vicinity, but its exact coordinate is not input to the planner.The target position is g = (xg, yg, 0)⊤ and the Cruise reference is gH = (xg, yg, zH)⊤.Source: 1) High-Level Waypoint Planner · p. 8
  • The same high-level planner outputs each waypoint in the UAV body frame and converts it to world coordinates using the current position and rotation.The textual cue changes with the navigation stage.Source: 1) High-Level Waypoint Planner · p. 8
  • During Cruise, the planner uses coarse direction and distance guidance; during Seek, it uses the target description to generate waypoints toward the visually identified object.If the stage changes while a Cruise waypoint is active, it is discarded and a Seek waypoint is generated from current observations.Source: 1) High-Level Waypoint Planner · pp. 8–9
  • Fig. 6 illustrates six successive moments: the UAV approaches the target region during Cruise (1–4), then visually localizes and approaches the target during Seek (5–6).The figure shows front- and bottom-view observations and representative instructions for both stages.Source: Figure 6 · p. 9

2) Low-Level Motion Controller:

The low-level controller uses front- and bottom-view images, waypoint-relative state, and estimated directional free-space distances to predict acceleration. It executes a discrete action and tracks the active waypoint until arrival, while training supervises both acceleration and DFD estimates.

2) Low-Level Motion Controller:

  • At each control step, the controller takes two camera views and a waypoint-relative state vector and predicts the active control acceleration.This is the control acceleration in the kinodynamic model; total acceleration also includes velocity-dependent drag.Source: 2) Low-Level Motion Controller · p. 9
  • The waypoint-relative state encodes direction, normalized distance, and the UAV’s horizontal and vertical velocities.The distance uses logarithmic normalization.Source: 2) Low-Level Motion Controller · p. 9
  • A shared visual encoder processes front and bottom views, while a DFD decoder estimates directional free-space distances for both views.The controller combines visual features, waypoint-relative state, and estimated DFDs in its acceleration predictor.Source: 2) Low-Level Motion Controller · p. 9
  • The predicted body-frame acceleration is projected to the nearest discrete action, rotated into the inertial frame, and combined with velocity-dependent drag before tracking the next reference position.The waypoint remains active until its distance is within the arrival radius; the planner is then invoked again unless the task has terminated.Source: 2) Low-Level Motion Controller · pp. 9–10
  • Training uses mean squared error to supervise both control acceleration and DFD prediction, with separate weights for the two objectives.The weights are λaction and λdist.Source: 2) Low-Level Motion Controller · p. 10

A. Implementation Details

The implementation specifies the simulation and training setup, evaluation metrics, and comparison protocols for learned navigation and vision-language models. Evaluation spans seen and unseen test splits and reports both navigation outcomes and motion characteristics.

A. Implementation Details

1) Hyperparameter Settings:

  • Dataset construction sets UAV mass to 1 kg, maximum gravity-compensated control force to 1 N, and waypoint arrival radius to 3 m.The stated linear and quadratic drag coefficients are 0.2 s−1 and 0.01 m−1; evaluation uses a 12 m cruise altitude and 5 m stage-transition threshold.Source: 1) Hyperparameter Settings · p. 10

B. Evaluation Metric

  • Evaluation reports success, oracle success, navigation error, path-length-weighted success, collision rate, mean speed, and mean absolute yaw rate.Success requires finishing within 6 m of the target; oracle success counts a trajectory passing within 20 m, and unsuccessful episodes contribute zero to SPL.Source: B. Evaluation Metric · p. 10

C. Comparison Experiments

  • Comparisons retrain CMA, TravelUAV, and AerialVLA on the dataset and assess multiple fine-tuned vision-language models, with Qwen3-VL also tested zero-shot.Performance is evaluated on Test Seen, Test Unseen Object, and Test Unseen Map; Table II covers Test Seen and Table III covers the two unseen splits.Source: C. Comparison Experiments · pp. 10–11, Table II · p. 11, Table III · p. 11

1) Evaluation of KinoUAV Test Seen:

On KinoUAV Test Seen, the fine-tuned KinoUAV-Agent improves success and collision rates over AerialVLA; it also generalizes to unseen targets and maps. Ablations indicate that training data composition, guidance, and distance information affect navigation performance.

1) Evaluation of KinoUAV Test Seen:

  • KinoUAV-Agent improves SR by 37.0 percentage points and reduces CR by 22.3 percentage points versus AerialVLA, while achieving higher mean speed and lower mean absolute yaw rate.With the low-level controller unchanged, fine-tuning Qwen3-VL-8B increases SR from 3.7% to 89.3% and reduces CR from 32.8% to 6.0%.Source: 1) Evaluation of KinoUAV Test Seen · pp. 11–12

2) Evaluation of KinoUAV Test Unseen Object:

  • On Test Unseen Object, KinoUAV-Agent reaches 87.1% SR, 35.6 percentage points above AerialVLA, and 6.2% CR versus TravelUAV’s 17.2%.Its success, collision, and kinematic statistics show no significant changes from Test Seen.Source: 2) Evaluation of KinoUAV Test Unseen… · p. 12

3) Evaluation of KinoUAV Test Unseen Map:

  • On Test Unseen Map, KinoUAV-Agent achieves 64.6% SR, 25.2 percentage points above AerialVLA, while maintaining the lowest collision rate.Its average speed and average absolute yaw rate remain consistent across the three test subsets.Source: 3) Evaluation of KinoUAV Test Unseen Map · p. 12

D. Ablation Studies

  • The ablation study tests training data, DFD supervision, navigation strategy, initial heading, and acceleration quantization under otherwise shared settings.Variants are evaluated on KinoUAV Test Seen; retraining is used when changing training data, supervision, or prediction targets.Source: D. Ablation Studies · p. 12

1) Analysis of the training dataset:

  • Adding Auxiliary Subset trajectories increases SR by 18.1 percentage points; using more training data further improves performance, while the 1% setting rarely completes tasks and has the highest collision rate.Auxiliary Subset trains the low-level controller on random-waypoint tracking; the 1% comparison uses limited data for both modules, unlike the cited zero-shot controller trained on full data.Source: 1) Analysis of the training dataset · pp. 12–13

D. Ablation Studies

  • Compared with Cruise–Seek navigation without distance information during Cruise, retaining that information improves SR and further reduces collisions; random initial heading lowers SR by 7.2 percentage points.Source: 1) Analysis of the training dataset · p. 13

2) Analysis of the Navigation Strategy:

DFD supervision improves navigation success and collision avoidance, while increasing DFD noise worsens some navigation outcomes. The trajectory comparison and conclusion describe the model’s motion behavior, benchmark scope, and current simulation boundary.

2) Analysis of the Navigation Strategy:

  • Random initial headings increase CR by 5.6 percentage points, while removing acceleration quantization yields similar SR but increases collisions, indicating quantization reduces collision risk without a large change in task success.Without quantization, mean speed and mean absolute yaw rate also increase.Source: 2) Analysis of the Navigation Strategy · p. 13

3) Analysis of the DFD:

  • Removing the DFD branch increases CR by 4.3 percentage points versus the full model, while the model trained with distance supervision performs best.At σn = 0.1, CR remains close to the result without noise; at σn = 0.5, CR reaches 10.7%.Source: 3) Analysis of the DFD · p. 14

E. Generated Trajectory Analysis

  • In the illustrated urban episode, KinoUAV reaches the target on a smoother street path, while TravelUAV and AerialVLA collide with nearby structures.Maximum absolute yaw rate is 0.393 rad/s for KinoUAV, versus 0.967 rad/s for TravelUAV and 0.857 rad/s for AerialVLA in this example.Source: E. Generated Trajectory Analysis · pp. 14–15, Figure 8 · p. 14

V. CONCLUSION

  • KinoUAV combines kinodynamic trajectory generation and DFD-labeled observations with a waypoint planner and acceleration controller, and reports higher success and fewer collisions across its test settings.The benchmark contains 300K trajectories from 16 simulated scenes; current work uses simulated scenes and a simplified dynamics model.Source: §V · p. 15

Related papers

Loading 2610.30609v1…