PaperlayerView exact revision on arXiv

Source-linked AI summary

DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments

Wen Jiang, Hanfang Liang, Li Wang, Kangyao Huang, Wang Xu, Wei Fan, Jinyuan Liu, Shaoyu Liu, Hongwei Duan, Bin Xu, Xiangyang Ji, Huaping Liu

arXiv:2606.31654v2cs.ROcs.CV

TL;DR

UAV-VLN methods commonly predict discrete actions, local targets, or sparse waypoints, while the transition to executable continuous flight remains weakly modeled. DynFly adds a B-spline-based trajectory-generation layer with dynamic-aware supervision while preserving visual-language reasoning. Experiments on UAV-VLN benchmarks report improvements in both navigation performance and trajectory quality.

  • Problem

    UAV-VLN methods commonly output discrete actions, local targets, or sparse waypoints, leaving the transition from navigation intent to continuous UAV motion weakly modeled.

    Source: §I · p. 1

  • Method

    DynFly adds a trajectory-generation layer that uses B-spline control points, a Spline-DiT generator trained through flow matching, and UAV-oriented dynamic-aware supervision.

    Source: §I · p. 2

  • Results

    DynFly improves both navigation performance and trajectory quality on UAV-VLN benchmarks.

    Source: §I · p. 2

  • Takeaways & Limitations

    The reported results support continuous trajectory generation as a bridge between high-level navigation reasoning and executable UAV motion.

    Source: §V · pp. 13–14

    Evaluation is limited to simulation; real-UAV performance remains unverified, and low-level flight-control and long-horizon navigation connections have not been studied.

    Source: §V · p. 14

Abstract

from arXiv · show

Recent advances in multimodal large models have significantly improved UAV vision-language navigation (UAV-VLN) by enhancing high-level perception and reasoning. However, existing methods mainly focus on predicting discrete actions, local targets, or sparse waypoints, while the continuous transition from navigation intent to executable UAV motion remains weakly modeled. This motion-interface gap limits the continuity, stability, and executability of generated UAV trajectories. To address this gap, we propose DynFly, a dynamic-aware continuous trajectory generation framework that bridges high-level navigation reasoning and executable UAV motion. DynFly bridges high-level navigation intent and continuous UAV motion through a lightweight trajectory generation layer. Specifically, it represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation via flow matching. Furthermore, we introduce UAV-oriented dynamic-aware supervision over position, finite-difference velocity, finite-difference acceleration, heading consistency, and local target alignment, enabling the generated trajectories to better satisfy UAV motion characteristics. And our trajectory generation framework can also be integrated with an existing UAV-VLN framework while preserving its original visual-language reasoning pipeline. Extensive experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality. On the Test Unseen Full split, DynFly improves the strongest baseline by 4.69 NDTW, 2.40 SDTW, 2.14 SR points and 4.87 OSR points, while reducing NE by 4.51 m.

I. INTRODUCTION

UAV-VLN systems often predict targets or sparse waypoints but leave the transition to continuous, executable flight weakly modeled. DynFly adds a trajectory-generation layer while preserving visual-language reasoning, and the related work frames this as a gap between navigation reasoning and continuous motion generation.

I. INTRODUCTION

  • Existing UAV-VLN methods commonly output discrete actions, local targets, or sparse waypoints, leaving transitions between targets weakly modeled.UAVs require continuous displacement, velocity, acceleration, and heading changes rather than instantaneous jumps between sparse targets.Source: §I · p. 1
  • DynFly adds an independent motion layer that converts high-level navigation intent into executable continuous UAV trajectories while preserving visual-language reasoning.The framework represents expert trajectories in B-spline control-point space and trains a lightweight Spline-DiT generator using flow matching.Source: §I · p. 2
  • Dynamic-aware supervision incorporates position, finite-difference velocity and acceleration, heading consistency, and target alignment.The framework is described as producing smoother, more stable, and physically executable trajectories.Source: §I · p. 2
  • DynFly improves navigation performance and trajectory quality on UAV-VLN benchmarks.The introduction also presents the framework as compatible with existing visual-language reasoning pipelines.Source: §I · p. 2

A. Vision-Language Navigation for UAVs

  • UAV vision-language navigation systems improve UAV navigation through stronger perception, spatial reasoning, historical memory, or multimodal foundation models, yet most represent outputs as discrete actions, local targets, or sparse waypoint sequences.The conversion from navigation intent to executable continuous UAV motion remains weakly modeled.Source: A. Vision-Language Navigation for UAVs · p. 3

B. Continuous UAV Trajectory Modeling

  • UAV trajectory planning and generative action models address continuous motion, but the transition from visual-language navigation intent to executable UAV trajectories remains underexplored.DynFly addresses this gap with dynamic-aware continuous trajectory generation.Source: B. Continuous UAV Trajectory Modeling · p. 3

III. METHOD

DynFly keeps visual-language reasoning for local intent and adds a motion layer that generates continuous UAV trajectories from that intent. Its trajectories are generated in B-spline control-point space and supervised for motion consistency and target alignment.

A. Problem Formulation

  • Given visual-language context and a local 3D target, DynFly predicts a short-horizon future 3D trajectory rather than waypoint-centric motion.Source: A. Problem Formulation · p. 3

B. System Overview

  • DynFly separates visual-language reasoning from continuous motion generation, connecting them through a compact motion interface that turns navigation intent into executable UAV trajectories.Source: B. System Overview · pp. 3–4
  • The visual-language front end supplies semantic and spatial reasoning, while an independent motion layer transforms its intent into spline-structured continuous trajectories.The implementation uses Qwen2.5-3B as the front end.Source: B. System Overview · pp. 3–4, Figure 2 · p. 4
  • During training, expert waypoints become B-spline control-point labels for conditional flow matching; at inference, generated control points are decoded into continuous trajectories.Source: B. System Overview · p. 4
  • The decoded trajectory is supervised for position matching, displacement consistency, second-order motion variation, direction consistency, and local target alignment.Source: B. System Overview · p. 4

C. Pseudo Control-Point Label Generation

DynFly fits expert waypoints into B-spline control-point space to create compact targets for trajectory generation. The representation captures continuous curve structure, while pseudo-inverse fitting yields a deterministic minimum-norm label when the fitting problem is underdetermined.

C. Pseudo Control-Point Label Generation

D. DiT-Based Control Point Flow Generation

DynFly generates continuous UAV trajectories by predicting B-spline control points conditioned on visual-language context, a local target, and flow time. Flow matching trains the control-point vector field, and inference samples control points before decoding them into a trajectory.

D. DiT-Based Control Point Flow Generation

  • DynFly formulates trajectory generation as conditional control-point generation from the front-end context and local target.The generator reuses compact visual-language context as a global semantic condition rather than adding a task-specific visual decoder head.Source: D. DiT-Based Control Point Flow… · p. 6
  • Flow matching trains a conditional vector field that transforms Gaussian noise toward pseudo control-point labels.At inference, the model initializes control points from Gaussian noise, applies Euler integration, and decodes the resulting control points with the B-spline basis.Source: D. DiT-Based Control Point Flow… · pp. 6–7
  • The condition encoder fuses visual-language context, the local target, and flow time into a shared condition for the generator.The condition vector is paired with B-spline control-point tokens and learnable positional embeddings before DiT processing.Source: Figure 4 · p. 6, D. DiT-Based Control Point Flow… · p. 6
  • The DiT module processes control-point tokens and predicts a vector field in control-point space.Its tokens represent B-spline control points rather than image patches, and its output is a flow vector field rather than image noise.Source: D. DiT-Based Control Point Flow… · p. 6

E. Loss Design

DynFly combines control-point flow matching with trajectory-space losses applied after B-spline decoding. The auxiliary losses supervise position, finite-difference motion, heading, and alignment toward an intermediate local target.

E. Loss Design

  • The joint objective connects flow-matching supervision in control-point space to trajectory-level losses after B-spline decoding.The losses cover position, velocity, acceleration, heading, and target alignment.Source: E. Loss Design · p. 7, Figure 5 · p. 7
  • Finite-difference velocity and acceleration losses regularize decoded trajectories beyond waypoint-position matching.These terms capture local displacement and curvature patterns; the acceleration penalty also suppresses overly large second-order changes.Source: E. Loss Design · p. 7
  • Heading consistency constrains local flight direction to discourage trajectories that are positionally close but directionally inconsistent.The loss is computed from normalized velocity directions.Source: E. Loss Design · p. 7
  • The local target is anchored at an intermediate trajectory index, encouraging motion toward it without forcing the final point to equal the target.Later points remain available for smoothing, local adjustment, or motion variation.Source: E. Loss Design · p. 7

1) Datasets and Experimental Environment:

The evaluation uses the OpenUAV UAV-VLN benchmark in AirSim, with training and test data divided across seen and unseen map or object splits.

Datasets and Experimental Environment

  • OpenUAV includes 9,152 training trajectories and Test Seen, Test Unseen Map, and Test Unseen Object sets of 1,418, 958, and 629 trajectories, respectively.The test sets are further divided into Easy and Hard subsets, and experiments use the AirSim simulator.Source: 1) Datasets and Experimental Environment · p. 8

2) Evaluation Metrics:

The evaluation reports standard navigation measures alongside trajectory smoothness and turning-stability metrics. Across seen and unseen settings, DynFly improves navigation results over SpatialFly, with the strongest reported comparison on the unseen Full split.

2) Evaluation Metrics:

  • SmoothMean and SmoothVar summarize the mean and variance of the second-order difference magnitude, while TurnMean and TurnVar summarize the mean and variance of consecutive-direction angles.The implementation clips the direction cosine to [-1,1] before applying arccos.Source: 2) Evaluation Metrics · p. 8

3) Implementation Details:

  • DynFly predicts a seven-step 3D trajectory using eight cubic B-spline control points and generates trajectories with 10 Euler integration steps.The Spline-DiT uses six Transformer blocks, hidden size 512, and eight attention heads.Source: 3) Implementation Details · p. 8

1) Quantitative Evaluation on the OpenUAV-Seen Dataset

  • For low-data Test Seen, DynFly reports best NE, SR, OSR, and SPL on Full, Easy, and Hard; on Full it reduces NE from 113.31 m to 63.96 m versus SpatialFly.SR rises from 16.84% to 32.93%, OSR from 38.36% to 64.25%, and SPL from 14.58% to 27.24%.Source: 1) Quantitative Evaluation on the… · pp. 8–9
  • On Test Seen Full with 100% training data, DynFly improves SR from 38.54% to 40.62% and reduces NE from 58.05 m to 53.66 m versus SpatialFly.On the Hard split, NE falls from 84.76 m to 76.85 m, while SR and OSR reach 38.18% and 64.85%.Source: 1) Quantitative Evaluation on the… · p. 9
  • On Test Unseen Hard, DynFly reduces NE from 108.56 m to 99.73 m versus SpatialFly and leads on NDTW, SDTW, SR, OSR, and SPL.On Easy, it leads on those five metrics while NE is slightly higher than SpatialFly's.Source: 2) Quantitative Evaluation on the… · p. 9

3) Performance on Unseen Map and Unseen Object :

DynFly achieves the strongest overall performance across the Unseen Overall, Unseen Object, and Unseen Map protocols, with larger gains on unseen objects. Ablations also show that its B-spline representation, Spline-DiT generator, and dynamic-aware loss contribute to the final performance.

3) Performance on Unseen Map and Unseen Object :

  • Gains are larger on Unseen Object, while all methods face a smaller performance gap on Unseen Map with completely unseen layouts.DynFly nevertheless achieves the best overall performance on the Unseen Map split.Source: 3) Performance on Unseen Map and… · p. 9
  • DynFly achieves the lowest NE and highest SR, OSR, and SPL across Unseen Overall, Unseen Object, and Unseen Map.The comparison is shown for the Full split; the figure reports these three unseen protocols.Source: 3) Performance on Unseen Map and… · p. 9, Figure 6 · p. 10

1) Ablation on Different Components:

  • The B-spline head raises Full-split NDTW from 12.30 to 13.72 and SDTW from 6.38 to 7.60; adding Spline-DiT raises SR to 27.28% and reduces NE from 92.43 m to 89.35 m.The table evaluates the B-spline representation, Spline-DiT generator, and dynamic-aware loss across Full, Easy, and Hard splits.Source: 1) Ablation on Different Components · p. 10, Table III · p. 10
  • Adding dynamic-aware loss yields Full-split NDTW of 15.89, NE of 83.31 m, OSR of 49.28%, and SPL of 23.57%.On Hard, it leads on NDTW, SDTW, NE, OSR, and SPL, while SR ties the B-spline DiT model.Source: 1) Ablation on Different Components · p. 10, Table III · p. 10
  • Together, the components form a motion interface that converts high-level navigation intent into continuous, executable UAV trajectories.The reported interpretation is that B-splines represent continuous trajectories, Spline-DiT learns a suitable trajectory distribution, and dynamic-aware loss constrains motion properties.Source: 1) Ablation on Different Components · p. 10

2) Sensitivity Analysis on Control-Point:

Eight B-spline control points provide the best overall balance in the tested sensitivity analysis. Moderate dynamic-aware loss weights improve both navigation metrics and trajectory quality, and DynFly can be integrated into SpatialFly without changing its visual-language front-end.

2) Sensitivity Analysis on Control-Point:

  • Eight control points yield the best overall performance; increasing the count to twelve lowers most metrics, possibly because the larger search space is harder to learn with the available training data.Two points perform substantially worse, while four improve all metrics; eight is the default configuration.Source: 2) Sensitivity Analysis on Control-Point · pp. 10–11, Figure 7 · p. 11

3) Sensitivity Analysis on Dynamic-Aware Loss:

D. Integration with Existing UAV-VLN Frameworks

  • With SpatialFly’s original visual-language front-end and training configuration unchanged, DynFly improves all navigation metrics; Full-split NE falls from 87.82 m to 82.54 m.The module replacement improves NDTW from 11.20 to 13.76, SDTW from 6.50 to 7.71, and SR, OSR, and SPL by 2.14%, 4.36%, and 1.89%, respectively.Source: D. Integration with Existing UAV-VLN… · pp. 11–12, Table VI · p. 12

E. Generated Trajectory Analysis

Qualitative comparisons show DynFly reaching targets with longer, more stable continuous trajectories than SpatialFly. Multi-view examples show closer alignment to ground truth, and the conclusion reports improved navigation and trajectory quality while identifying simulation-only evaluation as a limitation.

1) Qualitative Navigation Comparison:

2) Multi-view Trajectory Visualization:

  • Across overall and enlarged views, DynFly stays closer to ground truth and follows the target direction more continuously than SpatialFly and SpatialFly(BS).The views show local oscillations in the baselines; near the target, SpatialFly(BS) collides with a building, while DynFly avoids it and stays closer to ground truth.Source: 2) Multi-view Trajectory Visualization · pp. 12–13, Figure 9 · p. 13

V. CONCLUSION

  • The conclusion reports that B-spline representation, Spline-DiT, and dynamic-aware supervision improve trajectory quality and navigation performance.It characterizes explicit continuous-motion modeling as a bridge between high-level reasoning and low-level flight execution.Source: §V · pp. 13–14
  • DynFly is evaluated only in simulation, and its connection to low-level flight control and long-horizon navigation remains unstudied.The authors also identify lightweighting, real-time inference, and sim-to-real transfer as areas needing further improvement.Source: §V · p. 14

Related papers

Loading 2606.31654v2…