PaperlayerView exact revision on arXiv

Source-linked AI summary

Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

Zhishan Zou, Guoyan Sun, Zhiwei Wei, Jiancheng Pan, Yujie Li, Mugen Peng, Wenjia Xu

arXiv:2607.12477v2cs.CV

TL;DR

Existing UAV benchmarks largely overlook explicit modeling of the UAV as an embodied agent. SIS-Bench evaluates space and self across cognitive levels, revealing weaker self-modeling and progressive degradation, while motion-aware modeling improves both dimensions and transfers to navigation.

  • Problem

    Existing UAV studies remain environment-centered, largely overlooking explicit modeling of the UAV’s internal state and its interaction with the surrounding space.

  • Method

    SIS-Bench evaluates spatial cognition and self-awareness across perception, memory, and reasoning using 4,856 QA pairs from 1,646 UAV videos and 13 tasks.

  • Results

    Current MLLMs model space more strongly than self and degrade from perception to memory to reasoning, while SIS-Motion improves both dimensions and transfers to navigation.

  • Takeaways & Limitations

    SIS-Bench highlights the need to evaluate UAV embodied intelligence through joint modeling of external space, agent self-awareness, and their interaction.

  • Takeaways & Limitations

    The benchmark is limited to standardized source datasets, multiple-choice open-loop UAV video understanding, and a motion-aware probe with modest, inconsistent reasoning gains.

Abstract

from arXiv · show

Autonomous UAV systems increasingly rely on multimodal large language models (MLLMs) to operate in complex real-world environments. Such embodied scenarios require not only understanding the surrounding space but also maintaining a coherent representation of the agent itself. However, existing UAV-oriented approaches and benchmarks remain largely environment-centric, primarily focusing on spatial understanding tasks, with the agent's self-awareness remaining implicit. To address this gap, we introduce SIS-Bench, a benchmark for evaluating embodied spatial intelligence in UAV scenarios under a unified self-in-space formulation. SIS-Bench organizes evaluation along two complementary dimensions, space and self, and a three-level hierarchy of perception, memory, and reasoning. It contains 4,856 question--answer pairs across 13 tasks derived from 1,646 real-world UAV videos through a task-conditioned construction pipeline with expert verification. Extensive evaluations reveal that current MLLMs exhibit fundamental limitations in modeling dynamic and agent-centered processes. In particular, we observe a clear imbalance between spatial cognition and self-awareness, as well as a progressive performance degradation across cognitive levels. Motivated by these findings, we further explore a motion-aware representation that incorporates self-related dynamics through optical flow and visual feature fusion. Experimental results show that modeling agent motion consistently improves perception and memory performance, not only in spatial cognition but also in self-awareness, and generalizes to downstream UAV decision-making tasks. Our results highlight the importance of self-awareness for advancing embodied spatial intelligence, and provide both a new benchmark and empirical evidence for motion-aware self-in-space modeling.

1 Introduction

SIS-Bench frames UAV embodied intelligence as jointly modeling external space, the UAV’s self-state, and their interaction. It evaluates this capability across spatial cognition and self-awareness, revealing weaker self-modeling and progressive degradation across cognitive levels, while SIS-Motion improves both dimensions and transfers to navigation.

  • Benchmark: SIS-Bench evaluates UAV embodied intelligence across spatial cognition and self-awareness within a perception–memory–reasoning hierarchy.The benchmark contains 4,856 question–answer pairs across 13 tasks derived from real-world aerial videos.
  • Benchmark: 4,856 QA pairs from 1,646 real-world UAV videos support 13 tasks spanning diverse environments and video durations from 10 seconds to over 2 minutes.Construction uses heterogeneous video types, task-specific annotation pipelines, and dual-expert verification.
  • Motion-aware modeling: SIS-Motion fuses optical-flow-based motion features with visual embeddings to jointly capture environmental context and agent dynamics.It is presented as a controlled motion-aware extension of a standard video MLLM rather than a new flagship model.
  • Findings: 26 video MLLMs reveal weaker self-modeling than spatial cognition and progressive performance degradation from perception to memory to reasoning.The evaluation includes 6 proprietary and 20 open-source models.
  • Motion-aware modeling: SIS-Motion improves spatial cognition and self-awareness on SIS-Bench and transfers to downstream UAV navigation tasks.The exploration tests whether explicitly strengthening joint space-and-self modeling yields measurable gains.

2 Related Work

Related work has advanced MLLMs for UAV applications and spatial intelligence from visual understanding toward navigation, geometric reasoning, and embodied environments. However, explicit modeling of UAV agent-related motion remains underexplored despite its importance for viewpoint change and self-motion.

  • MLLMs for UAV Applications: UAV-oriented MLLM research has progressed from aerial visual understanding to trajectory planning, target-oriented navigation, and spatio-temporal scene interpretation.Applications include image understanding, object detection, and change detection before extending to navigation and decision-making.
  • MLLMs for Spatial Intelligence: Spatial-intelligence research has evolved from vision–language alignment to 3D geometric reasoning, structured spatial grounding, and embodied understanding in egocentric and interactive environments.These advances improve agents’ ability to perceive, represent, and reason about spatial structure in the physical world.
  • Motion-aware Modeling for Embodied UAVs: Most video-based MLLMs capture high-level semantics through CLIP-style pretraining but remain less effective at fine-grained motion and temporal dynamics.Recent work adds structured spatial cues and embodied video reasoning to improve spatio-temporal understanding.
  • Motion-aware Modeling for Embodied UAVs: Explicit modeling of agent-related motion remains underexplored in UAV scenarios, where viewpoint change and self-motion are fundamental.This gap motivates motion-aware modeling for embodied UAVs.

3 SIS-Bench

SIS-Bench evaluates UAV embodied spatial intelligence through a self-in-space formulation that jointly models the environment and the UAV’s evolving state. It organizes 13 tasks across spatial cognition and self-awareness, with progressive perception, memory, and reasoning levels supported by task-conditioned video construction and expert verification.

  • Hierarchical cognitive design: The benchmark contains 13 tasks organized across perception, memory, and reasoning to measure progressively higher cognitive demands.Perception uses Single Video inputs, memory uses Concatenated Video inputs, and reasoning uses Long Video and Shuffled Video inputs.
  • Design principles: SIS-Bench evaluates both spatial cognition and self-awareness, covering external scenes and the UAV’s motion, action history, and future behavior.Spatial cognition includes objects, landmarks, spatial relations, and scene consistency, while self-awareness captures the UAV’s own dynamics.
  • Hierarchical cognitive design: Perception tests immediate scene and action understanding, memory tests temporal retention, and reasoning integrates spatial context with agent dynamics.Memory includes Landmark Recall, Landmark Order, Positional Relationship, Action Sequence, and Action Recall; reasoning includes Spatial Consistency, Spatio-temporal Consistency, Action Prediction, and Path Planning.
  • Video construction: Four task-conditioned video types align inputs with evaluation demands: Single Video, Concatenated Video, Long Video, and Shuffled Video.Concatenated clips add cross-segment memory dependencies, Long Video preserves extended motion and scene evolution, and Shuffled Video disrupts chronology for recovery.
  • Benchmark construction: SIS-Bench uses a four-stage construction pipeline—Data Processing, Task-specific Annotation, QA Construction, and Dual-expert Verification—drawing on AirScape, UrbanVideo-Bench, and VisDrone.Processing includes converting VisDrone frame sequences at 15 FPS, concatenating 2–4 AirScape clips, and shuffling selected UrbanVideo-Bench videos.

4 How Well Do MLLMs Handle Self-Awareness and Spatial Cognition in UAV Scenarios?

SIS-Bench evaluations show that current MLLMs understand external UAV environments better than the embodied agent itself, with performance declining from perception through memory to reasoning. Human performance reaches 91.7% overall accuracy versus 71.6% for the best model, leaving substantial room for improvement.

  • Evaluation Setup: Evaluations cover 26 video-capable MLLMs under zero-shot prompting with greedy decoding, reporting task-level and overall accuracy.The models span diverse families, parameter scales, and training paradigms.
  • Overall Performance: 91.7% overall accuracy is achieved by human evaluators, compared with 71.6% for the best model, leaving a gap of over 20 points.The benchmark remains far from saturated, especially on self-awareness and reasoning tasks.
  • Spatial Cognition vs. Self-Awareness: Spatial cognition performance is consistently higher than self-awareness performance across most models.Models better interpret object layouts, attributes, and landmark relations than the UAV’s state, motion history, and action dynamics.
  • Cognitive Hierarchy: Performance is highest on perception, drops on memory, and declines further on reasoning across proprietary and open-source models.The hierarchy reflects increasing demands for temporally distributed evidence retention and coherent spatial or action-level inference.
  • Model Families and Scaling: Proprietary models generally outperform open-source models, but stronger-model gains are uneven and less evident for UAV motion, action recognition, and action recall.The proprietary–open-source gap remains modest relative to the gap from the best model to human performance.

5 A Motion-aware Exploration of Self in Space

SIS-Motion augments a video MLLM with optical-flow motion cues and visual feature fusion to jointly model self-related dynamics and spatial context. This motion-aware integration improves both self-awareness and spatial cognition, with stronger benefits for perception and memory and transfer to UAV navigation.

  • Motion-aware architecture: SIS-Motion uses a dual-encoder architecture combining visual scene representations, an optical-flow-based motion encoder, and a lightweight connector for feature fusion.The fused representation is fed to the language model, isolating gains attributable to motion-aware cues.
  • SIS-Bench results: Compared with visual-only SFT, SIS-Motion raises Spatial Avg from 72.0 to 74.2 and Self Avg from 60.3 to 63.7.The gains indicate benefits for both UAV self-understanding and interpretation of the surrounding environment under changing viewpoints.
  • SIS-Bench results: Motion-aware improvements are strongest on perception and memory tasks, while reasoning gains are limited: Path Planning improves, but Action Prediction does not.The training data focuses on perception and memory rather than high-level reasoning.
  • Robustness: All four tested motion encoders outperform the visual-only SFT baseline at 66.4% overall, demonstrating robustness to the choice of motion encoder.The encoders are RAFT, Sea-RAFT, MemFlow, and MOFNet, evaluated with the rest of the SIS-Motion setup fixed.
  • Downstream transfer: On a zero-shot UAV navigation benchmark with 3,895 questions from 22 simulated environments, SIS-Motion achieves 92.2% accuracy versus 71.2% for Qwen2.5-VL 3B.The benchmark covers urban, desert, forest, port, and island scenes.

6 Conclusion … A.2 Video construction protocol

SIS-Bench benchmarks embodied UAV intelligence through a unified self-in-space formulation, combining spatial cognition and self-awareness across 13 tasks. Its construction integrates diverse real-world UAV sources and video transformations to produce four temporally structured video types.

  • 6 Conclusion: SIS-Bench contains 4,856 question–answer pairs from 1,646 real-world UAV videos spanning 14.9 hours of aerial footage and 13 tasks.The benchmark is built with task-conditioned processing, task-specific annotation, and dual-expert verification, organized around spatial cognition and self-awareness.
  • A.1 Source datasets and collection protocol: The benchmark construction uses short clips, long continuous videos, diverse agent motions, and varied authentic UAV scenes to support perception, memory, and reasoning evaluation.Source selection follows criteria covering temporal characteristics, motion and action diversity, and real-world scene diversity.
  • A.1 Source datasets and collection protocol: AirScape supplies generally short action-intent videos, with its adjusted splits preventing test leakage when UrbanVideo-Bench is also used as an independent source.The original AirScape test split is used for benchmark construction, while the original training split is reserved for subsequent training-data construction.
  • A.1 Source datasets and collection protocol: UrbanVideo-Bench contributes 174 real-world videos from authentic urban environments, typically preserving longer continuous flight trajectories than AirScape clips.The selected scenes include streets, residential communities, schools, and parks in Shenzhen, Guangdong Province, China.
  • A.1 Source datasets and collection protocol: VisDrone contributes 63 low-altitude urban scene-level frame sequences, which are converted into videos before annotation and question–answer construction.The sequences cover settings such as streets, parks, and residential neighborhoods and enrich benchmark scene diversity.
  • A.2 Video construction protocol: Video processing applies Converted, Concatenated, and Shuffled operations while retaining some long videos in their original form.These operations yield four benchmark types: Single Video, Concatenated Video, Long Video, and Shuffled Video.
  • A.2 Video construction protocol: VisDrone frame sequences are converted at 15 FPS, making every 15 frames correspond to 1 second of video time.This conversion produces temporally coherent video inputs that can be processed consistently with the other benchmark sources.
  • A.2 Video construction protocol: AirScape processing produces 689 Single Video samples and 467 Concatenated Video samples by randomly stitching 2–4 short clips with balanced composition proportions.The adjusted AirScape pool contains 2067 short videos: 923 from UrbanVideo, 444 from NAT2021, and 700 from WebUAV3M.

A.3 Summary of the processed benchmark videos … B.5 Benchmark summary

SIS-Bench combines four temporally diverse UAV video types with 13 task formulations spanning spatial cognition and self-awareness across perception, memory, and reasoning. Its 4,856-question benchmark is built through task-specific annotation and generation pipelines followed by dual-expert review.

  • A.3 Summary of the processed benchmark videos: 689 Single Video, 467 Concatenated Video, 237 Long Video, and 253 Shuffled Video samples form the processed benchmark video pool.Videos span about 5 seconds to more than 2 minutes and cover urban streets, residential areas, schools, parks, and waterfront regions.
  • B.1 Task formulation details: 13 tasks are organized across spatial cognition and self-awareness, with perception, memory, and reasoning matched to different video construction types.Perception uses Single Video, memory uses Concatenated Video, and reasoning uses Shuffled Video or Long Video inputs.
  • B.1 Task formulation details: Spatial cognition tasks progress from object and landmark perception to route-memory retrieval and consistency reasoning under concatenated or shuffled observations.The reasoning tasks reconstruct stable spatial relations or original flight trajectories from fragmented, reordered clips.
  • B.1 Task formulation details: Self-awareness tasks evaluate instantaneous action recognition, ordered action tracking, action recall, action prediction, and path planning across short, concatenated, and long videos.Action Prediction and Path Planning require continuous long-horizon motion context, while Action Sequence and Action Recall track actions across stitched segments.
  • B.2 Annotation details: Three annotation pipelines cover self-awareness metadata, model-assisted spatial metadata, and expert-produced reasoning metadata, with manual correction or verification by four trained annotators.Pipeline-A reorganizes AirScape action annotations, Pipeline-B filters and corrects model-produced scene metadata, and Pipeline-C uses expert annotation for complex routes and long temporal contexts.
  • B.3 QA generation details: 13 task-specific QA templates use multiple LLMs to generate diverse multiple-choice questions and plausible, non-ambiguous distractors grounded in structured annotations.The process removes explicit directional expressions where needed and checks for lexical shortcuts, duplicates, and answer leakage.
  • B.4 Dual-expert review details: 6,174 generated QA pairs underwent dual-expert review, yielding 1,944 approvals, 2,912 modified-and-retained samples, 1,318 rejections, and 4,856 final QA pairs.The corresponding approval, modification, and rejection rates were 31.5%, 47.2%, and 21.3%, respectively.
  • B.5 Benchmark summary: 4,856 multiple-choice QA pairs span 13 tasks, comprising 2,516 spatial-cognition pairs, 2,340 self-awareness pairs, 1,765 perception pairs, 2,120 memory pairs, and 971 reasoning pairs.Action Recall, Action Recognition, Object Existence, and Landmark Recall are the largest subsets, while reasoning tasks are comparatively compact.

B.6 Benchmark comparison

SIS-Bench differs from prior UAV benchmarks by using real-world urban UAV videos and emphasizing real-flight visual grounding and motion understanding. It jointly evaluates spatial cognition and self-awareness within a unified hierarchy for fine-grained embodied UAV assessment.

  • Benchmark comparison: SIS-Bench uses real-world UAV videos from common urban scenes, unlike AerialVLN, CityNav, and OpenUAV, which rely on simulator-based production pipelines.This design emphasizes real-flight visual grounding and motion understanding over simulator-generated trajectories and test instances.
  • Benchmark comparison: SIS-Bench jointly evaluates spatial cognition and self-awareness within a unified perception–memory–reasoning hierarchy.Its task construction also spans four complementary video types, including Single Video, Concatenated Video, and Long Video.
  • Benchmark comparison: SIS-Bench enables fine-grained evaluation of embodied UAV cognition in real urban aerial environments.Compared with UrbanVideo-Bench and MM-UAVBench, it is comparable in overall scale while differing in benchmark organization.

C Evaluation Protocol · D Deeper Diagnosis

The evaluation protocol standardizes video processing and inference across 26 models, while a task-balanced human baseline provides reference performance. Deeper diagnosis supplements aggregate accuracy with controlled comparisons, error analysis, and instance-level failure investigation.

  • C Evaluation Protocol: Adaptive sampling retains at most 32 frames per video, using 2 FPS below 16 seconds, 1 FPS from 16–32 seconds, and uniform sampling beyond 32 seconds.The strategy is designed to reduce evaluation cost while preserving denser temporal evidence for short videos.
  • C Evaluation Protocol: 26 models are evaluated, comprising 20 open-source and 6 proprietary systems.Open-source models run on four RTX 4090 GPUs with vLLM, while proprietary models use API-based inference.
  • C Evaluation Protocol: Open-source evaluation uses a unified 32,768-token context window, up to 128 generated tokens, and at most 32 frames per video.The setup also specifies no fixed image resizing and pixel bounds from 256 × 28 × 28 to 512 × 28 × 28.
  • C Evaluation Protocol: The human baseline samples 80 QA pairs from each of 13 tasks, producing 1,040 questions and 6,240 independent responses from six participants.Participants have master’s-level EECS or AI training, did not contribute to annotation or verification, and may replay videos.
  • C Evaluation Protocol: 91.7% is the reported human accuracy, averaged across the six participants’ individual responses.The human-study subset contains 1,040 questions balanced across the 13 benchmark tasks.
  • D Deeper Diagnosis: Deeper diagnosis combines controlled evaluations, matched model comparisons, an error taxonomy, and instance-level failures to examine trends and reasoning limitations.These analyses extend beyond aggregate accuracy to characterize remaining failure modes.

D.1 Robustness controls and model-type analysis

Confounding controls leave the perception>memory>reasoning trend unchanged, while explicit reasoning training improves weighted reasoning accuracy by 2.5 points for Qwen3-VL-8B and 0.6 points for MiMo-VL-7B. These gains are task-dependent, most visible in temporal action reasoning, and can reverse in Path Planning.

  • Confounding-factor controls: Uniform 32-frame sampling changes level-wise results by at most 0.7 percentage points for Qwen3-VL-8B and InternVL3.5-8B, leaving the perception>memory>reasoning trend unchanged.This control equalizes the frame budget across video lengths.
  • Reasoning-oriented models: Switching from Qwen3-VL-8B-Instruct to Qwen3-VL-8B-Thinking improves weighted reasoning accuracy by 2.5 points.The comparison uses matched model pairs.
  • Reasoning-oriented models: Switching from MiMo-VL-7B-SFT to MiMo-VL-7B-RL produces a smaller 0.6-point gain in weighted reasoning accuracy.The gain is smaller than the matched Qwen3-VL-8B comparison.
  • Reasoning-oriented models: Reasoning-training improvements are most visible on temporal action reasoning, but gains are task-dependent and Path Planning can decrease.The passage links explicit reasoning training to selected-task improvements rather than universal gains.

D.2 Error taxonomy and cognitive bottlenecks … E.2 Training details

The analysis identifies action understanding and spatial reasoning as dominant failure categories, with errors increasing as tasks require self-motion estimation, temporal binding, and reasoning. SIS-Motion-54K supplies perception- and memory-focused supervision, while its training uses parameter-efficient LoRA configurations and does not directly supervise planning.

  • D.2 Error taxonomy and cognitive bottlenecks: Action Understanding Error (33.0%) and Spatial Reasoning Error (18.2%) are the two largest categories among 7,987 incorrect responses.The taxonomy covers eight visual, temporal, spatial, logical, action, memory, instruction, and hallucination categories across four representative models.
  • D.2 Error taxonomy and cognitive bottlenecks: Failures progress from local spatial perception to self-motion tracking, cross-time binding, and joint temporal, spatial, and action-consequence reasoning.The reported diagnosis attributes self-awareness difficulty to estimating and tracking the UAV’s own motion state.
  • D.3 Below-random Path Planning performance: 20.6% SFT, 25.2% random baseline, and 23.5% SIS-Motion results belong to Path Planning, whereas Action Prediction reaches 45.2% with visual-only SFT and 44.1% with SIS-Motion.Action Prediction’s task-specific random baseline is 24.9%, so the below-random observation is specific to Path Planning.
  • D.3 Below-random Path Planning performance: Path Planning remains near chance because SIS-Motion-54K supervises nine perception and memory tasks but contains no planning-oriented examples.Planning requires goal-conditioned, long-horizon inference over scene constraints, action consequences, and evolving UAV pose, which local optical-flow cues alone do not provide.
  • D.4 Representative failure cases: Representative failures involve weak-visibility motion grounding, segment-level temporal organization, and fine-grained discrimination between rotation and translation.Examples include mistaking vertical descent with rightward yaw for curved forward motion and corrupting action order across concatenated clips.
  • E.1 SIS-Motion-54K construction: SIS-Motion-54K is built from the adjusted AirScape training pool using benchmark task templates and strict separation from benchmark construction.The pool contains 0 UrbanVideo clips, 1349 NAT2021 clips, and 7596 WebUAV3M clips, with MCQ, OpenQA, and descriptive supervision.
  • E.2 Training details: All experiments use parameter-efficient LoRA fine-tuning on 4 RTX 4090 GPUs, with one epoch, BF16, sequence length 2048, and 2 FPS video sampling.Common settings include batch size 1, gradient accumulation 8, learning rate 1 × 10−5, cosine scheduling, and LoRA rank 32, alpha 64, dropout 0.05.
  • E.2 Training details: SIS-Motion freezes the vision and motion encoders, trains the multimodal connector with LoRA enabled, and uses a projector learning rate of 2 × 10−5.Baseline fine-tuning freezes the vision encoder and trains remaining components; SIS-Motion checkpoints are saved every 100 steps, retaining at most three.

E.3 Ablation settings and results · F Downstream Navigation Transfer · G Limitations and Future Directions

The ablations show that explicit motion cues improve performance across optical-flow estimators and that scaling the backbone further strengthens the motion-aware design. A downstream OpenUAV path-planning benchmark transfers this modeling approach to UAV decision-making, while broader limitations and future directions remain open.

  • E.3 Ablation settings and results: Two ablation groups vary the optical-flow estimator and Qwen2.5-VL backbone capacity to isolate motion-design and scaling effects.The comparisons include RAFT, Sea-RAFT, MemFlow, and a 7B replacement for the 3B backbone.
  • E.3 Ablation settings and results: All motion-enhanced variants outperform the corresponding vanilla 3B baseline, indicating benefits from explicit motion cues across estimators.The result does not depend on a single optical-flow estimator.
  • E.3 Ablation settings and results: 69.1 to 76.9: the 7B variant improves over the 3B setting and remains stronger than the corresponding vanilla backbone, which rises from 73.1 to 76.9.These comparisons support the contribution of both motion modeling and increased backbone capacity.
  • F Downstream Navigation Transfer: The downstream task adapts OpenUAV into a multimodal path-planning evaluation using target-pose trajectories, front-view videos, and derived flight operations.Altitude changes larger than 5 meters define takeoff or landing, while consecutive identical operations form flight segments with distance, altitude change, and turning angle.
  • F Downstream Navigation Transfer: The adapted navigation data become four-choice questions whose correct option is the true path and whose distractors use direction reversal, distance perturbation, or altered operation stages.An LLM rewrites rule-based path descriptions into natural question wording.
  • F Downstream Navigation Transfer: 3895 questions across 22 simulated scene maps evaluate cross-environment generalization in urban streets, deserts, forests, ports, and tropical islands.The downstream evaluation follows the procedure used in Section 3.
  • G Limitations and Future Directions: The ablation and downstream studies isolate motion-modeling contributions and demonstrate transfer to UAV path planning, while several directions remain open beyond the current evaluation.The limitations section identifies open directions without specifying them in the provided passage.

G.1 Scope and limitations … G.4 Autonomous flying agents

The paper frames SIS-Bench as a structured evaluation of space and self, while identifying limitations in benchmark scope, multimodal modeling, sensing, and autonomous UAV deployment. It motivates unified embodied cognition that connects perception, reasoning, and action in open environments.

  • G.1 Scope and limitations: SIS-Bench explicitly factorizes space and self across perception, memory, and reasoning rather than introducing entirely new atomic tasks.Its scope is limited to standardized existing datasets, multiple-choice evaluation, and open-loop UAV video understanding.
  • G.1 Scope and limitations: SIS-Motion is presented as a controlled motion-aware probe rather than a highly optimized architecture.The supplied passage begins reporting an accuracy improvement on Qwen2.5-VL-3B, but does not provide the complete value.
  • G.2 Native multimodal models: Native multimodal foundation models increasingly unify language, vision, motion, and environmental feedback.This direction is relevant to UAV agents that interpret dynamic scenes, follow instructions, track ego-motion, and decide under changing viewpoints.
  • G.2 Native multimodal models: Future work should pursue unified multimodal embodied cognition rather than isolated text reasoning.The motivation is to integrate the capabilities required by dynamic UAV operation.
  • G.3 Integrated sensing: CLIP-style visual encoders remain limited for fine-grained spatial structure, subtle motion, and long-horizon continuity in UAV videos.These limitations are especially pronounced with small targets, viewpoint changes, altitude variation, and trajectory-dependent reasoning.
  • G.3 Integrated sensing: Integrated sensing should combine appearance, motion, depth, geometry, and temporal correspondence for stable scene reconstruction and action understanding.The proposed combination addresses limitations of coarse semantic visual representations in dynamic UAV video.
  • G.4 Autonomous flying agents: Although SIS-Bench evaluates spatial cognition and self-awareness and SIS-Motion improves action perception, question answering remains an intermediate capability.The paper therefore identifies a gap between benchmark performance and autonomous UAV behavior.
  • G.4 Autonomous flying agents: Future autonomous UAV agents should use self-in-space understanding for navigation, planning, interaction, and long-horizon decision making in open environments.This direction aims to close the perception–reasoning–action loop.
Loading 2607.12477v2…