Source-linked AI summary
SkyAgent: Revisable Target Memory for Continuous Aerial Search
Kaibo Yang, Xingyu Wu, Enze Zhu, Yajie Chai, Xuyang Li, Jiahui Cui, Xiaoxuan Liu, Lei Wang, Zhan Chen
TL;DR
Continuous aerial search can leave contradicted candidates available to planning, creating false candidate persistence. SkyAgent’s Revisable Target Memory associates evidence with candidates and binds their generated waypoints to them, so rejection can cancel stale goals. Across four paired benchmark splits, RTM improves task performance while reducing FCP and keeping false rejection low.
Problem
In continuous aerial search, sparse viewpoint-dependent evidence can contradict a stored candidate that nevertheless remains active or continues triggering actions.
Source: §1 · pp. 1–2Method
RTM associates signed evidence with persistent candidates, revises their lifecycle states, and cancels unexecuted waypoints tagged to a rejected candidate.
Source: §1 · p. 2, §4.2 · p. 5Results
Across four paired benchmark splits, RTM improves task performance while reducing FCP and keeping false rejection low, with SR gains of 2.90–6.31 points.
Source: §5.3 · p. 6, §6 · p. 7Takeaways & Limitations
For static-target search, candidate-bound waypoints connect memory revision to planner behavior by canceling queued goals from rejected candidates.
Source: §3 · p. 3, §4.2 · p. 5RTM assumes reliable cross-view association, covers static objects, and its tested flights do not establish a general safety guarantee.
Source: §6 · p. 7
Abstract
from arXiv · showhide
Continuous aerial search builds target hypotheses from sparse, viewpoint-dependent observations. When later evidence contradicts a stored hypothesis, however, the candidate can remain available to the planner and continue triggering approach or verification actions. We call this trajectory-level failure false candidate persistence (FCP). We equip SkyAgent with Revisable Target Memory (RTM), which associates evidence with persistent candidates, discounts correlated views, and revises each candidate through Tentative, Confirmed, and Rejected states. The key interface binds every generated waypoint to its source candidate: rejection therefore invalidates stale goals, while conservative reactivation protects temporarily occluded targets. Across four splits from the CityAVOS, UAV-ON, and OpenUAV benchmarks, our SkyAgent equipped with RTM outperforms all competitors, improving SR by 2.90--6.31 points while substantially reducing Behavior-FCP and post-contradiction travel with a low false-rejection rate. Across 35 matched real-world DJI scenarios per method, RTM consistently improves task success and behavioral reliability, further validating its state-to-action correction under physical deployment conditions.
1 Introduction
Continuous aerial search must revise target hypotheses when later viewpoints contradict them, rather than leaving disproved candidates available to planning. SkyAgent’s RTM makes candidate revision affect both memory state and the goals already generated from it.
1 Introduction
- A later independent view can expose a candidate as wrong, yet append-only memory may keep it active and waste route on it.Figure 1 contrasts that behavior with RTM rejecting the candidate, canceling its pending waypoint, and resuming search.Source: Figure 1 · p. 1, §1 · p. 1
- False candidate persistence (FCP) is a trajectory-level failure in which a contradicted candidate remains active or continues triggering approach, revisit, or confirmation actions.Unlike a transient detection error, it can consume distance, time, and planning cycles through stale memory.Source: §1 · pp. 1–2
- Revisable memory must retain supporting and opposing evidence, discount redundant adjacent frames, and invalidate every pending waypoint sourced by a rejected candidate.Rejection should not be permanent because temporary occlusion or low-quality frames can produce misleading negative evidence.Source: §1 · p. 2
- RTM combines Candidate Evidence Memory with Revise-and-Suppress to associate observations with persistent candidates, accumulate signed evidence, revise lifecycle states, and cancel stale waypoints.The existing search policy, event trigger, inspection skills, and safety-projected controller remain execution infrastructure.Source: §1 · p. 2
- The paper introduces trajectory-level FCP measures and evaluates RTM with controlled comparisons and mechanism ablations on three benchmarks and 35 matched DJI scenarios per method.Its planner binding cancels goals sourced by rejected candidates while allowing conservative reactivation after new independent positive evidence.Source: §1 · p. 2
2 Related Work
Prior aerial-search work improves memory representation, retrieval, coverage, or verification before decisions. RTM instead targets what happens to a stored candidate and its already-generated goals after contradictory evidence arrives.
2 Related Work
- Aerial-search and navigation methods use visibility-aware planning, maps, hierarchical memory, VLM reasoning, and learned policies to improve coverage or persistent representations.The related work also describes open-world aerial object-goal search in UAV-ON and CityAVOS, alongside systems that scale retrieval through maps, octrees, or voxel–ray representations.Source: §2 · p. 2
- Ground ObjectNav and lifelong-navigation methods attach semantics to maps or scene graphs, compress observations into long-term memory, or retrieve experience across goals and episodes.These approaches chiefly address representation, retention, retrieval, or coverage.Source: §2 · p. 2
- Verification methods acquire or revisit informative views before committing, while CARe replans decisions induced by an erroneous pre-explored semantic map.RTM’s distinction is that each waypoint records its source candidate, so rejection invalidates all unexecuted goals from that candidate.Source: §2 · p. 2
- VLM self-correction and episodic reflection revise errors or feedback, but RTM corrects candidate memory online and cancels still-pending waypoints within the same episode.RTM retains supporting and contradicting observations for audit and permits recovery after independent positive views arrive.Source: §2 · p. 2
3 Problem Setup
The setup treats continuous aerial search as evidence accumulation over localized, persistent candidate hypotheses, with identity belief updated as viewpoint-dependent observations arrive. It defines candidate lifecycle and external reference events to distinguish persistence, behavioral effects, and recovery.
3 Problem Setup
- Continuous aerial search builds persistent candidate hypotheses from partial, viewpoint-dependent observations rather than treating each detection as independent.At each step, the agent receives an observation and the perception front end produces an evidence item.Source: §3 · p. 3
- Each evidence item is localized to one candidate and contains a crop or mask, estimated 3-D location, multimodal feature, signed query-relative evidence, and quality.The observation tuple includes the current image, vehicle pose, and image or semantic target query.Source: §3 · p. 3
- The setup covers physically static targets: spatial location remains fixed while identity judgment and belief score are updated with new evidence..Source: §3 · p. 3
- A candidate has spatial identity, independent supporting and opposing evidence, a bounded belief score, a Tentative/Confirmed/Rejected state, and a count of new independent positive groups since rejection.Confirmed permits approach and benchmark-defined final verification; benchmark stopping rules still determine task success.Source: §3 · p. 3
- False-candidate and true-target recovery event sets are fixed before results, using simulator labels or blinded annotation for contradiction and new independent positive views after degradation or occlusion for recovery.The sets are disjoint, so temporary loss of a true target is not classified as FCP.Source: §3 · p. 3
- FCP is measured at state and behavior levels, while post-contradiction waste attributes distance and time to candidate-directed actions after the contradiction window.False candidates are stratified as same-category distractors, reflections, repeated texture, or scale/viewpoint confusions; the recovery window closes after new positive views.Source: §3 · p. 3
4 Revisable Target Memory
RTM treats candidates as revisable hypotheses, linking accumulated evidence and lifecycle state to the planner. Rejection removes a candidate’s pending goals, while integration preserves the existing flight-execution infrastructure.
4.1 Candidate Evidence Memory
- Candidate Evidence Memory links observations to persistent candidates and accumulates signed evidence grouped by independent viewpoint.Correlated frames update one bounded group contribution rather than adding repeated votes; the accumulator is not claimed to be a calibrated Bayesian posterior.Source: §4 · p. 3, §4.1 · pp. 4–5
4.2 Revise-and-Suppress
- Lifecycle transitions depend on both the candidate’s current score and state, with stricter recovery requiring new independent positive evidence after rejection.Reactivation returns a candidate to Tentative; a later update is required for confirmation.Source: §4.2 · p. 5
4 Revisable Target Memory
- RTM binds each waypoint to its source candidate, so rejection cancels that candidate’s unexecuted goals before the next planning cycle.If eligible candidates remain, planning continues from them; otherwise the agent resumes coverage search.Source: §4 · p. 3, §4.2 · p. 5
4.2 Revise-and-Suppress
- The planner reads only non-rejected candidates, and cancellation removes queued goals that verify-before-act methods leave intact.Waypoints store their source candidate ID; an action already under safety-critical execution is handled by the safety controller rather than interrupted by memory.Source: §4.2 · p. 5
4.3 Integration with Flight Execution
- The fast search policy, event trigger, inspection skills, and safety-projected controller remain execution infrastructure, while RTM alone manages candidate state.The language router cannot directly set candidate state, reactivate candidates, or bypass waypoint cancellation.Source: §4.3 · p. 5
5 Experiments
The experiments evaluate RTM on three aerial benchmarks and matched DJI flights, measuring task outcomes alongside false-candidate persistence, suppression, and recovery. RTM improves benchmark task outcomes while reducing FCP, and ablations and physical trials examine its mechanisms and transfer.
5.1 Setup
- The experiments use CityAVOS Hard and Medium controls, 1,000 UAV-ON episodes, and 816 OpenUAV-Hard trajectories, with benchmark-standard observations and goal information.RTM variants share task IDs, targets, perception, action budgets, controllers, and evaluators; calibration choices are fixed on disjoint episodes.Source: §5.1 · p. 5
5.2 FCP in Unmodified Search
- 89 of 197 reference-event episodes retain contradicted candidates in append-only CityAVOS-Hard search (BFCPR 45.18%), accruing 120.76 m of post-contradiction travel.Two annotators label candidate events and attributable distance under fixed thresholds across five distractor, appearance, viewpoint, occlusion, and degradation strata.Source: §5.2 · p. 6
5.3 Controlled Benchmark Results
- BFCPR falls from 45.18% to 15.27% and post-contradiction waste from 120.76 m to 44.21 m on CityAVOS-Hard; other splits show the same pattern, with FRR at 1.94–3.52%.CityAVOS-Medium, UAV-ON, and OpenUAV-Hard BFCPR reductions are 12.70, 17.63, and 21.25 points, respectively.Source: §5.3 · p. 6
- SR gains of 2.90–6.31 points and improved path efficiency appear on every benchmark split, with the largest SR gain on ambiguity-rich CityAVOS-Hard.The comparisons use identical paired episodes and complete splits; OpenUAV-Hard baselines combine two published Hard aggregates.Source: §5.3 · p. 6
5.4 Mechanism, Recovery, and Stress
- On CityAVOS-Hard, action-only rechecking leaves BFCPR at 38.19% and Waste at 96.17 m, versus 15.27% and 44.21 m for Full RTM; removing goal cancellation yields 28.86% and 74.24 m.Permanent rejection raises FRR to 11.65% and lowers ORR to 65.63%; Full RTM has 2.91% FRR and 92.71% ORR. Oracle association reaches 42.11% SR and 8.82% BFCPR, identifying cross-view association as the main remaining bottleneck.Source: §5.4 · p. 6
- On 816 OpenUAV-Hard trajectories, BFCPR drops from 35.32% to 14.07% and Waste from 120.90 m to 50.41 m, while SR rises from 18.01% to 22.06%.The CityAVOS-Hard control confirms the FRR/ORR trade-off; Figure 3 audits trajectories, candidate states, and cancellation.Source: §5.4 · pp. 6–7
5.5 Real-Flight Evaluation
- In matched DJI flights, RTM raises SR from 57.14% to 71.43%, reduces distance from 264.37 m to 223.81 m and BFCPR from 45.45% to 13.64%, and lowers interventions from 14.29% to 5.71%.The four methods share 35 starts, regions, layouts, weather strata, and budgets; order is randomized and outcomes are retained across five conditions.Source: §5.5 · p. 7
6 Limitations and Conclusion
RTM improves task performance across four paired benchmark splits while reducing FCP and keeping false rejection low. Its evaluation assumes reliable cross-view association and covers static objects without establishing a general safety guarantee.
6 Limitations and Conclusion
- Reliable cross-view association is assumed; merging or splitting candidates distorts FCP counts, and oracle association estimates the tracking contribution.Source: §6 · p. 7
- Closed-loop results measure system behavior because RTM changes later routes and observations; common-trajectory replay is needed to isolate revision.Source: §6 · p. 7
- The scope covers static objects, and the tested flights do not establish a general safety guarantee.Source: §6 · p. 7
- RTM improves task performance across four paired benchmark splits while reducing FCP and keeping false rejection low.Source: §6 · p. 7
- Candidate Evidence Memory aggregates signed evidence across viewpoints, while Revise-and-Suppress cancels waypoints sourced by rejected candidates.Source: §6 · p. 7