Source-linked AI summary
Learning to Fly: Stable Vision-Guided UAV Servoing with Compact Target-Centric Cues and Reinforcement Learning
Saurbh Singh Jamwal, Nived Chebrolu
TL;DR
Vision-guided UAV reinforcement learning is challenged by unstable optimisation and costly high-dimensional perception. The paper uses compact target-centric cues with low-dimensional measurements, compares matched-budget PPO curricula, and finds comparable nominal performance with improved robustness under perturbations and unseen target motion.
Problem
Vision-guided UAV reinforcement learning lacks stable, robust long-horizon learning under difficult visual observations and operating conditions.
Method
The framework extracts segmented image-space offsets and relative depth, combines them with quadrotor measurements into a 12D observation, and compares Direct PPO with Visual, Dynamics, and Joint curricula.
Results
Comparable nominal performance is achieved across learned strategies, while learned policies show greater robustness than classical controllers and Visual Curriculum PPO has the smallest degradation under unseen target motion.
Takeaways & Limitations
Compact target-centric representations support long-horizon aerial visual servoing, and visual curriculum training can improve robustness to dynamic distribution shifts despite limited nominal gains.
Takeaways & Limitations
The study is simulation-based and relies on simplified target segmentation and simulator-provided depth, so it does not address real-world perception and dynamics mismatch or cluttered, occluded, multi-target scenes.
Abstract
from arXiv · showhide
Vision-guided reinforcement learning for Unmanned Aerial Vehicles (UAVs) remains challenging due to unstable policy optimisation, aggressive exploration, and the cost of high-dimensional visual perception. In this work, we investigate long-horizon UAV visual servoing using compact target-centric cues combined with low-dimensional sensor measurements. Rather than learning directly from RGB images, lightweight target segmentation provides image-space offsets and relative depth, which are combined with quadrotor velocity and projected-gravity measurements into a compact 12D policy observation. We compare Direct PPO with three matched-budget curriculum strategies: a Visual curriculum that progressively expands target placement difficulty, a Dynamics curriculum that gradually relaxes action constraints and smoothing, and a Joint curriculum that combines both progressions. All strategies reach comparable nominal performance, with complementary advantages across tracking metrics. Observation ablations show that proprioceptive measurements are critical for stable flight and image-space cues for target alignment, while explicit depth is not necessary for strong performance in the evaluated setting. Against tuned classical visual-servo controllers, learned policies show greater robustness to strong control and visual perturbations, while the Visual curriculum exhibits the smallest degradation under unseen target motion. Overall, the results demonstrate that compact target-centric representations can support robust long-horizon aerial visual servoing and that visual curriculum training can improve robustness to dynamic distribution shifts despite limited gains in nominal performance.
1 Introduction
The paper addresses unstable vision-guided UAV reinforcement learning by using compact target-centric observations and matched-budget curriculum strategies. It finds comparable nominal performance across strategies, with complementary robustness and information requirements.
- Vision-guided UAV control remains difficult because policy optimisation is unstable, visual observations are high-dimensional, and operating conditions can be challenging.
- The framework combines segmented image-space target offsets, relative depth, and quadrotor measurements into a 12D policy observation without high-dimensional visual feature extraction.
- Direct PPO and Visual, Dynamics, and Joint curricula achieve comparable long-horizon nominal performance under matched optimisation budgets, with complementary tracking and control trade-offs.
- Observation ablations show that proprioception is critical for stable flight, image-space cues support servo alignment, and explicit target depth is unnecessary for strong performance in the evaluated setting.
- Learned policies show greater robustness than tuned classical visual-servo controllers under strong control and visual perturbations, while Visual curriculum training reduces degradation under unseen target motion.
2 Related Work
Prior UAV control research spans classical model-based methods, reinforcement learning, and vision-based navigation and servoing. This work focuses that landscape on compact target-relative geometry and controlled curriculum comparisons for long-horizon visual servoing.
- Classical UAV controllers can provide strong stabilisation and trajectory tracking but often require system identification, expert tuning, accurate state estimation, and carefully designed pipelines.
- Reinforcement learning enables data-driven UAV control, yet stable long-horizon learning, sample efficiency, and recovery remain challenging, especially with noisy visual observations.
- Vision-based UAV research includes classical visual servoing, end-to-end navigation, obstacle avoidance, target following, and reinforcement-learning-based visual control.
- Because visual servoing is naturally expressed through target-relative geometry, the paper asks whether task-relevant information can be represented with compact cues rather than high-dimensional visual features.
- The study separately varies visual difficulty, control difficulty, and their joint progression while keeping the policy, reward, optimisation budget, and final task fixed.
3 Methodology
The method uses compact target-centric visual cues and low-dimensional quadrotor measurements for PPO-based visual servoing, then compares learned and classical controllers under matched training and robustness conditions.
- Compact observation design: Target segmentation supplies compact image-space offsets and relative depth instead of high-dimensional RGB features.The target-centric representation is designed to retain alignment and distance information while separating perception from downstream control.
- Reinforcement learning framework: The policy combines target cues with body-frame linear velocity, angular velocity, and projected gravity in a 12-dimensional observation.PPO outputs continuous thrust and roll, pitch, and yaw moment commands for closed-loop quadrotor control.
- Reward design: The reward balances image-space alignment, depth regulation, visibility, angular-motion regularisation, and altitude stabilisation.The same reward formulation and weights are used across Direct PPO and curriculum strategies.
- Curriculum learning strategy: Direct PPO is compared with Visual, Dynamics, and Joint curricula using identical policies, rewards, PPO settings, and optimisation budgets, differing only in difficulty progression.Visual curricula expand target placement difficulty, Dynamics curricula relax action limits and smoothing, and Joint curricula apply both progressions.
- Classical baselines: Classical 2D and 3D visual-servo controllers use the same compact observations and produce the same four-dimensional action vector as PPO.Their gains are tuned in the nominal static-target setting and then fixed for moving-target and robustness evaluations.
- Robustness evaluation: Robustness evaluation applies fixed level-0.2 action noise, visual noise, and visual dropout only at test time.Direct PPO, Visual Curriculum PPO, and classical baselines are compared using reward, tracking, visibility, successful-step, and survival metrics.
4 Experiments
The experiments compare matched-budget PPO curricula, observation variants, unseen target motion, evaluation-time perturbations, and camera-calibration changes. Results show comparable nominal curriculum performance, distinct observation roles, and stronger robustness of learned policies under several shifts.
- Experimental Setup: The full 12D observation combines target-centric visual cues with quadrotor measurements for the ablation study.The evaluation uses PPO policies and reports reward, center error, depth error, successful-step rate, visibility rate, and survival rate.
- Equal-Budget Curriculum Analysis: All curriculum strategies reach comparable long-horizon performance under matched optimisation budgets, but each favours different tracking trade-offs.Dynamics Curriculum PPO has the highest reward and episode length, Visual Curriculum PPO the highest successful-step rate and lowest depth error, and Direct PPO the lowest center error.
- Observation Ablation: Removing proprioception reduces survival from 97.1% to 4.4%, while the state-only policy preserves 100% survival but substantially worsens target centering and successful-step rate.These results distinguish flight stability from accurate visual servoing.
- Observation Ablation: Removing explicit depth does not degrade performance in the evaluated operating regime, with the 11D no-depth policy achieving the highest reward, lowest center error, and highest successful-step rate.The authors caution that this does not establish depth as unnecessary under broader target-motion or depth-variation conditions.
- Unseen Target Motion: Visual Curriculum PPO degrades 3.94% from static to hard-helix evaluation and retains 97.7% survival, versus 15.82% degradation and 84.9% survival for Classical Servo 3D.The learned policies were evaluated without retraining, while classical controllers were not gain-retuned for the new target motion.
- Evaluation-Time Robustness: Under visual noise, reward degradation is 27.09% for Direct PPO and 25.43% for Visual Curriculum PPO, compared with 36.62% and 39.05% for the classical controllers.PPO policies retain survival above 97%, whereas classical control retains approximately 85–87%; dropout has little effect across methods.
- Camera Calibration Robustness: With focal length varied by ±20%, reward changes remain at or below 1.2% across methods, while successful-step-rate changes remain small.Both PPO variants retain performance with camera intrinsics not encountered during training, and the authors attribute the shared robustness largely to normalized target-centric representation.
5 Discussion
The experiments show that curriculum learning offers limited nominal benefit but can improve robustness under distribution shifts, while observation components serve complementary control roles. Learned policies trade some nominal accuracy for greater perturbation tolerance than classical controllers.
- Direct PPO matches curriculum variants on nominal static-target performance, so curriculum learning is not required for the nominal task.
- Visual Curriculum PPO has the smallest reward degradation under unseen target motion while retaining high survival.
- Classical controllers achieve stronger nominal tracking, whereas learned policies degrade less under action and visual perturbations and maintain higher survival under challenging shifts.
- Proprioception supports stable flight, image-space cues support visual alignment, and explicit depth is less critical in the evaluated range.
6 Limitations and Future Work
The study’s limitations concern simplified perception and simulation-based evaluation rather than the policy interface itself. Future work targets realistic perception, broader failure modes, and sim-to-real deployment.
- Simplified segmentation and simulator-provided depth leave clutter, severe occlusion, multiple targets, semantic understanding, and real-world perception failures unaddressed.
- The perception block can be replaced with stronger detectors, trackers, segmentation models, or depth estimators without changing the policy interface.
- Simulation-based experiments do not capture the full perception and dynamics mismatch of physical flight or exhaustive deployment conditions.
7 Conclusion
The paper presents compact target-centric observations for long-horizon aerial visual servoing and evaluates them against curriculum strategies and classical controllers. The resulting policies retain comparable nominal performance while offering robustness advantages beyond training conditions.
- The framework combines mask-derived image offsets, relative depth, and quadrotor state measurements into a compact control-oriented interface.
- Direct PPO and three curriculum variants achieve comparable nominal performance with different tracking trade-offs.
- Observation ablations identify complementary roles for proprioceptive and image-space information, while explicit depth is not essential in the evaluated setting.
- Learned policies sacrifice some nominal performance but show greater relative robustness to sensing and control perturbations than tuned classical visual-servo controllers.
- Visual Curriculum PPO has the smallest reward degradation under the hard helical trajectory, despite training only with static targets.