PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty-Aware Vision-Based Autonomous Aerial Refueling

Minhyuk Jang, Thinh Nguyen, Vivek Khatana, Sandeep Banik, Naira Hovakimyan, Petros G. Voulgaris

arXiv:2610.07501v1eess.SYcs.SY

TL;DR

Vision-based PDR docking must align a probe with an uninstrumented moving drogue despite uncertainty in state estimation and closed-loop motion. The paper combines conformal estimator calibration and covariance-based operational uncertainty design to configure geometric and command margins for deterministic MPC. In simulation, the combined-margin controller achieved more centered crossings than nominal MPC.

  • Problem

    Vision-based docking methods estimate relative state, but generally do not quantify estimation and operational uncertainty in a way that informs the control authority needed for robust docking.

    Source: §I · p. 1

  • Method

    The architecture uses conformal calibration for fused-estimator error and offline covariance design for operational position-dispersion and corrective-command margins, which configure deterministic range-conditioned MPC.

    Source: §I · p. 1, §IV · p. 4, A. Conformal Prediction for Estimation… · p. 4

  • Results

    45/50 centered crossings with the combined-margin controller versus 34/50 with nominal MPC in high-fidelity simulation.

    Source: §VI · p. 8

  • Takeaways & Limitations

    The results provide a principled basis for incorporating uncertainty bounds and corresponding control-input margins while retaining deterministic MPC.

    Source: §VI · p. 8

Abstract

from arXiv · show

Probe-and-drogue refueling (PDR) requires accurate terminal alignment of a receiver probe with an uninstrumented, dynamically moving drogue despite uncertainty in both relative-state estimation and closed-loop motion. Existing vision-based docking approaches provide relative-state estimates, but generally do not quantify estimation and operational uncertainty in a way that informs the control authority required for robust docking performance. We present an uncertainty-aware vision-based PDR docking architecture that fuses keypoint-based drogue measurements, differential GPS, and avionics in an extended Kalman filter. Split conformal calibration characterizes near-contact error in the fused probe-to-drogue displacement estimate, while a covariance-based operational uncertainty quantification designs position-dispersion limits along with corrective-command variance budgets under modeled handoff variability and disturbances. The resulting uncertainty statistics define geometric and command margins for a range-conditioned deterministic predictive controller. In high-fidelity FlightGear/JSBSim simulations with four controller variants and 50 runs per variant, the combined-margin controller achieved 45/50 centered basket-plane crossings versus 34/50 for nominal MPC. Median planar crossing error decreased from 0.231 to 0.175~$\mathrm{m}$ and the empirical 90th percentile from 0.403 to 0.286~$\mathrm{m}$, while the roll-command RMS increased from 0.22$^\circ$ to 0.48$^\circ$.

I. INTRODUCTION

Terminal probe-and-drogue docking must align with a moving, uninstrumented drogue despite estimation error, disturbances, and limited receiver control authority. This work connects separate uncertainty characterizations to geometric and command margins, while evaluating terminal probe passage within a modeled capture region.

I. INTRODUCTION

  • The architecture separates fused-estimator error from operational uncertainty, using conformal calibration and offline covariance design to set geometric and coupled position/command margins for deterministic range-conditioned MPC.The controller supplies roll, pitch, and airspeed references to the existing AFCS.Source: §I · p. 1

II. RELATED WORK

  • Prior work addresses visual measurement, uncertain docking control, or calibrated and stochastic uncertainty separately; this approach connects uncertainty to geometric tolerance and corrective-command authority before online planning.The related-work discussion frames the approach as complementary to these existing methods.Source: §II · p. 2

A. Docking Task and Coordinate Frames

  • The study considers terminal docking after a DGPS-guided approach, evaluating probe passage through a modeled capture region at the first basket-plane crossing, not mechanical engagement or fuel transfer.The EKF fuses drogue vision, DGPS, and avionics; the MPC commands roll, pitch, and airspeed references within AFCS limits.Source: A. Docking Task and Coordinate Frames · pp. 2–3

B. Relative Dynamics and AFCS Model

The model represents relative drogue motion, receiver kinematics, and receiver response to roll, pitch, and airspeed commands. It includes disturbance and response residuals as stochastic inputs, under a steady, straight-and-level tanker assumption.

B. Relative Dynamics and AFCS Model

  • A double-integrator model represents drogue motion relative to the tanker, with lumped disturbance acceleration capturing hose dynamics, aerodynamic wake, and atmospheric turbulence.The tanker track is assumed steady, straight, and level.Source: B. Relative Dynamics and AFCS Model · p. 3
  • The receiver model includes attitude, angular rate, and translational velocity, while its AFCS accepts roll, pitch, and airspeed commands with first-order lag response.Heading is not independently commanded; the AFCS maintains coordinated flight.Source: B. Relative Dynamics and AFCS Model · p. 3
  • Drogue acceleration disturbances and AFCS roll–pitch response residuals enter the stochastic dynamics through a zero-mean white-noise process with diagonal spectral-density matrix.The diagonal matrix reflects an assumption that the five noise sources are mutually uncorrelated.Source: B. Relative Dynamics and AFCS Model · p. 3

C. Vision-Based Drogue Detector

The vision-based detector uses learned keypoints on the drogue rim to estimate a camera-frame drogue measurement, including approximate depth from the projected ring size.

C. Vision-Based Drogue Detector

  • YOLO11s-pose detects the drogue throat center and four evenly spaced rim points in the receiver-mounted monocular camera image.The detector uses the known drogue diameter and calibrated camera intrinsics.Source: C. Vision-Based Drogue Detector · p. 3
  • Approximate drogue depth is computed from the known diameter, focal length, and projected ring semimajor axis using Zd ≈ fD/(2a).The method uses the pinhole camera model.Source: C. Vision-Based Drogue Detector · p. 3
  • The resulting camera-frame measurement uses depth along the optical axis as its first component.Source: C. Vision-Based Drogue Detector · p. 3

D. Multisensor State Estimator

A continuous-discrete EKF combines vision, DGPS, and avionics measurements to estimate the system state and propagate its covariance between sensor updates.

D. Multisensor State Estimator

  • The estimator combines three sensor groups: DGPS relative displacement, vision-based drogue position, and avionics attitude and airspeed.The combined observations form a composite measurement vector and measurement function.Source: D. Multisensor State Estimator · p. 4
  • A continuous-discrete EKF propagates the state and covariance between sensor updates, assuming zero-mean measurement noise uncorrelated across sensor groups.Source: D. Multisensor State Estimator · p. 4

IV. UNCERTAINTY-AWARE DOCKING PLANNING

The docking planner separately characterizes estimator error and operational uncertainty, then uses their margins in the predictive controller. Split conformal calibration turns fused-estimator displacement errors into an estimator allowance, with coverage dependent on exchangeability.

IV. UNCERTAINTY-AWARE DOCKING PLANNING

A. Conformal Prediction for Estimation Uncertainty

  • Calibration residuals are scored with a covariance-normalized nonconformity measure to form a conformal prediction set.The empirical residual mean and covariance are used in the scoring procedure.Source: A. Conformal Prediction for Estimation… · p. 4
  • Marginal coverage is 1 − αc under exchangeability, while bounding the ellipsoidal set with a box is more conservative and preserves coverage.The text notes that exchangeability is often violated in practice and its coverage lower bound is not computable.Source: A. Conformal Prediction for Estimation… · p. 4

B. Operational Uncertainty Quantification

Operational uncertainty combines handoff-state variability with disturbances accumulated during docking. Offline covariance design propagates these sources to jointly constrain near-contact alignment dispersion and command variability.

B. Operational Uncertainty Quantification

  • The operational model combines handoff-state covariance with uncertainty accumulated from disturbances during docking.The handoff covariance captures pre-contact variability, while process-noise covariance captures uncertainty during the subsequent maneuver.Source: B. Operational Uncertainty… · p. 4
  • The covariance design constrains lateral and vertical alignment standard deviations near contact alongside stagewise control-input standard deviations.The design selects gains subject to the specified position-dispersion and command-dispersion limits.Source: B. Operational Uncertainty… · p. 5
  • Propagated alignment and command standard deviations become operational-uncertainty statistics for the online controller; the auxiliary gains are not applied online.The covariance propagation characterizes coupled state dispersion and control-input variability under the operational model.Source: B. Operational Uncertainty… · pp. 4–5

C. Uncertainty-Aware Docking MPC

The docking MPC uses the current estimated range to select an offline stage and prediction horizon. It tightens terminal capture and command constraints using estimator and operational-uncertainty margins, then sends the first optimized correction to the AFCS.

C. Uncertainty-Aware Docking MPC

  • The estimated range selects the offline stage and remaining prediction horizon for each MPC update.The reference is propagated under constant trim commands, and the MPC optimizes deviations from it.Source: C. Uncertainty-Aware Docking MPC · p. 5
  • The terminal capture region is reduced by both the estimator allowance and the operational alignment-dispersion margin.The nominal terminal trajectory must lie inside this reduced region to account for estimation error and modeled closed-loop dispersion.Source: C. Uncertainty-Aware Docking MPC · p. 5
  • Control-input bounds are tightened using stagewise command standard deviations to reserve authority for corrective maneuvers.The geometric and command margins are coupled: smaller near-contact position dispersion can require greater command variability and reduce available authority.Source: C. Uncertainty-Aware Docking MPC · p. 5
  • The online problem softens terminal-alignment and first-command slew constraints, and the AFCS receives the first optimized command correction.At each update, the applied input combines the reference command with the MPC correction.Source: C. Uncertainty-Aware Docking MPC · pp. 5–6

V. EXPERIMENTAL EVALUATION

The evaluation assesses estimator-error calibration, covariance-based operational-uncertainty statistics and feasibility, and closed-loop docking performance in a high-fidelity simulator.

V. EXPERIMENTAL EVALUATION

  • Estimator-error calibration and operational-uncertainty statistics are assessed before closed-loop docking performance.The evaluation also examines covariance-margin feasibility.Source: §V · p. 6
  • Closed-loop docking performance is evaluated in a high-fidelity simulator.The section reports a feasibility study alongside the simulator evaluation.Source: §V · p. 6
  • The evaluation examines covariance-margin feasibility as a distinct part of the study.The section also assesses the operational-uncertainty statistics obtained through covariance design.Source: §V · p. 6

A. Testbed and Implementation

The testbed models an F-14B receiver approaching an A330 MRTT tanker in FlightGear/JSBSim, with vision-based drogue keypoints and an avionics control stack.

A. Testbed and Implementation

  • FlightGear 2024.1 and JSBSim simulate an F-14B receiver approaching an A330 MRTT tanker at FL220 and 183 m/s in calm air.The tanker flies straight and level.Source: A. Testbed and Implementation · p. 6
  • A receiver-mounted camera uses a YOLO11s-pose model, finetuned on synthetic images, to predict five drogue keypoints.Rendering varies distance, lateral and vertical offsets, and receiver attitude; training augmentation includes flipping, translation, scaling, and color variation.Source: A. Testbed and Implementation · p. 6
  • The online quadratic program is solved with Clarabel 0.11, and experiments run on an Intel Core i7-14700F with 32 GB RAM and an NVIDIA RTX 4060 Ti.Detection uses Ultralytics 8.4 and PyTorch 2.5.Source: A. Testbed and Implementation · p. 6

B. Uncertainty Characterization

The section characterizes estimator and operational uncertainty, then evaluates how their margins affect docking accuracy, command activity, and computational feasibility.

B. Uncertainty Characterization

  • On one held-out closed-loop flight, 716/790 samples were contained, giving 90.6% empirical pointwise coverage against a 90% target; calibrated estimator margins were (0.082, 0.118, 0.093) m.Calibration used 481 samples from seven baseline-LQR runs; within-run dependence and differences from deployment mean exchangeability was not met.Source: B. Uncertainty Characterization · p. 6
  • The offline covariance design combined handoff, drogue-disturbance, and receiver-response uncertainty, yielding propagated lateral and vertical dispersions of approximately 0.12962 m each.Handoff position standard deviations were 0.55 m laterally and 0.23 m vertically; drogue acceleration intensities were estimated from 125 trials, and receiver residuals from 34 runs.Source: B. Uncertainty Characterization · p. 6
  • Combined estimator and covariance margins reduced terminal half-widths to approximately 0.0032 m laterally and 0.0290 m vertically, while requiring command-variance margins.Without margins, the corresponding half-widths were approximately 0.3889 m; the estimator-only and covariance-only variants yielded (0.2706, 0.2964) m and (0.1215, 0.1215) m, respectively.Source: B. Uncertainty Characterization · p. 6
  • At a 2.1° roll-command SD limit, a 3-cm nominal capture request was compatible but a 5-cm request was not; the 5-cm request became compatible at 2.4°.The allocation analysis varies position requirements against roll-command variability while retaining the other design inputs and constraints.Source: B. Uncertainty Characterization · pp. 6–7, Figure 3 · p. 7

C. Closed-Loop Docking Evaluation

  • Trials began 90–110 m behind the tanker and used handoff offsets up to 1.5 m laterally and 0.6 m vertically; four variants shared the same seed pairs and physical command limits.Handoff required remaining within ±2.0 m axially of the nominal station with closing speed above 0.1 m/s for 1 s.Source: C. Closed-Loop Docking Evaluation · p. 7
  • 45/50 centered crossings were achieved with both margins, and median crossing error fell from 0.231 m to 0.176 m versus nominal MPC.The combined-margin variant had the highest centered-crossing count and lowest median and P90 error among the four variants; its P90 was 0.286 m versus 0.304 m with covariance margins only.Source: Table II · p. 7, Figure 4 · p. 7, C. Closed-Loop Docking Evaluation · p. 7
  • Online MPC took a median of 24 ms and a maximum of 126 ms against a 250 ms target update period; all online QPs were optimal and logged terminal alignment slacks were zero.Offline covariance uncertainty quantification required 5.3 s on average.Source: C. Closed-Loop Docking Evaluation · p. 7
  • Both-margin trajectories converged toward the basket from varied handoff offsets, while the plotted covariance envelope represents the offline design, not a probabilistic tube for executed MPC trajectories.Roll activity was similar near handoff but remained higher with both margins near contact.Source: C. Closed-Loop Docking Evaluation · p. 7, Figure 5 · p. 8

VI. CONCLUSION AND FUTURE WORK

The paper concludes that separately characterizing estimator and operational uncertainty provides geometric and control-input margins for deterministic MPC. Simulations show more centered crossings than nominal MPC, while trajectory-wide calibration and adaptive control remain future work.

VI. CONCLUSION AND FUTURE WORK

  • The architecture translates estimation and operational uncertainty into geometric and control-input margins while retaining deterministic MPC.Conformal calibration supplies the estimator allowance; covariance uncertainty quantification supplies position dispersion and control-input margins for operational uncertainty.Source: §VI · p. 8
  • 45/50 centered crossings were achieved with the proposed controller, versus 34/50 with nominal MPC.Source: §VI · p. 8
  • Future work will extend conformal calibration to trajectory-wide uncertainty tubes for corridor planning and integrate L1 adaptive control for improved transient response and disturbance rejection.Source: §VI · p. 8
Loading 2610.07501v1…