PaperlayerView exact revision on arXiv

Source-linked AI summary

Autonomous fixed-wing interception using reinforcement learning

Daan van Hamersveld, Ewoud Smeur, Iurii Kriachko

arXiv:2610.37470v1cs.ROcs.AIcs.NEcs.SYeess.SY

TL;DR

Fixed-wing interceptors offer endurance for longer-range and higher-altitude targets, but terminal guidance remains mostly manual and real-world learning-based fixed-wing evidence is lacking. This paper trains a monocular-vision RL controller to command a fixed-wing interceptor, reporting better simulation performance than classical guidance under most conditions and successful real-flight interception of maneuvering targets.

  • Problem

    Terminal guidance for fixed-wing interceptors is mostly manual, and real-environment evidence for learning-based control with realistic perception is lacking.

    Source: §I · p. 1

  • Method

    A reinforcement-learning neural network maps filtered monocular-vision observations directly to actuator commands for a fixed-wing interceptor.

    Source: §I · p. 2

  • Results

    The controller outperformed classical guidance algorithms in simulation and achieved comparable performance in real-flight experiments, ultimately intercepting real fixed-wing MALE targets.

    Source: §VI · p. 18

  • Takeaways & Limitations

    The successful real-flight validation shows the potential of autonomous intelligent interceptors in a modern air-defense context.

    Source: §VI · p. 18

    The policy does not distinguish real targets from false positives or other detected objects, a noise source not simulated in training.

    Source: B. Perception · p. 17

Abstract

from arXiv · show

Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers show potential for automation. In this work, a fixed-wing interceptor is presented that is controlled by a neural network that maps filtered monocular vision observations directly to actuator commands. The policy was trained in a custom Python environment using reinforcement learning and proximal policy optimization. The policy was then deployed on a fixed-wing platform and tested in real interception scenarios. As a baseline, two classical guidance controllers were also developed and tested in simulation. The policy is able to intercept targets both in simulation and in real flight. In simulation, the policy outperforms the classical controllers under most conditions. The interceptor was ultimately successfully validated in real flight by intercepting maneuvering fixed-wing targets. This novel system shows the potential of artificial intelligence for autonomous control of fixed-wing interceptor drones.

I. INTRODUCTION

The paper addresses the gap in autonomous fixed-wing interception using limited onboard vision, where existing guidance struggles against maneuvering targets and real-world learning-based evidence is lacking. It presents a monocular-vision RL interceptor and describes the problem, aircraft, and perception setup.

I. INTRODUCTION

  • PN and PP guidance struggle against maneuvering targets, so current interceptors are mostly manually controlled during terminal guidance.Terminal guidance requires meter-level accuracy, motivating vision-based sensing.Source: §I · p. 1
  • Fixed-wing, vision-based learning control is presented for real-world MALE interception, addressing a literature gap where most studies use simulation or quadrotor models and omit realistic perception and transfer.The controller maps filtered visual and flight-controller information directly to actuator commands.Source: §I · p. 2

A. Problem formulation

  • Interception is defined as bringing the interceptor within a maximum radius of the target so an explosive device can neutralize it.The target and interceptor are modeled in North-East-Down three-dimensional space.Source: A. Problem formulation · p. 2

B. Dynamics model

  • A six-degree-of-freedom rigid-body fixed-wing dynamics model, based on prior literature [29], supports RL controller training in simulation.Source: B. Dynamics model · p. 2

A. System identification

  • Three system-identification flights and ordinary least-squares regression were used to estimate the aircraft’s aerodynamic coefficients.A magnetic angle encoder with a wind-vane measured angle of attack.Source: A. System identification · p. 2

B. Perception

  • A front-facing monocular camera feeds a target detector whose bounding box supplies center, dimensions, and confidence as measurements.The bounding-box center and width and height are measured in pixels; confidence indicates whether it encapsulates a drone.Source: B. Perception · p. 2, 1) Bounding box · p. 2

2) Pinhole model:

The section describes how the simulation maps target geometry into camera measurements and models imperfections in the resulting detections.

2) Pinhole model:

  • The pinhole model uses target width and its image width to estimate distance along the camera’s optical axis, then derives horizontal and vertical offsets from pixel coordinates.For fixed-wing targets, effective width is adjusted for viewing orientation because apparent width varies with the angle of view.Source: 2) Pinhole model · pp. 2–3

3) Camera noise and detection probability:

  • Camera-noise simulation includes pixel noise that grows with range, persistent bounding-box bias, and probabilistic detections that degrade with range and require the target to remain in the field of view.The bias models stretches of consistently misplaced boxes; detection outages are clustered, and simulated detections also lag the camera frame.Source: 3) Camera noise and detection… · p. 3

C. Kalman filtering

A constant-acceleration Kalman filter turns noisy bounding-box observations into a relative target-state estimate. The RL policy uses that estimate and aircraft state to command actuators, with rewards shaping interception, target visibility, and stable flight.

C. Kalman filtering

  • The same constant-acceleration Kalman filter, with identical parameters, estimates relative target state in simulation, training, and real flight.It is used because noisy bounding boxes do not provide a continuous relative-state estimate.Source: C. Kalman filtering · p. 3

D. RL formulation

  • The RL network maps filtered monocular-vision target estimates, filter confidence, and interceptor state directly to normalized throttle and control-surface commands, bypassing the low-level autopilot.Source: D. RL formulation · p. 3

1) Observable state:

  • The observation stacks three frames spaced 0.5 seconds apart and includes filtered state and uncertainty; training directly on raw bounding boxes and finite-difference velocity did not converge to successful interception.The stacked input spans a one-second window.Source: 1) Observable state · pp. 3–4

2) Reward function:

  • The reward combines distance progress and an interception bonus with rewards for keeping the target in view and penalties for target loss, aggressive body rates, abrupt actuator changes, and high angle of attack.The centering reward encourages a pursuit-like behavior, while the field-of-view penalty grows with time since the last detection.Source: 2) Reward function · p. 4

E. RL algorithm

The policy is trained with PPO in an adapted fixed-wing simulation using a staged curriculum. The real-flight testing process also uses virtual targets to separate perception challenges from control evaluation.

E. RL algorithm

  • PPO training uses a reduced clip range of 0.05 and an increased discount factor of 0.999 to limit policy updates and encourage longer-term planning.Source: E. RL algorithm · p. 4

F. Training environment

  • The custom training environment adapts a prior simulation framework for fixed-wing aircraft, models a single moving or accelerating target, and terminates episodes for interception, misses, aircraft limits, or timeout.Domain randomization increases from zero to 30 percent around nominal aircraft parameters during training.Source: F. Training environment · pp. 4–5

G. Curriculum learning

  • A ten-stage curriculum progresses from nearby, slow, non-accelerating targets when the policy reaches a 75 percent interception rate, transferring the best policy between stages.The final stage uses an impossible 101 percent interception target and continues until its timestep budget runs out or training crashes.Source: G. Curriculum learning · p. 5

H. Real interception test flights

  • Real-target testing initially faced perception difficulties: the detector was less robust than expected, particularly for tail engagements against unclear backgrounds and thin target profiles.The interceptor must be aligned with the target before the RL policy takes actuator control.Source: H. Real interception test flights · p. 5

1) Virtual stationary target:

  • A virtual stationary target is projected onto the camera image from predefined GNSS coordinates and aircraft pose, preserving the deployment pipeline while removing detection dropouts and simplifying alignment.Source: 1) Virtual stationary target · p. 5

2) Virtual moving target:

  • The virtual moving-target test projects a bounding box from a looping series of predefined coordinates, altitudes, and fixed ground speed for systematic evaluation.Source: 2) Virtual moving target · p. 5

3) Real fixed-wing target:

The section describes a maneuvering fixed-wing target, vision-based PN and PP baselines, and the cascaded autopilot used to realize their commands. The identified aerodynamic model follows out-of-sample trends, though residual autocorrelation remains for some coefficients.

3) Real fixed-wing target:

  • The moving target flew a 90-meter-altitude rectangular waypoint path and received pilot evasive stick inputs while airspeed remained under Ardupilot throttle control.The target pilot's inputs allowed it to depart from its waypoint mission until released.Source: 3) Real fixed-wing target · p. 5

I. Baseline classical controller

  • Vision-based PN and PP controllers served as baselines, and all three controllers shared the same perception pipeline so control differences could be compared.Source: §I · p. 5

1) Guidance laws:

  • PN commands acceleration perpendicular to line-of-sight rotation, while PP turns to reduce the angle between camera boresight and target line of sight.The PN implementation uses filtered LOS rotation and interceptor speed rather than a noisy relative-velocity estimate.Source: 1) Guidance laws · pp. 5–6

2) Cascaded autopilot:

  • The cascaded autopilot converts acceleration demands into specific force, then uses roll and pitch loops to align lift and regulate normal force.Inner body-rate loops support the outer loops; the derivative-based simulated specific-force estimate is low-pass filtered to limit noise.Source: 2) Cascaded autopilot · p. 6

A. System identification

  • Three system-identification flights estimated the aerodynamic derivatives; flights 2 and 3 used a wind vane for angle of attack, while flight 1 used the EKF estimate.The flights also differed in flight mode and axis excitation.Source: A. System identification · p. 6, Table I · p. 7
  • Flights 2 and 3 agree especially well; a flight-3 model follows flight-2 coefficient trends out of sample despite sizeable residuals.The flight-by-flight comparison and the ten-second flight-2 segment show agreement and trend tracking, respectively.Source: A. System identification · p. 6, Figure 3 · p. 7, Figure 4 · p. 7
  • Residuals are mostly uncorrelated over time, but CL and dependent CD show significant low-lag autocorrelation, indicating possible aerodynamic-model deficiency.The authors attribute this likely to the simplified thrust model and deem the fit adequate for training because domain randomization alleviates the need for high accuracy.Source: A. System identification · p. 7, Figure 5 · p. 7

B. Cascaded autopilot for baseline classical guidance

The cascaded autopilot tracks roll, pitch, sideslip, and normal-acceleration demands, while RL training progresses through a staged curriculum. In a simulated engagement, the learned policy intercepts a maneuvering target and maintains conservative angles of attack, though its control surfaces oscillate.

B. Cascaded autopilot for baseline classical guidance

  • The cascaded autopilot’s inner rate loops respond faster than its outer loops, while normal-acceleration tracking is slowest and least damped.The roll-rate loop reaches 90 percent demand in 0.10 s; the pitch-rate loop takes 0.06 s, while normal acceleration takes 1.0–1.5 s to settle.Source: Figure 6 · p. 8, B. Cascaded autopilot for baseline… · p. 8

C. RL learning

  • Across the 10-stage curriculum, initial interception rate rises from about 30 percent to nearly 95 percent, but stage 8 ends below 75 percent.Training stopped at stage 8 due to CPU constraints.Source: Figure 7 · p. 8, C. RL learning · p. 8

1) Single episode analysis:

  • The policy commands full throttle immediately and reduces it after prolonged target loss; this behavior emerged after reward changes favored keeping the target in view.The reward for quick interceptions was reduced while the reward for keeping the target inside the field of view was increased.Source: 1) Single episode analysis · pp. 8–9
  • Within 6 seconds, the policy intercepted a maneuvering target under randomized aircraft parameters, wind, gusts, and detection latency.The target moved at 10 m/s; aircraft parameters varied ±15 percent, with 0.06-second detection latency.Source: 1) Single episode analysis · p. 8, Figure 8 · p. 9
  • During the engagement, the Kalman Filter estimate tracked closing distance through periods without target detection, and angle of attack stayed within ±10-degree penalty thresholds.Airspeed ranged from 21 to 33 m/s, above stall speed at these low angles of attack.Source: 1) Single episode analysis · p. 8, Figure 9 · p. 9
  • The policy oscillates rudder deflection, producing yaw oscillations; earlier policies with less strict angle-of-attack penalties instead drove elevator oscillations to their limits.All three interceptors—the learned policy and two classical controllers—intercepted the target in the illustrated engagement.Source: 1) Single episode analysis · p. 9

2) Multiple episodes analysis:

The controller intercepted moving virtual targets in real flight, while simulations showed RL generally leading the classical controllers as conditions became more difficult. Performance remained sensitive to approach geometry and out-of-training-range estimates.

2) Multiple episodes analysis:

  • As curriculum difficulty increased, RL generally performed best, matched only by ideal PP and slightly exceeded by ideal PN in the final stages.Ideal RL degraded more slowly; real PP was less affected by camera noise than PN because its guidance kept the target in view.Source: Figure 16 · p. 10, 2) Multiple episodes analysis · p. 10
  • Across single-variable sweeps, acceleration and longer camera dropouts reduced all controllers’ interception rates, while PP and RL were mostly unaffected by wind and PP excelled at higher target speeds.The sweeps varied target speed, acceleration, wind, dropout duration, initial range, and approach geometry.Source: Figure 17 · p. 11, 2) Multiple episodes analysis · p. 11
  • Approach geometry produced the largest performance spread: RL and PP were weakest at 45°, while PN was least sensitive and outperformed RL at that angle.RL’s interception rate fell from 83.3 percent in a tail chase to 47.2 percent at 45°; PP fell to 29.2 percent at 45° and 34.0 percent head-on.Source: Figure 17 · p. 11, 2) Multiple episodes analysis · p. 11

1) Stationary virtual target:

  • Five of six stationary virtual-target flights passed through or shortly after pilot interruption; one clear failure may have involved a poor KF estimate.After passing the target, the controller aggressively tried to reacquire it in the field of view.Source: Figure 19 · p. 12, 1) Stationary virtual target · pp. 12–13

2) Multiple episodes analysis:

  • At 20 m/s target speed, RL was more affected than PN and PP; increasing initial range also eventually destabilized the KF range estimate beyond the training envelope.RL reached 82 percent interception at 150–200 m, but higher ranges caused erratic behavior; classical-controller episodes increasingly ended in target loss.Source: 2) Multiple episodes analysis · pp. 11–12

2) Moving virtual target:

1) Stationary virtual target:

  • In stationary-target flights, the policy oscillated the rudder, while telemetry was standardized to 40–50 Hz after earlier tests used 10–20 Hz.The authors report successful controller-wise sim-to-real transfer after matching deployment telemetry frequency to training.Source: Figure 20 · p. 12, 1) Stationary virtual target · p. 13

2) Moving virtual target:

  • Four of eight moving-target flights intercepted (distance ≤3 m), two missed, and two were aborted; the first achieved a 0.3 m closest point of approach.The moving-target trajectories are shown in Fig. 21.Source: Figure 21 · p. 13, 2) Moving virtual target · p. 13

3) Real target:

Six real engagements tested the controller against a maneuvering fixed-wing target, with three successful interceptions; simulation generally favored RL over classical guidance, while rudder behavior varied with perceived range. Real-flight proximity and state estimates were reconstructed from video and onboard logs, with important limits on the available target trajectory data.

3) Real target:

  • Three of six RL-controlled engagements successfully intercepted the maneuvering target, with pilots taking over to avoid collision.The target flew a rectangular pattern at 90 m altitude while its operator actively maneuvered to avoid interception.Source: 3) Real target · pp. 13–14
  • Video-based range estimates were 3.3 m in engagement 3 and yaw-corrected 6.0 m and 8.1 m in engagements 5 and 6; the latter CPAs may have been closer.Target logging failed 17 seconds after launch, so reconstructed CPAs relied partly on the planned route and an assumed 15 m/s target speed; actual CPA may have been smaller.Source: 3) Real target · pp. 14–15, Figure 23 · p. 14, Figure 24 · p. 15
  • Final onboard range estimates were 3, 4, and 6 m in engagements 3, 5, and 6, while engagement 5 reacquired the real target after a 1.7-second detection loss.In engagement 1, the filter continued to indicate closing during a 3.7-second loss even though video showed the detector had tracked something else; the authors relate this behavior to training with detection dropouts.Source: 3) Real target · p. 16

F. Rudder behavior in RL

  • Rudder oscillation amplitude correlated nearly linearly with perceived range and decreased as the target approached; artificial range estimates shifted amplitude regardless of actual range.The figure plots RMS rudder deflection against range, contrasting real range during stationary-target interceptions with artificial range estimates.Source: F. Rudder behavior in RL · p. 16, Figure 28 · p. 16

A. Performance

  • In simulation, RL outperformed classical controllers under almost all conditions, but PP reached a 91.8% interception rate versus RL’s 70.2% against 20 m/s non-maneuvering targets.This speed partly exceeded the training distribution, which reached only 12 m/s target speed per axis.Source: A. Performance · p. 16
  • Real-flight tests validated sim-to-real transfer, including successful interceptions of maneuvering fixed-wing targets, though operators repeatedly intervened to prevent collision.The policy also performed accurately against virtual targets across varied initial geometries, attitudes, and wind speeds.Source: A. Performance · p. 16

B. Perception

The work demonstrates fixed-wing, vision-based reinforcement-learning interception in real flight, while discussing perception and transfer constraints, controller behavior, and next steps. The policy’s conservative design favors robust transfer, with performance and autonomous target acquisition remaining open challenges.

B. Perception

  • Kalman filtering supplies relative-state estimates through prolonged detection loss, but does not reject false targets; range estimates rely on a nominal 1.7-m wingspan.During a real engagement, the policy intercepted despite at least 1.7 seconds without detections.Source: B. Perception · pp. 16–17

C. Flight model

  • Aircraft parameters were imperfectly identified, yet domain randomization and other techniques overcame simulation-to-flight gaps without requiring a high-fidelity simulator.Three identification flights produced no correlation scores above 0.8; lift and drag residuals were autocorrelated under OLS.Source: C. Flight model · p. 17

D. Training

  • After discovering that training made turns approximately twice as hard as realistic, the authors retained the existing policy for comparable experiments rather than retraining it.Simulation performance declined only slightly after the physics bug was fixed.Source: D. Training · p. 17

E. Transferability

  • Domain randomization of up to 30% supports robustness to flight-model gaps, while engagement time hindered transfer because real control authority is toggled by an operator.Engagement time quickly exceeded the training envelope and added no decision-making value.Source: E. Transferability · pp. 17–18

F. RL controller flight behavior

  • The policy’s persistent rudder oscillations remained unexplained; replacing RL rudder control with the baseline sideslip damper did not significantly reduce interception rate.The authors hypothesize that the behavior optimizes some reward, but did not confirm this.Source: F. RL controller flight behavior · p. 18

G. Future work

  • The conservative policy and strong incentives to keep targets visible improve robustness to camera noise but hinder interception performance; 30% randomization may also be more than needed.The authors propose investigating when transfer fails and where these design choices reach their limits.Source: G. Future work · p. 18
  • The proof of concept requires an initial target detection and lacks real-flight testing of classical baselines; target reacquisition, sensor fusion, and other airframes remain future directions.The authors note that fixed-wing results could extend to tailsitters or X-wing interceptors.Source: G. Future work · p. 18

VI. CONCLUSION

  • The end-to-end reinforcement-learning controller outperformed classical guidance algorithms in simulation and successfully intercepted real fixed-wing MALE targets in flight.The policy maps filtered monocular-vision observations directly to actuator commands and achieved sim-to-real transfer.Source: §VI · p. 18

APPENDIX A AERODYNAMICS MODEL

The appendix describes the rigid-body aircraft model used in Python simulation and training, including its state representation and translational dynamics.

1) State vector: 

  • The state vector represents position and velocity in NED, quaternion attitude, body angular rates, and normalized control-surface and throttle states with first-order actuator lag.Elevator, aileron, and rudder time constants are 0.05 s; throttle is 0.4 s.Source: 1) State vector:  · p. 19

2) Translational dynamics:

  • Translational dynamics combine aerodynamic lift, drag, and sideforce with propeller thrust and gravity to determine the aircraft’s total force.The aerodynamic coefficients depend on angle of attack, sideslip, control deflections, and induced drag.Source: 2) Translational dynamics · p. 19
  • The simulator propagates position and velocity in NED coordinates, transforming between inertial and body frames using the aircraft attitude.The model uses an ISA troposphere density model and specifies sea-level reference density as 1.225 kg m−3.Source: 2) Translational dynamics · p. 19

3) Rotational dynamics:

This block specifies the aircraft’s rotational and attitude equations, its V-tail actuator mapping, and the OLS regression used to estimate flight-model parameters.

3) Rotational dynamics:

  • Rotational dynamics use the aircraft inertia tensor and aerodynamic moments to describe changes in body angular rates.The moment model uses dimensionless aerodynamic coefficients for rolling, pitching, and yawing moments.Source: 3) Rotational dynamics · p. 19

4) Attitude kinematics: 

5) V-tail configuration:

  • The V-tail configuration maps normalized left and right ruddervator deflections through the aircraft’s dihedral angle.Source: 5) V-tail configuration · p. 19

APPENDIX B ORDINARY LEAST SQUARES REGRESSION

  • OLS estimates model parameters by minimizing squared residuals between measured dependent variables and a linear model of measured regressors.The resulting parameter vector supplies a least-squares fit to flight data used to create the RL training simulation.Source: Appendix B · p. 20

APPENDIX C KALMAN FILTER

The appendix describes a Kalman filter that turns target bounding-box detections into relative-state estimates for interception.

1) State vector and process noise model:

2) Measurement model:

  • Bounding-box coordinates and width are back-projected using a nominal 1.7-m wingspan, transformed into NED, and used as position measurements; velocity and acceleration are estimated over time.Detections with confidence c ≥ 0.3 are used, and measurement noise is modeled as zero-mean Gaussian noise.Source: 2) Measurement model · p. 20

3) Range-dependent measurement covariance:

The filter models range-dependent uncertainty, initializes unobserved motion states with broad priors, and propagates and corrects its state estimate as detections arrive.

3) Range-dependent measurement covariance:

  • Range uncertainty grows with distance, while bearing is more certain; the filter constructs covariance in the line-of-sight frame and maps it into NED.The three error sources are assumed uncorrelated, and bearing can be trusted from fewer measurements than range.Source: 3) Range-dependent measurement… · pp. 20–21

4) Initialisation:

  • The first detection above the confidence threshold seeds the filter with zero velocity and acceleration and large priors on unobserved states.The velocity and acceleration prior standard deviations are 5 m/s and 2 m/s2, respectively, so initial updates are measurement-dominated.Source: 4) Initialisation · p. 21

5) Prediction step:

  • At every control step, the filter propagates its state and covariance, whether or not a detection is available.Source: 5) Prediction step · p. 21

6) Correction step:

  • Accepted detections correct the predicted state using the measurement residual and Kalman gain, which updates all nine states through dynamics-built state covariances.Although the innovation contains only position, off-diagonal covariance terms make velocity and acceleration observable.Source: 6) Correction step · p. 21

7) Detection handling and coasting:

Without an accepted detection, the filter coasts on its motion model and increasing uncertainty; its outputs are transformed into the interceptor body frame for controller use. The appendices specify implementation and training parameters and describe policy performance in the old simulation environment.

7) Detection handling and coasting:

8) Outputs:

  • The filter expresses its estimated target state and covariance relative to the interceptor, then rotates them into the body frame to compute controller line-of-sight angles and rates.Source: 8) Outputs · p. 21

APPENDIX D ACTUATOR LAG MODEL

  • The actuator model represents each commanded deflection as a first-order lag, with 0.05 s time constants for flight-control surfaces and 0.40 s for the propeller.Source: Appendix D · p. 21

APPENDIX E CLASSICAL GUIDANCE CASCADED AUTOPILOT GAINS

  • The cascaded autopilot gains are tabulated, and integrator clamping and conditional accumulation protect its loops against windup during actuator saturation.The clamping limits vary by loop, and integration resumes when the error reverses.Source: Appendix E · p. 21, Table II · p. 21, 8) Outputs · p. 21

APPENDIX F REWARD FUNCTION COEFFICIENTS

APPENDIX H AERODYNAMIC AIRCRAFT COEFFICIENTS

APPENDIX I RL POLICY PERFORMANCE IN THE OLD TRAINING

  • The old, unfixed simulation environment’s policy-performance figure breaks down episode termination reasons across conditions, including conditions beyond the training envelope.Its comparison uses stage 8 nominal conditions; Table IV describes curriculum variables changed by stage, and the accompanying text defines aCV and CA.Source: Appendix I · p. 22, Figure 29 · p. 22, Table IV · p. 22
Loading 2610.37470v1…