PaperlayerView exact revision on arXiv

Source-linked AI summary

ISB-NMPC: Interception-Oriented Spherical-Bearing Nonlinear Model Predictive Control for Quadrotors

Li Hong, Linkai Liu, Xiangyu Ren, Shuli Lv, Chen Min, Chunjiang Wu, Yujun Zhang, Dingchen Zeng, Quan Quan

arXiv:2610.34789v1cs.RO

TL;DR

Strapdown-camera interceptors must balance rapid target closing against maintaining visual contact as vehicle attitude also changes camera pointing. ISB-NMPC jointly predicts spherical bearings and quadrotor dynamics to coordinate closing and visibility, with a compensated bearing estimator supporting detection. In paired simulations it achieved 86.7% interception success and 99.11% mean target-visible time.

  • Problem

    A strapdown camera couples vehicle maneuvering with camera pointing, creating a conflict between rapid closing and maintaining target visibility.

    Source: §I · pp. 1–2

  • Method

    ISB-NMPC jointly predicts spherical bearings and quadrotor dynamics to optimize closing motion and visibility, with attitude- and delay-compensated bearing estimates and predicted ROIs supporting detection.

    Source: §I · p. 2

  • Results

    86.7% interception success and 99.11% mean target-visible time were achieved by ISB-NMPC across 30 simulation trials.

    Source: A. Simulation Validation and Comparison · p. 7, §VI · p. 7

  • Takeaways & Limitations

    The paired simulations show that closing speed should be coordinated with visibility rather than maximized alone.

    Source: §VI · pp. 7–8

    The nonconvex real-time optimization generally returns a local solution, and bounded FOV slack does not guarantee recursive feasibility.

    Source: 3) Discrete Optimal Control Problem · p. 6

Abstract

from arXiv · show

Quadrotor interceptors equipped with strapdown cameras offer a low-cost solution for physically countering unauthorized UAVs. However, this configuration inherently couples vehicle maneuvering and camera pointing, creating a conflict between rapid closing and maintaining target visibility. We present interception-oriented spherical-bearing nonlinear model predictive control (ISB-NMPC), which couples spherical-bearing prediction with nonlinear quadrotor dynamics to jointly optimize target-directed closing motion and visibility. An attitude-compensated extended Kalman filter on S^2 estimates target bearing motion, compensates for perception delay, and predicts target bearings to guide scale-adaptive detection. Across 30 paired simulation trials, ISB-NMPC achieves an interception success rate of 86.7%, the highest among the evaluated controllers, with a mean per-trial target-visible time of 99.11%. Outdoor flights demonstrate acquisition of a 30 cm target at approximately 150 m, with interceptor speed reaching 17 m/s in the long-range flight and approximately 14 m/s while pursuing a quadrotor-towed maneuvering target. The software and hardware are open source at https://github.com/HOLYNOTBAD/ISB-NMPC and https://github.com/HOLYNOTBAD/holydrone, respectively.

I. INTRODUCTION

Strapdown-camera interception couples rapid closing with target visibility because thrust-driven attitude changes also steer the camera. The paper proposes joint bearing-aware control and perception, then evaluates closing and visual retention in simulations and outdoor flights.

Motivation

  • Thrust-driven attitude changes rotate the strapdown camera, so target-directed acceleration can reduce FOV margin while recentering can restrict closing acceleration.Camera rotation, detection delay, target-scale variation, nonlinear dynamics, and actuator limits also affect visual feedback and control.Source: §I · p. 1

Related work

  • Existing work spans trajectory planning, visual servoing and LOS guidance, and perception-aware MPC; the paper frames rapid capture as requiring coordinated closing demand and visibility-preserving motion.Prior methods establish useful feedback structures, but the introduction identifies coordinating closing with vehicle motion as an additional requirement.Source: §I · pp. 1–2

Contributions

  • ISB-NMPC predicts SO(3)-lifted spherical bearings jointly with quadrotor dynamics, using LOS velocity tracking, bearing-dependent speed demand, and visual costs and constraints.The controller jointly optimizes desired closing speed and vehicle motion under actuator limits.Source: §I · p. 2
  • The attitude- and delay-compensated spherical-bearing estimator supplies current-time bearing feedback and absolute-time ROI predictions to support detection during attitude changes and target-scale variation.The front end draws on bearing estimation, small-object slicing, and delayed-feedback estimation.Source: §I · p. 2

Evaluation

  • The evaluation pairs controller comparisons across 30 simulation trials with outdoor flights against a quasi-static small target and a maneuvering target.The reported assessment considers controller performance and operation of the integrated perception-control system.Source: §I · p. 2

II. SYSTEM ARCHITECTURE AND PROBLEM FORMULATION

The onboard system links camera detections and ego-state to bearing estimation, interception control, and PX4 actuation. Its formulation uses camera-relative spherical bearings and a quadrotor model that treats commanded body rates as tracked inputs.

A. System Architecture

  • The onboard architecture combines a strapdown camera, PX4 ego-state, and a Jetson Orin NX; 30 Hz detections feed DC-S2EKF, and 50 Hz ISB-NMPC sends thrust and body-rate commands to PX4.The detector is scale-adaptive YOLOv5n, and the estimator provides current-time bearings and ROI predictions.Source: A. System Architecture · p. 2, Figure 3 · p. 2

B. Coordinate Frames and Quadrotor Dynamics

A. System Architecture

  • The onboard architecture figure integrates scale-adaptive detection, DC-S2EKF bearing estimation, ISB-NMPC, and PX4 actuation.Source: Figure 3 · p. 2

B. Coordinate Frames and Quadrotor Dynamics

  • The camera is rigidly mounted, and the model treats commanded body rates as directly tracked because the PX4 body-rate loop is faster than the NMPC update.Rotational rigid-body, motor, and propeller dynamics are excluded from the prediction model.Source: B. Coordinate Frames and Quadrotor… · p. 3

C. Visual Interception Problem

The interception problem chooses thrust and body rates from bearing feedback, with target-state estimates used when available, to approach an interception region while retaining the target in view. Because future target motion is unknown, the controller uses a finite-horizon surrogate rather than a guaranteed minimum-time solution.

Visibility

  • Visibility is constrained through the camera-frame bearing’s normalized image coordinates and bounds set by the usable camera field of view.The projection is defined for a target bearing in front of the camera.Source: C. Visual Interception Problem · p. 3

Closing reference

  • LOS alignment maximizes instantaneous closing at fixed geometry and target velocity, but does not establish minimum-time capture.Source: C. Visual Interception Problem · p. 3
  • In bearing-only operation, the nominal relative-velocity reference sets estimated target velocity to zero and aligns the reference with the LOS at the chosen closing-speed demand.The physical closing speed is positive when range decreases.Source: C. Visual Interception Problem · p. 3

Dynamic coupling

  • Closing acceleration is dynamically coupled to transverse motion: the bearing changes with transverse velocity, and increasing thrust improves closing acceleration only under a stated LOS-alignment condition.Source: C. Visual Interception Problem · p. 3

Interception problem

  • The controller must reach r(tf) ≤ rint rapidly while keeping the target within the camera FOV, using bearing feedback and any available target-state estimates.The target does not communicate its state; bearing-only prediction initializes nominal range and zero target velocity.Source: C. Visual Interception Problem · p. 3
  • The formulation is a finite-horizon surrogate, not a guaranteed minimum-time interception solution, because the non-cooperative target’s future motion is unknown a priori.Performance is assessed jointly by success rate, successful-trial interception time, and target-visible time.Source: C. Visual Interception Problem · p. 3

III. ONBOARD VISUAL PERCEPTION FRONT END

The visual front end converts timestamped imagery and egomotion into bearings and ROI predictions, while the DC-S2EKF compensates camera rotation and perception delay. It maintains unit-bearing estimates and propagates them to controller and image-acquisition times.

III. ONBOARD VISUAL PERCEPTION FRONT END

  • Timestamped images and egomotion become bounding boxes, current-time unit bearings, residual bearing motion, and future ROI predictions.The bounding-box center and camera intrinsics provide the image-time bearing.Source: §III · p. 3, A. Bearing Estimation, Temporal… · p. 3

A. Bearing Estimation, Temporal Filtering, and Delay Compensation

  • The DC-S2EKF compensates camera rotation before integrating residual motion, preserving unit bearings and using two-dimensional tangent innovations.Figure 5 schematically shows spherical back-projection, rotation compensation, propagation, and tangent-space correction.Source: A. Bearing Estimation, Temporal… · pp. 3–4, Figure 5 · p. 4
  • A constant-velocity model propagates bearing and residual-velocity coordinates, with covariance accounting for sample intervals, residual-acceleration variance, and IMU rotation uncertainty.The residual tangent velocity is distinct from the total spin-free angular velocity used in the control cost.Source: A. Bearing Estimation, Temporal… · p. 4
  • Bearing covariance derives from pixel uncertainty, while confidence, box area, boundary proximity, and temporal consistency adjust the measurement-noise floor.Joseph-form updates, basis transport, and the exponential map maintain consistency and bearing norm.Source: A. Bearing Estimation, Temporal… · p. 4
  • Detections update the filter at image time, then buffered attitude and IMU data propagate estimates to controller time or image-acquisition time for ROI prediction.Missing timestamps use calibrated delay; stale IMU data or prolonged detection loss triggers reacquisition.Source: A. Bearing Estimation, Temporal… · p. 4, Figure 6 · p. 4

B. Multi-Resolution Detection and ROI-Based Tracking

Detection switches between reacquisition and tracking modes using target-scale and confidence thresholds, with predictions bridging brief dropouts. The controller uses compensated bearings in a short-horizon optimization, while an SO(3) lift represents bearing direction separately from range.

B. Multi-Resolution Detection and ROI-Based Tracking

  • The detector switches among tiled high-resolution reacquisition, prediction-centered tracking, and downsampled full-frame tracking using hysteretic size and confidence thresholds.Valid predictions bridge brief dropouts, while repeated loss triggers reacquisition.Source: B. Multi-Resolution Detection and… · pp. 4–5

IV. INTERCEPTION-ORIENTED SPHERICAL-BEARING NMPC

  • At each update, the controller schedules closing speed from current LOS error, solves a constrained short-horizon problem, and applies only the first thrust and body-rate command.Its inputs include current ego-state, compensated camera-frame bearing, and available target-motion estimates.Source: §IV · p. 5

A. Spherical-Bearing Model and Prediction Dynamics

2) Spherical-Bearing Kinematics:

The spherical-bearing model propagates the target direction while accounting for relative translation and camera rotation. The NMPC state and controls combine vehicle motion with bearing, range, desired closing speed, and relaxable field-of-view bounds.

2) Spherical-Bearing Kinematics:

  • Bearing propagation accounts for relative target translation and camera rotation, while only transverse relative velocity changes the bearing.The model uses a spin-free angular velocity for the bearing dynamics.Source: 2) Spherical-Bearing Kinematics · p. 5
  • Quaternion kinematics propagate the SO(3)-lifted visual feature while preserving the quaternion unit-norm constraint.The lifted bearing is represented as nt = R(cf)e3.Source: 2) Spherical-Bearing Kinematics · p. 5

3) NMPC State and Control Input:

  • The prediction state combines vehicle motion, visual feature, and interception variables; desired closing speed s is distinct from physical closing speed −˙r.The FOV slack accumulators are solver decision inputs, not physical states.Source: 3) NMPC State and Control Input · p. 5
  • Controls comprise thrust, body rates, a desired-speed drive, and horizontal and vertical FOV-bound relaxations.The slack accumulators integrate their respective relaxation inputs.Source: 3) NMPC State and Control Input · p. 5

4) Prediction Dynamics:

The prediction dynamics couple vehicle attitude and motion with bearing, range, and desired closing speed, neglecting the small camera–body translational lever arm. The objective combines LOS-directed approach, image-plane visibility, terminal cross-track, and actuation costs.

4) Prediction Dynamics:

  • The prediction model uses calibrated camera rotation and relative velocities, evolves range and desired closing speed, and propagates world-frame bearing with predicted attitude and visual feature.The differential kinematics neglect the small camera–body translational lever arm; only the objective reference bearing is frozen.Source: 4) Prediction Dynamics · p. 5

1) Objective:

  • The interception objective tracks LOS-aligned closing motion, schedules desired speed from current LOS errors, and lets optimization balance approach demand against visual and actuation requirements.Using current bearing for scheduling prevents predicted errors from reducing the optimizer’s own speed reference.Source: 1) Objective · p. 5
  • The visual objective penalizes normalized image-plane target offset and bearing motion to account for camera-induced displacement during acceleration.The image projection is regularized, and its offset is measured relative to the image center.Source: 1) Objective · pp. 5–6
  • The terminal cross-track penalty discourages predicted endpoint deviation from the current LOS, while regularization penalizes aggressive actuation and closing-speed changes.The regularizer penalizes thrust deviation from gravity, body rates, closing-speed rate, and both FOV slacks.Source: 1) Objective · p. 6

2) Constraints:

Actuator bounds and soft field-of-view constraints encode visibility requirements alongside range and speed bounds.

2) Constraints:

  • The constraints combine actuator bounds and soft FOV constraints with range and speed bounds.The passage states that these requirements supplement the input set.Source: 2) Constraints · p. 6
  • Normalized image bounds πmin and πmax define the camera-FOV limits.A bounded, nonnegative slack vector softens the FOV constraint, and a large running penalty discourages violations.Source: 2) Constraints · p. 6
  • A lower bound on nz keeps the predicted target in front of the camera, while input bounds impose |us| ≤ aℓ,max.The latter follows from ˙s = us.Source: 2) Constraints · p. 6

3) Discrete Optimal Control Problem:

The controller solves a finite-horizon problem using a discretized prediction model and running and terminal costs. Under stated feasibility assumptions, a global minimizer exists, but the real-time solver generally returns a local solution.

3) Discrete Optimal Control Problem:

  • At each control update, a finite-horizon problem uses the discretized continuous-time prediction model F.The prediction discretization interval is ∆t.Source: 3) Discrete Optimal Control Problem · p. 6
  • The objective sums interception, visibility, and regularization costs across the horizon, together with a terminal cost.The supplied expression identifies the running terms as Lint,i, Lvis,i, and Lreg,i, plus LN.Source: 3) Discrete Optimal Control Problem · p. 6
  • A global minimizer exists under the stated bounded-input, nonzero-projection-denominator, and feasible-sequence assumptions, but the nonconvex real-time solver generally returns a local solution.Bounded FOV slack does not guarantee recursive feasibility; that would require a compatible invariant terminal set and controller.Source: 3) Discrete Optimal Control Problem · p. 6

C. Real-Time Implementation

ISB-NMPC is evaluated in paired simulations and outdoor flights, with simulations isolating controller performance using ground-truth target inputs. It runs at 50 Hz with typical onboard solve time of about 10 ms, and leads the evaluated controllers in simulation success and target visibility.

C. Real-Time Implementation

  • About 10 ms typical onboard solve time fits within the 20 ms control period; persistent solver failure resets the warm start and commands level hover.The controller runs at 50 Hz with a 0.5 s horizon; the solve time is typical, not a worst-case guarantee.Source: C. Real-Time Implementation · p. 6

V. SIMULATION AND EXPERIMENTAL VALIDATION

  • Simulations use ground-truth target inputs to isolate controller performance, while outdoor flights evaluate the integrated bearing-only system.Thirty paired trials use identical platform settings and random seeds, with a three-dimensional random-walk target bounded by 5 m/s and 1.5 m/s2.Source: §V · p. 6, A. Simulation Validation and Comparison · p. 6

A. Simulation Validation and Comparison

  • 86.7% success and 99.11% mean target-visible time; ISB-NMPC was faster than PS-LOS and backstepping among successful trials, while LOS-IBVS closed faster but succeeded twice and often lost visibility.In the paired seed shown in Fig. 8, ISB-NMPC was the only successful controller; its four failures coincided with sudden lateral target maneuvers.Source: Table I · p. 6, A. Simulation Validation and Comparison · p. 7
  • Closing speed must be assessed jointly with interception success and target visibility.The comparison uses shared target states, trajectories, bearings, and initial conditions across ISB-NMPC, PS-LOS, LOS-IBVS, and backstepping.Source: A. Simulation Validation and Comparison · pp. 6–7

B. Experimental Setup and Flight Validation

Paired simulations favor ISB-NMPC on interception success and target-visible time, while outdoor flights demonstrate long-range acquisition and pursuit of a maneuvering target. The flight evidence is complementary to the paired comparison, and its bearing-estimate agreement is qualitative.

1) Experimental Platform:

  • Bearing-only flight prediction used zero target velocity and a nominal range, with a separate quadrotor towing the maneuvering target.The simulations used ground-truth target inputs to isolate controller behavior, whereas outdoor flights demonstrated the integrated perception–control pipeline.Source: 1) Experimental Platform · p. 7, §VI · p. 8
  • The long-range flight progressed from REACQUIRE-HR through TRACK-HR to TRACK-LR as apparent target size increased, while speed reached 17 m/s.Source: 2) Experiment 1: Long-Range… · p. 7
  • During pursuit of an undulating target traveling approximately 5 m/s, filtered bearings followed the offline-detection trend and supplied estimates between detections.Agreement was qualitative because the offline detections were not independent ground truth; Figure 9 compares image coordinates u and v and enlarges highlighted intervals.Source: 3) Experiment 2: Pursuit–Evasion… · p. 7, Figure 9 · p. 7
  • Speed reached approximately 14 m/s within 4 s and was largely sustained; visual cost dominated near the image boundary, while LOS-alignment cost grew near encounter.These observations are consistent with modeled coupling, but are not a causal test or proof of constraint satisfaction.Source: 3) Experiment 2: Pursuit–Evasion… · p. 7, Figure 10 · p. 8

VI. DISCUSSION AND CONCLUSION

  • ISB-NMPC achieved 86.7% interception success and 99.11% mean target-visible time, while LOS-IBVS was faster when successful but succeeded only twice.Jointly optimizing closing speed and vehicle motion provides a way to allocate shared attitude authority between approach and visual retention; the comparison does not isolate individual cost terms.Source: §VI · pp. 7–8
  • Nominal-range, zero-velocity flight prediction can be limited during strong maneuvers; arbitrary-motion capture and recursive feasibility are not guaranteed.The four simulation failures and long occlusions motivate better lateral-motion prediction and active-constraint analysis.Source: §VI · p. 8
  • The evaluated scenarios support ISB-NMPC’s utility, but prediction uncertainty and sustained visibility remain deployment boundaries.Long-range acquisition and maneuvering-target pursuit complement rather than replace the paired comparison.Source: §VI · p. 8
Loading 2610.34789v1…