Source-linked AI summary
Aggressive Quadrotor Flight through Narrow Gaps with Onboard Sensing and Computing using Active Vision
Davide Falanga, Elias Mueggler, Matthias Faessler, Davide Scaramuzza
TL;DR
The paper addresses autonomous aggressive quadrotor flight through narrow gaps when gap pose must be estimated onboard rather than supplied externally. It fuses forward-camera gap detection with an IMU and generates dynamically and geometrically constrained trajectories that support active vision and replanning. The system was demonstrated in real experiments, achieving an 80% success rate across the reported trials.
Problem
Autonomous flight through narrow gaps is challenging because pose uncertainty grows quadratically with distance and the quadrotor must actively orient toward the gap for state estimation.
Method
The system fuses gap detection from a single onboard forward-facing camera with an IMU and plans trajectories satisfying perception, geometric, and dynamic constraints.
Results
80% success rate was achieved across 35 experiments involving gaps with roll angles from 0° to 45° and pitch angles from 0° to 30°.
Takeaways & Limitations
The approach demonstrated autonomous aggressive flight through narrow gaps using only onboard sensing and computing without prior knowledge of the gap pose.
Abstract
from arXiv · showhide
We address one of the main challenges towards autonomous quadrotor flight in complex environments, which is flight through narrow gaps. While previous works relied on off-board localization systems or on accurate prior knowledge of the gap position and orientation, we rely solely on onboard sensing and computing and estimate the full state by fusing gap detection from a single onboard camera with an IMU. This problem is challenging for two reasons: (i) the quadrotor pose uncertainty with respect to the gap increases quadratically with the distance from the gap; (ii) the quadrotor has to actively control its orientation towards the gap to enable state estimation (i.e., active vision). We solve this problem by generating a trajectory that considers geometric, dynamic, and perception constraints: during the approach maneuver, the quadrotor always faces the gap to allow state estimation, while respecting the vehicle dynamics; during the traverse through the gap, the distance of the quadrotor to the edges of the gap is maximized. Furthermore, we replan the trajectory during its execution to cope with the varying uncertainty of the state estimate. We successfully evaluate and demonstrate the proposed approach in many real experiments. To the best of our knowledge, this is the first work that addresses and achieves autonomous, aggressive flight through narrow gaps using only onboard sensing and computing and without prior knowledge of the pose of the gap.
I. INTRODUCTION
Autonomous flight through narrow, arbitrarily oriented gaps requires aggressive trajectory execution, precise state estimation, and onboard perception tightly coupled to control. This work targets those constraints without relying on motion capture, external computing, or prior gap pose knowledge.
- Narrow, arbitrarily oriented gaps require precise trajectory following and aggressive motion to align the quadrotor with the gap.
- Previous approaches relied on external motion capture, accurate prior gap pose knowledge, or external trajectory-generation computers.
- The proposed system fuses gap detection from a forward-facing camera with an IMU to estimate the quadrotor state relative to the gap.
- Active vision requires trajectories that keep the gap visible, while replanning addresses pose uncertainty that increases quadratically with distance.
- During traversal, the vehicle must maximize distance from the gap edges without visual feedback when the gap is very close.
- The evaluation uses a black-and-white rectangular gap pattern, although the trajectory-generation approach is independent of the detection algorithm.
B. Contributions
The paper combines onboard visual-inertial sensing with trajectory generation that supports perception and respects geometric and dynamic constraints, without prior gap pose knowledge or iterative learning.
- The method relies solely on onboard visual-inertial sensors and computing.
- Trajectory generation facilitates perception while satisfying geometric and dynamic constraints.
- The approach does not require iterative learning or prior knowledge of the gap position and orientation in the world frame.
A. Traverse Trajectory
The traverse trajectory is designed in a plane orthogonal to the gap, passes through its center, and maximizes clearance while remaining dynamically feasible. A closed-form ballistic trajectory then enables onboard execution without visual feedback.
- The traverse trajectory lies in a plane orthogonal to the gap and intersects its center to reduce collision risk.
- The plane is spanned by e1 and e2, with e2 orthogonal to the gap and e1 = e2 × e3.
- Gravity is compensated along the plane normal so the vehicle motion can be described in two dimensions within the plane.
- The traverse motion follows a second-order polynomial and is ballistic, combining uniformly accelerated and uniform-velocity motion when gΠ,2 = 0.
- Parameters l and d determine the initial position, velocity, and time tc needed to reach the gap center, and are selected through optimization.
- The trajectory can be tracked without visual feedback because traversal is short and uses constant thrust with zero angular velocities.
B. Optimization of the Traverse Trajectory
Traverse optimization reduces sensitivity to initial-condition errors by shortening flight time while enforcing velocity and clearance constraints, and it runs onboard in tens of milliseconds.
- Initial-condition errors propagate during traversal, so reducing traverse duration is the viable way to reduce collision risk.
- The optimization adjusts l to reduce flight time while accounting for the resulting initial velocity and minimum-distance constraints.
- The nonlinear optimization is solved with Sequential Quadratic Programming and runs onboard in a few tens of milliseconds.
C. Approach Trajectory
The approach trajectory is generated to match the traverse trajectory’s initial conditions while maintaining robust gap visibility. Yaw is chosen so the forward camera faces the gap as closely as vehicle constraints allow.
- The approach trajectory connects a suitable start position to the known initial position, velocity, and acceleration of the traverse trajectory.
- Candidate trajectories are evaluated for whether they keep the gap in the forward-facing camera’s field of view.
- The desired camera orientation minimizes the angle between the camera-to-gap vector d and optical axis r3 subject to the quadrotor’s vertical-axis constraint.
- The constraint fixes the angle between the body vertical axis zb and optical axis r3 according to the camera mounting; for the forward-facing setup, k = 0.
- The optimal optical axis lies in the plane spanned by d and zb, and the quadrotor yaw is computed to align the actual axis r3 with it.
E. Selection of the Approach Trajectory to Execute
The system selects among candidate approach trajectories using a cost that balances image-centered gap viewing against starting distance. Normalization constants make these terms dimensionally compatible.
- Candidate trajectories differ in start position and execution time, and the selected trajectory provides the most reliable state estimate relative to the gap.
- The cost includes the RMS angle θrms evaluated over every sample along each candidate trajectory.
- The cost also includes the straight-line distance d0 from the gap at the start of the approach.
- Normalization constants make the angle and distance terms dimensionless so they can be summed.
- The resulting trajectory keeps the gap center close to the image center while preventing the vehicle from starting too far away.
F. Recovery after the Gap
After traversal, localization with respect to the gap is lost, so the quadrotor must recover a vision-based state estimate before hovering safely.
- Traversal leaves the quadrotor without a state estimate relative to the gap.
- The recovery system stabilizes the vehicle after the aggressive maneuver, enabling fixed-position hovering without collision.
III. STATE ESTIMATION
The onboard state-estimation pipeline detects a known rectangular gap from a forward-facing camera, estimates its pose, and fuses that pose with IMU measurements.
- The trajectory-generation method is independent of the gap detector, while the implementation uses a black-and-white rectangular pattern to simplify detection.
- Detection applies Canny edges, undistortion, edge grouping, quadrangle searches, geometric checks, and sub-pixel corner refinement.
- Knowing the gap’s metric size enables 6-DOF pose estimation with an eight-point Perspective-n-Points problem and reprojection-error verification and refinement.
- The detector accelerates processing by searching around the last detection and falling back to the full image only when necessary.
- The estimated gap pose is fused with IMU measurements to provide a full state estimate.
A. Experimental Setup
The custom quadrotor uses onboard sensing and computing for aggressive gap-flight experiments across varied gap orientations. The experiments achieved 80% success, with 3 m/s speed and 400 °/s angular velocity at a 45° inclination.
- Platform: The custom 830 g quadrotor uses an onboard computer, PX4 autopilot, forward-facing fisheye camera, and IMU-based onboard processing.A distance sensor and downward-facing camera are reserved solely for recovery.
- Platform: The vehicle measures 55 × 12 cm, while the gap measures 80 × 28 cm, leaving only 12.5 cm and 8 cm center-gap tolerances.These clearances require centimeter-accurate traverse tracking to avoid collision.
- Trajectory parameters: The traverse trajectory uses v0,max = 3 m/s and dmin = 0.25 cm, while tuning keeps the gap detectable and near the image center.The normalization constants are manually tuned to balance reliable vision-based pose estimation with centered perception.
- Experiments: Across 35 experiments, gap roll ranged from 0° to 45° and pitch from 0° to 30°; at 45° roll, the quadrotor reached 3 m/s and 400 °/s.Experiments included multiple inclined-gap configurations.
- Results: The experiment success rate was 80%, where success required collision-free traversal followed by recovery and locking to a hover position.Failures were attributed to persistent loss of pose estimates during approach, causing traverse initialization errors and frame collisions.
V. DISCUSSION
The discussion reports accurate state tracking across varied gap orientations and fast onboard trajectory computation, while noting a feasibility-check limitation for short approach trajectories.
- State estimation: Figure 6 compares estimated and ground-truth position, velocity, and orientation for gap configurations of 20° roll, 45° roll, and 30° roll with 30° pitch.The approach runs from t = 0 to t0, traversal reaches the gap center at tc, and recovery begins at each plot’s final time.
- Replanning: The approach method can fail to verify trajectory feasibility, usually when the trajectory duration is short; then the system skips replanning and reuses the last available approach trajectory.This is an explicit boundary of the replanning procedure.
- Trajectory Computation Times: During execution-time replanning, each trajectory requires (0.018 ± 0.011) ms on average because the current vehicle state constrains initialization and no cost function is evaluated.This is faster than the full approach-planning computation.
C. Gap configuration
The system combines onboard camera–IMU sensing with perception-aware trajectory generation for autonomous narrow-gap flight. It was demonstrated in real-world experiments, while evaluations constrained gap orientations despite broader formulation feasibility.
- Gap configuration: The formulation supports feasible trajectories for any gap configuration, but experiments limit roll angles to 0°–45° and pitch angles to 0°–30°.Heavily pitched or extreme gaps can reduce visibility, increase motion blur, and demand angular velocities beyond the experimental platform’s capabilities.
- Gap configuration: The system safely passes through a narrow inclined gap using only onboard sensing and computing.
- Gap configuration: Full state estimation fuses gap detections from a forward-facing onboard camera with IMU readings.The fusion supports state estimation during the approach maneuver.
- Gap configuration: Trajectories facilitate state estimation by keeping the gap continuously in the onboard camera image.This couples perception and control to address varying uncertainty in vision-based state estimation.
- Gap configuration: The approach was successfully evaluated and demonstrated in many real-world experiments without prior knowledge of the gap pose.The authors characterize it as autonomous aggressive narrow-gap flight using only onboard sensing and computing.