Source-linked AI summary
Learning to Fly by Crashing
Dhiraj Gandhi, Lerrel Pinto, Abhinav Gupta
TL;DR
Indoor UAV navigation is difficult because imitation learning offers limited data, while simulation may not transfer reliably to real environments and perception systems add cost. The paper collects large-scale real crash data autonomously, combines negative collision examples with positive trajectory samples, and trains a simple policy. The resulting model navigates cluttered indoor environments, including settings with dynamic obstacles, and is comparable to human control on some environments.
Problem
Indoor UAV navigation lacks a clearly adequate data and representation strategy because imitation datasets are small, simulation transfer is uncertain, and sensor-based perception can be costly or error-prone.
Method
The paper autonomously collects 11,500 real crash trajectories and combines near-collision negative samples with far-from-object positive samples to train a binary navigation classifier.
Results
The learned policy navigates extremely cluttered indoor environments with dynamic obstacles, outperforms depth-based methods across tested environments, and is comparable to human control on some environments.
Takeaways & Limitations
Large-scale self-supervised crash data can support effective indoor UAV navigation without relying on depth prediction as an intermediate representation.
Abstract
from arXiv · showhide
How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. We show that this simple self-supervised model is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans. For supplementary video see: https://youtu.be/u151hJaGKUo
I. INTRODUCTION
The paper addresses indoor UAV navigation without costly intermediate perception systems or small imitation-learning datasets. It proposes learning from large-scale real crash data, combining negative and positive trajectory samples for effective navigation.
- I. INTRODUCTION: Indoor UAV navigation requires avoiding obstacles while perception errors and sensor costs make traditional two-step approaches infeasible.Traditional systems estimate maps or depth before issuing motor commands.
- I. INTRODUCTION: Imitation learning uses human demonstrations and corrective actions, but its training data remains small.The passage identifies the limited dataset size as a restriction of this approach.
- I. INTRODUCTION: The method targets indoor cluttered environments and avoids reliance on multiple sensors, explicit 3D reconstruction, or depth-based intermediate representations.These alternatives are described as costly, bulky, computationally expensive, or unreliable on some surfaces.
- I. INTRODUCTION: Simulation-based drone policies can transfer to real environments, but real-world transfer is questioned and may require additional training.The related-work passage contrasts simulation with real-world deployment.
- I. INTRODUCTION: The proposed approach collects 11,500 crashes and combines negative collision data with positive samples from the same trajectories to learn navigation.The crash dataset represents how the UAV should not fly, while the combined samples train the policy.
B. Deep learning for robots
Deep learning models require substantial data for generalization, motivating self-supervised collection with minimal human supervision. The paper applies this strategy to real-world indoor drone flight.
- B. Deep learning for robots: Deep learning requires high amounts of data to train generalizable models.This motivates collecting data at scale rather than relying on limited manual supervision.
- B. Deep learning for robots: Self-supervised learning enables large-scale data collection with minimal human supervision.The paper connects this approach to navigation and other robotic tasks.
- B. Deep learning for robots: The work reports more than 40 hours of real drone flight data and presents it as crucial for learning to fly.The data were collected in real-world indoor flight rather than only in simulation.
A. Hardware Specifications:
The data-collection system uses a low-cost drone to autonomously generate collision trajectories across diverse indoor environments. Random straight-line crashes, recovery, and iterative hard-negative collection produce a large real-flight dataset.
- A. Hardware Specifications:: The Parrot AR-Drone 2.0 uses its built-in camera, accelerometer, and safety hull without additional sensors in the flying space.The camera provides 720p images at 30 hz, while the hull permits collisions without rotor damage.
- 1) Collecting collision data:: The collection procedure samples naive trajectories that collide with varied objects, then learns a navigation policy from those trajectories.The policy can subsequently collect harder negatives when it fails.
- 1) Collecting collision data:: The drone randomly chooses a direction and follows a straight path until collision, then returns to its original position before repeating.PTAM helps compensate for inaccurate backtracking from the low-accuracy IMU.
- 1) Collecting collision data:: Each trajectory stores timestamped camera images, estimated trajectories, accelerometer data, and collision timing information.Accelerometer signals identify the exact collision moments and enable automatic segmentation.
- 1) Collecting collision data:: 11,500 collision trajectories were collected across 20 diverse indoor environments over 40 drone flying hours.Replaceable drone hulls make catastrophic failure inexpensive.
2) Data processing:
Collision trajectories are automatically segmented and divided into far-from-object positive samples and near-object negative samples. Intermediate images are discarded, producing binary classification labels.
- 2) Data processing:: Accelerometer data automatically segments each trajectory up to the collision time.This establishes the portion used for subsequent positive-negative partitioning.
- 2) Data processing:: Positive samples are far from the collision object, while negative samples are close to it.Positive images correspond to continuing forward; negative images correspond to the near-collision portion.
- 2) Data processing:: The first N+ timesteps are labeled positive and the last N− timesteps negative, while middle images are ignored.This heuristic segmentation converts the selected trajectory portions into binary classification data.
C. Learning Methodology
The methodology section introduces the learning procedure for the binary classification dataset collected from UAV collisions.
- The learning methodology is organized into the network architecture and test-time execution.
1) Network architecture:
The UAV policy uses an ImageNet-initialized AlexNet classifier to assess directional motion from image crops and convert those predictions into flight actions.
- Network architecture:: The network uses AlexNet with ImageNet-pretrained convolutional weights and a randomly initialized fully connected layer learned from collision data.The architecture is illustrated in Figure 4.
- Network architecture:: The binary classifier predicts whether the drone should move forward in a straight line from an input image.Its final layer is a binary softmax trained with negative log-likelihood.
- Test time execution:: At test time, the policy evaluates right, complete, and left image crops to predict probabilities for right, straight, and left motion.
- Test time execution:: If P(S) exceeds α, the drone moves forward with linear velocity β and angular velocity proportional to P(R) - P(L).
- Test time execution:: When P(S) is below α, the drone stops its forward motion and turns toward the direction with the higher predicted probability until the straight-motion condition is met.
IV. EXPERIMENTAL EVALUATION
The evaluation tests the method in indoor environments represented by floor plans, including corridors, transparent doors, and additional chair obstacles.
- The testing environments include NSH Entrance, Wean Hall, NSH 4th Floor, Glass Door, and Hallway.Hallway with Chairs uses the Hallway floorplan with chairs added as obstacles.
- The floor plans organize evaluation across distinct indoor layouts and obstacle configurations.The supplied figure identifies five floorplan categories and notes the additional Hallway with Chairs condition.
A. Baselines
The evaluation compares the proposed method with straight-line, monocular-depth, and human-controlled policies using distance and collision-free flight time.
- Baselines: The baselines comprise a Straight line policy, a depth prediction based policy, and a human controlled policy.
- Baselines: The oracle straight-line baseline selects the best direction but can still fail in narrow corridors because path errors compound into wall collisions.
- Evaluation metrics: Performance is evaluated with average distance and average time of flight without collisions, with runs terminated when the drone enters small loops.Quantitative results are presented in Table I.
- Depth prediction based policy:: The depth-based baseline predicts depth maps from monocular images and infers the movement direction from those maps.The cited depth models are often trained on around 220k indoor depth data.
3) Human policy:
The human policy baseline uses joystick commands from participants viewing only the drone’s monocular images. It is evaluated across diverse indoor environments, including transparent obstacles and narrow corridors.
- Human policy:: Participants control the drone with a joystick using only monocular images observed by the drone.Participants first fly a test trial to become familiar with the drone’s response.
- Human policy:: The evaluation covers six complex indoor environments with different starting positions and orientations.The environments are intended to represent challenges encountered in general-purpose indoor navigation.
- Human policy:: The Glass Door environment requires turning near transparent doors, where depth sensing is poor but visual learning methods can perform reasonably.
- Human policy:: The NSH 4th Floor environment tests navigation through a larger office space containing several turns in narrow corridors.
- Human policy:: The NSH Entrance environment requires maneuvering past glass walls into a cluttered atrium containing dining tables and chairs.
4) Wean Hall:
The Wean Hall and related cluttered-corridor settings test turning, long-term flight, and navigation through narrow gaps. Some environments were seen during training, while others were entirely novel.
- Wean Hall:: The corridor-turn setting connects short straight stretches at 90-degree angles, requiring the drone to turn at intersections to avoid collisions.
- Wean Hall:: The narrow 2m dead-end corridor tests centered flight, repeated turnarounds, and long-term autonomous operation.The drone must turn at the dead end and continue navigating after returning through the corridor.
- Wean Hall:: The chair-modified hallway tests whether the controller can identify gaps between chairs and walls, including gaps narrower than 1m.
- Wean Hall:: Two of the six evaluation environments were also present during training, while the remaining four were completely novel.
C. Results
The crash-trained self-supervised method outperforms depth prediction across tested environments and can match or exceed human control in difficult cluttered settings. Its collision experience helps with glass and untextured obstacles, while hallway flight lasts over three minutes.
- Results: The evaluation measures average distance and average collision-free flight time, terminating runs that enter small spinning loops.Quantitative results are reported in Table I, with trajectory comparisons shown in Figure 6.
- Results: The method performs much better than the depth baseline in every tested environment or setting.Human-controlled baselines are higher in most environments.
- Results: In ‘Hallway with Chairs’, the method surpasses human control because clutter makes narrow-space navigation difficult for participants.
- Results: Depth-based methods fail on glass walls, glass doors, and untextured flat walls, while collision examples may help the proposed method avoid glass obstacles.
- Results: Training on untextured environments helps the model identify corridor corners and avoid flying toward them.
- Results: More than 3 minutes of autonomous hallway navigation was achieved against a 5-minute flight battery life.
- Results: The conclusion attributes performance to more than 11,500 crash trajectories and emphasizes the importance of large-scale negative data for navigation.A standard deep network performs better than depth prediction methods and is comparable to human control in some environments.