Source-linked AI summary
Deep Neural Network for Real-Time Autonomous Indoor Navigation
Dong Ki Kim, Tsuhan Chen
TL;DR
Indoor MAV navigation is challenging because GPS precision is limited indoors and carrying heavy or power-consuming sensors is difficult. The paper presents a single-camera ConvNet system that imitates an expert pilot while navigating to find a book bag; real-time tests report 70–80% success in Test Loc 1–4, and the authors identify extending experiments to stairways as future work.
Problem
Indoor navigation is challenging because GPS has limited precision indoors and MAVs cannot readily carry heavy or power-consuming sensors such as range finders.
Source: §1 · p. 1Method
A ConvNet trained on a custom dataset uses a single camera’s visual input to imitate an expert pilot’s flight-command choices.
Source: §1 · pp. 1–2Results
70–80% success rate in Test Loc 1–4, including similar performance in unseen Test Loc 3 and 4.
Source: §5 · p. 8Takeaways & Limitations
The system is presented as practical because it uses one camera and avoids 3-D map reconstruction, and the authors report real-time indoor navigation and target finding.
Source: §1 · p. 2, §7 · p. 12The experiments use constant-height control at 1 meter, which the authors describe as generally sufficient except for stairways.
Source: §4.2 · p. 5
Abstract
from arXiv · showhide
Autonomous indoor navigation of Micro Aerial Vehicles (MAVs) possesses many challenges. One main reason is that GPS has limited precision in indoor environments. The additional fact that MAVs are not able to carry heavy weight or power consuming sensors, such as range finders, makes indoor autonomous navigation a challenging task. In this paper, we propose a practical system in which a quadcopter autonomously navigates indoors and finds a specific target, i.e., a book bag, by using a single camera. A deep learning model, Convolutional Neural Network (ConvNet), is used to learn a controller strategy that mimics an expert pilot's choice of action. We show our system's performance through real-time experiments in diverse indoor locations. To understand more about our trained network, we use several visualization techniques.
1. Introduction
Indoor navigation is difficult because GPS is imprecise indoors and common sensing approaches can be impractical or unreliable for MAVs. The paper presents a single-camera system that learns expert flight commands, finds a book bag, and is examined through real-time tests and network visualizations.
1. Introduction
- Indoor navigation is challenging because GPS has limited precision indoors, while SLAM and stereo vision have computational, feature-poor-scene, or hardware constraints.SLAM’s 3-D mapping is computationally heavy, and stereo depth can perform poorly in texture-less regions or under specular reflections.Source: §1 · p. 1
- The system uses a single camera and a ConvNet trained to mimic an expert pilot’s actions, achieving a 70–80% success rate in real-time target-finding navigation.It navigates while seeking a book bag; the reported rate is from real-time test experiments.Source: §1 · pp. 1–2
- The approach is presented as computationally less expensive, requiring only a single camera and no 3-D map reconstruction.The authors also state that it does not require high-resolution cameras and can prevent wall collisions.Source: §1 · p. 2
- The paper contributes a custom dataset from seven visually distinct indoor locations and visualizations of the trained model’s learned features and representations.The authors describe the dataset’s diversity as potentially useful for other indoor research.Source: §1 · p. 2
2. Related Works
Prior indoor-flight approaches use range sensing, SLAM, stereo vision, vanishing points, or learned control policies, each with specific constraints. This work is framed as monocular visual imitation learning that also performs target finding and wall avoidance.
Range Sensor
- Range-sensor systems can support autonomous flight, but the sensors are described as impractical for many public quadcopters because of weight and power demands.The paper’s approach instead uses the built-in monocular camera.Source: §2 · p. 3
SLAM
- SLAM’s 3-D reconstruction is computationally expensive and can delay perception-to-action, while sparse trackable features in indoor walls can reduce accuracy.The authors say their system does not perform path planning and reacts to the currently faced situation.Source: §2 · p. 3
Stereo Vision
- Stereo depth estimation can fail in texture-less regions where corresponding features are hard to match, and stereo cameras are uncommon on public quadcopters.The paper states its system performs robustly in texture-less environments.Source: §2 · p. 3
Other Approach
- Vanishing-point navigation may use an added short-range sensor for collision avoidance, while prior learned controllers imitate human driving or pilot actions; this work uses ConvNets for visual flight and target finding.The authors also distinguish their approach by reporting monocular-camera collision avoidance without the additional range sensor.Source: §2 · p. 3
3. Overview
The system trains a visual classifier to imitate an expert pilot’s flight commands. During flight, it processes new camera frames, hovers when confidence is low, lands when it identifies the target, and otherwise issues a movement command.
3. Overview
- A supervised ConvNet classifier is trained from expert demonstrations to map real-time MAV images to the expert’s flight commands.The network is fine-tuned from pre-trained CaffeNet using a custom dataset; its architecture has five convolutional and three fully connected layers.Source: §3 · p. 4
- The flight loop obtains a new frame and predicts a command; low confidence triggers hovering and another prediction cycle.When the predicted class is the target, the drone stops and lands; otherwise it returns the corresponding movement command.Source: §3 · p. 4
- The classifier receives an RGB image and predicts one of six flight commands using a CaffeNet-like architecture with six output nodes.The commands are Move Forward, Move Right, Move Left, Spin Right, Spin Left, and Stop.Source: §3 · p. 4, Figure 2 · p. 5
4. Deep Neural Network for Autonomous Navigation
The system trains a visual flight-command classifier using expert demonstrations from varied indoor locations, then fine-tunes a pretrained ConvNet for six commands. Its dataset and training choices reflect a fixed-height indoor flight setting.
4.1. Hardware Platform
- The Parrot Bebop streams 640×368 images over WiFi to a host computer that runs the classifier.The drone has a forward-facing camera, an ultrasound sensor for ground altitude, and an onboard computer.Source: §4.1 · p. 5
4.2. Dataset
- The custom dataset covers seven distinct corridor or corner locations, with varied building structures, brightness, and objects.Public indoor-image datasets lacked ground-truth flight commands, motivating collection of expert-labeled demonstrations.Source: §4.2 · p. 5
- At a constant height of 1 meter, the quadcopter uses six commands, including Stop when it finds the book-bag target.Expert pilots demonstrated flights multiple times, including responses to possible wall collisions; Gaussian noise augmentation doubled the dataset size.Source: §4.2 · pp. 6–7
4.3. Fine-tuning
- Fine-tuning replaces CaffeNet’s fc8 layer with six output nodes and raises the new layer’s learning rate relative to the rest of the model.CaffeNet has five convolutional and three fully connected layers; mirror augmentation is disabled because flipping an image would not change its Move Right label.Source: §4.3 · pp. 7–8
5. Experiments
Real-time trials tested target finding and obstacle-free flight across five indoor locations, including three not represented in training. Success was 70–80% in four locations and 60% in the most visually distinct location.
5. Experiments
- Five test locations varied in objects, geometry, and lighting; Test Loc 3–5 were unseen during training, and fake objects were placed as distractors.Test Loc 4 had a narrow hallway, while Test Loc 5 had dim lighting and a largely glass wall.Source: §5 · p. 8, Figure 7 · p. 10
- The classifier achieved similar performance in unseen Test Loc 3 and 4 to seen environments, while the uniquely appearing Test Loc 5 had lower success.The authors describe Test Loc 5’s 60% success as comparable with the others.Source: §5 · p. 8
- 70–80% success in Test Loc 1–4 and 60% in Test Loc 5, using a trial criterion that required finding the correct target without any collision.The main result is summarized in Table 2.Source: §5 · p. 8, Table 2 · p. 9
6. Discussion
The discussion uses visualization to inspect learned representations and diagnose potential model problems. Class visualizations suggest distinct features for flight commands, while image-specific saliency maps highlight edges.
6.1. Class Model Visualization
- Class-model visualization optimizes a synthetic image to produce a high score for a selected flight-command class.The method uses back-propagated gradients with L2 decay and Gaussian blur, starting from a random image.Source: §6.1 · p. 10
- Class visualizations suggest distinct learned features, but Spin Right and Spin Left are less clear and may benefit from more training images.The Stop visualization resembles book-bag features; Figure 9 shows no well-visualized indoor environment for Spin Right.Source: §6.1 · p. 10, Figure 9 · p. 11
6.2. Image-Specific Class Saliency Visualization
- Image-specific saliency maps for 50 high-scoring images per command highlight common edges, suggesting edges affect classification performance.Brighter regions indicate parts of an input image with greater effect on a class score.Source: §6.2 · p. 10, Figure 8 · p. 12
7. Conclusion
The paper concludes that a single-camera deep-learning system can navigate indoors and find a target, with experiments across diverse indoor locations. It is computationally efficient because it does not reconstruct a 3-D map.
7. Conclusion
- The authors propose expanding the dataset and experiments to more diverse indoor environments, including stairways.They also report using visualization techniques to investigate the trained network.Source: §7 · p. 12