Source-linked AI summary
Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness
Nikolai Smolyanskiy, Alexey Kamenev, Jeffrey Smith, Stan Birchfield
TL;DR
Forest-trail navigation requires MAVs to follow trails while remaining aware of people, pets, and previously unseen obstacles. The paper presents TrailNet and complementary vision modules running onboard in real time, and reports smoother, more robust control with autonomous navigation of a 1 km trail.
Problem
Autonomous low-flying MAV trail following requires both trail estimation and environmental awareness for avoiding branches, people, pets, and obstacles.
Method
The system uses TrailNet to estimate view orientation and lateral offset, plus DNN object detection and visual-odometry depth estimation for environmental awareness.
Results
The six-class TrailNet quickly returned the MAV to a straight course after disturbances, while only TrailNet autonomously completed the entire 250 m test trail.
Takeaways & Limitations
Label smoothing and entropy reward reduced overconfident predictions, enabling smoother trail-direction estimates and more stable control.
Abstract
from arXiv · showhide
We present a micro aerial vehicle (MAV) system, built with inexpensive off-the-shelf hardware, for autonomously following trails in unstructured, outdoor environments such as forests. The system introduces a deep neural network (DNN) called TrailNet for estimating the view orientation and lateral offset of the MAV with respect to the trail center. The DNN-based controller achieves stable flight without oscillations by avoiding overconfident behavior through a loss function that includes both label smoothing and entropy reward. In addition to the TrailNet DNN, the system also utilizes vision modules for environmental awareness, including another DNN for object detection and a visual odometry component for estimating depth for the purpose of low-level obstacle detection. All vision systems run in real time on board the MAV via a Jetson TX1. We provide details on the hardware and software used, as well as implementation details. We present experiments showing the ability of our system to navigate forest trails more robustly than previous techniques, including autonomous flights of 1 km.
I. INTRODUCTION
The paper presents an environmentally aware MAV system for autonomous forest-trail following, combining TrailNet with vision modules for detecting people, pets, and obstacles. Its contributions include real-time onboard operation, a six-category trail-estimation DNN, and autonomous navigation of a 1 km forest trail.
- Autonomous forest-trail following is challenging, while MAVs can avoid terrain limitations, fly faster, and switch trails through the forest.
- Environmental awareness is critical because low-flying MAVs may collide with branches, people, or pets on the trail.
- TrailNet estimates view orientation and lateral offset for continuous trail-following control, while additional vision modules detect people, pets, and obstacles.
- The complete vision system runs in real time onboard the MAV using an NVIDIA Jetson TX1 computer.
- The system autonomously navigates a narrow forested trail for 1 km, according to the authors’ stated contribution.
- TrailNet uses a less-confident classification scheme and additional categories for view orientation and lateral offset to support more stable control.
II. PREVIOUS WORK
Previous work spans visual navigation, mapping, obstacle avoidance, and deep reinforcement learning for MAVs. The paper is most closely related to trail-following DNNs that predict view orientation from camera data.
- Earlier MAV systems used structure-from-motion for depth estimation or RGBD cameras for mapping, localization, and planning.
- Deep reinforcement learning has been applied to visual recognition, continuous control, forest obstacle avoidance, indoor flight, and simulated quadrotor navigation.
- Giusti et al. trained a convolutional DNN to predict left, right, or straight view orientation from footage collected with a three-camera head-mounted rig.
- The paper adapts an image-based steering approach in which multiple camera views provide training signals for a single-camera vehicle controller.
III. SYSTEM DESCRIPTION
The system combines an off-the-shelf quadcopter, onboard computing, flight-control software, and three vision-processing nodes. Sensor and firmware adaptations target reliable operation in forest conditions.
- The platform uses a 3DR Iris+ quadcopter, Pixhawk autopilot, Jetson TX1 computer, and a forward-facing 720p camera operating at 30 fps.
- PX4FLOW optical flow with sonar and a Lidar Lite V3 provide positioning-related sensing and distance-to-ground information.
- The software architecture uses PX4 on Pixhawk, ROS and Ubuntu L4T on Jetson TX1, with communication through MavLink.
- Three vision nodes run TrailNet, YOLO-based object detection, and DSO visual odometry converted into a camera-centric depth map for obstacle avoidance.
- The controller converts trail predictions, detected obstacles or objects, and teleoperation commands into waypoints sent to PX4.
- A wide-angle PX4FLOW lens, firmware changes, and improved exposure control increased performance in low light and rotor-generated debris.
IV. VISION-BASED PROCESSING
This section introduces processing by three vision-based modules using imagery from the MAV’s forward-facing camera.
- The vision-processing section covers three modules operating on forward-facing camera data.
- The forward-facing camera is the stated visual input for the described vision-based processing.
- The section focuses on how the system processes camera data through its vision-based modules.
A. Estimating lateral offset and view orientation with DNN
TrailNet estimates both the MAV’s view orientation and lateral offset relative to the trail, using a six-category DNN and wide-baseline training data. A less-confident loss and probabilistic controller convert these estimates into stable flight commands.
- TrailNet uses a modified ResNet-18 with separate output heads for view orientation and lateral offset.The architecture removes batch normalization, replaces ReLUs with SReLU activations, and adds a double-headed fully connected output layer.
- Six categories represent three trail orientations and three lateral positions, allowing the MAV to distinguish being edge-aligned from facing straight.The authors state that these additional lateral-offset categories are essential for accurate state estimation and reliable trail following.
- A 1 m camera baseline enables the network to disambiguate lateral offset from view orientation.The center camera was positioned within 0.5 cm of the midpoint, and the wide separation differs from the earlier rig in.
- The training loss combines cross entropy, entropy reward, and a side-swap penalty to reduce overconfident predictions and support stable flight.Label smoothing and entropy reward are used for both heads, while the side-swap penalty applies only to the lateral-offset head.
- The controller mixes probabilistic orientation and lateral-offset predictions into a turning angle, then sends a waypoint to PX4 for execution.The turning-speed parameters are β1 = β2 = 10°, and waypoint distance affects vehicle speed.
B. Object detection with DNN
The object-detection module identifies people, cars, animals, and other objects, while hardware constraints determine which DNN architecture can run efficiently on the Jetson TX1.
- R-CNN variants were rejected because they could not run in real time on the Jetson TX1.
- SSD was unsuitable because its custom layers were unsupported by TensorRT, preventing efficient execution on the Jetson TX1.
- YOLO was fast and accurate but required modifications because some features were unsupported by TensorRT or FP16 mode.
- The selected detector was a modified YOLO retrained on PASCAL VOC for efficient FP16 TensorRT execution on the Jetson TX1.
C. Obstacle detection and avoidance with Monocular SLAM
The system uses monocular direct sparse odometry to estimate depth and camera pose for low-level obstacle detection, then aligns visual measurements with MAV state estimates to recover metric scale.
- Low-level obstacle detection must handle previously unseen static and dynamic obstacles, including trees, walls, people, animals, and vehicles.
- The system chooses monocular techniques to reduce complexity and support eventual miniaturization instead of traditional stereo matching.
- DSO produces semi-dense 3D maps, camera-pose estimates, and depth images at 30 Hz on the Jetson TX1.
- Inverse-depth measurements are converted into camera-frame 3D points and transformed into the MAV frame with metric scale.
- Procrustes analysis aligns paired DSO and MAV poses to estimate the rigid similarity transform between visual-odometry and MAV frames.
- Figure 6 contrasts the onboard camera image with DSO’s sparse depth map, where red denotes close points and blue denotes far points.
- Figure 7 reports mean and ±1σ across five trials while the aligned distance estimate settles to ground truth after several seconds.
A. DNN comparison
The authors compared several trail-detection architectures using offline classification and closed-loop autonomy, selecting TrailNet primarily for reliable long-distance autonomous flight.
- The comparison included ResNet-18, SqueezeNet, miniature AlexNet, Giusti et al.’s architecture, and TrailNet.
- The proposed architecture was selected primarily for reliable autonomous flight over long distances, alongside accuracy, computational efficiency, and power efficiency.
- All architectures produced six categories except Giusti et al.’s model, which produced three, and evaluation used the IDSIA trail dataset.
- Offline classification accuracy did not necessarily correlate with closed-loop autonomous navigation performance.
- VGGNet-16 was excluded because it was too large to fit on the Jetson TX1.
- Only TrailNet autonomously completed the entire 250 m test trail despite not achieving the highest offline classification accuracy.
B. Autonomous trail following
TrailNet was evaluated across forest-trail environments and disturbance experiments, with less-confident predictions improving control behavior. Compared with alternative DNN controllers, TrailNet recovered more quickly from injected rotations while operating alongside real-time perception modules.
- Experimental evaluation: The system autonomously navigated 100 m, 250 m, and 1 km forest trails, plus a difficult open-field trail.The forest trails were approximately 1.5 m wide; the 250 m trail included six turns.
- Control behavior: Overconfident predictions delayed turning because the controller was strongly biased toward flying straight.The original network assigned 0.9–0.99 probability to the winning class, which the authors attributed to overfitting.
- Control behavior: Entropy reward and label smoothing reduced overconfidence, enabling probability mixing for smoother trail-direction angles and better control.These terms were added to the loss function in Equation (1).
- Disturbance recovery: After 2-second left or right rotation disturbances, the 6-class TrailNet returned quickly to the straight course and produced a nearly straight trajectory.The comparison used a 2 m wide trail at 2 m/s and 2 m altitude.
- On-board execution: All modules ran simultaneously on the Jetson TX1, using 80% of the CPU and 100% of the GPU under the reported operating frequencies.DSO and TrailNet ran at 30 Hz, while YOLO ran at 1 Hz.
VI. CONCLUSION
The conclusion presents TrailNet and complementary perceptual modules as a real-time, on-board system for autonomous forest-trail navigation. The authors report smoother, more robust control and progress toward safe navigation in unstructured environments.
- VI. CONCLUSION: TrailNet combines SReLU activations, additional categories for lateral-offset and view-orientation estimation, and a loss function designed to prevent overconfidence.The conclusion frames these changes as improvements over existing work.
- VI. CONCLUSION: The system adds DNN-based object detection and monocular visual odometry for environmental awareness and obstacle detection.All software runs simultaneously in real time on a Jetson TX1 aboard the MAV.
- VI. CONCLUSION: Together, the reported improvements yield smoother and more robust control and move toward fully autonomous, safe robotic navigation in unstructured environments.