PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards bio-inspired visual navigation: a neuromorphic navigation paradigm for MAVs

Junjie Jiang, Ankang Zhou, Zheng Fang

arXiv:2610.21978v1cs.RO

TL;DR

Energy-constrained MAVs still face a gap in robust visual navigation in complex environments. This paper integrates event-based sensing, spiking neural networks, and neuromorphic hardware into an end-to-end navigation system, reporting performance comparable to ANN-based methods.

  • Problem

    Visual navigation for small aerial robots remains separated from the robust, low-resource navigation of flying animals, motivating approaches for complex environments and tight energy budgets.

    Source: Introduction · p. 2

  • Method

    The system combines event-camera sensing, spiking perceptual representation and navigation, and neuromorphic hardware in an end-to-end MAV navigation loop.

    Source: Introduction · p. 5, Discussion · p. 19

  • Results

    Navigation performance is comparable to the ANN-based method in success rate, average trajectory distance, and average navigation time.

    Source: Results · p. 13

  • Takeaways & Limitations

    Physical-system validation demonstrates that end-to-end neuromorphic navigation can operate on a resource-constrained MAV.

    Source: Introduction · p. 5, Discussion · p. 20

    The physical system does not yet deploy all spiking neural networks onboard neuromorphic hardware and still relies on wireless data transmission.

    Source: Discussion · p. 20

Abstract

from arXiv · show

Biological visual navigation achieves remarkable robustness at extremely low power, whereas frame-based robotic systems relying on artificial neural networks remain limited in both respects, particularly for energy-constrained micro aerial vehicles (MAVs) operating in complex environments. Inspired by the event-driven, sparse, and asynchronous principles of biological visual systems, neuromorphic sensing and computation provide a promising alternative for robust and energy-efficient navigation. In this paper, we propose the first end-to-end event-driven neuromorphic paradigm for MAV visual navigation in complex environments by integrating neuromorphic vision, spiking neural networks, and neuromorphic processors. The proposed method first extracts the environmental representation from event streams using a spiking perceptual representation module. This representation is fed into a spiking navigation module, where guidance learning and population coding enable efficient policy learning with few time steps. The navigation module then outputs a waypoint converted into executable trajectory points. Experiments show that the proposed neuromorphic method achieves navigation performance comparable to ANN-based methods while markedly reducing power consumption. The spiking perceptual module consumes only 19.57% of the power of its ANN counterpart, while the spiking navigation module deployed on Intel Loihi consumes only 12.37% of that required by its ANN counterpart running on NVIDIA Orin NX. Beyond prior neuromorphic systems focused mainly on relatively simple navigation subtasks, we demonstrate a complete neuromorphic visual navigation system spanning simulation and real-world deployment. Our results establish the feasibility of the neuromorphic paradigm for autonomous navigation and provide a bio-inspired pathway toward robust and low-power MAV navigation.

Introduction

The paper presents an end-to-end event-driven neuromorphic framework for MAV navigation, combining event-based perception with spiking computation and hardware execution. It evaluates the system in simulation and physical flight as a route to resource-efficient navigation.

Introduction

  • Biological visual systems navigate robustly with limited resources, while frame-based MAV systems use dense synchronous computation and may depend on salient ground features, motivating sparse asynchronous neuromorphic sensing and processing.Event cameras respond to pixel-level brightness changes, and SNNs process asynchronous spikes; prior neuromorphic technologies have demonstrated visual-task capability and potential energy savings (17–31).Source: Introduction · pp. 2–4
  • The paper introduces an end-to-end event-driven MAV navigation paradigm integrating an event camera, spiking neural networks, and neuromorphic hardware, addressing prior systems that used neuromorphic components at only one stage.Earlier work used event cameras with ANNs (37, 38) or SNNs with frame-based inputs (39,40); the paper identifies a complete real-world neuromorphic navigation system as an open challenge.Source: Introduction · pp. 3, 5
  • The perceptual module converts event streams into compact geometric features to compensate for missing explicit structure and reduce rendering-related sim-to-real mismatch; guidance learning and population coding support efficient SNN policy training.Population coding supports informative spike representations with few time steps, and the learned policy runs on a neuromorphic processor.Source: Introduction · pp. 5–6

Results

  • The proposed SNN navigation method achieves performance comparable to its ANN counterpart in success rate, average trajectory distance, and average navigation time across six simulation environments.The comparison evaluates maximum speeds of 0.5, 1.0, and 2.0 m/s; Table 1 summarizes results across 48 trials per setting.Source: Results · pp. 9, 13, Table 1 · p. 13
  • Simulation results show that slower maximum speeds reduce navigation success, while removing depth supervision causes collisions, ground contacts, and larger deviations from targets.The paper links slow-speed degradation to fewer events from distant objects, less reliable depth estimates, and delayed obstacle avoidance.Source: Results · pp. 13, 15, Figure 9 · p. 15
  • In real-world trials, the system reached its target in 7/10 indoor and 9/10 outdoor attempts, demonstrating transfer from simulation to physical MAV flight.The paper attributes reduced real-world success partly to the event camera’s limited horizontal field of view and noisy depth supervision, especially indoors.Source: Results · pp. 13, 16, Figure 10 · p. 15
  • The SNN perceptual representation network consumes 0.507 mJ per inference versus 2.591 mJ for its ANN counterpart, or 19.57% of the ANN energy.The estimates use operation counts under a 45-nm CMOS assumption: 563.02M ACs for the SNN and 563.18M MACs for the ANN.Source: Results · pp. 17–18, Figure 11 · p. 18
  • On Intel Kapoho Bay, the SNN navigation model uses 8.09 times less per-inference energy than the ANN on Orin NX in high-performance mode and 3.73 times less in power-saving mode.The comparison also reports that the ANN on an NVIDIA GeForce RTX 4090 uses approximately 25.54 times the per-inference energy of the SNN on Kapoho Bay.Source: Table 2 · p. 17, Results · pp. 18–19

Discussion

  • The neuromorphic system achieves navigation performance comparable to its ANN counterpart while substantially reducing inference energy consumption; real-world flight experiments further validate the feasibility of the complete framework.Source: Discussion · p. 19
  • The physical system does not yet deploy all spiking networks onboard and still relies on wireless transmission and a frame-based camera for pose estimation.The authors attribute these constraints to Intel Kapoho Bay’s supported neuron models and capacity, and limitations in event-based visual odometry; future work targets more capable hardware and lightweight, accurate event-based odometry.Source: Discussion · p. 20
  • The framework unifies event-based sensing, spiking neural computation, and neuromorphic hardware in a physical MAV perception–decision–execution loop.The authors present physical-system validation as evidence that biological asynchronous, sparse, and event-driven principles can operate in robotic navigation.Source: Discussion · pp. 19–20

Materials and Methods

  • Event cameras asynchronously emit sparse events from pixel luminance changes, while spiking depth estimation and feature-extraction networks turn event streams into compact obstacle-relevant geometric representations.The method accumulates events into polarity-separated event frames, estimates depth with a spiking U-Net, then extracts geometry-aware features with a spiking residual network.Source: Materials and Methods · pp. 21–22
  • Population coding represents continuous inputs with Gaussian receptive-field spike trains and decodes output spike trains into continuous actions; the navigation state includes motion, goal direction, and visual features.The network predicts waypoint offsets, velocity, and acceleration.Source: Materials and Methods · pp. 23–24
  • Training combines depth-based geometric supervision with guidance learning that optimizes smoothness, obstacle safety, and goal consistency using trajectory gradients and surrogate-gradient backpropagation.The trajectory between initial and end states is parameterized by a quintic polynomial.Source: Materials and Methods · pp. 24–26
  • Energy estimation compares ANN multiply–accumulate operations with spike-triggered SNN accumulations, while hardware measurements derive per-inference energy from dynamic power and inference speed.The system uses ROS for communication, event-stream acquisition and processing, and Intel NxSDK for navigation-network deployment and neuromorphic-processor communication.Source: Materials and Methods · pp. 27–28

Towards bio-inspired visual navigation: a neuromorphic

The supplementary methods describe how simulation training data and real-world depth supervision are prepared for the spiking depth-estimation module, alongside visualizations of navigation and depth predictions.

Materials and Methods

  • 20 forestlike training scenarios each contribute 2,500 grayscale image pairs, with corresponding depth images and MAV poses collected in Flightmare.Small pose perturbations generate consecutive image pairs for rendering event streams with vid2e; the dataset also includes environmental point-cloud maps.Source: Materials and Methods · pp. 37–38, Figure S1 · p. 38
  • For real-world fine-tuning, synchronized EVK4 event streams and RealSense D435i depth images are calibrated, aligned, and resized from 1280 × 720 to 320 × 180.The aligned event frames and depth images provide pixel-level supervision for fine-tuning the spiking depth-estimation module.Source: Materials and Methods · pp. 37–38, Figure S2 · p. 38
  • Supplementary visualizations show predicted depth in simulation and indoor and outdoor settings, while movies demonstrate simulated and real-world navigation.Movie S1 includes navigation at different maximum flight speeds; Movie S2 presents indoor and outdoor obstacle configurations.Source: Figure S4 · p. 43, Materials and Methods · pp. 43–44
Loading 2610.21978v1…