PaperlayerView exact revision on arXiv

Source-linked AI summary

2022 Roadmap on Neuromorphic Computing and Engineering

Dennis V. Christensen, Regina Dittmann, Bernabé Linares-Barranco, Abu Sebastian, Manuel Le Gallo, Andrea Redaelli, Stefan Slesazeck, Thomas Mikolajick, Sabina Spiga, Stephan Menzel, Ilia Valov, Gianluca Milano, Carlo Ricciardi, Shi-Jun Liang, Feng Miao, Mario Lanza, Tyler J. Quill, Scott T. Keene, Alberto Salleo, Julie Grollier, Danijela Marković, Alice Mizrahi, Peng Yao, J. Joshua Yang, Giacomo Indiveri, John Paul Strachan, Suman Datta, Elisa Vianello, Alexandre Valentian, Johannes Feldmann, Xuan Li, Wolfram H. P. Pernice, Harish Bhaskaran, Steve Furber, Emre Neftci, Franz Scherr, Wolfgang Maass, Srikanth Ramaswamy, Jonathan Tapson, Priyadarshini Panda, Youngeun Kim, Gouhei Tanaka, Simon Thorpe, Chiara Bartolozzi, Thomas A. Cleland, Christoph Posch, Shih-Chii Liu, Gabriella Panuccio, Mufti Mahmud, Arnab Neelim Mazumder, Morteza Hosseini, Tinoosh Mohsenin, Elisa Donati, Silvia Tolu, Roberto Galeazzi, Martin Ejsing Christensen, Sune Holm, Daniele Ielmini, N. Pryds

arXiv:2105.05956v3cs.ETcond-mat.dis-nncond-mat.mtrl-sci

TL;DR

The Roadmap addresses the energy and learning limitations of von Neumann computing by surveying neuromorphic materials, devices, circuits, algorithms, applications, and ethics. It synthesizes perspectives from leading researchers on the field’s current state, challenges, and future directions, including in-memory and edge computing. The paper concludes that neuromorphic engineering offers promising technological advances, while commercial viability, device control, algorithm scalability, and ethical issues remain important boundaries.

  • Problem

    Von Neumann computing incurs substantial energy costs from data movement and lacks intrinsic brain-like capabilities for learning and handling complex data.

  • Method

    The Roadmap synthesizes expert perspectives on neuromorphic materials, devices, circuits, algorithms, applications, and ethics.

  • Results

    The Roadmap presents neuromorphic engineering as offering technological advances for perception, computation, cognition, and low-power edge applications.

  • Takeaways & Limitations

    Neuromorphic computing provides a roadmap toward efficient computation across applications including autonomous transport, neuromorphic audition, and wearable EMG processing.

  • Takeaways & Limitations

    Widespread adoption requires compelling commercial advantages, while brain-science applications face competition from GPUs and HPC and clear wins remain difficult.

Abstract

from arXiv · show

Modern computation based on the von Neumann architecture is today a mature cutting-edge science. In the Von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 1018 calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this Roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The Roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges. We hope that this Roadmap will be a useful resource to readers outside this field, for those who are just entering the field, and for those who are well established in the neuromorphic community. https://doi.org/10.1088/2634-4386/ac4a83

Introduction

The Roadmap surveys neuromorphic materials, devices, circuits, algorithms, applications, and ethics as alternatives to energy-intensive data movement and limited learning capabilities in conventional computing. It presents current field perspectives, open challenges, and future opportunities, including in-memory processing and edge applications.

  • Motivation: Neuromorphic computing addresses conventional systems’ limited intrinsic ability to learn or handle complex data like the human brain.The Roadmap frames neuromorphic systems as computing systems inspired by or mimicking brain information processing.
  • Scope: The Roadmap covers neuromorphic materials and devices, circuits, algorithms, applications, and ethics.These areas are presented as contributions to advancing the field and assessing its potential applications.
  • Architectures and devices: Neuromorphic architectures seek energy and space efficiency by colocating memory and computation and using spike-mediated communication.Memristive devices are proposed as hardware representations for synapses and neurons, while non-von Neumann circuits can implement dense, efficient synaptic layers.
  • Algorithms: The Roadmap identifies algorithm development as a central challenge because neuromorphic hardware benefits require new strategies for real-world problems.Spiking data, stochasticity, noise, and non-ideal nonlinear synapses complicate direct training and continuous learning on portable devices.
  • Ethics: The Roadmap also highlights ethical concerns involving limited transparency in complex systems and autonomous decision making.Its final section addresses ethical questions arising from advances in neuromorphic computation.
  • Applications: In-memory matrix-vector multiplication with PCM crossbars supports DNN inference and promises improved latency and energy consumption over existing solutions.PCM devices can map synaptic layers to crossbar arrays, although accuracy is challenged by device nonidealities.
  • Device challenges and responses: Projected PCM improves in-memory scalar multiplication precision to 8-bit fixed-point arithmetic while enabling array-level temperature compensation.The approach addresses PCM 1/f noise and conductance drift; multilayer PCM devices have also been proposed to reduce drift.

1.2 – Ferroelectric Devices

Ferroelectric devices offer CMOS-compatible, low-power building blocks for neuromorphic memory and computation, but their scalability, endurance, switching behavior, and material integration remain important challenges.

  • Ferroelectric device landscape: <50ns switching at <5V has been reported for fully FEOL-integrated FeFETs in >1Mbit memory arrays.Fine-grained co-integration with CMOS logic is intended to reduce data-transfer constraints associated with the von Neumann bottleneck.
  • Device challenges: FeFET scaling requires uniform nanoscale polarization in HfO2 films, while silicon-based devices typically exhibit endurance near 10^5 cycles because of SiO2 interfacial-layer breakdown.The endurance limitation is linked to the dielectric interfacial layer between the silicon channel and ferroelectric gate insulator.
  • Device challenges: FTJs provide small current density for massively parallel analogue matrix-vector multiplication, but read speed is constrained by self-capacitance, layer thickness, leakage, and interfacial defects.Thin ferroelectric layers are difficult to form without dead layers and increased leakage from defects and grain boundaries.
  • Materials challenges: Open questions include stabilization of the ferroelectric orthorhombic Pca21 phase and interactions among electrodes, tunneling barriers, interfacial layers, charge trapping, and switching mechanisms.Resolving these issues is presented as necessary for optimizing material stacks and electrical operating conditions.
  • Ferroelectric device landscape: Hafnium-oxide ferroelectricity has renewed research into CMOS-compatible FeCAP, FeFET, and FTJ memory devices for dense, non-volatile, ultra-low-power systems.The roadmap describes this device family as a trinity of ferroelectric memory technologies and connects it to neuromorphic architectures.

1.5 Nanowire Networks

Self-organized memristive nanowire networks offer a biologically plausible alternative to rigid crossbar architectures by using collective network dynamics for neuromorphic processing. Their development requires multidisciplinary study across scales, while emerging 2D materials and heterostructures extend opportunities for low-latency in-sensor and in-memory computing.

  • 1.5 Nanowire Networks: Self-organized memristive nanowire networks better emulate biological topology, connectivity, and adaptability than rigid grid-like crossbar arrays.Their promise comes from self-organization and collective interactions among many nanoscale components.
  • 1.5 Nanowire Networks: Nanowire-network development requires integrated work spanning material physics, electronics engineering, neuroscience, and network science.The roadmap emphasizes network-level computing paradigms rather than focusing only on individual devices.
  • 1.5 Nanowire Networks: Scanning probe microscopy, particularly conductive atomic force microscopy, can characterize local nanowire-network conductivity and dynamics.These measurements help connect nanoscale junction behavior with collective macroscale behavior.
  • 1.5 Nanowire Networks: Reservoir computing and sensor-driven unconventional computing are highlighted as routes toward adaptive nanowire hardware for robotic behavior.Future architectures could combine multiple interconnected networks or heterogeneous stimuli.
  • 1.5 Nanowire Networks: A WSe2-based homojunction vision sensor was reported capable of processing images within 50 ns.This example illustrates the potential of computational sensing devices to reduce the delay associated with transferring raw sensor data.
  • 1.5 Nanowire Networks: 2D materials and van der Waals heterostructures support ultralow-latency, reconfigurable in-sensor computing and promising in-memory computing architectures.Reported directions include ultrafast vision sensing, memristive devices, and integration of material growth, device physics, arrays, and peripheral circuits.

1.7 – Organic materials

The supplied passage identifies Tyler J. Quill, Scott T. Keene, and Alberto Salleo.

  • The passage lists Tyler J. Quill, Scott T. Keene, and Alberto Salleo.

1. Status

Organic semiconductors are promising neuromorphic materials because their chemical and microstructural properties support tunable, low-energy synaptic behavior. Commercial deployment remains constrained by speed, density, integration, and stability challenges, although the roadmap identifies strategies addressing each.

  • Capabilities: Organic semiconductors support low-energy, tunable synaptic devices through filament formation, charge trapping, ion migration, and three-terminal transistor mechanisms.Their large free volume facilitates ion migration, while chemical and microstructural control influences device performance.
  • Capabilities: Organic neuromorphic devices have demonstrated synaptic weight representation, excitatory postsynaptic potentials, global connectivity, and pulse shaping.These functions support potential applications from high-performance computing to biological interfacing.
  • Challenges and strategies: Device speed still lags inorganic counterparts because electronic and ionic mobilities, defects, and stray capacitances can limit operation.Side-chain engineering has improved mixed ionic/electronic conducting organic devices, but does not eliminate the speed gap.
  • Challenges and strategies: Patterning organic semiconductors is difficult because solvents and photon wavelengths used in photolithography can be incompatible with the materials.Vertical architectures, hard-mask strategies, and additive manufacturing are proposed routes toward higher density.
  • Challenges and strategies: Integration is limited by organic-material degradation above typically 150 °C during back-end processing, while environmental exposure can destabilize interfaces and retention.BEOL alternatives, access devices, molecular engineering, crystallinity optimization, and encapsulation are identified as mitigation strategies.
  • Applications and outlook: Organic materials are promising for brain-machine interfaces and adaptive prosthetics because of their biocompatibility and softer mechanical properties, but commercial systems still require major improvements.The roadmap states that no fundamental barriers prevent meeting the required speed, density, integration, and stability metrics.

1.8 – Spintronics

Spintronics uses electron spin, magnetic non-volatility, and nonlinear magnetization dynamics to implement neuromorphic synapses and neurons. The roadmap projects near-term ST-MRAM-based AI chips, while more complex spintronic computing remains experimentally incomplete.

  • Device principles: Spintronic devices combine nanomagnet non-volatile memory with nonlinear magnetization dynamics to mimic synapses and neurons.This multifunctionality allows the same materials to support essential neural operations.
  • Synapses: Magnetic tunnel junctions can act as memristive synapses whose resistance stores synaptic weights and multiplies input currents.Stable magnetization supports weight retention, while bistability makes magnetic tunnel junctions suitable for neural-network implementations.
  • Neurons: Spintronic nonlinear dynamics can reproduce neuron-like voltage-to-spike-frequency relationships and may support local and unsupervised learning.The approach exploits device dynamics beyond simply applying a static nonlinear activation function.
  • Interfaces and speed: Spintronic chips could process digital, radio-frequency, and biological-timescale inputs, while emerging materials may enable THz information processing and transmission.Superparamagnetic junctions and magneto-electric effects span seconds-to-milliseconds timescales; antiferromagnets and optical interfaces target THz operation.
  • Future directions: Complex spin-wave and magnetic-particle systems have been modeled for learning, but experimental demonstrations remain to be carried out.Examples include arrays of spin-wave transmitters and receivers, skyrmions, and domain walls.
  • Outlook: Near-term development is expected to commercialize AI chips using ST-MRAM weights before hardware neuron circuits and longer-term in-materio computation.The roadmap frames this as a progression from established memory technology toward more exotic materials and magnetic textures.

2.2 Spiking neural networks

Spiking-neural-network hardware aims to combine in-memory computation, online processing, and biological dynamics while avoiding the von Neumann memory bottleneck. Progress depends on jointly developing local spike-based learning theories, compatible memory technologies, and co-designed hardware systems.

  • Circuit approaches: Neuromorphic circuit research spans large-scale digital CMOS platforms for general-purpose simulation and analog circuits for real-time emulation of neural dynamics.The analog approach targets specific sensory-motor online-processing tasks and continues the original neuromorphic-engineering objective.
  • Learning: A central learning challenge is implementing BPTT-level capabilities with local signals in spiking hardware whose synapses have limited resolution.This constraint follows from the in-memory-computing organization of spiking neural networks.
  • Memory and time: Spiking architectures avoid external memory transfers but must retain short-term traces of recent inputs for real-time sensory-stream computation.Their online, nonlinear-filter operation creates a coupled challenge of managing memory and time.
  • Core challenges: The field needs a computation theory combining fading memory, nonlinear dynamics, and local spike-based learning with volatile and nonvolatile memories compatible with CMOS analog circuits.Both theoretical mechanisms and supporting memory technologies are identified as interlinked requirements.
  • Biological inspiration: Biological brains provide an existence proof that robust computation can use analog, inhomogeneous, and imprecise elements.This motivates studying how biological systems achieve stable computation despite non-ideal components.
  • Outlook: A co-design approach is positioned as the route for advancing neuromorphic circuits and memristive devices together, especially for low-power edge applications.Edge systems process locally measured data without remote servers and often require low latency and compact packaging.
  • Current boundary: No general-purpose solution or established formal methodology yet exists for designing and programming analog CMOS–memristive spiking systems.Existing demonstrations remain few and focused on specific sensory or biomedical tasks.

2.3 – Emerging Hardware Approaches for Optimization

Physics- and brain-inspired hardware complements heuristic optimization algorithms by implementing diverse electronic, magnetic, optical, and quantum approaches. The roadmap emphasizes that practical success depends on scalable, reliable systems matched to diverse problem instances and integrated across the computing stack.

  • Approaches: Emerging optimization hardware includes quantum annealers, optical and coherent Ising machines, CMOS digital annealers, resistive-memory systems, coupled oscillators, and probabilistic-bit logic.These approaches complement meta-heuristics, Boltzmann machines, Ising models, and Hopfield-network variants.
  • Benchmark comparison: A coherent Ising machine solved a 200-node cubic Max-Cut graph in about 50 ms, while a D-Wave annealer solved one in 11 ms at around 25 kW cryogenic power.The comparison illustrates differing architectures and operating requirements rather than a universal ranking.
  • Benchmark comparison: A Hitachi CMOS Ising chip achieved 50x lower energy-to-solution than a CPU using a greedy algorithm on a 200-node random cubic graph.The cited comparison concerns energy-to-solution for that benchmark and baseline.
  • Device requirements: Future solvers require multi-bit levels, high endurance, robustness, low variability, and rapid re-programmability across device technologies.Binary Max-Cut representations do not cover optimization problems requiring more than two states.
  • Scaling: Large applications may require many thousands of variables and constraints, forcing multiple processing units to communicate with low latency, low energy, and high bandwidth.Railway crew scheduling is given as an example involving tens of thousands of trains per week.
  • Scaling: Increasing scale and fan-out introduce parasitics, variability, and declining success probability in finding globally optimal solutions.Oscillator systems additionally face interconnect capacitance and frequency variability as networks grow.
  • Problem diversity: Optimization instances vary substantially even within one problem class, so domain-specific parameter choices and techniques remain important.Relevant differences include barrier scales, saddle-point density, and the closeness of local and global minima.
  • Outlook: The roadmap concludes that the leading approach will need to combine flexibility, performance, reliability, and an active user community linking software and hardware researchers.This conclusion frames adoption and solver quality as joint success criteria.

2.6 – Large-Scale Neuromorphic Computing Platforms

Large-scale neuromorphic platforms have made system-level experimentation accessible through reliable hardware and software, but widespread adoption still requires compelling advantages over conventional technologies.

  • Current platforms: SpiNNaker, BrainScaleS, and Loihi are large-scale CMOS neuromorphic systems with distinct architectural approaches and established user communities.SpiNNaker uses embedded processors and packet switching; BrainScaleS uses analogue circuits running 10,000 times faster than biology.
  • Current platforms: High-level software stacks such as PyNN let users develop applications without detailed knowledge of the underlying hardware.These stacks support experiments focused on network architectures, learning rules, and brain modelling.
  • Current platforms: Reliable, accessible platforms mean that access to neuromorphic technology is no longer a limiting factor for exploring its capabilities.The systems can also be used to model future neuromorphic technologies at minimal cost.
  • Challenges: Widespread adoption depends on demonstrating commercial viability through significant capability, performance, or energy-efficiency advantages over competing technologies.Existing demonstrations of neuromorphic superiority remain relatively few, while brain-science applications compete with GPUs and high-performance computing.
  • Challenges: Scaling spiking-neuron systems amplifies challenges including homeostasis, adaptation to failures, unsupervised learning, synaptogenesis, neurogenesis, and online learning.A future challenge is combining current system scalability with characteristics offered by advanced device technologies.
  • Learning and algorithms: Spiking neural networks exploit asynchronous technologies and spatiotemporal sparsity, making them promising for efficient processing of dynamical signals.Three-factor rules combine presynaptic and postsynaptic factors with task-dependent modulation such as error, surprise, or reward.
  • Learning and algorithms: Off-chip learning generally achieves the best inference accuracy on practical tasks, while chip-in-the-loop approaches have succeeded at smaller scales but face scalability limits.The limitation arises from access to local chip states needed for plasticity.
  • Learning and algorithms: Task-objective-driven three-factor rules currently offer practical advantages over Hebbian STDP variants, while backpropagation remains difficult to implement physically because it requires non-local signals.Future progress requires algorithms and software designed for neuromorphic constraints rather than simply mapping conventional networks onto SNNs.

3.2 – Learning-to-Learn for Neuromorphic Hardware

Learning-to-Learn proposes separating extensive offline priming from rapid on-chip adaptation, so neuromorphic hardware can learn new user tasks from very few examples. Its feasibility depends mainly on implementing the required on-chip rule and training the offline outer loop effectively.

  • Motivation: Fast on-chip learning requires both a sufficiently powerful learning method and rapid convergence, ideally from a single example.The motivation is one-shot or few-shot learning of new classes, analogous to rapid learning in brains.
  • Offline priming: Learning-to-Learn optimizes initial synaptic weights and other hyperparameters offline for a family of user tasks before loading them onto the neuromorphic chip.The optimized system is intended to generalize rapidly to structurally related tasks not encountered during priming.
  • Offline priming: Offline priming can select learning rates, synaptic weights, initial weights, or parameters of an auxiliary learning-signal generator.These choices determine which components are optimized before the user performs on-chip learning.
  • Training challenges: The main training challenge is the efficacy of the offline outer loop, with demanding network optimizations tending to require backpropagation through time.Simpler learning-rate optimization can often use gradient-free methods, whereas broader network optimization is more demanding.
  • Hardware realization: Most neuromorphic hardware could support the approach when the inner loop uses simple local plasticity rules, while some options require emulating adapting spiking neurons.The hardware must be able to run the selected on-chip learning algorithm.
  • Applications: A sample application is learning a new spoken command from one example and recognizing it across different acoustic conditions and speakers.This illustrates the intended user-facing benefit of rapid on-chip adaptation.
  • Concluding remarks: The proposed two-phase strategy performs extensive offline priming for a broad task family, then learns remaining parameters on-chip from very few examples.The outer phase may optimize hyperparameters and network architecture on hardware or a software model.
  • Concluding remarks: The outer loop may also impose priors that prevent on-chip learning from entering unsafe or otherwise undesired operating regimes.The passage reports that powerful priors for subsequent RSNN computing and learning have already been verified.

3.4 – Stochastic Computing

Stochastic computation uses randomness and noise as computational resources, with potential links to biological neural processing and energy-efficient artificial systems. Its broader adoption remains constrained by immature foundations and implementation challenges.

  • Stochastic computation encompasses systems that exploit noise or randomness and systems processing intrinsically random or noisy inputs.The supplied definition distinguishes randomness used to enable otherwise inaccessible states from intrinsically noisy data streams.
  • Random projection layers may improve neural-circuit versatility and selectivity, motivating stochastic models for real-world computation.The passage connects this possibility to evidence about biological neural systems and parallels in artificial networks.
  • The foundations of stochastic computation remain immature, with mostly proofs-of-concept and few generic principles for broad application.Noise and randomness are still commonly treated as signal-processing obstacles, limiting routine adoption.
  • A proposed research priority is foundational probabilistic-computing theory integrating Bayesian methods, stochastic resonance, and random projections.The goal is to exploit noisy, nonlinear inputs or systems rather than simply filtering those properties out.
  • Stochastic computation models the noisy and variable conditions of biological brains and may guide artificial computation on similarly noisy signals.The motivation is that the brain computes successfully despite noisy hardware and inputs.
  • Convolutional spiking networks face energy, training, robustness, and interpretability challenges when processing temporal spike representations.Rate coding can generate redundant spikes; deep direct training has convergence issues, while robustness and internal spike behavior remain active concerns.
  • Alternative temporal and phase coding schemes reduce spike counts or encode information through spike timing and patterns.Examples include one-spike temporal coding, rank-order coding, signed timing spikes, and oscillator-based phase coding.
  • Recent studies report greater adversarial robustness for some SNNs and introduce heatmaps to visualize discriminative attention.The cited work links robustness to Poisson coding and non-differentiable neuronal dynamics, while visualization supports interpretability research.

Coding with Spiking Neurons

Coding with spiking neurons replaces conventional rate-based representations with temporal alternatives that can transmit information using fewer or more structured spikes. These codes support efficient learning and large-scale neuromorphic systems, while robotics adds challenges of scalable event-driven sensing, perception, decision-making, and control.

  • Coding strategies: Rate coding counts spikes over a fixed observation window, whereas temporal schemes encode activation through firing order, latency, or selected spike combinations.The compared strategies include conventional rate coding, rank-order coding, and N-of-M coding.
  • Coding strategies: Rank Order Coding assigns meaning to firing order, but 16 inputs already permit nearly 21 trillion possible orders.A feedforward shunting-inhibition mechanism progressively reduces the effectiveness of later spikes.
  • Coding strategies: N-of-M coding uses feedback inhibition to stop firing after a target number of inputs, enabling inexpensive winner-take-all behavior with binary synapses.Each neuron responds to how well the selected input spikes match its binary weight pattern.
  • Learning: Temporal coding can support more efficient learning than standard backpropagation, which requires many labelled training trials.The motivation is rapid learning from sparse presentations, closer to the cited biological example.
  • Learning: JAST uses binary synapses and connection-location swaps to detect repeating patterns in as few as 2-5 presentations.This contrasts with continuously weighted STDP rules that typically require tens of repeats.
  • Hardware: A Spartan-6 FPGA implementation supported 4096 inputs, 1024 output neurons, 100,000 updates per second, and on-chip learning.The cited implementation integrated the learning algorithm directly into the circuit.
  • Hardware: Spiking neural networks address inefficiencies in rate-based or floating-point coding through temporal codes and spike-timing-dependent plasticity.Binary connections and sparse spike communication are presented as routes toward large networks with billions of neurons and trillions of connections.
  • Robotics: Neuromorphic robotics must scale event-driven sensing, perception, decision-making, and control while accounting for interactions among brain, body, and environment.The roadmap calls for event-driven readout across platform sensors and architectures informed by neuroscience, machine learning, and engineering.

4.2 – Self-Driving Cars

Neuromorphic engineering is presented as a potential route toward higher-level autonomous driving by improving perception, cognition, and energy efficiency. Event-based vision offers relevant advantages, but deployment remains constrained by sensor resolution, integration, and post-processing challenges.

  • Level 4 ADAS is considered attainable and useful, whereas full Level 5 autonomy is unlikely within five years.
  • Human-quality visual perception, world modeling, and low-power real-time perception-cognition-action computation are central challenges for autonomous vehicles.
  • Current event-based sensors face low spatial resolution, asynchronous variable-rate output, and difficult integration with downstream processing.
  • Event-based vision provides high dynamic range and spatio-temporal resolution for high-speed, low-latency, resource-constrained applications.
  • Advanced semiconductor integration and sparse-data processing could support tightly packaged, ultra-low-power edge perception systems.

4.5 – Neuromorphic Audition

Neuromorphic audition uses cochlea-inspired asynchronous and sparse temporal encoding to support low-latency, energy-efficient audio processing. Progress depends on combining suitable silicon technology with algorithms that preserve robustness across difficult acoustic conditions.

  • Technology and biological inspiration: Biological and silicon cochleas encode sound through asynchronous pulses across broadly frequency-selective channels.
  • Technology and biological inspiration: Sparse sampling and temporal event encoding support low-latency spatial audition through extraction of interaural time differences.
  • Audio processing architectures: Spiking cochlea features can support always-on voice activity detection and keyword spotting before more energy-intensive speech recognition.
  • Audio processing architectures: Spiking cochlea features show lower clean-condition accuracy than spectrogram features but retain lower accuracy loss as signal-to-noise conditions decrease.
  • Hardware challenges: Neuromorphic audition still requires advances in low-leakage, low-mismatch silicon, algorithms, and on-chip learning.
  • Hardware challenges: Recent ASICs combining binary networks and spectral features report keyword-spotting power below 1 uW, motivating comparisons with spiking-cochlea front ends.

4.7 – Embedded Devices for Neuromorphic Time-Series Assessment

Neuromorphic embedded devices target low-power, low-latency analysis of multimodal human time-series signals, including EMG. The roadmap emphasizes better signal conversion and decomposition, tighter sensor-chip integration, and eventual fully spiking pipelines.

  • Scope and challenges: Embedded neuromorphic systems address speech, health monitoring, activity recognition, and other human time-series tasks requiring compact hardware.
  • Scope and challenges: Raw multimodal time-series data are difficult for DNNs because variables encode concurrent actions and may be imbalanced or correlated.
  • EMG processing: Neuromorphic EMG processing reduces power consumption and latency by up to three orders of magnitude, with a 5−7% accuracy loss.
  • EMG processing: High-density EMG improves measurement precision through hundreds of electrodes but increases computational resource requirements.
  • Roadmap: Current ML-to-neuromorphic implementations face an accuracy gap and substrate mismatch because many neuromorphic platforms primarily target SNNs.
  • Roadmap: The roadmap prioritizes improved front-end signal-to-spike conversion, real-time decomposition, neuromorphic-chip deployment, and smart electrodes that directly record motor-unit activity.
  • Roadmap: Mixed-signal SNN processors integrated with sensors could perform real-time in-situ EMG processing without external computation.

4.9 – Collaborative Autonomous Systems

Collaborative autonomous systems coordinate agents and humans in complex, uncertain environments, requiring advances in planning, perception, communication, cognition, and control. Their development also raises ethical, social, legal, and sustainability questions that require context-sensitive governance.

  • System goals and challenges: Collaborative autonomous systems cooperate with humans and other agents while operating with variable levels of human intervention in unknown environments.
  • System goals and challenges: Deployment requires online mission planning and adaptive allocation of heterogeneous agents under uncertainty.
  • Technology directions: Proposed mission-planning advances include closed-loop task allocation and automated survivability prediction.
  • Technology directions: Existing human-supporting CAS can improve productivity, speed, and accuracy while reducing heavy work and collision risk.
  • Ethics: Ethical concerns include privacy, opacity, bias, manipulation, autonomous decision responsibility, automation-related unemployment, and sustainability.
  • Ethics: Ethically defensible development may require regulation, developer responsibility, citizen oversight, or combinations chosen according to the concrete situation.
Loading 2105.05956v3…