PaperlayerView exact revision on arXiv

Source-linked AI summary

Event-based Vision: A Survey

Guillermo Gallego, Tobi Delbruck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Joerg Conradt, Kostas Daniilidis, Davide Scaramuzza

arXiv:1904.08405v3cs.CVcs.AIcs.LGcs.RO

TL;DR

Event cameras acquire visual information asynchronously, offering advantages for high-speed and high-dynamic-range scenarios but requiring specialized processing. This paper surveys their sensors, algorithms, applications, computing approaches, and remaining challenges, concluding that the field shows potential while retaining substantial room for improvement.

  • Problem

    Event cameras offer high temporal resolution, high dynamic range, and low power, but their unconventional asynchronous output requires novel methods to unlock these properties.

  • Method

    The paper provides a comprehensive overview of event-camera principles, available sensors, processing techniques, applications, specialized processors, and open challenges.

  • Results

    The survey reports event cameras’ potential for computer vision and robotics in scenarios involving low latency, low power, high speed, and high dynamic range.

  • Takeaways & Limitations

    Event-based vision may support perception and control in scenarios previously inaccessible to traditional cameras, while many research opportunities remain.

  • Takeaways & Limitations

    The field remains less mature than frame-based vision, and event cameras face practical limitations including low spatial resolution and high cost.

Abstract

from arXiv · show

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.

1 INTRODUCTION AND APPLICATIONS

The paper introduces event cameras as bio-inspired sensors that acquire visual information asynchronously and reviews their applications, advantages, processing challenges, and organization of the field.

  • Introduction: Event cameras sample light according to scene dynamics rather than an unrelated clock, producing asynchronous visual information.This differs from standard frame-based sensors that provide synchronous image sequences.
  • Advantages and applications: Microsecond temporal resolution, 140 dB dynamic range, and low power consumption make event cameras promising for challenging robotic and wearable scenarios.The paper highlights high-speed operation and uncontrolled lighting as representative use cases.
  • Processing challenge: Because events encode asynchronous per-pixel brightness changes rather than absolute brightness at a constant rate, novel processing methods are required.The paper frames this processing requirement as necessary to unlock the sensors’ potential.
  • Applications: Applications include object tracking, surveillance, recognition, depth estimation, structured-light 3D scanning, optical flow, and HDR reconstruction.These examples span real-time interaction, monitoring, geometric estimation, motion analysis, and image reconstruction.
  • Paper organization: The paper is organized around sensor principles and challenges, event-processing methodologies, applications from low-level to high-level vision, and future opportunities.The outline connects the technology, algorithms, applications, and open research directions.

2 PRINCIPLE OF OPERATION OF EVENT CAMERAS

Event cameras acquire visual information asynchronously from per-pixel brightness changes rather than clocked full images. Their event representation offers low latency and high temporal responsiveness, but introduces hardware and modeling constraints that require specialized processing.

  • Event-based sensing: Event cameras respond asynchronously and independently at each pixel, producing a variable-rate stream of events rather than clocked full images.Each event encodes a brightness change at a pixel, with its location, timestamp, and polarity.
  • Event-based sensing: Each pixel emits an event when the change from its memorized log intensity exceeds a threshold, recording x, y, time t, and ON/OFF polarity.The pixel continuously monitors brightness relative to its last event-triggering value.
  • Event-based sensing: Microsecond timestamps and sub-millisecond transmission latency allow event cameras to react quickly to visual stimuli, with event rates increasing as scene motion accelerates.The sensors are data-driven: faster brightness changes generate more events per second.
  • Sensor response: DVS pixels have finite bandwidth, so photoreceptor filtering suppresses intensity variations that change too quickly; the reciprocal bandwidth corresponds to an exposure-like rise and fall time.The frequency response is measured using sinusoidally varying illumination and compared with a 60 fps camera response that includes aliasing.

3 EVENT PROCESSING

Event processing converts asynchronous, sparse events into representations and processing strategies suited to each task. The review covers event-by-event and grouped processing, grid-based representations, motion compensation, and learned representations.

  • Processing strategies: Event-by-event methods update system state upon individual arrivals, whereas grouped methods aggregate events and introduce latency.Grouped processing can improve the signal-to-noise ratio by combining several events.
  • Processing strategies: Task-dependent event representations extract meaningful information from individual events that otherwise convey little information alone.Representations range from simple hand-crafted transformations to more elaborate forms.
  • Event representations: Event frames collapse events into 2D histograms, while point-set representations retain events as sparse 2D or 3D geometric data.Point sets support methods such as mean-shift, ICP, plane fitting, and PointNet.
  • Event representations: Time surfaces retain recent-event timing per pixel, while voxel grids preserve temporal structure by distributing events across space-time bins.Interpolated voxel grids provide sub-voxel accuracy by spreading event polarity among nearby voxels.
  • Motion compensation: Motion-compensated event images warp events to a reference time and maximize alignment to estimate motion and produce sharp edge images.The technique supports ego-motion, optical flow, depth, motion segmentation, and feature-motion estimation.
  • Design choices: Representation choices include separate or joint polarity processing, and learned systems can optimize representations for task performance.Event representations may be used at preprocessing or within core processing stages.

4 ALGORITHMS / APPLICATIONS

Event-based algorithms address low-level vision tasks such as feature detection and tracking, using asynchronous event data and, in some approaches, complementary intensity information. The section emphasizes motion-dependent appearance, data association, and evaluation limitations.

  • Feature detection and tracking: Event cameras enable asynchronous, low-latency feature tracking adapted to scene dynamics, including during the intervals between standard-camera frames.Tracking methods must handle event-specific space-time representations and motion-dependent appearance.
  • Challenges: Motion direction changes event-based edge appearance, making correspondences between events or derived features difficult to establish.Sensor noise and camera-motion clutter add a second major tracking challenge.
  • Tracking methods: Early blob-based methods supported surveillance and high-speed tracking but worked only for a limited class of object shapes.Later iterative methods tracked more complex, user-defined high-contrast shapes, including at a 200 kHz equivalent frame rate.
  • Combining events and frames: Combining frames with events supplies an intensity-based feature map that simplifies data association for arbitrary edge patterns.Frame-based feature locations and descriptors can be tracked asynchronously with events.
  • Combining events and frames: The generative tracker achieved the highest accuracy among five compared feature trackers, with sub-pixel performance, but was computationally expensive.Reconstructed event images produced similar detection and tracking results to frames in the cited study.
  • Opportunities: Feature-tracking methods are rarely evaluated on common datasets, motivating benchmark datasets, common procedures, and adaptive parameter tuning.These opportunities are presented as ways to foster comparison and broaden tracker operation.

4.2 Optical Flow Estimation

Event-based optical flow estimation adapts geometric, filter-based, joint-estimation, and deep-learning methods to asynchronous brightness-change data. The review covers challenging data assumptions, representative approaches, and remaining evaluation and computational constraints.

  • Problem: Optical flow is difficult because events provide brightness changes rather than absolute images, so conventional brightness-constancy methods require additional assumptions.Sparse event generation can make spatial and temporal derivative estimates unreliable.
  • Applications: 750 ◦/s is the fidget-spinner speed in an example of dense optical flow estimation under dark conditions.The figure also shows sparse flow in an indoor flying scene.
  • Model-based methods: Time-surface plane fitting estimates normal flow from the slopes of event surfaces in x-t and y-t space-time cross sections.The method interprets moving-edge events geometrically in space-time.
  • Model-based methods: Motion-compensation methods estimate flow by warping event cuboids and maximizing the sharpness of the resulting image patches.This acts as an adaptive filter whose coefficients select the spatio-temporal direction with the strongest response.
  • Learning-based methods: Deep networks estimate dense flow from event representations such as time surfaces or voxel grids using supervised or unsupervised photometric and motion-compensation losses.Unsupervised extensions jointly learn optical flow and ego-motion with motion-compensation losses.
  • Evaluation: GPU requirements affect several accurate methods, while lightweight approaches are less accurate and only a few have been implemented in hardware logic.Comprehensive real-scene datasets with ground-truth flow remain needed for comparison and reproducibility.

4.3 3D reconstruction. Monocular and Stereo

Event-camera 3D reconstruction spans instantaneous stereo, monocular depth, active structured light, and space-sweeping methods, each using different assumptions about motion, synchronization, and scene structure.

  • Instantaneous Stereo: Instantaneous stereo reconstructs 3D from synchronized rigidly attached cameras by matching events across image planes and triangulating their locations.Methods use event frames, time surfaces, temporal correlations, and constraints such as epipolar geometry, ordering, uniqueness, edge orientation, and polarity.
  • Instantaneous Stereo: Global stereo methods reduce correspondence ambiguities by enforcing spatial regularity through cooperative networks, Markov Random Fields, belief propagation, or semiglobal matching.These approaches improve depth estimates over local methods at the expense of computational effort.
  • Instantaneous Stereo: Dedicated GPUs enable brute-force space-sweeping, while neuromorphic processors support fully event-based stereo systems operating at 100 mW.The survey identifies an efficiency-versus-accuracy trade-off between these approaches.
  • Instantaneous Stereo: Event-based depth methods are demonstrated mainly with static cameras and few moving objects, where correspondences are easier to obtain; matching can run at approximately 1 kHz with low power.These results indicate promise for high-speed 3D reconstruction of moving objects or uncluttered scenes.
  • Monocular Depth Estimation: Monocular methods integrate events from a moving camera over time to recover semi-dense 3D structure and therefore require camera-motion knowledge.One space-sweep method instead back-projects events into a ray-density volume and identifies scene structure from local maxima without event matching or intensity-image recovery.
  • Stereo Depth for SLAM: Stereo depth for SLAM estimates semi-dense 3D structure by optimizing local spatio-temporal event consistency across image planes using time surfaces, without explicit event correspondences.The approach is presented as opening event-based stereo SLAM applications such as self-driving cars.
  • Depth Estimation using Structured Light: Active reconstruction uses emitted light, including pulsed line lasers and structured-light techniques, rather than relying only on passive scene illumination.These methods support applications such as fast terrain reconstruction and 3D line scanning.
  • Opportunities: Comparing event-based depth methods remains difficult because existing approaches are not evaluated on a common dataset.The survey calls for a comprehensive dataset, testbed, and benchmark to enable direct performance comparisons.

4.4 Pose Estimation and SLAM

Event-based pose estimation and SLAM progressed from restricted motion and scene settings toward full 6-DOF tracking and mapping, but robustness, scale, and loop closure remain open issues.

  • Pose Estimation and SLAM: Event-based SLAM is difficult because conventional-camera techniques such as feature detection, matching, and image alignment do not directly apply to fundamentally different event data.New techniques must exploit event cameras’ advantages in scenarios challenging for frame-based cameras.
  • Pose Estimation and SLAM: Event-based SLAM maps are naturally semi-dense edge maps because events arise from apparent motion of intensity edges, while edge properties and camera motion must be estimated jointly.Event cameras measure temporal changes rather than intensity gradients directly.
  • Camera Tracking Methods: Early camera-tracking systems addressed planar, rotational, or artificial line-based scenes using particle filters, Bayesian filters, or hand-crafted reprojection-based methods.These systems progressively expanded the motion and scene settings considered.
  • Pose Estimation and SLAM: Table 3 indicates that only [25] and [26] address the most general 6-DOF natural-scene tracking-and-mapping scenario using only events.The table distinguishes motion dimensionality, tracking, depth estimation, scene type, and additional requirements.
  • Camera Tracking Methods: Generative pose-estimation methods include robust probabilistic filtering and nonlinear photometric-error optimization, with the latter giving slightly better results.The comparison concerns brightness-increment image predictions given the scene map.
  • Tracking and Mapping: Full 6-DOF natural-scene SLAM uses interleaved probabilistic filters or real-time nonlinear optimization, but one filter-based system has limited robustness during initialization and high computational cost.Its limitation follows from assuming depth, intensity gradient, and camera motion are uncorrelated.
  • Tracking and Mapping: Processing event groups trades latency for efficiency, while simultaneous processing can also reduce noise.Probabilistic filters operate on small event groups, and nonlinear optimization methods process multiple events in real time on CPUs.
  • Opportunities: Existing event-based SLAM methods lack loop closure, and demonstrated scene scales remain considerably smaller than those of frame-based SLAM.The survey cautions that matching frame-based scale may not be a sensible goal because the sensors can be complementary.

4.5 Visual-Inertial Odometry (VIO)

Event-based visual-inertial odometry combines asynchronous events with synchronous high-rate IMU measurements through several temporal-fusion strategies and predominantly feature-based pipelines.

  • Visual-Inertial Odometry: Event-camera VIO improves robustness by rigidly combining event measurements with an IMU, but differs from VO because it drifts rather than losing its output pose.Some event cameras therefore integrate an IMU as a vestibular sensing modality.
  • Visual-Inertial Odometry: Temporal fusion of synchronous IMU data and asynchronous events uses asynchronous probabilistic filtering, IMU pre-integration, or continuous-time formulations.These options express measurements at compatible times or within a shared temporal framework.
  • Feature-based VIO: Most event-based VIO systems first extract feature tracks from events and then fuse those geometric trajectories with IMU data using established VIO algorithms.The front end converts event photometric information into geometric information for optimized geometric pipelines.
  • Feature-based VIO: Each successive feature-based VIO method outperforms its predecessor on the 6-DOF motion dataset, including a resource-constrained quadrotor demonstration in low-light and HDR scenarios.The later system also fuses intensity frames and recovers a camera trajectory with a sparse 3D landmark map.
  • Reprojection-error–based VIO: A reprojection-error approach fuses events and inertial data continuously by optimizing inertial- and event-reprojection terms over a camera-trajectory segment.Its formulation resembles visual-inertial bundle adjustment rather than feature-based processing.
  • Opportunities: The survey identifies direct recovery of motion and scene structure from events as an alternative to first converting events into geometric information.A suggested direction is joint optimization of photometric event-firing-rate and inertial error terms.
  • Opportunities: Stereo event-based VIO remains unexplored, while learning-based approaches are also an open direction in a literature dominated by model-based methods.The survey suggests combining event-based depth estimation with SLAM and VIO.

4.6 Image Reconstruction

Image reconstruction converts event streams into brightness images, supporting high-speed and high-dynamic-range scene recovery while requiring methods that address noise, missing offsets, and computational cost.

  • Principles: Event integration can recover absolute brightness because events encode non-redundant per-pixel brightness changes.An offset image at the interval start is normally needed to recover brightness increments.
  • Methods: Early reconstruction methods relied on rotational motion and static-scene assumptions, whereas later variational methods handled generic motions and scenes.The later approach jointly estimated image brightness and optical flow using motion-compensation and regularization terms.
  • Methods: Deep learning replaced hand-crafted motion restrictions and regularizers with perceptual data-driven priors that produced more natural-looking reconstructions.Recent methods also mitigated visual artefacts and achieved considerable gains over previous approaches.
  • Methods: Event-frame fusion combines frames’ low-frequency brightness with events’ high-frequency changes, improving temporal resolution and HDR information while limiting noise.Per-pixel temporal filtering can reduce corruption, and event timing can additionally remove motion blur from frames.
  • Applications and limitations: Reconstruction captures high-speed and HDR scenes but incurs computational cost, latency, and power consumption.Image quality is also affected by contrast-threshold noise, pixel mismatch, dynamic effects, and sensor spatial resolution.
  • Applications: Reconstructed representations can facilitate event correspondences and support recognition, SLAM, and optical-flow estimation, although reconstruction is not required for those tasks.Appearance maps can be more motion-invariant than raw events.

4.7 Motion Segmentation

Motion segmentation is difficult with moving event cameras because ego-motion generates events across the image alongside object motion. Methods progressively reduce prior information and use motion-consistent event structure to separate objects.

  • Problem: Moving cameras trigger events from both objects and the static scene, making event-wise causal classification challenging because each event carries little information.With a stationary camera and constant illumination, events can be attributed to moving objects more simply.
  • Approach: Segmentation methods become more unsupervised as they remove assumptions about known object shape or motion.Clustering jointly optimizes object motion parameters and event-object associations.
  • Approach: Distinctive event traces enable inference of object trajectories and segmentation through joint optimization of motion parameters and event assignments.This approach reduces reliance on externally specified object shape or expected motion.
  • Approach: Motion-compensated event images associate events with motion hypotheses by seeking sharp warped contours.Linear motion models are sufficiently expressive for short-time scenes containing objects with different motions.
  • Approach: Greedy fitting can cluster events by optical flow, while dominant-motion compensation can identify objects as inconsistencies with the background model.These methods have been tested in HDR and high-speed scenarios inaccessible to standard cameras.
  • Limitations: Segmentation remains relatively unexplored in event-based vision as increasingly complex scenes and techniques create open research challenges.The survey anticipates more work on this problem.

4.8 Recognition

Event-based recognition has progressed from simple shape detection to machine-learning and deep-learning recognition, but difficult dynamic and mobile-platform scenarios remain open challenges. Event representations often convert asynchronous spikes into tensors or frames for conventional models, while some systems operate directly on event data.

  • Methods: Recognition evolved from template matching simple shapes to classifying arbitrary edge patterns with hand-crafted features or deep learning.The progression increases expressibility and robustness to data distortions.
  • Representations: Learning-based approaches commonly convert events into dense tensors using time surfaces, histograms, average timestamps, exponential decay, or reconstructed images.These representations support image-based hierarchical models such as artificial neural networks.
  • Representations: Some recognition systems convert spikes to frames during inference, whereas others convert trained artificial neural networks into spiking networks operating directly on event data.The choice determines whether inference uses frame-like or native event processing.
  • Results and tasks: 98 % accuracy has been achieved on several event-based MNIST datasets, but digit recognition is not considered a strength of event-based vision.Harder datasets and recognition from moving vehicles remain comparatively uncommon and often use generated event frames with traditional deep learning.
  • Applications: Event recognition suits dynamic objects, gestures, and actions because these tasks include the temporal dimension in which event cameras excel.Such tasks are typically more challenging than static object recognition.
  • Limitations: Recognition remains a long way from competing with modern frame-based approaches, and useful applications on resource-constrained moving platforms remain an open challenge.Event-based and frame-based sensors have different strengths, so comparisons depend on the acquisition scenario.
  • Limitations: Event-based datasets still lack standardized annotation tools or formats, and frame-derived datasets may favor frame-based vision.Improved data-collection and annotation pipelines are identified as a significant field contribution.

4.9 Neuromorphic Control

Neuromorphic control architectures use event-based sensing and estimation to support low-latency, computationally efficient control, while event-based control theory remains only partly suited to embodied neuromorphic systems.

  • Neuromorphic-vision-driven Control Architecture: Neuromorphic control architectures combine a neuromorphic sensor, an event-based estimator, and a traditional controller that is unaware of asynchronicity.The estimator computes a state, which the controller uses to compute control.
  • Neuromorphic-vision-driven Control Architecture: Neuromorphic-vision-driven control has demonstrated low latency and computational efficiency in early and later systems.These architectures include spike-based target tracking and robot-goalie demonstrations.
  • Neuromorphic-vision-driven Control Architecture: A pencil-balancing robot estimated the pencil’s 3D state in below 1 ms latency and maintained balance with a hand-tuned PID controller on constrained hardware.Two DVS cameras observed the inverted pendulum, and the system was demonstrated on an embedded platform.
  • Event-based Control Theory: Event-based control changes the control signal asynchronously, with events generated exogenously or through self-triggered rules.One example recomputes control when trajectory tracking error exceeds a threshold.
  • Event-based Control Theory: Event-based control trades computation or communication cost against control performance, with diminishing returns from increasingly frequent control updates.Control frequency should depend on the time constant of the plant and sensor.
  • Event-based Control Theory: Existing event-based control theory has restricted utility for embodied neuromorphic systems because autonomous vehicles may process millions of events per second and exhibit nonlinearities.These nonlinearities prevent clean closed-form results, making specialization to neuromorphic settings a young research area.
  • Open questions in Neuromorphic Control: Open questions include task-driven sensing and integrating fast, low-latency control with slower deliberative components of an agent’s cognitive architecture.Current hardware offers limited software-modulated control of sensing, while fast sensorimotor processing remains weakly integrated with deliberative systems.

5 EVENT-BASED SYSTEMS AND APPLICATIONS

Event-based systems pair asynchronous sensors with neuromorphic processors or embedded computing to deliver low-latency, low-power perception and control in constrained applications.

  • 5.1 Neuromorphic Computing: Neuromorphic processors are natural partners for event cameras because both use inherently asynchronous spike-based computation.Neuromorphic engineering typically provides massively parallel hardware acceleration for spiking neural networks.
  • 5.1 Neuromorphic Computing: Neuromorphic processors span analog, digital, and software neuron implementations, with some architectures supporting on-chip learning.Evaluation should also consider the software development ecosystem and deployment workflow.
  • 5.1 Neuromorphic Computing: Widespread neuromorphic-processor adoption requires friendlier programming ecosystems, greater hardware capabilities, and broader device availability.The review identifies these as developments needed beyond early-access programs.
  • 5.1 Neuromorphic Computing: TrueNorth has produced end-to-end event-based vision systems for gesture recognition, stereo reconstruction, and optical flow estimation.The systems are described as low-power and low-latency.
  • 5.2 Applications in Real-Time On-Board Robotics: Speck combines a DVS with the Dynap-se neuromorphic CNN processor for near-sensor processing, with peak power below 1 mW and latency below 30 ms.Target applications include continuous object detection, surveillance, and automotive systems.
  • 5.2 Applications in Real-Time On-Board Robotics: Event cameras have supported autonomous landing and flight on quadrotors with limited computational resources in challenging scenes.A down-looking DAVIS was used for autonomous flight, and event speed and dynamic range supported difficult illumination conditions.

6 RESOURCES

Event-based vision resources include software libraries, datasets, simulators, and community activities, but standardization and benchmark coverage remain incomplete.

  • Software and community resources: The Event-based Vision Resources list collects papers, software, drivers, code, datasets, simulators, and other tools for the field.It is presented as a collaborative effort to organize relevant resources.
  • Software and community resources: No open-source standard library integrated with OpenCV provides event-based vision algorithms, despite the desirability of such a resource for adoption.Several developed open-source utilities nevertheless exist across different ecosystems.
  • Software and community resources: Available software includes jAER, libcaer, ROS DVS, event-driven YARP, pyAER, and DV for sensor access, processing, robotics integration, and deployment.These tools differ in language, middleware, supported hardware, and intended use.
  • Datasets and simulators: Datasets and simulators reduce the cost of event-camera research and enable quantitative benchmarking across tasks.They support motion estimation, image reconstruction, recognition, and end-to-end labeled scenarios such as driving.
  • Datasets and simulators: Optical-flow datasets have incomplete coverage because some provide only rotational motion, while others derive flow from camera motion and measured scene depth.The rotational-only setting lacks translational flow caused by parallax.
  • Datasets and simulators: Recognition datasets remain limited in size relative to traditional computer-vision datasets and cover small class sets or neuromorphic conversions of frame-based benchmarks.Examples include cards, faces, digits, gestures, and cars.
  • Datasets and simulators: Simulators generate event streams from virtual scenes and moving DAVIS trajectories, with extensions adding photorealism, noise, and estimated optical flow.These tools can also return intensity frames and depth maps.
  • Software and community resources: Summer schools, workshops, and conference demonstrations provide recurring venues for advancing event-based vision research.The review notes activity at neuromorphic workshops and major robotics and vision conferences.

7 DISCUSSION

Event-based vision spans multiple communities and offers potential benefits in latency, power, and information availability, but its methods, end-to-end integration, and deployment remain unsettled.

  • Discussion: Different communities prioritize event-based vision’s low power, low latency, or high temporal-resolution and HDR information according to their application needs.These priorities range from always-on embedded systems to highly reactive systems and richer environmental perception.
  • Discussion: Event cameras show potential for scenarios previously inaccessible to frame-based cameras, although comparisons remain uneven because frame-camera hardware and software are more mature.The review emphasizes considerable room for further research and development.
  • Discussion: No single event-processing method or representation is best across applications because latency, power, accuracy, sensitivity, bandwidth, and processing capacity trade off.Lower contrast thresholds or higher resolution produce more events for finite-capacity algorithms and platforms.
  • Discussion: A major challenge is developing efficient, natively event-based systems spanning perception, control, and actuation while adapting deep-learning methods to event data.Active vision is especially relevant because events depend distinctly on motion, including robot actuation.
  • Discussion: Near-sensor event processing could transmit only high-level, non-redundant information, reducing bandwidth, latency, and power consumption.The review points to sensor-integrated hardware such as Speck and cellular processor arrays as possible implementations.

8 CONCLUSION

Event cameras offer low latency, low power, high speed, and high dynamic range, motivating their use in challenging computer-vision and robotic applications. The paper surveys event-based perception, computing, and control while identifying continuing challenges and opportunities.

  • Event cameras combine low latency, low power, high speed, and high dynamic range, offering potential where traditional cameras remain limited.
  • The overview covers perception, computing, and control, emphasizing event-camera principles and algorithms across low- and high-level vision.
  • Neuromorphic perception and control remain emerging topics with many challenges and opportunities for more efficient, bio-inspired machine interaction.
Loading 1904.08405v3…