Source-linked AI summary
Recent Event Camera Innovations: A Survey
Bharatesh Chakravarthi, Aayush Atul Verma, Kostas Daniilidis, Cornelia Fermuller, Yezhou Yang
TL;DR
Researchers seeking a consolidated view of event cameras need an account of their principles, models, applications, datasets, and simulators. This survey reviews those areas and concludes that event cameras have significantly impacted visual sensing, with diverse applications demonstrating flexibility and potential.
Problem
Researchers need a consolidated understanding of event-camera principles, models, applications, datasets, and simulators.
Source: §1 · pp. 2–3Method
The survey reviews event-camera operation and comparisons, models, milestones, applications, datasets, simulators, and supporting resources.
Source: §1 · pp. 2–3Results
The survey concludes that event cameras have significantly impacted visual sensing, and their diverse applications demonstrate flexibility and potential.
Source: §10 · p. 14Takeaways & Limitations
The authors state that consolidating and sharing knowledge is essential for addressing new challenges and promoting innovation, and describe the GitHub page as a resource for researchers.
Source: §1 · p. 3, §10 · p. 14
Abstract
from arXiv · showhide
Event-based vision, inspired by the human visual system, offers transformative capabilities such as low latency, high dynamic range, and reduced power consumption. This paper presents a comprehensive survey of event cameras, tracing their evolution over time. It introduces the fundamental principles of event cameras, compares them with traditional frame cameras, and highlights their unique characteristics and operational differences. The survey covers various event camera models from leading manufacturers, key technological milestones, and influential research contributions. It explores diverse application areas across different domains and discusses essential real-world and synthetic datasets for research advancement. Additionally, the role of event camera simulators in testing and development is discussed. This survey aims to consolidate the current state of event cameras and inspire further innovation in this rapidly evolving field. To support the research community, a GitHub page (https://github.com/chakravarthi589/Event-based-Vision_Resources) categorizes past and future research articles and consolidates valuable resources.
1 Understanding Event-based Vision – An Introduction
The introduction describes event cameras as asynchronous sensors that report pixel-level light changes, then outlines the survey’s coverage of their operation, models, applications, datasets, simulators, and supporting resources.
- Event cameras asynchronously report significant light-intensity changes at each pixel, capturing dynamic scenes without the motion blur typical of frame cameras.Because they focus on changes rather than static information, they can reduce redundancy and bandwidth requirements.Source: §1 · pp. 1–2
- Logarithmic change detection supports operation across varied lighting, while real-time event processing can reduce data bandwidth and computational power needs.The introduction notes relevance to battery-powered devices and long-term monitoring.Source: §1 · p. 2
- The survey covers research trends, camera operation and frame-camera comparisons, manufacturer models, milestones, applications, datasets, and simulators.Its stated scope includes both practical applications and research resources.Source: §1 · pp. 2–3
- A companion GitHub page consolidates event-vision resources and is intended to be updated with developments to support informed decisions and continued innovation.Source: §1 · p. 3
2 The Rise of Event-based Vision: A Background
The background traces event-based vision from early neuromorphic work through later technical milestones and documents increasing publication activity, especially from 2019 onward.
- Event-based vision publications increased substantially from 2019 onward, reaching a 2023 peak as cameras and simulators became more available.The passage also notes growing representation at major computer-vision conferences and specialized workshops.Source: §2 · p. 3
- Late-1990s and early-2000s neuromorphic vision work included sensors, spiking controllers, biologically inspired systems, and AER dynamic vision sensors.The period also included a frame-free dynamic vision discussion, stereo vision, and activity-driven sensors.Source: §2 · p. 3
- From the early 2010s to 2020, milestones included asynchronous stereo, robotic vision, event-based flow and SLAM, tracking, and recognition methods.The account spans embedded hardware and algorithms for visual tasks.Source: §2 · pp. 3–4
- Figure 2 depicts independent pixels converting light to voltage and event generation as a function of logarithmic light intensity over time.Source: Figure 2 · p. 4
- The account highlights mid-2010s advances in continuous-time trajectory estimation, stereo matching, event-driven classification, noise filtering, and low-latency tracking.It also notes late-2010s surveys and spatiotemporal feature learning before the rapid arrival of cameras and simulators in the early 2020s.Source: §2 · p. 4
3 How Event Cameras Work: An Inside Look
Each event-camera pixel independently processes incoming light and emits an asynchronous event when a logarithmic intensity change crosses a threshold, forming a sparse spatiotemporal stream.
- Each pixel converts incoming light into voltage and compares it continuously with a reference to detect logarithmic intensity changes.Pixel operation is independent, continuous, and asynchronous.Source: §3 · p. 4
- A threshold crossing triggers an event containing pixel coordinates, change time, and polarity indicating whether light intensity increased or decreased.Events are output as they occur, producing a continuous stream rather than static frames.Source: §3 · pp. 4–5
- The event stream can be represented in spatial and temporal coordinates, and its sparse structure minimizes redundancy for processing scene dynamics.Source: §3 · p. 5
4 Event Camera vs. Frame Camera: A Comparison
The comparison attributes event cameras’ low latency, reduced motion blur, efficient data output, and lighting robustness to their change-driven, asynchronous operation.
- Event cameras exceed 10,000 fps in temporal resolution and respond immediately to changes, reducing latency and motion blur relative to frame cameras.The passage identifies robotics and autonomous driving as applications requiring rapid response.Source: §4 · p. 5
- By recording scene changes rather than full frames, event cameras produce less data, while high frame rates in frame cameras can require substantial bandwidth and storage.The reduced output is particularly relevant to embedded and edge systems with limited resources.Source: §4 · pp. 5–6
- Event sensors have a dynamic range above 120 dB, compared with no more than 95 dB for high-quality frame cameras.The passage links logarithmic intensity-change sensing with robustness to challenging lighting, including outdoor scenes.Source: §4 · p. 6
- Event cameras’ low-light cutoff is 0.08 Lux, a capability that has prompted exploration of low-light applications.Source: §4 · p. 6
5 Event Camera Models: An Overview
The survey reviews event-camera models from iniVation and Prophesee, comparing their features and outlining associated software and development kits.
5 Event Camera Models: An Overview
- The survey focuses on iniVation and Prophesee models after noting other manufacturers and the prominence of these two companies in research.It names the DAVIS 346, Prophesee EVK4, and DAVIS 240 among prominent models.Source: §5 · p. 6
- iniVation’s lineup spans event-only and event-plus-frame cameras, with stated differences in resolution, dynamic range, event rate, and sensor integration.The DVXplorer is listed at VGA, 110 dB, and 165 million events per second; the DAVIS 346 AER provides event and frame output with a 120 dB dynamic range.Source: §5 · p. 6
- iniVation no longer offers several earlier models, while its software includes visualization, C++/Python processing, ROS integration, and a low-level library.Discontinued models listed include the DVXplorer Mini, DVS 240, DAVIS 240, eDVS, DVS 128, DVL-5000.Source: §5 · pp. 6–7
- Prophesee’s offerings include USB cameras and embedded starter kits, with the Metavision SDK providing visualization tools, guides, and C++/Python APIs.The kits include an FPGA-based development option and an STM32-F7 MCU option described for low-power applications.Source: §5 · pp. 7–8
6 Pioneering Progress: Milestones in Event-based Vision
The survey traces milestones from early gesture-recognition systems, datasets, and simulators to newer architectures, reconstruction methods, and expanded data resources. It describes progress across applications including tracking, optical flow, and 3D reconstruction.
6 Pioneering Progress: Milestones in Event-based Vision
- From 2017–2019, milestones included event-based gesture recognition, CIFAR10-DVS, ESIM, EV-FlowNet [301], and event-based video reconstruction methods.Other work introduced a dataset and simulator and methods for optical flow and depth prediction.Source: §6 · pp. 8–9
- In 2020–2021, milestones included a 1Mpx dataset and recurrent architecture, the DSEC stereo dataset, and v2e for synthesizing DVS events from intensity frames.Other advances included synthetic event data for detection and segmentation, image reconstruction, Time Lens, event-based stereo-visual odometry, and N-ImageNet.Source: §6 · p. 9
- In 2022–2023, milestones included spiking-transformer tracking, graph-based event processing, EventNeRF, and recurrent vision transformers for object detection.The survey also notes restoration, interpolation, and neural-radiance-field approaches.Source: §6 · p. 9
- The 2024 milestones include high-resolution tracking data and hierarchical knowledge distillation, synthetic multi-view traffic data [2], and 10 hr of traffic-monitoring data.The survey presents these developments as evidence of rapid progress and growing potential.Source: §6 · p. 9
7 Event Cameras in Action: Diverse Tasks and Impacts
The survey organizes event-based vision applications across recognition, estimation, segmentation, reconstruction, and navigation. It connects event-camera characteristics and data fusion with work spanning these task areas.
7 Event Cameras in Action: Diverse Tasks and Impacts
- Detection and tracking work includes scene-adaptive sparse transformers, spiking and recurrent vision transformers, and self-supervised learning for settings including surveillance and autonomous driving.The survey also describes classification and recognition advances in object, gesture, gait, and action tasks.Source: §7 · p. 10
- Optical-flow, motion, pose, and depth estimation draw on event-camera speed and low latency, with cited advances including spatiotemporal alignment and contrast maximization.The survey also covers event stereo, photometric stereo, shape from polarization, segmentation, and fusion with frame, lidar, or infrared data.Source: §7 · pp. 10–11
- The survey covers reconstruction, deblurring, frame interpolation, denoising, video generation, event conversion, super-resolution, visual odometry, and SLAM applications.It points to notable works across these tasks, including video reconstruction, event denoising, and visual odometry.Source: §7 · p. 11
8 Data for Innovation: Event-based Vision Datasets
The survey reviews real-world datasets captured with event cameras and synthetic datasets generated using simulators, describing both as resources for training and evaluation.
8 Data for Innovation: Event-based Vision Datasets
- The survey distinguishes real-world event-camera recordings from synthetic datasets generated by simulators for controlled experimentation.It points to summary tables and a GitHub page for additional dataset listings.Source: §8 · p. 12
8.1 Real-world Datasets
- Real-world datasets cover tracking, traffic monitoring, action recognition, lip reading, stereo driving, automotive detection, gestures, and pose estimation, visual odometry, and SLAM.Examples include EventVOT, eTraM, DSEC, GEN1 [41], DvsGesture, and the Event Camera Dataset.Source: §8.1 · pp. 12–13
8.2 Synthetic Datasets
- Synthetic datasets include multi-view traffic data from CARLA [2], Event-KITTI, ESfP-Synthetic, N-ImageNet, CIFAR10-DVS, N-MNIST, and N-Caltech.The survey describes these datasets as supporting varied traffic-perception, shape-from-polarization, and recognition research.Source: §8.2 · p. 13
9 Simulating Reality: The Event-based Simulators
Event-based simulators provide synthetic data for validating algorithms and exploring applications in controlled, cost-effective settings. The survey reviews simulators that generate event streams and ground truth, model sensor noise, or convert conventional video into event data.
9 Simulating Reality: The Event-based Simulators
- Simulators support algorithm validation and application exploration with synthetic data in controlled, cost-effective settings.The survey presents them as important to advancing event-based vision systems.Source: §9 · p. 13
- DAVIS Simulator generates event streams, intensity frames, and depth maps through time interpolation; ESIM models camera motion in 3D scenes and produces events with comprehensive ground truth.Source: §9 · p. 14
- v2e, ICNS, DVS-Voltmeter, and V2CE [291] address video-to-event conversion or realistic sensor noise and timing.CARLA DVS simulates high-frequency event generation with microsecond resolution, while Prophesee Video to Event Simulator converts frame-based video; Table 6 summarizes open-source simulators.Source: §9 · p. 14, Table 6 · p. 14
10 Conclusion
The conclusion consolidates the survey’s coverage of event-camera evolution, operation, models, milestones, applications, datasets, and simulators. It emphasizes shared knowledge and an updated GitHub resource as supports for future research and innovation.
10 Conclusion
- The survey outlines event-camera evolution, operational principles, differences from frame cameras, models, and key milestones.Source: §10 · p. 14
- Applications across fields demonstrate event cameras’ flexibility and potential, while real-world and synthetic datasets and simulators support research, testing, and development.Source: §10 · p. 14
- The GitHub page provides research resources and is intended to be continuously updated with ongoing work and relevant materials.The conclusion identifies knowledge consolidation and sharing as important for addressing new challenges and promoting innovation.Source: §10 · p. 14