Source-linked AI summary
LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices
Riadul Islam, Joey Mule, Dhandeep Challagundla, Shahmir Rizvi, Sean Carson, Rachit Saini
TL;DR
Sparse, asynchronous, and noisy event streams challenge conventional models on low-power embedded platforms. The paper introduces a compact configurable autoencoder classifier and evaluates it across event thresholds, datasets, and hardware, achieving competitive accuracy with substantially lower model size and energy use.
Problem
Sparse, asynchronous, noisy event streams make conventional deep learning models inefficient for accurate, power-constrained edge inference.
Method
The paper uses a compact convolutional event autoencoder with an integrated classifier, dense event representations, adaptive thresholds, and embedded hardware evaluation.
Results
The proposed classifier reaches about 93% accuracy across thresholds, remains within 4–6% of YOLOv9 with 35.6× fewer parameters, and uses 16.19 J—approximately 726.3× less energy than YOLOv9.
Takeaways & Limitations
The system provides a deployable event-driven autoencoder classifier for real-time, low-power perception on resource-constrained vision hardware.
Abstract
from arXiv · showhide
Event-based vision has emerged as a promising paradigm for energy-aware artificial intelligence (AI), offering sparse, low-latency visual signals that reduce redundant data processing and support sustainable edge computing. However, the asynchronous and noise-prone nature of event streams creates challenges for conventional deep learning models, which are often too computationally intensive for low-power embedded platforms. This work presents a compact and configurable event-driven autoencoder that efficiently compresses neuromorphic data while preserving essential spatiotemporal structure for downstream inference. The architecture integrates lightweight convolutional encoding with robust performance under adaptive event thresholding and a minimal classifier head, enabling substantial reductions in computational cost without degrading recognition fidelity. Extensive evaluations on the Smart Event Face Dataset (SEFD) and Event-Based Crossing Dataset (EBCD) show that the proposed framework achieves competitive or superior accuracy compared to YOLOv9 while requiring up to 35.6$\times$ fewer parameters. To assess real-world sustainability, the model is deployed on resource-constrained hardware: a Raspberry Pi 4B and a NVIDIA Jetson Nano. On NVIDIA Jetson Nano, it delivers real-time throughput of 44.8 FPS. On a Raspberry Pi 4B CPU, the 50\% autoencoder classifier consumes 16.19 J for the evaluated inference workload, corresponding to approximately 726.3$\times$ lower energy consumption than YOLOv9 under the same evaluation protocol. These results demonstrate the potential of compact event-driven models to advance environmentally conscious, low-power AI systems for high-speed perception in autonomous, mobile, and embedded computing environments.
I. INTRODUCTION
Event-based vision addresses the latency and redundancy limits of frame-based sensing, but sparse, asynchronous, noisy streams require lightweight models for accurate, power-efficient edge inference. This work introduces a configurable event-based autoencoder classifier designed to compress event data and support efficient embedded deployment.
- Event-driven sensors asynchronously register per-pixel brightness changes, enabling continuous motion tracking with low latency and high temporal fidelity.
- Sparse, asynchronous, noisy event streams make direct use of conventional CNNs inefficient and often inaccurate on embedded devices.
- Static event frames allow standard convolutions while jointly addressing representation efficiency, latent feature compression, and edge inference complexity.
- The proposed configurable autoencoder is tailored for event-based object classification and adapts across multiple event activity thresholds.
- The proposed event classifier achieves up to 35.6× fewer parameters, 22.07× higher frame rates, and over 700× lower energy consumption than compared SOTA CNNs.
A. Existing Event Vision Applications
Event vision has been applied across automotive, surveillance, classification, gesture, action, flow, and face-detection tasks. This research focuses on multi-threshold datasets to characterize robustness under adaptive event sensitivity.
- Event vision supports applications including automotive sensing, surveillance, object classification, gesture recognition, action detection, flow detection, and face detection.
- Most event vision datasets concentrate on single-threshold data, whereas this research examines multi-threshold datasets for robust autoencoder characterization.
- Adaptive threshold selection dynamically modulates event detection sensitivity in response to environmental conditions and signal characteristics.
B. Existing Event Vision Object Classification Methods
Event vision object detection models use sparse or dense representations: sparse formats retain continuous-stream temporal information, while dense formats create frame-like tensors compatible with standard vision methods.
- Sparse event representations seek to exploit temporal information in continuous event streams.
- Dense event representations coalesce asynchronous streams into structured, frame-like tensors for standard vision processing.
- Dense representations can leverage advances from traditional computer vision because they resemble conventional image frames.
C. Existing Evaluation Methods
Event-vision evaluation commonly uses benchmark datasets and computer-vision metrics alongside parameter count and runtime. Prior recurrent and encoder-fusion approaches can be computationally and energetically burdensome for resource-limited platforms.
- Evaluation typically characterizes event-vision models using MS COCO metrics, parameter count, and runtime across datasets such as SEFD and EBCD.
- CNN-based autoencoders have been used for medical image compression and diagnostic classification, illustrating autoencoder applications beyond event vision.
- RNN-based autoencoders encode temporal dependencies in event streams for optical-flow estimation and tracking.
- RNN models and encoder-fusion pipelines can impose substantial computational and power demands that hinder deployment on low-cost embedded platforms.
III. PROPOSED METHODOLOGY
The proposed methodology uses a convolutional autoencoder to compress event representations into latent features and reconstruct the input. A pretrained encoder can then serve as a fixed feature extractor for a lightweight classifier suited to resource-constrained devices.
- Autoencoder design: The autoencoder learns compact visual representations by reconstructing inputs from a compressed latent space.Its encoder maps high-dimensional data to a lower-dimensional embedding, while the decoder reconstructs the original input.
- Feature transformation: The encoder feature map is computed from the previous layer using convolution kernels, biases, and a nonlinear activation function.The initial feature map is the input x, and the encoder output is the latent representation z = h(L).
- Decoder: The decoder reconstructs event frames by progressively upsampling latent features using transposed convolutions.The reconstruction objective encourages retention of salient spatial structure and activity patterns while discarding high-frequency noise and threshold-induced artifacts.
- Encoder: The encoder progressively condenses each event frame through convolutional blocks with ReLU activation and spatial downsampling.ReLU introduces nonlinearity, suppresses noise through thresholding, and promotes sparse activations aligned with event data.
- Autoencoder-based classifier: The classifier reuses the pretrained encoder as a fixed feature extractor and appends two fully connected layers to the latent output.This separates representation learning from task-specific discrimination and minimizes parameter count for real-time embedded inference.
IV. EXPERIMENTAL RESULTS AND DISCUSSION
Experiments evaluate the configurable event autoencoder on SEFD and EBCD, compare it with established detectors, and assess reconstruction, threshold robustness, embedded throughput, energy use, and encoder-size scaling. The results show competitive recognition and reconstruction with substantially lower model complexity and energy consumption, alongside faster inference on embedded platforms.
- Experimental setup: The study trains and tests the proposed autoencoder classifier and baseline detectors on SEFD and EBCD using threshold-generated event frames.Baselines include YOLOv4, YOLOv7, YOLOv9, EfficientDet-b0, MobileNet-v1, and YuNet.
- Recognition performance: YOLOv9 achieves the best accuracy across SEFD and EBCD, while the proposed classifier uses 35.6× fewer parameters than YOLOv9.The proposed model achieves higher accuracy than several other baselines and remains within approximately 4–6% of YOLOv9 on EBCD.
- Training and scaling: 0.0884 average training loss is achieved by the 50% filter model, compared with 0.1593 for the full-capacity autoencoder.Reducing the architecture to 50% filters produces a fourfold reduction in overall model size.
- Reconstruction quality: SSIM decreases from 0.939 to 0.915 on SEFD when reducing from the 100% to 50% autoencoder, while the model size decreases by 4×.On EBCD, the 50% model’s SSIM is only 0.028 lower than the full-sized model.
- Embedded efficiency: 16.19 J is consumed by the 50% autoencoder classifier, making it approximately 726.3× more energy efficient than YOLOv9.The energy evaluation averages ten runs of sequential inference on the same 100 test images for all models.
- Training and scaling: A 25% encoder configuration further reduces parameters and FLOPs while maintaining competitive recognition performance.The ablation evaluates progressively reduced convolutional-filter capacity beyond the 50% model.
V. CONCLUSION
The deployable event-driven autoencoder achieves robust classification, reconstruction, and embedded-hardware efficiency under substantial parameter reduction. The paper identifies richer event-based perception scenarios for real-time detection as future work.
- 93% classification accuracy across multiple thresholds outperforms YOLOv4, YOLOv7, MobileNet-v1, and YuNet while remaining within 4–6% of YOLOv9.
- 35.6× fewer parameters than YOLOv9 accompanies competitive classification accuracy on the SEFD and EBCD datasets.
- 99.97% reconstruction accuracy with the full model remains 93.54% at 50% of the filters, indicating resilience under aggressive parameter reduction.
- Embedded deployment reaches up to 87.84× higher FPS than MobileNet-v1 and over 700× lower energy consumption than YOLOv9.
- Future work will explore richer event-based perception scenarios for real-time detection.