PaperlayerView exact revision on arXiv

Source-linked AI summary

LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices

Riadul Islam, Joey Mule, Dhandeep Challagundla, Shahmir Rizvi, Sean Carson, Rachit Saini

arXiv:2608.21764v1cs.CVcs.AIeess.IV

TL;DR

Sparse, asynchronous, and noisy event streams challenge conventional models on low-power embedded platforms. The paper introduces a compact configurable autoencoder classifier and evaluates it across event thresholds, datasets, and hardware, achieving competitive accuracy with substantially lower model size and energy use.

  • Problem

    Sparse, asynchronous, noisy event streams make conventional deep learning models inefficient for accurate, power-constrained edge inference.

    Source: §I · p. 1, C. Existing Evaluation Methods · p. 3

  • Method

    The paper uses a compact convolutional event autoencoder with an integrated classifier, dense event representations, adaptive thresholds, and embedded hardware evaluation.

    Source: §III · p. 3, §IV · pp. 6, 8

  • Results

    The proposed classifier reaches about 93% accuracy across thresholds, remains within 4–6% of YOLOv9 with 35.6× fewer parameters, and uses 16.19 J—approximately 726.3× less energy than YOLOv9.

    Source: §IV · pp. 5, 8, §V · p. 9

  • Takeaways & Limitations

    The system provides a deployable event-driven autoencoder classifier for real-time, low-power perception on resource-constrained vision hardware.

    Source: §IV · p. 7, §V · p. 9

Abstract

from arXiv · show

Event-based vision has emerged as a promising paradigm for energy-aware artificial intelligence (AI), offering sparse, low-latency visual signals that reduce redundant data processing and support sustainable edge computing. However, the asynchronous and noise-prone nature of event streams creates challenges for conventional deep learning models, which are often too computationally intensive for low-power embedded platforms. This work presents a compact and configurable event-driven autoencoder that efficiently compresses neuromorphic data while preserving essential spatiotemporal structure for downstream inference. The architecture integrates lightweight convolutional encoding with robust performance under adaptive event thresholding and a minimal classifier head, enabling substantial reductions in computational cost without degrading recognition fidelity. Extensive evaluations on the Smart Event Face Dataset (SEFD) and Event-Based Crossing Dataset (EBCD) show that the proposed framework achieves competitive or superior accuracy compared to YOLOv9 while requiring up to 35.6$\times$ fewer parameters. To assess real-world sustainability, the model is deployed on resource-constrained hardware: a Raspberry Pi 4B and a NVIDIA Jetson Nano. On NVIDIA Jetson Nano, it delivers real-time throughput of 44.8 FPS. On a Raspberry Pi 4B CPU, the 50\% autoencoder classifier consumes 16.19 J for the evaluated inference workload, corresponding to approximately 726.3$\times$ lower energy consumption than YOLOv9 under the same evaluation protocol. These results demonstrate the potential of compact event-driven models to advance environmentally conscious, low-power AI systems for high-speed perception in autonomous, mobile, and embedded computing environments.

I. INTRODUCTION

Event-based vision addresses the latency and redundancy limits of frame-based sensing, but sparse, asynchronous, noisy streams require lightweight models for accurate, power-efficient edge inference. This work introduces a configurable event-based autoencoder classifier designed to compress event data and support efficient embedded deployment.

  • Event-driven sensors asynchronously register per-pixel brightness changes, enabling continuous motion tracking with low latency and high temporal fidelity.Source: §I · p. 1
  • Sparse, asynchronous, noisy event streams make direct use of conventional CNNs inefficient and often inaccurate on embedded devices.Source: §I · p. 1
  • Static event frames allow standard convolutions while jointly addressing representation efficiency, latent feature compression, and edge inference complexity.Source: §I · p. 2
  • The proposed configurable autoencoder is tailored for event-based object classification and adapts across multiple event activity thresholds.Source: §I · p. 2
  • The proposed event classifier achieves up to 35.6× fewer parameters, 22.07× higher frame rates, and over 700× lower energy consumption than compared SOTA CNNs.Source: §I · p. 2

A. Existing Event Vision Applications

Event vision has been applied across automotive, surveillance, classification, gesture, action, flow, and face-detection tasks. This research focuses on multi-threshold datasets to characterize robustness under adaptive event sensitivity.

B. Existing Event Vision Object Classification Methods

Event vision object detection models use sparse or dense representations: sparse formats retain continuous-stream temporal information, while dense formats create frame-like tensors compatible with standard vision methods.

C. Existing Evaluation Methods

Event-vision evaluation commonly uses benchmark datasets and computer-vision metrics alongside parameter count and runtime. Prior recurrent and encoder-fusion approaches can be computationally and energetically burdensome for resource-limited platforms.

III. PROPOSED METHODOLOGY

The proposed methodology uses a convolutional autoencoder to compress event representations into latent features and reconstruct the input. A pretrained encoder can then serve as a fixed feature extractor for a lightweight classifier suited to resource-constrained devices.

Autoencoder design

  • The autoencoder learns compact visual representations by reconstructing inputs from a compressed latent space.Its encoder maps high-dimensional data to a lower-dimensional embedding, while the decoder reconstructs the original input.Source: §III · p. 3

Feature transformation

  • The encoder feature map is computed from the previous layer using convolution kernels, biases, and a nonlinear activation function.The initial feature map is the input x, and the encoder output is the latent representation z = h(L).Source: §III · p. 3

Decoder

  • The decoder reconstructs event frames by progressively upsampling latent features using transposed convolutions.The reconstruction objective encourages retention of salient spatial structure and activity patterns while discarding high-frequency noise and threshold-induced artifacts.Source: §III · p. 3

Encoder

  • The encoder progressively condenses each event frame through convolutional blocks with ReLU activation and spatial downsampling.ReLU introduces nonlinearity, suppresses noise through thresholding, and promotes sparse activations aligned with event data.Source: §III · p. 3

Autoencoder-based classifier

  • The classifier reuses the pretrained encoder as a fixed feature extractor and appends two fully connected layers to the latent output.This separates representation learning from task-specific discrimination and minimizes parameter count for real-time embedded inference.Source: §III · pp. 3–4, Figure 2 · p. 4

IV. EXPERIMENTAL RESULTS AND DISCUSSION

Experiments evaluate the configurable event autoencoder on SEFD and EBCD, compare it with established detectors, and assess reconstruction, threshold robustness, embedded throughput, energy use, and encoder-size scaling. The results show competitive recognition and reconstruction with substantially lower model complexity and energy consumption, alongside faster inference on embedded platforms.

Experimental setup

  • The study trains and tests the proposed autoencoder classifier and baseline detectors on SEFD and EBCD using threshold-generated event frames.Baselines include YOLOv4, YOLOv7, YOLOv9, EfficientDet-b0, MobileNet-v1, and YuNet.Source: §IV · p. 4

Recognition performance

  • YOLOv9 achieves the best accuracy across SEFD and EBCD, while the proposed classifier uses 35.6× fewer parameters than YOLOv9.The proposed model achieves higher accuracy than several other baselines and remains within approximately 4–6% of YOLOv9 on EBCD.Source: §IV · p. 5

Training and scaling

  • 0.0884 average training loss is achieved by the 50% filter model, compared with 0.1593 for the full-capacity autoencoder.Reducing the architecture to 50% filters produces a fourfold reduction in overall model size.Source: §IV · p. 5

Reconstruction quality

  • SSIM decreases from 0.939 to 0.915 on SEFD when reducing from the 100% to 50% autoencoder, while the model size decreases by 4×.On EBCD, the 50% model’s SSIM is only 0.028 lower than the full-sized model.Source: §IV · pp. 5–6

Embedded efficiency

  • 16.19 J is consumed by the 50% autoencoder classifier, making it approximately 726.3× more energy efficient than YOLOv9.The energy evaluation averages ten runs of sequential inference on the same 100 test images for all models.Source: §IV · p. 8

Training and scaling

  • A 25% encoder configuration further reduces parameters and FLOPs while maintaining competitive recognition performance.The ablation evaluates progressively reduced convolutional-filter capacity beyond the 50% model.Source: §IV · pp. 8–9

V. CONCLUSION

The deployable event-driven autoencoder achieves robust classification, reconstruction, and embedded-hardware efficiency under substantial parameter reduction. The paper identifies richer event-based perception scenarios for real-time detection as future work.

  • 93% classification accuracy across multiple thresholds outperforms YOLOv4, YOLOv7, MobileNet-v1, and YuNet while remaining within 4–6% of YOLOv9.Source: §V · p. 9
  • 35.6× fewer parameters than YOLOv9 accompanies competitive classification accuracy on the SEFD and EBCD datasets.Source: §V · p. 9
  • 99.97% reconstruction accuracy with the full model remains 93.54% at 50% of the filters, indicating resilience under aggressive parameter reduction.Source: §V · p. 9
  • Embedded deployment reaches up to 87.84× higher FPS than MobileNet-v1 and over 700× lower energy consumption than YOLOv9.Source: §V · p. 9
  • Future work will explore richer event-based perception scenarios for real-time detection.Source: §V · p. 9

Related papers

Loading 2608.21764v1…