Source-linked AI summary
ANTShapes Benchmarking Datasets for Event-Based Neuromorphic Object Classification
M. Middleton, H. Kayan, B. Sen Bhattacharya, T. Ali, E. Baikas, M. Vousden, C. Perera, O. Rhodes, E. Gheorghiu, M. A. Trefzer
TL;DR
Event-based object classification lacks datasets that specifically target the task, while existing datasets can have limited temporal structure or mismatched classification targets. This paper creates four ANTShapes datasets of varying difficulty, benchmarks them with existing event-based datasets using a convolutional SNN, and finds comparable classification results while validating ANTShapes for event-based computer vision experiments.
Problem
Existing event-based vision datasets do not specifically target object classification, and some lack meaningful temporal structure or use action rather than object classes.
Method
The paper creates four ANTShapes datasets with varying visual challenges and benchmarks them against established event-based datasets using a convolutional SNN.
Results
The four ANTShapes datasets achieve comparable results to existing object-classification dataset approximations, validating the simulation tool as suitable for event-based computer vision tasks.
Takeaways & Limitations
ANTShapes provides high-resolution datasets with many classes and examples for future event-based object-classification experiments.
Takeaways & Limitations
N-MNIST and CIFAR10-DVS contain no meaningful temporal structure within samples because their apparent motion is induced by camera movement rather than natural scene dynamics.
Abstract
from arXiv · showhide
Object classification in event-based computer vision is a task that is attracting considerable research attention. Event-based object classification is a fundamental task in the fields of security and applied computer vision, which typically use synchronous frame-based cameras and computing pipelines for operation. This approach has several practical flaws. The size, weight and power consumption of the device could prohibit deployment at the extreme edge or in covert sensing environments. Besides this, there are security concerns inherent in cloud-based or other off-device computation approaches due to the requirement of sending and receiving potentially sensitive data. Furthermore, this transmission of data introduces latency and requires consistent connectivity to the cloud infrastructure to function. The use of Spiking Neural Networks (SNNs) hosted on neuromorphic devices attempts to solve several issues present in this conventional approach. Research into event-based object classification methods are hindered by the lack of high-quality vision datasets to use. To this end, the ANTShapes simulation tool has been previously proposed to create and label event-based vision datasets. In this paper, four novel datasets of varying difficulties are created using the tool and are benchmarked against existing spiking datasets commonly used for event-based vision research (N-MNIST, CIFAR10-DVS, DVSGesture and POKER-DVS). Classification is performed using a convolutional SNN. This work simultaneously provides four datasets with rich details for future experiments to use and validates the output of the ANTShapes dataset simulation tool as being suitable for its purpose.
1 Introduction
Event-based object classification is attractive for low-power, low-latency edge vision, but progress is constrained by limited benchmark datasets that capture suitable spatiotemporal object features.
- SNN inference on neuromorphic hardware can operate at microsecond time scales and use less power than conventional deep-learning hardware.
- Event-driven neuromorphic computation suits extreme-edge deployments where device size, weight, power, or latency must be tightly managed.
- Object classification supports scene understanding, autonomous navigation, and anomaly detection in security and monitoring applications.
- Existing neuromorphic datasets lack naturally evolving spatiotemporal features and purpose-built benchmarks for classifying 3D objects.
2 Background
The four established neuromorphic datasets differ in data generation and task relevance, but none specifically targets full 3D object classification. Their limitations motivate simulated ANTShapes datasets with controlled noise and richer temporal structure.
- The benchmarked datasets are N-MNIST, CIFAR10-DVS, POKER-DVS and DVSGesture, which are frequently used for event-based classification research.
- N-MNIST: N-MNIST converts moving-camera recordings of static handwritten digits into event streams, with 60,000 training and 10,000 test samples across 10 classes.Each event records x and y address, timestamp, and polarity; recordings span approximately 360 ms.
- N-MNIST: N-MNIST contains little meaningful temporal information because its static scene content generates events from sensor motion.This limits its suitability as a proxy for 3D object classification, which can involve varying viewing angles and richer spatial detail.
- CIFAR10-DVS: CIFAR10-DVS provides richer event structure from complex image textures, but its source images remain spatial rather than encoding evolving scenes.Its positive-polarity event dominance can also create class confusions and overlap in feature clusters.
- POKER-DVS and DVSGesture: POKER-DVS contains real spatiotemporal recordings but primarily represents suit-symbol surface detail, while DVSGesture captures human actions rather than changing object identities.DVSGesture therefore differs from object classification because the human remains the object while the gesture supplies the class.
- Summary of Existing Datasets: None of the four existing datasets specifically targets object classification, and the simulated ANTShapes datasets are proposed to address these needs.The review also identifies inherent DVS sensor noise and artificially induced temporal structure as dataset limitations.
3 ANTShapes Dataset Generation
ANTShapes generates four event-based benchmarking datasets from simulated 3D objects, using controlled parameter variations to create progressively different classification challenges. The datasets are exported in AER format, with standard, translation, distortion, and rotation variants.
- Simulation framework: ANTShapes simulates 12 distinct 3D object classes and exports event streams in AER format, representing changing scene luminance as coordinate, timestamp, and polarity packets.The simulated datasets are computationally generated to control represented features and eliminate stochastic DVS noise.
- Dataset overview: Four benchmarking datasets comprise a standard baseline plus three variants that introduce additional object behaviour as classification challenges.The variants are shape translation, shape distortion, and random rotation.
- Shape Translation: The translation dataset adds independent X- and Y-axis motion, reflecting objects from scene boundaries while allowing variation in movement speed.The translation variance is set to 0.15, and some examples move very quickly or slowly, increasing class-separation difficulty.
- Shape Distortion: The distortion dataset applies independent random scaling across all three axes, producing objects that range from subtly distorted to strongly squashed or stretched.The relevant global scaling-parameter variance is set to 0.1, altering the spatiotemporal patterns generated during rotation.
- Random Rotation: The rotation dataset assigns each object a random 3D rotation vector, producing independent rotational directions and variable speeds between examples.Two 1,500-example sets with opposite mean rotation vectors are merged to ensure meaningful rotation in both directions.
4 Evaluation
The evaluation benchmarks four ANTShapes datasets against established event-based datasets using convolutional SNNs across repeated trials. Results support ANTShapes as suitable benchmarking data, while exposing dataset-specific classification difficulty and experimental limitations.
- Evaluation design: Convolutional SNNs were trained across ANTShapes and four established datasets, with performance averaged over independent trials on out-of-sample test data.The benchmark compares mean performance and trial variance.
- ANTShapes results: The standard ANTShapes dataset achieved 92.2% mean accuracy, while the distorted-shape dataset achieved 83.5%.The standard dataset is easiest because examples lack rotational, translational, and shape distortion variance.
- Benchmark comparison: ANTShapes performance was largely comparable to N-MNIST, POKER-DVS, and DVSGesture, with clear class separation but recurring L-block/T-block confusion.Partial rotations and distorted proportions increased similarity between those classes.
- Benchmark comparison: 55.6% mean accuracy made CIFAR10-DVS a clear outlier, with overlapping class features and more frequent confusions than the other datasets.The authors identify positive-polarity event dominance as one possible contributor.
- Conclusions: The study concludes that ANTShapes can generate DVS-like, high-resolution datasets with many classes and examples for event-based object classification.The standard dataset provides a baseline, while translation, distortion, and rotation variants add example-level variance.
- Limitations: The evaluation is limited to one convolutional SNN architecture, and the added ANTShapes difficulty may be insufficient because distortion reduced average performance by less than 10%.The authors suggest evaluating other network models and simulating more sophisticated examples.
5 Conclusions
The four ANTShapes datasets fulfil their intended object-classification purposes and validate the simulation tool for event-based computer-vision experiments. Their simple, noise-free scenes support 3D-object classification, while the current task remains time invariant.
- The four ANTShapes datasets achieved comparable object-classification results to existing event-based datasets used as approximations.
- The datasets encode spatiotemporal changes in simple, idealised single-object scenes without noise from real-world event-based sensors.
- The ANTShapes simulation tool is suitable for producing data specialised toward event-based computer-vision tasks.
- ANTShapes simulates and labels rich 3D scenes, including anomalies defined by deviations from expected normal behaviours.The tool provides pixel-perfect masks and bounding boxes for anomalous objects.
- The classification task is limited because each example contains one time-invariant shape class and requires only one class-label prediction.A more complex task could involve multiple shape types or object replacement over time.
6 Data Availability
The paper provides access to the ANTShapes simulation tool and the benchmarking datasets, while acknowledging the stated funding and disclosure information.
- The ANTShapes simulation tool is available on GitHub.The paper gives the repository URL.
- The benchmarking datasets produced for the experiments are available on Zenodo.The paper gives the Zenodo record URL.
- The work was funded and supported by EPSRC, NimbleAI, Horizon Europe, and UKRI grants.
- The authors disclose no competing interests.
7 Additional Figures
The additional figures visualise average class-confusion patterns and principal-component structure across the evaluated datasets. PCA plots highlight dataset-specific separation and overlap among classes.
- Figure 10 shows confusion matrices for each dataset, averaged over 10 trials.The matrices illustrate class confusions predicted by the convolutional SNN.
- Figure 11 presents PCA plots for each evaluated dataset.
- DVSGesture shows clear PCA separation but overlap between clockwise and counter-clockwise motion classes.The overlap makes those classes difficult to separate.
- POKER-DVS shows clear PCA separation but contains only four classes and a limited number of samples.
- CIFAR10-DVS shows extreme overlap of principal elements across classes.