Real-time Unsupervised Object Discovery from Asynchronous Event Streams

Pratham G. Shenwai; Hemant Kumar Singh; Sridhar Ravi

Source-linked AI summary

Real-time Unsupervised Object Discovery from Asynchronous Event Streams

Pratham G. Shenwai, Hemant Kumar Singh, Sridhar Ravi

arXiv:2608.26644v1Aug 27, 2026cs.CV

TL;DR

Event-based object discovery must identify moving entities from sparse, noisy streams without semantic labels, while denoising and clustering remain mutually dependent. This paper proposes the training-free SPEF–EMCC pipeline, which combines probabilistic spatio-temporal filtering with Morton-ordered gap detection. Across the reported benchmarks, it improves classical object-discovery accuracy and execution time while remaining applicable to resource-constrained deployment.

Problem
Unsupervised event-based discovery must localize coherent moving entities without category supervision, despite noise that corrupts clustering neighborhoods and costly density-based searches.
Method
SPEF probabilistically filters events using spatial activity and temporal coherence, while EMCC clusters denoised events through Morton-code sorting and gap detection.
Results
Across FRED and eTraM, EMCC achieves the strongest reported combination of object-discovery accuracy and execution time, while SPEF performs competitively in denoising without offline training.
Takeaways & Limitations
Geometry-based processing of native event structure provides a training-free framework that is practical for real-time, resource-constrained object discovery.
Takeaways & Limitations
Evaluations assume motion-compensated input, with discovery benchmarks using statically mounted sensors and zero-ego-motion conditions.

Abstract

from arXiv · show

Event cameras capture pixel-level intensity changes with microsecond resolution to produce highly sparse asynchronous data streams. For visual perception in latency-critical environments, we propose a lightweight, training-free framework for discovery of moving objects based on spatio-temporal clustering. This framework is driven by two core contributions. First, a linear-time Spatio-temporal Probabilistic Event Filter (SPEF) that introduces an adaptive event acceptance threshold to distinguish salient motion structures from background noise. Second, an Event Morton Code Clustering (EMCC) module that bypasses expensive distance matrix computation to efficiently group events for unsupervised discovery of moving objects. On the E-MLB dataset benchmark, SPEF achieves the best denoising performance among classical filtering methods and remains competitive with learning-based approaches without requiring any offline training. On object discovery, EMCC achieves the highest overall accuracy and lowest execution time across the FRED and eTraM datasets, outperforming established density-based clustering baselines by a substantial margin. Overall, this work establishes a new performance benchmark for classical object discovery in event data, providing a highly scalable, training-free solution for resource-constrained visual perception. The code is available at https://github.com/PrathamShenwai/SPEF_EMCC

1 Introduction

Unsupervised event-based object discovery must jointly address motion-dependent denoising and efficient spatio-temporal clustering. The proposed framework aligns both stages with event geometry through adaptive filtering and Morton-ordered gap detection.

Event noise from illumination flicker, background motion, and hot pixels overlaps true motion and distorts clustering neighborhoods.
The central challenge is balancing discovery accuracy with the unfavorable neighborhood-search costs of classical density-based clustering.
SPEF accepts events according to the product of spatial activity and temporal coherence, deriving its threshold from the live event distribution without calibration or learning.
EMCC converts clustering into gap detection on Morton-ordered events, requiring O(N log N) sorting and an O(N) scan instead of neighborhood search.
The combined SPEF–EMCC pipeline is fully unsupervised and training-free, targeting real-time operation on resource-constrained hardware without optical flow or iterative optimization.

2 Related Work

Existing event-based approaches rely either on supervised category-specific representations or on unsupervised flow-based scene decomposition. The proposed approach instead directly clusters denoised events, while addressing the computational cost of conventional density-based methods.

Supervised event detectors require fixed temporal buffering and recognize only categories represented during training.
Unsupervised contrast-maximization methods decompose scenes by compensating background motion and treating independently moving objects as structured residuals.
Noise filtering methods range from recent-neighbor checks and FIFO designs to learned denoisers with higher suppression accuracy but greater latency and resource cost.
DBSCAN and HDBSCAN handle unknown cluster counts and varying densities, but repeated neighborhood searches become costly as event count and neighborhood size increase.
EMCC sorts denoised events by Morton code to preserve spatio-temporal proximity while replacing neighborhood search with gap-based clustering.

3 Method

The method couples adaptive probabilistic denoising with Morton-code clustering to preserve event structure while replacing costly spatio-temporal proximity computations with efficient gap detection.

Spatio-temporal Probabilistic Event Filter (SPEF): SPEF maps asynchronous events to regions and combines spatial activity with temporal coherence to probabilistically retain structurally informative events.Its adaptive threshold is derived from live counter distributions, while correlation scores capture temporal regularity.
Spatio-temporal Probabilistic Event Filter (SPEF): SPEF accepts an event with probability pkeep = po · pt, suppressing regions that are active without coherence or coherent without sufficient activity.Uniform rejection sampling makes acceptance probability equal to the event’s combined spatio-temporal evidence.
Spatio-temporal Probabilistic Event Filter (SPEF): The filter preserves object-periphery events according to temporal coherence, maintaining boundary structure needed by downstream object discovery.This avoids discarding events that narrowly miss a hard activity threshold.
Event Morton Code Clustering (EMCC): EMCC normalizes event coordinates, encodes them as n-bit Morton codes, sorts them, and partitions clusters where adjacent-code gaps exceed an adaptive percentile threshold.Morton ordering preserves locality while reducing clustering to one-dimensional gap detection rather than pairwise proximity computation.
Event Morton Code Clustering (EMCC): The pipeline extracts bounding boxes from gap-defined clusters, retains candidates using geometric and statistical constraints, merges nearby proposals, and applies non-maximum suppression.The shared post-clustering stage uses box percentiles, density criteria, merge distance, and NMS to produce final detections.

4 Experiments

Experiments evaluate the training-free SPEF–EMCC pipeline on denoising and unsupervised object discovery, including benchmark comparisons, computational efficiency, and ablations. SPEF preserves event structure under noise, while EMCC achieves strong discovery accuracy with substantially lower latency than density-based baselines.

Denoising evaluation: SPEF achieves the highest MESR scores at ND16 for daytime sequences and at ND16 and ND64 in challenging nighttime environments.It also ranks second-best in Daytime ND1, ND4, and ND64 cases.
Denoising evaluation: SPEF narrows the gap with learning-based denoisers while eliminating uncorrelated background scatter and preserving moving-target structure.The resulting sparse stream reduces the downstream search space for spatio-temporal grouping.
Object discovery: On FRED, EMCC reaches F1 0.441 and IoU 0.647, while processing each 33 ms segment in 6.3 ms.This is a 4.3× improvement over ST-DBSCAN at 27.3 ms, although ST-MeanShift achieves higher precision at 517.1 ms.
Object discovery: On eTraM, EMCC achieves the highest precision of 0.378, matches the top baseline F1 of 0.273, and maintains a 15.4 ms latency bound.This is a 2.4× reduction against the next-fastest baseline.
Ablation study: Without filtering, EMCC’s F1 falls to 0.256 at 14.6 ms, indicating that noise disrupts the density structure required by clustering.The result supports the complementary role of SPEF before EMCC.
Ablation study: A 5-bit linear SPEF counter yields F1 0.441 and recall 0.517, while a probabilistic counter achieves F1 0.434 at 5.9 ms versus 6.3 ms.Under a 2-bit memory limit, the probabilistic counter reaches F1 0.353 versus 0.319 for the linear counter.
Ablation study: EMCC’s Morton bit-depth strongly affects performance: 5 bits preserve clear gap bimodality, whereas 20 bits reduce F1 to 0.229.The link distance reaches an F1 plateau of 0.442 between 60 and 80 px, and 90% gap percentile balances over-fragmentation and under-segmentation at F1 0.440.

5 Discussion and Conclusion

The framework combines SPEF and EMCC into a training-free, real-time object-discovery system. Across multiple benchmarks, geometry-based processing achieves higher accuracy and lower execution time than classical alternatives while remaining practical for resource-constrained hardware.

SPEF and EMCC combine probabilistic noise suppression with Morton-ordered clustering for real-time, training-free event-based object discovery.The framework is designed for strict latency and resource constraints.
SPEF ranks first or second across six of eight E-MLB noise conditions without offline training.This substantially narrows the performance gap with learned denoisers.
23.2% improvement in F1 score and 8.9% increase in IoU over ST-DBSCAN accompany a 4.3× latency reduction on FRED.ST-DBSCAN is also the next fastest baseline.
On eTraM, EMCC matches top baseline accuracy while recording the lowest execution time across evaluated methods.Across two benchmarks, Morton-ordered gap detection reduces computational cost without sacrificing accuracy.
Geometry-based processing grounded in native event structure is more accurate and faster than existing classical approaches across multiple benchmarks.The framework also makes pipeline decisions traceable to spatio-temporal structure, supporting straightforward failure diagnosis.

S1 Pipeline Modularity and Compatibility with Ego-Motion Compensation

SPEF and EMCC are modular downstream components that separate coordinate stabilisation from object discovery. Upstream ego-motion compensation can improve EMCC's discrimination by widening Morton-space gaps between background and moving targets.

SPEF and EMCC operate on pre-processed event streams, decoupling coordinate stabilisation from object discovery.This supports stage-isolated evaluation because ego-motion compensation methods differ in accuracy and hardware dependency.
The modular interface prevents ego-motion compensation errors from being conflated with SPEF or EMCC contributions.The stated purpose is to evaluate the discovery modules independently.
Upstream ego-motion compensation widens the Morton-space gap between static background clutter and independently moving targets.The resulting gap distribution becomes bimodal, making the adaptive threshold τ = percentile(∆, pgap) more discriminative.

S2 EMCC: Full Implementation Details

The supplementary procedure expands EMCC beyond the compact main-paper algorithm. It specifies inputs, density filtering, aggregation, non-maximum suppression, and qualitative comparison details.

Algorithm SA1 provides a fully specified EMCC procedure that complements the compact Algorithm 1 summary.It makes explicit details referenced but not fully described in the main text.
The expanded procedure specifies parameter inputs and the density filter condition used by EMCC.
Aggregation and NMS steps are detailed in the supplementary procedure, with additional qualitative comparison in Figure SF2.

S3 Hyperparameter Ranges

The supplementary material defines dataset-aware hyperparameter search spaces for EMCC and the baselines. Ranges address scale variation, density differences, cluster fragmentation, and box merging while validation search limits overfitting.

1000 Optuna trials per algorithm per dataset define the hyperparameter-search budget on a held-out validation split.The budget was selected after the TPE sampler showed no meaningful composite-score improvement, while test sequences remained disjoint.
ST-DBSCAN tunes eps ∈[0.005, 0.3] to cover tight and diffuse clusters, extending the upper bound for eTraM's wider traffic-scene variance.
ST-HDBSCAN restricts min_cluster_size ∈[5, 200] because hierarchy adds little benefit above 200 events and fragments isolated detections below 5.
ST-MeanShift tunes bandwidth ∈[0.02, 0.5] because undersmoothing fragments objects while oversmoothing merges adjacent targets.Dataset-specific bounds reflect differing object densities between FRED and eTraM.
EMCC tunes cluster_gap_percentile ∈[60.0, 99.8] so partitioning adapts to scene density rather than using a globally fixed threshold.The eTraM lower bound is relaxed for higher event density and closer object proximity.
EMCC's procedure normalises events, Morton-encodes sorted coordinates, measures consecutive-code gaps, and adapts the partition threshold to stream density.The supplementary algorithm also defines inputs including pgap, box percentiles, density multiplier α, and merge distance dlink.

S4 Hyperparameter Tuning — Optimization Framework

The framework uses Bayesian hyperparameter optimization to jointly tune filtering, clustering, and bounding-box generation. Its composite objective balances spatial accuracy, temporal coverage, and false-positive suppression while keeping detection frequency subordinate to localisation quality.

Optimization Framework: The composite score multiplies Precision + Recall + mIoU + λ · Fdet by the false-positive penalty PFP.This score is reported to the optimizer for selecting the best parameter configuration.
Optimization Framework: Precision, Recall, and mIoU use an IoU threshold of 0.5, balancing localisation accuracy with reliable object discovery.These metrics are combined with detection frequency in the optimisation objective.
Optimization Framework: Fdet measures frames with valid predictions divided by total frames containing ground-truth objects.It captures temporal consistency across the sequence.
Optimization Framework: λ = 0.1 makes Fdet a tie-breaker, and rankings remain stable when λ is swept from 0.01 to 0.5.The small weight prevents detection frequency from encouraging oversized bounding boxes at the expense of localisation precision.
Optimization Framework: Hyperparameters are independently optimised for each clustering algorithm, including shared bounding-box parameters, on FRED and eTraM.The search ranges and method-specific settings are documented for the evaluated algorithms.
Optimization Framework: PFP penalizes false-positive ratios while preserving recall, accounting for unannotated background motion such as vegetation.The tolerance threshold of 1.3 was selected from false-positive distributions across both datasets.

S5 Probabilistic vs. Deterministic Event Acceptance: An Empirical Analysis

SPEF uses probabilistic, scene-adaptive event acceptance rather than a uniform fixed threshold. Across multiple objects, sparse targets, and changing motion, this adaptability outperforms or remains competitive with fixed-threshold acceptance.

Probabilistic Acceptance: SPEF accepts an event when r ∼U(0, 1) is below the product of per-event spatial and temporal acceptance probabilities.The decision boundary adapts to local spatio-temporal activity instead of remaining uniform across scene states.
Multiple Objects: RNG suppresses background noise during inactivity and tracks coherent object motion as activity appears in the multiple-object scenario.Its performance is comparable to θfix = 0.4 without the degradation observed at higher fixed thresholds.
Sparse Distant Target: RNG significantly outperforms every tested fixed threshold for the sparse distant target, where weak coherence keeps acceptance probability low.The fixed thresholds over-reject events and degrade both precision and recall.
Dynamic Motion: RNG maintains high F1 and IoU@50 while acceptance rises during approach and drops during recession.Fixed thresholds cannot respond to this rate of motion change and develop worsening precision-recall trade-offs as the object recedes.
Overall Comparison: Across all three scenarios, RNG consistently maintains higher F1 scores while θfix = 0.4 degrades outside its tuning conditions.The acceptance traces indicate adaptation to local spatio-temporal coherence without manual intervention.