Source-linked AI summary
Efficient Multi-Timescale Event Representations for Feed-Forward Object Detection
Fredrik Lundell, Per-Erik Forssen, Mårten Wadenbäck, Astrid Lundmark
TL;DR
Event detection needs efficient temporal representations for sparse, rapidly changing scenes, where recurrent architectures add sequential dependencies and complexity. The paper introduces confidence-normalized logarithmic B-spline encoding with local confidence support and a recursive approximation, then evaluates it using a fixed feed-forward detector. Across PEDRo and Gen1, the proposed representations consistently outperform compact CSTR, while the approximation largely preserves analytical spline performance and captures a substantial portion of recurrent temporal information.
Problem
Event detection needs temporal representations that retain useful information across timescales without relying on recurrent architectures with sequential dependencies and added complexity.
Method
The paper combines confidence-normalized logarithmic B-spline multi-timescale encoding with geometry-aware local support and an exponential-polynomial approximation for recursive event-by-event updates.
Results
Across PEDRo and Gen1, the proposed representations consistently outperform compact CSTR; spline AP50:95 improves from 0.566 to 0.630 on PEDRo and from 0.365 to 0.406 on Gen1.
Takeaways & Limitations
The results indicate that event representations can encode a substantial portion of temporal information learned through recurrent modeling for feed-forward and event-driven perception.
Takeaways & Limitations
The choice and coverage of temporal functions remain scenario-dependent, and efficient recursive implementations for event-driven and neuromorphic processing remain future work.
Abstract
from arXiv · showhide
Autonomous systems require robust low-latency perception under rapidly changing scene dynamics and challenging illumination. In event cameras object detection commonly relies on recurrent architectures to accumulate sparse temporal information over time. This work investigates how temporal information can be encoded directly within the event representation. We propose a confidence-normalized continuous multi-timescale representation based on logarithmic B-spline temporal encoding together with a geometry-aware local confidence mechanism that exploits the spatial structure of event generation. Using a fixed feed-forward EventCenterNet detector, we show that the proposed representations consistently outperform the compact CSTR representation on PEDRo and Gen1 datasets. We further introduce a recursive exponential-polynomial approximation that enables efficient event-by-event updates while largely preserving detection performance. These results demonstrate that carefully designed event representations can capture a substantial portion of the temporal information learned through recurrent temporal modeling, providing a promising foundation for efficient feed-forward, event-driven, and future neuromorphic object detection.
1 Introduction
The paper asks whether temporal information can be encoded directly in event representations rather than recurrent detector architectures. It proposes confidence-normalized multi-timescale encoding and reports consistent advantages over compact CSTR representations.
- Event detection must balance short windows that preserve localization against long windows that retain weak evidence but introduce stale structures.
- Recurrent hidden states and attention capture temporal information but add sequential dependencies and architectural complexity that hinder parallel and event-driven execution.
- The proposed representation combines logarithmic B-spline temporal encoding with confidence normalization and geometry-aware local support across multiple timescales.
- Logarithmic B-splines provide smoothly overlapping, lower-correlation temporal functions, while an exponential-polynomial approximation restores efficient recursive event-by-event updates.
- The experiments indicate that carefully designed event representations capture much of the temporal information learned through recurrent temporal modeling.
2 Related Work
Existing event representations commonly use fixed temporal bins or exponential decay, while recurrent aggregation adds temporal memory at greater architectural complexity. This work explores continuous temporal-function design and recursive representations for efficient event-driven processing.
- Common event representations include event counts, time surfaces, spatio-temporal histograms, and voxel grids processed by conventional neural networks.
- Compact CSTR encodes event count, polarity, and temporal information in three channels, while learned and recurrent representations also target asynchronous vision.
- Spatial local support can stabilize coherent event contours under sparse noise, motivating confidence-weighted local support in the proposed representation.
- Existing methods leave continuous temporal-function design relatively unexplored because they rely mainly on fixed bins or exponential decay.
- Recurrent and spiking detectors accumulate intermittent event evidence but increase architectural complexity compared with encoding temporal information before detection.
- Recursive temporal states support constant-cost processing of incoming events, and the proposed approximation preserves multi-timescale spline structure for neuromorphic processing.
3 Method
The method encodes event history with confidence-normalized multi-timescale temporal functions, using logarithmic B-splines to preserve recent detail while compressing older context. It adds geometry-aware confidence support and an exponential-polynomial approximation for recursive event-by-event updates.
- Event age is projected onto temporal functions, whose responses are accumulated separately for positive and negative polarity evidence.
- Events are represented in a compact 3N-channel tensor containing positive, negative, and confidence/support channels.
- Confidence support normalizes polarity channels by measuring available temporal support, while preserving relative positive and negative temporal structure.
- Log-Time B-Spline Encoding: Logarithmic B-splines allocate higher temporal resolution to recent events, compress older history, and provide smoother, less correlated multi-timescale encoding than exponential decay.The shifts are distributed logarithmically between τ0 and τmax, and the representation uses second-order B-spline basis functions in this work.
- Local Confidence: Geometry-aware local confidence aggregates neighboring confidence channels with Gaussian weighting to reinforce spatio-temporal support without blurring polarity-specific information.The motivation is that temporally correlated neighboring events often occur along moving structures and can support weak or fragmented object evidence.
- Polynomial Approximation: Polynomial combinations of exponential kernels approximate spline functions so recursively updated exponential states can reconstruct spline-like responses for event-driven computation.The polynomial mapping is learned by least-squares regression over sampled temporal points; the paper notes that an analytic Gram-matrix change of basis is left for future work.
4 Experimental Setup
Experiments use PEDRo and Gen1 with a fixed fully feed-forward EventCenterNet detector, identical training settings within each dataset, and standard COCO-based evaluation metrics.
- Datasets: PEDRo contains mostly medium and large pedestrian objects, whereas Gen1 includes more small and distant objects with sparser events and greater detection difficulty.The datasets therefore provide complementary evaluation settings.
- Detector: All experiments use a fully feed-forward EventCenterNet with a ResNet34 encoder, attention U-Net-style decoder, and CenterNet prediction heads.The detector predicts center heatmaps, bounding-box sizes, and center offsets without recurrent hidden state or temporal attention.
- Protocol: Within each dataset, training settings remain identical while only the event representation changes.Augmentation includes horizontal flips, polarity flips, and timestamp jittering.
- Metrics: Performance is reported with COCO metrics including AP50:95, AP50, AP75, APs, APm, APl, and ARs.AP50:95 averages Average Precision over IoU thresholds from 0.50 to 0.95; AR denotes Average Recall.
5 Results
Across PEDRo and Gen1, multi-timescale representations outperform CSTR in the fixed feed-forward detector, while temporal support and function placement affect dataset-specific performance. The recursive exponential-polynomial approximation enables event-by-event updates, and representation design captures substantial temporal information without recurrent state, although longer recurrent context remains beneficial on Gen1.
- Representation Comparison: Both multi-timescale representations outperform CSTR across all evaluated metrics on PEDRo and Gen1.Spline encoding with local confidence is strongest on several PEDRo metrics, while spline encoding without local confidence leads on Gen1 AP50:95, AP50, and AP75.
- Representation Comparison: APs rises from 0.015 to 0.062 and ARs from 0.142 to 0.219 when spline encoding with local confidence is compared with exponential decay on PEDRo.On PEDRo, spline encoding with local confidence achieves the highest AP50:95, AP50, AP75, APs, APm, and ARs.
- Representation Comparison: Local confidence consistently improves spline encoding on PEDRo but provides little or no benefit for exponential decay on either dataset.On Gen1, local confidence primarily improves APl, while the no-confidence spline variant leads on AP50:95, AP50, and AP75.
- Temporal Function Placement: A 150 ms window and τmax = 75 are sufficient on Gen1, because extending the window to 300 ms with τmax = 90 provides only marginal additional benefit.Increasing context from 50 ms to 150 ms substantially improves performance for sparse event observations.
- Temporal Function Placement: Reducing Gen1’s τ0 from 1.2 ms to 0.6 ms improves all reported AP metrics, while 0.45 ms adds no improvement and reducing τmax to 50 ms changes performance only slightly.This indicates that Gen1 benefits from higher recent-event temporal resolution.
- Temporal Function Placement: On PEDRo, τ0 = 0.6 ms and τmax = 20 ms maximize AP50:95, but τ0 = 1.2 ms and τmax = 35 ms produce the best AP50, AP75, and APs.Small-object performance is especially sensitive to the shortest temporal scale.
- Recursive Approximation: Five exponential functions reduce approximation MSE from 1.89×10^-2 to 1.36 × 10^-6 compared with three functions on Gen1.Matching exponential decay constants to the spline temporal support is important for accurate approximation.
- Comparison with Recurrent Detectors: On PEDRo, the feed-forward detector achieves the highest AP50 among compared methods and remains competitive with the smallest ReYOLO model without recurrent state.Recurrent models still achieve the highest AP50:95 on PEDRo, and recurrent ReYOLO models remain strongest on Gen1 because they exploit substantially longer temporal context.
6 Discussion
The experiments show that representation design strongly affects feed-forward event-based detection, with performance depending on temporal function placement, spatial confidence support, and scene-specific temporal needs. Feed-forward representations can approach recurrent performance, but Gen1 still benefits from longer temporal aggregation and reported performance is slightly underestimated by missing annotations.
- The proposed multi-timescale representations consistently outperform compact CSTR on PEDRo and Gen1 using the same fully feed-forward detector.
- Local confidence support consistently improves logarithmic spline encoding on PEDRo but provides little benefit for exponential decay.This supports jointly designing temporal encoding and spatial support.
- The feed-forward detector achieves the highest AP50 among compared recurrent and non-recurrent YOLO-based detectors on PEDRo, while Gen1 retains a gap to recurrent models.The Gen1 gap indicates a benefit from longer temporal aggregation in urban driving scenes.
- Missing annotations slightly underestimate reported performance because some detections labeled false positives correspond to unannotated vehicles.
- The polynomial approximation preserves much of analytical spline performance, but accurate approximation depends on selecting suitable exponential functions.The authors identify temporal-function coverage, transfer to other event-based perception tasks, and efficient recursive implementations as future directions.
7 Conclusion
This work studies whether temporal information can be encoded directly in event representations rather than recurrent hidden states. It introduces a confidence-normalized multi-timescale design and recursive approximation, achieving consistent gains over CSTR while retaining competitive performance without recurrent state.
- The study targets efficient feed-forward object detection by encoding temporal information directly in event representations instead of recurrent hidden-state propagation.
- The proposed approach combines confidence-normalized logarithmic B-spline encoding, local confidence support, and a recursive exponential-polynomial approximation.
- AP50:95 improves from 0.566 to 0.630 (+6.4) on PEDRo and from 0.365 to 0.406 (+4.1) on Gen1 versus compact CSTR.On PEDRo, small-object APs also rises from 0.010 to 0.062 (+5.2).
- EventCenterNet achieves the highest AP50 among compared recurrent and non-recurrent YOLO-based detectors while requiring no recurrent state.Its AP50:95 is also competitive with recurrent ReYOLO models, although Gen1 retains a long-horizon aggregation gap.
- The recursive polynomial formulation replaces analytical splines with compact recursive states and causes only minor detection-performance changes.Accurate approximation and efficient event-by-event updates depend on matching exponential functions to the spline temporal support.
- The results indicate that a substantial portion of recurrent temporal information can be encoded directly within event representations for efficient feed-forward and event-driven perception.