PaperlayerView exact revision on arXiv

Source-linked AI summary

Tight Learned Event-Inertial Odometry (TLEIO): a Transformer-Based Architecture for End-to-End Motion from Event Streams

Matteo Cascone, David Jensen, Alessandro Pirini, Angelo Stefanini, Pietro Venè, Marta Tintoré Gazulla, Javier Hidalgo Carriò

arXiv:2610.33356v1cs.CV

TL;DR

Event-based odometry must extract motion from sparse, asynchronous events, while inertial propagation drifts over time. TLEIO predicts event-based motion constraints and fuses them with IMU propagation in a stochastic-cloning EKF. It reports competitive benchmark performance and up to 48.9% lower runtime than optimization-based event-odometry pipelines.

  • Problem

    Sparse, asynchronous event streams make stable motion estimation difficult, while IMU integration drifts over time, motivating sequential fusion of event cues and inertial propagation.

    Source: §1 · p. 2

  • Method

    TLEIO predicts short-window relative displacements and per-axis uncertainty from event voxels, then fuses these constraints with IMU propagation in a stochastic-cloning EKF.

    Source: §1 · p. 2

  • Results

    Up to 48.9% lower end-to-end runtime is reported than optimization-based event-odometry pipelines, with competitive results on TartanAir/TartanEvent and lower aggregate runtime than evaluated optimization-based pipelines.

    Source: §1 · p. 2, §5 · pp. 12–13

  • Takeaways & Limitations

    Results support learning useful motion constraints directly from event streams in a single forward pass and fusing them with inertial propagation to recover full 6DoF motion.

    Source: §5 · p. 13

Abstract

from arXiv · show

Event cameras provide microsecond temporal resolution, high dynamic range, and sparse asynchronous measurements, making them compelling for odometry under fast motion and challenging illumination. Yet, accurate event-based state estimation remains difficult: existing systems often use events as a complement to frames or rely on feature tracking, correspondence search, and optimization-heavy backends, limiting the efficiency promised by event sensing. We introduce Tight Learned Event-Inertial Odometry (TLEIO), a compact filter-based pipeline that learns short-window relative motion directly from voxelized event streams and tightly fuses the resulting constraints with IMU propagation. Its learned front-end, EventsFormer, predicts local displacements and per-axis uncertainty from event voxels in a single forward pass, while a stochastic-cloning EKF fuses these overlapping predictions with IMU propagation, producing metric trajectories without explicit feature tracks and pose optimization at inference time. The network is trained directly from pose displacements, without requiring event-flow labels. We evaluate TLEIO on the TartanAir/TartanEvent benchmark and on real DAVIS240c sequences. On TartanAir, TLEIO reduces average ATE from 2.26 m to 1.10 m over the IMU-only TLIO baseline, while achieving up to 48.9% lower runtime than optimization-based event odometry methods.

1 Introduction

Event cameras offer high temporal resolution and dynamic range, but sparse asynchronous events make stable motion estimation difficult, while IMU integration drifts. TLEIO learns event-based motion constraints and fuses them with IMU propagation, avoiding explicit feature tracking and iterative pose optimization at inference.

1 Introduction

  • Event cameras provide microsecond temporal resolution and high dynamic range, addressing frame-camera limitations in motion blur, low light, and challenging illumination.Their sparse, irregular output makes event-based odometry fundamentally different from frame-based visual odometry.Source: §1 · p. 1
  • Sparse, asynchronous event streams make stable geometric correspondences difficult, while IMU integration provides metric short-term motion but drifts over time.Event-inertial odometry must extract motion cues from events and fuse them with inertial propagation sequentially.Source: §1 · p. 2
  • TLEIO combines an EventsFormer front-end that predicts relative translation and per-axis uncertainty from event voxels with SC-EKF fusion of those constraints and IMU propagation.The filter recovers the full 6DoF trajectory without explicit feature tracking, optical-flow estimation, or iterative pose optimization.Source: §1 · p. 2
  • Up to 48.9% lower end-to-end runtime is reported against optimization-based event-odometry pipelines.The pipeline uses an SC-EKF for full 6DoF estimation without iterative pose optimization.Source: §1 · p. 2

2 Related Work

Earlier event-odometry systems adapted geometric visual-odometry components, while learned approaches predict motion from event representations. TLEIO builds on learned filtering and temporal modeling to fuse event-based motion estimates with IMU propagation.

Event-based odometry

  • Geometric event-odometry systems use event features with filtering or optimization, while some stronger systems combine events with frames and IMU measurements.EKLT-VIO and Ultimate SLAM are cited as later systems combining multiple sensors or using model-based event tracking.Source: §2 · p. 3

Learned event-based odometry

  • Learned event-odometry methods predict motion from event representations, but DEVO, DEIO, and RAMP-VO retain correspondence estimation and iterative backend optimization.Source: §2 · p. 3

Learned filtering and temporal modeling

  • TLIO uses stochastic-cloning filtering to fuse learned displacement estimates with IMU propagation; TLEIO adapts this principle to event motion estimates.EventsFormer applies divided space–time attention to event voxels and predicts relative displacement and uncertainty for SC-EKF fusion.Source: §2 · p. 3

3 Methodology

TLEIO converts event streams into voxel clips, predicts short-window motion and uncertainty with EventsFormer, and fuses these constraints with IMU propagation in an SC-EKF. Its training uses pose displacements to learn motion and uncertainty.

3 Methodology

  • TLEIO converts raw events to voxel grids, predicts short-window body-frame 3DoF displacements and per-axis uncertainty, then fuses them with high-rate IMU propagation to recover a 6DoF trajectory.The filter applies overlapping predictions to pose clones while they remain in the window.Source: §3 · p. 3, Figure 2 · p. 4, §3.2 · p. 5

3.1 Pre-processing

  • Events within [ti − ∆t, ti) are accumulated into voxel grids with temporal bins after spatial down-sampling and local spatio-temporal background-activity filtering.Each grid has sensor resolution H × W and C temporal bins.Source: §3.1 · p. 3

3.2 Network

  • EventsFormer applies temporal attention across voxels at each spatial location, then spatial attention within each voxel, and predicts T −1 displacements with per-axis log standard deviations.It uses RoPE2D for spatial encoding, learned temporal embeddings, and a classification token to summarize the clip.Source: §3.2 · pp. 3–4, Figure 4 · p. 5

3.3 Stochastic-Cloning EKF

  • The SC-EKF fuses network motion estimates with high-rate IMU propagation using four past poses and four consecutive 3D displacement predictions in a 12D measurement update.It preserves pose cross-covariances but does not explicitly model correlations between overlapping network predictions, so the diagonal measurement model is an approximation.Source: §3.3 · p. 5, §3.2 · p. 5

3.4 Training

  • Training first uses mean squared error to stabilize displacement estimates, then maximum likelihood to learn uncertainty from regressed log standard deviations.Training uses TartanEvent; the first 60 epochs use MSE loss and the final 10 use ML loss.Source: §3.4 · p. 7

4 Experiments

Across TartanAir and real DAVIS240c data, TLEIO combines event-derived motion estimates with IMU filtering and achieves lower error than several baselines. Its runtime advantage depends on the recording: it is fastest on two EDS sequences, while event-count-driven RAMP-VO is fastest on the lower-event sequence.

4 Experiments

  • Experiments use the official 16-sequence TartanAir Visual SLAM test split with corresponding TartanEvent streams [19], reporting ground-truth-aligned ATE on TartanAir and path-length-normalized MPE on DAVIS240c.IMU-based pipelines use SE(3) alignment; remaining TartanAir pipelines use Sim(3) alignment.Source: §4 · pp. 7–8

4.1 Results on the TartanAir Competition Split

  • 1.10 m average ATE on TartanAir, down 51.3% from IMU-only TLIO’s 2.26 m, with lower average ATE than EKLT-VIO, ORB-SLAM3, and COLMAP.TLEIO improves on 12 of 16 sequences, while RAMP-VO, DPVO, and DROID-VO remain more accurate.Source: §4.1 · p. 8
  • On TartanAir sequences with substantial IMU error, TLEIO reduces ATE from 7.31 to 1.37 m on ME03 and from 3.72 to 0.22 m on MH03.These correspond to reductions of 81.3% and 94.1%, respectively.Source: §4.1 · p. 8
  • Standalone EventsFormer with RoPE2D reaches AUC 0.74, versus 0.71 without RoPE2D, confirming the benefit of spatial positional encoding; the whole TLEIO pipeline reaches AUC 0.90.The ±3ˆσ intervals in panel (c) illustrate how the network adapts its uncertainty over time.Source: §4.1 · pp. 8–9

4.2 Real-Data Adaptation on DAVIS240c

  • On DAVIS240c, TLEIO reaches 0.45% average MPE, ranking third among nine methods and outperforming six of eight baselines, including DEVO on both 6DoF sequences.Adaptation updates only the final linear head—9,240 of approximately 31M parameters—for 15 epochs on two training sequences excluded from evaluation; the transformer backbone remains frozen.Source: §4.2 · p. 9

4.3 Runtime and Computational Profile

  • Across EDS sequences 00, 01, and 09, summed mean runtime was 171.915 s for TLEIO versus 204.555 s for RAMP-VO, 292.130 s for DEVO, and 301.051 s for DEIO, reductions of 16.0%, 41.2%, and 42.9%.On sequence 09, RAMP-VO is fastest at 39.454 ± 0.575 s, followed by TLEIO at 48.863 ± 0.332 s; TLEIO requires 44.5% less time than DEVO and 48.9% less than DEIO.Source: §4.3 · p. 12

5 Conclusions

TLEIO pairs event-derived motion measurements with IMU propagation in a stochastic-cloning EKF, avoiding explicit event feature tracking and pose-optimization backends at inference. Experiments report competitive benchmark results, adaptation to different sensors with lightweight head-only fine-tuning, and lower aggregate runtime than evaluated optimization-based pipelines.

  • TLEIO combines EventsFormer’s event-based short-window displacement predictions with IMU propagation in a stochastic-cloning EKF, without explicit event feature tracking or pose-optimization backends at inference.Source: §5 · p. 12
  • TLEIO reports competitive results against state-of-the-art odometry pipelines, outperforming IMU-only learned odometry and geometric event-inertial baselines on the TartanAir/TartanEvent competition split.Source: §5 · p. 12
  • TLEIO achieves lower aggregate runtime than the evaluated optimization-based event-odometry pipelines.Source: §5 · p. 13
  • On DAVIS240c, TLEIO adapts to different sensors with lightweight head-only fine-tuning while preserving a compact learned front-end/filter interface.Source: §5 · p. 13

Related papers

Loading 2610.33356v1…