PaperlayerView exact revision on arXiv

Source-linked AI summary

Event-based Stereo Visual Odometry

Yi Zhou, Guillermo Gallego, Shaojie Shen

arXiv:2007.15548v2cs.CVcs.RO

TL;DR

Event-based robot perception needs algorithms that process streams of intensity changes, while systems combining event cameras with additional sensors can be limited by those sensors’ speed and dynamic range. This paper estimates stereo-rig motion and reconstructs a semi-dense 3D map from stereo events; on synthetic scenes, its Student’s t-based objective yields depth-error standard deviations 2–3 times smaller than standard least squares.

  • Problem

    Robot perception needs algorithms for processing event streams, while approaches combining event cameras with additional sensors can be limited by those sensors’ speed and dynamic range.

    Source: §I · p. 1

  • Method

    The system estimates stereo-rig pose and reconstructs semi-dense depth maps from raw events using interleaved mapping and tracking modules.

    Source: §III · p. 3

  • Results

    On synthetic three-plane scenes, the Student’s t-based objective yielded depth-error standard deviations 2–3 times smaller than standard least squares.

    Source: B. Comparison of Mapping Optimization… · p. 10

  • Takeaways & Limitations

    The authors present the work as the first published stereo visual-odometry algorithm for event cameras.

    Source: §I · p. 2

Abstract

from arXiv · show

Event-based cameras are bio-inspired vision sensors whose pixels work independently from each other and respond asynchronously to brightness changes, with microsecond resolution. Their advantages make it possible to tackle challenging scenarios in robotics, such as high-speed and high dynamic range scenes. We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig. Our system follows a parallel tracking-and-mapping approach, where novel solutions to each subproblem (3D reconstruction and camera pose estimation) are developed with two objectives in mind: being principled and efficient, for real-time operation with commodity hardware. To this end, we seek to maximize the spatio-temporal consistency of stereo event-based data while using a simple and efficient representation. Specifically, the mapping module builds a semi-dense 3D map of the scene by fusing depth estimates from multiple local viewpoints (obtained by spatio-temporal consistency) in a probabilistic fashion. The tracking module recovers the pose of the stereo rig by solving a registration problem that naturally arises due to the chosen map and event data representation. Experiments on publicly available datasets and on our own recordings demonstrate the versatility of the proposed method in natural scenes with general 6-DoF motion. The system successfully leverages the advantages of event-based cameras to perform visual odometry in challenging illumination conditions, such as low-light and high dynamic range, while running in real-time on a standard CPU. We release the software and dataset under an open source licence to foster research in the emerging topic of event-based SLAM.

I. INTRODUCTION … B. Event-based Camera Pose Estimation

The paper presents a real-time stereo event-camera visual odometry system for natural scenes and arbitrary 6-DoF motion, combining spatio-temporal stereo mapping with event-based pose tracking. It also situates this approach among prior event-camera reconstruction and localization methods.

I. INTRODUCTION

  • Event cameras report pixel-wise intensity changes asynchronously with microsecond resolution and high dynamic range, supporting high-speed motion capture and broad illumination conditions.The introduction gives 140 dB as an event-camera example versus 60 dB for standard cameras.Source: §I · p. 1
  • The system estimates stereo-rig ego-motion and a 3D scene map for natural scenes with arbitrary 6-DoF motion; the authors describe it as the first published stereo event-camera VO algorithm.It follows a parallel tracking-and-mapping approach, with the modules operating in an interleaved fashion.Source: §I · pp. 1–2
  • The mapping method optimizes spatio-temporal consistency across stereo event streams and probabilistically fuses inverse-depth estimates to improve 3D structure density and accuracy.Depth estimates are obtained from multiple local viewpoints.Source: §I · p. 2
  • The tracking method uses 3D–2D registration and the distance-field nature of a compact event representation to estimate camera pose.This registration is the tracking module’s pose-estimation approach.Source: §I · p. 2
  • The system was evaluated on public datasets and the authors’ recordings, runs in real time on a standard CPU, and is released with its software, stereo-rig design, and datasets.The paper presents the evaluation as extensive.Source: §I · p. 2

II. RELATED WORK

  • Event-camera structure-and-motion estimation has been actively researched since the Dynamic Vision Sensor became commercially available in 2008.The related-work review covers 3D reconstruction and ego-motion estimation.Source: §II · p. 2

A. Event-based Depth Estimation (3D Reconstruction)

  • Instantaneous event-based stereo typically matches events across synchronized cameras and triangulates them, while monocular depth methods integrate events over time using camera-motion knowledge to produce semi-dense reconstruction.Instantaneous monocular depth estimation is described as brittle, and event simultaneity does not apply across time.Source: A. Event-based Depth Estimation (3D… · p. 2

B. Event-based Camera Pose Estimation

  • Prior event-camera localization progressed from constrained motion and artificial patterns to arbitrary 6-DoF motion in natural scenes with arbitrary 3D structure and photometric variation.Probabilistic filters provide event-by-event updates with microsecond latency, whereas frame-based techniques trade latency for more stable and accurate results.Source: B. Event-based Camera Pose Estimation · p. 2

C. Event-based VO and SLAM · III. SYSTEM OVERVIEW · A. Event Representation

The paper presents a stereo event-camera visual odometry system that combines time-surface event representation, mapping, and tracking to estimate rig motion and reconstruct the scene. Its design uses temporal consistency for mapping and event-to-map registration for tracking, aiming for real-time operation on commodity hardware.

C. Event-based VO and SLAM

  • Prior monocular methods include interleaved Bayesian filters requiring a GPU for real-time operation and a geometric, CPU real-time approach that avoids recovering absolute intensity.The latter builds on semi-dense mapping and focuses on events.Source: C. Event-based VO and SLAM · p. 2
  • To the authors’ knowledge, this is the first published stereo event-camera VO algorithm, and the experiments compare it with ICP, the demonstrator’s underlying technology.Source: C. Event-based VO and SLAM · pp. 2–3
  • The proposed method uses forward projection and time surfaces for depth estimation without explicit event correspondences, then optimizes tracking on time surfaces without recovering absolute intensity.It reworks prior mapping work and reuses mapping data structures for its tracker; it is designed to run in real time on a commodity CPU without a GPU.Source: C. Event-based VO and SLAM · p. 3

III. SYSTEM OVERVIEW

  • The system takes raw events from calibrated stereo cameras and combines event preprocessing, mapping, and tracking to estimate rig pose while reconstructing the environment with semi-dense depth maps.The overview flowchart shows raw stereo events as the only input and rig poses plus a 3D scene-edge point cloud as outputs.Source: §III · p. 3, Figure 2 · p. 3
  • Time surfaces are refreshed at practical intervals, their history is stored, and tracking poses are interpolated in SE(3) for mapping to refresh the local probabilistic semi-dense depth map.Refresh can follow a threshold number of events or a fixed rate such as the implementation’s 100 Hz; local maps are stored in a global point-cloud database for visualization.Source: §III · p. 3
  • Initialization uses a modified SGM stereo method to create a coarse map, allowing tracking to start while mapping builds a more accurate and dense semi-dense inverse-depth map.Source: §III · p. 3

A. Event Representation

  • Each event records pixel coordinates, timestamp, and the polarity of a predefined-size intensity change; a time surface stores each pixel’s last-event time and exponentially emphasizes recent events.The decay-rate parameter η is a small constant, for example 30 ms in the experiments.Source: A. Event Representation · pp. 3–4
  • Time surfaces provide compact, efficient representations of recent moving edges, enabling stereo pixel-wise temporal-consistency comparisons for mapping and anisotropic-distance-field registration of events to the 3D map for tracking.They replace standard-camera photo-consistency for mapping and are rescaled from [0, 1] to [0, 255] for visualization and processing.Source: A. Event Representation · p. 4

IV. MAPPING: STEREO DEPTH ESTIMATION BY SPATIO-TEMPORAL CONSISTENCY AND FUSION · A. Inverse Depth Estimation for an Event

The mapping module estimates event depths from stereo temporal consistency, then fuses those estimates into a populated depth map. It uses time-surface neighborhoods to handle timing differences between corresponding events and a real-time design based on single-map optimization and depth fusion over longer spans.

IV. MAPPING: STEREO DEPTH ESTIMATION BY SPATIO-TEMPORAL CONSISTENCY AND FUSION

  • The mapping module first estimates event depths, then fuses them into an accurate, populated depth map.Source: §IV · p. 4
  • Stereo matching minimizes temporal inconsistency across time-surface neighborhoods, rather than requiring simultaneous events at corresponding pixels.Pixel-level coincidence can fail because of delays, jitter, and differences in event firing rates.Source: §IV · p. 4
  • The real-time design optimizes temporal inconsistency over one stereo time-surface map and uses a novel fusion algorithm to estimate depth across longer spans.The fusion algorithm follows investigation of temporal-residual and inverse-depth uncertainty characteristics.Source: §IV · p. 4
  • Block matching provides the initial optimization guess more efficiently than brute-force search.Source: §IV · p. 4
  • Signed per-pixel residuals replace negative per-patch residuals, guaranteeing non-zero Jacobians for uncertainty propagation and fusion.Source: §IV · p. 4

A. Inverse Depth Estimation for an Event

  • For an event preceding a stereo observation, inverse depth is optimized along its left-camera viewing ray using the camera trajectory and paired time surfaces.The method parametrizes depth in the left camera; the stereo observation is a pair of time surfaces at time t.Source: A. Inverse Depth Estimation for an Event · p. 4

1) Problem Statement: · 2) Non-Linear Solver for Depth Estimation:

The method estimates depth by finding inverse-depth hypotheses that maximize spatio-temporal consistency across stereo event data, combining event matching and triangulation in one forward-projection step. It solves the resulting objective iteratively with non-linear least squares.

1) Problem Statement:

  • The objective measures inconsistency between left-right event-history time surfaces, replacing the photometric error used in frame-based stereo.Each event is warped using the camera pose at its timestamp; calibration is assumed known, and the left-camera pose is available over the observation interval.Source: 1) Problem Statement · pp. 4–5, Figure 5 · p. 5
  • The method uses known camera calibration to undistort and stereo-rectify event coordinates, and uses the constant left-to-right camera transform in its projections.Projection maps 3D points to the image plane, while inverse projection back-projects pixels into 3D given inverse depth.Source: 1) Problem Statement · p. 5
  • On a real stereo event-camera sequence with ground-truth depth, the objective profile confirms that the proposed function leads to optimal depth for a generic event.Figure 5 shows the objective variation with inverse depth and marks ground-truth inverse depth.Source: 1) Problem Statement · p. 5, Figure 5 · p. 5
  • An inverse-depth hypothesis defines a 3D point whose projections form a candidate stereo event match; the best match minimizes the objective function.This combines matching and triangulation in a single forward-projection step, unlike classical two-step methods that match events before triangulating.Source: 1) Problem Statement · p. 5

2) Non-Linear Solver for Depth Estimation:

  • The objective is optimized with non-linear least squares, such as Gauss-Newton, by iteratively finding roots of the necessary optimality condition.The residual is linearized as r(ρ+∆ρ) ≈ r(ρ)+J(ρ)∆ρ, yielding the normal equation J⊤J∆ρ = −J⊤r.Source: 2) Non-Linear Solver for Depth… · p. 5
  • Analytical derivatives are used to speed up the depth-estimation computations.Source: 2) Non-Linear Solver for Depth… · p. 5

3) Initialization of the Non-Linear Solver: … 1) Probabilistic Model of Estimated Inverse Depth:

The method initializes inverse-depth optimization with efficient stereo block matching, then estimates event depths and fuses them probabilistically into a semi-dense map. A Student’s t residual model supports uncertainty propagation and robust estimation in the presence of noise and outliers.

3) Initialization of the Non-Linear Solver:

  • Stereo block matching initializes inverse depth by maximizing ZNCC over an integer-pixel disparity grid, avoiding exhaustive inverse-depth search.The approximation omits motion compensation between the event time and stereo-observation time, with temporal consistency refined during nonlinear optimization.Source: 3) Initialization of the Non-Linear… · p. 5

4) Summary:

  • Each event’s inverse depth is estimated independently from the event, stereo time surfaces, inter-time rig motion, and camera extrinsics, making the computations parallelizable.Algorithm 1 initializes with ZNCC matching, iteratively computes residuals and analytical Jacobians, and updates inverse depth until convergence.Source: 4) Summary · p. 5, B. Semi-Dense Reconstruction · p. 6

B. Semi-Dense Reconstruction

  • Individual event inverse-depth estimates have different timestamps, so probabilistic fusion refers them to a common time, reduces uncertainty, and improves reconstruction density.The fused reconstruction approaches semi-dense coverage, producing depth values for most edge pixels.Source: B. Semi-Dense Reconstruction · pp. 5–6

1) Probabilistic Model of Estimated Inverse Depth:

  • Inverse depth is modeled as a probability distribution rather than a single estimate, allowing uncertainty around the estimated average to be propagated.The affine-transformation property of multivariate Student’s t distributions underlies the update’s approximate Student’s t model.Source: 1) Probabilistic Model of Estimated… · p. 6
  • Student’s t distributions fit residual histograms from two datasets, and their near-zero centers support setting the residual mean to approximately zero.The empirical histograms and fitted model are shown in Fig. 6; fitted parameters are reported in Table I.Source: 1) Probabilistic Model of Estimated… · p. 6, Figure 6 · p. 6, Table I · p. 6
  • The heavy tails of the Student’s t model support robust inverse-depth estimation by reweighting squared residuals with IRLS.This replaces the Gauss–Newton solver in Algorithm 1; the weighting factor depends on each residual and the probabilistic model.Source: 1) Probabilistic Model of Estimated… · p. 6

2) Inverse Depth Filters:

The mapping module propagates inverse-depth estimates to a common stereo-observation time and fuses them pixelwise using a robust Bayesian Student’s t filter. It combines 20 stereo observations to create an inverse-depth map.

2) Inverse Depth Filters:

  • Inverse-depth estimates are propagated to the current stereo-observation time, then updated with pixelwise priors using a robust Bayesian Student’s t filter.The filter approximates the posterior as a Student’s t distribution.Source: 2) Inverse Depth Filters · p. 6
  • The mapping module fuses estimates from 20 stereo observations (M = 19) to create an inverse-depth map.Source: 2) Inverse Depth Filters · p. 7, Figure 7 · p. 7
  • For fusion from t − 1 to t, a projected 3D point at mean depth and its distribution influence distributions within a 3 × 3 image region by assigning, fusing, or replacing them.Source: 2) Inverse Depth Filters · p. 7

3) Probabilistic Inverse Depth Fusion: · 4) Summary:

The mapping module estimates inverse depth from stereo observations and events, then propagates and probabilistically fuses estimates into a semi-dense map for tracking. Depth is estimated selectively to support real-time operation with limited computational resources.

3) Probabilistic Inverse Depth Fusion:

  • Propagated inverse-depth distributions affect the four nearest integer pixels when their projected target-image coordinate is non-integer.The section describes the propagated estimate as a distribution and distributes its effect across those nearby pixels.Source: 3) Probabilistic Inverse Depth Fusion · p. 7
  • At a pixel without a prior distribution, the propagated estimate initializes it; otherwise, the two inverse-depth hypotheses are checked for compatibility.The compatibility check determines whether the estimates may be fused.Source: 3) Probabilistic Inverse Depth Fusion · p. 7
  • Compatible inverse-depth hypotheses are fused using equation (12), while incompatible ones retain the distribution with the smallest variance.The section illustrates this strategy with propagation and update from estimates at t−1 to the inverse-depth map at t.Source: 3) Probabilistic Inverse Depth Fusion · p. 7

4) Summary:

  • At each timestamp, inverse depth is estimated by IRLS from stereo observations and events, then estimates across timestamps are propagated and fused into the latest map distribution.The overall mapping procedure combines the inverse-depth estimation described in Sec. IV-A with probabilistic fusion.Source: 4) Summary · p. 7
  • The fusion produces a semi-dense inverse-depth map with reasonably good signal-noise ratio, which the tracking method requires.Source: 4) Summary · p. 7
  • All events contribute to time surfaces for tracking and mapping, but depth is not estimated for every event to limit computation and enable real-time operation.Source: 4) Summary · p. 7
  • The number of fused stereo observations was set empirically to M + 1 = 20 as a sensible choice for map density; an adaptive statistical criterion is left for future work.Possible criteria include apparent point density or reduction in fused-depth uncertainty.Source: 4) Summary · p. 7

V. CAMERA TRACKING · A. Exploiting Time Surfaces as Distance Fields

The tracking module estimates the stereo rig’s pose relative to a local map by globally registering time-surface negatives with the warped inverse-depth map. It uses the time surfaces already built for mapping and is designed for robust, high-throughput tracking.

V. CAMERA TRACKING

  • Tracking takes events and a local map as input and estimates the stereo rig’s pose, using a stereo-observation pose rather than a separate pose for each event.The authors note that microsecond-resolution poses are typically unnecessary.Source: §V · p. 7
  • Minimizing stereo spatio-temporal inconsistency alone is insufficient: a wrong pose can project the map into blank regions of both time surfaces and create an undesired minimum.The alternative of aligning only the left time surface using the monocular edge-map method requires additional event images.Source: §V · pp. 7–8
  • The method performs global image-like registration of time-surface negatives, using the forward compositional Lucas–Kanade method.The paper also describes a later step to improve tracking robustness while maintaining high throughput.Source: §V · p. 8

A. Exploiting Time Surfaces as Distance Fields

  • Time-surface ramps act as anisotropic distance fields toward current edges; negating the surface makes small values mark edges and turns registration into minimization.The negative transformation is rescaled to [0, 255].Source: A. Exploiting Time Surfaces as… · p. 8
  • The method aligns dark regions of the time-surface negative with the warped inverse-depth map’s support, treating scene edges as “zero time”.In the Fig. 8 example, the warped point cloud aligns with minima in the negative time surface.Source: A. Exploiting Time Surfaces as… · p. 8, Figure 8 · p. 8
  • Tracking uses only the left time surface because adding the rigidly attached right camera’s surface does not significantly improve accuracy and doubles computational cost.A successful edge-map alignment example is shown in Fig. 8(b).Source: A. Exploiting Time Surfaces as… · p. 8

B. Tracking Problem Statement · C. Compositional Algorithm · D. Robust and Efficient Motion Estimation

Tracking estimates the stereo rig pose by aligning a warped semi-dense depth-map support with minima in the current left-camera time-surface negative. The method uses forward compositional pose refinement and robust, sampled optimization.

B. Tracking Problem Statement

  • Pose tracking seeks a transform that aligns the warped support of the reference semi-dense depth map with minima in the current left-camera time-surface negative.The reference support contains pixels with valid inverse depth.Source: B. Tracking Problem Statement · p. 8
  • The warp back-projects reference pixels using inverse depth, applies candidate 3D motion, and projects them into the current left image under a rectified, undistorted stereo assumption.Motion is parameterized by Cayley orientation parameters and translation.Source: B. Tracking Problem Statement · p. 8

C. Compositional Algorithm

  • Forward compositional Lucas–Kanade refines incremental pose parameters and updates the warp, allowing constant Jacobian components to be precomputed for efficiency.Linearization at zero increment keeps those components constant across iterations.Source: C. Compositional Algorithm · p. 8
  • The objective is smooth and differentiable with a unique local optimum near ground truth; blurring the time-surface negative with a 5-pixel kernel widens the convergence basin.The objective slices are evaluated around the ground-truth relative pose.Source: C. Compositional Algorithm · p. 9

D. Robust and Efficient Motion Estimation

  • To improve robustness to noise and outliers in inverse depth, the method applies a Huber norm and solves the resulting problem with iteratively reweighted least squares.The nonlinear least-squares solver is described as already accurate enough before adding this robust treatment.Source: D. Robust and Efficient Motion… · p. 9
  • Stochastic Levenberg–Marquardt samples batches of typically 300 reference-frame 3D points, runs one iteration per batch, and often converges in five iterations from a close initial pose.Sampling is used to speed optimization.Source: D. Robust and Efficient Motion… · p. 9

VI. EXPERIMENTS … C. Comparison of Stereo 3D Reconstruction Methods

The experiments assess the mapping module using ground-truth poses and compare stereo reconstruction methods, then evaluate the full system using events and camera trajectories. They use public datasets and simulators alongside recordings from a custom stereo event-camera rig.

VI. EXPERIMENTS

  • The mapping-method comparison uses ground-truth poses to isolate mapping performance, while the full-system evaluation uses only events and compares estimated trajectories with ground truth.The experiments also assess low-light and HDR operation, computational performance, limitations, and difficult motions for space-time consistency.Source: §VI · p. 9

A. Experimental Setup and Datasets Used

B. Comparison of Mapping Optimization Criteria: IRLS vs LS

C. Comparison of Stereo 3D Reconstruction Methods

  • The stereo comparison includes GTS, SGM, and CopNet, with depth estimates from GTS and SGM propagated using ground-truth poses for comparison with incremental fusion.CopNet is evaluated only at the evaluation time because software incompatibility prevents propagation; its inverse-depth map is sufficiently dense when given 15,000 events.Source: C. Comparison of Stereo 3D… · p. 10
  • Across the compared methods, our method produces the best inverse-depth maps in compactness and signal-to-noise ratio, while maintaining stable map density as event rates vary.The method models depth and uncertainty to fuse estimates across viewpoints and prune unreliable ones; SGM produces denser maps by regularizing ambiguous regions, while CopNet’s accuracy is affected by pixel-quantized disparity and smoothing.Source: C. Comparison of Stereo 3D… · p. 10, Figure 12 · p. 11

GTS [26]

  • GTS maps are considerably less dense on upenn than rpg sequences, and relaxing its event-matching time limit could increase density at the cost of less accurate, noisier depth.The sequences differ in depth range and motion: upenn has a larger depth range and dominantly translating drone motion, while rpg uses hand-held cameras with general 3D motion.Source: C. Comparison of Stereo 3D… · p. 10, SGM [45] · p. 11

SGM [45]

  • Results on rpg reader and rpg bin are less sharp than on rpg box and rpg monitor because their ground-truth poses are less globally consistent.Source: SGM [45] · p. 11
  • On the two sequences with LiDAR ground-truth depth, the proposed method outperforms all baseline methods in mean, median, and relative depth error.Relative error is measured with respect to the depth range.Source: SGM [45] · p. 11

D. Full System Evaluation

The full stereo event-based VO system is evaluated against an event-based SGM+ICP baseline and ORB-SLAM2 using relative pose error and absolute trajectory error. It outperforms the event-based baseline on all six sequences, with results versus ORB-SLAM2 varying by dataset and flickering conditions affecting performance.

D. Full System Evaluation

  • The SGM+ICP baseline combines dense SGM depth estimates with ICP pose estimates between successive point clouds, concatenating relative poses into a trajectory.SGM provides depth estimation and ICP estimates relative pose between successive depth maps.Source: D. Full System Evaluation · p. 12
  • The trajectories in Fig. 13 show that the full system significantly outperforms SGM+ICP while keeping mapping and tracking mutually consistent.This evaluation tests whether each module remains robust to errors induced by the other.Source: D. Full System Evaluation · p. 12
  • Despite rpg flickering degrading stereo time-surface consistency, the system performs robustly unless flickering occurs during initialization.The flickering is induced by the motion-capture system and does not appear in the grayscale frames used by ORB-SLAM2.Source: D. Full System Evaluation · p. 12
  • Upenn accuracy is lower than rpg because dominant translation leaves structures parallel to the stereo baseline unreconstructed, while sensor resolution also limits accuracy.Hovering generates few events, making constant-rate time surfaces unreliable and potentially forcing reinitialization; evaluation therefore uses only the dynamic dataset section.Source: D. Full System Evaluation · p. 12
  • Our method outperforms SGM+ICP on all six sequences, is slightly less accurate than ORB-SLAM2 on rpg, and performs better on upenn indoor flying.The evaluation reports RPE and ATE; global bundle adjustment was disabled for the fair ORB-SLAM2 comparison, with enabled results also shown for reference.Source: Table V · p. 12, Table VI · p. 12, D. Full System Evaluation · p. 12
  • On the HKUST lab sequence, the paper visualizes the estimated sensor trajectory, 3D point cloud, and inverse-depth maps from selected views.The sequence was collected with the authors’ stereo event-camera rig in a cluttered machine-facility environment, moving handheld from left to right with locally loopy motion.Source: D. Full System Evaluation · p. 12

E. Experiments in Low Light and HDR Environments · F. Computational Performance

The system is tested in low-light and HDR scenes, where event-based visual odometry remains robust despite limited visibility for the DAVIS frame sensor. The paper also reports the real-time implementation’s computational costs and node performance.

F. Computational Performance

  • The C++ ROS system runs in real time on an Intel Core i7-8750H laptop, with hyper-threading used for mapping and tracking.The number of threads used by each node is indicated alongside its name in Table VII.Source: F. Computational Performance · p. 13, Table VII · p. 14
  • Time-surface map creation takes 5–10 ms, and initialization takes 12–17 ms to produce the first local map using SGM depth masked by an event map.The durations depend on sensor resolution; initialization is active only during bootstrapping.Source: F. Computational Performance · pp. 13–14

E. Experiments in Low Light and HDR Environments

  • In dark and HDR scenes, event-based VO works robustly despite the DAVIS frame sensor’s limited visibility in dark regions (55 dB versus 120 dB); results are shown in Fig. 15.The HDR sequence uses a directional lamp to increase scene brightness variation.Source: E. Experiments in Low Light and HDR… · p. 12, Figure 15 · p. 14

F. Computational Performance

  • The four-thread mapping node takes about 41 ms and typically runs at 20 Hz; matching takes ≈6 ms, depth refinement 15 ms, and fusion 20 ms.Matching searches 1000 corresponding patches with a ≈40–50% success rate; depth refinement returns 500 inverse depth estimates, and fusion performs 60000 operations.Source: F. Computational Performance · p. 14
  • Using 500 events gives the sparsest depth map, while 1000 and 2000 yield nearly the same density; 1000 is chosen for real-time performance because 2000 costs more.The comparison holds the number of fusion steps constant and draws events randomly from the latest 10000; computation time is approximately proportional to event count.Source: F. Computational Performance · pp. 14–15, Figure 16 · p. 15
  • The two-thread tracking node solves pose estimation in ≈10 ms with IRLS and can run up to 100 Hz.Each iteration randomly samples 300 points, with at most five iterations.Source: F. Computational Performance · p. 15

G. Discussion: Missing Edges in Reconstructions · H. Dependency of Spatio-Temporal Consistency on Motion · VII. CONCLUSION

The paper presents an efficient event-based stereo visual odometry system whose experiments demonstrate performance comparable to mature frame-based methods in normal conditions and potential advantages in difficult illumination. Its reconstructions can miss edges parallel to the stereo baseline, while tested motions did not cause significant temporal inconsistency that broke the system.

G. Discussion: Missing Edges in Reconstructions

  • Edges parallel to the stereo baseline are difficult to recover across stereo methods, but uncertainty estimation lets this method reject their unreliable depth estimates.Thresholding the depth-uncertainty map produces a more reliable but sparser depth map.Source: G. Discussion: Missing Edges in… · p. 15
  • Improving the completeness of reconstructions affected by missing baseline-parallel edges is left for future work.Source: G. Discussion: Missing Edges in… · p. 15

H. Dependency of Spatio-Temporal Consistency on Motion

  • 14.995 and 21.838 are the temporal-residual scale values for translation along Y and rotation around Z, respectively.Under pure rotation around Z, fused depth maps were slightly worse but accurate in most pixels, and the residual histogram was broader than for translation along X/Y.Source: H. Dependency of Spatio-Temporal… · pp. 15–16, Figure 18 · p. 15
  • The method performed well in practice and showed no significant temporal inconsistency that broke down the system under a priori difficult stereo motions.The authors leave a more theoretical and detailed analysis of such motions for future research.Source: H. Dependency of Spatio-Temporal… · p. 16

VII. CONCLUSION

  • Experiments show performance comparable to mature frame-based visual odometry in normal conditions, potential advantages in difficult illumination, and real-time operation on a standard CPU.The software, stereo-rig design, and evaluation datasets were open sourced.Source: §VII · p. 16
  • The system combines probabilistic fusion of inverse-depth estimates for mapping with time-surface-based 3D–2D registration for tracking.Mapping optimizes spatio-temporal consistency across stereo event streams; tracking leverages the distance-field nature of time surfaces.Source: §VII · p. 16
  • Future work may fuse inertial observations and investigate temporal correspondence methods for events on each camera.These directions are closely related to event-based stereo visual odometry.Source: §VII · p. 16

Related papers

Loading 2007.15548v2…