PaperlayerView exact revision on arXiv

Source-linked AI summary

Event-Based Motion Estimation via Oriented Distance Fields

Lei Sun, Yuqin Ma, Weilun Li, Haoran Liang, Runyi Yang, Kaiwei Wang, Danda Pani Paudel, Luc Van Gool

arXiv:2608.24223v1cs.CVcs.RO

TL;DR

Event-based motion estimation is often slowed by iterative optimization or repeated hypothesis comparison, limiting the practical benefit of high-temporal-resolution sensors. The paper introduces ODF motion estimation, which averages event distance vectors from a precomputed oriented field and reuses the result for deblurring and asynchronous eye tracking. Across public and self-collected datasets, it reports competitive or superior accuracy and downstream performance with low latency and compact models.

  • Problem

    Existing event-based motion estimators rely on iterative optimization or repeated hypothesis comparisons, which reintroduce latency that limits rapid motion recovery.

  • Method

    ODF motion estimation precomputes a distance-vector field and replaces optimization or search with a closed-form per-event averaging step, adaptive event selection, and parameter-free directional filtering.

  • Results

    The method reaches state-of-the-art or competitive accuracy at substantially lower latency, while supporting competitive deblurring and tracking performance with compact models.

  • Takeaways & Limitations

    The same motion-estimation output generalizes across non-blind image deblurring and low-power asynchronous pupil and glint tracking.

Abstract

from arXiv · show

Event-based motion estimation is central to tasks that demand high temporal resolution and robustness to fast motion. Existing methods typically rely on iterative optimization or repeated hypothesis comparison, offsetting the sensor's low-latency advantage. We propose Oriented Distance Field Motion Estimation (ODF Motion Estimation), which replaces this optimization with a single averaging step over a precomputed field of event distance vectors, combined with an adaptive event-count selection strategy and a parameter-free trail filter. On public and self-collected datasets, ODF motion estimation reaches sub-pixel accuracy at the lowest latency among compared methods. We validate its generality on two downstream applications rather than treating them as separate contributions. First, the estimated trajectory is converted into a blur kernel and paired with a compact iterative-unfolding network, trained on simulated motion-estimation noise, for real-time non-blind image deblurring, attaining competitive or superior PSNR/SSIM with under 1M parameters. Second, the same precomputed field is repurposed for directional event filtering in a low-power asynchronous pupil and glint tracker, sustaining stable tracking for tens of seconds while lowering a near-eye module's power draw.

I. INTRODUCTION

The paper addresses the latency bottleneck of event-based motion estimation by replacing iterative optimization and hypothesis search with ODF motion estimation, a closed-form distance-field averaging method. Its accuracy and low latency are evaluated through motion estimation and two downstream applications.

  • Motivation: Existing event-based methods use iterative optimization or per-event hypothesis comparisons, reintroducing latency despite event cameras’ microsecond temporal resolution.The resulting computational burden limits how quickly motion can be recovered on fast-moving platforms.
  • Method: ODF motion estimation precomputes an oriented distance field and estimates motion by averaging per-event distance vectors without optimization, search, or learned inference.The method targets dominant 2-DoF translation and also includes adaptive event-count selection and a parameter-free trail filter.
  • Downstream validation: The recovered trajectory is reused as a blur kernel for non-blind deblurring and as a directional event filter for asynchronous pupil and glint tracking.These applications test the estimator’s generality rather than introducing separate deblurring or eye-tracking contributions.
  • Results: Evaluation on public and self-collected datasets reports state-of-the-art or competitive motion-estimation accuracy at substantially lower latency, with competitive downstream performance and fewer parameters.The paper presents the downstream tasks as evidence that the estimator’s accuracy transfers across applications.

II. RELATED WORK

Prior work estimates event-based motion through optical-flow machinery, event-generation models, iterative alignment, or hypothesis search. ODF instead precomputes distance vectors once per template and averages them per event, while its tracking application reuses those vectors for directional filtering.

  • Motion estimation: Existing approaches include classical optical flow, contrast maximization, time-surface alignment, closed-form line solvers, multi-hypothesis tracking, and iterative frame or event alignment.These methods span both event-generation-model approaches and extensions of conventional tracking machinery.
  • Motion estimation: ODF motion estimation precomputes a distance vector field once per template, reducing motion estimation to a single average per event.This distinguishes it from methods that compare multiple candidate states or iteratively align features.
  • Image deblurring: Non-blind deblurring uses a known or estimated blur kernel, whereas blind deblurring estimates both the kernel and sharp image from the blurred input.Event cameras provide a direct route to kernel estimation, although event double-integral methods are described as noise-sensitive.
  • Eye tracking: Eye-tracking systems commonly use PCCR or appearance-based pupil detection, while this work reuses translation vectors for parameter-free directional event filtering.The reused vectors avoid learning a separate pupil model.

III. EVENT-BASED MOTION ESTIMATION

The method recovers short-window motion by aligning event-derived edges with an oriented distance field, avoiding iterative or discrete search under a 2-DoF translation model. Adaptive event batching addresses scene-dependent edge coverage, while orientation diversity resolves the aperture ambiguity.

  • A. Edge Alignment: Event streams record edge trajectories during exposure, which can be recovered by aligning early-event edges with subsequent events.The paper applies edge alignment between an event-derived template and incoming events.
  • A. Edge Alignment: The target motion is 2-DoF translation, such as handshake or panning, and trajectory recovery replaces iterative optimization or discrete search with an oriented distance field.The approach exploits temporal continuity in the event stream under this motion assumption.
  • B. Pattern Generation: Adaptive event batching stops template collection when downsampled events increasingly collide with occupied bins, avoiding fixed thresholds that can produce sparse or overly dense templates.The overlap ratio is set empirically because collision rates depend on scene content, but small changes shift stopping points by only a few events.
  • C. Distance Field: A distance field stores translation magnitude and descent direction at each location, allowing subsequent events to retrieve distance vectors for motion estimation.The field is precomputed on an edge template and supports direct per-event lookup.

C. Oriented Distance Field

ODF constructs an oriented distance field that directly stores translation vectors, allowing event alignment through lookup and averaging rather than per-event optimization. It addresses aperture ambiguity, edge-density bias, and distant noise through orientation diversity, valid-region selection, and distance-based rejection.

  • Field construction: ODF precomputes an oriented distance field whose values and descent directions directly encode the translation needed to align each image location with the reference edge.The field stores the distance-vector pair (D, ˆd), with v(x)=D(x)ˆd(x).
  • Field construction: Each event reads its precomputed distance vector, and averaging vectors across the current batch produces the displacement that shifts the field forward.This replaces solving alignment parameters independently for each event.
  • Ambiguity and valid regions: Orientation diversity resolves the aperture problem because differently oriented edges constrain different components of the 2-DoF translation.Averaging is unreliable when the template is dominated by one edge orientation.
  • Ambiguity and valid regions: The valid region is formed by dilating sparse-edge regions and intersecting them with the edge active region, avoiding zero-biased estimates in edge-dense areas.Sparse regions are identified where the field exceeds 5 pixels.
  • Ambiguity and valid regions: Events more than 5 pixels from an edge are discarded because they are likely noise and would pull the average away from the true trajectory.The cutoff also thins spurious events that lack a concentrated distance distribution.

D. Extracting Trajectory

The exposure trajectory is reconstructed by averaging per-event translation vectors in sequential batches and concatenating the resulting incremental displacements. Batch size and trail denoising balance noise suppression against temporal blur, while fixed-template anchoring prevents incremental errors from compounding.

  • Batch selection: M is set to 2.5% of the event count used to build the edge template, balancing noise-dominated oscillation against blur from overly large batches.The setting was chosen empirically because no closed-form batch-noise model was available.
  • Batch selection: The batch-size setting was not formally sensitivity-swept, although the two failure modes were considered visually distinct.This leaves the robustness of the chosen percentage less formally characterized.
  • Denoising: The method suppresses trailing events with a Spatio-Temporal-Contrast filter before averaging.Trailing events left behind by the moving edge would otherwise blur the displacement estimate.
  • Trajectory reconstruction: The full trajectory is reconstructed by processing events sequentially and concatenating each batch’s averaged incremental displacement.The batch average approximates full 2-DoF motion through orientation diversity.
  • Trajectory reconstruction: Each incremental estimate is read from the fixed initial template, so errors do not compound relative to previous noisy estimates.The process remains noniterative despite sequential event processing.

IV. APPLICATIONS

The estimated trajectory is converted into a motion-blur point spread function by weighting trajectory positions according to exposure time. The kernel remains continuous and is centered at the trajectory’s temporal midpoint to match image-deblurring ground truth.

  • Kernel generation: The motion-blur PSF is derived from the trajectory s(t) over exposure time T.Each trajectory position contributes through a Dirac delta at s(t).
  • Kernel generation: Each trajectory position receives a value equal to the fraction of exposure time spent there, with sub-pixel linear interpolation.The resulting kernel is kept continuous and unquantized during non-blind deblurring.
  • Kernel alignment: The kernel is centered on the trajectory position at the temporal midpoint of exposure because most clean-image ground truth corresponds to that time.ODF motion estimation provides the position at any instant during exposure.

2) Deep Unfolding Network for Deblurring with Noisy Blur Kernels:

The deblurring network handles estimated-kernel noise with an iterative deep-unfolding architecture that alternates closed-form data fidelity and learned priors. It is trained directly on motion-noise-infused kernels rather than frame-locked event simulations, with event masks focusing correction on blurred regions.

  • Motivation: Estimated kernels deviate from ground truth because of event noise, sensor bandwidth limits, and kernel quantization, while prior noise models do not transfer directly.This motivates training for the specific kernel-estimation noise encountered here.
  • Network design: The network alternates a closed-form data module with a learned prior module using the blur kernel, blurred image, and previous prior output.The architecture is inspired by half-quadratic splitting methods such as USRNet and DPIR.
  • Network design: The prior module is conditioned on the original blurred image and an event mask rather than an unavailable true kernel-noise level.The binary mask marks pixels whose exposure event count exceeds a threshold and focuses correction on genuinely blurred regions.
  • Training: Training generates motion-noise-infused blur kernels directly from constructed trajectories instead of replaying synthetic trajectories through frame-locked event simulators.Low-quality blurred inputs are supervised by the original high-quality images.
  • Training: Training varies kernel sizes from 11 to 81 pixels and adds noise from simplified BSRGAN and Real-ESRGAN pipelines with simulated pixel saturation.Oversized patches are blurred before cropping to avoid cyclic-boundary artifacts.

1) Pupil and Glint Event Signatures:

Pupil motion produces polarity-specific boundary events, while trailing events and glints contaminate the stream. The proposed directional filtering and stabilized ellipse fitting select motion-consistent events for tracking.

  • Pupil motion generates negative events on the leading edge and positive events on the trailing edge, while glints produce the opposite polarity pattern.
  • Fixed-polarity trail filtering fails when a pixel crosses two same-sign edges consecutively, a routine occurrence during pupil motion.
  • The directional filter flips negative-event vectors, aggregates nearby vectors, and discards events opposing the inferred pupil-motion direction.
  • Valid events are refit to an ellipse with direct least-squares fitting, while previous-boundary samples stabilize fits against noise and drift.
  • A polarity-reversed directional template scores glints, with thresholding and decay accommodating their smaller motion and intermittent disappearance.

A. Event-Based Motion Estimation

ODF motion estimation is evaluated against event-based alternatives for accuracy, update rate, and downstream deblurring. Its estimated kernels support competitive restoration with a compact network and real-time execution.

  • A. Event-Based Motion Estimation: All three motion estimators reach sub-pixel error, while ODF updates fastest on every sequence and has the lowest error on slider far and slider hdr far.
  • A. Event-Based Motion Estimation: ODF trades the last fraction of a pixel iterative refinement can buy for an order-of-magnitude higher update rate when latency matters more than peak accuracy.
  • B. Image Deblurring with Estimated Kernel: EventAid-B spans near-sharp to over 100-pixel blur and includes row-scanning artifacts and low-light noise without dataset-specific fine-tuning.
  • B. Image Deblurring with Estimated Kernel: With 0.6M parameters, the proposed method attains the highest PSNR on EventAid-B, while plain USRNet-tiny with the same kernel reaches 26.35/0.843.
  • B. Image Deblurring with Estimated Kernel: On the real dataset, replacing ground-truth kernels with estimated kernels costs each network under 1 dB in PSNR and 0.02 in SSIM.
  • B. Image Deblurring with Estimated Kernel: ODF processes each event in under 0.7 µs on one CPU thread, while the deblurring network runs at 55 ms per 1280 × 720 image.

C. Pupil Motion Estimation for Eye Tracking

The asynchronous tracker uses event streams to estimate pupil and glint positions on near-eye data. It maintains long tracks with sub-pixel-scale center errors and reduces total module power relative to a frame-camera design.

  • C. Pupil Motion Estimation for Eye Tracking: Across 24 subjects, median pupil IoU is 0.85 for left eyes and 0.86 for right eyes, with median center errors of 1.45 and 1.41 pixels.
  • C. Pupil Motion Estimation for Eye Tracking: The longest continuous track reaches 1745 frames, approximately 70 seconds at 25 fps.
  • C. Pupil Motion Estimation for Eye Tracking: On a near-eye AR prototype, the tracker recovers pupil and glint positions throughout recorded sequences, processing 28 µs per event on one thread and 10 µs on separate threads.
  • C. Pupil Motion Estimation for Eye Tracking: The event-driven module draws 127.1 mW versus 215.0 mW for the frame-camera module at 60 fps, with the gap widening as frame rate increases.

VI. CONCLUSION

ODF motion estimation replaces iterative optimization with averaging over a precomputed distance-vector field and generalizes to deblurring and eye tracking. Its low latency is retained end to end, subject to motion and event constraints.

  • ODF replaces iterative optimization with a single average over a precomputed distance-vector field and supports two downstream tasks.
  • The recovered trajectory forms a blur kernel for lightweight deblurring, while the same distance vectors directionally filter events for pupil and glint tracking.
  • Motion estimation runs at sub-microsecond latency per event, preserving event cameras’ low-latency advantage through the full pipeline.
  • The method assumes globally consistent motion, sufficient edge-orientation diversity, and enough events to constrain the estimate.
  • Both applications assume 2-DoF translation rather than general 6-DoF motion, excluding larger rotation or affine motion.
Loading 2608.24223v1…