PaperlayerView exact revision on arXiv

Source-linked AI summary

Lossy Event Compression: From Event Stream Distortion to Task Performance

Zahra Rezaee, Catarina Brites, João Ascenso

arXiv:2608.28429v1cs.CVeess.IV

TL;DR

Event cameras can generate hundreds of millions of events per second, while existing distortion metrics do not reliably predict how lossy compression affects downstream tasks. The paper compares aggregation-based JPEG 2000 and frame-free G-PCC pipelines within a task-driven framework and finds that five proposed classification-based metrics reliably predict task degradation across coding frameworks. This supports using event-stream distortion assessment to guide future codec development and optimization.

  • Problem

    High event rates create bandwidth and storage demands, but existing metrics have not reliably captured compression-induced effects on downstream task performance.

  • Method

    The paper evaluates aggregation-based JPEG 2000 and frame-free G-PCC event compression within a unified framework linking distortion to four downstream tasks.

  • Results

    All five proposed metrics achieve SROCC and PLCC values above 0.80 across every task-codec combination and pooled scores, while existing benchmarks generally do not.

  • Takeaways & Limitations

    Event-stream distortion assessment can guide codec design and optimization as an alternative to repeated task-specific evaluation.

Abstract

from arXiv · show

Event cameras generate asynchronous, sparse data streams with microsecond temporal resolution, but in moderate-to-high motion scenes they can produce as many as hundreds of millions of events per second, creating significant bandwidth and storage challenges. Lossy compression is therefore essential for practical deployment, yet existing event stream distortion metrics fail to reliably predict compression-induced degradation at the task level, forcing codec optimization to rely on expensive task-specific evaluations. To address this gap, this paper introduces two fundamentally different event compression pipelines: i) an aggregation-based pipeline that converts the event stream into polarity-based histogram frames for compression with the conventional image codec JPEG 2000, and ii) a frame-free point cloud-based pipeline that codes events natively as 3D points using the octree-based codec G-PCC. Both pipelines are then assessed within a unified task-driven evaluation framework that relates event stream distortion to downstream application performance across four representative tasks: i) video reconstruction, ii) object detection, iii) optical flow estimation, and a delay-sensitive task iv) asynchronous feature tracking under a reference-relative protocol. Building on this framework, five classification-based distortion metrics are applied to event compression for the first time, to the best of the authors' knowledge, and benchmarked against existing event stream metrics. Experimental results demonstrate that the proposed metrics reliably predict compression-induced task degradation across different coding frameworks. This demonstrates that event stream distortion assessment can be an efficient alternative to repeated task-specific evaluation, providing direct guidance for the development and optimization of future event data coding solutions.

I. INTRODUCTION

Event cameras provide sparse, high-temporal-resolution data but can generate enormous event rates, making compression important while complicating evaluation of its task-level effects. This paper addresses the gap by introducing complementary compression pipelines and systematically relating event distortion to downstream performance.

  • Motivation: Event cameras asynchronously emit events when pixel brightness changes exceed a threshold, producing sparse streams with microsecond temporal resolution.Each event records spatial coordinates, timestamp, and polarity.
  • Motivation: Hundreds of millions of events per second can arise in moderate-to-high motion, creating substantial bandwidth and storage demands.These demands motivate practical event-stream compression.
  • Research gap: The paper asks whether intrinsic event-stream distortion measures can reliably characterize lossy-compression effects on downstream task performance.The introduction identifies missing comprehensive evaluations, benchmark datasets, and established best practices.
  • Contributions: The proposed study combines aggregation-based image coding with frame-free point-cloud coding and evaluates their distortion against multiple downstream applications.The aggregation pipeline uses polarity-based histograms and JPEG 2000, while the point-cloud approach codes event data in 3D.
  • Research gap: Existing compression studies mainly emphasize bitrate and compression ratio rather than systematic effects on downstream vision and visualization tasks.Compression errors can remove informative events or introduce spurious ones, affecting machine vision and reconstructed visual content.
  • Contributions: Classification-based distortion metrics are introduced to event compression and compared with existing event-stream metrics for predicting functionally relevant distortions.The paper motivates these metrics because prior objective measures had not been evaluated for downstream task prediction.

III. AGGREGATION-BASED EVENT STREAM COMPRESSION PIPELINE

The aggregation-based pipeline converts asynchronous events into polarity-separated histogram frames, compresses them with an image codec, and reconstructs an event stream through inverse aggregation.

  • Aggregation and representation: Temporal aggregation converts raw event tuples into structured histogram frames by separating polarities and partitioning events into fixed-duration windows.The resulting representation is designed for standard 2D image compression.
  • Image coding: The two polarity-specific histogram frames are independently compressed and decoded using JPEG 2000.JPEG 2000 supports both lossless and lossy modes and is selected for sparse-data compression, spatial redundancy exploitation, and rate control.
  • Inverse aggregation: Inverse aggregation converts decoded histogram counts back into an event stream suitable for downstream processing.Each positive or negative count is reconstructed as events carrying the corresponding polarity.
  • Inverse aggregation: For a pixel with N events in a window of duration ∆t, reconstructed timestamps are uniformly spaced with interval ∆t/N.Pixels with zero counts produce no events, while the original intra-window timing is discarded and approximated.

B. CODEC OPERATING POINTS

The aggregation-based codec varies its operating points through the temporal aggregation interval and JPEG 2000 target bitrate. These parameters trade bitrate against temporal, count, and spatial fidelity in the reconstructed stream.

  • Operating parameters: The codec’s rate-distortion operating points are defined by the temporal aggregation interval and JPEG 2000 target bitrate.Together they control the number of frames, compression level, and reconstructed-stream fidelity.
  • Temporal control: Larger aggregation intervals generally increase temporal distortion because more events are accumulated while their original timestamps are discarded.The interval also determines the number of histogram frames generated per second.
  • Bitrate control: Lower JPEG 2000 target bitrates increase compression of histogram frames and can reduce reconstructed event-count and spatial fidelity.Rate-distortion optimization selects coding-pass truncation points under the target bit budget.
  • Experimental operating points: The experiments combine lossless mode with target bitrates of 2, 1.5, 1, 0.5, 0.25, and 0.1 bpp.Aggregation intervals vary by dataset, spanning 5–100 ms for ECD, 5–50 ms for MVSEC, and 50–150 ms for the sparser dataset.
  • Lossless versus lossy coding: In lossless mode, histogram counts are preserved and downstream variation is attributed to temporal aggregation, whereas lossy mode can also alter counts and spatial distribution.This separates aggregation effects from additional image-coding errors.

IV. POINT CLOUD-BASED EVENT STREAM COMPRESSION PIPELINE

The point cloud-based pipeline encodes event streams natively as polarity-separated 3D point clouds with G-PCC, avoiding an intermediate frame representation. Fixed temporal chunking and common-grid coordinate mappings support invertible reconstruction into the standard event-stream format.

  • Pipeline overview: G-PCC directly encodes events in the native (x,y,t) domain as 3D point clouds, unlike aggregation-based coding through histogram frames.The encoder partitions the stream into chunks, separates polarities, and independently compresses the resulting point clouds.
  • Temporal partitioning: Fixed-duration temporal chunks bound each point cloud’s temporal extent, system latency, and encoder memory footprint.The 100 ms duration used here was selected to keep point clouds small enough for efficient octree coding while maintaining acceptable latency.
  • Point-cloud creation: Each chunk’s events are separated by polarity and mapped into independent 3D point clouds using spatial coordinates and timestamp-derived depth.Polarity is represented by stream identity rather than an encoded point attribute.
  • Coordinate mapping: Spatial and temporal coordinates are mapped onto a common 10-bit integer grid using fixed, data-independent ranges.A shared spatial scale preserves the sensor aspect ratio, while the within-chunk timestamp offset is mapped over the full chunk duration.
  • Decoding and reconstruction: Known chunk, sensor, duration, and grid parameters allow coordinate inversion without per-chunk side information.Decoded coordinates are mapped back to native pixels and absolute timestamps, polarity is restored from stream identity, and events are globally sorted by timestamp.
  • G-PCC coding: G-PCC independently compresses each point cloud with octree geometry coding, where duplicate quantized points may merge and reduce the reconstructed event count.The operating parameter positionQuantizationScale controls geometry quantization and the bitrate–geometric-distortion tradeoff.

B. CODEC OPERATING POINTS

The point-cloud codec controls rate–distortion operating points through geometry quantization rather than temporal aggregation. Quantization jointly affects spatial and temporal coordinates on the shared grid.

  • Codec operating points: The parameter positionQuantizationScale, denoted q, controls the point cloud codec’s bitrate–geometric-distortion tradeoff.q = 1.0 provides lossless geometry coding, while lower values reduce coordinate precision and bitrate.
  • Codec operating points: Unlike aggregation-based coding, the point-cloud pipeline quantizes each event timestamp jointly with spatial coordinates on a common grid.It processes timestamps individually and does not introduce an explicit temporal aggregation step.

V. TASK-DRIVEN EVALUATION FRAMEWORK

The evaluation uses a reference-relative protocol to isolate compression effects on downstream tasks. The two pipelines are evaluated independently across representative tasks using established models and benchmark datasets.

  • Reference-relative evaluation: The reference-relative protocol compares task outputs from uncompressed and decoded event streams for each codec operating point.Rate–utility curves relate coding rate to downstream task performance.
  • Reference-relative evaluation: Comparing decoded outputs with uncompressed references isolates compression effects from intrinsic downstream-model errors.The same protocol is used across all downstream tasks, with only the task-specific metric changing.
  • Downstream tasks: Three representative tasks are evaluated with established models and widely adopted benchmark datasets, alongside a fourth task in a later section.The stated task set covers video reconstruction, object detection, optical flow estimation, and asynchronous feature tracking.
  • Evaluation design: The two compression pipelines are assessed independently at their respective operating points using the same reference-relative protocol.This enables comparison of compression impacts across codecs and downstream tasks.

B. IMPACT ASSESSMENT OF EVENT CODECS

The impact assessment relates codec operating points to downstream task performance through rate–utility curves. Lossless coding-engine modes can still retain preprocessing-induced degradation, while JPEG 2000 preserves sparse histogram structure better than the tested alternatives.

  • Impact assessment: Rate–utility curves show how coding rate relates to downstream task performance for the aggregation-based and point-cloud codecs.The curves are produced by evaluating every codec operating point under the reference-relative protocol.
  • Lossless operating modes: In lossless JPEG 2000 and G-PCC modes, any residual task degradation comes from preprocessing quantization rather than coding loss.JPEG 2000 reconstructs count frames bit-exactly, while G-PCC with q = 1.0 reconstructs point-cloud geometry losslessly.
  • Image-codec comparison: At 0.2 bpp, AVIF retained 19% and JPEG XL retained 37% of original events, whereas JPEG 2000 preserved 80%.JPEG XL also introduced phantom events at its highest quality setting, while JPEG 2000 became accurate at higher bitrates.
  • Image-codec comparison: Block-transform codecs can spread isolated histogram energy into neighboring regions, creating spurious values and altering sparse event structure.This mechanism helps explain why their natural-image compression advantages may not transfer to event histogram frames.

2) Impact on Task Performance

The evaluation compares downstream task performance under aggregation-based and point cloud-based event compression, while additional image-codec experiments examine reconstruction quality and sparse-event preservation.

  • The aggregation-based and point cloud-based codecs are evaluated through rate-utility curves across three downstream tasks under the RR protocol.Aggregation-based curves use convex-hull operating points, while each point cloud-based curve corresponds to one sequence.
  • JPEG 2000 reconstructions preserve scene structure and degrade gracefully as bitrate decreases, while JPEG XL progressively loses structural detail.Figure 8 compares four lossy bitrates and lossless coding at the same time instant and grayscale scale.
  • JPEG XL introduces phantom events during lossy coding, whereas JPEG 2000 remains close to a count ratio of 1.0 across operating points.Across seven ECD sequences, JPEG XL reaches count ratios between 1.13 and 1.26 at lossy settings.

VI. EVENT STREAM DISTORTION METRICS

Existing event-stream metrics measure reconstruction fidelity but do not explicitly separate compression-induced event loss from spurious event generation.

  • Figure 6 illustrates sparse-histogram distortions produced by three image coding schemes under specified operating conditions.The example uses ECD boxes_6dof, frame 0, negative polarity, with 98.9% zeros and 483 events.
  • Prior metrics include ASTSM, PSNR E2E, Spatial Fidelity, Temporal Fidelity, Frame PSNR, and Frame MSE.These metrics capture complementary aspects of spatiotemporal reconstruction quality.
  • Existing metrics generally measure overall fidelity without distinguishing event loss from spurious event generation.The distinction is important because compression can remove original events or create false activity.

A. PROPOSED CLASSIFICATION-BASED METRICS

The paper adopts classification-based metrics that distinguish event-level compression errors by comparing binary activity locations in original and reconstructed streams.

  • The classification-based approach separates event loss from event hallucination, which conventional PSNR combines into one score.Event loss removes original events, while hallucination introduces events at originally inactive locations.
  • Metric computation aggregates both streams into binary frames marking whether any event occurred at each pixel during an interval.This focuses evaluation on activity-location preservation rather than exact event counts and addresses severe class imbalance.
  • The 15 ms metric interval is decoupled from codec aggregation settings to support codec-agnostic comparisons.The interval is used for the ECD, Gen1, and MVSEC datasets in this work.
  • Recall isolates event survival, IoU measures overlap, Markedness detects hallucinated activity, and MCC and Cohen’s κ provide imbalance-aware summaries.Cohen’s κ exposes trivial all-zero predictions that can achieve high accuracy while missing rare event pixels.
  • Figure 7 averages HyperE2VID rate-distortion curves over seven ECD sequences, with shaded regions showing ±1 standard deviation.Figure 8 provides corresponding same-time visual comparisons across lossy bitrates and lossless coding.
  • The five metrics enable event-level codec comparison without computationally expensive downstream task evaluation.They require only original and reconstructed event streams and apply across aggregation-based, point cloud-based, and learned codecs.

B. METRIC-TO-TASK CORRELATION ASSESSMENT

Metric-to-task assessment fits distortion-to-performance mappings and evaluates how well they predict downstream degradation under the RR protocol.

  • Correlation pools pair distortion metric values X with task-performance values Y across selected lossy operating points and dataset sequences.For the aggregation codec, only rate-utility Pareto-optimal points are retained per sequence.
  • A cubic polynomial maps each distortion metric to predicted task performance before correlation is measured.Least-squares fitting accounts for differences in scale and curvature between distortion and task-scoring metrics.
  • PLCC and SROCC quantify prediction accuracy between fitted predictions and observed task-specific performance.SROCC ≥0.80 is treated as a reliable prediction threshold when the mapping is monotonic over the data range.
  • Correlations are reported separately for JPEG 2000 and G-PCC to test whether predictive power depends on codec architecture.The evaluation also considers cross-codec predictive behavior.

C. CORRELATION ASSESSMENT RESULTS

The proposed distortion metrics correlate reliably with downstream task degradation across codecs and tasks, while benchmark performance varies by task and codec. Asynchronous feature tracking is more temporally sensitive and exposes residual event-level distortions that other tasks partly mask.

  • Frame PSNR and Frame MSE correlations depend on both task and codec, declining substantially when codecs are pooled for object detection and optical flow estimation.For pooled object detection and optical flow, Frame PSNR reaches 0.516 and 0.722, while Frame MSE reaches 0.590 and 0.775.
  • Across pooled operating points, proposed metrics have mean SROCC 0.878–0.890 and mean PLCC 0.858–0.872 across the three tasks.All five reach at least 0.834 SROCC and 0.801 PLCC on every individual task, with no single metric consistently dominating.
  • All five proposed metrics achieve SROCC ≥0.80 across three tasks for each codec separately and for pooled codec operating points.None of the six benchmark metrics meets this criterion consistently across all three tasks and codec settings.
  • Tracking performance never recovers the reference at the highest bitrate because temporal aggregation and quantized point-cloud coordinates introduce irreversible event-level distortions.These distortions are clearer for event-by-event tracking than for tasks that re-bin decoded streams before inference.
  • For asynchronous feature tracking, proposed metrics reach pooled SROCC values of 0.788–0.808, with Frame PSNR the closest benchmark at 0.787.Four proposed metrics exceed the 0.80 threshold, although none does so for the point-cloud codec alone; pooled PLCC instead favors Frame PSNR at 0.818.
  • Feature tracking correlations are highest with short temporal windows and decrease as the window increases, especially for the aggregation-based codec.The 15 ms setting remains within 0.05 SROCC of the maximum for every task-codec combination, while aggregation-based tracking peaks at 0.85 at 3 ms versus 0.81 at 15 ms.

VII. CONCLUSIONS

The paper evaluates two distinct event compression pipelines with a unified task-driven framework and introduces classification-based distortion metrics for event compression. The proposed metrics predict task degradation across codec architectures, supporting codec-independent evaluation, while future work targets direct rate-distortion optimization and adaptive temporal windows.

  • The paper combines JPEG 2000 aggregation-based coding and frame-free G-PCC point-cloud coding with reference-relative evaluation across four downstream vision tasks.
  • Classification-based and anomaly-detection metrics are applied to event compression for the first time and compared with six existing benchmarks.The five proposed metrics are Cohen’s Kappa, MCC, Markedness, IoU, and Recall.
  • All five proposed metrics exceed 0.80 in SROCC and PLCC on every task-codec combination and pooled codec scores, while existing benchmarks generally do not.On asynchronous feature tracking, pooled proposed-metric SROCC remains 0.788–0.808.
  • The metrics can serve as codec-independent performance indicators for replacing computationally expensive task-specific evaluation during rate-distortion optimization.
  • Future work includes using the metrics directly for codec rate-distortion optimization and adapting temporal aggregation windows to content sparsity and downstream task characteristics.
Loading 2608.28429v1…