EventSeeColor: Learning to Reconstruct Color from Monochrome Events

1. Introduction

• Grayscale event reconstructions recover scene structure but omit color distinctions, including differences between surfaces with similar intensity.The paper gives road markings and vehicle surfaces as examples.Source: §1 · p. 1
• Monochrome event activity may provide color-inference cues because event timing and polarity depend on temporal changes in measured intensity.Measured intensity depends on incident light and the sensor’s spectral response.Source: §1 · pp. 1–2
• Semantic-only colorizers may not faithfully reconstruct color, while event-driven video generation can introduce hallucinated details and artifacts.The paper notes that prior methods do not explicitly test whether aligned events improve color accuracy beyond grayscale guidance.Source: §1 · p. 2
• On paired real PEOD and CoSEC recordings, EventSeeColor achieves the best structural similarity, perceptual fidelity, and chromatic-region color accuracy among external methods.The evaluation also reports the lowest CF-MAE and NIQE, and the second-lowest FID.Source: §1 · p. 2
• EventSeeColor jointly uses reconstructed grayscale’s spatial and semantic content and aligned events’ temporal cues to guide a diffusion colorizer.Balanced Chroma Refinement supports regional color recovery, while Local Detail Refinement corrects brightness, color, and texture in rendered RGB.Source: §1 · p. 2

2. Related Work

• Grayscale reconstruction methods recover intensity rather than color differences, while EventSeeColor uses their spatial structure and aligned events to infer color.Ordinary monochrome event recordings lack the additional spectral measurements used by event-based color and spectral sensing approaches.Source: §2 · p. 2
• Grayscale colorizers infer chroma from image content and appearance priors but do not directly use event timing, polarity, or activity.EventSeeColor adds aligned-event features and regional chroma supervision while preserving neutral appearance.Source: §2 · p. 2
• Event-to-video methods either use external color references or generate RGB frames from sparse events with diffusion priors, which can introduce artifacts.The related work notes that these methods do not directly evaluate whether events improve grayscale image colorization.Source: §2 · pp. 2–3

3. Methodology

• The pipeline uses E2VID grayscale and aligned event data to guide a diffusion colorizer, then applies chroma refinement and local RGB correction.Validity masks identify reliably aligned pixels; RGB images serve as training supervision and evaluation targets, while scene-dependent inference inputs come from the event stream.Source: §3 · p. 3

3.1. Event and Grayscale Fusion

• EGF combines grayscale structure with multiscale aligned-event features so temporal activity can guide color inference.Its learned gate uses grayscale context, while the support map places event conditioning at observed activity; pooled projections guide the diffusion backbone through ControlNet.Source: §3.1 · p. 4

3.2. Balanced Chroma Refinement

• BCR adjusts local hue and chroma to recover colored surfaces while preserving neutral appearance, then renders an intermediate RGB image anchored to reconstructed intensity.Separate image and event encoders feed a joint context network that predicts channelwise gains and a chroma residual.Source: §3.2 · p. 4

3.3. Local Detail Refinement

• LDR corrects brightness and texture errors that chroma correction alone cannot remove, using intermediate RGB, grayscale, event activity, and validity masks.Shallow features preserve fine structure and a downsampled context branch supplies wider receptive field; the network predicts a bounded RGB residual.Source: §3.3 · p. 4

3.4. Training Objectives

• Diffusion adaptation trains EGF, an auxiliary chroma head, and LoRA parameters while keeping pretrained weights fixed.The objective combines valid-area denoising error with auxiliary chroma supervision from the fused representation.Source: §3.4 · p. 4
• BCR’s region-balanced objective gives target regions equal weighting, preventing large regions from dominating supervision for smaller colored surfaces.Region assignments use training targets and are absent at inference.Source: §3.4 · pp. 4–5
• LDR training combines RGB reconstruction and gradient losses with region-balanced chroma regression, LPIPS supervision, and a clipping penalty.Source: §3.4 · p. 5

4. Dataset Construction and Curation

• PEOD and CoSEC were not designed for supervised color reconstruction, and some RGB targets have over- or underexposed regions that obscure color and texture.Such targets can encourage the model to learn exposure artifacts.Source: §4 · p. 5
• Each paired sample contains a 256 × 256 RGB crop, aligned events, and an event-reconstructed grayscale image, with validity masks restricting supervision to aligned regions.PEOD uses a fixed temporal offset and sequence-level geometric correction; CoSEC uses coaxially aligned RGB and event coordinates.Source: §4 · p. 5
• Curation rejects crops with large residual offsets, weak edge correspondence, excessive clipping, or large bright or dark areas.The retained training split contains 8,068 crops from 1,755 target frames, and evaluation uses 2,220 crops from 491 frames; all methods share retained targets and valid regions.Source: §4 · p. 5

5.2. Comparison with Existing Methods

• EventSeeColor achieves the best PSNR, SSIM, LPIPS, DISTS, and chromatic CIEDE2000 among compared methods on the combined evaluation set.It also has the lowest CF-MAE and NIQE, and the second-lowest FID.Source: Table 1 · p. 6, §5.2 · p. 6

5.1. Experimental Setup

• The evaluation compares colorization, an alternative grayscale front end, and event-video reconstruction baselines using native RGB outputs and aligned crops.The setup reports 256 × 256 aligned crops and 512 × 512 diffusion-colorizer inputs; PSNR and SSIM assess fidelity, while LPIPS, DISTS, and CIEDE2000 assess perceptual and chromatic agreement.Source: §5.1 · pp. 5–6

5.2. Comparison with Existing Methods

• Across vegetation, buildings, roads, and vehicles, EventSeeColor recovers surface colors and details while reducing false coloration on neutral surfaces.The examples include concrete, foliage, sky, and road; LongE2V produces nearly achromatic outputs on many colored surfaces.Source: §5.2 · p. 6, Figure 4 · p. 7, Figure 5 · p. 8
• Green regions have the lowest regional CIEDE2000 error, followed by blue and yellow among the chromatic groups.Figure 6 reports crop-averaged regional CIEDE2000, for which lower is better.Source: §5.2 · p. 6, Figure 6 · p. 8

5.3. Ablation Studies

• Among component and event-guidance configurations, EventSeeColor has the highest SSIM and lowest LPIPS, DISTS, and chromatic error.Removing BCR increases chromatic error and desaturation; removing LDR retains more chroma but lowers RGB fidelity and perceptual agreement.Source: §5.3 · p. 8, Table 2 · p. 8

• EventSeeColor reconstructs scene color from monochrome events through joint grayscale and event guidance.Region-balanced chroma supervision and local RGB refinement improve surface colors, brightness, and detail.Source: §6 · p. 8
• The study curates paired data from PEOD and CoSEC to support supervised color reconstruction.Source: §6 · p. 8
• On the combined evaluation, EventSeeColor achieves the best structural similarity, perceptual fidelity, and chromatic-region color accuracy among external baselines.Source: §6 · p. 8
• The ablations provide empirical evidence that temporal event activity contributes to color inference beyond grayscale semantics.The conclusion frames color reconstruction as expanding the role of monochrome event sensing in recovering scene appearance.Source: §6 · p. 8

A. Research Motivation

• Event cameras provide spatial and temporal measurements used in applications including 3D reconstruction, pose estimation, and optical flow.Prior work also combines image appearance with temporal events for tracking, interpolation, and low-light enhancement.Source: A. Research Motivation · p. 13
• Grayscale event reconstructions omit color differences between surfaces with similar intensity, although color can distinguish road markings, vegetation, and vehicle surfaces.Source: A. Research Motivation · p. 13
• Grayscale images provide spatial and semantic cues, while aligned events retain local timing, polarity, and activity for investigating color reconstruction beyond grayscale guidance.Source: A. Research Motivation · p. 13
• The paper proposes reconstructed color as a potential bridge to appearance-aware visual understanding using monochrome event recordings as the sole sensor input.The authors suggest that fusing RGB appearance features with event-derived structural and motion features could improve recognition, pose estimation, and 3D reconstruction.Source: A. Research Motivation · p. 13

B.1. Event Representation and Lightness Calibration

• For each target time, E2VID reconstructs grayscale from one second of events through twenty 50 ms updates; the aligned event tensor covers the final 50 ms and has 24 channels.The grayscale is percentile-normalized, while the event tensor combines interpolated polarity bins, count maps, and timestamp surfaces.Source: §B.1 · p. 13
• A nonnegative-slope affine calibration maps E2VID lightness using fixed dataset-specific coefficients; gamut mapping then reduces out-of-gamut chroma while preserving Lab lightness and hue.The calibrated lightness defines diffusion RGB targets and the lightness used to render BCR outputs.Source: §B.1 · p. 14

B.2. Fusion and Fixed Prompt Settings

• Event and Grayscale Fusion combines separate grayscale and event branches across three scales, initializing its projected fused hint to the pretrained grayscale hint.For 256×256 inputs, the fused hint has 320 channels at 64×64 resolution; a fixed photographic prompt is shared across images.Source: §B.2 · p. 14

B.3. Balanced Chroma Refinement

• BCR uses separate image and event encoders to predict channelwise chroma gains and residuals, initialized to preserve the input chroma.The gain adjusts the a and b channels independently, while the residual corrects local hue and chroma.Source: §B.3 · p. 14

B.4. Local Detail Refinement; B.6. Local Detail Refinement Objectives

• LDR predicts a bounded RGB correction from the intermediate image, grayscale, event activity, and validity mask, preserving the intermediate image outside valid regions.Its objectives include RGB reconstruction and neighbor-gradient supervision, region-balanced chroma regression, LPIPS, and a clipping penalty.Source: §B.4 · p. 14, §B.6 · p. 15

B.5. Training Objectives

• BCR training combines weighted chroma regression with global and region terms, giving nonempty neutral, transitional, and chromatic subsets equal region weight.Additional objectives supervise chroma shortfall, neutral chroma, and warm-colored regions; auxiliary chroma supervision uses pooled targets and valid-area weighting.Source: §B.5 · pp. 14–15

B.7. Optional Grayscale Scene Prompt

• On 2,220 evaluation samples, adding the Grayscale Scene Prompt changes PSNR from 16.010 to 16.023 dB but increases LPIPS from 0.3329 to 0.3375 and chromatic-region chroma RMSE from 8.894 to 9.031, so the fixed prompt is used.The optional prompt describes objects, arrangement, and materials from grayscale, with explicit color words removed; it uses no RGB reference.Source: §B.7 · p. 16

C.1. Paired Data and Sequence Splits

• The paired data split is disjoint by recording sequence within each dataset, and evaluation uses fixed target frames and crop coordinates across methods.PEOD evaluation uses sequences 036, 063, 070, 078, and 096; CoSEC evaluation uses City Day 000.Source: §C.1 · p. 16

C.2. Native RGB Evaluation

• Methods are scored in native RGB against the same aligned targets, without fitting outputs to evaluation images.Image colorizers generally receive the same normalized event-reconstructed grayscale crop, while direct event reconstruction methods use their released procedures.Source: §C.2 · p. 16
• Each crop receives equal weight in combined means, and regional scores include only crops containing the corresponding target region.The evaluation includes 2,076 crops with chromatic support, 1,905 with warm support, and 2,220 with neutral support.Source: §C.2 · p. 16

C.3. Reference Image and Regional Color Metrics

• Paired fidelity and perceptual measures include PSNR, SSIM, LPIPS, and DISTS; regional color evaluation uses chroma RMSE and CIEDE2000 on target-defined regions.Chromatic pixels have target C* > 10 and neutral pixels C* < 5; warm pixels are chromatic targets in the specified hue interval.Source: §C.3 · pp. 16–17

C.4. Distribution and Naturalness Metrics

• FID and KID compare image distributions, CF-MAE measures per-crop colorfulness error, and NIQE and BRISQUE assess natural-image statistics independently of paired targets.Paired perceptual and regional color metrics instead measure reconstruction accuracy against the aligned RGB target.Source: §C.4 · p. 17

C.5. Data Construction and Curation Details

• Screening leaves 10,288 crops: 8,068 for training and 2,220 for evaluation, after exposure screening excludes 66,124 candidates and alignment screening excludes another 13,124.The evaluation set uses fixed target frames and common crop coordinates for every method.Source: §C.5 · pp. 17–18

D.1. EventSeeColor

• Adaptation trains EGF and LoRA parameters while keeping pretrained diffusion weights fixed, then trains BCR and LDR in separate stages with upstream components frozen.Training uses the training partition; RGB targets are accessed by losses during LDR training.Source: §D.1 · p. 18
• At inference, EventSeeColor samples with DDIM, decodes grayscale-conditioned latents, then applies BCR and LDR to produce the native RGB prediction.Grayscale supplies initial intensity structure, while event features guide color generation and refinement.Source: §D.1 · p. 18

D.3. Recent Baselines Not Included

• Four recent colorization methods are excluded from quantitative comparison because official pretrained checkpoints were not publicly accessible as of 15 October 2026.The excluded methods are SeAda, Beyond Fixed Luminance, Color-Turbo, and ColorFLUX.Source: §D.3 · p. 20

D.4. Ablation Settings

• The component ablations remove EGF event features, BCR, all event inputs, or LDR, while objective ablations remove individual BCR or LDR supervision terms.The achromatic-input experiment substitutes a calibrated-lightness achromatic image for BCR’s RGB output while retaining LDR inputs and training schedule.Source: §D.4 · p. 20

D.5. Efficiency Measurement

• EventSeeColor adds 371,255 inference parameters, or 0.0244% of Control Color’s count, and LDR alone averages 0.886 ms per crop on an RTX 5090.The latency excludes grayscale reconstruction, diffusion sampling, BCR, data transfer, file I/O, and metric computation.Source: §D.5 · p. 20

E.1. Distribution and Appearance

• EvDiff leads FID and KID, while EventSeeColor has the lowest CF-MAE and NIQE; EvDiff and LongE2V have the lowest full-image CIEDE2000 scores.Several image colorizers have lower BRISQUE scores.Source: §E.1 · p. 20

E.2. Regional Color Accuracy

• EventSeeColor has the lowest chromatic-region and warm-region chroma RMSE, while limiting strong warm coloration of neutral pixels.Lower desaturation can accompany larger chroma errors or stronger false color; nearly achromatic LongE2V has the lowest full-image and neutral-region chroma RMSE while discarding much target chromatic color.Source: §E.2 · pp. 20–21

E.3. Component, Event Guidance, and Supervision Results

• Event–grayscale fusion reduces ab RMSE by 6.56% relative to LoRA alone, while the complete method improves ab RMSE, approximate ∆E76, and PSNR under the common scoring protocol.LoRA alone reduces ab RMSE by 24.21% relative to frozen Control Color; adding a grayscale adapter reduces it by a further 3.45%, and event–grayscale fusion improves on that adapter by 3.22%.Source: §E.3 · p. 21
• The achromatic-input experiment has PSNR 16.086, SSIM 0.6497, and LPIPS 0.3313, but chromatic-region chroma RMSE 10.368 and desaturation 77.574%.EventSeeColor has chromatic-region chroma RMSE 8.894 and desaturation 36.603%; without LDR, desaturation is lowest, while EventSeeColor has stronger RGB fidelity and perceptual agreement.Source: §E.3 · p. 21

E.4. Additional Qualitative Comparisons

• Across foliage, roads, and structures, EventSeeColor preserves color separation and marking geometry, while several baselines introduce color casts or alter local texture and shape.Its road texture is smoother than the RGB targets, with local brightness differences.Source: §E.4 · p. 21

E.5. Qualitative Component and Supervision Comparisons

• Component and supervision comparisons show that removing BCR shifts color, removing LDR softens images and lowers local contrast, and the full configuration has the best overall visual quality.Removing perceptual supervision softens leaf boundaries and building detail; removing color supervision changes color intensity, while removing gradient supervision produces subtler visual differences in the examples.Source: §E.5 · pp. 21, 23