Depth Hypothesis Guided Iterative Refinement for Event-Image Monocular Depth Estimation

• Image-based methods can struggle with high-speed motion or poor illumination, while event cameras lack visual information in static regions, motivating complementary modalities for robust MDE.Source: §1 · p. 1
• Existing event-image methods focus on contextual feature improvement, but direct full-depth regression remains highly nonlinear and ill-posed.Source: §1 · p. 1
• HypoDepth discretizes continuous depth into candidate values within adaptive bounds from prior estimates, turning ill-posed regression into constrained search for stable residual optimization.Source: §1 · p. 1
• A 3D cost volume matches DHV embeddings with contextual features, and multi-scale correlation across resolutions guides refinement from distant layout to fine geometric detail.Source: §1 · p. 2
• HypoDepth combines resolution-specific cross-modal fusion, a shared DHV, multi-scale correlation lookup, and a GRU-based iterative unit to progressively refine depth across resolutions.Source: §1 · p. 2

• Image-based depth methods range from CNNs and discretization or scale-aware regression to Laplacian refinement, transformers, multi-frame cost volumes, and diffusion, but can fail under motion blur or poor illumination.Source: §2 · p. 2
• Event-based MDE uses recurrent, multi-scale, transformer, self-supervised, and distillation approaches, while fusion methods improve cross-modal alignment or contextual features; direct depth regression remains difficult because MDE is ill-posed.Source: §2 · p. 2
• PCDepth [22] iteratively optimizes feature-level fusion, whereas RAFT [35] uses cost-volume refinement for optical flow; HypoDepth introduces discrete depth hypotheses for iterative local search and explicit geometric refinement.Source: §2 · p. 2

3. Method

• HypoDepth combines image–event encoders, multimodal fusion, and iterative depth decoders to regress depth from coarse to fine.The decoder builds a Depth Hypothesis Volume (DHV) and a 3D cost volume between DHV and fused contextual features.Source: §3 · pp. 2–3, Figure 2 · p. 3

3.1. Event Representation

• Events are converted into an image-like voxel grid with B temporal bins for compatibility with neural-network inputs.The bin index is determined by each event’s timestamp relative to the minimum and maximum timestamps.Source: §3.1 · p. 3

3.2. Cross-Modal Context Encoder

• Separate image and event encoders extract features at 1/16, 1/8, and 1/4 resolutions, with fusion strategies adapted to each scale.Cross-attention fuses low-resolution features; convolution-based fusion combines 1/4-resolution features, with skip connections.Source: §3.2 · p. 3, Figure 3 · p. 4

3.3. Iterative Depth Decoder

• The decoder samples depth hypotheses within prior bounds, embeds them in the contextual feature space, and constructs a 3D cost volume for depth refinement.Bidirectional cross-attention lets contextual features incorporate hypothesis geometry and DHV embeddings incorporate scene semantics.Source: §3.3 · p. 4
• At each iteration, the decoder centers a local search on the DHV depth value closest to the previous estimate and uses multi-scale correlation cues to predict a residual update.A geometry encoder processes the correlation features alongside depth and contextual features before a GRU-based update unit predicts the residual.Source: §3.3 · pp. 4–5

3.4. Multi-resolution Depth Refinement

• Refinement proceeds from 1/16 to 1/4 resolution, using low-resolution stages for global consistency and higher-resolution stages for fine structural detail.The number of depth hypotheses increases with resolution; the initial depth at 1/16 resolution is zero.Source: §3.4 · p. 5

3.5. Loss

• Training uses SIlog loss between ground-truth depth and predictions from multiple iterations, with increasing weights.The loss uses variance and expectation operations, with α set to 10 and λ to 0.15.Source: §3.5 · p. 5

4. Experiments

• Experiments use MVSEC and DSEC, with different training and evaluation protocols; DSEC comparisons include base, small, and tiny variants and event-only or event-image inputs.MVSEC uses EventScape pretraining, fine-tuning on outdoor day2, and evaluation on day1 and night1; DSEC uses a split of its training set because test ground truth is unavailable.Source: §4 · p. 5, §4.1 · p. 5

4.1. Results

• 49% improvement in Abs Rel for Ours-E-B over DepthAnyEvent-R on DSEC, with HypoDepth also surpassing fusion-based PCDepth.Integrating the refinement strategy with PCDepth further improves performance across all metrics.Source: §4.1 · pp. 5–6, Table 1 · p. 6
• Ours-B achieves state-of-the-art Abs Rel results on both day and night MVSEC sequences.The comparison contrasts dynamic depth-level iterative refinement with methods focused on event-only modeling or feature-level fusion.Source: §4.1 · p. 6, Table 2 · p. 7
• After training on EventScape, HypoDepth achieves the best zero-shot performance across MVSEC and DSEC scenes, particularly on complex DSEC.The reported explanation is scene-adaptive depth hypotheses and multi-resolution refinement through multi-scale cost-volume lookups.Source: §4.1 · p. 6, Table 3 · p. 7
• Ours-E-T offers the best balance between accuracy and efficiency, making the Tiny model suitable for real-time or computationally constrained scenarios.Its lightweight 3D cost volume keeps multi-resolution computation below PCDepth, although E2Depth and DepthAnyEvent run faster at the cost of accuracy.Source: §4.1 · p. 7, Figure 6 · p. 7

4.2. Ablation Study

• Ablations report weaker performance when EIFusion, decoder cross-attention, GRU iteration, contextual features, multi-scale lookup, or DHV are removed or replaced.The reported explanations include weaker modality complementarity, semantic-geometric correspondence, iteration guidance, spatial consistency, scenario robustness, or tolerance to accumulated depth error.Source: §4.2 · p. 7, Table 4 · p. 8
• More iterations and larger lookup radii improve accuracy, while increasing iterations can raise computation and destabilize training; increasing D improves performance without added computation.Progressive low-to-high-resolution refinement performs best: 1/16 emphasizes global consistency, while 1/8 and 1/4 refine structural detail and reduce error regions.Source: Figure 7 · p. 7, §4.2 · p. 8, Figure 8 · p. 8

5. Conclusion

• HypoDepth reports the best performance on DSEC and MVSEC, with superior zero-shot generalization.Its lightweight cost volume supports refinement across multiple resolutions for robust global-to-fine depth estimation.Source: §5 · p. 8

• HypoDepth performs progressive depth refinement from 1/16 to 1/4 resolution using resolution-dependent lookup scales.At 1/16 resolution, the lookup radius already covers most or all of the feature map; further downsampling would make correlations redundant and weaken the multi-scale strategy.Source: §6 · p. 11
• Ours denotes event–image fusion, while Ours-E removes the image encoder and EIFusion module for an event-only variant.Tab. 8 summarizes the backbone, core-module channels, iterative updates, and cost-volume lookup parameters.Source: §6 · p. 11
• For a fair comparison with PCDepth, PCDepth-Ours refines at 1/8 resolution using PCDepth’s fused encoder output as contextual features.PCDepth performs iterative feature-level fusion refinement at 1/8 resolution; the refinement decoder parameters are listed in Tab. 8.Source: §6 · p. 11
• The scaled-down ablation model is trained with batch size 3 for 40k iterations.Source: §6 · p. 11

• Events consistently improve performance over image-only input, with larger gains at MVSEC on1 nighttime and on blurred DSEC; HypoDepth also generalizes better than PCDepth under blur.The blurred-DSEC model is evaluated zero-shot after training on the original DSEC; Fig. 9 provides qualitative comparisons in the blurred setting.Source: §7 · p. 11, Table 5 · p. 11, Figure 9 · p. 11
• 20 ms and 50 ms event windows achieve comparable performance, while longer windows degrade results through excess accumulation, misalignment, and motion artifacts.The paper adopts 20 ms to reduce latency and support fair performance comparison.Source: §7 · p. 11, Table 6 · p. 11
• An overly narrow DHV depth range degrades accuracy, whereas expanding the range further has only marginal impact.The DHV defines the coarse search space, while continuous residual updates refine depth and alleviate discretization errors.Source: §7 · p. 11, Table 7 · p. 12

• On DSEC [9], Ours-B produces more geometrically plausible, spatially accurate maps than PCDepth [22], while PCDepth-Ours gains sharper boundaries and Ours-B retains finer detail.The paper attributes Ours-B’s finer structural detail relative to PCDepth-Ours to its additional 1/4-resolution refinement stage.Source: §8 · p. 12
• On MVSEC, HypoDepth estimates distant and nearby structures in day and night sequences with sparse events, while PCDepth fails to recover distant regions.The paper notes stable, continuous distant estimates as relevant to perceiving distant scenes in advance.Source: §8 · p. 12
• Refinement progresses from globally consistent coarse estimates at 1/16 resolution to more pronounced geometry at 1/8 and sharper structures at 1/4.Source: §8 · p. 12

Limitations

• PCDepth-Ours outperforming Ours-B indicates that multimodal fusion still has room to improve through more adaptive use of image and event cues.The proposed direction emphasizes image features in static regions where events fail, and event spatiotemporal cues when image appearance becomes unreliable.Source: §9 · p. 12
• Non-forward camera motion and textureless regions can weaken event structure and cost-volume matching, leaving refinement reliant on contextual priors rather than meaningful correlation evidence.The authors identify weak parallax and ambiguous geometric boundaries in non-forward scenes; homogeneous surfaces also generate few events and lack discriminative image texture.Source: §9 · p. 12, Figure 10 · p. 12

Future Work

• Future work will explore multi-view models using temporal frames or multiperspective inputs to address depth-estimation limitations under non-forward motion.Source: §9 · p. 12
• The authors plan to add information such as semantic segmentation, structured light, or LiDAR, or use attention to reduce cost-volume matching ambiguity.They also plan asynchronous multimodal fusion to aggregate temporal event cues for temporally consistent, low-latency, high-frequency depth estimation.Source: §9 · pp. 12–13