PaperlayerView exact revision on arXiv

Source-linked AI summary

RIDE-Splat: Reflection Removal for In-the-Wild Driving Environments

Kevin David Hayes, Alex Hanson, Allen Tu, Pranav Asthana, Matthias Zwicker, Tom Goldstein

arXiv:2610.34632v1cs.CVcs.AI

TL;DR

Windshield reflections remain attached to the camera while road scenes exhibit parallax, undermining the world-space consistency assumed by 3DGS. RIDE-Splat assigns transmission to a 3DGS scene and reflections to a shared signed image, then restricts the reflection layer through self-localization. Across three benchmarks it outperforms the compared methods on every aggregate metric.

  • Problem

    Camera-attached windshield reflections and parallax-bearing road scenes violate standard 3DGS's world-space consistency, and reflected content can contaminate reconstructed scenes.

    Source: §I · p. 1

  • Method

    RIDE-Splat assigns transmitted content to a 3DGS scene and the dominant camera-attached reflection to one shared signed image, then uses self-localization to restrict where the reflection model acts.

    Source: §I · p. 1, C. Reflection Self-Localization · p. 3

  • Results

    Across the RIDE-Splat Dataset, XLD, and Wayve, RIDE-Splat ranks first on all nine aggregate metrics; on XLD, it improves PSNR by 3.59 dB over the next-best baseline.

    Source: §VI · p. 5

  • Takeaways & Limitations

    Historical dashcam archives could support closed-loop simulation, sensor-realistic augmentation, and rare-event replay without dedicated recapture.

    Source: §I · p. 1

    RIDE-Splat assumes dominant windshield reflections are semi-transparent and approximately image-fixed; substantial independent motion or deformation may leave residual contamination.

    Source: §VII · p. 7

Abstract

from arXiv · show

Driving video captured through a windshield violates the world-space consistency assumed by 3D Gaussian Splatting (3DGS): the road scene exhibits parallax while cabin reflections remain attached to the camera, causing standard 3DGS to embed reflected content into the reconstructed scene. We introduce RIDE-Splat, a pipeline for reflection-free 3D reconstruction from posed driving video without clean images, reflection masks, or pretrained networks. RIDE-Splat assigns world-consistent transmission through the windshield to a 3DGS scene and the dominant camera-attached reflection to one shared, signed RGB image. It builds the clean reconstruction in two passes. First, it learns the 3D geometry and the signed reflection image, representing both added radiance and transmission attenuation. Then, it uses the reflection-removed training images to fit a clean 3DGS model. At inference, the reflection image is discarded, preserving the standard 3DGS rendering path. We also introduce the RIDE-Splat Dataset, the first 3D reconstruction benchmark for reflection removal from driving video -- with real driving scenes, physical windshield reflections in training views, and separately captured non-reflective evaluation images. Across the RIDE-Splat Dataset, XLD, and Wayve, RIDE-Splat outperforms 3DGS, DC-Gaussian, and RobustSplat on all nine aggregate metrics, improving PSNR by up to 3.59 dB and reducing LPIPS by up to 0.0876. We achieve the highest metrics in 38 out of 42 per-scene comparisons and removes windshield reflections that remain visible in the baselines while preserving the transmitted scene.

I. INTRODUCTION

Windshield reflections are camera-attached while road content exhibits parallax, so standard 3DGS can embed reflections into the reconstructed scene. RIDE-Splat separates these signals by coordinate consistency, introduces a benchmark with real windshield reflections and clean evaluation traversals, and outperforms prior methods across benchmarks.

I. INTRODUCTION

  • Standard 3DGS can fit camera-attached windshield reflections as scene geometry or appearance, contaminating novel views.Single-image removal misses the multi-view cue, while rejecting contaminated observations discards valid transmission supervision.Source: §I · p. 1
  • Historical dashcam archives span diverse roads, weather, lighting, and rare events, but windshield reflections make them poorly suited to clean-imagery reconstruction pipelines.Reflection-free reconstructions from these recordings could support simulation, sensor-realistic augmentation, and rare-event replay without dedicated recapture.Source: §I · p. 1
  • RIDE-Splat assigns world-consistent transmission to a 3DGS scene and the dominant camera-attached reflection to one shared, signed RGB image.The image represents added reflected radiance and transmission attenuation, while the coordinate-based assignment prevents it from following parallax-bearing road texture.Source: §I · p. 1
  • The RIDE-Splat Dataset pairs reflective training traversals with separately captured non-reflective evaluation traversals for real driving scenes.The paired traversals use the same rigid camera mount and are recorded within minutes, with reflection-inducing materials removed for evaluation.Source: §I · p. 2
  • Across the RIDE-Splat Dataset, XLD, and Wayve, RIDE-Splat outperforms 3DGS, DC-Gaussian, and RobustSplat on every aggregate metric under physical and synthetic reflections.It improves PSNR by up to 3.59 dB and LPIPS by up to 0.0876, with higher performance in 38 of 42 per-scene metric comparisons.Source: §I · p. 2

II. RELATED WORK

  • Unlike reflection-aware radiance-field methods and Flash-Splat, which requires paired flash and no-flash captures, RIDE-Splat learns an image-fixed windshield layer from reflective driving video alone.DC-Gaussian instead predicts a per-view reflection field from camera pose; RIDE-Splat uses one shared signed image as a coordinate-based separation prior.Source: §II · p. 2

III. METHOD

RIDE-Splat reconstructs a transmitted 3DGS scene from posed reflective frames without paired clean images or reflection masks. It jointly fits a shared signed reflection image, refits using fixed spatial support, and discards the reflection layer at inference.

III. METHOD

  • At inference, RIDE-Splat renders reflection-free novel views from the 3DGS scene using standard rasterization.The model is trained on posed reflective frames without paired clean images or reflection masks.Source: §III · p. 2, Figure 2 · p. 3
  • A shared signed RGB image models the residual from both added reflected radiance and attenuation of transmitted light.The predicted observed frame is the rendered transmission image plus this shared reflection image.Source: A. 3D Scene and Signed Reflection Model · p. 2
  • Because the reflection image is shared across views, it can represent camera-attached content but cannot follow scene parallax.The model uses no view conditioning, network, hash grid, or per-frame coefficients.Source: A. 3D Scene and Signed Reflection Model · p. 2
  • The pipeline jointly optimizes a 3DGS scene and shared signed reflection image, then derives fixed spatial support and refits both representations from scratch.The fixed-support refit adds no optimized parameters, and the reflection layer is discarded at inference.Source: §III · p. 2, Figure 2 · p. 3

B. Coordinate-Based Joint Optimization

Coordinate-based joint optimization uses camera motion to favor world-consistent content in the 3DGS scene and camera-static residuals in the shared reflection image. A geometry warm-up and sparsity penalty guide this decomposition.

B. Coordinate-Based Joint Optimization

  • Joint optimization favors scene content in G and camera-static residuals in R because road texture exhibits parallax while cabin reflections remain nearly image-fixed.The separation is not unique: R can also absorb camera-static scene content such as the hood or a co-moving vehicle.Source: B. Coordinate-Based Joint Optimization · p. 3
  • A geometry warm-up delays R, while an ℓ1 penalty biases ambiguous pixels toward the 3DGS scene.The warm-up establishes initial scene geometry before R can absorb persistent image-space residuals.Source: B. Coordinate-Based Joint Optimization · p. 3
  • Averaging the reflection penalty over channels and pixels keeps it on the same scale as the photometric objective.The regularizer resolves ambiguous pixels in favor of the 3DGS scene.Source: B. Coordinate-Based Joint Optimization · p. 3

C. Reflection Self-Localization

Reflection self-localization estimates where the shared reflection layer should act, then refits the scene and reflection under that frozen spatial weight. This localizes reflection support while limiting scene leakage without adding optimized parameters.

C. Reflection Self-Localization

  • The method derives a continuous spatial weight from the recovered reflection image to restrict where the reflection model may act.It takes the maximum absolute RGB response per pixel, smooths those responses with a Gaussian kernel, and normalizes the result.Source: C. Reflection Self-Localization · p. 3
  • The weight is normalized across pixels, with Gaussian bandwidth σ controlling the estimated support's extent.A small ϵ ensures numerical stability, and W(u) lies in [0, 1].Source: C. Reflection Self-Localization · p. 3
  • The method restarts joint optimization of G and R from scratch with W frozen, training the final decomposition under localized reflection support.The refit retains the objective and gated 3DGS warm-up, rather than inheriting leakage from the joint fit.Source: C. Reflection Self-Localization · pp. 3–4
  • Where reflection evidence is weak, W suppresses the effective reflection so G must explain those pixels; inference then omits R and uses standard 3DGS rendering.The frozen-weight second pass adds no optimized parameters and provides gains over the shared reflection layer alone.Source: C. Reflection Self-Localization · p. 4

IV. RIDE-SPLAT DATASET

The RIDE-Splat Dataset pairs reflective training traversals of real driving scenes with separately captured non-reflective evaluation traversals. This provides a benchmark for transmission recovery under physical windshield reflections, with references that remain approximate across traversals.

IV. RIDE-SPLAT DATASET

  • The dataset addresses the lack of benchmarks that combine physical windshield reflections with non-reflective targets, enabling evaluation against actual clean images.It pairs reflective training traversals with separately captured non-reflective evaluation traversals of the same route.Source: §IV · p. 4
  • Reflective training footage uses common cabin objects to produce scene-specific, approximately camera-static windshield reflections.Objects included paper, laminated paper, aluminum, plastic, painted wood, and steel; the set remains fixed within each reflective traversal.Source: §IV · p. 4
  • The evaluated benchmark covers Leesburg, Martin, and Reston, spanning a historic town center, an institutional district, and suburban residential streets.The paired traversals use the same rigid camera mount and are recorded within minutes, with reflection-inducing materials removed between them.Source: §IV · p. 4
  • Evaluation frames are sampled from non-reflective traversals, and methods render at held-out poses for comparison with the corresponding clean images.Only reflective frames are used for training, without reflection masks or annotations.Source: §IV · p. 4
  • Traffic, illumination, and camera trajectories cannot be matched exactly, so the separately captured references are approximate despite controls on cross-traversal variation.The references contain minimal windshield reflections and are not used during optimization.Source: §IV · p. 4

V. EXPERIMENTS

The experiments assess RIDE-Splat across synthetic scenes, real imagery with synthetic reflections, and real driving imagery with physical windshield reflections. They also examine component contributions, hyperparameter sensitivity, and robustness to drift in the camera-static reflection prior.

V. EXPERIMENTS

  • The evaluation spans synthetic scenes, real driving imagery with synthetic reflections, and real driving imagery with physical windshield reflections.The settings test recovery of the transmitted scene as scene content and reflection formation become more realistic.Source: §V · p. 5
  • All methods use identical reflective training images and camera poses, while non-reflective targets are reserved for evaluation.Source: §V · p. 5
  • Beyond the main comparison, experiments isolate the Reflection Layer and self-localization, test hyperparameter sensitivity, and increase drift to stress-test the camera-static reflection prior.Source: §V · p. 5

A. Datasets

The evaluation covers three datasets with different combinations of scene content and reflection formation, using shared baselines and metrics. For DC-Gaussian, the authors distinguish transmission-recovery evaluation from reproduction of observed obstructed frames.

A. Datasets

  • The datasets span physical reflections in real driving scenes, synthetic scenes with synthetic reflections, and real driving imagery with synthetic reflections.RIDE-Splat uses Leesburg, Martin, and Reston; XLD includes six towns, and Wayve uses five sequences.Source: A. Datasets · p. 5

B. Baselines and Metrics

  • The comparison uses 3DGS, DC-Gaussian, and RobustSplat with identical training images, poses, and targets, reporting full-image PSNR, SSIM, and LPIPS.The baselines use their official implementations and published settings.Source: B. Baselines and Metrics · p. 5

C. DC-Gaussian Evaluation Protocol

  • DC-Gaussian's published metrics assess recomposed outputs against held-out obstructed frames, whereas this evaluation compares transmission renders directly with non-reflective targets.Targets are exact clean images for XLD and Wayve, and separately captured images for the RIDE-Splat Dataset.Source: C. DC-Gaussian Evaluation Protocol · p. 5
  • The oracle color-correction variant fits a global quadratic transform using test-target information and is excluded from best-result highlighting.The transform is not part of the published method or default evaluation and could be applied to any renderer.Source: C. DC-Gaussian Evaluation Protocol · p. 5

D. Implementation Details

RIDE-Splat achieves the strongest overall results in Table II, while qualitative comparisons show cleaner transmission than the baselines. The method’s implementation delays the shared reflection image until geometry is established and uses no pretrained image generator or reflection-removal network.

D. Implementation Details

  • Reflection-image optimization uses learning rate ηR = 0.01 and β = 0.01, and begins after a geometry warm-up ending at 2000 iterations.The self-localization pass uses σ = 8 and freezes the weight; no pretrained image generator or reflection-removal network is used.Source: D. Implementation Details · p. 5

VI. RESULTS

  • Table II reports RIDE-Splat’s strongest overall results across the RIDE-Splat Dataset, XLD, and Wayve, with metrics averaged across evaluated scenes.The + Oracle GT Color Correction variant uses each test target as reference and is excluded from best-result highlighting.Source: Table II · p. 6
  • RIDE-Splat removes reflections more effectively while preserving the transmitted scene, whereas 3DGS embeds reflections, DC-Gaussian leaves residual reflections and color shifts, and RobustSplat introduces broader artifacts.The resulting transmitted scene has sharper building and road-sign textures than DC-Gaussian and fewer artifacts than RobustSplat.Source: §VI · p. 6, Figure 4 · p. 6

A. Component Ablation

On XLD, coordinate-based joint optimization provides the primary improvement across metrics, and reflection self-localization further improves them without adding optimized parameters. Sensitivity and motion tests characterize tested settings, while the method assumes a semi-transparent, approximately image-fixed dominant reflection and has approximate real-reflection references from a limited scene subset.

A. Component Ablation

  • Coordinate-based joint optimization provides the primary improvement across all XLD metrics, and reflection self-localization further improves all metrics without adding optimized parameters.The components are applied cumulatively in the ablation.Source: A. Component Ablation · p. 7, Table IV · p. 7

B. Hyperparameter Sensitivity

  • Reducing β to 0.002 increases mean recovered |R| by 40% while PSNR and SSIM remain nearly unchanged; β = 0.05 degrades all metrics.Among tested smoothing scales, σ = 32 gives the best LPIPS, while σ = 2 lowers PSNR; performance remains stable across tested learning rates.Source: B. Hyperparameter Sensitivity · p. 7

C. Robustness to Reflection Motion

  • At 1.2 pixels-per-frame drift over 150 frames, cumulative displacement reaches 180 pixels and reflection-layer PSNR varies by 0.65 dB, versus 1.45 dB for DC-Gaussian.The test indicates tolerance to moderate drift; independent parallax or deformation remains outside the shared reflection model’s scope.Source: C. Robustness to Reflection Motion · p. 7

VII. LIMITATIONS

  • RIDE-Splat assumes the dominant reflection is semi-transparent and approximately fixed in image coordinates; substantial independent motion or deformation may leave residual contamination.Sparse forward-facing views remain a 3DGS challenge. Physical-reflection evaluation uses three scenes, and separate traversals make references approximate rather than exactly matched.Source: §VII · p. 7

VIII. APPLICATIONS AND FUTURE WORK

RIDE-Splat may turn existing dashcam video into reflection-free 3D assets for simulation and rare-event replay, and its camera-coordinate prior may extend to cameras behind transparent enclosures. The conclusion reports consistent benchmark gains and identifies the shared reflection image as the primary improvement, with self-localization as a refinement.

VIII. APPLICATIONS AND FUTURE WORK

  • RIDE-Splat can convert existing dashcam video into reflection-free 3D assets for closed-loop simulation and rare-event replay.The camera-coordinate prior may also extend to robot cameras behind protective windows or transparent housings.Source: §VIII · p. 7

IX. CONCLUSION

  • Across the RIDE-Splat Dataset, XLD, and Wayve, RIDE-Splat outperforms 3DGS, DC-Gaussian, and RobustSplat on every aggregate metric.The conclusion attributes the primary improvement to the shared reflection image and a consistent refinement to self-localization; discarding the reflection layer at inference preserves standard 3DGS rendering.Source: §IX · pp. 7–8

APPENDIX

The appendix specifies synthetic reflection settings for XLD and Wayve and reports scene-level metrics across the three evaluated datasets.

A. Synthetic Reflection Construction

  • XLD and Wayve use static synthetic reflections with zero flicker and image-space drift; Wayve’s annotated reflection support covers 31.8% of the image and is shared across five scenes.The support is used only to synthesize reflective training images, not provided to methods; Wayve uses reflection strength 0.5, no center glare, and lower obstruction strength.Source: A. Synthetic Reflection Construction · p. 9, Table VI · p. 9

B. Per-Scene Metrics

Related papers

Loading 2610.34632v1…