PaperlayerView exact revision on arXiv

Source-linked AI summary

EvenSplat: Coupled 2D-3D Decomposition for Gaussian Splatting under Exposure and Illumination Variation

Tongyu Wu, Jacob Edwards, Ziteng Cui, Caigui Jiang, Cheng Wang

arXiv:2610.01876v1cs.CV

TL;DR

Photometric inconsistency across views can entangle capture lighting with recovered scene geometry and color. EvenSplat couples image-space decomposition with Gaussian-level illumination and uses image-specific corrections for residual variation. Across real-world and simulated benchmarks, its largest gains occur under spatially uneven and high-contrast illumination.

  • Problem

    Photometric inconsistency challenges NVS methods that assume consistent appearance across views, and fitting disagreeing inputs can entangle capture lighting with recovered geometry and color.

    Source: §1 · p. 2

  • Method

    EvenSplat jointly couples online image-space illumination decomposition to shared Gaussian-level illumination, with global and local image-specific corrections for residual variation.

    Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 5

  • Results

    Across physical and simulated benchmarks, gains are largest under SIV and HCI; on real-world HCI, EvenSplat gains 3.17 dB PSNR over the strongest baseline.

    Source: §4.2 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    EvenSplat offers a practical route to more illumination-consistent Gaussian scene representations from ordinary sRGB captures.

    Source: §5 · p. 11

    Limitations include sRGB-domain ambiguity and saturation, training-image-specific ILEC parameters, and unmodeled specularities, motion blur, and sensor noise.

    Source: §5 · p. 11

Abstract

from arXiv · show

A surface photographed under even light presents nearly the same appearance from every angle; the same surface under uneven light does not. Exposure changes between views, illumination varies within a single image, and locally strong light sources leave one region bright and its neighbor in shadow. Multi-view reconstruction methods such as 3D Gaussian Splatting treat these lighting artifacts as if they were properties of the scene, entangling capture-specific illumination with the geometry and color they recover. We present EvenSplat, a framework that separates the two. EvenSplat couples an image-space illumination decomposition with an illumination field carried by the Gaussians, so that the same explanation of the lighting is shared between the two-dimensional and three-dimensional views of the scene; a camera-response network and a local exposure-compensation module absorb the global and residual differences that remain across training images. Through extensive experiments across multiple datasets and diverse forms of uneven illumination (cross-view exposure, spatial illumination variation, and high-contrast lighting) on both real-world captured and simulated benchmarks, EvenSplat generally outperforms state-of-the-art methods, particularly under high-contrast illumination.

1 Introduction

EvenSplat addresses photometric disagreement that can entangle capture lighting with reconstructed scene geometry and color. It couples image-space decomposition with Gaussian-level illumination and evaluates the approach on real-world captures spanning three uneven-lighting conditions.

1 Introduction

  • Exposure shifts, within-image lighting variation, and bright–dark imbalance make views of the same surface disagree, challenging reconstruction methods that assume photometric consistency.Fitting such inputs can entangle capture lighting with recovered geometry and color.Source: §1 · p. 2
  • The evaluation includes nine real-world scenes: three CEV, four SIV, and two HCI scenes, with evenly illuminated reference captures for appearance evaluation.Controlled simulated benchmarks and a directional-lighting robustness experiment complement the captured dataset.Source: §1 · p. 2
  • EvenSplat couples image-space decomposition with Gaussian-level illumination, using global and local image-specific correction to handle residual variation across training views.The framework also contributes a nine-scene real-world dataset with reference captures for appearance evaluation.Source: §1 · p. 2

2 Related Work

Related work addresses challenging lighting through appearance modeling, image enhancement, and inverse rendering. EvenSplat draws on image-space decomposition while targeting severe uneven lighting without calibrated light geometry.

2.1 NVS in Challenging Lighting Conditions

  • Prior NVS work addresses appearance variation through embeddings, HDR and low-light modeling, harmonization, changing-light models, and relighting; EvenSplat targets severe uneven lighting without calibrated light geometry.Its stated aim is illumination-consistent base appearance for novel-view synthesis while retaining 3DGS’s explicit rendering structure.Source: §2.1 · pp. 2–3

2.2 Image-Space Illumination Decomposition

  • Retinex-inspired enhancement estimates illumination, while intrinsic decomposition separates appearance from shading; these lines motivate EvenSplat’s image-space branch for guiding shared 3D illumination.The related work also identifies multi-view inverse rendering that models material, spatial illumination, and camera response.Source: §2.2 · p. 3

3 EvenSplat

EvenSplat learns a shared Gaussian base-appearance representation from inconsistent views by coupling online image-space decomposition with Gaussian illumination. Image-specific corrections absorb residual variation during training, and the coupled objective links the branches to captured observations.

3 EvenSplat

  • An online image-space decomposition guides a Gaussian-level illumination field, while global and local image-specific corrections absorb residual capture variation during training.The resulting representation supports base-appearance rendering from unseen viewpoints without fitting a test image.Source: §3 · pp. 3–4, Figure 2 · p. 4

3.1 Gaussian Appearance and Illumination

  • Gaussian spherical-harmonic color represents base color, while a shared neural field predicts multiplicative illumination from position, surface normal, and viewing direction.Because the decomposition is learned in sRGB without radiometric calibration, the paper treats Gaussian base color as distinct from physical albedo.Source: §3.1 · p. 4

3.2 Image-Specific Appearance Corrections

  • The camera-response network applies a spatially uniform per-image gain, while ILEC uses a low-resolution spatial log-gain map for remaining residuals.These image-specific corrections operate during training and are set to identity at novel views.Source: §3.2 · pp. 4–5

3.3 Online Image-Space Decomposition

  • A multi-scale image-space network predicts illumination and derives base appearance by division, regularized by adaptive curves, edge-aware smoothness, and white preservation.The priors moderate extreme base-appearance intensities, smooth illumination without blurring observed boundaries, and retain approximately achromatic input regions.Source: §3.3 · p. 5, Figure 3 · p. 6

3.4 Coupled Optimization and Rendering

  • The coupled loss aligns adapted Gaussian and image-space illumination, recombines illumination with base appearance, and supervises corrected Gaussian rendering against captured images.Training runs for 10,000 iterations; ILEC activates at iteration 5,000, while the image-space decomposition and coupling loss remain active throughout.Source: §3.4 · pp. 5–6

4 Experiments

Experiments test EvenSplat on physically captured and controlled uneven-light settings, then probe its components and a heritage-digitization use case. The strongest reported gains occur under spatially uneven real-world illumination, while other settings show distinct trade-offs.

4.1 Datasets

  • The real-world benchmark has nine scenes—three CEV, four SIV, and two HCI—and the simulated benchmark perturbs six Mip-NeRF 360 scenes across the same settings.Real captures retain camera response, highlight saturation, flash falloff, and cast shadows; evenly illuminated references support appearance evaluation.Source: §4.1 · pp. 6–7, Figure 4 · p. 7

4.2 Experimental Results

  • EvenSplat leads every reported metric for real-world SIV and HCI, with a 3.17 dB PSNR gain over the strongest HCI baseline and a 1.40 dB gain over PPISP for SIV.For CEV, it has the best PSNR, while PPISP retains better SSIM and LPIPS.Source: §4.2 · p. 7, Table 1 · p. 8, Table 3 · p. 9
  • On simulated data, EvenSplat leads all metrics for CEV and SIV; under HCI it has the best PSNR (+1.56 dB) but trails PPISP in SSIM.The reported HCI trade-off is attributed to correction under severe clipping; the trend is not driven by a single scene.Source: §4.2 · pp. 7–8, Table 2 · p. 8
  • On HDR-NeRF, EvenSplat achieves 37.6 dB mean PSNR across scenes while also extracting base appearance under severe but view-consistent illumination.The method supports both rendered-observation and base-appearance predictions.Source: §4.2 · p. 8

4.3 Ablation Study

  • Ablations assign distinct roles to coupled 2D–3D decomposition, global response correction, local compensation, and image-space priors; colored illumination improves some PSNR and LPIPS values but lowers SSIM in every setting.Removing image-space decomposition costs 3.60 dB on CEV and 4.94 dB on HCI; removing ILEC costs 2.88 dB on HCI. Adaptive curve regularization matters most under HCI, while edge-aware smoothness supports structural and perceptual quality.Source: §4.3 · pp. 9–10

4.2 Experimental Results

  • 15.57 dB PSNR, 2.84 dB above Luminance-GS, gives EvenSplat the strongest measured left/right-light appearance consistency, though the score alone does not establish absolute base-appearance accuracy.The comparison uses reconstructions from capture sets that differ in illumination direction; Figure 7 includes appearance renderings and color-chart crops.Source: §4.2 · p. 8, Figure 7 · p. 10

4.4 Real Application: Digital Heritage

  • In a large indoor heritage site, EvenSplat novel views reveal architectural detail across darker regions of an unevenly exposed ceiling.The application uses ceiling digitization photographs with reliefs unevenly exposed by windows and local lighting.Source: §4.4 · p. 10, Figure 8 · p. 10

5 Conclusion

EvenSplat’s largest gains occur under spatially uneven and high-contrast illumination, where global appearance models are insufficient. Its complementary 2D and 3D branches offer a route to more illumination-consistent Gaussian representations from ordinary sRGB captures, with stated limitations.

5 Conclusion

  • EvenSplat’s largest gains occur under spatially uneven and high-contrast illumination, where global appearance models are insufficient.Ablations confirm that the 2D and 3D branches are complementary.Source: §5 · pp. 10–11
  • EvenSplat couples image-space decomposition with shared Gaussian illumination and global–local view-specific correction.The resulting representation is presented as a practical route to more illumination-consistent Gaussian scenes from ordinary sRGB captures.Source: §5 · pp. 10–11
  • The approach remains limited by sRGB-domain ambiguity and saturation, training-image-specific ILEC parameters, and unmodeled specularities, motion blur, and sensor noise.Source: §5 · p. 11

A.1 Method and Loss Details

This section details Gaussian compositing, image-space decomposition, and the regularizers grouped in R in Eq. (8).

A.1 Method and Loss Details

  • The section covers Gaussian compositing, image-space decomposition, and regularizers grouped in R in Eq. (8).Source: §A.1 · p. 15

A.1.1 Gaussian Compositing and Appearance Estimates

Gaussian compositing uses front-to-back weights based on opacity and transmittance, and reuses those weights to render both illumination-modulated color and illumination.

A.1.1 Gaussian Compositing and Appearance Estimates

  • Front-to-back compositing weights combine each Gaussian’s opacity contribution with transmittance through preceding Gaussians.Source: §A.1.1 · p. 15
  • The same compositing weights render illumination-modulated color and illumination.Source: §A.1.1 · p. 15
  • The base-appearance quotient of composited maps generally differs from P_j during training and retains image-specific corrections.Source: §A.1.1 · p. 15

A.1.2 Image-Space Architecture and Losses

The image-space branch predicts illumination and derives base appearance, then regularizes the decomposition with adaptive curve, edge-aware smoothness, and white-preservation terms.

A.1.2 Image-Space Architecture and Losses

  • A multi-scale, U-Net-style image-space network predicts illumination, and element-wise division yields base appearance.The architecture uses a view-specific embedding and skip connections between encoder and decoder features.Source: §A.1.2 · p. 15, Figure 9 · p. 15
  • Illumination alignment and image recombination connect the image-space branch to the scene.The image-space branch predicts L2D and derives R2D.Source: §A.1.2 · p. 16
  • Adaptive curve regularization selects overly dark or bright base-appearance regions and pulls them toward preferred intensities.Thresholds select where the loss applies; target intensities determine the direction of correction.Source: §A.1.2 · p. 16
  • Edge-aware smoothness encourages smooth illumination while preserving image edges by weighting illumination gradients using input-image gradients.The smoothness weight decays according to the input-gradient scale.Source: §A.1.2 · p. 16
  • White preservation uses a soft mask for bright, approximately achromatic input regions, but clipped highlights can also receive high mask values.The mask’s reliability is limited in saturated regions.Source: §A.1.2 · p. 16

A.1.3 Gaussian-Level Losses

Gaussian-level losses combine observation reconstruction with regularization of appearance and illumination, including a gradient-exclusion term and higher-order spherical-harmonic regularization.

A.1.3 Gaussian-Level Losses

  • Observation reconstruction uses the loss from the main text, and its contribution is already counted in Eq. (8).The appendix notes that the observation loss is not counted again here.Source: §A.1.3 · p. 17
  • Gradient exclusion penalizes correlated gradients between rendered base appearance and illumination maps across a multi-scale pyramid.The operator uses reflectance and illumination maps at each pyramid level, with ϕ(z) = tanh(z) to suppress small gradient noise and normalize gradient responses.Source: §A.1.3 · p. 17
  • Higher-order spherical-harmonic coefficients are regularized to discourage illumination variation from being absorbed into Gaussian appearance.The passage identifies c_i as a higher-order SH coefficient.Source: §A.1.3 · p. 17

A.1.4 ILEC Regularization

ILEC regularization uses a normalized illumination guidance map to constrain image-specific local exposure residuals, balancing limited correction with brighter-region exposure promotion.

A.1.4 ILEC Regularization

  • The normalized illumination map identifies relatively bright regions for stronger guidance and lower-illumination regions for less guidance.Its minimum guidance weight, transition steepness, and threshold control the weighting; the map weights residual regularizers, while Eq. (4) defines forward exposure adaptation.Source: §A.1.4 · p. 17
  • An ℓ1 residual-magnitude penalty limits overall log-exposure correction and favors zero correction where other objectives provide little evidence.This constrains the amplitude of local exposure residuals.Source: §A.1.4 · p. 18
  • Guidance-weighted shrinkage applies stronger shrinkage where the guidance map is smaller, while its continuous weight concentrates correction in more strongly guided regions.The spatial preference varies across the image with 1 − M_rel.Source: §A.1.4 · p. 18
  • Bright-region exposure promotion favors positive log-exposure residuals where the guidance map is large.For a fixed guidance map, positive residuals increase the multiplicative exposure factor; magnitude and shrinkage penalties and the remaining training objective balance this effect.Source: §A.1.4 · p. 18
  • The local regularization objective combines the residual constraints, with local weights distinct from the SSIM mixture coefficient in Eq. (6).The text distinguishes the local weights from λ_ssim.Source: §A.1.4 · p. 18

A.2 Experiment and Implementation Settings

The appendix specifies shared experiment settings, real and simulated data construction, and supplementary comparisons. Those evaluations include ILEC ablations, per-scene uneven-lighting results, and HDR-NeRF tests of observation rendering and base-appearance recovery.

A.2 Experiment and Implementation Settings

  • Methods use identical poses, resolutions, and train–test splits, with every eighth view held out; the reported primary comparisons do not apply test-time pose refinement, exposure fitting, affine RGB correction, or color matching.PSNR, SSIM, and LPIPS compare rendered images directly with corresponding references; training optimizes each scene for 10,000 iterations with Adam.Source: §A.2 · pp. 18–20

A.3.1 Real-world Capture Datasets

  • The real benchmark contains nine scenes: three CEV, four SIV, and two HCI scenes, with reference captures used for appearance evaluation.Captures are handheld, and automatic bracketing and burst capture reduce motion between variants, though small residual misalignment can remain.Source: §A.3.1 · p. 21, Table 8 · p. 22

A.3.2 Simulated Datasets

  • Simulated benchmarks perturb training images from six Mip-NeRF 360 scenes while retaining original images as evaluation references.CEV changes exposure across views, SIV applies opposing spatial exposure factors with blended transitions, and HCI increases contrast by three and clips the result.Source: §A.3.2 · p. 22

A.4 Supplementary Results

  • Removing guidance-weighted shrinkage drops HCI PSNR by 0.62 dB, and the ablation also reports contributions from magnitude penalty and exposure promotion.The magnitude penalty limits arbitrary local edits; exposure promotion recovers persistently dark regions; the supported conclusion favors a constrained, illumination-aware local model over an unconstrained per-image correction map.Source: §A.4 · pp. 22–23, Table 9 · p. 23
  • Real-world results report HCI gains on both high-contrast scenes and aggregate SIV improvement beyond one example, while CEV remains variable and global-correction methods can stay competitive.Table 10 provides per-scene PSNR, SSIM, and LPIPS comparisons across CEV, SIV, and HCI.Source: §A.4 · p. 23, Table 10 · p. 23
  • Simulated results extend aggregate CEV and SIV gains across scenes, while HCI outcomes remain metric-dependent and the clearest visual advantage is near exposure transitions and in shadows.Correcting clipped high-contrast inputs can improve intensity agreement while altering local structure and perceptual similarity; Figures 11–13 show the qualitative comparisons.Source: §A.4 · p. 23, Table 11 · p. 25, Figure 11 · p. 26
  • 17.91 dB mean base-appearance PSNR, versus 13.23 dB for the strongest baseline, is EvenSplat’s highest result across all four HDR-NeRF scenes.SSIM and LPIPS remain mixed, particularly on CatRoom, so the reported gain concerns recovery of the uniform-light target rather than a universal perceptual advantage.Source: §A.4 · p. 24, Table 12 · p. 25
  • 37.6 dB mean PSNR on HDR-NeRF shows high-quality full-scene observation rendering comparable visually to methods dedicated to fitting illuminated observations.This evaluation retains the severe, view-consistent illumination in its target.Source: §A.4 · pp. 23–24

Related papers

Loading 2610.01876v1…