PaperlayerView exact revision on arXiv

Source-linked AI summary

COREA: Coupled Relightable 3D Gaussians and SDFs for Efficient Normal Alignment

Jaeyoon Lee, Hojoon Jung, Sungtae Hwang, Jihyong Oh, Jongwon Choi

arXiv:2512.07107v3cs.CV

TL;DR

Inverse PBR with 3DGS is bottlenecked by oversmoothed, unstable normal estimates, while a unified framework supporting novel-view synthesis, surface reconstruction, and inverse PBR is lacking. COREA couples relightable 3D Gaussians with an SDF for shared geometric supervision, and reports superior SH-based NVS and inverse-PBR performance alongside high-quality reconstructed surfaces.

  • Problem

    Inverse PBR with 3DGS is bottlenecked by normal estimation because discrete splats can produce oversmoothed, unstable normals.

    Source: §2.1 · p. 4

  • Method

    COREA couples an SDF and relightable 3DGS on a shared surface, using Gaussian depth signals to refine SDF geometry and continuous SDF normals to supervise Gaussian normals.

    Source: §1 · pp. 2–3

  • Results

    COREA is the only framework reported to support all three tasks, with superior SH-based NVS and inverse-PBR performance and high-quality reconstructed surfaces.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The coupled representations provide a consistent geometric foundation for inverse PBR, including BRDF-lighting decomposition and relighting under novel illumination.

    Source: §5 · p. 14

    The depth-gradient normal supervision assumes local planarity.

    Source: Depth-guided SDF Alignment (DSA) · p. 7

Abstract

from arXiv · show

We present COREA, the first unified three-tasks framework that couples an SDF and relightable 3D Gaussians (3DGS) to jointly support SH-based novel-view synthesis (NVS), surface reconstruction, and inverse physically-based rendering (inverse PBR). While recent relightable 3DGS methods have progressed, inverse PBR remains bottlenecked by normal estimation, as the discrete nature of 3DGS often yields oversmoothed and unstable normals. To address this limitation, COREA couples the complementary geometric properties of an SDF and relightable 3DGS on a shared underlying surface, where geometry-constrained relightable 3DGS provides reliable depth signals to anchor SDF geometry and the continuous SDF normal field provides spatially consistent supervision for Gaussian normal learning. We couple these signals through depth-guided alignment and normal supervision with normal-aware densification, and introduce Dual-Density Control to regulate densification by balancing photometric and geometric gradients for stable, memory-efficient training. Experiments on standard benchmarks show that COREA is the only framework that supports all three tasks, achieving competitive performance overall, with particularly superior results in inverse PBR.

1 Introduction

COREA addresses unstable normal estimation and weak geometric interaction between representations by coupling an SDF with relightable 3D Gaussians. It supports three rendering and reconstruction tasks, with reported strong performance in novel-view synthesis and inverse PBR.

1 Introduction

  • Inverse PBR depends on accurate normals, but discrete Gaussian splats can oversmooth them, while many hybrid methods lack complementary geometric interaction and still use blurred alpha-blended normal supervision.Pixel-wise depth gradients preserve sharper local structure for aligning SDF normals.Source: §1 · p. 2
  • COREA couples an SDF and relightable 3DGS on a shared surface, using Gaussian depth signals to refine SDF geometry and the continuous SDF normal field to stabilize Gaussian normals.It aligns SDF normals to pixel-wise depth gradients, projects Gaussians onto the SDF surface, and uses normal inconsistency for normal-aware densification.Source: §1 · pp. 2–3
  • With geometrically aligned Gaussians and stabilized normals, COREA optimizes BRDF parameters and lighting for faithful relighting under novel illumination.The paper describes this as robust BRDF-lighting disentanglement.Source: §1 · p. 3
  • COREA is the only framework reported to support SH-based NVS, surface reconstruction, and inverse PBR together, with superior performance in SH-based NVS and inverse PBR and high-quality surfaces.The task-support comparison is presented in Table 1; the introduction reports the performance and surface-quality outcomes in Fig. 1.Source: §1 · p. 3, Table 1 · p. 3
  • Dual-Density Control balances photometric and geometric gradients to regulate normal-aware densification, preventing excessive splitting while preserving geometric fidelity and stable rendering.The method is intended to keep densification stable and efficient.Source: §1 · p. 3

2 Related Work

Prior inverse-rendering work ranges from computationally demanding NeRF methods to explicit Gaussian approaches, while hybrid Gaussian–implicit methods often provide limited interaction between branches. COREA is presented as a mutually supervised coupling of relightable 3DGS and an SDF.

2.1 Inverse Physically-based Rendering

  • NeRF-based inverse-rendering approaches jointly estimate geometry, reflectance, and lighting but typically require expensive volumetric rendering and remain computationally demanding.Other reflectance-aware work models specular reflections and indirect illumination.Source: §2.1 · pp. 3–4
  • Gaussian inverse-rendering methods incorporate factorized geometry, material, and lighting, normal regularization, ray tracing, deferred rendering, or learned Gaussian-light interactions.The section also describes geometry-aware optimization and structured parameter disentanglement.Source: §2.1 · p. 4
  • Alpha-blended Gaussian normals can be oversmoothed, making BRDF-lighting decomposition unreliable; COREA couples an SDF and relightable 3DGS to provide complementary geometric supervision and stable normals.The stated aim is faithful BRDF and lighting decomposition under complex illumination.Source: §2.1 · p. 4

2.2 Hybrid 3DGS Methods for Surface Reconstruction

  • Hybrid methods combine Gaussian rendering with implicit geometry, including SDF alignment and geometry extraction, but often assign separate roles or use auxiliary constraints and pruning.The cited examples include Gaussian-center alignment to an implicit surface and pulling centers toward the SDF zero-level set.Source: §2.2 · p. 4
  • COREA instead jointly optimizes geometry-constrained relightable 3DGS and an SDF through mutual geometric supervision, using depth signals for SDF alignment and SDF normals to stabilize Gaussian normals.Its coupling also supports normal-aware densification for fine-scale geometry.Source: §2.2 · p. 5, Figure 3 · p. 5

3 Proposed Method: COREA

COREA alternates depth-guided SDF alignment and SDF-guided Gaussian normal refinement, then uses Dual-Density Control to regulate densification. After geometry converges, it optimizes BRDF parameters and lighting for inverse PBR.

Algorithm 1 Training pipeline of COREA

  • COREA alternates depth-guided SDF alignment and SDF-guided Gaussian alignment, then optimizes BRDF parameters and lighting on the learned geometry.The geometric-coupling stage alternates DSA and NGA; inverse PBR follows after geometry converges.Source: Algorithm 1 Training pipeline of COREA · p. 6

3.1 Geometric Coupling Between SDF and Relightable 3DGS

3.2 Dual-Density Control (DDC)

  • DDC balances photometric and geometric gradients and splits a candidate Gaussian only when the combined splitting matrix has a negative minimum eigenvalue.This curvature-guided rule is intended to constrain excessive Gaussian growth while maintaining geometric fidelity and stable rendering.Source: Figure 4 · p. 8, §3.2 · p. 9

3.3 Inverse Physically-Based Rendering (Inverse PBR)

  • With learned geometry and normals fixed, inverse PBR optimizes per-Gaussian BRDF parameters and environment lighting, then relights under a novel lighting condition.The inverse-rendering objective supervises rendered images against ground-truth images using an L1 loss.Source: §3.3 · p. 9

4 Experiments

COREA is evaluated on DTU and Tanks&Temples under a shared single-GPU setting. It uniquely supports all three tasks in one framework, with especially strong inverse PBR, while ablations favor pixel-wise depth-gradient supervision and alternating alignment updates.

4.2 Evaluation Results

  • COREA is the only framework supporting SH-based NVS, surface reconstruction, and inverse PBR together, with strong NVS and inverse PBR and competitive surface reconstruction.The comparison covers DTU and Tanks&Temples under the same single-GPU setting; Table 2 marks unsupported tasks and out-of-memory cases.Source: Table 2 · p. 10, §4.2 · p. 10

4.1 Experimental Setup

  • Evaluation covers DTU and four object-centric Tanks&Temples scenes, using PSNR, SSIM, and LPIPS for rendering and Chamfer Distance for mesh geometry.All methods are evaluated on a single NVIDIA A6000 GPU with 48 GB under the same hardware setting.Source: §4.1 · pp. 9–10

4.2 Evaluation Results

4.3 Ablation Study

5 Conclusion

COREA couples an SDF and relightable 3D Gaussians on a shared surface, combining their geometric signals to support three tasks. The paper reports strong NVS and inverse-PBR results, competitive surface reconstruction, and unique coverage of all three tasks.

5 Conclusion

  • COREA anchors SDF geometry with 3DGS depth signals and stabilizes Gaussian normals with the continuous SDF normal field.The shared geometric foundation supports accurate BRDF and lighting decomposition and faithful relighting under novel illumination.Source: §5 · p. 14
  • Dual-Density Control balances photometric and geometric gradients during normal-aware densification, limiting excessive Gaussian growth while maintaining stable rendering.Source: §5 · p. 14
  • COREA achieves strong SH-based NVS and inverse-PBR results, competitive surface reconstruction, and remains the only framework supporting all three tasks.Source: §5 · p. 14

Supplementary Material

The demo video presents COREA’s framework and qualitative comparisons across SH-based NVS, surface reconstruction, and inverse PBR.

Supplementary Material

  • The demo includes comparisons across SH-based NVS, surface reconstruction, and inverse PBR.Source: A Demo Video · p. 19
  • The video is organized into an overall teaser, a framework visualization, and additional qualitative-performance materials.Source: A Demo Video · p. 19
  • Source clips use Supersplat for SH-based NVS and inverse PBR, Blender for surface reconstruction, and After Effects for final editing.Source: A Demo Video · p. 19

B Implementation details

The implementation assigns BRDF and illumination parameters to each Gaussian, renders physically based color from sampled incident directions, and changes the environment map for novel-lighting relighting.

B Implementation details

  • Each Gaussian carries albedo, roughness, learned indirect illumination, and a shared global environment map for inverse rendering.Source: §B.1 · p. 19
  • Incident radiance combines visible environment lighting with learned indirect illumination, and sampled BRDF responses produce outgoing radiance.The rendering uses Gaussian normals and solid-angle weights over sampled incident directions.Source: §B.1 · p. 19
  • For novel illumination, COREA replaces the global environment map and omits learned indirect illumination when reevaluating physically based color.Source: §B.1 · p. 19
  • Visibility is estimated with R3DG’s BVH-based ray tracing scheme, which models Gaussian transmittance along sampled incident rays.Source: §B.1 · p. 19

B.2 Training Objectives

Training first couples SDF and relightable 3DGS geometry through depth and normal alignment, then optimizes appearance through inverse PBR.

B.2 Training Objectives

  • Training uses geometric coupling first and inverse PBR second.Source: §B.2 · p. 20
  • During geometric coupling, alternating updates optimize the SDF and 3DGS through depth and normal alignment.The depth-consistency loss uses an L1 difference between their rendered depth maps to encourage a shared geometric structure.Source: §B.2 · p. 20
  • The SDF objective combines reconstruction, depth, Eikonal, curvature, and normal terms.The reconstruction term supports coarse alignment, while the Eikonal and curvature terms enforce their stated constraints.Source: §B.2 · p. 20
  • The 3DGS loss includes image, depth, mask, depth-distribution, and normal terms; mask and uncertainty constraints suppress rendered-depth noise.The stabilized pixel-wise depth gradients provide sharper, more reliable DSA supervision than alpha-blended normals.Source: §B.2 · p. 20

Algorithm 2 Step 1: Depth-guided SDF Alignment (DSA)

Algorithm 3 Step 2: Normal-guided Gaussian Alignment (NGA)

  • NGA freezes the SDF, renders both representations, and updates Gaussians using selected depth and normal losses; SDF normal gradients also supply DDC’s geometric gradients.The 3DGS normal loss is a cosine loss aligning Gaussian normals with SDF normals.Source: §B.2 · p. 20, Algorithm 3 Step 2: Normal-guided… · p. 21
  • After geometric coupling, inverse PBR optimizes BRDF parameters and lighting using 64 incident directions per Gaussian and rendered-appearance supervision.Lighting and BRDF regularizers support stable illumination decomposition and regularize BRDF parameters.Source: Algorithm 3 Step 2: Normal-guided… · p. 21

B.3 Training Schedule

COREA uses a staged training schedule for geometric coupling and inverse PBR, with specified loss weights and reported DTU runtime and memory use.

B.3 Training Schedule

  • COREA warms up 3DGS for 15k iterations, couples geometry for 30k, then performs inverse PBR for 10k.Geometric coupling alternates SDF and 3DGS updates, using depth consistency for the first 10k iterations and pixel-wise normal consistency for the remaining 20k.Source: §B.3 · pp. 21–22

B.4 Hyperparameter Settings

  • The shared depth and normal consistency weights are 0.01 and 0.001, and the DDC weight is α=0.2.For inverse PBR, geometry and Gaussian normals are fixed while BRDF parameters and lighting are optimized.Source: §B.4 · p. 22

B.5 Complexity and Runtime

  • On DTU, COREA takes approximately 2.4 hours per scene and uses roughly 24 GB of peak GPU memory on average.Experiments used a single NVIDIA A6000 GPU.Source: §B.5 · p. 22

C.1 Trade-off Analysis of α in DDC

The DDC weight balances photometric and geometric gradients: α=0.2 gives the best reported inverse-PBR balance, while DDC reduces Gaussian growth without sacrificing rendering fidelity.

C.1 Trade-off Analysis of α in DDC

  • Inverse-PBR PSNR peaks at α=0.2, while SH-based NVS PSNR remains stable across tested weights.For α≥0.5, splitting shifts toward normal-driven directions; larger weights slightly reduce rendering quality.Source: Figure S1 · p. 23, §C.1 · p. 23

C.2 Dual-Density Control for Efficient Gaussian Densification

  • DDC reduces the Gaussian count by 17% on average while preserving rendering fidelity across DTU scenes.The comparison is against Adaptive Density Control (ADC).Source: §C.2 · p. 23, Table S1 · p. 24
  • Qualitative scenes show sharp novel views, accurate geometry, and stable relighting under white and HDR environment lighting, with clean object-background separation.The figure presents the three tasks in a unified framework.Source: Figure S2 · p. 24
  • DDC densifies when combined photometric and geometric gradients indicate meaningful refinement, suppressing redundant growth and reducing memory use and inverse-PBR computation.Fewer Gaussians participate in ray sampling and shading evaluations.Source: §C.2 · p. 24

D.1 Per-scene Results

COREA supports all three evaluated tasks in one framework and maintains competitive reconstruction accuracy alongside strong rendering results; fine-scale surface detail is not fully captured by CD.

D.1 Per-scene Results

  • Per-scene quantitative results cover SH-based NVS, surface reconstruction, and inverse PBR, with additional qualitative results across diverse scenes.The quantitative results are reported in Tables S3–S7.Source: §D.1 · pp. 24–25

D.2 Surface Reconstruction Comparison

  • COREA is the only compared framework supporting SH-based NVS, surface reconstruction, and inverse PBR, while maintaining competitive DTU CD and strong NVS quality.Table S2 compares representative mesh-oriented 3DGS methods under a unified protocol.Source: §D.2 · p. 25, Table S2 · p. 25
  • Sparse observations or thin structures can make Gaussian coverage and SDF sampling unreliable, leading to oversmoothed local geometry and unstable relighting.Source: Figure S4 · p. 26
  • Although COREA has higher CD than PGSR and MILo, it preserves sharper local surface structures that influence normal fidelity.The comparison indicates that lower CD does not necessarily imply greater fine-scale geometry fidelity.Source: Figure S3 · p. 25, §D.2 · p. 26

E Limitations

COREA’s cross-representation geometric consistency can become fragile in geometrically ambiguous regions; the authors identify pixel-weighted Gaussians as a possible direction for improving difficult cases.

E Limitations

  • Sparse-view regions and depth discontinuities can weaken alignment, allowing early errors to cascade into blurred geometry and unstable relighting.The described chain includes Gaussian collapse or sparsity and biased SDF ray sampling around inaccurate depth anchors.Source: E Limitations · p. 26
  • Pixel-weighted Gaussian representations may improve densification and Gaussian formation under sparse initial point clouds and low visibility.The paper presents this as a promising direction for stabilizing challenging cases.Source: E Limitations · p. 26
Loading 2512.07107v3…