Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction

1 Introduction

• Radiance fields sacrifice material editability, increase geometric ambiguity, and do not support relighting.Source: §1 · p. 2
• Physically based global illumination is costly in inverse-rendering optimization, so prior methods simplify visibility, indirect illumination, geometry, or lighting conditions.Data-driven relighting methods may instead require more than 500 OLAT images for training.Source: §1 · p. 2
• The method applies radiosity to Gaussian surfels, which serve as discrete finite elements for light transport without additional discretization or refinement.Continuous, non-binary visibility avoids complicated meshing techniques.Source: §1 · p. 2
• Radiosity-style solving supports view-independent rendering, avoiding recomputation of light transport across viewpoint changes and enabling hundreds of FPS during inference.The authors also describe parallel-friendly solvers and optimization-oriented approximations.Source: §1 · p. 2
• The framework combines direct and indirect illumination for Gaussian surfels and supports view-independent rendering and view-dependent effects.It extends radiosity to non-binary visibility and semi-opaque primitives.Source: §1 · p. 2
• The authors report ∼10× faster gradient calculation and significantly less memory use than automatic differentiation, alongside high-quality relighting and accurate geometry reconstruction.The reported applications include material editing, relighting, and geometry reconstruction.Source: §1 · pp. 2–3, Figure 2 · p. 2

Relighting and geometry reconstruction in radiance fields

• For discrete primitives, efficiently solving physically based rendering remains open; prior relighting approaches simplify indirect illumination or visibility, while data-driven methods require dense OLAT datasets.The proposed radiosity-based method retains indirect illumination and visibility and differentiates the light transport.Source: §2 · pp. 3–4
• Radiance-field geometry methods can increase ambiguity on textureless or heavily shadowed areas, whereas Gaussian surfels provide a geometry definition that can use light transport to explain those areas.This approach requires awareness of light sources.Source: §2 · p. 3

Differentiable global illumination

• Prior differentiable global-illumination work uses ray or path tracing, while this work studies differentiation with finite elements and continuous non-binary visibility.The authors state that their formulation avoids discontinuity issues and derives geometry and material gradients more efficiently than automatic differentiation.Source: §2 · p. 3

Radiosity

• Classical radiosity models element-to-element transport as a linear system and enables view-independent rendering, but historically faced visibility meshing and complexity challenges.For Gaussian surfels, the paper extends radiosity to transparency, adds solvers, derives efficient gradients, and uses approximations.Source: §2 · p. 4

3 Preliminaries

• The preliminaries establish notation for the geometry field and spherical harmonics used throughout the paper.Source: §3 · p. 4

3.1 Geometry Field

• A Gaussian surfel is a 2D Gaussian kernel defined on a local tangent plane, with its normal derived from its principal tangential directions.The surfels are used to parameterize a stochastic geometry field that is converted into a density field for rendering.Source: §3.1 · p. 4, Figure 3 · p. 5
• In the existing geometry-field formulation, each surfel has learnable outgoing radiance and ray contributions are ordered by intersection depth, with opacity weighting the incident radiance.The rendering formulation is exact except for the sorting approximation.Source: §3.1 · p. 5
• This paper computes outgoing radiance from geometry, materials, and lighting through differentiable light transport rather than treating it as directly learnable.Source: §3.1 · p. 5

3.2 Spherical Harmonics Representation

• Spherical harmonics provide a basis for representing spherical rendering functions, including BRDFs, radiance, and emission, through coefficients.The paper describes representing a spherical function as a weighted sum of basis functions.Source: §3.2 · p. 5
• The paper represents the BRDF in spherical harmonics, including a formulation for BRDFs that depend on the dot product of incoming and outgoing directions.It notes a closed-form spherical-harmonics expansion for the Phong BRDF and approximations to other BRDF models.Source: §3.2 · p. 5

4 Overview

• The method represents Gaussian surfel radiance with spherical-harmonic coefficients, accounts for inter-surfel light transport, and extends radiosity to transparency.The authors contrast this with path tracing, which they describe as potentially noisy and slow in this setting.Source: §4 · p. 6
• Given training images with known or unknown light sources, the method targets relighting at novel viewpoints and lighting conditions.For geometry reconstruction, meshes are extracted from rendered training-view depth maps using TSDF fusion.Source: §4 · p. 6
• The pipeline initializes surfels from SfM points when available or random points, computes outgoing radiance from geometry, materials, and lighting, then shades and rasterizes them.The rasterizer is differentiable and tile-based.Source: §4 · p. 6
• Optimization follows Jiang et al. [2025], adding an alpha-mapping mask loss and an optional environment-map intensity regularizer.The mask loss has weight 0.1; the optional regularizer has weight 0.05.Source: §4 · p. 6
• Adaptive density control grows the primitives during optimization.Source: §4 · p. 6

5 Forward Rendering

• The framework presents direct illumination as a special case and also derives a general global-illumination formulation.Source: §5 · p. 6

5.1 Geometry, Material, Emission, Outgoing Radiance, and Lighting Representations

• Gaussian surfels use semi-opaque, non-binary visibility; material, emission, and outgoing radiance are constant within each surfel, with diffuse-Phong BRDFs represented using spherical harmonics.Light sources are represented as emissive Gaussian surfels; directional lights and environment maps are supported approximately.Source: §5.1 · pp. 6–7

5.2 Direct Illumination

• Direct illumination averages responses over each kernel’s support using incident radiance from light sources, and its cached radiance enables view-independent rendering.The method retains continuous visibility and supports both near-field and far-field lights.Source: §5.2 · p. 7

5.3 Global Illumination

• Global illumination derives a recursive rendering equation under a non-overlap assumption, adapting classical radiosity to semi-opaque primitives and non-binary visibility.After solving outgoing radiance per surfel, the image can be rendered from arbitrary viewpoints.Source: §5.3 · pp. 7–8

• Explicit gradients exploit a duality between forward and backward passes to yield accurate, efficient, non-recursive analytical derivatives.The solvers cache quantities within surfels to avoid redundant backward-pass computation.Source: §6 · pp. 8–9
• Emission gradients mirror forward rendering with reversed decay, propagating gradients rather than emissions and extending the duality to other derivatives.Source: §6 · p. 8
• BRDF gradients are analytically computed from the previously calculated outgoing radiance and emission gradients, giving a closed-form alternative to automatic differentiation.Source: §6 · p. 8
• Geometry gradients have analytical non-recursive forms for decay and directional effects, and computation is restricted to visible kernel pairs.Source: §6 · pp. 8–9

Solver Design

• Solving the radiosity linear system exactly at every optimization step is computationally infeasible because geometry and materials keep changing.The solver discussion therefore turns to progressive refinement, Monte Carlo estimation, and a hybrid of the two.Source: §7 · p. 9

Hybrid Solver

• Progressive refinement alone cannot efficiently reach the converged solution for hundreds of thousands of kernels, but its remaining-radiance problem has the same recursive form as light transport.That shared form allows progressive refinement to be hybridized with other solvers, including the Monte Carlo solver.Source: §7 · p. 9

Monte-Carlo Solver

• The Monte Carlo solver selects the next kernel as a light-transport event rather than selecting a direction, enabling off-policy TD(0) updates of outgoing radiance for all kernels.It maintains an estimate for every kernel and iteratively updates those estimates.Source: §7 · p. 9
• The Monte Carlo update is an unbiased estimate, and the authors empirically set the maximum time step to T=64 to balance efficiency and variance.As iterations increase, transport depth grows, avoiding the need to set a maximum depth.Source: §7 · p. 10

Hybrid Solver

• The hybrid solver is more efficient and has lower variance at similar compute time than using the Monte Carlo solver alone for global illumination.It applies progressive refinement to estimate illumination from light sources, then uses Monte Carlo estimation for indirect illumination.Source: §7 · p. 10

Motivation

• Visibility variation across rays makes outgoing radiance sensitive to small geometric changes, potentially complicating joint geometry and material optimization.The authors attribute this sensitivity to assuming material, emission, and outgoing radiance are constant within each kernel's support.Source: §8 · p. 11

Center-to-Center Approximation

• The variance-free, efficient center-to-center approximation considers only the ray connecting the central points of the Gaussian surfels.It assumes the BRDF, outgoing-radiance, and decay terms are nearly constant inside the integral.Source: §8 · p. 11
• An optimizable positive scaling factor compensates for approximation error, while redefining the fused decay term absorbs the factor and integral terms to remove double integrals.This lets the gradient calculation and solvers use the simplified terms without changing the general framework.Source: §8 · p. 11

9.1 Qualitative Comparison

• Qualitatively, global-illumination rendering mostly leads relighting PSNR results, while reconstructed meshes show detail without the cracks or holes seen in comparisons.SSIM and LPIPS can be less sensitive to shadow errors.Source: Figure 7 · p. 12, Figure 8 · p. 13, §9.1 · p. 14

9.2 Quantitative Comparison

• On Stanford-ORB, the method achieves the best PSNR and Chamfer distance across baselines for novel-view synthesis, geometry reconstruction, and relighting.The comparison covers Tables 3 and 4.Source: §9.2 · p. 14, Table 3 · p. 14, Table 4 · p. 14

9.3 Ablation Study

• Across a 2,000-image test set, without approximation the baseline scored PSNR 26.44, SSIM 0.9255, and LPIPS 0.0681; with approximation, ours scored 30.01, 0.9542, and 0.0430, respectively.The authors attribute the performance boost to relaxing the dependency between geometry and material optimization.Source: §9.3 · p. 15
• The hybrid solver reduces rendered-image variance at slightly higher cost; Monte-Carlo-only training lowers PSNR by 0.25 dB and increases training time by 35.77%.The proposed backward pass takes 0.26 s versus 2.34 s for auto-differentiation and uses 28.25 GB less memory in the reported setup.Source: §9.3 · pp. 15–16, Figure 11 · p. 15
• Batch-size-4 training uses a little over one quarter of the original training time, improves view synthesis, and slightly compromises geometry reconstruction.It still outperforms the baselines reported in Table 3.Source: Table 5 · p. 15, §9.3 · p. 16
• With explicit light transport, the method achieves the lowest Chamfer distance and avoids concave or protruding geometry errors reported for PGSR and GFSGS variants.The comparison includes GFSGS using latent color representation.Source: §9.3 · p. 16, Table 6 · p. 16, Figure 12 · p. 16

9.4 Additional Experiments

• On PlateOfFruit, the method achieves competitive relighting with five training lighting conditions versus the full 50-condition setup; rotating environment maps also yields adapting shadows.The light-source location can additionally be optimized from L1 image-space supervision.Source: §9.4 · pp. 16–17, Table 7 · p. 16
• Hybrid-solver transport time grows roughly linearly with primitive count, while global illumination produces reasonable results in the enclosed-box test where direct illumination fails.The scaling also depends on primitive distribution, and the box experiment uses rich inter-reflection.Source: §9.4 · pp. 17–18, Figure 14 · p. 17, Figure 17 · p. 18

10 Conclusions, Limitations and Future Work

• The work derives differentiable global illumination on Gaussian surfels and supports view-independent rendering with diffuse and specular effects.The framework extends radiosity to non-binary visibility and semi-opaque primitives and derives an efficient gradient calculation.Source: §10 · p. 18
• For objects that are too shiny, finite-degree spherical harmonics can leave visible artifacts.The paper illustrates this limitation with an overly shiny object.Source: Figure 18 · p. 18, A BRDF details · p. 21
• Under sparse views, the Gaussian-surfel layout may not optimally capture all details; scene-level extensions also raise efficiency and variance challenges.The authors point to jaggy shadows and imperfect geometry reconstruction as evidence of these limitations.Source: §10 · p. 18

A BRDF details

• The BRDF specular component is clamped to zero on the invisible hemisphere, and the implementation uses up to 9-degree spherical harmonics for reasonably glossy materials.The BRDF coefficient expressions distinguish odd and even degrees.Source: A BRDF details · p. 21

B Forward rendering details

• The derivation starts from the rendering equation without assuming constant outgoing radiance, then introduces a spatially invariant, directionally varying approximation within each kernel.Material, emission, and outgoing radiance are treated as constant within the kernel support.Source: B Forward rendering details · pp. 21–22
• Swapping the integral and summation reorganizes incident radiance as contributions summed over kernels rather than directions.The paper illustrates the two interpretations in Fig. 19.Source: B Forward rendering details · p. 22, Figure 19 · p. 22
• A weighted residual aggregates the approximation error across tangent-plane points; assuming non-overlapping kernels simplifies the resulting expression but may cause energy-conservation issues.The weighting function is nonzero only at the selected kernel point, and may use a Galerkin form or alternatives such as Gaussian or Dirac delta functions.Source: B Forward rendering details · p. 22
• The backward pass reverses propagation by sending weighted gradients from the receiving kernel toward the emitting kernel, using reciprocity to reverse direction.The gradient propagation is equivalent to a forward pass with weighted gradients and a flipped asymmetric decay term.Source: B Forward rendering details · p. 23
• The resulting transport is converted into spherical-harmonic coefficient space, where visibility decay and BRDF terms propagate radiance between kernels.The derivation also accounts for semi-transparent primitives through the opacity factor.Source: B Forward rendering details · pp. 22–23

Gradients with respect to emissions

• Emission gradients accumulate contributions from every path between kernels, yielding a non-recursive closed-form solution.The backward pass reverses forward propagation, and its relationship to the forward pass expresses their duality.Source: C Backpropagation details · p. 23

Gradients with respect to the BRDF

• BRDF gradients account for paths through each bridge by combining propagated weighted gradients with emissions.The required propagated emissions and weighted gradients are those computed in the forward and backward passes.Source: C Backpropagation details · p. 24

Gradients with respect to geometry

• Geometry gradients for a kernel’s decay term enumerate every pair of kernels forming a decay term that contains it.The gradient for a decay term is the product of propagated emissions from the start kernel and propagated weighted gradients from the end kernel; the boundary term comes from differentiating through the integration domain or boundary.Source: C Backpropagation details · p. 24
• Directional gradients enumerate pairs involving the target kernel and comprise two symmetric terms for outgoing radiance and BRDF response.The paper also identifies central points and scaling among the geometric properties whose partial derivatives are needed.Source: C Backpropagation details · p. 24

Variance-adaptive sampling

• Kernels with higher tracked variance receive more sampling steps through variance-proportional selection with replacement.At each step, N elements are selected from {1, 2, ..., N}.Source: D Monte-Carlo Solver Details · p. 24

Grouped next-event estimation

• K-Means groups kernels for a two-stage next-event estimate, sampling a group and then a kernel within that group.Group proxy values use averaged central points and normal directions, plus summed current outgoing-radiance estimates; the resulting tree has depth 2.Source: D Monte-Carlo Solver Details · p. 24
• The double-summation approach performs well in practice, but non-binary visibility still requires transmitted radiance to be calculated for every kernel separately.The passage contrasts this with hierarchical radiosity methods that group kernels with approximately identical transmitted radiance.Source: D Monte-Carlo Solver Details · p. 24

Integral transformation

• The integral to evaluate is the opacity α_j over the plane P_j of kernel j.The local coordinates u and v describe x in that kernel’s tangent plane.Source: E Approximation Details · p. 25
• The method transforms the integral of α_j over P_j into an integral over the local tangent plane.The plane coordinates are defined as u and v.Source: E Approximation Details · p. 25

Closed-form evaluation

• The resulting expression has a closed-form solution involving the exponential integral.The passage relates the integral of (1−exp(−y))/y to that function.Source: E Approximation Details · p. 25