PaperlayerView exact revision on arXiv

Source-linked AI summary

VolS-GS: Relightable Gaussian Splatting with Volumetric Subsurface Scattering

Junyeong Ahn, Jaegul Choo

arXiv:2610.04007v1cs.CV

TL;DR

Gaussian relighting methods leave subsurface light that enters one location and emerges elsewhere without an interior transport path. VolS-GS solves transport over Gaussian support and achieves the best average performance across three datasets while relighting under novel lights and views.

  • Problem

    Existing Gaussian methods model subsurface response per primitive, without an interior path for light entering at one location and emerging elsewhere.

    Source: §1 · p. 2

  • Method

    VolS-GS solves differentiable finite-volume transport over Gaussian occupancy, adds a separate single-scattering term, and uses shared transport cues and regularization for shadow and specular modeling.

    Source: §1 · pp. 2–3

  • Results

    VolS-GS achieves the best average performance across all three datasets and relights with high visual quality under novel lights and views.

    Source: §3.2 · p. 7, §4 · p. 9

  • Takeaways & Limitations

    The shared transport field and specular prior keep components separable and the learned decomposition interpretable.

    Source: §4 · p. 9

    The diffusion-based solve may not suffice for directional radiance and does not model light exchanged between surfaces on concave regions.

    Source: §4 · p. 9

Abstract

from arXiv · show

We present VolS-GS, a relightable Gaussian splatting framework that reconstructs objects from one-light-at-a-time (OLAT) captures and renders them under novel lighting and viewpoints. Relightable Gaussian Splatting methods typically model appearance independently at each primitive, which makes non-local effects difficult to represent. This limitation is particularly apparent for subsurface scattering, where light entering the object at one location can emerge at another. Rather than modeling this effect solely with a neural network or a local kernel at each primitive, we use the spatial support of the Gaussian scene as the domain of a differentiable finite-volume transport solver, so that light can propagate through the object's interior. A small network predicts scattering and absorption coefficients for each Gaussian, and the solve redistributes incident light through the resulting field. The coefficients are fit to images rather than measured, so the solve supplies a transport-shaped path for aggregating per-primitive appearance, not a measurement of the material. To keep the learned shadow and specular terms from taking over the other components, our shadow term is predicted from visibility together with the transmittances and the scattering the solve produces, and a regularizer suppresses specular highlights in regions the shadow term predicts to be unlit. Experiments on three OLAT benchmarks show that VolS-GS consistently improves relighting quality on held-out lights and views.

1 INTRODUCTION

Relightable Gaussian methods model appearance per primitive, leaving subsurface light transport through an object's interior unresolved. VolS-GS uses the Gaussian scene as the domain for a differentiable volumetric transport solve, alongside separate shadow and specular controls.

1 INTRODUCTION

  • Per-primitive appearance models do not capture subsurface light entering at one surface location and emerging elsewhere through the object's interior.In the C-shaped example, a dipole kernel's Euclidean-distance support crosses the gap, unlike aggregation through the learned medium.Source: §1 · p. 2, Figure 2 · p. 2
  • Prior Gaussian methods route non-local light outside the object or evaluate subsurface response from a primitive's own attributes, leaving no interior path.The paper contrasts its approach with transport through the medium, where light enters at one surface point and exits at another.Source: §1 · p. 2
  • VolS-GS turns Gaussian support into a coarse-grid occupancy domain and solves finite-volume transport there, without requiring an extracted mesh.Each Gaussian reads the solved field at its position; an adjoint formulation makes the solver differentiable without unrolling iterations.Source: §1 · p. 2
  • A separate single-scattering term adds once-scattered light integrated along the view chord, which the angle-averaged fluence cannot represent.The multiply scattered field comes from the transport solve, and the medium is learned end to end from images.Source: §1 · p. 2
  • The shadow model shares transport transmittance cues, while a regularizer discourages the specular branch from absorbing transported light.Together, these controls keep learned shadow and specular components from trading against each other.Source: §1 · pp. 2–3

2 METHOD

VolS-GS uses the Gaussian scene as the support for volumetric light transport while retaining its rasterizer. Its method combines direct reflection, subsurface scattering, and shadow/specular disambiguation, trained jointly from OLAT images.

2.1 PRELIMINARIES

  • Diffusion approximates multiple scattering when scattering dominates absorption, while the direction-dependent first event is handled separately.The approximation is weakest at the boundary and for that first still-directional event.Source: §2.1 · p. 3

2.2 RELIGHTING FORMULATION

  • Each Gaussian’s outgoing radiance combines shadowed direct diffuse and specular reflection with a subsurface-scattering channel.The learned shadow decay attenuates the two direct terms, not subsurface light already attenuated inside the solve.Source: §2.2 · p. 4

2.3 GAUSSIAN-SUPPORTED TRANSPORT DOMAIN

  • A fractional-occupancy grid over the Gaussian cloud forms the transport domain, filling its hollow surface shell to provide an interior.The grid adds no learnable parameters; Gaussian coefficients are splatted onto it and extended inward from surface cells.Source: §2.3 · p. 4

2.4 VOLUMETRIC SUBSURFACE SCATTERING

  • The finite-volume system transports fluence between neighboring cells, absorbs it within cells, and models escape at the boundary.Its Robin boundary condition represents light leaving the surface and internal reflection; gradients are computed with one additional adjoint solve.Source: §2.4 · p. 5
  • The solve uses image-fitted, light- and view-conditioned coefficients, which are not optical constants of the material.A separate single-scattering term complements the angle-averaged diffusion solve, and the subsurface channel falls back to local shading when the medium is thin.Source: §2.4 · p. 5

2.5 SHADOW AND SPECULAR DISAMBIGUATION

  • A network predicts shadow decay from geometric visibility and transport cues, while a shadow-aware prior penalizes specular radiance where decay predicts unlit regions.The two learned branches can otherwise trade off, allowing specular to explain transported illumination when shadow suppresses direct light.Source: §2.5 · p. 6

2.6 TRAINING

  • All components train jointly, with the subsurface channel enabled after a 9k-iteration warm-up and the transport domain rebuilt during densification and every 300 iterations.Training uses photometric, opacity, leakage, and medium-regularization losses; each post-warm-up step runs a forward and adjoint solve.Source: §2.6 · p. 6

3 EXPERIMENTS

VolS-GS achieves the best average performance across three OLAT datasets, with gains most consistent on subsurface-dominant scenes. Its ablations show distinct roles for volumetric scattering, transport cues, and the shadow term.

3.1 SETUP

  • Evaluation covers subsurface-dominant, opaque, and anisotropic regimes; qualitative baselines are retrained with matched resolutions, splits, and training budgets.SSS-GS, NRHints, and GS3 contribute synthetic and real scenes; the baseline set includes GS3, SSD-GS, RNG, and SSS-GS.Source: §3.1 · p. 7

3.2 MAIN RESULTS

  • VolS-GS achieves the best average performance across all three datasets, with the most consistent improvement on SSS-GS and gains on NRHints and GS3.Table 1 reports the main results across the three benchmarks.Source: §3.2 · p. 7, Table 1 · p. 7
  • Across novel-light scenes, GS3 and SSS-GS lose accuracy around shadows and fine details, RNG loses high-frequency detail, and SSD-GS struggles with intricate shadows and complex subsurface effects.Figure 4 compares synthetic GS3 scenes and real NRHints captures, with per-frame metrics and shared-scale error maps.Source: §3.2 · pp. 7–8, Figure 4 · p. 8
  • The learned decomposition places broad backlit glow in the SSS channel, while specular and shadow components remain more localized.Figure 5 shows the channels for PIXIU and CRYSTAL.Source: §3.2 · p. 8, Figure 5 · p. 8

3.3 ABLATIONS

  • Removing volumetric SSS changes appearances where transported light contributes; removing transport cues changes the shadow pattern, while removing the shadow term flattens shading.Applying the shadow to SSS changes the shadow channel and slightly changes the render, with a held-out-light cost reported in Table 2.Source: §3.3 · pp. 8–9, Table 2 · p. 9

4 CONCLUSION

VolS-GS adds learned volumetric subsurface scattering to Gaussian relighting and reports faithful relighting under novel lights and views. The conclusion notes limits of diffusion for directional radiance and of the method for concave-region surface-to-surface exchange.

4 CONCLUSION

  • VolS-GS augments local shading with learned volumetric subsurface scattering, and its reconstructions relight under novel lights and views across three OLAT benchmarks.The method uses transport cues for shadow and a prior to keep the specular lobe out of unlit pixels, supporting a separable, interpretable decomposition.Source: §4 · p. 9
  • The diffusion solve may be insufficient while radiance remains directional and does not model light exchanged between surfaces across concave regions.Directional subsurface scattering in a discrete-volume regime and a unified representation for diverse non-local effects are left for future work.Source: §4 · p. 9

A.1 RELIGHTABLE NEURAL AND GAUSSIAN REPRESENTATIONS

  • Relightable neural and Gaussian methods span explicit reflectance models and illumination-conditioned appearance; GS3 leaves transported light in a residual rather than an explicit subsurface component.The cited Gaussian approaches include OLAT Gaussians, GS3, RNG, and SSD-GS.Source: §A.1 · p. 13

A.2 NON-LOCAL LIGHT TRANSPORT IN GAUSSIAN SCENES

  • Gaussian transport methods model light between surfaces, whereas subsurface scattering carries light through the object's interior.Examples of surface transport include PRTGS, Ref-Gaussian, and Radiosity-GS (Jiang et al., 2025).Source: §A.2 · p. 13

A.3 SUBSURFACE SCATTERING

  • Analytic BSSRDF kernels assume homogeneous semi-infinite media; numerical solvers address spatial variation, while VolS-GS discretizes diffusion on Gaussian occupancy and learns coefficients whose transport is determined by the solve.Related approaches include image-based medium recovery and learned subsurface appearance.Source: §A.3 · p. 13

B THEORETICAL PROPERTIES

The appendix establishes stability and differentiability properties of VolS-GS’s transport solve, and clarifies how it behaves under changing scattering strength and illumination. It also specifies the conventions used for the material coefficients and boundary exit.

B THEORETICAL PROPERTIES

  • The appendix assumes an isotropic phase and fixes g = 0 because the network predicts the reduced scattering coefficient, leaving g unidentifiable separately.It defines α = σs/σt and J = −κ∇ϕ as the single-scattering albedo and P1 flux vector.Source: B THEORETICAL PROPERTIES · p. 14
  • F is symmetric positive definite, so the discrete solution exists uniquely and conjugate gradients converge under the stated coefficient conditions.The solution norm is bounded by ∥b∥/λmin(M).Source: B THEORETICAL PROPERTIES · p. 14
  • Backpropagation uses one additional solve with the same operator to obtain gradients with respect to the source and operator parameters.The adjoint system equals the forward system because F is symmetric.Source: B THEORETICAL PROPERTIES · p. 14
  • The heterogeneous diffusion operator is self-adjoint, giving a reciprocal Green’s function, and the discretization preserves operator symmetry exactly.The appendix states G(x, x′) = G(x′, x) for arbitrary κ > 0 and σa ≥ 0.Source: B THEORETICAL PROPERTIES · p. 14
  • Transport superposes across lights for a fixed medium, although direction-conditioned material coefficients mean the medium itself does not superpose exactly.The single-scattered and diffused radiance split is not strictly disjoint, but both terms are assembled per light.Source: B THEORETICAL PROPERTIES · p. 14
  • As σs approaches zero, the source and multiply scattered radiance vanish while the operator remains symmetric positive definite, leaving opaque-material surface terms unperturbed.The scattering channel switches off continuously rather than causing the solve to degenerate.Source: B THEORETICAL PROPERTIES · p. 14
  • The exit factor follows from the Robin boundary condition and angular expansion; at η = 1 it is 1/(2π), with no additional σs factor.Scattering already enters through the source, so another σs factor would make exitant radiance quadratic in that coefficient.Source: B THEORETICAL PROPERTIES · p. 14

C IMPLEMENTATION DETAILS

The implementation solves transport on a fixed-resolution occupancy grid derived from the Gaussian cloud, with practical choices for convergence, inference speed, and training. The model uses one configuration across scenes, while the reported runtime depends on primitive count rather than pixel count.

C IMPLEMENTATION DETAILS

  • At inference, eight preconditioned conjugate-gradient iterations keep test PSNR within −0.009 to +0.000 dB of the converged solve on four measured scenes.The 643 grid has 5.1 × 104 to 1.4 × 105 active cells across 21 scenes, with a median of 7.4 × 104.Source: C IMPLEMENTATION DETAILS · p. 15
  • The 643 transport grid needs a smoothed interface-area estimate for grid convergence; training truncates both solves, making gradients approximate.Without smoothing, fluence correlations between 643 and 963 are 0.08 versus 0.97 with smoothing; source correlation is 0.96 at both resolutions.Source: C IMPLEMENTATION DETAILS · pp. 15–16
  • The material head predicts per-channel scattering and absorption coefficients, while the specular lobe uses eight anisotropic spherical Gaussians and fixes η = 1.0 for every scene.The exit index choice makes Fresnel factors identity; the decay MLP has three hidden layers of 32 units.Source: C IMPLEMENTATION DETAILS · p. 16
  • Training adds a scale-consistency penalty to the inherited base-system settings, uses one configuration across scenes, and evaluates models at each dataset’s protocol iteration.The penalty weight is 0.05; protocol iterations are 60k or 100k as specified by each table.Source: C IMPLEMENTATION DETAILS · p. 16

D.1 PER-SCENE RESULTS

The appendix provides per-scene comparisons across three dataset families, then checks relighting as the light moves and demonstrates an environment-map rendering procedure. The environment-map results are not benchmark evaluations or comparisons against methods designed for that setting.

D.1 PER-SCENE RESULTS

  • Tables 4 and 5 provide per-scene results for NRHints and GS3, while Figure 7 adds five test frames spanning all three dataset families.The figure prints PSNR, SSIM, and LPIPS per panel and shows clipped render-to-ground-truth error insets.Source: §D.1 · p. 16, Table 4 · p. 17, Figure 7 · p. 17

D.2 RELIGHTING UNDER A LIGHT SWEEP

  • Across a full azimuth sweep at the training elevation, the cast shadow and subsurface glow remain spatially consistent as the light moves.The sweep tests coherence within the training elevation distribution, not generalization to unseen elevations.Source: §D.2 · p. 19

D.3 ENVIRONMENT MAP RELIGHTING

  • Environment-map rendering samples 256 directional sources and sums their renders without retraining, but the paper does not evaluate environment-map benchmarks or compare against methods built for them.Figure 9 shows four maps rotated between paired frames; the evaluated setting otherwise uses one known-position light at a time.Source: §D.3 · p. 19

E ABLATIONS ON THE REMAINING SCENES

Ablations show that the most consequential component varies by scene: shadow removal usually has the largest effect, while subsurface transport dominates in selected scenes. The reported comparison extends PSNR ablations across seven scenes.

E ABLATIONS ON THE REMAINING SCENES

  • Removing the shadow term is the largest single change on six of seven scenes, while removing subsurface scattering costs BUNNY 5.48 dB, the largest effect on any scene.Table 6 extends the PSNR ablations to five further scenes beyond BUNNY and STATUE.Source: E ABLATIONS ON THE REMAINING SCENES · p. 19, Table 6 · p. 19
  • The second-most important component depends on the scene: subsurface transport on four translucent cases, but transport cues on STATUE and DRUMS.On weakly translucent DRUMS, removing subsurface scattering costs 0.10 dB, the smallest such cost across the scenes.Source: E ABLATIONS ON THE REMAINING SCENES · p. 19
  • The specular leakage regularizer does not change the rendered image in the reported ablation.The regularizer is discussed in Appendix F.Source: E ABLATIONS ON THE REMAINING SCENES · p. 19

F THE SPECULAR LEAKAGE REGULARIZER

The per-primitive dipole loses accuracy against VolS-GS’s transport solve on most scenes, while the specular leakage regularizer primarily keeps the decomposition identifiable rather than improving PSNR. Together, these comparisons show why the subsurface and specular components are evaluated by their placement and separation, not PSNR alone.

F THE SPECULAR LEAKAGE REGULARIZER

  • Turning on the regularizer changes test PSNR by −0.30 to +0.14 dB across seven scenes, with a mean of −0.08 dB and scene-dependent signs.The full-model change when switching it off is 0.18 dB, within the reported per-scene range.Source: F THE SPECULAR LEAKAGE REGULARIZER · pp. 19, 21
  • The leakage regularizer keeps the decomposition identifiable by penalizing specular energy where light does not reach; on BUNNY, the specular footprint falls from 68% to 23% at unchanged PSNR.Without the regularizer, the specular lobe can paint a ghost into shadowed regions while diffuse washes out.Source: F THE SPECULAR LEAKAGE REGULARIZER · pp. 19, 21, Figure 10 · p. 21

G REPLACING THE SOLVE WITH A PER-PRIMITIVE KERNEL

  • The per-primitive dipole substitution costs accuracy on six of seven scenes, exceeding 2 dB on CUPFABRIC and PIKACHU; DRUMS is the exception on PSNR and SSIM.With other components held fixed, Figure 10(b) shows misplaced transported light on BUNNY and more surface residual in the dipole channel on DRUMS.Source: Figure 10 · p. 21, G REPLACING THE SOLVE WITH A… · p. 21
  • A diffusion profile is a kernel rather than outgoing radiance, and the per-primitive dipole spreads it by Euclidean distance without reading the medium.Unlike that kernel, the BSSRDF integrates the profile against light arriving over the surface, requiring knowledge of where light entered.Source: G REPLACING THE SOLVE WITH A… · p. 21

H DECOMPOSITION ACROSS METHODS

The decomposition comparison finds that SSD-GS can render backlit glow without representing it in its explicit subsurface channel. VolS-GS’s channel carries that radiance, while SSS-GS’s reported SSS output is residual-network energy rather than a physical-channel explanation.

H DECOMPOSITION ACROSS METHODS

  • On STATUE, SSD-GS’s subsurface channel is empty (ratio 0.00 against VolS-GS’s 0.52) although its render contains the backlit glow; VolS-GS’s channel carries that radiance.Figure 11 compares each method’s decomposition on PIXIU and STATUE; the SSS-GS column shows residual-network output.Source: H DECOMPOSITION ACROSS METHODS · pp. 21–22, Figure 11 · p. 22
  • SSS-GS’s physical channels are nearly dark on both scenes, while its residual network carries more energy than its final render on STATUE (ratio 1.5).The reported residual energy is not exposed as the method’s explicit subsurface component.Source: H DECOMPOSITION ACROSS METHODS · pp. 21–22
  • The empty SSD-GS channel on STATUE matches the controlled dipole substitution, where the kernel either misplaces transported light or fires where there is no medium.With no reading of the medium, the kernel has no stable quantity to place in the subsurface channel, so the glow remains in the render but not the decomposition.Source: H DECOMPOSITION ACROSS METHODS · p. 22

I FAILURE CASES

The reported failure cases expose two boundaries: very thin geometry remains difficult for the diffusion approximation, and a broad, nearly uniform highlight-like glow on an opaque object is not targeted by the decomposition.

I FAILURE CASES

  • On BUNNY’s edge-on ear tip, the render is darker than capture; single scattering narrows but does not close the gap where the lobe is only a few mean free paths thick.Increasing transport-grid resolution from 64^3 to 96^3 changes the region by only 0.01 dB; Figure 12 marks the failing region and shows per-frame metrics.Source: I FAILURE CASES · p. 22, Figure 12 · p. 23
  • On opaque HOTDOG, the plate under the bun renders darker than capture because the broad, nearly uniform highlight-like glow is not targeted.Figure 12 marks the region at matching image coordinates and includes an error map.Source: I FAILURE CASES · p. 22, Figure 12 · p. 23

Related papers

Loading 2610.04007v1…