PaperlayerView exact revision on arXiv

Source-linked AI summary

Gaussian Light Transport

Patrick Attimont, Kartic Subr, Cyril Soler

arXiv:2609.11430v1cs.GR

TL;DR

Efficient global illumination remains difficult because the light transport equation is recursive. The paper fits a view-independent radiance field represented by 13D Gaussians, reporting competitive quality with faster training and rendering and low memory use.

  • Problem

    The recursive light transport equation expands into infinite-dimensional integrals, making efficient global illumination in 3D a longstanding challenge.

    Source: §1 · p. 1

  • Method

    The method represents radiance as a 13D Gaussian mixture over position, direction, normal, and material properties, optimizes its equation residual, and uses culling for efficient evaluation.

    Source: §1 · p. 2, §5 · p. 6

  • Results

    The method matches or improves existing neural methods in quality, trains 10–23× faster, renders 2–4× faster, and uses only a few megabytes of memory.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The low-memory representation supports real-time exploration of global illumination solutions in static scenes, including in an architectural viewer.

    Source: §7 · p. 8

    Directional culling becomes less effective when training or GS + MC queries have weaker spatio-directional correlation, causing up to a 1.5× inference slowdown in the worst case.

    Source: §5 · p. 6

Abstract

from arXiv · show

We present a novel method for computing global illumination by expressing the solution to the light transport equation as a 13D Gaussian mixture model over positions, directions, surface normals, and material properties. We show that including scene properties in the Gaussian representation drastically reduces the number of functions and speeds up evaluation. As opposed to traditional light transport methods based on Neumann series, the parameters of our model are directly estimated by minimizing the residual of the rendering equation. While both optimization and rendering require repeated evaluations of a linear combination of high-dimensional Gaussian functions, we introduce an efficient culling strategy to keep the optimization tractable and produce renderings in real time. Our representation enables to render fast, view-independent solutions to the light transport equation, achieving rendering times on the order of milliseconds, with a fraction of the memory requirements of conventional neural rendering approaches.

1 Introduction

The paper addresses the difficulty of efficiently solving global illumination’s recursive light transport equation. It represents radiance with 13D Gaussians optimized against the equation’s residual, reporting competitive quality, faster training and rendering, and low memory use.

1 Introduction

  • The recursive light transport equation expands into infinite-dimensional integrals, making efficient global illumination in 3D a longstanding challenge.Solving it would yield a radiance field usable for photo-realistic images from any viewpoint.Source: §1 · p. 1
  • View-independent solutions compute the scene’s light field once, after which camera-specific images are produced by slicing that field.Source: §1 · p. 2
  • The method represents the light transport solution with 13D Gaussians over positions, directions, normals, and material properties, fitted to minimize the equation’s residual.The representation is view-independent and its kernels are not restricted to scene surfaces.Source: §1 · p. 2
  • The method matches or improves existing neural methods in quality, trains 10–23× faster, renders 2–4× faster, and uses only a few megabytes of memory.Source: §1 · p. 2

2 Related Work

Prior global-illumination methods trade view-independent scene solutions against representation cost, high-frequency fidelity, and computation. The paper situates its Gaussian approach among these methods and related precomputed and learned representations.

2.1 Global illumination methods

  • Galerkin and directional finite-element methods provide view-independent solutions, but geometry-linked bases struggle with complex scenes and glossy materials, while fine 5D bases are costly for caustics and glossy reflections.Collocation methods ease some discretization constraints without resolving scalability issues.Source: §2.1 · p. 2
  • Monte Carlo path tracing handles complex transport phenomena flexibly, but high variance on long paths can make it expensive and poorly suited to view-independent global illumination.Reuse, temporal reprojection, and gradient-domain methods mitigate the cost by reusing or improving computed transport samples.Source: §2.1 · p. 2

2.2 View-independent precomputed radiance

  • Precomputed radiance transfer supports interactive scene changes but struggles with local radiance adaptation and can require substantial memory because its bases are tied to geometry.Source: §2.2 · p. 2
  • Radiance caches render faster than PRT in the cited comparison but require gigabytes and retrieve indirect illumination, whereas the paper’s evaluated field includes direct and indirect illumination.The passage reports cache rendering can be an order of magnitude faster (10 ms) than PRT.Source: §2.2 · p. 2
  • Image-based Gaussian and neural approaches can model novel-view appearance, but typically approximate observed radiance rather than solving the rendering equation.The paper’s illustration shows its own Gaussians fitted to the rendering-equation residual and queried at ray-traced points, so only the first visible surface contributes and visibility needs no explicit treatment.Source: §2.2 · p. 3, Figure 2 · p. 3

3 Technical Description

The method optimizes a radiance field represented by a variable mixture of 13D Gaussians to minimize rendering-equation residuals. It adapts the mixture and uses spatial culling to make optimization and rendering practical.

3 Technical Description

  • Radiance is parameterized as a weighted sum of a variable number of 13D Gaussians over position, direction, normal, albedo, and roughness, with parameters optimized to minimize the rendering-equation residual.The objective is the squared L2 residual norm over surface area and angular domains.Source: §3 · p. 3
  • Including position, direction, normal, albedo, and roughness in each Gaussian lets the representation adapt to illumination variation and share contributions across spatially separate points with similar scene features.The passage gives spatially varying materials with identical albedo and roughness as an example of sharing.Source: §3 · p. 3
  • Stochastic-gradient training estimates the residual by Monte Carlo integration, while a semi-gradient formulation drops the inner-integral gradient, trading some bias for lower gradient variance.Source: §3 · p. 4
  • The mixture is refined by splitting Gaussians with large subspace gradients and spawning new ones at high-residual samples, then pruning kernels with negligible relative contribution.Spawning adds coverage where splitting cannot; pruning uses a relative contribution threshold.Source: §3 · p. 4

3.1 Model evaluation and rendering

  • Rendering either evaluates learned radiance at visible surface points or traces one camera-ray bounce and evaluates the model as incident illumination.The representation also uses query-point albedo, normal, and roughness as selection features; albedo and normal Gaussians have no rotation in the diagonal representation.Source: §3.1 · p. 4, Figure 3 · p. 4
  • World-space tile culling groups queries by shape and position, bounds their 13 input dimensions, and rejects irrelevant Gaussian kernels before evaluation.On Bedroom, it evaluates an average of 76 kernels per pixel out of 22K total.Source: §3.1 · p. 4

4 Implementation

The implementation fits an adaptive Gaussian representation with a loss that balances radiance regions and controls kernel anisotropy. It accelerates training and rendering with Gaussian management and separable culling.

Implementation

  • The implementation uses Mitsuba/Python and PyTorch for scene management, Gaussian representation, and training, with custom Taichi kernels for evaluation and analytic derivatives.Source: §4 · pp. 4–5

Covariance matrices

  • Factoring each 13D Gaussian into independent position, direction, material, and normal subspaces improves performance while sacrificing cross-subspace correlations, with relatively low impact on converged-model quality.The paper uses this factorization in all reported results and maintains spatial locality in 3D.Source: §4 · p. 5

Loss function

  • Normalizing the linear-radiance loss by the current prediction prevents bright regions from dominating darker ones; the prediction is held constant during differentiation.The paper reports this normalization as effective.Source: §4 · p. 5
  • Near-coincident surfaces can prompt elongated positional Gaussians; an anisotropy penalty addresses this and avoids artifacts such as the thin bright stripe (ε = 2.5).The penalty is weighted by λ_aniso = 0.008 in all examples.Source: §4 · p. 5, Figure 4 · p. 5

Optimization

  • Gradient estimation samples 32 incident directions and 32 explicit light samples for each outgoing direction to reduce variance from direct illumination.Source: §4 · p. 5
  • Combining both residual evaluations into one tensor saves a spatial reordering and culling pass per iteration, and reuses culling during backpropagation.Source: §4 · p. 5
  • Adam training splits Gaussians every 400 iterations, spawns 500 every 500 iterations until iteration 15,000, and prunes every 2,000 iterations.Splitting and spawning stop at iteration 15,000; pruning continues throughout training.Source: §4 · p. 5

Rendering

  • Separable Mahalanobis tests cull Gaussians against tile bounds, using cheap 1D rejection before spatial and directional 3D tests; perfectly specular surfaces are not directly learned.For such paths, rendering and optimization extend paths to the first smooth surface.Source: §4 · pp. 5–6

5 Discussion

The discussion relates the adaptive Gaussian representation to kernel methods and identifies limits in sampling, culling, scene scope, and discontinuity handling. It also interprets residual optimization as recursive training.

5 Discussion

  • The representation centers Gaussians on scene features and learns their weights, positions, shapes, and number, yielding an adaptive nonstationary Gaussian-process kernel.Unlike the compared N-dimensional Gaussian method, it solves the rendering equation itself; fewer principal-component axes support more efficient culling and lower memory use.Source: §5 · p. 6
  • The adaptive representation avoids the pitfalls of bandwidth-limited constructions.Source: §5 · p. 6
  • Repeated evaluation of the model for outgoing and incident radiance can be viewed as recursive training using its previous state as the current solution estimate.Source: §5 · p. 6
  • Uniform surface sampling favors Gaussians with large positional footprints, while curvature-based sampling learns glossy reflections on curved surfaces quicker.The authors suggest sampling according to local angular and spatial bandwidth to focus work where needed.Source: §5 · p. 6
  • The current implementation handles static scenes only; an extra temporal Gaussian dimension is proposed as one possible route for predefined changes.Source: §5 · p. 6
  • Culling is less effective when training or GS + MC queries have reduced spatio-directional correlation, causing a worst-case 1.5× inference slowdown.The authors suggest direction-aware tiling or clustering to preserve locality.Source: §5 · p. 6
  • Because the representation is smooth by construction, position- or direction-only radiance discontinuities are softened, especially near specular reflections.Albedo, normal, and roughness jumps at material and geometric edges remain sharp.Source: §5 · p. 6

6 Results

The method trains and renders faster than vertex-feature Neural Radiosity, with a compact stored model and satisfactory LIP comparisons against path-traced references. Ablations and scene examples show how the representation performs across rendering modes and complex illumination conditions.

6 Results

  • LIP comparisons with the path-traced reference are reported as very satisfactory.Source: §6 · p. 6
  • Without albedo in the Gaussian dimensions, textured regions require 37.9K scene-total Gaussians, compared with 35.2K when albedo is included.Source: Figure 6 · p. 7
  • Except on Rings, adding dimensionality, spawning, splitting, and regularization clearly improves image quality; the splitting-disabled run starts with 30K Gaussians.The Rings difference is described as little to no impact.Source: §6 · pp. 6–7
  • Training time is reduced 10.0–23.4× and rendering is 2.2–4.5× faster than vertex-feature Neural Radiosity at one sample per pixel.The comparison is against the stronger of the two baselines.Source: §6 · p. 7, Table 2 · p. 7
  • The representation occupies 1–5.9 MB on disk, versus 35.7 MB for the hash grid and 0.9–26.4 MB for vertex features across scenes.The model size is bounded by the number of Gaussians rather than grid resolution or geometry tessellation.Source: §6 · p. 7
  • GS + MC is noisier at one sample but more accurate once converged, overtaking GS at 1024 samples on every scene.GS + MC traces one Monte Carlo bounce first.Source: §6 · p. 7
  • The method handles geometric complexity, caustics, glossy and specular surfaces, and textures; multi-lobed BSDFs yield diffuse and glossy Gaussian components.Source: §6 · p. 7

7 Conclusion

The Gaussian-mixture method enables real-time exploration of global illumination solutions in static scenes. Its low memory footprint supports embedding a viewer for exploring architectural constructions, while future work includes participating media and light-field export.

7 Conclusion

  • The Gaussian-mixture method enables real-time exploration of global illumination solutions in static scenes, and its low memory footprint makes it easy to embed in a viewer for architectural exploration.Source: §7 · p. 8
  • Future work includes extending the method to participating media and exporting light fields; the representation could also support frequency-based filtering and global-illumination baking.Source: §7 · p. 8

Related papers

Loading 2609.11430v1…