Source-linked AI summary
Stochastic Graph Compression for Constant-Memory Differentiable Light Tracing
Linas Beresna, Eugene Fiume
TL;DR
Light-tracing gradients can require memory that grows with path depth and sensor connections, unlike viewpoint path replay, whose non-branching paths can be reconstructed from a PRNG seed. ResLRB retains one weighted-reservoir connection per light path and replays the path for an unbiased adjoint estimator; it holds memory constant at 142 MB and, at depth 16, runs in 132 ms versus naive AD’s 260 ms.
Problem
Reverse-mode AD records operations across paths and can exceed available GPU memory, while PRB's non-branching replay structure does not directly apply to light paths that connect to the sensor at every vertex.
Source: §1 · pp. 1–2Method
ResLRB retains one sensor connection per light path in a streaming weighted reservoir, then replays the path in the adjoint pass and backpropagates through the retained connection.
Source: §1 · p. 2Results
ResLRB holds memory constant at 142 MB; at depth 16 its gradient step costs 132 ms versus naive AD's 260 ms, and naive AD exhausts GPU memory beyond depth 16.
Source: §3 · pp. 3–4Takeaways & Limitations
The retained-connection estimator is unbiased, with higher variance than evaluating all k connections; reservoirs of capacity m > 1 offer a memory–variance trade-off.
Source: §2.3 · p. 3, §4 · p. 4The method is developed for the detached case, where sensor coordinates are independent of scene parameters and only connection flux depends on them.
Source: §2.1 · p. 2
Abstract
from arXiv · showhide
We describe Reservoir Light Replay Backpropagation (ResLRB), a method for reverse-mode differentiable light tracing whose memory is constant in path length and in the number of sensor connections per light path. Naive automatic differentiation of a light tracer records a computation graph that grows with the number of valid sensor connections along each path, since flux can be splatted from every scattering vertex; adjoint path replay removes the analogous dependence on depth for viewpoint path tracing, but its non-branching structure does not extend to the splatting case. We compress the adjoint graph during the primal pass by stochastically retaining a single representative sensor connection per light path via a streaming weighted reservoir. The adjoint pass reconstructs the path deterministically by replaying its pseudorandom number sequence and backpropagates only through the retained connection, yielding an unbiased gradient estimator whose peak memory is that of differentiating a single scattering event.
1 Introduction
Reverse-mode AD can exhaust GPU memory because it stores operations across every scattering event, while path replay provides constant memory for viewpoint path tracing. Light tracing branches at each sensor connection, so ResLRB uses a weighted reservoir to retain one connection and replay the path during the adjoint pass.
1 Introduction
- Reverse-mode AD stores operations across every scattering event and can exceed available GPU memory for deep paths or high sample counts.Radiative Backpropagation recasts gradient computation as a transport problem, trading memory for time.Source: §1 · p. 1
- PRB stores a PRNG seed, then replays the path in the adjoint pass to achieve constant memory in path depth.The replay regenerates the same random sequence and vertices, at the cost of one additional traversal.Source: §1 · pp. 1–2
- Unlike viewpoint paths, light paths can connect to the sensor at every scattering vertex, producing up to k separate contribution graphs for a path of depth k.Prior adjoint light tracing targets view-independent object-space irradiance [Lipp et al. 2024]; image-space splatting introduces the branching considered here.Source: §1 · p. 2
- ResLRB retains one sensor connection per light path in a streaming weighted reservoir, then replays the path and backpropagates through that connection for an unbiased estimator.Its memory is constant in path length and sensor-connection count, and it preserves PRB’s two-pass structure.Source: §1 · p. 2
2 Method
ResLRB compresses each light path’s sensor connections into one weighted-reservoir sample, then replays the path to compute an unbiased gradient while keeping only one local scattering graph in memory. The method also extends to attached differentiation, where scene parameters can move the splatted sensor position.
2.1 Notation
- A light path starts at an emitter and can contribute a flux-and-sensor-coordinate pair at each scattering vertex; each flux depends on scene parameters, while the detached formulation holds sensor positions fixed.The attached extension allows sensor positions to depend on scene parameters as well.Source: §2.1 · p. 2
2.2 Stochastic Selection of One Connection
- A streaming weighted reservoir retains one sensor connection per path, with luminance weights favoring connections that dominate the path’s image contribution.Only the selected connection’s data and the final weight sum are kept; the other candidates are discarded.Source: §2.2 · p. 2
2.3 Unbiased Estimator
- The gradient estimator is unbiased for the per-path gradient when the splat functions are sufficiently smooth for differentiation and expectation to be interchanged.Its variance increases relative to evaluating all k connections.Source: §2.3 · p. 3
2.4 Two-Pass Evaluation via Path Replay
- PRNG replay reconstructs the path to the retained connection, while a fresh local AD scope per vertex keeps peak memory bounded by one scattering event.The primal pass stores the seed, selected index, and final weight sum; the adjoint pass re-traces and differentiates local scattering computations.Source: §2.4 · p. 3
2.5 Extension to Attached Differentiation
- For attached differentiation, cached path Jacobians propagate parameter-induced motion of the sensor position through the selected connection’s splat and flux Jacobians.Rank-deficient Jacobians at perfectly diffuse vertices are regularized because their inverse is otherwise ill-defined.Source: §2.5 · p. 3
3 Verification
ResLRB’s memory remains bounded as path depth grows, while naive reverse-mode AD’s memory grows until it exhausts GPU capacity. Their gradients agree up to Monte Carlo noise, and replay overhead is offset at greater depths.
3 Verification
- ResLRB and naive AD gradients agree in sign and spatial structure, differing only by Monte Carlo noise.Figure 5 shows the gradient of an L2 image loss with respect to floor diffuse reflectance at path length 32 and 32 spp; red denotes positive, blue negative, and grey zero.Source: §3 · p. 3, Figure 5 · p. 4
- 142 MB is ResLRB’s constant memory use; naive AD grows linearly and exhausts GPU memory beyond depth 16.The bounded graph is limited by differentiating a single scattering event.Source: §3 · p. 3, Figure 6 · p. 4
- At depth 4, ResLRB takes 1.5× as long as naive AD (93 vs. 63 ms), reaches parity by depth 8, and is roughly twice as fast at depth 16 (132 vs. 260 ms).Naive AD no longer runs beyond depth 16; ResLRB’s cost rises from 93 to 197 ms between depths 4 and 128.Source: §3 · p. 4
4 Conclusion
The conclusion presents a two-pass formulation that compresses each light path’s sensor connections into a single weighted-reservoir sample, then replays the path with local AD scopes. It identifies larger reservoirs, multi-pass gathering, and the attached extension as directions supported by the framework.
4 Conclusion
- The two-pass formulation keeps memory constant in path length and sensor connections per path by retaining one weighted-reservoir connection, replaying the path, and using a local AD scope per vertex.The conclusion presents this as the framework’s central result.Source: §4 · p. 4
- Reservoirs with capacity m > 1 interpolate along the memory–variance trade-off.The conclusion also notes that multi-pass gathering could retain every connection at the cost of further traversals.Source: §4 · p. 4
- The attached extension opens the framework to geometric inverse problems.The conclusion describes this extension as sketched above.Source: §4 · p. 4