Source-linked AI summary
StochasticSplats: Stochastic Rasterization for Sorting-Free 3D Gaussian Splatting
Shakiba Kheradmand, Delio Vicini, George Kopanas, Dmitry Lagun, Kwang Moo Yi, Mark Matthews, Andrea Tagliasacchi
TL;DR
Sorted 3DGS can make rendering cost hard to control and can produce artifacts. This paper uses stochastic transparency with sorting-free Monte Carlo rendering and gradient estimators; at one sample per pixel, its OpenGL implementation renders up to 4× faster than the original CUDA implementation.
Problem
Depth-sorted 3DGS can cause rendering artifacts and offers no straightforward way to balance rendering cost against visual fidelity.
Source: §1 · p. 2, §3.1 · p. 3Method
The method uses stochastic transparency with Monte Carlo estimators for forward alpha blending and gradient computation, rendering samples without sorting.
Source: §1 · p. 2Results
At one sample per pixel, the OpenGL implementation renders up to 4× faster than the original CUDA implementation.
Source: §1 · p. 2Takeaways & Limitations
Sampling can be adjusted at rendering time to trade latency against quality, including for latency-critical applications such as open-vocabulary 3D object localization.
Source: §1 · p. 2Results are noisy when rendered with low sample counts.
Source: §5 · p. 9
Abstract
from arXiv · showhide
3D Gaussian splatting (3DGS) is a popular radiance field method, with many application-specific extensions. Most variants rely on the same core algorithm: depth-sorting of Gaussian splats then rasterizing in primitive order. This ensures correct alpha compositing, but can cause rendering artifacts due to built-in approximations. Moreover, for a fixed representation, sorted rendering offers little control over render cost and visual fidelity. For example, and counter-intuitively, rendering a lower-resolution image is not necessarily faster. In this work, we address the above limitations by combining 3D Gaussian splatting with stochastic rasterization. Concretely, we leverage an unbiased Monte Carlo estimator of the volume rendering equation. This removes the need for sorting, and allows for accurate 3D blending of overlapping Gaussians. The number of Monte Carlo samples further imbues 3DGS with a way to trade off computation time and quality. We implement our method using OpenGL shaders, enabling efficient rendering on modern GPU hardware. At a reasonable visual quality, our method renders more than four times faster than sorted rasterization.
1. Introduction
3DGS offers efficient rendering, but depth sorting can cause costly rendering, popping artifacts, and little control over the cost–fidelity tradeoff. The paper introduces stochastic transparency for 3DGS and reports faster rendering with sampling that trades latency against quality.
1. Introduction
- 3DGS represents scenes with anisotropic 3D Gaussians that can be rasterized for real-time viewing and efficient training, with visual quality comparable to NeRF.NeRF produces high-quality visuals at low rendering speed, while 3DGS enables efficient rendering.Source: §1 · p. 2
- Depth-sorting 3DGS billboards can prevent accurate volumetric intermixing, cause view-dependent popping, and offer no straightforward way to balance rendering cost against visual fidelity.The authors observe that reducing image resolution does not necessarily make rasterization faster.Source: §1 · p. 2
- Prior approaches address popping, rendering cost, or portability but typically trade one against another, while per-pixel sorting still cannot fully solve volumetric intermixing.Constant-density ellipsoids allow analytical intermixing but use slower ray tracing; other approaches may require complicated GPGPU kernels.Source: §1 · p. 2
- Stochastic transparency replaces sorted alpha blending with an unbiased Monte Carlo estimator, probabilistically selecting opaque samples and averaging them with multisampling.Individual samples use order-independent Z-buffering without sorting, and the method provides forward blending and gradient estimators compatible with a portable OpenGL implementation.Source: §1 · p. 2
- At one sample per pixel (SPP), the OpenGL implementation renders up to 4× faster than the original CUDA implementation; higher SPP increase quality and render time, approaching the PSNR of alpha-blended 3DGS.Downstream applications can dynamically adjust sampling at rendering time to trade latency against quality.Source: §1 · p. 2
2. Related Work
Related work spans stochastic and non-stochastic order-independent transparency, methods addressing popping and rendering speed, and differentiable Monte Carlo techniques. The paper positions stochastic transparency for 3DGS as unexplored in rendering and training.
Order-independent transparency
- Order-independent transparency methods include stochastic fragment-discard estimators and non-stochastic approaches such as linked lists, depth peeling, weighted blending, moments, and deep learning.Recent preprints combined 3DGS with non-stochastic transparency, while stochastic variants remained unexplored.Source: §2 · pp. 2–3
Popping artifacts
- Per-pixel sorting addresses popping but can be prohibitively costly, while ray tracing constant-density ellipsoids computes volume rendering more fully but is slower than rasterization.Stochastic transparency instead trades sorting for image noise to eliminate popping.Source: §2 · p. 3
Fast 3DGS rendering and training
- Reducing Gaussian count and compressing attributes can further speed 3DGS, and such approaches complement this work.Implementation improvements to training and rendering routines also provide speedups.Source: §2 · p. 3
Differentiable Monte Carlo methods
- Differentiable Monte Carlo methods have been used in inverse rendering, neural radiance fields, surface reconstruction, Gaussian ray tracing, and other fields; this work introduces a Monte Carlo gradient estimator for stochastic transparency.Prior applications also include nuclear engineering, finance, atmospheric physics, and inverse partial differential equation problems.Source: §2 · p. 3
3. Method
The method replaces depth-sorted alpha blending with stochastic transparency, then extends the estimator to gradients and volumetric intermixing. It also approximates Gaussian surfaces to reduce popping while retaining efficient hardware depth computation.
3.1. Sorted alpha blending
- Sorted alpha blending accumulates radiance in depth order, so sorting is required and render cost cannot be traded against accuracy by changing pixel count.The method projects Gaussians onto camera-oriented billboards using a linear approximation of projection.Source: §3.1 · p. 3
3.2. Stochastic transparency
- Stochastic transparency estimates alpha blending by sampling opaque fragments without sorting, and its estimator is unbiased when every nonzero summand has positive sampling probability.Multiple independent samples per pixel reduce estimator noise; standard depth testing and discard implement selection for any primitive ordering.Source: §3.2 · pp. 3–4
3.3. Differentiable stochastic transparency
- The stochastic gradient estimator samples a summand without differentiating its probability, assigning color gradients only to the sampled Gaussian and opacity gradients to it and nearer Gaussians.The derivation applies beyond 3DGS to representations rendered with stochastic transparency.Source: §3.3 · p. 4
- Decorrelated loss-gradient samples yield unbiased gradients for L2 loss and reduce bias for other losses, but unbiased estimation for arbitrary losses remains open.The two noisy factors are evaluated using different random samples.Source: §3.3 · p. 4
- The stochastic gradient images are noisy but closely match sorted alpha-blending gradients in structure.Validation compares the estimators at 128 and 512 SPP; red and blue encode positive and negative values.Source: §3.3 · p. 5, Figure 3 · p. 5
3.4. Removing popping artifacts
- Approximating each Gaussian’s maximum-density surface with a plane lets hardware interpolation compute splat depth without extra render time, with popping-free results at similar PSNR to [36] at higher sample counts.The approximation avoids per-fragment depth changes that inhibit hardware early discard.Source: §3.4 · p. 5
- For full volumetric intermixing, stochastic transparency samples free-flight distances and takes the minimum across overlapping Gaussians, resolving arbitrary overlaps without sorting, ray tracing, or extra data structures.The method reparameterizes opacity with a spatially varying extinction function.Source: §3.4 · p. 6
4. Experiments
Experiments assess rendering quality, computation time, resolution changes, and single-sample use on MipNerf360 and LERF. StochasticSplats offers a sample-count quality–time trade-off, with single-sample speed and demonstrated use for semantic localization.
4. Experiments
- Experiments use MipNerf360 scenes for rendering evaluation and LERF for semantic localization, compare against 3DGS, Splatapult, and StopThePop, and report image-quality metrics and per-frame processing time.The authors fine-tune trained scenes rather than training from scratch.Source: §4 · p. 6
4.1. Rendering quality
- Room and Kitchen renderings are reasonable at 1 SPP, while 16 SPP closely matches alpha blending; increasing samples also brings novel-view quality toward alpha blending.The authors note that Monte Carlo noise can be less distracting than structured artifacts and may be mitigated with temporal anti-aliasing.Source: §4.1 · p. 6, Table 1 · p. 6, Figure 7 · p. 7
4.2. Computation time
- With one sample, StochasticSplats achieves 2× to 4× lower latency compared to alpha-blended 3DGS, and is described as fast, portable, and pop-free.The paper also reports CUDA backward-pass timings because training typically uses CUDA devices.Source: Table 2 · p. 7, §4.2 · pp. 7–8
4.3. Lowering resolution does not enhance speed
- Lowering resolution does not necessarily speed alpha blending: render time decreases only to a point, then increases as more Gaussians contribute per tile; no tested configuration matches StochasticSplats at 1 SPP.The authors attribute this to increased Gaussian counts per pixel as resolution falls.Source: §4.3 · p. 8
4.4. Making the most out of a single sample
- Temporal anti-aliasing reduces rendering noise by accumulating warped prior-frame samples, while 1-SPP semantic localization retains correct object locations despite moderate accuracy reduction and renders 4× faster.The localization comparison uses the LERF dataset and contrasts StochasticSplats with alpha blending.Source: Table 4 · p. 8, Figure 10 · p. 8, §4.4 · p. 8
5. Conclusion
The conclusion presents sorting-free stochastic rendering for 3DGS with forward and backward passes, a quality–cost trade-off, and pop-free rendering. It reports reasonable-quality OpenGL rendering 2×–4× faster, while noting low-sample noise.
5. Conclusion
- The method derives forward and backward stochastic rendering for 3DGS and is technically compatible with any semi-transparent representation.Source: §5 · p. 8
- OpenGL rendering is 2×–4× faster with reasonable quality, and supports a quality-versus-cost trade-off without popping artifacts.Source: §5 · p. 9
- Low-sample results remain noisy; suggested future directions include variance reduction and lightweight neural denoising.Source: §5 · p. 9
1. Per-scene results
Tables 5 and 6 report rendering times across scenes and rendering techniques.
1. Per-scene results
- Tables 5 and 6 show times for different scenes and rendering techniques, presenting rendering-time results across scene choices and technique choices in the reported comparison.Source: §1 · p. 12
2. Tighter image-space bounding box
The section describes a tighter image-space bounding box that accounts for a projected Gaussian’s eigenvectors and eigenvalues, reducing unnecessary per-tile Gaussian overhead.
2. Tighter image-space bounding box
- Reducing the number of Gaussians within each tile helps the CUDA implementation, which rasterizes all Gaussians without an early-termination transmittance threshold.The method uses ϵo = 1/255 in all results, including 3DGS.Source: §2 · p. 12
- 3DGS uses the larger eigenvalue, scaled by tO, to bound the screen-space Gaussian radius and form a square bounding box.The box is axis-aligned and centered on the projected Gaussian.Source: §2 · p. 12
- For elongated Gaussians, accounting for both eigenvalues and eigenvectors yields a tighter fit than the square approximation, reducing unnecessary per-tile Gaussian overhead.The OpenGL implementation computes a tighter, non-axis-aligned fit and can refine it by increasing the number of corner samples.Source: §2 · p. 12
3. Temporal anti-aliasing
Temporal accumulation reprojects prior-frame samples and blends them with current samples when their world-space positions match; mismatches reset accumulation.
3. Temporal anti-aliasing
- At the first frame, the method records each pixel’s world-space position and color.The recorded positions are used in subsequent frames to check whether warped samples correspond to current pixels.Source: §3 · p. 12
- On later frames, prior samples are warped into the new view and blended with current color and position when their 3D coordinates closely match.The color update averages the warped prior color and current color using the accumulated sample count.Source: §3 · p. 12
- Tables 5 and 6 give average rendering times on 3090 and 4090 GPUs, respectively.The supplied passages do not state specific timing values.Source: Table 5 · p. 12, Table 6 · p. 12
- New samples more than threshold τ from the warped mesh are treated as occluded, and accumulation is reset for those pixels.Average position is well defined for surfaces but poorly approximated in less-confident volumetric regions.Source: §3 · p. 12
4. Free-flight distance sampling
The method samples free-flight distance using an analytical density integral and an inverse-CDF routine for the resulting probability density.
4. Free-flight distance sampling
- The section defines a volumetric density for a 3D Gaussian, with σt scaling the Gaussian’s density uniformly and x denoting a 3D position.The density is the starting point for deriving free-flight distance samples.Source: §4 · p. 12
- The density integral along ray (o, d) is computed analytically, enabling an analytical free-flight probability density p(t).Here, t is distance along the ray.Source: §4 · p. 13
- An analytical inverse-CDF routine samples distance t proportional to p(t), using u drawn from U(0, 1).The procedure follows [5].Source: §4 · p. 13