PaperlayerView exact revision on arXiv

Source-linked AI summary

Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives

Alex Hanson, Allen Tu, Geng Lin, Vasu Singla, Matthias Zwicker, Tom Goldstein

arXiv:2412.00578v3cs.CVcs.GR

TL;DR

3D-GS rendering speed is limited by high parameter counts and algorithmic inefficiencies, particularly for resource-constrained settings. Speedy-Splat combines precise Gaussian localization with training-pipeline pruning; across evaluated scenes, it improves rendering speed, model size, and training speed while maintaining high image quality.

  • Problem

    3D-GS rendering remains limited by high parameter counts and algorithmic inefficiencies, while real-time rendering on resource-constrained edge devices has yet to be achieved.

    Source: §1 · p. 1

  • Method

    Speedy-Splat precisely localizes Gaussians during rendering and integrates pruning into the 3D-GS training pipeline.

    Source: §1 · p. 2

  • Results

    6.71× average rendering-speed acceleration, 10.6× model-size reduction, and 1.47× training-time improvement are reported across Mip-NeRF 360, Tanks & Temples, and Deep Blending scenes.

    Source: §7 · p. 8

  • Takeaways & Limitations

    The approach targets both pixels processed per Gaussian and the total number of Gaussians, the two factors identified as proportional to rendering cost.

    Source: §1 · p. 2

    Speedy-Splat produces slightly lower image quality than 3D-GS, a degradation the authors note at high compression ratios.

    Source: §6 · p. 8

Abstract

from arXiv · show

3D Gaussian Splatting (3D-GS) is a recent 3D scene reconstruction technique that enables real-time rendering of novel views by modeling scenes as parametric point clouds of differentiable 3D Gaussians. However, its rendering speed and model size still present bottlenecks, especially in resource-constrained settings. In this paper, we identify and address two key inefficiencies in 3D-GS to substantially improve rendering speed. These improvements also yield the ancillary benefits of reduced model size and training time. First, we optimize the rendering pipeline to precisely localize Gaussians in the scene, boosting rendering speed without altering visual fidelity. Second, we introduce a novel pruning technique and integrate it into the training pipeline, significantly reducing model size and training time while further raising rendering speed. Our Speedy-Splat approach combines these techniques to accelerate average rendering speed by a drastic $\mathit{6.71\times}$ across scenes from the Mip-NeRF 360, Tanks & Temples, and Deep Blending datasets.

1. Introduction

3D-GS rendering speed remains limited by parameter counts and algorithmic inefficiencies, while few compression approaches directly target speed. Speedy-Splat combines precise Gaussian localization with pruning to improve rendering speed, model size, and training time.

1. Introduction

  • Rendering cost depends on both the number of Gaussians and the pixels processed per Gaussian, and Speedy-Splat targets both factors.Its four named components are SnugBox, AccuTile, Soft Pruning, and Hard Pruning.Source: §1 · p. 2
  • SnugBox precisely bounds each Gaussian and AccuTile identifies exact tile intersections, yielding 1.82× and 1.99× average inference speed-ups without changing 3D-GS renderings.Both methods are plug-and-play alternatives to conservative Gaussian-to-tile localization.Source: §1 · p. 2
  • Soft Pruning removes 80% of Gaussians at regular intervals during densification, and Hard Pruning removes an additional 30% after densification.The pruning approach extends an existing method, reduces its memory requirement by 36×, and is integrated into training to preserve visual fidelity.Source: §1 · p. 2
  • 6.71× average rendering-speed acceleration, 10.6× model-size reduction, and 1.4× training-speed improvement are reported across evaluated scenes while maintaining high image quality.Speedy-Splat integrates both techniques into the 3D-GS training pipeline.Source: §1 · p. 2

2. Related work

Related work addresses 3D-GS training and inference performance, including Gaussian pruning and changes to densification, optimization, and GPU scheduling. Speedy-Splat's pruning approach extends PUP 3D-GS with lower memory requirements.

2. Related work

  • The related-work discussion frames these methods as efforts to improve 3D-GS training and inference performance in resource-constrained environments.It identifies desktop-GPU real-time rendering as motivation for that research.Source: §2 · p. 2

2.1. Pruning

  • Prior pruning methods generally score Gaussians for removal, including approaches based on aggregated ray contributions [6] [7] [18] [25] or opacity with gradient information [1] [2] [20].Other criteria include resolution- and scale-aware redundancy [26], total transmittance [10], and efficiency-aware inference with foveated rendering [19].Source: §2.1 · p. 2
  • PUP 3D-GS derives a Hessian-based sensitivity for each Gaussian and achieves state-of-the-art post-hoc pruning, but its score's storage needs limit training use [11].Speedy-Splat directly extends PUP 3D-GS and improves its memory efficiency by 36×.Source: §2.1 · p. 2

2.2. Other Methods

  • Other work changes densification, optimization, Gaussian-count constraints, or GPU scheduling; most approaches are orthogonal and can be applied alongside Speedy-Splat.StopThePop also uses precise tile intersections, while the paper reports Speedy-Splat is notably faster in Appendix A.4 [27].Source: §2.2 · p. 2

3. Background

The background describes 3D-GS primitives, optimization, and tile-based rendering, then outlines the processing stages from Gaussian projection and tile mapping through pixel-wise rendering.

3.1. 3D Gaussian Splatting Overview

  • 3D-GS represents scenes with differentiable 3D Gaussians initialized from an SfM point cloud; training optimizes image reconstruction losses and periodically densifies and prunes Gaussians.Each primitive has geometry parameters (mean, scale, rotation) and color parameters (spherical harmonics and opacity).Source: §3.1 · pp. 2–3

3.2. 3D Gaussian Splatting Rendering Specifics

  • 3D-GS divides rendered images into 16×16-pixel tiles, maps Gaussians to intersected tiles, and sorts those mappings by tile and depth for pixel-wise rendering.Rendering runtime is dominated by six key functions, whose execution times are analyzed in Table 1.Source: §3.2 · p. 3

3.2.1. Preprocessing

  • Preprocessing projects each Gaussian and its covariance into image space, then uses the largest eigenvalue of the projected 2D covariance to estimate intersected tiles.The projected mean, depth, inverse covariance, and view-dependent color are stored for later processing.Source: §3.2.1 · p. 3

3.2.2. Sorting

  • InclusiveSum allocates Gaussian-to-tile arrays, duplicateWithKeys creates tile-and-depth keys, RadixSort orders them, and identifyTileRanges prepares keys for pixel rendering..Source: §3.2.2 · p. 3

3.2.3. Rendering

  • The render kernel processes tile Gaussians in depth order; per-Gaussian alpha is computed from opacity and the projected 2D Gaussian, then contributes to pixel-color compositing above the stated alpha threshold.The passage identifies the 2D Gaussian value at pixel p and gives the alpha-compositing expression for pixel color.Source: §3.2.3 · pp. 3–4

4. Methods

The methods target two 3D-GS inefficiencies: overestimated Gaussian extents and redundant primitives. They refine tile localization and add memory-efficient pruning to the training pipeline.

4. Methods

  • 3D-GS overestimates Gaussian extent, while pruning work indicates its models are heavily overparameterized.These are the two inefficiencies motivating the methods.Source: §4 · p. 4

4.1. Precise Tile Intersect

  • Using the alpha threshold for pixel contribution gives a more concise set of intersected tiles than the conservative extent based on projected covariance.Pixels with alpha below 1/255 do not contribute to rendering.Source: §4.1 · p. 4

4.1.1. SnugBox

  • 1.82× average overall speed-up: SnugBox computes a tight bounding box around each Gaussian and converts its edges to tile indices.The computation uses a constant number of operations and is called twice in the rendering pipeline.Source: §4.1.1 · p. 4, §4.1 · p. 4

4.1.2. AccuTile

  • 1.99× average overall speed-up: AccuTile extends SnugBox to find exact intersected tiles, processing along the shorter side of the tile extent.Its tile counting time is proportional to the shorter side; tile processing time is proportional to the tile count.Source: §4.1.2 · pp. 4–5

4.2. Efficient Pruning

  • The pruning approach builds on PUP 3D-GS with an efficient score and applies Soft Pruning during densification and Hard Pruning afterward.The two modalities operate before and after iteration 15000, respectively.Source: §4.2 · p. 6

4.2.1. Efficient Pruning Score

  • Reparameterizing the Hessian around each Gaussian’s projected 2D value makes its score use gradients already computed in rendering’s backward pass and cuts storage by 36×.The resulting storage requirement is proportional to the number of Gaussians, N.Source: §4.2.1 · p. 6

4.2.2. Soft Pruning

  • Soft Pruning removes Gaussians immediately before opacity resets at iterations 6000, 9000, and 12000; visual fidelity is preserved at 80% pruning in the experiments.The L1 loss is observed to be low by iteration 6000 except after an opacity reset.Source: §4.2.2 · p. 6

4.2.3. Hard Pruning

  • Hard Pruning removes 30% of Gaussians every 3000 iterations from iteration 15000, and pairing it with Soft Pruning reduces total Gaussians by 10.6× across scenes.The authors report that post-densification iterations can further refine the pruned model.Source: §4.2.3 · p. 7

5. Experiments

The experiments evaluate Speedy-Splat on scenes from three datasets and compare its cumulative rendering optimizations, full pipeline, and pruning score with alternatives. The full pipeline achieves the strongest compression, rendering, and training speed-ups across datasets while retaining competitive image quality.

5.1. Datasets

  • The evaluation covers nine Mip-NeRF 360 scenes, two Tanks & Temples scenes, and two Deep Blending scenes, using dataset-provided COLMAP poses and sparse point clouds.The Mip-NeRF 360 scenes include four indoor and five outdoor scenes.Source: §5.1 · p. 7

5.2. Implementation Details

  • Rendering times and FPS are measured with CUDA events on an Nvidia RTX A5000 GPU, and Speedy-Splat metrics average three independent runs per scene.The renderer builds on 3D-GS, with Python training-pipeline modifications for pruning schedules and execution.Source: §5.2 · p. 7

5.3.1. Additive method performance

  • 6.71× overall speed-up over baseline 3D-GS follows additive gains from SnugBox, AccuTile, soft pruning, and hard pruning.The reported overall speed-ups are 1.82× with SnugBox, 1.99× with AccuTile, and 3.14× after soft pruning, before hard pruning reaches 6.71×.Source: §5.3.1 · p. 8

5.3.2. Overall Performance

  • Speedy-Splat’s full pipeline has the highest compression, rendering, and training speed-ups across datasets, with competitive image quality.Qualitative results also report nearly identical images to PUP 3D-GS while achieving over double its FPS.Source: §5.3.2 · p. 8

5.4. Pruning Score Comparison

  • The Speedy-Splat pruning score outperforms PUP 3D-GS on PSNR and is competitive on the other metrics in the post-hoc pruning pipeline.Table 4 compares the scores across scenes using the PUP 3D-GS pipeline.Source: §5.4 · p. 8, Table 4 · p. 8

6. Limitations

Speedy-Splat produces slightly lower image quality than 3D-GS; the authors note this degradation is expected at high compression ratios and is also observed in comparable techniques.

6. Limitations

  • Speedy-Splat produces slightly lower image quality than 3D-GS, a degradation the authors associate with high compression ratios and also observe in comparable techniques.They also report a slight but noticeable performance gap between their efficient pruning score and PUP 3D-GS’s score.Source: §6 · p. 8

7. Conclusion

Speedy-Splat combines precise Gaussian localization during rendering with Soft and Hard Pruning to reduce model size. Across Mip-NeRF 360, Tanks & Temples, and Deep Blending scenes, it averages 6.71× faster rendering, 10.6× smaller model size, and 1.47× faster training.

7. Conclusion

  • Across all evaluated scenes, Speedy-Splat averages 6.71× rendering speed-up, 10.6× model-size reduction, and 1.47× training speed-up.The approach combines SnugBox and AccuTile localization with Soft and Hard Pruning.Source: §7 · p. 8

A.1. AccuTile Proof of Correctness Sketch

AccuTile’s correctness sketch considers tile rows and relies on ellipse symmetry to extend the reasoning to columns. It selects row extents from bounding-box extrema or boundary intersections across the cases where the ellipse meets a tile row.

A.1. AccuTile Proof of Correctness Sketch

  • AccuTile’s row-based proof also covers tile columns by symmetry, exchanging x and y and coefficients a and c.The sketch focuses on rows because the column argument is equivalent.Source: §A.1 · p. 11
  • When an ellipse lies entirely within a tile row, AccuTile selects SnugBox’s xmin and xmax extrema as the row extents.Figure 4 illustrates this case.Source: §A.1 · p. 11, Figure 4 · p. 11
  • When the ellipse crosses one row boundary, AccuTile uses in-row extrema or boundary intersections as the row’s minimum and maximum extents.The proof distinguishes cases based on whether xmin, xmax, ymin, or ymax lies in the row.Source: §A.1 · p. 11, Figure 5 · p. 11, Figure 6 · p. 11
  • When the ellipse crosses both row boundaries, in-row xmin or xmax extrema define the corresponding extent; otherwise, monotonicity makes boundary points the extrema.Figure 6 illustrates the cases where an ellipse intersects both boundaries.Source: §A.1 · p. 11, Figure 6 · p. 11

A.2. Overall Pruning Percent Metrics

The pruning sweep selected a favorable balance between speed and quality, and Table 6 reports average pruning-method metrics across Deep Blending scenes.

A.2. Overall Pruning Percent Metrics

  • (80%, 30%) pruning was selected as a favorable speed-quality balance after sweeping hard pruning from 0%–40% and soft pruning from 0% and 50%–95% in 5% intervals.The sweep excluded SnugBox and AccuTile to assess pruning in isolation.Source: §A.2 · p. 12
  • Each pruning experiment was run three times per scene, with metrics averaged across runs.The evaluated scenes are listed in Section 5.1.Source: §A.2 · p. 12
  • Table 6 reports average metrics for each pruning method across Deep Blending scenes.Source: Table 6 · p. 12

A.3. Additional Datasets Evaluation

The additional evaluation reports pruning metrics on Tanks & Temples and Deep Blending, compares AccuTile with Tile-Based Culling, and provides per-scene metrics after cumulative pipeline changes.

A.3. Additional Datasets Evaluation

  • Tables 5 and 6 report compression, rendering-speed, training-speed, and image-quality metrics for pruning methods on Tanks & Temples and Deep Blending.FPS and training speed-ups are relative to baseline 3D-GS.Source: §A.3 · p. 12

A.4. StopThePop Tile-Based Culling Ablation

  • AccuTile outperforms Tile-Based Culling on all scenes, with a notable overall-runtime margin in Table 7.Source: Table 8 · p. 13, Table 7 · p. 13
  • AccuTile outperforms Tile-Based Culling on Preprocess and Duplicate with Keys; apparent downstream advantages for Tile-Based Culling stem from under-counted mappings and disappear with padded alpha thresholds.Source: §A.4 · p. 13

A.5. Per-Scene Metrics

Loading 2412.00578v3…