Source-linked AI summary
Reducing the Memory Footprint of 3D Gaussian Splatting
Panagiotis Papantonakis, Georgios Kopanas, Bernhard Kerbl, Alexandre Lanvin, George Drettakis
TL;DR
3DGS provides fast, high-quality, real-time novel-view synthesis but has excessive memory requirements for storage and transmission. The paper combines resolution-aware primitive pruning, adaptive spherical-harmonics allocation, and codebook-based quantization with half-floats. It reports a 27× memory reduction, a 1.7× rendering-speed increase, and 20–30× faster mobile downloads.
Problem
3DGS achieves fast, high-quality, real-time rendering, but its memory footprint is problematic for storage, processing, and mobile streaming.
Method
The method reduces primitive redundancy, adaptively selects spherical-harmonics bands per primitive, and compresses attributes with codebooks and half-float representation.
Results
27× reduction in memory is reported with a 1.7× rendering-speed increase, while mobile download times are reduced 20–30×.
Takeaways & Limitations
The method preserves high visual quality while improving the storage, rendering, and streaming profile of 3DGS.
Takeaways & Limitations
Further reducing primitives and avoiding initial over-densification remain difficult future-work problems.
Abstract
from arXiv · showhide
3D Gaussian splatting provides excellent visual quality for novel view synthesis, with fast training and real-time rendering; unfortunately, the memory requirements of this method for storing and transmission are unreasonably high. We first analyze the reasons for this, identifying three main areas where storage can be reduced: the number of 3D Gaussian primitives used to represent a scene, the number of coefficients for the spherical harmonics used to represent directional radiance, and the precision required to store Gaussian primitive attributes. We present a solution to each of these issues. First, we propose an efficient, resolution-aware primitive pruning approach, reducing the primitive count by half. Second, we introduce an adaptive adjustment method to choose the number of coefficients used to represent directional radiance for each Gaussian primitive, and finally a codebook-based quantization method, together with a half-float representation for further memory reduction. Taken together, these three components result in a 27 reduction in overall size on disk on the standard datasets we tested, along with a 1.7 speedup in rendering speed. We demonstrate our method on standard datasets and show how our solution results in significantly reduced download times when using the method on a mobile device.
1 INTRODUCTION
The paper targets 3DGS's excessive memory footprint while preserving its speed and visual quality. It reduces storage through primitive pruning, adaptive spherical-harmonics allocation, and quantization, with reported gains for rendering and mobile downloads.
- 3DGS combines fast training, high visual quality, and real-time rendering, but its representation has problematic memory requirements for storage, processing, and mobile streaming.
- The paper identifies three storage-reduction targets: primitive count, spherical-harmonics bands, and attribute precision.
- A resolution-aware pruning method eliminates approximately 60% of baseline primitives, while adaptive SH adjustment removes bands unnecessary for view-dependent appearance.
- Codebook quantization and half-float representation further reduce the stored representation size.
- 27× reduction in representation size is reported alongside a 1.7× increase in rendering speed and 20–30× faster mobile downloads.
2 RELATED WORK
Prior novel-view-synthesis methods trade off quality, speed, and memory in different ways. The related work spans image-based rendering, NeRFs, voxel and sparse-grid methods, and emerging 3DGS compression approaches.
- Image-Based Rendering: Image-based rendering reprojects input-view colors through proxy geometry, but storing all input images on the GPU scales poorly as scene complexity and view count increase.
- Neural Radiance Fields: NeRFs use compact MLP-based volumetric representations with high visual quality, but original training can take days and rendering can require minutes per frame.
- Voxel-Based Methods: Voxel-based methods improve rendering and training speed using grids and shallow MLPs, while sparse structures and hash-based compression address voxel grids' cubic memory complexity.
- Trade-offs: Across these approaches, quality, speed, and memory remain significantly traded off, especially for large real-world scenes such as MipNeRF360.
- 3D Gaussian Splatting: 3DGS is inherently sparse and amenable to rasterization, while concurrent compression methods either target a single compression strategy or omit redundant-Gaussian culling.
3 ANALYZING 3DGS MEMORY USAGE
The analysis attributes 3DGS's memory use to dense primitive sets, uniformly allocated spherical-harmonics bands, and over-precise attributes. It motivates resolution-aware pruning, adaptive SH allocation, and codebook compression.
- 3DGS Representation: 3DGS represents scenes with Gaussian primitives carrying opacity, covariance, and spherical harmonics, initialized from a camera-calibration point cloud and densified during optimization.
- Primitive Density: The original densification process can create unnecessarily dense primitive sets, whose redundancy depends strongly on the scale and resolution of observed details.
- Attribute Memory: With three SH bands, each primitive requires 59 floats, including 45 floats, or 76%, for view-dependent effects.
- Spherical Harmonics: Most scene regions are diffuse and can be modeled with base RGB color, motivating variable SH-band allocation only where view-dependent appearance is required.
- Attribute Precision: Opacity, scale, rotation, and SH coefficients tolerate lower dynamic range and accuracy, motivating clustering-based codebook compression while preserving higher accuracy for positions.
4 MEMORY REDUCTION FOR 3DGS
The method reduces 3DGS memory through resolution-aware primitive pruning, adaptive per-primitive SH-band assignment, and post-training quantization. Together, these stages achieve approximately 96.3% memory reduction, or almost 27×, with limited visual-quality loss.
- Overview: The pipeline combines pruning during optimization, adaptive SH assignment at 15K iterations, and codebook quantization afterward.Quantization applies to remaining values except primitive positions.
- 4.1 Scale- and Resolution-aware Redundant Primitive Removal: The redundancy score uses camera-dependent pixel footprints to count overlapping Gaussians at the smallest observable detail scale.A k-NN search and approximate ellipsoid-point intersection test make the computation practical for millions of Gaussians.
- 4.1 Scale- and Resolution-aware Redundant Primitive Removal: The pruning criterion is conservative across views: it preserves information for high-resolution observations while avoiding degradation at smaller resolutions.The method uses the closest visible view to determine the smallest pixel footprint around each primitive.
- 4.1 Scale- and Resolution-aware Redundant Primitive Removal: 60% of primitives are culled by combining redundancy-based removal with low-opacity culling, with minimal effect on visual quality.The combined strategy removes 3% of the lowest-opacity primitives each time, subject to a maximum opacity threshold of 0.05.
- 4.2 Adaptive Adjustment of Spherical Harmonics Bands: Adaptive SH selection evaluates color changes from dropping higher bands and retains the lowest band count whose average transmittance-weighted distance is below 0.04.In the main model, 89% of points use 0 bands, 0.1% use 1, 2.7% use 2, and 8.2% use 3 bands.
- 4.3 Quantization of the Final Representation: 87% memory reduction follows primitive reduction and adaptive SH adjustment, while half-float quantization raises the average total reduction to 96.3%, or almost 27×.The 16-bit representation applies to remaining uncompressed floating-point values, including positions and codebook entries, without significantly affecting quality.
5 IMPLEMENTATION
The implementation extends the original open-source 3DGS system with modified neighbor search, continued opacity culling, variable-band storage, and WebGL support for compressed representations.
- Implementation: The method is implemented on top of the original open-source 3DGS implementation.The authors state that they will release their implementation upon acceptance.
- Implementation: The modified CUDA k-NN routine identifies nearest-neighbor primitive IDs, while low-opacity Gaussians continue to be culled after 15K iterations.Gaussians with opacity below 1/255 are neither rendered nor optimized and therefore contribute no image quality.
- Implementation: Four primitive sets store the variable numbers of SH bands—0, 1, 2, or 3—with minimal changes to the file format and parsing.Each set corresponds to one allowed SH-band count.
- Implementation: Pruning and adaptive SH assignment reduce rendering memory after peak training usage, while rendering benefits from fewer Gaussians and fewer dynamic SH coefficients.Fetching a variable number of SH coefficients requires only minor modifications to the rendering process.
- Implementation: A WebGL implementation incorporates the modified representation to evaluate SH coefficients and support streaming compressed 3DGS models.The implementation targets real-world network delivery of 3DGS representations.
6 RESULTS AND EVALUATION
The evaluation shows that progressive primitive reduction, adaptive spherical harmonics, and quantization substantially reduce 3DGS storage while preserving visual quality and improving rendering or download speed.
- 27× average size reduction is achieved across the tested datasets, with each major component contributing approximately a 3× reduction.The resulting memory footprint is 32%–52% of the original method, while average PSNR changes range from −0.32 to +0.16 dB.
- Table 1 progressively evaluates primitive reduction, adaptive SH adjustment, and quantization across SSIM, PSNR, LPIPS, total memory size, and memory reduction.
- The full method produces virtually imperceptible visual degradation, particularly on smaller displays.
- 120 sec to 5 sec is the measured mobile download reduction, while rendering increases from 16 FPS to 45 FPS.This corresponds to a ×24 download-time speedup in the WebGL phone application.
- The comparison covers original 3DGS, MeRF, 3DGS*, the full solution, and low- and high-compression variants across three representative datasets.
7 CONCLUSION
The paper concludes that its combined pruning, adaptive SH, and quantization approach substantially reduces 3DGS memory while increasing rendering speed and preserving high visual quality. It also identifies avoiding initial over-densification as an important direction for further reduction.
- The method combines resolution-aware primitive reduction, adaptive SH-band selection, and codebook-based quantization.Primitive reduction cuts the number of primitives by half.
- ×27 memory reduction and ×1.7 rendering-speed increase are reported for the proposed method.
- 20–30 times faster downloads and approximately 3 times faster rendering are reported for the WebGL streaming setup.The paper presents this as a high-quality streaming/mobile 3DGS solution.
- Further reducing primitives and avoiding over-densification remain future-work directions.Initial tests indicate that avoiding over-densification is difficult; data-driven priors are suggested as one possible direction.
APPENDIX
The appendix compares the method with concurrent preprints using consistent datasets and reports per-scene results across three standard benchmarks. The low-compression variant offers a favorable file size/quality tradeoff while retaining image quality close to 3DGS.
- Concurrent-method comparisons: 26MB versus 54MB: the full method's relevant average file size is lower than Compact3D's reported three-dataset average.The low-compression variant is 41MB for the same comparison.
- Concurrent-method comparisons: Tables 5 and 6 compare against unpublished concurrent methods using the full Mip-NeRF360 scene set and a version excluding two licensed scenes.The excluded scenes are treehill and flowers.
- Concurrent-method comparisons: The low-compression variant achieves smaller files than the most compact competitors while maintaining image quality closest to 3DGS across datasets.For Deep Blending, some competitors improve metrics through more invasive regularization, while this variant remains competitive with less storage.
- Per-scene results: Per-scene results are provided for the Mip-NeRF360, Tanks&Temples, and DeepBlending datasets.These results are reported in Tables 7–9.