Source-linked AI summary
CompGS: Smaller and Faster Gaussian Splatting with Vector Quantization
KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani, Hamed Pirsiavash
TL;DR
3DGS is fast but storage-intensive because it stores many Gaussian parameters. CompGS combines quantization, compressed assignments, and opacity regularization to reduce this overhead. It achieves large storage and rendering-time reductions with small reported quality changes, although PSNR variability across repeated runs can be comparable to differences between methods.
Problem
3DGS requires at least an order of magnitude more parameters than NeRF variants, limiting storage and memory-constrained applications.
Method
CompGS uses K-means vector quantization with codebooks and assignments, compresses assignments by sorting and run-length-style encoding, and removes low-opacity Gaussians.
Results
CompGS achieves comparable performance to non-quantized 3DGS while producing models 65× and 54× smaller on MipNerf-360 and TandT, respectively.
Takeaways & Limitations
CompGS makes 3DGS more practical by reducing its storage disadvantage while retaining fast rendering and comparable image quality.
Takeaways & Limitations
Repeated experiments show PSNR variability that can be comparable to differences in performance between methods.
Abstract
from arXiv · showhide
3D Gaussian Splatting (3DGS) is a new method for modeling and rendering 3D radiance fields that achieves much faster learning and rendering time compared to SOTA NeRF methods. However, it comes with a drawback in the much larger storage demand compared to NeRF methods since it needs to store the parameters for several 3D Gaussians. We notice that many Gaussians may share similar parameters, so we introduce a simple vector quantization method based on K-means to quantize the Gaussian parameters while optimizing them. Then, we store the small codebook along with the index of the code for each Gaussian. We compress the indices further by sorting them and using a method similar to run-length encoding. Moreover, we use a simple regularizer to encourage zero opacity (invisible Gaussians) to reduce the storage and rendering time by a large factor through reducing the number of Gaussians. We do extensive experiments on standard benchmarks as well as an existing 3D dataset that is an order of magnitude larger than the standard benchmarks used in this field. We show that our simple yet effective method can reduce the storage cost for 3DGS by 40 to 50x and rendering time by 2 to 3x with a very small drop in the quality of rendered images.
1 Introduction
3DGS enables fast, real-time radiance-field rendering but requires substantially more storage than NeRF methods. CompGS addresses this trade-off by quantizing Gaussian parameters and reducing redundant Gaussians while retaining speed and image quality.
- 1 Introduction: CompGS-32K achieves around 45× compression and 2.5× inference speedup with little performance loss, while Ours-BitQ reaches 65× compression with hardly noticeable quality differences.These comparisons are made against 3DGS in the speed-versus-memory analysis.
- 1 Introduction: NeRF methods are slow to train and render, whereas 3DGS achieves real-time rendering by projecting and rasterizing explicit 3D Gaussians.3DGS also stores scene structure explicitly in Gaussian parameters.
- 1 Introduction: 3DGS requires at least an order of magnitude more parameters than NeRF variants, increasing storage, communication, and inference-memory requirements.This can limit deployment on smaller devices and AR/VR headsets.
- 1 Introduction: CompGS vector-quantizes Gaussian parameters during learning and stores a codebook with each Gaussian’s code index.The index can act as a pointer to shared parameters, reducing replicated memory at rendering time.
- 1 Introduction: K-means quantization uses the quantized model in the forward pass while updating the non-quantized model in the backward pass.Assignments are updated less frequently to reduce K-means computation overhead.
- 1 Introduction: Opacity regularization promotes invisible Gaussians, reducing the number of stored and rendered Gaussians.Together with quantization and index compression, the final model is 40× to 50× smaller and 2× to 3× faster to render than 3DGS.
2 Related Work
Prior novel-view synthesis methods improved image quality or efficiency but retained important limitations in speed, representation, or storage. CompGS applies vector quantization to explicit Gaussian representations rather than NeRF-style grids or neural networks, while also pruning low-opacity Gaussians.
- 2 Related Work: NeRF improved novel-view synthesis quality but remained constrained by large neural backbones, empty-space representation, image quality, and rendering-speed challenges.3DGS provides superior quality and faster rendering without implicit learning, but has higher storage requirements.
- 2 Related Work: Bit quantization reduces neural-network memory footprints by representing parameters with fewer bits, including 8-bit and binary representations.CompGS uses vector quantization for Gaussian parameters and separately incorporates bit quantization as a compression strategy.
- 2 Related Work: CompGS differs from prior vector-quantization approaches by compressing collections of 3D Gaussians rather than grids or neural networks.It additionally reduces storage through opacity-based regularization and Gaussian pruning.
3 Method
CompGS represents 3DGS parameters with codebooks and integer assignments learned through quantization-aware training, then further compresses assignments and removes nearly transparent Gaussians. The method preserves the rendering pipeline while reducing stored parameter redundancy and model size.
- 3 Method: 3DGS represents scenes with Gaussians parameterized by position, covariance, opacity, and view-dependent color modeled with spherical harmonics.Covariance is factorized into scale and rotation matrices for optimization.
- 3 Method: Rendering projects and sorts Gaussian splats, then uses differentiable alpha blending and tile-based rasterization to produce pixels.Training minimizes image-space L1 and SSIM losses while pruning low-opacity Gaussians and adding high-gradient ones.
- 3 Method: K-means clusters N Gaussian parameter vectors into K codebook vectors and stores the codebook plus N integer assignments, with N typically much larger than K.Typical scenes contain a few million Gaussians and a few thousand clusters.
- 3 Method: Quantization-aware training renders with centroid-replaced parameters in the forward pass while retaining non-quantized parameters for backward updates.This avoids the performance degradation caused by clustering only after training.
- 3 Method: CompGS uses separate codebooks for DC color, spherical harmonics, scale, and rotation, while leaving position and opacity unquantized.Separate codebooks avoid a huge joint codebook when parameter types are weakly correlated.
- 3 Method: Sorting Gaussians by a quantized index groups equal codes, allowing run-length-style storage of code counts instead of one index per Gaussian.This reduces the representation from n integers to k integers for that index.
- 3 Method: Opacity regularization encourages transparent Gaussians, which are then removed below a threshold to reduce storage and inference time.The reduction in Gaussian count provides compression beyond parameter quantization.
4 Experiments
Experiments show that CompGS substantially compresses 3DGS while largely preserving its reconstruction quality and rendering advantages across standard and larger-scale benchmarks. Ablations attribute these gains to vector quantization, opacity regularization, and compact storage of Gaussian parameters and indices.
- Benchmark comparisons: 65× and 54× smaller models than 3DGS are reported on MipNerf-360 and Tanks&Temples, respectively, with only small quality changes.CompGS also increases rendering FPS by 2× to 3× while retaining low inference memory usage and training time.
- Parameter compression: K-Means vector quantization outperforms 3DGS-No-SH while using less than half its memory, and quantized harmonics retain nearly the original performance.The study also finds that combining correlated parameters into a larger codebook marginally reduces both memory and performance.
- Large-scale datasets: CompGS achieves nearly the same performance as 3DGS with ten times smaller memory on ARKit-200 and nearly 30× compression with a small quality drop on DL3DV-10K.DL3DV-10K results cover a 140-scene novel-view synthesis benchmark.
- Memory analysis: 68% and 81% of CompGS memory at 16- and 32-bit precision comes from non-quantized position and opacity parameters, while codebooks use less than 2% of quantized-parameter memory.CompGS quantizes 55 of each Gaussian’s 59 parameters; most quantized-parameter storage is assigned indices.
Appendix
The appendix reports comparisons with state-of-the-art novel-view synthesis methods, examines codebook generalization across scenes, and analyzes codebook assignments.
- CompGS is compared with state-of-the-art approaches on NeRF-Synthetic, while additional appendix sections study shared codebooks and code assignments.
A Results on NeRF-Synthetic dataset
On NeRF-Synthetic, CompGS improves PSNR over 3DGS-No-SH while using less than half its memory; the appendix also reports a higher 3DGS result than the original paper.
- 1.13 points higher PSNR than 3DGS-No-SH is achieved by CompGS while using less than half its memory.
- 0.5 points higher performance is observed for the authors’ 3DGS runs than the originally reported 3DGS numbers.
B Generalization of codebook across scenes
A codebook trained on one Mip-NeRF360 scene generalizes across the remaining scenes when only assignments are learned, with a small performance drop and no apparent qualitative difference.
- A codebook trained on the ‘Counter’ scene is frozen while assignments are learned for the other eight Mip-NeRF360 scenes.
- The shared codebook generalizes across scenes with a small performance drop relative to learning a separate codebook for each scene.
- Sharing the codebook can further reduce memory requirements and help speed CompGS training.
- CompGS and CompGS-Shared-Codebook show no apparent qualitative differences.
C Analysis of learnt code assignments
Code assignments are highly non-uniform: a few clusters dominate, including one of 512 codes assigned to nearly 5% of Gaussians for SH and DC parameters.
- Nearly 5% of Gaussians use a single code among 512 for both SH and DC parameters.
- A few clusters also dominate rotation and scale assignments, though less strongly than for SH and DC.
- The non-uniform cluster distribution suggests that Huffman coding could further compress assignment indices.
D Scene-wise Metrics
Table D.3 supplements the main submission’s averaged dataset metrics with detailed scene-wise metrics for three datasets.
- Table D.3 reports detailed scene-wise metrics for MipNerf-360, Tanks and Temples, and DeepBlending.
- The main submission reports averaged metrics over all scenes within each dataset.
- The detailed metrics are provided to complement the dataset-level averages.
E Ablations for Gaussian count reduction
Ablations compare baseline strategies for reducing the Gaussian count, identifying gradient-threshold modification as the best size–performance trade-off.
- Gradient-threshold modification provides the best trade-off between model size and performance among the evaluated baselines.
- The ablations vary minimum opacity, densification interval, densification end iteration, and gradient threshold.
- The selected hyperparameter settings are reported in the main paper’s table 3.
F Qualitative comparison on ARKit-200 dataset.
The qualitative material covers ARKit-200, Synthetic-NeRF, and Mip-NeRF360 comparisons, while the dataset includes challenging indoor video scenes with motion blur. Across these evaluations, CompGS is compared with uncompressed 3DGS and 3DGS-No-SH, with scene-wise results reporting compression and rendering speed without much quality loss.
- Scene-wise metrics: Scene-wise metrics report that CompGS 32K achieves high compression and fast rendering without much loss in rendering quality.
- Synthetic-NeRF comparison: On Synthetic-NeRF, 3DGS-No-SH can poorly model branch colors, shadows, and bright leaf lighting in the Ficus scene.
- Shared codebook: A codebook trained on Counter and frozen across Mip-NeRF360 scenes yields visually similar CompGS and CompGS-Shared-Codebook results to uncompressed 3DGS.
- ARKit-200 dataset: ARKit-200 contains indoor videos from multiple viewpoints and introduces motion blur as a distinctive challenge.
- ARKit-200 dataset: ARKit-200 visualizations compare CompGS with uncompressed 3DGS and 3DGS-No-SH, whose image quality is lower than on standard benchmarks.
- ARKit-200 dataset: 3DGS-No-SH commonly produces large noisy blobs on ARKit-200 and can fail to reproduce scene colors and lighting faithfully.