Source-linked AI summary
EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS
Sharath Girish, Kamal Gupta, Abhinav Shrivastava
TL;DR
3D-GS offers efficient rendering but requires substantial memory because high-quality scenes use millions of Gaussians. EAGLES combines attribute quantization, coarse-to-fine training, and influence pruning, achieving 10-20× lower storage requirements with faster training and inference while maintaining comparable reconstruction quality.
Problem
High-quality 3D-GS scenes require millions of Gaussians, creating substantial storage and runtime memory demands.
Method
EAGLES combines latent quantization, opacity quantization, progressive coarse-to-fine training, and pruning of redundant Gaussians.
Results
10-20× lower storage requirements are achieved with lower training cost, faster inference, and reconstruction quality on par with 3D-GS.
Takeaways & Limitations
EAGLES provides a more memory-efficient 3D Gaussian representation while preserving reconstruction quality and improving speed.
Abstract
from arXiv · showhide
Recently, 3D Gaussian splatting (3D-GS) has gained popularity in novel-view scene synthesis. It addresses the challenges of lengthy training times and slow rendering speeds associated with Neural Radiance Fields (NeRFs). Through rapid, differentiable rasterization of 3D Gaussians, 3D-GS achieves real-time rendering and accelerated training. They, however, demand substantial memory resources for both training and storage, as they require millions of Gaussians in their point cloud representation for each scene. We present a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements and a coarse-to-fine training strategy for a faster and more stable optimization of the Gaussian point clouds. Our approach develops a pruning stage which results in scene representations with fewer Gaussians, leading to faster training times and rendering speeds for real-time rendering of high resolution scenes. We reduce storage memory by more than an order of magnitude all while preserving the reconstruction quality. We validate the effectiveness of our approach on a variety of datasets and scenes preserving the visual quality while consuming 10-20x lesser memory and faster training/inference speed. Project page and code is available https://efficientgaussian.github.io
1 Introduction
EAGLES targets the high storage and runtime memory demands of 3D-GS while preserving novel-view quality. It combines quantization, progressive training, and pruning to improve efficiency.
- Motivation: 3D-GS can require over 1GB of storage and almost 20GB of GPU RAM for high-resolution scenes.These requirements limit practicality in memory-constrained or low-bandwidth systems.
- Approach: EAGLES compresses per-point attributes through quantization to reduce storage and runtime memory costs.The approach focuses especially on color and rotation attributes, which account for more than 80% of attribute memory.
- Approach: Opacity quantization improves Gaussian optimization, while progressive training improves stability, convergence speed, and reconstruction quality.The progressive strategy trains with increasing rendering resolution.
- Approach: A pruning stage removes redundant or low-influence Gaussians, reducing representation size and improving rendering and training speed.The stage addresses redundancy introduced by densification through cloning and splitting.
- Results: EAGLES achieves comparable quality to 3D-GS across varied datasets while being faster and more memory-efficient.The paper evaluates ablations and multiple datasets to assess the efficiency components.
2 Related Work
Prior work improves the efficiency of neural fields and their compression, but EAGLES focuses on compressing individual 3D Gaussian scene representations. It reports smaller memory use and higher inference FPS while retaining reconstruction quality.
- Neural-field efficiency: Neural fields are used broadly for representing 3D objects, audio, images, and videos, motivating work on faster and more efficient representations.Neural-network compression techniques have influenced this line of research.
- Compression methods: Neural-network compression commonly quantizes parameters to lower precision and may apply entropy coding or other lossless compression.Some methods instead train binary or finite-precision networks directly.
- Compression methods: Neural-field compression methods include meta-learning and vector quantization for hierarchical feature grids.These approaches target compressed neural-field representations rather than the single-scene Gaussian representation addressed here.
- EAGLES: EAGLES compresses 3D Gaussian point-cloud representations while retaining high reconstruction quality with smaller memory and higher inference FPS.The paper presents this as a first demonstration for this representation.
3 Background
3D-GS represents scenes with anisotropic 3D Gaussians, projects them into 2D, and blends them differentiably to render views. Training optimizes the Gaussian attributes using gradient-based reconstruction losses and controls point growth through densification and removal.
- 3D Gaussian representation: Each 3D Gaussian stores position, covariance-related scale and rotation, opacity, and color attributes.The covariance is parameterized by a scale matrix and rotation matrix.
- Differentiable rendering: Rendering projects 3D Gaussians onto a 2D plane and blends overlapping points with a differentiable tile-based rasterizer.Pixel colors are formed from depth-sorted Gaussian contributions.
- Differentiable rendering: The projected covariance uses the camera transform and the Jacobian of its affine approximation.The projection maps the 3D covariance into a 2D covariance for rasterization.
- Optimization: Gaussians are initialized from SfM point clouds and optimized with stochastic gradient descent using a combined L1 and SSIM reconstruction loss.The rendering process is fully differentiable.
- Point control: Densification clones or splits Gaussians based on opacity, scale, and positional-gradient criteria, while low-opacity Gaussians are removed.These operations occur after a warmup phase and at regular iteration intervals.
4 Method
EAGLES reduces Gaussian-scene cost through latent attribute quantization, coarse-to-fine training, and influence-based pruning. These components target memory, optimization stability, artifact reduction, and redundant point removal.
- Attribute quantization: A million full-precision Gaussians require about 236 MB, while color coefficients comprise more than 80% of the attribute-vector dimensions.EAGLES therefore targets per-attribute storage reduction using quantized representations.
- Attribute quantization: Quantized latent vectors are decoded by an MLP, with continuous training proxies rounded through a Straight-Through Estimator.After training, rounded latents are entropy-coded together with the decoder.
- Attribute quantization: Color, rotation, and opacity are quantized, while base-band color, scale, and position remain unencoded because quantizing them causes large performance drops.Opacity quantization additionally improves optimization and reduces rendered artifacts.
- Progressive training: Progressive training begins at low rendering resolution and increases it, simplifying optimization and helping remove floating or foggy artifacts.Higher resolutions allow finer scene features to be fitted later in training.
- Influence pruning: Influence pruning removes Gaussians with negligible rasterization contribution, including occluded or very small points, without reducing reconstruction quality.Influence is accumulated from per-pixel alpha-transmittance weights across the scene.
5 Experiments
Experiments across 13 challenging scenes show that EAGLES preserves competitive reconstruction quality while reducing memory use and improving training and rendering efficiency. Ablations attribute these gains to quantization, progressive training, controlled densification, and pruning.
- Implementation and evaluation: The evaluation covers 13 high-resolution scenes from Mip-NeRF360, Tanks&Temples, and Deep Blending, using every eighth view for evaluation.Reconstruction quality is evaluated primarily with PSNR.
- Benchmark comparison: EAGLES performs competitively in reconstruction metrics while outperforming prior work in efficiency metrics across three datasets.Table 1 compares the approach with prior view-synthesis methods; asterisks denote runs on existing codebases.
- Benchmark comparison: Compared with INGP, EAGLES achieves better reconstruction quality at comparable training times on Deep Blending and Tanks&Temples, but requires higher training time on Mip-NeRF360.The Ours-Small configuration is smaller than INGP while retaining better reconstruction metrics, and rendering exceeds INGP by >15× on all datasets.
- Benchmark comparison: ∼15× lower storage and faster training and rendering than 3D-GS are reported on Deep Blending while maintaining good visual quality at 21K iterations.The 21K-iteration point marks the end of progressive scaling and is close to convergence.
- Benchmark comparison: EAGLES produces reconstructions on par with or better than 3D-GS and MiP-NeRF360, while avoiding some boundary artifacts and preserving fine detail better than INGP and Plenoxels.Qualitative examples include smoother scene boundaries and fewer floaters in areas sparsely observed during training.
- Ablations: ∼20−30× memory reduction is achieved for quantized attributes, while opacity quantization improves rendering quality by preventing erroneous Gaussians from saturating quickly.Quantized color, rotation, and opacity attributes shrink from 220 MB, 452 MB, and 1046 MB to 6 MB, 12 MB, and 28 MB across three scenes.
- Ablations: Progressive scaling improves PSNR, reduces floating artifacts, and lowers training time through more stable optimization at gradually increasing resolutions.It uses a cosine schedule that increases rendering resolution during training.
- Ablations: Decreasing the resize scale to 0.3 reduces Gaussian count and iteration time without affecting PSNR, whereas lower scales cause PSNR dropoffs.The analysis averages results over three random seeds on the Truck scene.
6 Conclusion
EAGLES improves 3D reconstruction and novel view synthesis by combining per-point attribute quantization, progressive training, and pruning. The approach reduces storage requirements while lowering training cost and inference time, with reconstruction quality on par with prior work.
- EAGLES combines attribute quantization, progressive training, and pruning for 3D reconstruction and novel view synthesis.These are presented as three major improvements over prior 3D Gaussian Splatting.
- 10-20× lower storage requirements are achieved with lower training cost, faster inference time, and on-par reconstruction quality.
- The method is evaluated through extensive quantitative and qualitative analyses of 3D representation.
Supplementary - EAGLES: Efficient Accelerated 3D
The supplementary section identifies EAGLES as a method for efficient 3D Gaussian representations with lightweight encodings.
- EAGLES is presented as a method for efficient 3D Gaussian representations with lightweight encodings.
1 Hyperparameters
The supplementary implementation compresses Gaussian attributes using latent variables and specifies stable hyperparameter settings for training.
- Color, rotation, and opacity attributes are compressed using a latent quantization framework with several associated hyperparameters.The listed hyperparameters include latent dimension, decoder parameter learning rate, latent learning rate, and decoder initialization.
- For linear decoders, least-squares approximation provides latent values, while latent learning rates are scaled using the original attribute rate and decoder norm.This scaling is intended to improve training stability and convergence when decoder norms are too high or too low.
- The selected compressible-attribute hyperparameters are used across experiments and reported as stable across various datasets.
2 Per scene metrics
The paper reports per-scene metrics across Mip-NeRF360, Tanks&Temples, and Deep Blending, with supplementary tables organizing the corresponding results.
- Per-scene metrics are reported across the Mip-NeRF360, Tanks&Temples, and Deep Blending datasets.
- A latent hyperparameter table is also included alongside the per-scene metrics.
- The supplementary material includes separate per-scene result tables for Mip-NeRF360, Tanks&Temples, and Deep Blending.