Source-linked AI summary
HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression
Yihang Chen, Qianyi Wu, Weiyao Lin, Mehrtash Harandi, Jianfei Cai
TL;DR
3DGS offers fast, high-fidelity novel-view synthesis but is costly to store because its Gaussians are numerous, sparse, and unorganized. HAC compresses the representation by using a binarized hash grid to model anchor context, combined with adaptive quantization and masking. It reports 75× size reduction versus vanilla 3DGS and 11× versus Scaffold-GS, with comparable or improved fidelity.
Problem
3DGS requires substantial storage for large scenes, while its sparse and unorganized Gaussians make structural-relation-based compression difficult.
Method
HAC jointly learns a compact binarized hash grid whose interpolated features predict anchor-attribute distributions for entropy coding, with adaptive quantization and masking.
Results
75× size reduction versus vanilla 3DGS and 11× versus Scaffold-GS are achieved on average across five datasets, with comparable or improved fidelity.
Takeaways & Limitations
HAC demonstrates that mutual information between structured hash grids and unorganized anchors can support compact 3DGS representations.
Takeaways & Limitations
HAC increases training time by approximately 0.9× over Scaffold-GS, its main limitation.
Abstract
from arXiv · showhide
3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis, boasting rapid rendering speed with high fidelity. However, the substantial Gaussians and their associated attributes necessitate effective compression techniques. Nevertheless, the sparse and unorganized nature of the point cloud of Gaussians (or anchors in our paper) presents challenges for compression. To address this, we make use of the relations between the unorganized anchors and the structured hash grid, leveraging their mutual information for context modeling, and propose a Hash-grid Assisted Context (HAC) framework for highly compact 3DGS representation. Our approach introduces a binary hash grid to establish continuous spatial consistencies, allowing us to unveil the inherent spatial relations of anchors through a carefully designed context model. To facilitate entropy coding, we utilize Gaussian distributions to accurately estimate the probability of each quantized attribute, where an adaptive quantization module is proposed to enable high-precision quantization of these attributes for improved fidelity restoration. Additionally, we incorporate an adaptive masking strategy to eliminate invalid Gaussians and anchors. Importantly, our work is the pioneer to explore context-based compression for 3DGS representation, resulting in a remarkable size reduction of over $75\times$ compared to vanilla 3DGS, while simultaneously improving fidelity, and achieving over $11\times$ size reduction over SOTA 3DGS compression approach Scaffold-GS. Our code is available here: https://github.com/YihangChen-ee/HAC
1 Introduction
3DGS enables rapid differentiable rendering but requires substantial storage because large scenes contain millions of sparse, unorganized Gaussians. HAC addresses this compression challenge by using a structured hash grid as context for anchor attributes, with adaptive quantization and masking, achieving large size reductions while preserving or improving fidelity.
- Motivation: Millions of Gaussians in large-scale scenes can require a few gigabytes of storage, motivating effective 3DGS compression.The storage burden arises from both the number of Gaussians and their associated attributes.
- Compression challenge: Sparse and unorganized Gaussians make compression difficult, while prior methods mainly target parameter values and overlook structural relations.Examples include pruning based on parameter thresholds and vector quantization of similar values.
- HAC framework: HAC jointly learns a compact binarized hash grid and uses interpolated hash features to predict anchor-attribute distributions for entropy coding.Scaffold-GS serves as the base model because its anchor-centered design supports relations with interpolated hash features.
- HAC framework: HAC also uses neural prediction of quantization-step refinement and learnable masks to prune ineffective Gaussians and anchors.These components support efficient entropy encoding and direct storage reduction.
- Results: 11× compression over Scaffold-GS and 75× over vanilla 3DGS are achieved on average across five datasets, with comparable or improved fidelity.The reported ratios summarize the framework’s averaged performance over all evaluated datasets.
2 Related Work
Prior compression methods for explicit representations reduce values or exploit structural relations. Because 3DGS Gaussians are sparse and unorganized, existing approaches largely emphasize values, motivating HAC’s structured hash-grid context for anchor relations.
- Compression approaches: Compression methods for explicit representations include value-based pruning, codebooks, quantization, and entropy constraints.Structural-relation approaches include wavelet decomposition, rank-residual decomposition, and spatial prediction.
- 3DGS compression: 3DGS compression mainly targets parameter values because sparse, unorganized Gaussians make structural relations difficult to establish.Existing techniques include pruning, codebooks, and entropy constraints.
- Motivation: Image and video compression show that exploiting spatial or temporal relations can eliminate structural redundancy in well-organized data.This motivates searching for analogous relations in 3DGS representations.
- HAC positioning: HAC introduces a structured hash grid as context for modeling consistencies among sparse, unorganized anchors, using Scaffold-GS as its base model.The approach targets a more compact 3DGS representation through structural relations.
3 Methods
HAC compresses Scaffold-GS by using interpolated binary hash-grid features as context for anchor-attribute quantization and entropy modeling, while masking redundant elements. The framework preserves the original anchor-based rendering structure and targets compactness without sacrificing fidelity.
- Hash-grid context: HAC jointly learns a compact binarized hash grid and queries interpolated features at anchor locations to expose spatial relations among unorganized anchors.The interpolated hash feature is used as context rather than directly replacing anchor features.
- Context modeling and entropy coding: The context model maps interpolated hash features to adaptive quantization refinements and Gaussian parameters for estimating quantized anchor-attribute probabilities during arithmetic coding.The adaptive quantization module produces finite-valued attributes, while predicted µ and σ model their distributions.
- Design rationale: Directly substituting hash features for anchor features lowers fidelity, destabilizes training through anchor spawning effects, and reduces testing FPS because of interpolation overhead.The paper therefore uses hash features only to estimate entropy rather than modifying the original Scaffold-GS representation.
- Adaptive quantization: Adaptive quantization uses learned step sizes because unit-step rounding is unsuitable for decimal-valued scaling and offset attributes.The module refines a predefined step size separately for feature, scaling, and offset components.
- Adaptive masking: An adaptive offset-masking module prunes redundant Gaussians and anchors, motivated by the impulse at zero observed in offset distributions.The masking loss regularizes the binary masks used to identify ineffective offsets.
4 Experiments
Experiments evaluate HAC against 3DGS compression baselines, analyze its components and spatial bit allocation, and measure training, coding, and inference costs.
- 4.2 Experiment Evaluation: HAC offers two size–fidelity tradeoffs by adjusting λe, with smaller λe producing larger models and improved fidelity.Table 1 measures size in MB and compares HAC with 3DGS and Scaffold-GS baselines.
- 4.2 Experiment Evaluation: HAC is evaluated against 3DGS compression approaches across Synthetic-NeRF, BungeeNeRF, DeepBlending, Mip-NeRF360, and Tanks&Temples.The evaluation uses relative rate changes under similar fidelity because BD-rate is incalculable when competing methods provide only one rate.
- 4.3 Ablation Study: Ablations show that removing hash-grid mutual information increases bit consumption, disabling AQM reduces fidelity, and masking saves rates in simpler or lower-rate scenes.These effects are assessed on BungeeNeRF and Synthetic-NeRF across variable-rate settings.
- 4.4 Visualization of Bit Allocation: Bit allocation concentrates more total bits in complex or sharp regions, while average bits per anchor remain smoother when those regions contain more anchors.The visualization voxelizes 3D space and encodes anchor count through ball radius, with colors representing total or average bit consumption.
- 4.5 Training and Execution Time: HAC training is approximately 0.9× longer than Scaffold-GS, while decoding removes the hash grid and preserves similar rendering speed.On BungeeNeRF, training takes 27.6 minutes for HAC versus 15.1 minutes for Scaffold-GS; HAC rendering reaches 283 FPS versus 232 FPS.
5 Conclusion
HAC uses mutual information between sparse anchors and structured hash grids to compress 3DGS representations and address their storage demands.
- 5 Conclusion: HAC leverages mutual information between unorganized anchors and structured hash grids for compact 3DGS representations.The conclusion describes this relationship as the basis of the framework’s compression performance.
–Supplementary Material–
The supplementary material documents the paper’s additional resources and gives a detailed training-process overview for HAC alongside Scaffold-GS.
- The supplementary material includes implementation details, an extra experiment, per-scene quantitative results, and a comprehensive notation table.
- Figure A contrasts HAC-related training steps, shown in blue, with Scaffold-GS training steps, shown in pink.
A More Implementation Details
Training progressively introduces noise, quantization-aware transitions, and the binary hash grid, while entropy training samples a subset of anchors for efficiency.
- During the initial 3000 iterations, HAC adds no techniques beyond Scaffold-GS training to stabilize anchor-attribute training and anchor spawning.
- From iterations 3000 to 10000, noise is added to anchor attributes so the model adapts to quantization.
- Table A reports bit allocation among the anchor attributes using λe = 4e −3 and counting only valid, unpruned anchors for per-parameter size.
- After iteration 10000, HAC jointly trains the binary hash grid once the model is considered adequately fitted to quantization.
- The hash-grid bounds are determined from the maximum and minimum anchor locations at iteration 10000.
- Each iteration entropy-trains a random 5% of anchors used for rendering, reducing training time and potential out-of-memory issues while preserving satisfactory rate-distortion performance.
B Additional Experiments
The bit-allocation experiment explains how attribute dimensionality, rendering use, redundancy, precision, and predictability shape per-parameter storage.
- Feature f_a receives the smallest per-parameter bit allocation because its high dimensionality creates substantial redundancy before MLP processing.
- Scaling l and offsets o consume more bits per parameter because they are directly used for rendering and contain higher-precision values that are harder to predict.
- The allocation reflects different compression difficulties across attributes rather than assigning equal precision to the anchor representation.
C.1 Detailed Results of Our HAC Framework
The supplementary material provides detailed per-scene results for HAC across Synthetic-NeRF and multiple large-scale real-scene datasets.
- Detailed per-scene results for Synthetic-NeRF are provided in Table B.
- Detailed per-scene results for Mip-NeRF360, Tanks&Temples, DeepBlending, and BungeeNeRF are provided in Tables C–F.
C.2 Detailed Results of the Base Models
This section provides detailed per-scene results for the paper’s two base models, 3DGS and Scaffold-GS, across all evaluated datasets.
- 3DGS: Per-scene results are reported for the 3DGS base model.The detailed results appear in Table G.
- Scaffold-GS: Per-scene results are also reported for the Scaffold-GS base model.The detailed results appear in Table H.
- All evaluated datasets: Tables G and H together cover all evaluated datasets for both base models.The section directs readers to these tables for scene-level comparisons.
D Notation Table
This section collects supplementary per-scene result tables for the proposed approach and baseline models, alongside the notation table defining hash-grid level notation.
- Per-scene results: Table B reports per-scene results for the Synthetic-NeRF dataset using the proposed approach.Synthetic-NeRF is identified as dataset.
- Per-scene results: Tables C, D, and F report per-scene results for Mip-NeRF360, Tank&Temples, and BungeeNeRF, respectively, using the proposed approach.The tables cover three datasets and identify their corresponding references.
- Base-model results: Tables G and H report per-scene results across all evaluated datasets for 3DGS and Scaffold-GS, respectively.These tables provide detailed results for the two base models.
- Notation: Table I provides detailed notation explanations, including L3d and L2d for the 3D and 2D hash-grid levels.The notation uses slight abuse to represent the number of levels in each hash-grid part.