PaperlayerView exact revision on arXiv

Source-linked AI summary

HAC++: Towards 100X Compression of 3D Gaussian Splatting

Yihang Chen, Qianyi Wu, Weiyao Lin, Mehrtash Harandi, Jianfei Cai

arXiv:2501.12255v4cs.CV

TL;DR

3DGS requires compression despite its high-fidelity, rapid rendering, while sparse, unorganized anchors contain redundancies that existing methods may overlook. HAC++ models relationships between anchors and structured hash grids, captures intra-anchor context, and uses adaptive quantization and masking; it achieves over 100× compression versus vanilla 3DGS and 20× versus Scaffold-GS while maintaining or improving fidelity.

  • Problem

    Existing methods may overlook internal redundancies within anchors, while the sparse, unorganized anchor structure makes compact 3DGS compression challenging.

  • Method

    HAC++ uses mutual information between unorganized anchors and a structured hash grid for context modeling, adds intra-anchor context, and applies adaptive quantization and masking for entropy coding and pruning.

  • Results

    Over 100× compression versus vanilla 3DGS and 20× versus Scaffold-GS are achieved on average across datasets while maintaining or improving fidelity.

  • Takeaways & Limitations

    HAC++ provides a compact and efficient 3DGS representation that the authors state supports deployment in large-scale scenes.

  • Takeaways & Limitations

    HAC++ requires increased training time compared with Scaffold-GS because of an additional loss term and the associated method components.

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis, boasting rapid rendering speed with high fidelity. However, the substantial Gaussians and their associated attributes necessitate effective compression techniques. Nevertheless, the sparse and unorganized nature of the point cloud of Gaussians (or anchors in our paper) presents challenges for compression. To achieve a compact size, we propose HAC++, which leverages the relationships between unorganized anchors and a structured hash grid, utilizing their mutual information for context modeling. Additionally, HAC++ captures intra-anchor contextual relationships to further enhance compression performance. To facilitate entropy coding, we utilize Gaussian distributions to precisely estimate the probability of each quantized attribute, where an adaptive quantization module is proposed to enable high-precision quantization of these attributes for improved fidelity restoration. Moreover, we incorporate an adaptive masking strategy to eliminate invalid Gaussians and anchors. Overall, HAC++ achieves a remarkable size reduction of over 100X compared to vanilla 3DGS when averaged on all datasets, while simultaneously improving fidelity. It also delivers more than 20X size reduction compared to Scaffold-GS. Our code is available at https://github.com/YihangChen-ee/HAC-plus.

I. INTRODUCTION

3DGS offers fast, high-fidelity novel-view synthesis but requires compression because its many sparse, unorganized Gaussians demand substantial storage. HAC++ addresses this by modeling anchor relationships with a structured hash grid and intra-anchor context, achieving over 100× compression versus vanilla 3DGS while maintaining or improving fidelity.

  • Motivation: 3DGS enables rapid, differentiable rendering with high photorealistic fidelity, but city-scale scenes may contain millions of Gaussians requiring a few gigabytes of storage.These storage demands motivate dedicated 3DGS compression techniques.
  • Motivation: Sparse, unorganized Gaussians make structural relationships difficult to exploit, while existing methods primarily reduce parameter values through pruning or vector quantization.Scaffold-GS clusters Gaussians into anchors but treats each anchor independently, leaving further redundancy unmodeled.
  • HAC++: HAC++ uses mutual information between unorganized anchors and a structured hash grid to predict anchor-attribute value distributions for entropy coding.Interpolated hash features are queried at anchor locations and used as context rather than directly replacing anchor features.
  • HAC++: HAC++ combines hash-grid context with intra-anchor context, adaptive quantization, and adaptive masking to improve compression and fidelity restoration.The intra-anchor model captures internal relationships, AQM adjusts quantization, and masks remove invalid Gaussians and anchors.
  • Results: Over 100× compression versus vanilla 3DGS and 20× versus Scaffold-GS is achieved on average across datasets while maintaining or even improving fidelity.The paper also reports comprehensive analyses of HAC++ and its technical components.

II. RELATED WORK

Prior explicit-representation compression methods mainly reduce parameter values, whereas 3DGS’s sparse, unorganized structure limits structural compression. HAC++ extends anchor-based Scaffold-GS with inter-anchor hash-grid context, intra-anchor context, and improved pruning.

  • NeRF and explicit representations: NeRF-based representations use explicit feature grids to improve training and rendering efficiency, motivating grid-based ideas for compression.These methods reduce reliance on large MLPs but can increase the size of explicit representations.
  • Compression methods: Value-based compression uses pruning, codebooks, quantization, and entropy constraints to reduce parameter counts or bit lengths.These techniques target parameter values rather than structural relations.
  • Compression methods: 3DGS Gaussians are sparse and unorganized, making structural relationships harder to establish than in well-structured feature grids.Consequently, many 3DGS compression methods primarily follow value-based strategies.
  • HAC++: HAC++ addresses remaining redundancy by combining inter-anchor and intra-anchor contexts with an improved pruning strategy.Its hash-grid context models anchor relations, while intra-anchor context exploits internal redundancy.
  • Anchor-based representations: Scaffold-GS clusters Gaussians with anchors and predicts Gaussian attributes from anchor attributes instead of storing them directly.Each anchor contains feature, scaling, and offset attributes used by MLPs during rendering.

B. Bridging Anchors and Hash Grid

HAC++ uses interpolated hash-grid features as context for unorganized anchor attributes, exploiting their mutual information to improve entropy modeling without replacing the original anchor representation. Adaptive quantization and Gaussian probability modeling support compact, fidelity-preserving coding.

  • Bridging Anchors and Hash Grid: HAC++ assumes spatial consistency between unorganized anchor attributes and structured hash-grid features, motivated by observed mutual information.The hash feature is obtained by interpolating the grid at each anchor location.
  • Bridging Anchors and Hash Grid: Directly replacing anchor features with hash features reduces fidelity and testing FPS while causing unstable training, so HAC uses them as coding context instead.The preliminary Synthetic-NeRF experiment reports moderate fidelity degradation from substitution, but additional interpolation reduces testing speed.
  • Bridging Anchors and Hash Grid: HAC predicts anchor-attribute distributions from interpolated hash features, maximizing conditional probability to reduce entropy-coded bit consumption.The hash features estimate Gaussian parameters and quantization refinements while leaving the original Scaffold-GS structure unchanged.
  • Bridging Anchors and Hash Grid: AQM predicts attribute-specific quantization refinements, restricting each step size to (0, 2Q0) for higher-fidelity reconstruction of decimal-valued attributes.The predefined step Q0 differs for feature, scaling, and offset attributes.
  • Bridging Anchors and Hash Grid: The anchor components exhibit Gaussian-like value distributions, although scaling is single-sided and offsets show a zero impulse indicating redundant Gaussians.The distribution analysis covers feature, scaling, and offset attributes; scaling values are visualized after multiplication by 100.

D. Improving Context Accuracy with Intra Information

HAC++ adds an intra-anchor context model to capture redundancies within anchor features and combines it with HAC through a Gaussian Mixture Model for more accurate probability estimation.

  • Improving Context Accuracy with Intra Information: The intra-anchor model divides each anchor feature into chunks and predicts each chunk’s distribution from previously decoded chunks in a causal flow.This exploits channel-wise redundancies within the anchor feature.
  • Improving Context Accuracy with Intra Information: The intra-anchor model is applied only to anchor features because scaling and other attributes offer negligible internal redundancy or limited storage savings.Scaling has dimensionality six, reducing the benefit of modeling its internal redundancies.
  • Improving Context Accuracy with Intra Information: The GMM combines Gaussian parameter sets from HAC and intra-anchor contexts, adaptively weighting their contributions for anchor-feature probability estimation.The combined model is intended to improve context accuracy and robustness.

E. Adaptive Offset Masking

HAC++ uses learnable Gaussian- and anchor-level masks to remove invalid primitives and incorporates mask information into differentiable rate estimation for adaptive pruning across rate constraints.

  • Adaptive Offset Masking: Offset distributions with a zero impulse suggest substantial unnecessary Gaussians and anchors that can be removed.The observation motivates the adaptive masking strategy.
  • Adaptive Offset Masking: Learnable binary Gaussian masks remove invalid offsets, and anchors with all offsets pruned are removed together with their locations and attributes.A mask value of one denotes a valid offset, while zero denotes an invalid one.
  • Adaptive Offset Masking: Mask information enters rendering and entropy estimation differentiably, so invalid Gaussians stop contributing while valid ones remain unaffected.Gaussian-level masks modify opacity and scale, while both Gaussian- and anchor-level masks affect bit-cost calculation.
  • Adaptive Offset Masking: Embedding masks directly in the rate calculation eliminates an additional mask-rate loss and dynamically identifies pruning ratios across different rate constraints.Backpropagation adapts the mask ratios as the rate constraint changes.

F. Hash Grid Compression

HAC++ compresses the hash grid by binarizing its entries and estimating their bit consumption from the frequency of the two binary symbols.

  • Hash Grid Compression: The hash table is binarized to {−1, +1} using straight-through estimation before storage-cost estimation.The binary representation reduces the hash-grid coding problem to two symbol frequencies.
  • Hash Grid Compression: The frequency of “+1”, together with the counts of “+1” and “−1”, estimates the hash grid’s bit consumption.The symbol counts are denoted M+ and M−.

G. Training and Coding Process

HAC++ jointly optimizes rendering fidelity and bitrate, then uses separately encoded anchor locations and hash-grid features to entropy-code anchor attributes. The evaluation covers implementation, comparative performance, ablations, and efficiency analyses.

  • Training objective: The training objective combines Scaffold-GS rendering loss with estimated entropy and hash-grid bit consumption, balanced by λ.The estimated bitrate includes anchor-attribute entropy and hash-grid costs.
  • Encoding and decoding: Anchor locations and the binary hash grid are encoded separately before interpolated hash features condition quantization refinement and Gaussian probability estimation.MLPq predicts quantization refinement, while MLPc predicts Gaussian parameters for entropy coding with arithmetic coding.
  • Evaluation scope: The evaluation compares HAC++ with existing 3DGS compression methods and includes ablations of its technical components.The experiments also analyze mask ratios, storage size, coding time, training and inference efficiency, and iteration-dependent performance.

A. Implementation Details

HAC++ is implemented on Scaffold-GS and evaluated across multiple datasets against diverse compression baselines. Ablations show that adaptive quantization, contextual probability modeling, masking, and GPCC location coding support its rate-distortion performance.

  • Implementation Details: HAC++ uses a mixed 3D–2D structured binary hash grid and increases the Scaffold-GS anchor-feature dimension to 50 while disabling its feature bank.The implementation uses 12 three-dimensional and 4 two-dimensional hash-grid levels, with other Scaffold-GS hyperparameters largely unchanged.
  • Implementation Details: The evaluation spans Synthetic-NeRF, BungeeNeRF, DeepBlending, Mip-NeRF360, and Tanks&Temples, comparing HAC++ with pruning, codebook, entropy-constrained, sorting, and context-model baselines.The quantitative table reports size in MB and varies λ to provide different size–fidelity trade-offs.
  • Ablation Study: Removing AQM causes a significant fidelity drop, especially at higher rates, because quantized anchor attributes retain insufficient rendering information.The resulting fidelity has no overlap with HAC++ for BD-rate calculation and is reported as N/A.
  • Ablation Study: Removing HAC or intra-anchor context increases model size, while replacing GMM fusion with concatenation yields suboptimal probability estimation.The GMM provides a more flexible distribution estimate for the conditional attribute distribution.
  • Ablation Study: Gaussian- and anchor-level masking, mask-aware rate calculation, and GPCC location coding each improve compression, with pruning also improving rendering efficiency.An additional mask loss is less effective than directly incorporating mask information into the differentiable rate calculation.

D. Mask Ratio Analysis

HAC++ adaptively changes valid anchor and Gaussian proportions under rate constraints, discarding invalid elements and encoding only valid parameters. Stricter constraints reduce storage, especially for offsets, through fewer valid parameters and lower per-parameter entropy.

  • Mask Ratio Analysis: As λ increases, the valid ratios of anchors and Gaussians decrease, indicating stricter rate constraints and more aggressive masking.Each anchor contains K = 10 Gaussians, and an anchor remains valid when at least one Gaussian is valid.
  • Mask Ratio Analysis: The Gaussian valid ratio is smaller than the anchor valid ratio because anchor validity requires only one valid Gaussian among its K = 10 offsets.The changing anchor ratio also indicates a reduced proportion of valid Gaussians inside valid anchors as λ increases.
  • Mask Ratio Analysis: Only valid anchors and valid offsets are encoded, while invalid anchors are directly discarded.This masking strategy produces a compact representation by removing invalid anchors and offsets within valid anchors.
  • Storage Analysis: Offset storage decreases most significantly, by 3×, as λ increases because both per-parameter bits and the ratio of valid offsets decline.Fewer valid offsets reduce the number of encoded parameters in addition to reducing their bit cost.
  • Storage Analysis: Increasing λ generally lowers per-parameter bits, while GPCC-compressed locations and increasingly zero-dominant masks exhibit distinct entropy trends.More positional redundancy lowers location bits, and fewer valid offsets reduce mask entropy.

F. Decomposition of Coding Time of Difference Components

HAC++ maintains efficient coding despite large scenes, while its training and rendering behavior reflects its Scaffold-GS foundation and contextual compression design. Coding-time analyses separate location, attribute, and model-operation costs.

  • Coding Time: Both encoding and decoding remain within 30 seconds on Mip-NeRF360 despite more than 400K valid anchors on average.Simpler datasets require less coding time because they contain fewer anchors.
  • Coding Time: GPCC makes anchor-location encoding relatively slow because of RD search, whereas arithmetic-code decoding is more complex because it involves index searching.The “Others” category includes model-related operations such as MLP processing.
  • Training and Rendering Efficiency: Scaffold-GS trains faster and renders at higher FPS than 3DGS because pre-filtering skips out-of-view computations and colors are precomputed.HAC++ inherits these Scaffold-GS efficiency features.
  • Training and Rendering Efficiency: HAC++ increases training time by 81% over Scaffold-GS and peak GPU memory by 16% on average, while retaining fast training.Training time remains consistent across rates because masks preserve gradients for valid and invalid elements during training.
  • Training and Rendering Efficiency: HAC++ can remove the hash grid after decoding anchor attributes, avoiding additional inference operations.This efficiency follows from using the hash grid for entropy estimation without modifying the original Scaffold-GS structure.

H. Visualization of Bit Allocation

HAC++ allocates more total bits to spatial regions with complex appearances or sharp edges. Longer training improves performance, primarily through fidelity gains, but increases training time and offers diminishing incremental benefit.

  • H. Visualization of Bit Allocation: HAC++ allocates more total bits to areas with complex appearances or sharp edges, including lego edges and mate specular objects.Bit allocation is visualized with voxelized colored balls on Synthetic-NeRF scenes.
  • H. Visualization of Bit Allocation: A substantial BD-rate improvement of −32.9% is observed when increasing training from 30k to 60k iterations.The comparison uses the default 30k iterations as the reference.
  • H. Visualization of Bit Allocation: Under the same λ, longer training primarily improves fidelity while producing minimal storage-size reductions.This indicates that iteration increases affect quality more than compressed size in the reported training curves.
  • H. Visualization of Bit Allocation: Additional training iterations provide diminishing incremental benefits as the total iteration count increases.The reported trade-off must therefore account for both training time and compression performance.
  • H. Visualization of Bit Allocation: Selecting training iterations can be tailored to application requirements and available computational resources.The paper frames iteration choice as a trade-off between training time and compression performance.
  • H. Visualization of Bit Allocation: HAC++ is evaluated with PSNR, SSIM, and LPIPS across all datasets, with per-scene results provided in the Appendix.The cited passage identifies the fidelity metrics and location of the detailed results.

–Appendix–

The Appendix reports HAC++ results across Synthetic-NeRF, DeepBlending, Tank&T emples, Mip-NeRF360, and BungeeNeRF, including comparisons across λ values for several datasets.

  • –Appendix–: Appendix Table A reports HAC++ results on the Synthetic-NeRF dataset.The supplied table heading identifies the dataset but not the reported metric values.
  • –Appendix–: Appendix Table B reports HAC++ results on the DeepBlending dataset.The supplied table heading identifies the dataset but not the reported metric values.
  • –Appendix–: Appendix Tables C and D report HAC++ results for different λ values on Tank&T emples and Mip-NeRF360, respectively.Both supplied headings explicitly identify λ as the varying condition.
  • –Appendix–: Appendix Table E reports HAC++ results on the BungeeNeRF dataset for different conditions.The supplied heading is truncated after identifying the dataset and that conditions differ.
Loading 2501.12255v4…