PaperlayerView exact revision on arXiv

Source-linked AI summary

RacerGS: Residual-Autoregressive Context for 3D Gaussian Splatting Compression

Wenqing Wang, Huimin Zeng, Yun Fu

arXiv:2610.34263v1cs.CVcs.GR

TL;DR

Large 3DGS scenes create substantial storage and transmission demands despite fast, high-quality rendering. RacerGS combines projected spatial context, residual entropy coding, and recurrent causal refinement; it reports substantially smaller representations with preserved rendering quality.

  • Problem

    Large 3DGS scenes can require millions of Gaussians, creating substantial storage and transmission demands.

    Source: §1 · p. 2

  • Method

    RacerGS combines hash-projected context features, adaptive residual quantization, and a FiLM-modulated autoregressive GRU to refine entropy-model parameters.

    Source: §1 · p. 2

  • Results

    RacerGS achieves 150× compression over vanilla 3DGS and 28× over Scaffold-GS, with PSNR gains of 2.31 dB and 0.56 dB, respectively.

    Source: §1 · p. 2

  • Takeaways & Limitations

    RacerGS reduces storage requirements while preserving rendering fidelity across diverse datasets.

    Source: §6 · p. 10

    Sequential decoding limits real-time streaming applicability, and per-scene optimization means unseen-scene generalization still requires retraining.

    Source: F Limitations · p. 15

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) has emerged as a promising approach for fast, high-quality novel-view rendering. However, its large parameter size remains a bottleneck for storage and transmission. We propose RacerGS, a compact 3DGS compression framework that achieves scene reconstruction at significantly reduced storage sizes with preserved fidelity. To obtain compact and spatially coherent features, we design a hash projection model that projects the anchor hash encodings into compact context features. Additionally, we introduce a FiLM-modulated autoregressive GRU context model that utilizes causal dependencies among anchor feature groups and uses recurrent hidden states to refine entropy-model parameters, improving bitrate efficiency. Furthermore, we adaptively entropy-code quantized anchor-attribute residuals relative to their context-predicted means, leveraging spatial consistency to reduce symbol entropy. Overall, RacerGS achieves more than 150x compression over vanilla 3DGS and 28x over Scaffold-GS, while providing comparable rendering quality. Extensive experiments across five datasets demonstrate that RacerGS outperforms prior 3DGS compression methods in both storage reduction and rate-distortion performance.

1 Introduction

3DGS renders quickly and with high visual quality, but large scenes create substantial storage and transmission demands. RacerGS combines compact spatial context, residual coding, and recurrent causal modeling to improve compression while preserving rendering quality.

1 Introduction

  • 3DGS offers substantially faster rendering than NeRF-based methods while maintaining high visual quality, but large scenes may require millions of Gaussians.That scale creates substantial storage and transmission demands.Source: §1 · p. 2
  • Prior compression methods either ignore spatial redundancy or leave anchor relationships and residual regularity underused.Scaffold-GS models anchors independently; HAC methods use concatenated hash features or directly quantize attributes.Source: §1 · p. 2
  • RacerGS combines hash-projected context features, a FiLM-modulated autoregressive GRU, and adaptive residual quantization for entropy coding.The components target context modeling and entropy estimation together.Source: §1 · p. 2
  • 150× compression over vanilla 3DGS and 28× over Scaffold-GS accompany PSNR gains of 2.31 dB and 0.56 dB, respectively.The reported gains preserve rendering quality across diverse datasets.Source: §1 · p. 2

2 Related Work

Prior scene-compression approaches balance representation size and reconstruction quality by exploiting parameter or structural redundancy. RacerGS builds on structure-aware 3DGS compression while addressing limits in spatial context, causal dependencies, and residual coding.

2 Related Work

  • Value-based NeRF compression reduces parameter precision or redundancy, while structure-based approaches expose dependencies through scene organization.Both seek to balance representation size against reconstruction quality.Source: §2 · pp. 2–3
  • RacerGS projects hash encodings into compact context features, codes residuals around predicted means, and refines parameters using recurrent GRU context with FiLM.The pipeline overview combines the refined parameters with a spatial prior for residual entropy coding.Source: Figure 2 · p. 3, §2 · p. 3
  • 3DGS compression methods use codebooks, entropy models, feature projections, grids, anchors, and hash-grid context to reduce storage.HAC-style methods are described as using poorly scaled hash contexts, missing higher-order causal dependencies, and coding raw attributes rather than spatially regular residuals.Source: §2 · p. 3

3 Preliminaries

The preliminaries describe 3DGS scenes as collections of Gaussians with geometric and appearance attributes. They also outline covariance construction, alpha compositing, and Scaffold-GS anchor-based attribute prediction.

3 Preliminaries

  • Each 3D Gaussian is characterized by opacity, color, a 3D covariance matrix, and a center position.Source: §3 · p. 3
  • The covariance is formed from rotation and scaling matrices, and pixel color is computed by alpha compositing.The passage gives the covariance construction as RSS⊤R⊤.Source: §3 · p. 3
  • Scaffold-GS groups Gaussians around anchors and uses anchor attributes to predict the attributes of associated Gaussians.Anchor attributes include features, scaling parameters, and offset parameters.Source: §3 · p. 3

4 Method

RacerGS compresses 3DGS by combining compact hash-derived context features, residual entropy coding, and causal refinement of anchor-feature distributions. Its context-predicted means and decoded feature groups guide coding of anchor attributes.

4.2 Context feature projection

  • A hash projection model maps anchor hash encodings to compact, fused context features that provide structured, low-dimensional input to the entropy model.Joint training with the entropy loss retains bits-relevant directions without inflating entropy-parameter cost.Source: §4.2 · p. 4, §4.1 · p. 4

4.3.1 Adaptive residual quantization

  • Adaptive residual quantization codes anchor-attribute residuals around context-predicted means, contracting their dynamic range and lowering residual entropy.Decoding uses the integer symbol and fused context to predict the mean and quantization step.Source: §4.3.1 · p. 4

4.3.2 Residual distribution modeling

  • Residual probabilities are modeled over quantization bins and arithmetic-coded, with code length based on the negative log probability of each residual.The method describes the predicted residual as centered near zero.Source: §4.3.2 · p. 5

4.4 Causal entropy model refinement

  • Causal refinement processes decoding-order feature chunks with a cached GRU state that summarizes preceding chunks at constant per-chunk input cost.A shared parameter predictor uses learnable group positional embeddings to specialize across chunks.Source: §4.4.1 · p. 5, §4.4 · p. 5, §4.4.3 · p. 6

4.4.2 Anchor-conditional FiLM modulation

  • Anchor-conditional FiLM modulates the chunk latent, improving optimization stability and refinement capacity without deepening the GRU or increasing its parameter footprint.The paper links this design to improved coding efficiency.Source: §4.4.2 · p. 6

4.4.4 Residual mixture distribution with refined context

  • A mixture combines base entropy parameters from hash context with refinement parameters from causal relations within anchor-feature groups.The base component captures coarse residual statistics, while refinement adjusts probability mass using causal dependencies.Source: §4.4.4 · p. 6

4.5.2 Training process

  • Table 1 reports RacerGS outperforming baselines in size reduction and fidelity preservation; Figure 3 shows better rate-distortion performance, and Figure 4 shows detail preservation with fewer artifacts.The comparisons span datasets and scenes.Source: Table 1 · p. 7, Figure 3 · p. 8, Figure 4 · p. 8

4.5.1 Residual entropy loss

  • Training jointly optimizes rendering and residual-entropy losses, while the entropy loss measures expected bit consumption and masks prune inactive anchors and offsets.The total loss also includes a rate-weighted term for coded parameters per anchor.Source: §4.5.1 · p. 6, §4.5.2 · p. 7

5 Experiments

RacerGS compresses 3DGS across five datasets while retaining rendering quality, and its experiments evaluate storage, visual fidelity, runtime, and component-level ablations. The results and ablations examine hash projection, residual quantization, and recurrent causal context modeling.

5.1 Implementation

  • Experiments span five datasets, compare codebook/pruning, entropy/sorting, and anchor-based baselines, and measure image quality, storage, and runtime.They use one RTX A6000 GPU and 30,000 training iterations, with bitrate operating points varied through λr.Source: §5.1 · pp. 8–9

5.2 Experimental results

  • Over 150× compression versus vanilla 3DGS and 28× versus Scaffold-GS on average, RacerGS preserves comparable rendering quality across datasets.At the high-rate Mip-NeRF360 point, RacerGS uses 13.61 MB versus HAC++’s 18.48 MB, with PSNR of 27.84 versus 27.82 dB.Source: §5.1 · p. 8, §5.2 · p. 9
  • Across diverse scenes, RacerGS shows fewer leaf artifacts, clearer book-title details, and sharper metal structures while preserving edges and material details at comparable or smaller sizes.The qualitative comparisons also report smooth backgrounds under tight storage constraints.Source: §5.2 · p. 9
  • RacerGS reaches 171 FPS on Mip-NeRF360 with the lowest peak GPU memory, while encoding takes 0.63–16.26 s and decoding 0.89–25.98 s across datasets.Training takes longer than for the compared baselines; anchor features dominate storage, while hash-grid and MLP modules add minor overhead.Source: §5.2 · p. 9, Table 4 · p. 9, Table 3 · p. 9

5.3 Ablation study

  • Without hash projection, PSNR falls from 27.84 to 27.62 and model size grows from 13.61 MB to 15.21 MB.A single linear projection yields 27.76 dB / 14.28 MB; the reported context-dimension sweep changes size by at most 0.35 MB and PSNR by at most 0.04 dB.Source: §5.3 · p. 10, Table 5 · p. 10
  • Without residual quantization, size rises from 13.61 MB to 18.24 MB and PSNR slightly decreases.Prediction-centered anchor-feature symbols have empirical entropy of 5.22 rather than 8.27 bits per element, and arithmetic-coded cost of 5.38 rather than 8.44 bits per element.Source: §5.3 · p. 10, Table 5 · p. 10
  • Replacing the GRU with decoded-chunk concatenation increases size from 13.61 to 16.87 MB at nearly equal PSNR; removing FiLM raises size to 17.32 MB.Both approaches cost 68 bits per anchor for the first chunk, while by the fifth chunk the GRU costs 39.0 versus 80.1 bits per anchor for concatenation.Source: §5.3 · p. 10, Table 6 · p. 10, Table 7 · p. 10

6 Conclusion

RacerGS reduces 3DGS storage while preserving rendering fidelity through hash-projected context, residual quantization, and a FiLM-modulated autoregressive GRU. The paper concludes that it achieves substantially lower storage with comparable quality across diverse datasets, while identifying parallelizable context models and content-adaptive control as future work.

6 Conclusion

  • RacerGS reduces storage while preserving rendering fidelity, and experiments report substantially lower storage with comparable quality across diverse datasets.The paper identifies parallelizable context models and content-adaptive control as future work for more efficient 3D rendering.Source: §6 · p. 10

A Entropy coding

  • Entropy coding assigns code lengths proportional to symbol information content, with expected length bounded below by Shannon entropy.Arithmetic coding can approach that bound when its probability model matches the true symbol distribution.Source: A Entropy coding · p. 14

B Hash-grid-assisted design

  • Hash-grid-assisted design leaves redundancy from spatial consistency in anchor-attribute residuals and causal dependencies among feature groups; RacerGS targets both with residual modeling and recurrent hidden states.The hash-grid-assisted design is described as achieving strong performance, while these remaining redundancies motivate the proposed compression approach.Source: B Hash-grid-assisted design · p. 14

C Per-component coding time

On Mip-NeRF360, anchor features dominate RacerGS coding time, while anchor positions and the remaining compact components add little overhead.

C Per-component coding time

  • Anchor features take about 46%–51% of encoding time and 50%–53% of decoding time on Mip-NeRF360.They contain the most parameters and require entropy modeling with the autoregressive GRU; scaling parameters and offsets take the next-largest share, while anchor positions contribute little.Source: C Per-component coding time · p. 14
  • Total coding time rises as the rate weight λr decreases, because smaller λr produces more anchors and higher-bitrate representations.Source: C Per-component coding time · p. 14
  • The remaining components add negligible coding overhead because their representations are compact.Source: C Per-component coding time · p. 14

D Quantitative per-scene results

Per-scene results are provided for five datasets: Synthetic-NeRF, Mip-NeRF360, Tanks&Temples, DeepBlending, and BungeeNeRF.

D Quantitative per-scene results

E Additional analyses

Additional analyses examine entropy-context projection, context-dimension sensitivity, residual coding, seed variability, and decoding-time breakdown.

E Additional analyses

  • Table 8 analyzes entropy-context projection, context-dimension sensitivity, anchor-feature symbol statistics, and variability across three seeds on Mip-NeRF360.The nonlinear projection outperforms a single linear layer; performance stays stable across a 4× context-dimension range, and halving the default dimension increases size by only 2.6%.Source: E Additional analyses · p. 14, Table 8 · p. 15
  • Residual coding reduces both the anchor-feature alphabet and its coding cost.Across three seeds, standard deviations are 0.02 dB for PSNR and 0.10 MB for size.Source: E Additional analyses · p. 14, Table 8 · p. 15
  • Arithmetic coding accounts for about 60% of decoding time across datasets, with the remainder spent on network evaluation.Source: Table 9 · p. 15

F Limitations

RacerGS has sequential anchor-feature decoding that limits real-time streaming, and its per-scene optimization means unseen scenes require retraining.

F Limitations

  • Sequential decoding of anchor feature groups limits RacerGS’s applicability to real-time streaming scenarios.Source: F Limitations · p. 15
  • Because RacerGS is optimized per scene, generalization to unseen scenes still requires retraining.The authors identify parallelizable context modeling and cross-scene generalization as future directions.Source: F Limitations · p. 15

G Societal impact

RacerGS is presented as reducing storage and transmission costs for high-fidelity 3D scenes, with potential benefits across immersive and other applications. The paper also flags privacy and deception risks when compression is used to distribute sensitive or misleading content.

G Societal impact

  • Reduced storage and transmission costs can lower the bandwidth and energy footprint of high-fidelity 3D scene representations.The stated potential applications include virtual and augmented reality, telepresence, digital cultural heritage preservation, robotics simulation, and remote collaboration.Source: G Societal impact · p. 15
  • More compact 3D content may be more accessible to edge and mobile users with limited compute or network resources.Source: G Societal impact · p. 15
  • More efficient distribution could facilitate unauthorized sharing of private-space or individual reconstructions and deceptive photorealistic 3D media.The authors recommend consent, watermarking, provenance practices, and access control for sensitive content.Source: G Societal impact · p. 15
Loading 2610.34263v1…