PaperlayerView exact revision on arXiv

Source-linked AI summary

CompGS: Efficient 3D Scene Representation via Compressed Gaussian Splatting

Xiangrui Liu, Xinju Wu, Pingping Zhang, Shiqi Wang, Zhu Li, Sam Kwong

arXiv:2404.09458v1cs.CVcs.GR

TL;DR

Gaussian splatting can require millions of primitives, while existing compression methods leave redundancies by compressing Gaussians independently and optimizing only rendering distortion. CompGS predicts primitives from anchors and balances bitrate with rendering distortion, reporting up to 110× compression alongside high-quality rendering and compact representations versus prior methods.

  • Problem

    Gaussian splatting can require millions of primitives for high-quality rendering, and existing compression methods leave redundancies by compressing Gaussians independently and optimizing only rendering distortion.

    Source: §1 · pp. 1–2

  • Method

    CompGS predicts most coupled primitives from a sparse set of anchors, encodes compact residuals, and optimizes bitrate jointly with rendering distortion.

    Source: §1 · p. 2

  • Results

    Up to 110× compression ratio on prevalent datasets accompanies reported high-quality rendering and compact representations versus prior methods.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CompGS provides a compact 3D scene representation while retaining high-quality rendering in the reported comparisons.

    Source: §1 · p. 2

Abstract

from arXiv · show

Gaussian splatting, renowned for its exceptional rendering quality and efficiency, has emerged as a prominent technique in 3D scene representation. However, the substantial data volume of Gaussian splatting impedes its practical utility in real-world applications. Herein, we propose an efficient 3D scene representation, named Compressed Gaussian Splatting (CompGS), which harnesses compact Gaussian primitives for faithful 3D scene modeling with a remarkably reduced data size. To ensure the compactness of Gaussian primitives, we devise a hybrid primitive structure that captures predictive relationships between each other. Then, we exploit a small set of anchor primitives for prediction, allowing the majority of primitives to be encapsulated into highly compact residual forms. Moreover, we develop a rate-constrained optimization scheme to eliminate redundancies within such hybrid primitives, steering our CompGS towards an optimal trade-off between bitrate consumption and representation efficacy. Experimental results show that the proposed CompGS significantly outperforms existing methods, achieving superior compactness in 3D scene representation without compromising model accuracy and rendering quality. Our code will be released on GitHub for further research.

1. Introduction

Gaussian splatting can require millions of primitives, making storage and bandwidth a practical concern; existing compression approaches leave redundancies by compressing Gaussians independently and focusing on rendering distortion. CompGS uses anchor-based prediction and rate-constrained optimization to pursue compact representations while retaining rendering quality.

1. Introduction

  • Existing compression methods independently compress Gaussians and optimize rendering distortion, leaving local and within-primitive redundancies.The paper identifies local similarities among Gaussians as a source of redundancy.Source: §1 · pp. 1–2
  • Up to 110× compression ratio on prevalent datasets is reported for CompGS.The contribution summary gives the maximum ratio without specifying a particular dataset.Source: §1 · p. 2
  • CompGS predicts most coupled primitives from a sparse set of anchors, representing them with compact residual embeddings.The hybrid structure is designed to capture predictive relationships among primitives.Source: §1 · p. 2
  • Rate-constrained optimization jointly minimizes rendering distortion and bitrate costs to improve primitive compactness.The stated objective is an optimal trade-off between bitrate consumption and representation efficiency.Source: §1 · p. 2

2. Related Work

Related work covers Gaussian splatting and its compression methods, then connects CompGS to predictive and rate-distortion ideas from video coding.

2.1. Gaussian Splatting Scene Representation

  • 3DGS represents scenes with explicit 3D Gaussians and renders them through parallel volume splatting, but maintaining many Gaussians can produce large models.The Gaussians encode geometry and appearance attributes; adaptive control prunes insignificant ones and densifies crucial ones.Source: §2.1 · pp. 2–3

2.2. Compressed Gaussian Splatting

  • Prior 3DGS compression combines pruning and quantization, but independent compression with distortion-only optimization leaves substantial representation redundancies.The cited approaches also include learned masks, neural fields, entropy coding, distillation, and latent embeddings.Source: §2.2 · p. 3

2.3. Video Coding

  • Video coding uses prediction and residual coding alongside rate-distortion optimization; CompGS adopts this philosophy to reduce primitive redundancies.Neural video coding also uses entropy models to estimate the rates of residuals.Source: §2.3 · p. 3

3. Methodology

CompGS represents scenes with anchor primitives that predict associated coupled primitives, which store compact residual embeddings. It combines inter-primitive prediction with rate-constrained optimization to balance bitrate and rendering distortion.

3.1. Overview

  • Each anchor primitive supplies geometry and reference embeddings for associated coupled primitives, which store residual embeddings that compensate for prediction errors.The predicted primitives’ geometry and view-dependent appearance are used for volume-splatting renders.Source: §3.1 · pp. 3–4

3.2. Inter-primitive Prediction

  • Prediction fuses anchor reference and coupled residual embeddings, then uses an affine transform for coupled geometry and view embeddings with neural networks for appearance.The affine parameters are decomposed into translation, scaling, and rotation components predicted by neural networks.Source: §3.2 · pp. 4–5

3.3. Rate-constrained Optimization

  • Scalar quantization is approximated with uniform quantization noise for differentiable optimization, while entropy estimation models anchor and coupled attributes’ bitrate.The implementation uses K = 10 coupled primitives per anchor and arithmetic coding for embeddings and anchor covariance; G-PCC compresses anchor locations.Source: §3.3 · p. 5, §3.4 · p. 6
  • CompGS jointly minimizes a rate-distortion cost over anchor and coupled primitives, using bitrate estimates and rendering loss to favor compact representations with high-quality rendering.The rate term sums estimated bitrate costs for both primitive types, while the distortion term uses rendering loss.Source: §3.3 · p. 6

4. Experiments

CompGS is evaluated across three view-synthesis datasets, with results showing substantially smaller bitstreams and competitive or better rendering quality. Ablations examine its hybrid primitives, rate-constrained optimization, residual embeddings, and coupled-primitive count; the paper also reports computational costs.

4.1. Experimental Settings

  • Experiments use Tanks&Templates, Deep Blending, and Mip-NeRF 360, with one of every eight views held out for testing and five repeated evaluations for comparison methods.Rendering quality is assessed with PSNR, SSIM, and LPIPS; model size and training, encoding, decoding, and rendering times are also measured.Source: §4.1 · pp. 6–7

4.2. Experimental Results

  • Across Tanks&Templates, Deep Blending, and Mip-NeRF 360, CompGS achieves smaller bitstreams with competitive or better rendering quality; Figure 6 compares rate-distortion curves.The paper reports the highest rendering quality and smallest bitstream on Tanks&Templates, and competitive rendering quality with size reduction on Mip-NeRF 360.Source: §4.2 · p. 7, Figure 6 · p. 7
  • CompGS reaches 45.25×–73.75× compression versus 3DGS on Tanks&Templates, 75.94×–110.45× on Deep Blending, and up to 89.35× on Mip-NeRF 360.At the highest bitrate on Deep Blending, rendering quality improves by 0.15 dB; Mip-NeRF 360’s Stump scene reaches 175.20× compression.Source: §4.2 · p. 7

4.3. Ablation Studies

  • Hybrid primitives reduce bitstream size from 257.44 MB to 48.58 MB for Train and from 611.31 MB to 30.38 MB for Truck.Figure 8 shows coupled primitives consume fewer average bits than anchor primitives despite their much higher count.Source: §4.3 · p. 8, Figure 8 · p. 9
  • Rate-constrained optimization further reduces Train’s bitstream from 48.58 MB to 8.60 MB, an additional 82.30% reduction, and cuts Truck bitrate by 65.08%.The ablation attributes the bitrate efficiency to learning compact primitive representations.Source: §4.3 · p. 8
  • Without residual embeddings, the primitive-derivation variant has inferior rendering quality because it struggles to capture coupled primitives’ unique characteristics.CompGS instead represents those characteristics with compact residual embeddings.Source: §4.3 · p. 8
  • On Train, K = 10 yields the best rendering quality; increasing K from 10 to 15 degrades it by 0.22 dB.The authors suggest excessive coupled primitives may lead to inaccurate prediction.Source: §4.3 · p. 9, Table 6 · p. 9

4.4. Complexity Analysis

  • On Tanks&Templates, training averages 37.83 minutes, shorter than Lee et al. [20] and longer than other methods; encoding and decoding each take under 10 seconds, and per-view rendering averages 5.32 milliseconds.Source: §4.4 · p. 9

5. Conclusion

CompGS represents 3D scenes with compact primitives by predicting coupled primitives from a limited set of anchors and encoding their residuals. Rate-constrained optimization further improves compactness, and the method achieves superior size reduction over existing compression methods without compromising rendering quality.

5. Conclusion

  • CompGS combines anchor-predicted coupled primitives with compact residual embeddings and rate-constrained optimization to reduce scene size while preserving rendering quality.Its rate-distortion cost jointly considers bitrate and rendering distortion.Source: §5 · p. 9

A. Per-scene Results on Evaluation Datasets

Per-scene evaluations across Tanks&Templates, Deep Blending, and Mip-NeRF 360 report substantial compression while maintaining comparable or preserved rendering quality. Compression ratios vary by scene, reaching 175.20× for Stump.

A. Per-scene Results on Evaluation Datasets

Loading 2404.09458v1…