PaperlayerView exact revision on arXiv

Source-linked AI summary

CC-4DGS: Computational Deformation and Point-Cloud Compression for Storage-Efficient Dynamic Gaussian Splatting

Kyungdae Park, Chae Eun Rhee

arXiv:2609.02184v1cs.CV

TL;DR

Dynamic 4D Gaussian Splatting is limited by the storage cost of deformation hash tables and high-dimensional Gaussian attributes. CC-4DGS replaces stored deformation features with a computational deformation field and compresses canonical attributes, achieving comparable reconstruction quality with substantially lower storage across N3DV and Technicolor datasets.

  • Problem

    Dynamic 4DGS methods can require hundreds of megabytes to several gigabytes per scene because deformation features are stored in large multi-resolution hash tables or voxel grids.

  • Method

    CC-4DGS uses a compact decoder with dense hash encoding for on-the-fly deformation features and a learned compression pipeline for canonical Gaussian attributes.

  • Results

    Across N3DV and Technicolor, CC-4DGS maintains comparable reconstruction quality to representative methods while reducing the model size to only a few tens of megabytes.

  • Takeaways & Limitations

    The full CDF+CCA configuration provides a favorable trade-off among storage efficiency, reconstruction quality, and rendering speed.

  • Takeaways & Limitations

    Quality gradually decreases as sequence length grows because the fixed-capacity canonical point set and deformation pathway must cover increasing temporal extent.

Abstract

from arXiv · show

Dynamic four-dimensional (4D) Gaussian Splatting has emerged as a powerful explicit representation for high-quality view synthesis, yet existing methods still require tens to hundreds of megabytes per scene due to their heavy reliance on large multi-resolution hash tables and high-dimensional Gaussian attributes. This paper presents CC-4DGS, a storage-efficient and scalable framework that rethinks both deformation modeling and canonical attribute storage. First, we introduce a computational deformation field (CDF) that replaces large multi-resolution learnable hash tables with deterministic dense hash encoding and compact neural decoders, enabling on-the-fly synthesis of deformation features while reducing deformation storage to only 1--3 MB per scene. Second, we propose a compression of canonical point-cloud attributes (CCA) pipeline that compresses high-dimensional spherical harmonic appearance terms and auxiliary Gaussian attributes via conditional autoencoding, selective quantization, and residual codebooks, achieving 3--5$\times$ point-cloud reduction with negligible quality loss. Together, these components yield a unified representation that preserves real-time rendering performance while reducing total storage to 20--30 MB. Extensive experiments across the N3DV and Technicolor Light Field datasets demonstrate that CC-4DGS achieves reconstruction accuracy comparable to state-of-the-art methods such as Swift4D, while offering significantly improved storage efficiency and favorable runtime-memory trade-offs.

I. Introduction

Dynamic 4D scene representations improve view synthesis across time but face substantial storage and computation challenges. CC-4DGS addresses these bottlenecks by replacing stored deformation features and compressing canonical Gaussian attributes.

  • 4D video captures static structure and temporal changes, enabling consistent rendering under varying viewpoints and environmental conditions.
  • Explicit dynamic representations render efficiently but incur substantial storage overhead as voxel grids, hash tables, and Gaussian attributes grow with scene complexity.
  • CC-4DGS replaces large learnable deformation hash tables with compact on-the-fly computation and compresses canonical Gaussian attributes.
  • Canonical-space deformation methods improve temporal consistency but still face computational and storage limitations in complex dynamic scenes.

C. Hash-Based Embeddings and Learnable Hash-Table-Free Feature Learning

Hash-based embedding methods motivate replacing large explicit parameter tables with compact functions that compute features on demand. In 3D Gaussian Splatting, these features complement explicit Gaussian primitives used for efficient rendering.

  • Deep Hash Embedding replaces explicit embedding tables with multiple universal hash functions and a compact MLP that computes embeddings on-the-fly.
  • 3D Gaussian Splatting represents scenes with Gaussian kernels parameterized by position, covariance, opacity, and spherical-harmonic color coefficients.
  • The covariance matrix is decomposed into scaling and rotation factors to ensure positive semi-definiteness.
  • Projected Gaussian primitives contribute to rendered pixels through alpha blending, with each primitive supplying a projected opacity contribution.

B. Dynamic Scene Representation and Storage Challenges

Canonical dynamic Gaussian splatting transforms a stored Gaussian set over time, but conventional deformation features stored in large hash tables or voxel grids can dominate scene storage. CC-4DGS computes these features with compact decoders and interpolated spatial representations instead.

  • Dynamic representation: A canonical Gaussian G is transformed into a time-dependent primitive G′ by adding a learned deformation ΔG.
  • Storage challenge: Large multi-resolution hash tables and voxel grids can make deformation features consume hundreds of megabytes to several gigabytes per scene.
  • Computational deformation field: CC-4DGS computes dynamic deformation on-the-fly using dense hash encoding, hash feature decoders, and a lightweight deformation network.
  • CDF construction: The CDF constructs multi-resolution voxel vertices from normalized canonical Gaussian centers before hashing and decoding their features.
  • Dense hash encoding: Dense hash encoding maps each voxel vertex through parallel hash functions to a dense real-valued signature used by the feature decoder.
  • Feature synthesis: A lightweight decoder synthesizes vertex-specific features, which are combined by trilinear interpolation into the spatial feature at each Gaussian center.
  • Multi-resolution design: Using only 4 to 5 resolution levels instead of the typically 16 levels in prior work streamlines the voxel hierarchy.

2) Collision-Resilient Multi-Resolution Design:

The CDF combines collision-resilient spatial encoding with temporal conditioning to synthesize deformation features and predict time-varying Gaussian attributes. Feature reuse and FiLM-based modulation further organize the spatiotemporal computation.

  • Collision-Resilient Multi-Resolution Design:: Full 128-dimensional hash signatures produced zero exact duplicates across queried N3DV voxel-corner coordinate sets.
  • Collision-Resilient Multi-Resolution Design:: The Hash Feature Decoder maps dense multi-hash signatures to continuous deformation features rather than resolving collisions through explicit table lookups.
  • Time Encoding and Deformation Field Estimation:: Sinusoidal timestamp encoding and a shallow temporal network produce the temporal feature ftime used in dynamic deformation estimation.
  • Time Encoding and Deformation Field Estimation:: Feature-wise Linear Modulation combines temporal and spatial features by applying time-dependent channel scales and shifts to the spatial representation.
  • Time Encoding and Deformation Field Estimation:: A shared trunk and task-specific branches predict offsets for position, scale, rotation, opacity, and spherical-harmonic coefficients.
  • Time Encoding and Deformation Field Estimation:: The predicted offsets are added to canonical attributes to produce smooth changes in Gaussian position, orientation, and appearance during rasterization.

C. Overcoming the Computational Overhead of CDF

CDF introduces redundant computation when shared voxel vertices are repeatedly encoded and decoded, motivating vertex reuse to avoid recomputation.

  • Shared vertex coordinates can cause repeated hash encoding and neural decoding for Gaussian primitives in the same or adjacent voxels.Naive execution redundantly processes identical vertices during grid construction.

1) Vertex Reuse for Redundant-Compute Reduction:

CC-4DGS compresses canonical Gaussian attributes according to their sensitivity and dimensionality, while caching and reuse mechanisms reduce the computational cost of on-the-fly feature generation.

  • CCA preserves position and SH-DC in full precision, quantizes scalar attributes, and compresses SH-AC with a conditional autoencoder and residual codebook.The three pathways target geometric fidelity, scalar compactness, and the dominant high-dimensional appearance storage.
  • SH-AC accounts for approximately 70% of total data size and is encoded using geometric conditioning attributes including position, scale, opacity, rotation, and dynamics.The encoder maps the joint SH-AC and conditioning vector into a low-dimensional latent representation.
  • Positions use full 32-bit precision, while rotation, scale, and opacity use 12-bit linear quantization.The choices reflect differing sensitivity to quantization error.
  • The dynamic indicator uses µ-law companding because its long-tailed distribution is poorly preserved by simple linear quantization.The transformation allocates more effective resolution where important information is concentrated.
  • Outputs from all pathways are combined with lightweight metadata and stored after lossless Zstandard compression.The compressed components are assembled into the final archive.

E. Loss Functions and Regularization

Training combines reconstruction supervision with selective smoothness regularization to stabilize dynamic rendering when explicit voxel-based deformations flicker.

  • Predicted deformations are applied to Gaussian position, scale, rotation, opacity, and spherical-harmonic coefficients during training.The rendered output is supervised with pixel-wise L1 and SSIM losses.
  • Smoothness regularization is enabled only for challenging scenes exhibiting noticeable temporal flickering.The regularizer is applied directly to embedded feature vectors before deformation-network inference.
  • The final loss combines reconstruction accuracy with deformation-field smoothness, with λr balancing the two objectives.λr is increased linearly during training.
  • The regularizer enforces similarity among deformation parameters in local spatio-temporal neighborhoods and suppresses flickering artifacts.This stabilizes rendering in dynamic regions.
  • Evaluation compares representative dynamic Gaussian splatting methods under a shared single-step initialization protocol using one canonical point cloud.COLMAP initializes only the first frame, while later deformations are learned from 2D supervision; methods using different geometric assumptions are excluded from the main comparison.

2) Datasets and Evaluation Metrics:

CC-4DGS is evaluated on N3DV and Technicolor using comparisons that emphasize the trade-off among reconstruction quality, rendering speed, and storage under specified experimental settings.

  • N3DV results: On N3DV, CC-4DGS reaches PSNR in the low 32 dB range and SSIM of approximately 0.94, slightly exceeding Swift4D in reconstruction quality.Swift4D achieves higher FPS, while CC-4DGS maintains real-time rendering through feature caching and dynamic separation.
  • Technicolor results: On Technicolor, CDF improves PSNR from 32.83 to 33.29 dB, SSIM from 0.900 to 0.912, and rendering speed from 46.2 to 64.7 FPS versus E-D3DGS.With CCA, the full system reaches about 32 MB while retaining PSNR 33.3 dB, SSIM 0.912, and roughly 64 FPS.
  • Overall trade-off: Across both datasets, CDF+CCA provides comparable reconstruction quality to representative methods while maintaining a model size of only a few tens of megabytes.The paper characterizes this as a favorable storage–quality–speed trade-off.
  • Comparison scope: The contextual N3DV comparison reports 32.06 dB PSNR but is not a uniform head-to-head benchmark because resolution, GPU, and task formulation differ.It is best interpreted as an operating-point comparison, with larger storage than bitrate-oriented methods such as GIFStream and Light4GS.
  • Comparison scope: Recent-method rows use reported values without reruns, and some FPS, resolution, and SSIM measurements are not directly comparable across methods.The table notes heterogeneous settings and different GPU measurements.

C. Qualitative Results

CC-4DGS preserves fine-grained visual details across N3DV and Technicolor Light Field scenes while substantially reducing storage. Its CDF and CCA components jointly provide favorable reconstruction-quality and storage trade-offs, with PSNR comparable to Swift4D at smaller model sizes.

  • Qualitative Results: Ours(CDF+CCA) remains nearly indistinguishable from Ours(CDF) despite additional point-cloud compression.This indicates that the CCA stage introduces negligible visible degradation in the shown N3DV comparisons.
  • Qualitative Results: CC-4DGS preserves sharper high-frequency details and thin structures than prior methods in representative N3DV scenes.The comparisons cover facial features, fingers, flame boundaries, bottle labels, textures, and fast motion.
  • Qualitative Results: CC-4DGS preserves intricate patterns and thin structures in high-resolution Technicolor scenes, including paper decorations, fabric folds, leaves, and building surfaces.The evaluation also examines regions with large depth parallax for geometric stability.
  • Storage and Rate Distortion Analysis: More than 60 MB of total storage is saved relative to Swift4D+CCA and Grid4D.The storage analysis separates deformation parameters from canonical point-cloud geometry and appearance.
  • Storage and Rate Distortion Analysis: Approximately 0.8 MB of Deform-Grid storage and 1.1 MB of Deform-Net storage replace the large learnable hash tables used by prior methods.Prior methods allocate most deformation storage to multi-resolution grids, whereas CC-4DGS reallocates it toward compact learned parameters.
  • Storage and Rate Distortion Analysis: CC-4DGS reaches PSNR comparable to default Swift4D with substantially smaller total and deformation-module storage.Rate–distortion points vary deformation capacity for CC-4DGS and hash-grid capacity for Swift4D under a fixed CCA profile.
  • Storage and Rate Distortion Analysis: Reducing the CCA profile from Default to Low cuts model size from 29.67 MB to 23.46 MB, while PSNR decreases by only 0.14 dB.This sweep uses the six-scene N3DV average with the CDF configuration fixed at Default.

E. Long-sequence Stress Test

The long-sequence stress test evaluates CC-4DGS and Swift4D on flame_salmon sequences from 300 to 1200 frames. Both methods degrade gradually, but CC-4DGS degrades less in this single-scene setting and retains a larger advantage at 1200 frames.

  • Long-sequence Stress Test: CC-4DGS declines from 29.62 dB at 300 frames to 29.12 dB at 1200 frames, a 0.50 dB decrease.The training and densification schedules are scaled with sequence length.
  • Long-sequence Stress Test: Swift4D declines from 29.48 dB at 300 frames to 28.24 dB at 1200 frames, a 1.25 dB decrease.The comparison uses the same flame_salmon stress-test sequence lengths.
  • Long-sequence Stress Test: At 1200 frames, CC-4DGS retains a +0.88 dB advantage over Swift4D.This is the largest reported advantage in the stress-test comparison.
  • Long-sequence Stress Test: The single-scene stress test does not establish general long-duration scalability.It indicates only that the tested frequency-based time encoding and depth-3 TimeMLP do not show a disproportionately worse failure mode than the baseline in this setting.

B. Acceleration Effects of Vertex Reuse and Feature Caching

CC-4DGS accelerates its procedural deformation path by caching spatial features and reusing duplicated vertices. Feature caching nearly saturates grid-path efficiency, while vertex reuse mainly reduces memory and supports longer-term runtime savings.

  • Acceleration Effects: Feature caching computes spatial hash features once and reuses them across subsequent frames, leaving only the time-dependent branch to evaluate.This exploits the separation between spatial and temporal feature computation in the CDF pathway.
  • Acceleration Effects: Vertex reuse merges duplicated spatial vertices into a unique set, reducing redundant hash encoding and memory usage.The optimization targets Gaussians that reference identical or nearby grid vertices.
  • Acceleration Effects: A roughly 65 ms difference in total sequence time falls within rasterization, other-module variation, and measurement noise.The result suggests that feature caching alone nearly saturates grid-path efficiency.
  • Acceleration Effects: Vertex reuse contributes mainly to memory reduction and long-term runtime savings rather than a large immediate sequence-time improvement.The rendering-time and peak-memory measurements are reported over six N3DV scenes.
  • FiLM Fusion: FiLM fusion improves average CDF reconstruction by +1.01 dB over direct concatenation, but its effect varies by scene.The deformation-module storage differs only slightly between the two settings, so the comparison isolates the fusion design rather than a storage-budget asymmetry.

D. TimeMLP Depth and Time Encoding Format

The ablation supports a three-layer TimeMLP with frequency-based time encoding, while optimization remains practical for offline content creation. The broader discussion identifies sequence-length degradation and several future extensions and design boundaries.

  • D. TimeMLP Depth and Time Encoding Format: Three hidden layers provide the best tested TimeMLP depth: one lowers average PSNR by 1.01 dB, while five lowers it by 0.74 dB.The comparison keeps frequency-based time encoding fixed and evaluates alternatives relative to the default protocol.
  • D. TimeMLP Depth and Time Encoding Format: Replacing frequency-based time encoding with raw t decreases average PSNR by 1.66 dB at the default TimeMLP depth.This indicates that frequency-based encoding materially supports deformation learning under the ablation protocol.
  • D. TimeMLP Depth and Time Encoding Format: The ablation table reports TimeMLP depth and time-encoding-format variants on six N3DV scenes, with ∆PSNR measured relative to the default.Each variant changes one knob while preserving the other settings.
  • E. Optimization Cost: On a single RTX 4090, CC-4DGS requires roughly 64–65 minutes to optimize a standard 300-frame N3DV scene, compared with about 25 minutes for Swift4D and 60 minutes for 4DGS and Grid4D.The reported runtime is comparable to other dynamic 3DGS methods and below 1.5 hours per scene.
  • VII. Limitations and Future Directions: CC-4DGS and Swift4D gradually lose quality as sequence length grows, reflecting the capacity limits of fixed canonical point sets and deformation pathways.This limitation concerns the canonical-plus-deformation 4DGS family.
  • VII. Limitations and Future Directions: GoP-based methods such as 4DGV and 4DGC offer a streaming-oriented alternative by partitioning sequences into per-segment models, while GoP segmentation within CC-4DGS remains future work.The alternative trades training-time integration for streaming-friendly per-segment storage.
  • VII. Limitations and Future Directions: The computation-over-storage principle could extend to Swift4D, Grid4D, and 4DGV, which also use trainable deformation hash grids.The proposed extension replaces stored trainable hash tables with small neural decoders evaluated on the fly.
  • VII. Limitations and Future Directions: Static representations and hash-grid compression schemes require separate designs because their rendering-query counts, entropy objectives, and compression baselines differ.The paper does not treat them as direct ports of the dynamic 4DGS approach.
Loading 2609.02184v1…