PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Gaussian Splatting Sequence Compression with Standard Video Codecs

Qi Yang, Shuting Xia, Le Yang, Geert Van Der Auwera, Zhu Li

arXiv:2610.07795v1cs.CV

TL;DR

Gaussian Splatting’s explicit primitive format creates large data volumes, and video-based sequence coding can be undermined by disrupted primitive correspondence. GSCV combines Inter-PLAS with high-bit-depth images and video codecs, reporting better performance than video- and point-cloud-based anchors on tracked and semi-tracked sequences.

  • Problem

    GS has large data volumes, and permutation changes or dissimilar attributes can undermine the spatial correspondence assumed by video-based sequence coding.

    Source: §1 · pp. 1–2

  • Method

    GSCV uses Inter-PLAS to align adjacent-frame maps and a high-bit-depth GS image compression pipeline based on state-of-the-art video codecs.

    Source: §1 · p. 2

  • Results

    GSCV reports better performance than video- and point-cloud-based anchors on tracked and semi-tracked sequences.

    Source: §6 · p. 7

  • Takeaways & Limitations

    High-bit-depth images provide wider bitrate and quality ranges for the video sequence pipeline.

    Source: §1 · p. 2

    Generic video codecs restrict GSCV’s compression ratios for PLAS images, and Inter-PLAS cannot completely recover primitive correspondence in tracked data after permutation.

    Source: §6 · p. 7

Abstract

from arXiv · show

This paper presents a novel effective Gaussian Splatting (GS) sequence Compression method that utilizes the Video codec (GSCV). Existing video-based GS sequence compression relies on the Parallel Linear Assignment Sorting (PLAS) and tracked primitive information to convert GS into smooth 2D videos. However, tracked information is not available for most practical applications, and without it, using the vanilla PLAS can generate images exhibiting weak inter-frame correlation, due to its stochastic nature. GSCV incorporates a simple yet efficient Inter-PLAS method to produce close images between the I- and P-frames of GS, enhancing the inter-frame performance of video codec greatly. GSCV also realizes a new pipeline based on the state-of-the-art video codecs with high bit-depth GS images, achieving higher compressibility while simultaneously providing a higher quality upper bound. Experimental results show that the proposed GSCV exhibits obviously improved performance over MPEG video and point cloud-based anchors in GS sequence compression. The code is available at https://github.com/Qi-Yangsjtu/GSCV.

1 Introduction

GSCV addresses weaknesses in video-based Gaussian Splatting sequence compression by improving inter-frame map alignment and using a high-bit-depth video-codec pipeline. It reports better performance than video- and point-cloud-based anchors on tracked and semi-tracked sequences.

1 Introduction

  • GS compression includes optimization-based A-3DGS and optimization-free I-3DGS, with lightweight requirements drawing attention to canonical codecs.A-3DGS trains models during encoding, whereas I-3DGS compresses trained data; optimization-based methods can involve complex coding and rigid GPU-based rate control.Source: §1 · p. 1
  • GSCodec Studio applies the I-frame PLAS index to following P-frames, assuming primitive-index spatial correspondence and smooth, close P-frame maps.The approach generates videos within a group of pictures using the I-frame PLAS index for subsequent frames.Source: §1 · pp. 1–2
  • Primitive shuffling can disrupt index correspondence, while alpha-blending means index-matched neighbors may still have substantially different attributes.The passage also notes that 16-bit coordinates and 8-bit quantization of other attributes can limit quality for complex scenes.Source: §1 · p. 2
  • GSCV uses Inter-PLAS to make adjacent-frame maps closer and a high-bit-depth image pipeline compatible with state-of-the-art video codecs.Its spatial-stable initialization addresses permutation invariance, and anchor-based refinement optimizes P-frame maps for inter-frame prediction.Source: §1 · p. 2
  • GSCV outperforms video- and point-cloud-based anchors on tracked and semi-tracked sequences.The comparison is reported for sequences in the current MPEG standardization study.Source: §1 · p. 2

2 Brief Review of PLAS

PLAS maps Gaussian Splatting attributes onto 2D grids and progressively rearranges pixels to create smooth images for video-codec compression.

2 Brief Review of PLAS

  • PLAS first prunes a minimal set of primitives when needed, then randomly sorts remaining attributes onto 2D grids.It produces m images from selected reference attributes; values at the same position across images come from the same primitive, so no extra alignment signaling is needed.Source: §2 · p. 2
  • PLAS selects blur-window radii from image resolution, decays them by τ, and derives corresponding block sizes for successive iterations.The minimum radius is 1, and vanilla PLAS sets the minimum block size to 16.Source: §2 · pp. 2–3
  • At each iteration, PLAS blurs the preceding image and permutes groups of four pixels within blocks to minimize L2 loss against the blurred target.As blur radii and block sizes decrease, the process moves from coarse to fine and yields a smoother final image.Source: §2 · p. 3

3 Inter-PLAS image Generation

Inter-PLAS addresses PLAS’s unstable frame-to-frame results with spatial-stable initialization (SSI) and anchor-based refinement. The two steps make P-frame images more similar to I-frame images, improving inter-prediction potential.

3 Inter-PLAS image Generation

  • Inter-PLAS combines SSI and anchor-based PLAS refinement to improve inter-prediction, addressing different images produced for identical or close GS content by vanilla PLAS.Vanilla PLAS uses random initialization and permutation, which can make its results unsuitable for inter-prediction.Source: §3 · p. 3

3.1 Spatial-stable initialization (SSI)

  • Morton-code sorting provides a lightweight, stable initialization that produces more similar I- and P-frame images; the P-frame converges in about 3,000 iterations versus 4,000 for the I-frame.Sorting takes around 0.1 seconds for GS with half a million primitives, and initialization preserves close I-frame all-intra compression performance.Source: §3.1 · pp. 3–4

3.2 Anchor-based PLAS refinement

  • Anchor-based refinement uses the I-frame PLAS image as the P-frame target, increasing prediction gain and enabling over 20% bitrate savings for P-frame coding.Direct smooth-image targeting conflicts with PLAS’s coarse-to-fine process and can slow convergence.Source: §3.2 · p. 4

4 Gaussian splatting sequence coding using video codec

GSCV organizes GS frames into groups of pictures and represents attributes as 2D images for video coding. It splits high-bit-depth coordinates for codec compatibility and uses different coding modes for coordinates and other attributes.

4 Gaussian splatting sequence coding using video codec

  • Each eight-frame group uses one I-frame and seven P-frames, with SSI plus vanilla PLAS for the I-frame and Inter-PLAS for P-frames to project primitives onto 2D grids.Coordinates are quantized at 20 bits and other attributes at 10 bits.Source: §4 · p. 4
  • GSCV generates 22 images per GS frame, splitting 20-bit coordinates into 10-bit high- and low-part images, then groups images across each GoP into 22 YUV444 videos.Coordinates use lossless compression, while other attributes use lossy compression.Source: §4 · p. 4
  • The pipeline treats video data as YUV directly, avoiding RGB-to-YUV conversion because it offers no compression benefit for most attributes and can introduce quality-degrading calculation errors.The text identifies coordinate high-part images at high bitrate as especially susceptible to degradation from the conversion.Source: §4 · p. 5

5 Experiments

On MPEG tracked and semi-tracked sequences, GSCV outperforms GSCodec Studio and shows strong performance against GPCC v1, with results varying by bitrate. The evaluation also tests initialization and Inter-PLAS ablations.

5.1 Dataset and GSCV Implementation

  • The evaluation uses three 32-frame MPEG sequences in tracked and semi-tracked forms, groups frames into eight-frame GoPs, and randomly shuffles tracked data before testing.The compared video-codec anchors are HM v18.0 and VTM v23.11.Source: §5 · p. 5, §5.1 · p. 5

5.2 Overall evaluation

  • GSCV performs better than GPCC v1 at middle-to-high bitstreams but worse at low bitstreams, and reports consistent performance across tracked and semi-tracked sequences.The paper notes semi-tracked sequences are more challenging, while random shuffling of tracked data simulates correspondence instability from preprocessing.Source: §5.2 · pp. 5–6
  • On the MPEG tracked and semi-tracked dataset, GSCV shows obvious superiority over GSCodec Studio and GPCC v1, with around 8 dB gain over GSCodec Studio on “bartender” at 10 MB.Source: §5.2 · p. 5

5.3.1 Influence of PLAS initialization on image generation and com-

  • Initialization variants yield nearly identical all-intra rate–distortion curves, while using another frame as context gives the shortest PLAS time; Inter-PLAS improves inter-prediction over vanilla PLAS and SSI alone.The ablations compare random, Morton-based, and cross-frame-context initialization, then compare separate vanilla PLAS, SSI initialization, and Inter-PLAS refinement.Source: §5.3.1 · p. 6, §5.3.2 · p. 6, Figure 8 · p. 6

6 Conclusions

GSCV combines Inter-PLAS with a video-codec pipeline and reports better performance than video- and point-cloud-based anchors on tracked and semi-tracked sequences. The conclusion also notes limits from PLAS image characteristics, underused feature correlations, and heuristic correspondence recovery.

6 Conclusions

  • GSCV reports better performance than video- and point-cloud-based anchors on tracked and semi-tracked sequences.The pipeline uses high-bit-depth images and is compatible with prevalent video codecs.Source: §6 · pp. 6–7
  • PLAS images differ from natural images, restricting compression ratios with generic codecs; separate video streams underuse intra-channel correlations, especially for color SH.For tracked data after permutation, Inter-PLAS cannot completely recover primitive correspondence because it inherits PLAS’s heuristic mechanism.Source: §6 · p. 7

A Related Work

  • A-3DGS methods compact representations through training constraints, while I-3DGS methods compress trained GS data without training optimization.The related-work discussion describes pruning, anchor-based and codebook approaches in A-3DGS, and voxelization, attribute coding, pruning, and learned compression in I-3DGS.Source: A Related Work · p. 9

B Bitrate Setting Details

  • The bitrate-setting appendix lists QPs for five GSCV bitstreams and HEVC/VVC configurations for lossy and lossless compression.The supplied text also mentions an HM18.0 low-delay configuration with different coding sequencing.Source: B Bitrate Setting Details · p. 9

C Baseline Anchors

  • GSCodec Studio uses FFMPEG x265 with GoP-wise inter-prediction; GPCC v1 uses octree coordinate coding and RAHT attribute coding and is tested in all-intra mode.The experiment changes GSCodec Studio’s default GoP size from 16 to 8 for fair comparison; GPCC v1 uses 18-bit coordinates and 12-bit other attributes.Source: C Baseline Anchors · p. 9

D Dataset Information

The evaluation covers tracked and semi-tracked MPEG sequences and reports rate-distortion and bitrate-allocation analyses. Its redistribution experiments show that component-specific quantization and chroma format affect the quality-bitrate trade-off.

D Dataset Information

  • The dataset includes tracked and semi-tracked versions of “bartender,” “breakfast,” and “cinema”; each sequence has 32 frames, with the first eight used as one GoP.Tracked sequences keep the same primitive count across frames; semi-tracked versions allow densification during refinement and have more primitives.Source: D Dataset Information · p. 9

F Bitrate Allocation

  • At high BPP, color SH uses more bits, while geometry-related features contribute more at low BPP; decoding is faster than encoding.The coding-time illustration uses tracked “bartender” rates 1, 3, and 5 with HEVC.Source: F Bitrate Allocation · p. 9, Figure 10 · p. 10

E BPP-based RD Curve

  • Table 2 reports BD-Rate results for HEVC and VVC, with negative values indicating bitrate savings relative to the corresponding reference method.The text also identifies BPP and BD-Rate as the reported performance measures.Source: E BPP-based RD Curve · p. 9, Table 2 · p. 10

F Bitrate Allocation

  • The HM/VTM implementation is relatively slow without engineering optimizations such as multithreading; optimized codec platforms can run faster.The reported coding-time figures are for complexity illustration.Source: F Bitrate Allocation · p. 10

G Bitrate Redistribution

  • C2 performs better than C1 at low-to-medium bitrates, while YUV420 for color SH improves those rates but substantially lowers the quality ceiling; YUV444 is preferred for high quality.The C2 setting uses QP 7 for color DC in rates R01–R03.Source: G Bitrate Redistribution · p. 10

H Influence of Data shuffling

The experiments examine data shuffling, refinement settings, feature channels, coding order, and dynamic decay. They show that performance and runtime depend on primitive alignment and on choices in Inter-PLAS and video coding.

H Influence of Data shuffling

  • Shuffling or pruning makes GSCodec Studio’s performance markedly worse: matching the unshuffled PSNR requires four times the bitrate.The method relies on I-frame index inheritance, which becomes invalid when I- and P-frame primitive alignment is disrupted.Source: H Influence of Data shuffling · p. 10

I Effectiveness of Anchor-based PLAS refinement

J All-Intra Performance

  • GSCV reports better all-intra performance than GSCodec Studio, while FCGS performs better than the other compared methods and requires GPUs for compression.HGSC also uses GPUs for pruning; GSCV, GSCodec Studio, and GPCC v1 are described as CPU-based and lightweight, although GSCV and GSCodec Studio require GPU-based PLAS.Source: J All-Intra Performance · p. 11

L Influence of Compression Sequencing

  • RandomAccess coding is slightly better than low-delay coding, with 3.63% RD gains; P-frame coding avoids dependence on future frames and is more suitable for real-time applications.The comparison uses B-frames in RandomAccess mode and P-frames in low-delay mode.Source: L Influence of Compression Sequencing · p. 12

M The Influence of Dynamic Decay Factor

  • A fixed τ of 0.95 performs best, while increasing τ yields around 2.12% gain over decreasing it; for relatively long sequences, the increasing-τ case is preferred at similar generation time.The analysis relates greater distance from I-frames to greater P-frame differences and a need for larger τ.Source: M The Influence of Dynamic Decay Factor · p. 13

N Residual Analysis

The residual analysis contrasts PLAS images with natural images and relates their distributions to video-codec behavior. The component analysis finds that rendered quality responds more strongly to opacity, scaling, and rotation than to color components.

N Residual Analysis

  • Natural images have the smallest residual standard deviation and histogram entropy, while PLAS images show heavier tails and lack semantic texture, consistent with challenging compression and close HM18.0/VTM23.11 performance.The analysis fits Gaussian and Laplace curves to residual statistics.Source: N Residual Analysis · pp. 13–14

O RD curve for different components

  • The component RD analysis covers color DC, color SH, opacity, scaling, and rotation; some vertically varying points arise when adjacent bitrate points share a QP for one feature but not others.The component curves are shown in Fig. 20.Source: O RD curve for different components · p. 14
  • Rendered quality is more sensitive to opacity, scaling, and rotation: smaller QPs for these components yield PSNR close to larger QPs for color DC and SH.The paper therefore uses relatively smaller QPs for opacity, scaling, and rotation in its main results.Source: O RD curve for different components · p. 14

P Influence of video codec types

Codec choice affects GSCV’s rate-distortion behavior and speed, while comparisons show trade-offs with A-3DGS methods and gains from a Sandwich network. The reported codec behavior is tied to PLAS images’ characteristics.

P Influence of video codec types

  • Video codecs decode faster than point-cloud codecs, and GSCV with libx265 encodes and decodes faster than GSCodec Studio, supporting FFMPEG for low-delay use cases.Table 5 reports encoding and decoding time for one GoP of eight frames.Source: Table 5 · p. 15, P Influence of video codec types · p. 16
  • FFMPEG libx265 and libx264 perform similarly, but HM18.0 outperforms libx265; PLAS’s texture and uneven smoothness may limit intra-mode gains and make partition search valuable.The analysis attributes libx265’s weaker performance partly to early skip or termination during coding-unit partitioning.Source: P Influence of video codec types · pp. 14–16, Figure 22 · p. 16

Q Comparison with A-3DGS based method

R Influence of using Sandwiched Network

S Results on more sequences

On the generated dance and basketball sequences, GSCV-HEVC outperforms GSCodec Studio and GPCC v1. The sequences are constructed from multiview video and generated GS samples.

S Results on more sequences

  • The dance and basketball sequences provide multiview videos, and the first eight frames are used to generate COLMAP files and 3D GS samples.The sequences are dance_dunhuang_pair from PKU-DyMVHumans and basketball from AVS-VRU.Source: S Results on more sequences · p. 17
  • For each time point, a canonical 3D GS model is fine-tuned using ground-truth images after sparse point clouds are merged and uniformly downsampled.The canonical model is trained with ground-truth images from all viewpoints and time points.Source: S Results on more sequences · p. 17
  • GSCV demonstrates better performance than GSCodec Studio and GPCC v1 on dance and basketball RD curves.The results are presented in Figure 24.Source: S Results on more sequences · p. 17

T Sample Visualization

The sample visualizations show ground truth and rates 5, 3, and 1 for three MPEG tracked GS sequences.

T Sample Visualization

Loading 2610.07795v1…