PaperlayerView exact revision on arXiv

Source-linked AI summary

Affine-Aligned Atlas for Canonical Gaussian Construction in Video Representation

Masaya Takabe, Hiroshi Watanabe, Sujun Hong, Tomohiro Ikai

arXiv:2610.01114v1cs.CV

TL;DR

Large global motion can misalign GOP frames in canonical-Gaussian video representations and increase the burden on temporal deformation. This paper aligns frames in an affine atlas before canonical construction, improving reconstruction especially for sequences with large camera motion while adding negligible parameter cost.

Abstract

from arXiv · show

Gaussian splatting has recently emerged as an efficient representation for images and videos due to its explicit structure and fast rendering capability. Existing Gaussian-based video representations often decompose a video into canonical Gaussians and temporal deformation. However, when a video contains large global motion such as camera movement, the canonical representation may become misaligned with individual frames, increasing the burden on the temporal deformation model. In this paper, we propose an affine-atlas canonical Gaussian representation, which constructs canonical Gaussians in a larger affine-aligned atlas space. Frame-wise affine transforms absorb global motion before canonical Gaussian construction, reducing the gap between the canonical representation and target frames. Since the proposed method only modifies the canonical construction stage, it can be integrated into existing canonical-Gaussian-based methods with negligible additional parameter cost. Experiments show that our method improves reconstruction quality especially for sequences with large camera motion.

I. INTRODUCTION

Canonical-Gaussian video methods separate shared spatial structure from temporal deformation, but large motion can leave GOP frames misaligned. The affine-atlas approach aligns frames before canonical construction, reducing this mismatch with negligible added parameter cost.

I. INTRODUCTION

  • Large camera or rigid motion can misalign frames within a GOP, forcing canonical Gaussians to represent shifted appearances in one space and increasing temporal-model burden.This can reduce representation efficiency and make temporal modeling more difficult.Source: §I · p. 1
  • Frame-wise affine transforms align GOP frames in a larger atlas before canonical construction, absorbing global motion and reducing the gap to target frames.The approach changes only canonical construction and can integrate with existing methods including D2GV [14], GSVR [16], and STGV [17].Source: §I · p. 1, Figure 2 · p. 2
  • The method targets motion-intensive sequences with negligible additional parameter cost.Source: §I · p. 2

A. Neural Video Representation

Video representations trade decoding speed and parameterization against how they capture temporal variation. Canonical-Gaussian methods separate shared spatial content from temporal change, but global motion can make their canonical space suboptimal.

A. Neural Video Representation

  • Neural video representations exploit spatial and temporal redundancy but require repeated network inference, while Gaussian primitives support faster rasterization.Source: A. Neural Video Representation · p. 2

B. Gaussian Splatting for Images and Videos

  • Gaussian video methods model temporal variation through explicit parameterization, spatio-temporal structures, or canonical Gaussians with temporal deformation.The canonical approach separates shared spatial content from temporal variation, a design suited to representation and compression.Source: B. Gaussian Splatting for Images and… · p. 2

C. Canonical-Gaussian-Based Video Representation

  • D2GV [14], GSVR [16], and STGV [17] use canonical Gaussians with different temporal models, but GOP-wide global motion can make the canonical space suboptimal.That misalignment increases the burden on temporal deformation and motivates improving canonical construction by absorbing global motion first.Source: C. Canonical-Gaussian-Based Video… · p. 2

III. PROPOSED METHOD

The method constructs canonical Gaussians in a larger atlas space rather than directly in original frame coordinates. Frame-wise affine transforms align GOP frames before construction, while existing temporal deformation models remain unchanged.

A. Overview

  • A larger atlas space replaces the original frame coordinate space as the location for canonical Gaussian construction.Source: A. Overview · p. 2
  • Each GOP frame receives a six-parameter affine transform, and the frames are aligned in atlas space before canonical construction.Source: A. Overview · p. 2
  • The affine alignment absorbs global translation, rotation, and scaling before the baseline temporal deformation model is applied unchanged.Source: A. Overview · p. 2

B. Affine-Atlas Canonical Space

The atlas defines a canonical Gaussian set over a larger space covering affine-aligned GOP frames. Precomputed transforms project atlas positions into frame coordinates, while Gaussian covariance and color are shared.

B. Affine-Atlas Canonical Space

  • Precomputed affine transforms map atlas coordinates to each target frame and remain fixed during optimization.Source: B. Affine-Atlas Canonical Space · p. 2
  • Table I reports PSNR and MS-SSIM for Big Buck Bunny and UVG, with UVG sequences ordered by estimated affine-motion magnitude and average baseline-to-ours deltas.Source: Table I · p. 3
  • The atlas Gaussian set contains centers, covariance matrices, and colors for its Gaussians.Source: B. Affine-Atlas Canonical Space · pp. 2–3
  • Affine transforms project atlas Gaussian positions into frame coordinates, while covariance and color remain shared from the atlas set.The same atlas Gaussian can thus explain corresponding regions across frames under global motion.Source: B. Affine-Atlas Canonical Space · p. 3

C. Canonical Gaussian Construction

Canonical Gaussian construction optimizes atlas Gaussians to match target GOP frames after affine transformation. The following temporal model deforms the atlas representation before final rendering, focusing on residual local motion and appearance changes.

C. Canonical Gaussian Construction

  • Atlas Gaussian parameters are optimized so rendered images match target GOP frames after the precomputed affine transforms.The reconstruction loss uses an image reconstruction measure such as L2.Source: C. Canonical Gaussian Construction · p. 3
  • The reconstruction loss encourages atlas Gaussians to represent shared GOP content in an aligned coordinate system.The loss function d(·, ·) may be an L2 loss.Source: C. Canonical Gaussian Construction · p. 3
  • During temporal modeling, the baseline deformation model further deforms the atlas, focusing on residual local motion and appearance changes.The formulation reduces the gap between the canonical representation and each target frame.Source: C. Canonical Gaussian Construction · p. 3

A. Experimental Setup

Experiments compare affine-atlas variants with their baselines on Big Buck Bunny and UVG, using 10-frame GOPs and PSNR and MS-SSIM. Reconstruction gains are strongest in sequences with large camera motion, while qualitative comparisons illustrate improved structure preservation and reduced canonical-frame discrepancy.

A. Experimental Setup

  • The evaluation uses 132 Big Buck Bunny frames at 1280 × 720 and UVG sequences at 1920 × 1080, subsampled by five and split into independently reconstructed 10-frame GOPs.Reconstruction quality is measured with PSNR and MS-SSIM; D2GV, GSVR, and STGV are each tested with and without affine-atlas construction.Source: A. Experimental Setup · p. 3
  • Frame-wise affine transforms are precomputed by composing adjacent-frame 2D ORB-feature-matching estimates and are fixed, not learnable, during training.Average adjacent-frame translation magnitude sorts UVG sequences in Table I.Source: A. Experimental Setup · p. 3

B. Results and Analysis

  • Reconstruction quality improves especially on high-camera-motion sequences, with the tendency observed across evaluated baseline methods.The comparison covers Big Buck Bunny and UVG; Table I reports the quantitative results.Source: B. Results and Analysis · p. 3
  • For relatively low-motion sequences, including Big Buck Bunny, Beauty, and HoneyBee, improvement is limited because the original canonical space is already well aligned.In these cases, conventional and affine-atlas canonical Gaussians become similar.Source: B. Results and Analysis · pp. 3–4
  • Affine-atlas variants better preserve local structures in camera-motion regions and reduce discrepancy between canonical renders and target frames.Fig. 3 highlights affected regions; Fig. 4 compares canonical renders and error maps, with baseline blur attributed to fitting misaligned frames in one coordinate space.Source: B. Results and Analysis · p. 4, Figure 3 · p. 4, Figure 4 · p. 4

C. Parameter Cost

The affine-atlas method adds only frame-wise affine matrices and leaves the temporal deformation model unchanged. Its parameter overhead is small relative to the Gaussian and temporal-model parameters.

C. Parameter Cost

  • The method adds only frame-wise affine transform matrices, without adding a neural network or changing the temporal deformation model.Only the canonical construction stage is modified.Source: C. Parameter Cost · p. 4
  • A GOP with T frames adds 6T affine parameters; a 10-frame GOP adds 60.Each 2D affine transform is represented by a 2×3 matrix.Source: C. Parameter Cost · p. 4
  • The added parameters are negligible beside Gaussian and temporal-model parameters, which typically number in the hundreds of thousands per GOP.The paper attributes the gain to better canonical alignment rather than simply increasing model size.Source: C. Parameter Cost · p. 4

V. CONCLUSION

The paper introduces an affine-aligned atlas for canonical Gaussian construction, absorbing global motion before temporal deformation and reducing the gap to target frames. Experiments show improved reconstruction, especially with large camera motion, at negligible parameter cost.

V. CONCLUSION

  • Canonical Gaussians are constructed in a larger affine-aligned atlas space to absorb global motion before temporal deformation.This reduces the gap between the canonical representation and target frames.Source: §V · p. 4
  • Experiments on Big Buck Bunny and UVG show improved reconstruction quality, especially for sequences with large camera motion.The reported parameter cost is negligible.Source: §V · p. 4
  • Future work includes more robust alignment-transform estimation and extending atlas construction beyond affine transformations.Source: §V · p. 4

Related papers

Loading 2610.01114v1…