PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoGSplat: Geometry-Grounded Non-Pixel-Aligned Gaussian Splatting

Jiale Wang, Chen Zhao, Haoyang Zhang, Yanhao Wu, Fei He, Yanhu Shan, Rui Wu, Wei Ke, Tong Zhang

arXiv:2610.36760v1cs.CV

TL;DR

Pixel-aligned feed-forward Gaussian Splatting scales primitive count with image resolution and views, while fixed-count non-pixel-aligned methods lack explicit geometric constraints. GeoGSplat combines Anchors and Triplane in a geometry-grounded encoder and hierarchical decoder. It outperforms both method families on DTU and NRGBD for novel view synthesis and reconstruction, with a greater advantage under large viewpoint changes.

  • Problem

    Pixel-aligned prediction increases primitive count with image resolution and input views, while fixed-count non-pixel-aligned methods lack ray-and-depth constraints that support accurate Gaussian centers.

    Source: §1 · p. 2, §2 · p. 3

  • Method

    GeoGSplat combines coarse geometric Anchors and fine-grained Triplane features, then hierarchically decodes dense 3D points and Gaussian primitives under Anchor-based constraints.

    Source: §1 · pp. 2–3

  • Results

    GeoGSplat outperforms pixel-aligned and non-pixel-aligned methods on DTU and NRGBD for novel view synthesis and 3D reconstruction, with a greater advantage under larger viewpoint changes.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The reported results support explicit geometric grounding as important for robust and generalizable feed-forward Gaussian Splatting, particularly under large viewpoint changes.

    Source: §5 · p. 10

Abstract

from arXiv · show

Feed-forward Gaussian Splatting enables efficient novel view synthesis, but pixel-aligned prediction makes the primitive count grow with image resolution and view count, leading to substantial redundancy and increased computational cost. Non-pixel-aligned methods address this limitation by predicting a fixed number of Gaussians. Without explicit ray and depth constraints, these methods yield misplaced Gaussian centers, causing rendering artifacts under large viewpoint changes. To address this issue, we introduce GeoGSplat, a geometry-grounded feed-forward model that explicitly incorporates geometric priors into the non-pixel-aligned framework. A dual-branch encoder constructs Anchors and Triplane to capture coarse 3D structure and fine-grained scene features, respectively. Subsequently, a hierarchical decoder progressively decodes dense 3D points and Gaussian primitives, enabling 3D reconstruction and novel view synthesis. GeoGSplat surpasses both state-of-the-art pixel-aligned and non-pixel-aligned Gaussian Splatting methods on DTU and NRGBD in terms of novel view synthesis and reconstruction. The advantage of our geometry-grounded paradigm further increases as target viewpoints move farther from the input cameras, demonstrating stronger generalization to large viewpoint changes. Project Page: https://comedy0913.github.io/projects/geogsplat

1 INTRODUCTION

Pixel-aligned prediction provides geometric constraints but scales primitive count with image resolution and views, while fixed-count non-pixel-aligned methods lack those constraints. GeoGSplat combines fixed-size prediction with geometry-grounded encoding and hierarchical decoding, and outperforms both method families on DTU and NRGBD, especially under larger viewpoint changes.

1 INTRODUCTION

  • Pixel-aligned methods gain geometric constraints from per-pixel depth but increase primitive count with input views, whereas non-pixel-aligned methods fix the count but lose those constraints.The paradigm comparison also identifies redundancy from overlapping image regions in pixel-aligned methods.Source: §1 · p. 2, Figure 2 · p. 2
  • GeoGSplat uses a dual-branch encoder to construct Anchors for coarse 3D structure and Triplane for fine-grained scene features.Anchors are sparse 3D spheres with learnable centers and adaptive radii; Triplane consists of three orthogonal 2D feature maps queryable at arbitrary 3D coordinates.Source: §1 · p. 2
  • Its hierarchical decoder samples Triplane features within Anchor regions to predict dense 3D points, then re-queries the Triplane at those points to predict Gaussian primitives.Anchor geometry constrains point reconstruction while fine-grained Triplane features support geometric refinement and appearance learning.Source: §1 · pp. 2–3
  • GeoGSplat outperforms pixel-aligned and non-pixel-aligned competitors on DTU and NRGBD for novel view synthesis and 3D reconstruction, with a larger advantage under greater viewpoint changes.DTU evaluates object-level scenes and NRGBD evaluates scene-level scenes.Source: §1 · p. 2

2 RELATED WORKS

Prior feed-forward Gaussian Splatting methods trade pixel-to-Gaussian geometric constraints for fixed-count prediction, or retain those constraints at the cost of scaling and redundancy. GeoGSplat combines coarse geometric Anchors with fine-grained Triplane features.

Pixel-aligned feed-forward Gaussian Splatting

  • Pixel-aligned methods use camera rays and per-pixel depths to constrain Gaussian centers, but their primitive count scales with image resolution and input views, increasing redundancy and computational cost.Overlapping regions across images can produce multiple Gaussians for the same scene content.Source: §2 · p. 3

Non-pixel-aligned models

  • Non-pixel-aligned methods predict Gaussians from aggregated features or scene representations, fixing primitive count but losing ray-and-depth constraints that help locate centers.Their geometry errors become more exposed under large viewpoint changes, where they can produce strong rendering artifacts.Source: §2 · p. 3

Scene Representations

  • GeoGSplat combines Anchors for coarse geometric constraints with Triplane for fine-grained scene features.The paper motivates this pairing by the complementary roles of spatial constraints and queryable structured features.Source: §2 · p. 3

3 METHOD

GeoGSplat keeps Gaussian count fixed while restoring explicit geometric priors through Anchors, Triplane features, and geometry-guided decoding. Its training combines geometry and rendering supervision.

3.1 PRELIMINARIES

  • Non-pixel-aligned prediction fixes Gaussian count but omits geometric priors; GeoGSplat uses fixed-count Anchors and Triplane features to retain geometric constraints.With M Anchors and K decoded points per Anchor, N = M × K is independent of input-view count and image resolution.Source: §3.1 · pp. 3–4

3.2 DUAL-BRANCH STRUCTURED ENCODER

  • The dual-branch encoder aggregates image features into Anchors for coarse geometry and progressively refined Triplane maps for fine-grained features queryable at 3D coordinates.Anchor radii adapt to scene geometry and image content; Triplane refinement increases resolution from S0 to 4S0.Source: §3.2 · pp. 4–5

3.3 GEOMETRY-GUIDED HIERARCHICAL DECODER

  • The hierarchical decoder expands each Anchor into dense 3D points using sampled Triplane features, then predicts Gaussian primitives from features queried at those points.Anchor-to-Point decoding refines points within Anchor regions; Point-to-Gaussian decoding uses their coordinates and sampled Triplane features.Source: §3.3 · p. 5

3.4 TRAINING LOSS

  • Training combines symmetric Chamfer-distance geometry supervision for Anchor centers and dense points with photometric rendering loss.The model first trains with geometry supervision alone for 15K iterations, then adds rendering loss to jointly optimize geometry and appearance.Source: §3.4 · p. 6

4 EXPERIMENTS

Experiments compare GeoGSplat with pixel-aligned and non-pixel-aligned methods on novel view synthesis and 3D reconstruction, then test viewpoint changes and ablate key components. Results favor GeoGSplat across both datasets and show a widening advantage over TokenGS at greater viewpoint distances.

4.1 EXPERIMENTAL SETUP

  • Training uses four 256×256 input images and 65,536 Gaussians; evaluations use DTU and NRGBD with four context views by default.Ablations use a smaller model for computational efficiency.Source: §4.1 · p. 6, §4.4 · p. 9

4.2 NOVEL VIEW SYNTHESIS AND GEOMETRIC RECONSTRUCTION

  • GeoGSplat achieves the best PSNR, SSIM, and LPIPS among compared methods and the lowest reconstruction CD on DTU and NRGBD.The comparisons use four input views.Source: §4.2 · p. 7, Table 1 · p. 7, Table 2 · p. 7
  • Versus TokenGS, DTU PSNR rises from 17.71 to 20.48 dB and LPIPS falls from 0.354 to 0.137; on NRGBD, PSNR rises from 23.82 to 24.78 and LPIPS falls from 0.173 to 0.120.Source: §4.2 · p. 7
  • Compared with TokenGS, CD falls from 0.082 to 0.018 on DTU and from 0.277 to 0.017 on NRGBD.GeoGSplat also lowers CD versus pixel-aligned baselines, including AnySplat∗.Source: §4.2 · p. 7, Table 2 · p. 7
  • Qualitatively, GeoGSplat preserves fine details and faithful geometry, while TokenGS blurs details and AnySplat∗ produces shifted or duplicated structures.The figure compares ground truth, AnySplat∗, TokenGS, and GeoGSplat.Source: §4.2 · p. 7, Figure 4 · p. 8

4.3 HOW GEOMETRY AFFECTS NOVEL VIEW SYNTHESIS

  • As target-context distance increases, GeoGSplat maintains higher PSNR and its performance gap over TokenGS grows; their performance is similar for nearby views.The comparison uses fixed context views and target-view trajectories on NRGBD.Source: §4.3 · p. 8, Figure 5 · p. 8

4.4 ABLATION STUDIES

  • Removing Anchors or Triplane drops PSNR by more than 1 dB on both datasets; removing either Triplane-query decoding stage also worsens performance.The ablations support the complementary encoder representations and progressive two-stage decoding.Source: §4.4 · p. 9
  • Removing geometry losses causes severe performance drops; removing either loss also degrades results, with dense-point supervision having the larger impact.Full achieves the lowest CD, supporting supervision of both Anchor centers and dense points.Source: §4.4 · p. 9, Table 3 · p. 9

5 CONCLUSION

GeoGSplat combines a fixed Gaussian budget and explicit geometric priors with a dual-branch encoder and geometry-guided hierarchical decoder. On DTU and NRGBD, it improves NVS and 3D reconstruction over pixel-aligned and non-pixel-aligned baselines, with larger advantages under large viewpoint changes.

5 CONCLUSION

  • GeoGSplat maintains a fixed Gaussian budget while combining Anchors and Triplane with a decoder that progressively reconstructs dense 3D points before predicting Gaussian primitives.Experiments on DTU and NRGBD show improvements in NVS and 3D reconstruction over pixel-aligned and non-pixel-aligned baselines, with a larger advantage under large viewpoint changes.Source: §5 · pp. 9–10

A.1 COMPARISON WITH RETRAINED TOKENGS

The retrained TokenGS comparison indicates that GeoGSplat’s gains are not explained simply by additional training data or geometric supervision. GeoGSplat also maintains leading performance across 2- and 12-view inputs without retraining, using the same fixed set of 65K Gaussians.

A.1 COMPARISON WITH RETRAINED TOKENGS

  • Both retrained TokenGS variants perform substantially worse than GeoGSplat on both datasets, although adding geometry loss improves their NVS and geometric accuracy.The comparison uses the same training data, input resolution, Gaussian budget, and schedule; the authors attribute GeoGSplat’s remaining advantage to its architecture.Source: §A.1 · p. 14, Table 4 · p. 14

A.2 GENERALIZATION ACROSS INPUT VIEW COUNTS

  • With 12 input views, GeoGSplat reaches 22.53 dB PSNR and 0.014 CD on DTU, and 25.21 dB PSNR and 0.015 CD on NRGBD, without retraining.With 2 views, it has the highest PSNR on both datasets and the lowest CD on NRGBD; it uses 65K Gaussians for both input settings.Source: Table 5 · p. 14, §A.2 · p. 14

A.3 EXPERIMENTS ON MIP-NERF 360

The additional experiments report strong Mip-NeRF 360 results and near-identical geometry measurements for dense points and final Gaussian centers. The appendix also describes the shared attention module, Triplane refinement, and fixed reference initialization.

A.3 EXPERIMENTS ON MIP-NERF 360

  • 21.59 dB PSNR and 0.015 CD are GeoGSplat’s best PSNR and lowest CD on Mip-NeRF 360 with 12 input views.The authors present this result as evidence of generalization to large-scale scenes.Source: §A.3 · p. 14, Table 6 · p. 15

A.4 CHAMFER DISTANCE COMPARISON BETWEEN DENSE POINTS AND GAUSSIAN CENTERS

  • Chamfer distances for dense points and final Gaussian centers are nearly identical across settings, while predicted offsets adjust centers for rendering quality.The authors state that the offsets preserve the dense points’ reconstructed geometry.Source: Table 7 · p. 15, §A.4 · p. 15

B.1 SHARED ATTENTION MODULE

  • The shared attention module applies cross-attention to aggregate image features, then self-attention among query tokens, followed by an FFN.The Anchor and Triplane branches share the module but use independent query tokens.Source: §B.1 · p. 15

B.2 TRIPLANE REFINEMENT BLOCK

  • Each Triplane refinement stage doubles resolution, applies residual convolutions, fuses information across planes, and re-injects image evidence through cross-attention.The refinement progresses through resolutions 64, 128, and 256.Source: §B.2 · pp. 15–16

A.5 ADDITIONAL VISUALIZATIONS

  • Figure 7 compares reconstruction and novel view synthesis; GeoGSplat is reported to produce cleaner Gaussian centers and higher-quality images than TokenGS.Source: §A.5 · p. 15, Figure 7 · p. 16

B.3 REFERENCE INITIALIZATION

  • Anchor centers and Anchor-to-Point offsets use references sampled once at initialization and then kept fixed or shared during training.The same set of offsets is shared across Anchors and training samples.Source: §B.3 · pp. 16–17

B.4 DATASET-SPECIFIC VIEW SAMPLING

Input-view sampling is adapted to how each dataset organizes its camera views.

B.4 DATASET-SPECIFIC VIEW SAMPLING

  • Four input views are sampled using dataset-specific strategies: pose-distance sampling for ScanNet++ and ARKitScenes, temporal-gap sampling for BlendedMVS, and uniform distinct-view sampling for CO3D and WildRGB-D.Source: §B.4 · p. 17

B.5 NORMALIZED GEOMETRIC EVALUATION

Geometry is evaluated in a shared normalized frame defined by the context cameras and ground-truth geometry. Viewpoint change is measured by target-context distance, with PSNR restricted to reconstructed regions visible in the inputs.

B.5 NORMALIZED GEOMETRIC EVALUATION

  • Ground-truth and predicted points are evaluated in a shared coordinate frame oriented and centered by context cameras, scaled by the ground-truth point cloud, and normalized to [−1, 1]3.Predictions are transformed into this frame using the known correspondence between context cameras.Source: §B.5 · p. 17

B.6 TARGET-CONTEXT DISTANCE

  • Target-context distance is the Euclidean distance from a target camera center to its nearest context camera center.Source: §B.6 · p. 17
  • PSNR is computed only on target pixels visible in at least one context view, excluding regions unseen in all input views.Source: §B.6 · p. 17

B.7 DETAILED MODEL ARCHITECTURE

Experiments use a smaller model for ablations and a main-table model for comparisons with prior methods; Table 8 summarizes their architectures.

B.7 DETAILED MODEL ARCHITECTURE

  • A smaller model is used for ablations to reduce computational cost, while the main-table model is used for comparisons with prior methods.Their architectures are summarized in Table 8; the supplied passage does not specify the architectural differences.Source: §B.7 · p. 17, Table 8 · p. 17

Related papers

Loading 2610.36760v1…