Source-linked AI summary
InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis
Jiawei Wang, Hao Yu, Yongzhen Hu, Xinyi Yang, Tao Ni, Xin Zhan, Junbo Chen, Xiaowei Zhou, Ruizhen Hu, Sida Peng
TL;DR
Single-image feed-forward 3DGS must generate coherent renderable scenes despite pixel-aligned primitives that weaken under large viewpoint shifts. InfiniSplat uses geometry-guided supports and query-conditioned implicit decoding to form a surface-aligned representation, achieving state-of-the-art cross-dataset results while retaining limitations from single-view ambiguity and imperfect depth priors.
Problem
Pixel-aligned single-image feed-forward 3DGS can produce promising nearby views but struggles to preserve coherent structures under large viewpoint shifts.
Method
InfiniSplat samples 2D supports from depth-induced surface structure and uses a query-conditioned implicit decoder to predict Gaussian attributes from queried image features.
Results
InfiniSplat achieves state-of-the-art quantitative results against feed-forward baselines across cross-dataset evaluations on ETH3D, ScanNet++, Tanks and Temples, and DL3DV.
Takeaways & Limitations
The results support moving single-image 3DGS beyond pixel-aligned splat expansion toward a more structurally stable, surface-aligned scene representation.
Takeaways & Limitations
Single-view ambiguity and imperfect pretrained depth priors can cause incomplete, stretched, or hallucinated geometry, especially in unseen regions and difficult scene types.
Abstract
from arXiv · showhide
Single-image feed-forward 3D Gaussian Splatting (3DGS) aims to directly generate a renderable 3D scene representation from one input image, avoiding the cost of multi-view capture and per-scene optimization. However, existing methods are often constrained by a pixel-aligned representation, where Gaussians are predicted from fixed image-grid locations. Such pixel-aligned primitives can produce promising nearby-view renderings, but they remain weakly coupled to underlying scene surfaces and struggle to preserve coherent structures under large viewpoint shifts. We present InfiniSplat, a feed-forward single-image 3DGS framework that moves from a pixel-aligned representation toward a surface-aligned representation. InfiniSplat constructs this representation by first using geometry-guided sampling to place 2D supports according to depth-induced local surface structure, and then applying a query-conditioned implicit decoder to predict Gaussian attributes from the image features queried at these supports. By grounding support locations in geometry while decoupling Gaussian prediction from fixed pixel centers, InfiniSplat produces Gaussian layouts that better follow scene surfaces and reduce scattered primitives caused by grid discretization. Across multiple cross-dataset NVS evaluations, InfiniSplat achieves state-of-the-art performance compared with single-image feed-forward baselines, and demonstrates zero-shot generalization from Hypersim indoor synthetic training to complex open-world scenes. Project page: https://zju3dv.github.io/InfiniSplat.
1 Introduction
InfiniSplat addresses the pixel-aligned bottleneck in single-image feed-forward 3DGS by organizing Gaussian supports around surface structure and decoding attributes implicitly. It targets coherent scene representations and stable rendering under large viewpoint changes.
- Motivation: Single-image feed-forward 3DGS seeks rapid renderable scene generation from one photograph for applications including spatial browsing, AR/VR, and handheld displays.The setting avoids multi-view capture and time-consuming per-scene optimization.
- Motivation: Pixel-aligned Gaussian primitives can look convincing near the input view yet reveal tearing, surface breaks, and loose structure under large viewpoint shifts.The limitation arises because primitives are generated from fixed image-grid locations rather than coherent scene surfaces.
- Approach: InfiniSplat moves toward a surface-aligned representation by sampling supports according to depth-induced local surface structures and decoupling attribute prediction from fixed pixel centers.These conditions allow neighboring primitives to assemble into coherent surfaces rather than remain independent local splats.
- Approach: A query-conditioned implicit decoder predicts Gaussian attributes from fused semantic and local-texture features queried at geometry-guided supports.A dual-branch encoder supplies DINO semantic representations and CNN texture cues.
- Results: InfiniSplat achieves state-of-the-art performance across multiple cross-dataset NVS datasets and major image-quality metrics.The reported results support more coherent structures under large-baseline viewpoint changes.
- Contributions: The framework combines geometry-guided support sampling with implicit Gaussian decoding, enabling support sets with different densities and spatial arrangements.The shared prediction function turns sampled supports and queried image features into Gaussian primitives.
2 Related Work
Prior single-image NVS methods span layered representations, generative models, meshes, feed-forward 3DGS, and implicit representations. InfiniSplat is most directly related to feed-forward 3DGS and query-based decoding, but applies implicit decoding to explicitly organized Gaussian scene generation.
- Single-image NVS: Depth warping, soft layering, layered depth images, and multiplane representations provide geometric biases but are limited in viewpoint range and do not directly yield explicit 3D scenes for real-time rendering.These approaches primarily support nearby or moderate viewpoint changes.
- Generative methods: Diffusion-based and transformer-based methods improve scene generation and large-baseline view synthesis, but many synthesize images directly or rely on iterative generation.This distinguishes them from methods that directly produce an explicit renderable 3D representation in one pass.
- Mesh-based methods: Mesh-based single-image-to-3D methods generally target object-centric asset reconstruction or generative 3D creation rather than real-scene NVS.Their emphasis differs from scene-level novel-view synthesis.
- Feed-forward 3DGS: Feed-forward single-image 3DGS methods predict explicit Gaussians in one forward pass but mostly retain a pixel-aligned generation paradigm.They are directly related to InfiniSplat because they target efficient novel-view rendering from one image.
- Implicit representations: Implicit neural representations model signals as functions over query locations, supporting applications including shape, radiance fields, point clouds, image representation, and dense prediction.InfiniSplat adapts this query-based idea to Gaussian scene generation rather than a single-channel image or depth signal.
- Query-based Gaussian decoding: C3G and TokenGS use learnable query tokens to decode compact Gaussians from unposed sparse multi-view images, where cross-view constraints support consistent layouts.InfiniSplat differs by operating in single-image NVS with explicit geometric organization.
3 Method
InfiniSplat shifts single-image Gaussian generation from fixed pixel-grid locations to surface-aligned supports, coupling geometry-guided sampling with query-conditioned implicit decoding. The method uses sampled surface structure and queried semantic/texture features to construct and regularize a feed-forward 3D Gaussian representation.
- Surface-aligned representation: InfiniSplat shifts Gaussian generation from a fixed pixel grid to surface-aligned locations guided by predicted depth geometry.The representation addresses where Gaussians should be placed and how their parameters should be predicted at irregular supports.
- Preprocessing: A frozen monocular depth model supplies dense depth and camera intrinsics, while a dual-branch encoder extracts DINO semantic and CNN texture features.These branches provide the geometric scaffold and complementary image-conditioning signals for downstream decoding.
- Geometry-guided sampling: Geometry-guided sampling back-projects depth to estimate local 3D surface areas, allocates supports proportionally to area, and initializes base Gaussians.Candidate triangles with large relative depth variation are discarded before area-weighted sampling within their 2D regions.
- Implicit Gaussian decoding: A shared implicit decoder queries DINO and CNN features at each support, fuses them, and predicts bounded updates to Gaussian position, scale, rotation, color, and opacity.Its shared function operates over arbitrary support coordinates rather than fixed pixel centers, enabling consistent updates across neighboring Gaussians.
- Coupled design: The sampling and decoding stages are mutually coupled: geometry supplies surface awareness for irregular supports, while implicit decoding converts those supports into complete Gaussians.The resulting representation is intended to assemble neighboring primitives into coherent surfaces instead of independent local splats.
- Variants and training: InfiniSplat includes RGB-only and LiDAR-conditioned variants, and trains rendered target views with RGB, perceptual, and Gaussian-regularization losses.The RGB-only variant uses DepthPro for its geometric scaffold; regularization constrains scales and local attribute consistency.
4 Experiments
InfiniSplat is evaluated in zero-shot cross-dataset novel-view synthesis, where it consistently outperforms feed-forward baselines quantitatively and preserves more coherent structures under large viewpoint changes. Ablations show that learned updates, dual-branch conditioning, Gaussian regularization, geometry-guided sampling, and implicit decoding all contribute to performance.
- Quantitative Comparison: InfiniSplat-RGB achieves the highest PSNR and SSIM and lowest LPIPS on all four RGB-only datasets.Its four-dataset average is 20.394/0.806/0.277, improving over SHARP by +1.919 PSNR and +0.048 SSIM while reducing LPIPS by 0.022.
- Quantitative Comparison: 22.548/0.851/0.225 is InfiniSplat-LiDAR’s average on three source-depth datasets, improving over ADGaussian by +10.299 PSNR and +0.212 SSIM while reducing LPIPS by 0.193.DL3DV is excluded because it lacks usable source-depth input.
- Qualitative Comparison: InfiniSplat-RGB preserves more coherent large planes, stable object boundaries, and fewer holes and cracks than baselines under large viewpoint shifts.Normal visualizations likewise show smoother, more coherent surfaces than SHARP, especially on walls, floors, columns, vehicles, and building facades.
- Qualitative Comparison: InfiniSplat-LiDAR produces fewer cracks and holes and more coherent surfaces than ADGaussian under larger viewpoint changes, while in-the-wild results show detailed reconstructions under complex geometry and varying lighting.These results support generalization beyond the evaluated training distribution and the effectiveness of the depth-conditioned variant.
- Ablation Study: Removing learned Gaussian updates reduces PSNR and worsens LPIPS, leaving coarse structure but blurry textures and local boundaries.The base-GS-only variant drops PSNR by 1.712 on ETH3D and 2.379 on ScanNet++, while LPIPS worsens from 0.220 to 0.237 and from 0.270 to 0.298, respectively.
- Ablation Study: Removing DINO, Gaussian regularization, geometry-guided sampling, or implicit decoding degrades metrics, with implicit decoding and DINO removal producing especially large ScanNet++ drops.The full model’s gains arise from the joint effect of learned updates, dual-branch conditioning, regularization, geometry-guided sampling, and implicit decoding.
- Ablation Study: 1.5M supports is the default because it nearly reaches the quality plateau while costing less inference and rendering time than 2.0M supports.Rendering time increases from 0.004s at 0.5M supports to 0.011s at 2.0M supports; controlled multiplicative depth noise causes smooth degradation without abrupt failure.
5 Limitations
InfiniSplat remains vulnerable to geometric errors caused by single-view ambiguity, imperfect depth priors, and extreme viewpoint extrapolation. These limitations are especially visible when unseen regions, unusual surfaces, or fine structures challenge the depth-induced support distribution.
- Single-image ambiguity can produce incomplete geometry, stretched structures, or hallucinated appearance when target views expose large unseen regions.This is especially problematic behind foreground objects or outside the source-view frustum.
- Incorrect pretrained depth on reflective, transparent, thin, textureless, or unusual scenes can place supports on inaccurate geometric scaffolds.
- Extreme viewpoint extrapolation remains challenging, particularly for non-Lambertian surfaces, repeated fine structures, very thin geometry, and strong depth discontinuities.
- Qualitative failure cases include incomplete geometry under large disocclusion and artifacts inherited from incorrect depth.
6 Conclusion
InfiniSplat combines geometry-guided support sampling with query-conditioned implicit Gaussian decoding to move single-image 3DGS toward surface-aligned scene representations. Across cross-dataset evaluations, it achieves state-of-the-art results and produces more coherent structures under large viewpoint changes.
- InfiniSplat combines geometry-guided support sampling with query-conditioned implicit Gaussian decoding for single-image 3D Gaussian scene generation.
- Decoupling Gaussian prediction from fixed pixel centers organizes supports around depth-induced surface structure for novel view synthesis.
- InfiniSplat achieves state-of-the-art quantitative results across ETH3D, ScanNet++, Tanks and Temples, and DL3DV in RGB-only and LiDAR-conditioned settings.
- Qualitative comparisons and ablations show fewer cracks and holes, cleaner normal organization, and more coherent surfaces under large viewpoint changes.