Source-linked AI summary
Less Gaussians, Texture More: 4K Feed-Forward Textured Splatting
Yixing Lao, Xuyang Bai, Xiaoyang Wu, Nuoyuan Yan, Zixin Luo, Tian Fang, Jean-Daniel Nahmias, Yanghai Tsin, Shiwei Li, Hengshuang Zhao
TL;DR
Feed-forward 3D Gaussian Splatting struggles at high resolution because primitive counts grow quadratically and appearance is coupled to geometry. LGTM decouples compact geometric primitives from per-primitive textures, enabling 4K novel-view synthesis with substantially lower scaling costs. The method improves multiple baseline families, while geometry quality and manually selected texture resolutions remain limitations.
Problem
Existing feed-forward 3DGS methods face quadratic primitive-count growth with resolution, while coupled appearance and geometry require excessive Gaussians for detailed textures.
Method
LGTM uses dual networks: one predicts compact geometry from low-resolution inputs, and the other predicts detailed per-primitive textures from high-resolution inputs.
Results
LGTM achieves 4K feed-forward novel-view synthesis, requiring only 1.80× memory and 1.47× time for a 64× increase in pixel count.
Takeaways & Limitations
LGTM provides a broadly applicable feed-forward textured-Gaussian framework that improves monocular, two-view, and multi-view baselines.
Takeaways & Limitations
Reconstruction quality still depends heavily on geometry, and texture resolutions are predefined and require manual tuning to balance quality and computational cost.
Abstract
from arXiv · showhide
Existing feed-forward 3D Gaussian Splatting methods predict pixel-aligned primitives, leading to a quadratic growth in primitive count as resolution increases. This fundamentally limits their scalability, making high-resolution synthesis such as 4K intractable. We introduce LGTM (Less Gaussians, Texture More), a feed-forward framework that overcomes this resolution scaling barrier. By predicting compact Gaussian primitives coupled with per-primitive textures, LGTM decouples geometric complexity from rendering resolution. This approach enables high-fidelity 4K novel view synthesis without per-scene optimization, a capability previously out of reach for feed-forward methods, all while using significantly fewer Gaussian primitives. Project page: https://yxlao.github.io/lgtm/
1 INTRODUCTION
LGTM targets the scalability limits of feed-forward 3D Gaussian Splatting by separating compact geometry prediction from high-resolution textured appearance prediction. This enables feed-forward novel-view synthesis up to 4K with fewer primitives.
- Existing feed-forward methods scale poorly because Gaussian counts grow quadratically with image size, making 4K prediction and rendering prohibitively expensive.Scaling from 512 to 4K requires 64× more Gaussians.
- Standard 3DGS couples appearance and geometry within each primitive, requiring excessive Gaussians for rich textures on simple surfaces.
- LGTM uses a dual-network architecture to decouple geometry parameters from per-primitive textures.The primitive network processes low-resolution inputs, while the texture network processes high-resolution inputs and predicts detailed texture maps.
- LGTM predicts compact geometric primitives and rich per-primitive textures for high-resolution rendering up to 4K.
- The framework is demonstrated across monocular, two-view, and multi-view baselines with and without camera poses.
2 RELATED WORK
Prior work moves from optimized neural or Gaussian representations toward feed-forward reconstruction, while textured splatting addresses the inefficiency of encoding detailed appearance with individual Gaussian colors.
- NeRF and conventional 3DGS provide powerful scene representations but require per-scene optimization, motivating generalizable feed-forward alternatives.
- Traditional Gaussian splatting couples appearance and geometry, so high-frequency textures or complex reflectance require excessive numbers of Gaussians.This inefficiency remains even on geometrically simple surfaces such as flat textured regions.
- Table 1 compares LGTM with NoPoSplat variants at different primitive resolutions while holding effective output resolution equal.
- Textured Gaussian methods improve appearance efficiency using either global UV atlases or individual textures assigned to each Gaussian.Global atlases can be difficult for scenes with complex geometric topologies, whereas per-primitive texturing is more flexible.
3 PILOT STUDY
The pilot study exposes the memory bottleneck of scaling feed-forward Gaussian prediction to high resolutions, while LGTM reaches 2K and 4K with compact geometry and textured upsampling.
- 61.85 GB of training memory is required by NoPoSplat at 1024×576 primitives with batch size 1.
- NoPoSplat training fails entirely at 2K and 4K because of memory constraints.
- LGTM trains successfully at 2K and 4K using under 30 GB of memory.
- A 64× increase in pixels adds only modest inference memory and time overhead for LGTM.The approach keeps geometry compact while scaling per-primitive textures to higher resolutions.
4 METHOD
LGTM predicts compact 2D Gaussian geometry from low-resolution inputs and detailed per-primitive textures from high-resolution features, then renders textured primitives through projective mapping and alpha blending.
- Feed-forward prediction: LGTM accepts multiple input settings and predicts textured 2D Gaussians for novel-view synthesis.The framework supports monocular, posed and unposed two-view, and multi-view inputs.
- 2D Gaussian Splatting: 2D Gaussian Splatting maps each pixel ray to local primitive coordinates through a view-dependent homography before evaluating the Gaussian.
- 2D Gaussian Splatting: Rendered pixels are formed by alpha-blending primitive contributions sorted by depth.
- Textured Gaussian Splatting: Textured splatting replaces Gaussian alpha falloff with sampled alpha textures and augments view-dependent base color with sampled color textures.
- Architecture: The primitive network predicts compact geometric primitives from low-resolution images, while the texture network predicts per-primitive texture maps from high-resolution images.
- Primitive network: The primitive network produces an h × w grid of 2DGS primitives from low-resolution images.
- Primitive network: High-resolution supervision trains low-resolution primitive predictions to render appropriately at full H×W resolution.Without this supervision, un-antialiased Gaussians may leave holes at higher resolutions.
- Texture network: The texture network combines patchified-image, projective, and shared primitive features to predict per-primitive textures.Projective priors are obtained by mapping primitive coordinates back to high-resolution source-image pixels.
5 EXPERIMENTS
Experiments evaluate LGTM across two-view, single-view, and multi-view novel view synthesis settings on RE10K and DL3DV, including 4K rendering. Results show consistent quality gains and efficient high-resolution inference, while 4K multi-view evaluation remains constrained by memory.
- Two-view: LGTM is evaluated with NoPoSplat and DepthSplat for two-view synthesis on RE10K and DL3DV, including 4K DL3DV results.The 4K settings use 4096×2304 for NoPoSplat and 3840×2048 for DepthSplat because of backbone requirements.
- Two-view: LGTM consistently outperforms 3DGS and 2DGS across tested two-view resolutions and metrics, reducing LPIPS by 23%–75%.The gains are reported for both pose-free NoPoSplat and posed DepthSplat.
- Single-view: LGTM achieves the best single-view performance at every tested resolution while using only 512×288 geometric primitives.Compared with Flash3D, qualitative results show finer details and textures that baselines often blur or lose.
- Multi-view: LGTM improves multi-view results at 1K and 2K when built with the VGGT backbone, but the study does not scale this setting to 4K.The 4K multi-view experiment is deferred because of memory constraints even with the VGGT backbone frozen.
- Performance benchmark: A 64× pixel increase from NoPoSplat 512×288 2DGS to LGTM 4096×2304 requires only 1.80× peak memory and 1.47× total time.The benchmark measures inference on one NVIDIA A100 GPU with batch size one, averaging ten runs after three warmups.
- Ablation study: The ablation study shows progressive gains from high-resolution supervision, patchified features, texture color maps, and texture alpha maps, with the full model performing best.All ablation models use 512×288 primitives and are evaluated on DL3DV at 2K.
6 CONCLUSION
LGTM enables 4K feed-forward novel view synthesis while avoiding the severe scaling costs of conventional approaches. Its gains are consistent across baseline methods, though reconstruction quality remains constrained by geometry and manually tuned texture resolution.
- LGTM achieves 4K novel view synthesis where traditional approaches fail due to memory constraints.It requires only 1.80× memory and 1.47× time for a 64× increase in pixel count.
- Consistent improvements across Flash3D, NoPoSplat, DepthSplat, and VGGT demonstrate broad applicability across input settings.
- Reconstruction quality still depends heavily on geometry, with marginal gains in multi-view settings where geometry is less precise.
- Pre-defined texture resolutions require manual tuning to balance quality and computational cost.
A.1 HIGH-RESOLUTION RE-TRAINING
High-resolution re-training supplies the geometry needed for LGTM’s high-resolution rendering. Direct rendering creates holes, while rendering and upsampling is blurry; re-training performs best among the compared strategies.
- High-resolution re-training is crucial for learning primitive scales and other attributes needed for high-resolution rendering.It provides a strong geometry prior for subsequent texture learning.
- Re-training produces the best results among direct rendering, rendering and upsampling, and re-training.
- Direct high-resolution Gaussian rendering produces holes because 3DGS lacks anti-aliasing.
- Rendering and upsampling eliminates holes but remains blurry because the effective image resolution stays low.
- Even re-trained vanilla 3DGS remains somewhat blurry because its texture complexity is limited.
A.2 BILINEAR TEXTURE SAMPLING
LGTM retrieves per-primitive texture values by bilinear sampling at ray-splat intersections, using border clamping to extend texture information beyond primitive boundaries. The method separates texture sampling behavior from Gaussian opacity handling.
- LGTM bilinearly samples a T × T texture map at local ray-splat coordinates (u, v) on each primitive plane.The scale σ controls the span of the texture region within the 2DGS primitive.
- The sampling procedure computes integer and fractional coordinate parts, clamps neighboring corner indices, and combines four texture corners bilinearly.
- Border clamping handles out-of-bounds coordinates, allowing texture information to extend beyond primitive boundaries.This provides smoother transitions and better coverage than zero-padding bounded sampling.
- Unlike BBSplat’s bounded color sampling, LGTM uses equal weights across bilinear corners and learns texture colors using only 2DGS opacity as alpha.
A.3 ROBUSTNESS TO LARGER INPUT VIEW GAPS
LGTM remains effective as input-view gaps widen from 10 to 40 frames, despite increasingly difficult viewpoint differences. It consistently improves sharpness, artifacts, and quantitative performance over the baseline.
- The evaluation uses two source views and four target views for each specified context-frame gap at 4096×2304 resolution.
- NoPoSplat + LGTM produces sharper details and fewer artifacts than the baseline across all tested gaps.
- As the context gap increases from 10 to 40 frames, viewpoint differences become more substantial and reconstruction becomes increasingly challenging.
- LGTM consistently outperforms the baseline across context view gaps of 10, 20, 30, and 40 frames.
- Both methods degrade as the gap increases, but LGTM remains superior even at gap 40.
A.4 COMPARISON WITH PER-SCENE OPTIMIZATION
DepthSplat + LGTM outperforms per-scene optimized 3DGS across all metrics while reconstructing instantly. It also maintains stable quality across target views, avoiding the context-view overfitting seen in per-scene optimization.
- DepthSplat + LGTM maintains stable performance across the 19 target views evaluated between two context frames.The comparison uses frames 0 and 20 as context views and evaluates frames 1-19.
- DepthSplat + LGTM outperforms per-scene optimized 3DGS across all metrics while being orders of magnitude faster.
- Per-scene optimization overfits to context views, with PSNR dropping to ∼20-22 dB for middle frames.
- DepthSplat + LGTM achieves instant reconstruction, compared with ∼30 minutes for per-scene optimization on a single A100 GPU.