Source-linked AI summary
GSPotential: Camera Potential Field for Sparse-View 3D Gaussian Splatting
Zeyuan An, Yanghang Xiao, Zhiying Leng, Yijun Feng, Xiaohui Liang
TL;DR
Sparse-view 3DGS suffers from overfitting and geometric artifacts because photometric supervision is uneven across viewing space. GSPotential quantifies this imbalance with a Camera Potential Field, placing virtual supervision in low-potential regions and regulating Gaussian updates. The framework reports improved reconstruction fidelity while maintaining competitive training efficiency.
Problem
Sparse-view 3DGS can suffer from overfitting and geometric instability because anisotropic camera coverage leaves some directions under-constrained.
Method
GSPotential uses a Camera Potential Field to sample virtual viewpoints in low-potential regions for point-cloud guidance and regulate pruning and densification by directional coverage.
Results
GSPotential achieves improved reconstruction quality over original 3DGS in sparse-view inputs while maintaining competitive training efficiency.
Takeaways & Limitations
Potential-guided supervision and structure rebalancing targets weakly covered directions rather than applying uniform regularization.
Takeaways & Limitations
Experiments focus on inward-looking, bounded scenes, and the current field does not explicitly encode scene geometry, occlusion, or visibility.
Abstract
from arXiv · showhide
3D Gaussian Splatting has achieved remarkable success in photorealistic rendering, yet it suffers from severe overfitting and geometric artifacts in sparse-view scenarios due to the inherent deficiency of photometric supervision. Recent advances have attempted to regularize optimization by incorporating external priors, such as depth, point clouds, or diffusion models. However, these methods typically overlook the non-uniform distribution of supervision across the viewing space, resulting in limited specificity in prior use and primitive control. In this paper, we propose GSPotential, a framework that quantifies view-space supervision imbalance using a Camera Potential Field. Our key insight is to identify supervision valleys where photometric constraints are most deficient, and use the potential field to guide reconstruction from two complementary aspects. First, we devise a probabilistic spherical sampling strategy that places informative virtual cameras in low-potential regions. Point-cloud renderings from these views then provide targeted geometric guidance. Second, the same field provides a directional coverage cue for conservative Gaussian updates in weakly covered spatial sectors. Extensive experiments demonstrate that GSPotential achieves high reconstruction fidelity while maintaining competitive training efficiency.
1. Introduction
Sparse-view 3DGS suffers from overfitting and geometric instability because camera supervision is uneven across viewing directions. GSPotential addresses this imbalance with a Camera Potential Field that targets weakly supervised regions for virtual geometric guidance and conservative Gaussian updates.
- Sparse-view 3DGS often produces overfitting, geometric instability, and novel-view artifacts under limited photometric supervision.
- Strong geometric initialization alone may not preserve geometry during optimization when viewing directions are insufficiently guided.
- Anisotropic input-camera coverage creates supervision imbalance, biasing optimization toward well-constrained directions and leaving angular regions under-constrained.
- The Camera Potential Field measures supervision strength over a canonical viewing sphere and separates reliable weak regions from unobservable blind spots.
- GSPotential samples virtual viewpoints in low-potential regions for point-cloud consistency and uses directional coverage to regulate pruning and densification.
2. Related Work
Related work uses geometric, generative, and structural priors to improve sparse-view reconstruction, but many approaches do not account for directional reliability. GSPotential instead models supervision induced by fixed cameras to guide both virtual supervision and structural updates in under-constrained regions.
- Sparse Views 3D Reconstruction: Sparse-view 3DGS remains vulnerable to overfitting and artifacts when limited image overlap weakens supervision.
- Sparse Views 3D Reconstruction: Existing methods address settings ranging from small-baseline and sparse 360° scenes to two-image reconstruction, often relying on specialized protocols or extensive pre-training.
- Sparse Views 3D Reconstruction: External priors include monocular depth, dense point clouds, diffusion-based appearance refinement, and structural regularization of Gaussian primitives.
- Geometry Priors and Point-based Supervision: Point clouds are commonly used for initialization or global constraints, but their geometry may be under-utilized in directions lacking photometric supervision.
- View Planning and Camera Modeling: Unlike active view-planning methods designed for data acquisition, GSPotential models supervision potential from a fixed camera set for passive reconstruction.
3. Methodology
GSPotential models view-space supervision imbalance with a Camera Potential Field, then uses it to allocate virtual geometric supervision and regulate Gaussian evolution in weakly covered directions. The method combines spherical viewpoint sampling, confidence-aware reconstruction, and coverage-aware primitive updates for sparse-view 3DGS.
- Motivation: Sparse-view 3DGS suffers from clustered camera coverage, leaving angular regions under-constrained and causing geometric instability.The resulting artifacts arise even when strong geometric priors are available, because optimization is biased toward well-constrained directions.
- Camera Potential Field: The Camera Potential Field represents supervision strength continuously on a canonical viewing sphere and identifies weakly supervised but geometrically plausible regions.It combines directional supervision modeling with a confidence mechanism that excludes unreliable extrapolation regions.
- Geometric Canonicalization: GSPotential estimates a pseudo-focus center from camera optical axes and places virtual cameras on a sphere oriented toward that center.The sphere radius uses the maximum camera distance to the estimated center, keeping virtual viewpoints at a safe distance from the scene.
- Potential-Guided Viewpoint Generation: Virtual viewpoints are sampled probabilistically from low-potential regions, concentrating auxiliary supervision where reconstruction quality is most likely to degrade.The sampling distribution supports a fixed set of virtual viewpoints and avoids repeated potential evaluation during training.
- Camera Potential Field: The potential formulation uses a heavier-tailed hyperbolic secant kernel because it preserves smoother, longer-range influence across large angular gaps than calibrated vMF kernels.The resulting potential values remain more informative in sparse regions and show a more monotonic relationship with novel-view rendering quality.
- Potential-Guided Reconstruction: The field guides reconstruction through point-cloud cross-training and directional coverage-aware Gaussian evolution, combining missing-supervision compensation with conservative primitive updates.Coverage-aware evolution includes anisotropy pruning and conservative densification, reducing unsupported primitive growth in weakly covered sectors.
4. Experiments
Experiments show that camera potential identifies rendering-quality variation across viewpoints and guides targeted virtual supervision and coverage-aware Gaussian updates. GSPotential improves reconstruction quality across datasets, initialization conditions, sampling strategies, and ablations while maintaining competitive efficiency.
- Potential and Rendering Quality: Higher camera potential generally corresponds to more faithful novel-view reconstruction, while low-potential viewpoints more often exhibit sparse-view artifacts.Camera potential also separates viewpoints that nearest-view angular distance alone treats similarly because supervision depends on the joint camera configuration.
- Point Cloud Fidelity: Up to 6 dB PSNR is lost when the initial point cloud is decimated, showing that point-cloud density strongly constrains reconstruction quality.The experiment identifies explicit geometric anchors as the primary source of structural integrity when dense photometric overlap is unavailable.
- Quantitative Evaluation: GSPotential achieves the best PSNR, SSIM, and LPIPS on Mip-NeRF 360 and the highest PSNR with best-matching SSIM on OmniObject3D.It remains competitive in LPIPS on OmniObject3D and is evaluated under a common sparse-view protocol.
- Efficiency: 13 s for potential computation and sampling plus approximately 35 s for point-cloud rendering preserves competitive efficiency despite added overhead.The total time remains close to 3DGS and faster than several geometry- or surface-oriented baselines.
- Qualitative Evaluation: GSPotential improves visual quality by producing sharper edges, cleaner planar regions, fewer floaters, and more stable geometry in unsupervised views.The reported improvements span Mip-NeRF 360 and OmniObject3D, where the method suppresses needle-like artifacts, drifting, and geometric distortion.
- Ablation Study: Uniform virtual-view supervision raises PSNR from 13.52 to 14.49, while Uinfo-guided viewpoint generation reaches 15.90 dB PSNR.The ablation attributes the larger gain to locating supervision valleys and allocating virtual supervision where photometric constraints are weakest.
- Ablation Study: Qi-guided pruning improves SSIM and LPIPS, Qi-guided densification yields a larger PSNR gain, and combining both updates improves all metrics.Pruning suppresses over-elongated primitives, while conservative densification limits unreliable growth in weakly covered sectors.
- Sensitivity Studies: GSPotential consistently outperforms 3DGS across point-cloud qualities and tested input-view settings, with larger gains from denser and more complete point clouds.The weakest point-cloud prior with GSPotential still outperforms 3DGS with the strongest prior; adding two more views yields only a 0.16 dB gain in the DUSt3R prior.
5. Limitation and Future Work
The current formulation is evaluated on inward-looking, bounded scenes and does not explicitly model scene geometry, occlusion, or visibility. GSPotential also depends on external geometric priors, while pose uncertainty and improved virtual-view supervision remain open directions.
- Experiments focus on inward-looking, bounded scenes whose input views approximately observe a shared region of interest.
- CPF models camera-layout imbalance without explicitly encoding scene geometry, occlusion, or visibility.
- Extending the formulation to outward-looking scenes, adding geometry-aware visibility cues, and refining virtual-view supervision with diffusion models are identified as future directions.
- GSPotential requires an external prior such as a point cloud, typically obtained from feedforward methods such as DUSt3R.
- Pose inaccuracies can perturb the potential field, motivating future incorporation of pose uncertainty.
6. Conclusion
GSPotential addresses view-space supervision imbalance in sparse-view 3DGS by locating weakly supervised regions and guiding both virtual supervision and Gaussian updates. Experiments on Mip-NeRF 360 and OmniObject3D show improved 4-view reconstruction quality, while ablations validate both guidance components.
- GSPotential measures supervision strength over the viewing sphere with a Camera Potential Field to identify regions where photometric constraints are insufficient.
- Potential-guided viewpoint generation places virtual cameras in weakly supervised yet geometrically plausible regions, whose point-cloud renderings provide targeted geometric guidance.
- The method uses directional coverage scores to regulate anisotropy pruning and conservative densification in weakly covered spatial sectors.
- Experiments on Mip-NeRF 360 and OmniObject3D demonstrate improved sparse-view reconstruction quality under 4-view settings.
- Ablation studies verify the effectiveness of potential-guided virtual supervision and coverage-aware primitive updates.