Source-linked AI summary
Beyond Monoscopic Viewing: A Study on 3D Gaussian Splatting Quality in VR
Shreyas Shivakumara, Gabriel Eilertsen, Karljohan Lundin Palmerius
TL;DR
Under reduced capture, 3DGS can produce localized geometry and texture artifacts that standard metrics may not reflect well. The paper compares SfM-only and combined SfM–VGGT initialization with other training components held fixed, then evaluates metrics and user preferences across monoscopic and stereoscopic viewing. Artifacts that are inconspicuous monoscopically become salient in stereo viewing, supporting perceptual HMD evaluation for VR use.
Problem
Method
The paper compares SfM-only initialization with a union of SfM and dense VGGT points while holding subsequent training components fixed, and evaluates metrics and user preferences in mono and stereo.
Source: §1 · p. 2Results
Takeaways & Limitations
For 3DGS reconstructions intended for VR, the paper recommends perceptual evaluation in stereo using an HMD.
Source: §1 · p. 2, §10 · p. 9The user study compared only baseline and hybrid initialization; including VGGT would have tripled scene-level pairwise comparisons and lengthened HMD sessions beyond a comfortable duration.
Source: §9 · p. 9
Abstract
from arXiv · showhide
Stereoscopy is fundamental to virtual reality (VR), providing depth perception through binocular viewing. Recent advances in 3D Gaussian Splatting (3DGS) enable high-quality novel view synthesis, making it well suited to immersive VR. We render 3DGS reconstructions stereoscopically and evaluate them in a head-mounted display, replicating how they would actually be viewed in VR. Real-world capture provides only a limited number of views, and under this constraint 3DGS reconstruction often produces localized floaters and misplaced structures. Standard metrics miss these localized artifacts, which become salient under stereoscopic viewing, where geometry is placed at the wrong depth. We investigate whether standard image-quality evaluation reflects the perceptual quality of 3DGS reconstructions under reduced capture. We compare SfM-only baseline with a union initialization that combines SfM with a dense VGGT network. All other training components are held fixed, isolating the effect of initialization coverage. We conduct a user study comparing preferences under monoscopic and stereoscopic HMD viewing, and test whether image-quality metrics predict the observed preferences. Monoscopically, preference for the more consistent reconstruction is weak, reaching 58.4\% overall. Stereoscopically, the same preference rises to 78.2\% and is consistent across all participants, while image-quality metrics (PSNR, SSIM and LPIPS) and stereo-aware metrics (iSQoe and StereoQA) show only modest differences and fail to penalize them. Our results indicate that monoscopic evaluation and standard image-quality metrics substantially underestimate perceptual artifacts observed in 3DGS reconstructions for VR, making stereoscopic assessment essential for 3DGS quality evaluation in VR.
1 INTRODUCTION
Reduced-view 3DGS can produce misplaced geometry and texture loss, while common image metrics may underrepresent localized artifacts that matter in stereo. The paper compares initialization strategies under controlled conditions and evaluates reconstructions with metrics and user preferences in monoscopic and stereoscopic viewing.
1 INTRODUCTION
- Sparse and uneven seed clouds under reduced capture can produce floaters and texture loss, with misplaced geometry especially problematic for VR depth perception.3DGS commonly uses SfM points to seed its initial geometry.Source: §1 · p. 1
- Localized artifacts occupy few pixels and receive modest penalties from full-frame image metrics, which do not explicitly model binocular consistency or perceived depth.The paper notes divergence between conventional image metrics and subjective assessments in prior neural-rendering work [27] [31].Source: §1 · p. 2
- The controlled comparison holds camera poses and all subsequent training components fixed, isolating the difference between SfM-only and SfM-plus-dense-prior initialization.The union initialization combines SfM points with a dense feed-forward geometric prior.Source: §1 · p. 2
- The evaluation combines held-out image metrics, stereo-aware metrics, and separate monoscopic and stereoscopic two-alternative forced-choice studies.The study tests whether metric differences correspond to preferences and whether preferences change with binocular depth cues.Source: §1 · p. 2
- Artifacts weakly reflected in metrics and inconspicuous in monoscopic images become salient under stereo viewing, motivating perceptual HMD assessment for reconstructions intended for VR.The paper recommends evaluating such reconstructions perceptually in HMDs.Source: §1 · p. 2
2 RELATED WORK
Prior work has adapted Gaussian splatting for VR, studied novel-view quality, and improved sparse-view initialization. This paper focuses on stereoscopic assessment under reduced coverage while leaving the vanilla 3DGS architecture unchanged.
3D Gaussian Splatting
3DGS for Virtual Reality
Evaluation of Novel-View Synthesis
- Novel-view evaluation commonly uses monoscopic full-frame PSNR, SSIM, and LPIPS [55], but no established metric captures localized floater artifacts under reduced coverage.The paper therefore supplements image and stereo-quality metrics with monoscopic and HMD two-alternative forced-choice studies.Source: §2 · p. 2
Initialization for Novel-View Synthesis
- Sparse-view initialization research uses depth supervision, learned or differentiable reconstruction, and feed-forward geometry, but related approaches modify vanilla 3DGS and are not evaluated in VR.Here, a dense prior seeds the initial point cloud without changing the vanilla 3DGS architecture.Source: §2 · p. 2
3 PROBLEM FORMULATION
The study models reduced scene coverage by training on widely spaced frames, where sparse initialization can leave artifacts that affect stereo depth. It uses controlled initialization differences to examine how these artifacts appear and are judged.
3 PROBLEM FORMULATION
- Training on every fourth, eighth, or sixteenth frame models reduced coverage while retaining scene-wide sampling.The sampled sequences contain far fewer viewpoints than the standard capture.Source: §3 · pp. 2–3
- Figure 2 contrasts smoothed cabinet grain in the SfM-only baseline with restored texture in VGGT and Union, and VGGT ghosting at the window that baseline and Union resolve.The passage reports that these localized visual differences are weakly represented by metrics but amplified in an HMD.Source: Figure 2 · p. 3
- With reduced coverage, SfM seeds become sparse and uneven, and densification cannot recover geometry absent from the initial cloud, leading to floaters or texture loss.Dense VGGT points can fill uncovered regions, but its far-field predictions may be inaccurate and appear hazy or blurred.Source: §3 · p. 3
- The controlled stimulus pairs vary initialization while holding other factors identical, so reconstruction differences can be attributed to seed geometry alone.Reduced-coverage artifacts motivate comparing identical content under monoscopic and stereoscopic viewing.Source: §3 · p. 3
4 CONTROLLED STIMULUS GENERATION
The stimulus-generation pipeline combines sparse SfM geometry with filtered, aligned dense VGGT points, then trains and evaluates standard 3DGS under controlled conditions. It tests reduced-coverage scenes using monoscopic and stereoscopic rendering alongside image and stereo-aware metrics.
4 CONTROLLED STIMULUS GENERATION
- All stimulus pairs are formed by aggregating SfM and VGGT initial clouds while keeping subsequent training, parameters, and optimization identical to vanilla 3DGS.This controls the comparison so the initial geometry is the changing component.Source: §4 · p. 3
4.1 Preliminaries: 3D Gaussian Splatting
- 3DGS optimizes Gaussian position, covariance, opacity, and view-dependent color, but densification only refines existing Gaussians and cannot add structure where the initial cloud is empty.The initial point cloud therefore determines which regions can be reconstructed during training.Source: §4.1 · p. 3
4.2 Hybrid Initialization
- SfM points are accurate but sparse, while VGGT is dense but degrades in the far field; the hybrid initialization keeps both sources to cover their complementary gaps.VGGT points are filtered, subsampled, aligned to SfM coordinates, and concatenated with SfM points before unchanged 3DGS training.Source: §4.2 · p. 4
4.3 Stereoscopic Rendering
- Held-out poses produce on-axis renders for image metrics and monoscopic study, and off-axis stereo pairs for HMD evaluation.The stereo rendering uses 6.5 cm eye separation and a shared image-plane convergence distance of 1.3 m.Source: §4.3 · pp. 4–5
4.4 Implementation Details
- Both baseline and Union models use the public 3DGS implementation and default settings, trained for 30,000 iterations on a single RTX 4090.Training takes approximately nine minutes per scene for the baseline and comparable time for Union.Source: §4.4 · p. 5
4.5 Data
- Evaluation uses eight indoor and outdoor scenes, subsampled at strides 4, 8, and 16; the table caption reports Union benefits as capture density decreases except on StereoQA.The data are from Mip-NeRF 360 and Deep Blending, with approximately 55, 28, and 14 training views at the three strides.Source: §4.5 · p. 5, Table 1 · p. 5
5 QUANTITATIVE EVALUATION
The evaluation compares standard held-out image metrics with stereo-aware quality metrics for reconstructions from different initializations. Metrics show broad but modest gains for the union initialization, yet cannot establish whether those gains matter perceptually.
5 QUANTITATIVE EVALUATION
- Held-out views are compared with ground truth using PSNR, SSIM, and LPIPS, while iSQoE and StereoQA assess stereoscopic quality.iSQoE is trained on 3DGS-rendered artifacts; StereoQA predicts perceptual stereo quality.Source: §5 · pp. 5–6
- Figure 5 contrasts ground-truth images with left-view renderings and highlights blur and texture loss that metrics and monoscopic viewing miss but stereo viewing reveals.Red boxes mark regions enlarged in adjacent columns.Source: Figure 5 · p. 6
5.1 Image and Stereo Quality Metrics
- Across eight scenes, union initialization improves PSNR, SSIM, and LPIPS over the SfM baseline, with gains over VGGT as views become sparser.Stereo-aware results are mixed: iSQoE shows marginal improvement at 1/4 and 1/8 stride, while StereoQA does not follow that trend.Source: §5.1 · p. 6
6 USER STUDY
The study compares SfM-baseline and union reconstructions in separate monoscopic and stereoscopic groups using randomized pairwise judgments. Union preference is stronger in stereo than mono, while objective metrics show only modest differences.
6 USER STUDY
- The study used 2AFC judgments in both viewing conditions and modeled participant and scene variation with crossed random intercepts.The GLMM used a binomial logit link.Source: §6 · p. 6
6.1 Setup
- Sessions used 28 trials per participant across seven scenes and four viewpoints; stereo stimuli were shown in a custom WebXR application on a Meta Quest 2.The stereo session was capped at approximately 30 minutes to limit fatigue and discomfort.Source: §6.1 · p. 6, §6.3 · p. 7
6.2 Participants.
- Forty participants were divided into distinct groups of 20 for stereo and mono, and half reported visualization or computer-graphics backgrounds.Participants were aged 22–36; the two groups were kept separate to avoid fatigue or carryover effects.Source: §6.2 · p. 6
6.3 Stimuli
- The mono group judged static image pairs on a 27-inch display using the same scenes, views, and underlying renders as the stereo condition.Only the participant sample differed between conditions.Source: §6.3 · p. 7
6.4 Results
- Union initialization was preferred in 78.2% of stereoscopic trials, versus 58.4% of monoscopic trials; six of seven scenes were significantly above chance in stereo.The kitchen was the sole stereoscopic scene not different from chance, at 55.0%.Source: §6.4 · pp. 7–8, Figure 6 · p. 7, Figure 8 · p. 7
- Stereo judgments varied more by scene than by participant, whereas the scene-to-participant variance ratio narrowed from about 4.4× in stereo to 2.0× in mono.The scene-variance estimate is based on seven scenes and has a wide interval.Source: §6.4 · p. 8
- On the 28 study frames, the initializations differed by +0.77 dB PSNR, +0.030 SSIM, and −0.040 LPIPS, while stereo-aware metric differences were small or inconsistent.Despite these metric differences, every participant preferred union in a majority of stereoscopic trials.Source: §6.4 · p. 8
7 DISCUSSION
The discussion links the mismatch between metrics and headset judgments to localized geometry errors that stereo makes perceptually salient. It also identifies legibility and near-field content as cues, while noting that added initialization points cannot repair artifacts already present in the source.
7 DISCUSSION
- A misplaced floater can appear as blur or texture duplication on a monoscopic screen, but as a wrong-disparity blob in stereo.Near-field geometry errors produce larger disparity errors than equivalent errors at a distance.Source: §7 · p. 8
- Readable text or fine markings can outweigh minor artifacts, and participants appeared to weight near-field content more heavily.The authors connect this near-field weighting to larger disparity errors from close-range geometry errors.Source: §7 · p. 8
- Union initialization helps when the initial geometry is missing, but cannot correct artifacts already present in its input.The kitchen’s near-field floater is inherited from VGGT; reported benefits include bicycle at 100%, room at 80%, and bonsai at 70%.Source: §7 · p. 8
8 ABLATION STUDY
The ablations test confidence filtering and initialization size in the Union initialization. Neither parameter has a single best setting across scenes and metrics, and initialization-size performance does not vary monotonically.
8 ABLATION STUDY
- The ablation varies VGGT confidence percentiles and point caps, treating the results as sensitivity analysis rather than evidence of statistical significance.Source: §8 · p. 8, §8.1 · p. 8
8.1 Confidence-threshold ablation
- No confidence threshold dominates across scenes and metrics, and the tested thresholds do not identify an optimum.Across p0–p90, PSNR varies by 1.2 dB on room and 0.71 dB on kitchen; SSIM varies by 0.11 and 0.021, and LPIPS by 0.007 and 0.026, respectively.Source: §8.1 · p. 8, Table 3 · p. 9
8.2 Initialization-size
- No initialization-size cap consistently performs best across scenes and metrics, and performance does not vary monotonically with size.Within the tested range, the 400k-point default provides dense coverage without the added computation of a larger initialization.Source: §8.2 · p. 8, Table 4 · p. 9
9 LIMITATIONS AND FUTURE WORK
The evaluation is bounded by static stereo image pairs rather than free 6-DoF exploration, and by study design choices that limit which comparisons can be made. The union initialization also inherits limitations of SfM and VGGT.
9 LIMITATIONS AND FUTURE WORK
- Static stereo image pairs do not characterize artifact behavior under continuous viewpoint change during free 6-DoF exploration.The study leaves comfort, presence, temporal artifacts, and depth inconsistencies during interactive head motion for future work.Source: §9 · p. 9
- Separate participant groups and different hardware mean the viewing-condition comparison does not isolate stereoscopy alone.Source: §9 · p. 9
- The perceptual study compares only baseline and hybrid initialization, leaving direct stereo comparison with VGGT untested.A three-way comparison was deferred because it would triple pairwise comparisons per scene and lengthen HMD sessions beyond a comfortable duration.Source: §9 · p. 9
- Artifacts inherent to either SfM or VGGT may persist in the fused initialization.The authors propose adaptive fusion using scene-space proximity or per-point confidence as future work.Source: §9 · p. 9
10 CONCLUSION
The conclusion reports that stereoscopic viewing reveals 3DGS artifacts missed by monoscopic viewing and standard image metrics. The authors argue that VR reconstruction quality should be evaluated perceptually in stereo in the headset.
10 CONCLUSION
- Stereoscopic viewing exposes 3DGS artifacts that monoscopic viewing and standard image metrics fail to resolve.Source: §10 · p. 9
- Because stimulus pairs differed only in initialization, the authors attribute observers’ responses to differences in reconstructed geometry.Source: §10 · p. 9
- For intended VR use, the authors conclude that reconstruction should be evaluated perceptually, in stereo, in the headset.Source: §10 · p. 9
SUPPLEMENTAL MATERIALS
The supplemental table reports run-to-run variability across five baseline training runs that differed only by random seed.
SUPPLEMENTAL MATERIALS
- Across five baseline runs, mean run-to-run standard deviations were 0.37 dB for PSNR and 0.010 for SSIM.Source: Table 5 · p. 12
- Across the same runs, mean run-to-run standard deviations were 0.011 for LPIPS and 0.013 for iSQoE.Source: Table 5 · p. 12
- The reported maxima were 0.72 dB for PSNR and 0.39 for StereoQA.Source: Table 5 · p. 12