Source-linked AI summary
From Explicit References to Scene Manifolds: Distributional Fidelity and Realism for Radiance Field Quality Assessment
Saeed Mahmoudpour, Gi-Mun Um, Hyon-Gon Choo, Peter Schelkens
TL;DR
Radiance-field artifacts complicate quality assessment, while full-reference and cross-reference approaches depend on reference images that may not suit arbitrary viewpoints. SCODA models a scene-level reference distribution and combines fidelity with a distortion-aware realism cue. Across 3DGS images, GS trajectories, and NeRF views, it reports strong agreement with human judgments and robust generalization.
Problem
View-dependent radiance-field artifacts complicate quality assessment, while full-reference metrics need aligned images and cross-reference methods still need usable nearby or overlapping views.
Source: §1 · pp. 1–2Method
SCODA scores a rendering's deviation from a high-quality scene distribution and combines this fidelity signal with a distortion-aware realism cue through bounded fusion.
Source: §1 · p. 2, §3.3 · p. 6Results
Across 3DGS images, GS trajectories, and NeRF-generated views, SCODA shows strong agreement with human judgments and robust generalization.
Source: §5 · p. 8Takeaways & Limitations
Abstract
from arXiv · showhide
Radiance field representations such as 3D Gaussian Splatting (3DGS) enable high-quality novel view synthesis but can introduce complex, view-dependent artifacts from reconstruction, rendering, and compression. Reliable perceptual quality assessment (QA) is thus essential for evaluating rendered views and guiding the design of perceptually faithful scene representations. Existing full-reference QA metrics require an aligned reference image, while recent cross-reference metrics relax this requirement by comparing a test view with non-aligned references. However, under wide-baseline radiance field settings, selecting a reliable nearby reference can be difficult, particularly when evaluating views along arbitrary trajectories and poses. We propose SCODA, a lightweight scene-conditioned objective QA method that shifts QA from explicit image-to-image comparison to scene-manifold modeling. High-quality observations of each scene are represented as a multivariate Gaussian distribution in deep feature space, producing a semantic fidelity score that measures deviation from the scene distribution. A weakly-supervised distortion-aware patch discriminator provides a complementary realism signal, and both cues are combined through an unsupervised bounded fusion strategy. Experiments on multiple benchmarks show strong agreement with human judgments and robust generalization across GS- and NeRF-generated views and trajectories. Code is publicly available at https://gitlab.com/saeedmp/scoda.
1. Introduction
Radiance-field artifacts are view-dependent and can propagate across viewpoints, making image-based quality assessment difficult. SCODA replaces explicit reference-view matching with a scene-conditioned distribution and combines its fidelity signal with a distortion-aware realism cue.
1. Introduction
- Structured, view-dependent artifacts from reconstruction, rendering, and compression complicate quality assessment and can propagate across viewpoints.In 3DGS, errors in Gaussian geometry, opacity, or appearance can produce misalignment, floaters, transparency artifacts, color shifts, and flickering.Source: §1 · p. 1
- Full-reference metrics can disagree with human judgments and are limited to test viewpoints with available ground-truth images.Conventional FR metrics such as PSNR and SSIM are often inconsistent with human judgments.Source: §1 · p. 2
- Cross-reference metrics still require nearby or sufficiently overlapping scene views, making usable references ambiguous for arbitrary wide-baseline poses.NR methods avoid explicit references but typically have lower accuracy without retraining and can struggle to generalize across scenes and datasets.Source: §1 · p. 2
- SCODA models high-quality scene observations as a scene-level distribution and scores each novel view by its deviation, retaining reference awareness without an explicit reference image.The method frames quality as per-scene distributional consistency rather than a generic image prior, set-level comparison, or local anomaly detection.Source: §1 · p. 2
- A weakly supervised distortion-aware discriminator adds a realism cue for localized artifacts, and an unsupervised fusion strategy combines it with semantic fidelity.The realism cue complements fidelity because fidelity does not explicitly capture the perceptual impact of artifacts.Source: §1 · pp. 2–3
2. Related Work
Prior work establishes datasets and benchmarks for neural-scene quality, but conventional metrics and reference-based methods face limits with arbitrary viewpoints, view-dependent distortions, and cross-scene generalization. SCODA uses a pristine-view scene distribution and a distortion-aware discriminator to address these challenges.
2. Related Work
- Existing benchmarks and datasets document inconsistent full-reference metric behavior and the difficulty of evaluating GS trajectories when reference views are limited.The literature includes subjective studies, trajectory benchmarks, and a 15,200-image compressed-3DGS dataset spanning 10 real-world scenes.Source: §2 · p. 3
- Full-reference metrics require paired images that may be unavailable for arbitrary novel views, and benchmarks report misalignment with human judgments on NeRF and 3DGS renderings.Common examples include PSNR, SSIM, LPIPS, and DISTS.Source: §2 · p. 3
- No-reference approaches either struggle to generalize when trained on human ratings or rely on natural-image priors that miss neural-scene distortions.Specialized NR work models spatial quality and angular consistency through self-supervised learning.Source: §2 · p. 3
- Cross-reference methods compare non-aligned same-scene views, but still depend on sufficient overlap, nearby viewpoints, or locally matchable content.The passage describes CrossScore, PuzzleSim, and NOVA as examples of these approaches.Source: §2 · p. 3
- SCODA addresses reference availability and generalization challenges by building a reference scene distribution from pristine views and combining it with a distortion-aware discriminator.The framework is reported to perform robustly across NeRF- and GS-based datasets.Source: §2 · p. 4
3. Method
SCODA combines scene-level semantic fidelity with patch-level distortion awareness, then fuses the cues through calibrated, bounded corrections. Its fidelity branch compares a view with a learned distribution of high-quality scene features rather than an explicit reference image.
3. Method
- SCODA combines a scene-fidelity branch and a distortion-aware realism branch, with default weak supervision limited to coarse groups for discriminator training.The fidelity branch and fusion stage are unsupervised, and no component regresses MOS as a continuous target.Source: §3 · p. 4
3.1. Semantic Fidelity Measure
- For each scene, SCODA models pristine-view features with a multivariate Gaussian and scores a test view by its covariance-normalized deviation from that distribution.The resulting Mahalanobis distance down-weights directions with high natural variation and penalizes deviations along stable scene-specific directions more strongly.Source: §3.1 · pp. 4–5
3.2. Distortion-Aware Discriminator
- A patch discriminator learns distortion-sensitive realism from coarse clean/degraded pseudo-labels, complementing the Gaussian model's semantic fidelity signal.Attention-weighted aggregation gives more weight to spatial regions indicative of distortions, while suppressing perceptually irrelevant areas.Source: §3.2 · p. 5
3.3. Score Fusion
- Fusion keeps fidelity primary and applies a softly clipped realism residual, with trust weights reducing correction strength at extreme fidelity scores.Small disagreements remain nearly linear, while large disagreements saturate to improve stability.Source: §3.3 · p. 6
- Before fusion, non-parametric quantile alignment matches the realism score's marginal distribution to the fidelity score using calibration scenes, without MOS-based score fitting.Evaluation is conducted on held-out scenes.Source: §3.3 · p. 6
4. Evaluation
SCODA is evaluated against full-, no-, and cross-reference metrics on 3DGS images, GS trajectories, and NeRF views. It performs strongly across these settings, with dataset-specific differences and ablations examining attention, supervision, reference-view count, and fidelity features.
Benchmark Datasets
- Evaluation spans 15,200 images from 10 scenes, 64 GS-QA video sequences, and 102 NeRF stimuli; cross-dataset training and calibration use 3DGS-IEval before transfer.The datasets cover compressed GS images, continuous camera trajectories, and NeRF representation generalization.Source: §4 · p. 6
Quantitative Results
- On 3DGS-IEval-15K, SCODA achieves the best overall correlation; on GS-QA it is comparable to DeepDC, and on NeRF QA it remains among the strongest methods while DISTS leads.The comparison includes SRCC, PLCC, and KRCC across all three datasets.Source: §4 · p. 7, Table 1 · p. 7
Effect of Attention and Training Size
- Attention consistently improves discriminator performance over uniform aggregation, and useful learning is reported with only a small subset of scenes.Training and calibration use a subset of 10 scenes, with held-out scenes for evaluation.Source: §4 · p. 7, Table 2 · p. 7
Supervision Analysis for the Realism Branch
- MOS-based pseudo-labels perform best; MOS-free QF quantiles come closest, while DISTS quantiles are weaker but still improve the fused score.The comparison uses five held-out scenes, totaling 7,600 images.Source: Table 3 · p. 7, §4 · p. 8
Fusion Mechanism Analysis
- With 10% of reference views—about 17–30 per scene—the standalone manifold model drops in performance, while the fused model remains substantially more stable.The figure reports medians over 10 random trials at each reference-view fraction.Source: §4 · p. 8, Figure 4 · p. 8
Backbone / PCA Ablation
- ResNet-50 layer3 performs strongest for fidelity, PCA performance improves from 10% to 30% retained components then saturates, and inference takes about 0.10 seconds per image on CPU.The reported image size is 1237 × 822; the authors report lower inference overhead than competing methods.Source: §4 · p. 8, Table 4 · p. 8
5. Conclusions
SCODA assesses radiance-field renderings against a scene-level reference distribution rather than an aligned or neighboring image. Its fidelity and realism cues are fused, and experiments report agreement with human judgments across 3DGS images, GS trajectories, and NeRF views.
Conclusions
- SCODA models high-quality scene observations as a reference distribution and scores each rendering by deviation from that scene-level model.Source: §5 · p. 8
- A distortion-aware realism signal complements fidelity, with bounded corrections applied only when the two cues disagree.Source: §5 · p. 8
- Across 3DGS images, GS trajectories, and NeRF-generated views, experiments show strong agreement with human judgments and robust generalization.Source: §5 · p. 8