PaperlayerView exact revision on arXiv

Source-linked AI summary

View-Structured Conformal Prediction for 3D Gaussian Splatting

Junzheng Chu, Bin Pan, Zhenwei Shi

arXiv:2609.10307v1cs.LGcs.CV

TL;DR

3DGS uncertainty heatmaps and pixel-pooled calibration do not certify that individual novel views meet a required pixel-coverage event. VSCP factorizes spatial and view difficulty structure, while View-CP calibrates over views; it achieves substantially higher view-event coverage than pixel pooling and transfers beyond the source setting.

  • Problem

    3DGS uncertainty fields are mainly evaluated by pixel-ordering metrics, while pixel pooling controls average pixel coverage rather than the probability that a new view covers a required fraction of pixels.

  • Method

    VSCP combines a renderer-structured spatial scale with a transferable view-difficulty factor and uses a held-out view-level conformal quantile to construct finite-sample valid RGB boxes.

  • Results

    View-CP reaches 91.7–92.0% view-event coverage at a 90% target, versus 61.4% for pixel pooling, and the approach transfers to an unseen dataset family while beating the closest single-model baseline at 1/10 of the training cost.

  • Takeaways & Limitations

    Separating spatial width from view-level validity makes conformal RGB intervals both statistically targeted to new views and efficient under rendering structure.

  • Takeaways & Limitations

    The guarantee is marginal over exchangeable views, not conditional coverage for every fixed camera pose.

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) renders novel views in real time, but an uncertainty heatmap does not certify that a rendered view meets a certain prediction coverage. We treat novel-view synthesis as structured regression and ask that, with probability at least $1-α$, RGB prediction boxes cover at least a $1-β$ fraction of pixels in a new view. We propose View-Structured Conformal Prediction (VSCP). It splits the pre-calibration scale into a spatial shape from the renderer and a transferable view-difficulty factor, which predicts the smallest view-wise multiplier that shape needs. A held-out quantile over views (View-CP) then gives finite-sample validity even when transferring to new scenes. The same factorization makes the analysis exact: a conformity score is the ratio of oracle to predicted view difficulty, and excess width separates into a test-side and a calibration-side term. Across 13 real scenes, pixel-pooled calibration reaches 89.9\% marginal pixel coverage but only 61.4\% view-event coverage at a 90\% target, while View-CP reaches 91.7--92.0\%. At matched coverage VSCP cuts width by 22.1\% against a constant scale, and matches a ten-model ensemble's 21.0\% reduction using only one model per scene and four rather than ten rasterization passes per query. VSCP also improves on the closest single-model baseline, the 3DGS-U field, by 4.7 points ($p=0.0225$). The view predictor transfers from bounded source families to all nine unbounded Mip-NeRF~360 scenes. There the full scale beats the constant scale with 20.7\% width saving on all nine scenes. It also keeps an 18.3\% saving under a different densification backbone and runs at 216--280 FPS on an RTX~4090.

1 INTRODUCTION

The paper reframes uncertainty for 3DGS as a view-level coverage problem and proposes VSCP, which factorizes spatial and view difficulty before calibrating a finite-sample-valid multiplier over held-out views.

  • Motivation: With probability at least 1 −α, the method targets coverage of at least a 1 −β fraction of pixels in a new view.This treats views, rather than dependent pixels, as the statistical unit.
  • Method: VSCP factorizes the scale into a renderer-derived spatial shape and a transferable view-difficulty factor, then applies a held-out view quantile.The view factor predicts the smallest multiplier needed for the fixed spatial shape.
  • Analysis: The conformity score is the ratio of oracle to predicted view difficulty, yielding an exact excess-width decomposition rather than only a bound.The decomposition separates test-view error from calibration-view order statistics.
  • Results: VSCP transfers across scene families and matches a ten-model ensemble’s width reduction while using one model per scene and four rather than ten rasterization passes.The introduction also reports improvement over the closest single-model baseline.

2 RELATED WORK

Prior work estimates uncertainty in radiance fields and applies conformal prediction to structured or adaptive outputs, while this paper targets finite-sample view-event guarantees with cross-scene view factors.

  • Uncertainty for radiance fields and 3DGS: Existing radiance-field and 3DGS methods estimate uncertainty, information, visibility, or residual-derived fields for prediction, pruning, view selection, and reconstruction.The paper distinguishes its target from these uncertainty representations.
  • Structured and image-valued conformal prediction: Structured conformal methods provide image-valued intervals, heterogeneous scalar intervals, risk control, structured prediction, or grouped-pixel calibration.These methods motivate conformal treatment of structured outputs but do not supply the paper’s specific view event.
  • Adaptive conformal scales: Input-adaptive conformal scales learn heterogeneous thresholds or predictive distributions, whereas this work preserves an exact marginal-over-views guarantee while using structure to reduce width.The paper’s excess-width analysis complements general non-asymptotic analyses.
  • Relation to 3DGS: The method reuses renderer view dependence twice: residual fields shape within-view widths, while directional observation support predicts whole-view widening.It does not modify the 3DGS renderer.

3 PROBLEM SETUP: VIEW-STRUCTURED COVERAGE

The paper models each rendered view as a structured regression response and defines RGB boxes whose success is the event that at least a required fraction of pixels is covered.

  • 3DGS as functional regression: A view is one covariate producing an image-valued response, so conformal calibration treats views as observations while allowing arbitrary pixel dependence.The renderer maps camera and image-formation conditions to an RGB image, while the true image is unknown.
  • 3DGS as functional regression: 3DGS point rendering combines alpha-transmittance contributions from Gaussians with view-dependent spherical-harmonic colors, and this rendering equation only constructs the scale.The proposed calibration does not modify the point predictor.
  • RGB boxes: The method uses scalar RGB residuals and axis-aligned RGB boxes whose sup-norm condition is equivalent to residuals not exceeding the scaled half-width.Unclipped half-width is the width metric; clipping preserves containment but changes that functional.
  • View event: The view event requires at least a 1 −β fraction of pixels to be covered with probability 1 −α over a new view exchangeable with calibration views.This is marginal over views, not conditional coverage for every fixed camera pose.

4 VIEW-STRUCTURED CONFORMAL PREDICTION

VSCP combines a renderer-derived spatial shape with a label-free transferable view factor, then uses view-level split conformal calibration to produce valid RGB boxes without target-label feature selection.

  • Overview: VSCP is a factorized scale under View-CP, the view-level conformal calibration layer applied consistently to the baselines.The method’s guarantee concerns the explicit view event rather than pixel-pooled coverage.
  • Data separation: Target-scene training, source-scene meta-training, target calibration, and test evaluation have disjoint roles, preserving the split-conformal guarantee.The target scene receives no feature or shrinkage updates from calibration labels.
  • Scale construction: The spatial shape is learned from target-scene residual attribution, while the view predictor uses label-free camera, rendering, exposure, and directional-support features.The spatial field is a relative shape whose overall scale is removed before calibration.
  • View factor: The view-regression target is the smallest multiplier that makes a fixed spatial shape reach the required within-view coverage.It is computed from pixel residual quantiles on source scenes and modeled with a ridge-regularized log-linear predictor.
  • View-CP calibration: Calibration computes view-wise residual quantiles, takes a discrete order statistic over calibration views, and outputs boxes with half-width bq aV ebV p.The resulting width varies across pixels and views while bq is a single scene-level multiplier.
  • Implementation: The complete query uses one backbone and four rasterization passes by packing auxiliary per-Gaussian scalars into three additional RGB rasterizations.Packing is an algebraic reorganization rather than an approximation.

5 VALIDITY AND EFFICIENCY

View-structured calibration treats views, not pixels, as the exchangeable units, yielding finite-sample view-event validity and an exact decomposition of efficiency into prediction and calibration effects.

  • Validity: The view event is equivalent to exceeding the discrete order statistic of pixelwise ratios, so interpolated quantiles would break the guarantee.This equivalence follows because within-view coverage is the empirical distribution function of the ratios.
  • Validity: 1 −α + 1/(m + 1) is the finite-sample view-event coverage guarantee when calibration and test scores are exchangeable and ties are absent.The guarantee applies to boxes calibrated by the stated conformal procedure.
  • Validity: Target-scene mismatch can change interval width but not coverage, provided target calibration and test views are exchangeable given the fixed construction.The fixed objects include the trained renderer, source data, and fitted scale construction.
  • Efficiency: The oracle view factor is the smallest multiplier that attains the within-view event, making the conformity score a ratio of oracle to used view difficulty.This factorization motivates predicting log view difficulty rather than average regression accuracy.
  • Efficiency: At α = 0.1 with the smallest usable calibration size m = 9, one underestimated calibration view determines the width inflation.Global multiplicative bias cancels exactly, while the lowest calibration error controls the inflation.

6 EXPERIMENTS

Across 13 scenes, View-CP controls the view-level coverage event that pixel pooling misses, while VSCP reduces interval width through structured spatial and view factors, transfers across scene families, and remains efficient across backbones.

  • 6.2 DOES CALIBRATION CONTROL THE RIGHT EVENT?: 91.7–92.0% view-event coverage is achieved by View-CP, versus 61.4% for pixel-pooled calibration despite 89.9% marginal pixel coverage at the 90% target.The failure persists across operating points: pooled calibration stays nearly flat while View-CP tracks the requested level.
  • 6.3 MATCHED-COVERAGE SCALE EFFICIENCY: 4.7 points is the gain over 3DGS-U across 13 scenes, with 11/13 wins and p = 0.0225.This is the main same-center comparison; VSCP makes no efficiency claim over the ten-model ensemble, whose paired gap is 1.2 points with p = 1.0.
  • 6.4 CROSS-FAMILY TRANSFER: 20.7% width saving against the constant scale is retained on all nine Mip-NeRF 360 scenes without refitting after training only on Tanks & Temples and Deep Blending.A source pool containing Mip-NeRF 360 improves the saving to 21.9%, only 1.2 points higher.
  • 6.3 MATCHED-COVERAGE SCALE EFFICIENCY: 22.1% width saving results from separately estimating and shrinking spatial shape and view difficulty, compared with 17.4% for a scene-global shrinkage factor.Table 2 separates the value of view information from the benefit of the factorized structure.
  • 6.5 FACTORIZATION, METRIC BLIND SPOTS, AND THEORY AUDIT: 39.0% oracle-factor performance under strict holdout identifies view-difficulty prediction as the main remaining room for improvement, while the two factors remain complementary.Figure 2(c) reports complementarity across all nine target scenes.
  • 6 EXPERIMENTS: 18.3% width saving survives replacement of FastGS density control with vanilla 3DGS rules despite 5.6–7.7× more Gaussians, while exact channel packing enables 216–280 FPS.The reported throughput is specific to FastGS and the stated hardware.

7 CONCLUSION

Reliable 3DGS rendering is formulated as structured regression with view-level calibration for pixel-coverage events. Renderer-derived spatial and angular structure controls interval width, while conformal quantiles protect validity.

  • View-CP calibrates the event that most pixels in a new 3DGS view are covered, rather than calibrating pooled pixels or relying on uncertainty rankings.

A ADDITIONAL EFFICIENCY RESULTS

The efficiency analysis identifies the oracle view factor and connects prediction error to excess interval width. It shows that better view-difficulty prediction can reduce width when view difficulty is heterogeneous.

  • The oracle factor equals view difficulty up to a global constant when the inner noise quantile is view-invariant.With bounded noise-quantile variation and factor prediction error, the width ratio is bounded by e2(ϵ+η).
  • The factorization yields an excess-width bound of e2ϵ − 1 = 2ϵ + O(ϵ2) when deployed log-factor errors are uniformly bounded by ϵ.
  • A perfect view model replaces an upper quantile of view difficulty with its mean, with larger gains when view difficulty is more heterogeneous.This exact finite-sample identity holds numerically to 6.2 × 10−7 over all 13 scenes.

B PROOFS

The proofs establish finite-sample validity through exchangeable view-level scores and characterize how scale factorization affects ranking metrics and conformal width. Global multiplicative scale changes cancel exactly.

  • Multiplying a positive scale by a global constant leaves every calibrated interval unchanged because the calibration multiplier cancels the rescaling.This invariance also underlies the paper’s normalization conventions and multiplicative non-identifiability.
  • View-CP validity follows from exchangeability of calibration and test scores, with finite thresholds requiring kα ≤ m and otherwise becoming uninformative.
  • Permutation-equivariant construction from a whole query batch preserves score exchangeability when calibration views are uniformly split after scale construction.Separate normalization constants for calibration and test batches would break this argument.
  • Per-view ranking metrics cannot detect the view factor, even though that factor divides conformal width.Thus AUSE and Spearman correlation can miss the scale information needed for calibrated interval size.

C IMPLEMENTATION AND PROTOCOL DETAILS

The implementation separates source learning, target calibration, and test evaluation while using unlabeled query-batch covariates. Protocol analyses show that normalization and calibration unit determine valid coverage and reported width.

  • Target test views supply only camera poses and renderer features, while target calibration images enter only the final conformal order statistic.The protocol keeps source learning, target calibration, and test evaluation in disjoint roles.
  • 22.1% saving is reported for VSCP under scene-level normalization, while per-view normalization would make the comparison misleadingly stronger.The selected table values use the stronger normalization for all three baselines.
  • View-CP fails on 7.9% of views versus 38.9% for pixel-pooled CP when the distribution is checked against the view-event corner.Across three scale maps, the failure ranges are 7.9–8.3% for View-CP and 38.0–40.0% for pooled CP.

D.5 FACTORIZATION, RANKING METRICS, AND THEORY AUDIT

Across controlled studies, VSCP preserves view-event coverage while improving interval efficiency, transfers across scene families and backbones, and exposes why calibration tails matter. Its gains are supported by per-scene comparisons, predictive-center controls, and runtime measurements.

  • Factorization and theory audit: The theory audit attributes most excess width to the calibration tail: the test-side factor is 0.975, the calibration-tail factor is 1.420, and the observed oracle width ratio is 1.384.The three identities hold to relative error below 1.1 × 10^-6 across 13 scenes and 200 splits; one underestimated calibration view determines the relevant order statistic.
  • Runtime: 216–280 FPS is achieved on an RTX 4090 after reducing the implementation from eight to four rasterization passes without changing predictions.The reported figures use one target-scene model and are specific to FastGS and this hardware.
  • Per-scene comparisons: 4.7 points separate VSCP from 3DGS-U, with VSCP winning on 11 of 13 scenes while losses remain small on kitchen and train.Against the ensemble, results are mixed: VSCP wins on 7 of 13 scenes, supporting parity rather than an advantage.
  • Predictive-center and efficiency controls: 21.0% ensemble-level width reduction is matched using one model per scene and four rather than ten rasterization passes per query.The comparison concerns how the view-difficulty signal is obtained, not a claimed width advantage over the ensemble.
  • Predictive-center and efficiency controls: 20.0% versus 18.1% saving under strict holdout shows that using the ensemble mean as predictive center leaves the conclusion unchanged.The paired gap is 1.9 points, with 5/9 wins and p = 1.0.

E LIMITATIONS

The guarantee is marginal over exchangeable views rather than conditional for every camera pose, and several data, scene, representation, and comparison settings remain outside scope.

  • Statistical scope: The guarantee is marginal over exchangeable views, so ordered camera paths can break the exchangeability assumption.It does not provide conditional coverage for every camera pose.
  • Data and transfer scope: The method requires labeled target-scene views to be held out for calibration, and using them to train 3DGS would require a different calibration argument.Only one held-out family is large enough to be useful, and family is confounded with bounded versus unbounded capture.
  • Representation and evaluation scope: Axis-aligned RGB boxes with a shared radius exclude richer color geometry, dynamic scenes, and time-correlated calibration units.The 13-scene results use FastGS; the vanilla-densification study covers only three scenes and changes only the density rule.
Loading 2609.10307v1…