Spackle: Completing Large View Single Image NVS with Adaptive Gaussians

• Prior decoupled frameworks use diffusion-generated large-view image pairs to train efficient feed-forward 3DGS models, avoiding generative sampling at deployment [1] [14] [13] [27].Source: §1 · pp. 1–2
• Under large viewpoint changes, fixed-budget models shift Gaussians from seen regions to disocclusions, degrading fidelity and detail in the original content.The paper terms this failure mode capacity competition.Source: §1 · pp. 2–3
• Spackle detects poorly reconstructed disocclusions with a binary mask, trains a residual 3DGS only there, and combines its Gaussians with the pretrained backbone at inference.Source: §1 · p. 2
• Spackle achieves state-of-the-art performance in large-view single-image NVS, according to the paper’s comprehensive experiments.Source: §1 · p. 3

2.1 Single Image Novel View Synthesis (NVS)

• Earlier image-space and 3D-representation methods improved view synthesis but remained limited to local extrapolation, failing under large viewpoint deviations.Intermediate representations included layered depth images, MPIs, point-based models, and conditional neural radiance fields.Source: §2.1 · p. 3
• Feedforward 3DGS enables efficient rendering without per-scene optimization, but fixed Gaussian budgets struggle with disoccluded regions during large viewpoint shifts.Generative approaches such as ZeroNVS and CAT3D [7] can produce photorealistic disoccluded content at extreme deviations, but iterative sampling incurs prohibitive computational cost.Source: §2.1 · p. 3

2.2 Decoupled Frameworks with Generated Supervision

• Spackle unifies feedforward reconstruction and generative-model strengths for large-view-deviation NVS while maintaining computational efficiency.It proposes residual learning to mitigate capacity competition in decoupled methods without sacrificing efficiency.Source: §2.1 · p. 3, §2.2 · p. 3
• Decoupled methods transfer generated supervision or generative priors into 3D representations to provide useful signals for occluded or unseen regions.Examples include Lyra [1], DiFix3D+ [38], and RI3D.Source: §2.2 · p. 3
• Spackle trains a residual 3DGS on baseline RGB, depth, opacity, and mask cues, using ProPainter [45] pseudo targets for masked regions.At inference, only predictions inside the binary mask become augmented Gaussians, which are combined with baseline Gaussians to improve unseen regions while preserving visible-region fidelity.Source: Figure 2 · p. 4

3.1 Base 3DGS Prediction

• The pretrained SHARP model predicts the baseline 3DGS from the input image and renders it from the target viewpoint.Spackle can also be applied to other feed-forward 3DGS-based NVS models.Source: §3.1 · p. 4

3.2 Occlusion Identification

• A binary mask marks low-opacity target-view pixels as occluded or unobserved, and its marked regions guide residual-model optimization.Pixels outside the mask correspond to reliably reconstructed visible areas.Source: §3.2 · pp. 4–5

3.3 Residual 3DGS Optimization

• The residual strategy preserves the fixed total number of Gaussians during training while improving occluded-region reconstruction without reallocating capacity from visible areas or modifying the base Gaussians.The baseline Gaussians remain unchanged, preserving visible-region fidelity.Source: §3.3 · p. 5

3.4 Residual-Guided Inference

• At inference, Spackle combines the baseline Gaussians with augmented Gaussians restricted to the mask to render the final novel view.In the qualitative comparison, Spackle better reconstructs unseen regions while preserving visible-region fidelity under large viewpoint changes.Source: §3.4 · p. 5, Figure 3 · p. 6

4.1 Experimental Setup

• Training uses 20,000 generated target views from 1,000 COCO 2017 images, with the baseline frozen and only the residual 3DGS optimized.Evaluation covers six datasets and three increasingly challenging disparity settings from 0.3 to 0.5.Source: §4.1 · pp. 6–7

4.2 Main Results

• Under large viewpoint deviations, Spackle reconstructs disoccluded regions more completely than baseline 3DGS while preserving the appearance of visible areas.Qualitative comparisons show more coherent structures with fewer missing regions.Source: §4.2 · p. 7
• CLIPIQA improves across most datasets and settings, especially at larger deviations, while NIQE and MUSIQ changes are smaller or mixed.The authors report competitive perceptual quality alongside improved visual completeness.Source: §4.2 · pp. 7–8, Table 1 · p. 7
• Spackle consistently reduces Hole Area Ratio across six datasets and three disparity settings, indicating fewer uncovered pixels in large-view renderings.Table 2 evaluates hole coverage under controlled camera changes that expose newly disoccluded regions.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 Ablation Studies

• Directly injecting generated supervision into baseline 3DGS degrades DISTS and LPIPS, whereas Spackle preserves the baseline and trains a residual model on poorly reconstructed regions.The authors report improved large-view reconstruction without degrading standard-view rendering quality.Source: §4.3 · p. 8
• Binary-mask-guided allocation introduces fewer augmented Gaussians than dense grid initialization while maintaining nearly identical DISTS and LPIPS.The mask constrains additional Gaussians to poorly reconstructed unseen regions.Source: §4.3 · p. 8

4.4 Efficiency and Gaussian Analysis

• 11.5K→28.0K augmented Gaussians as disparity rises from 0.3 to 0.5; median runtime overhead rises from 2.7% at S1 to 4.4% at S3, with 8.02M added parameters (1.14%).The Gaussian and runtime measurements are reported in Table 3; the parameter overhead appears in the expanded analysis.Source: §4.4 · p. 9, Table 3 · p. 9

5 Conclusion and Limitations

• Spackle allocates additional Gaussian capacity to occluded regions while retaining feed-forward rendering and improving large-view completion.Generative completion priors provide pseudo supervision during training; inference does not require diffusion or video-inpainting models.Source: §5 · p. 9
• Performance relies on accurate disocclusion estimation and pseudo-supervision quality, and may degrade when the baseline 3DGS model is poorly initialized.The authors also note modest computational overhead.Source: §5 · p. 9

A Additional Quantitative Ablations

• Spackle achieves comparable or better results than SHARP in most cases, while the naive decoupled framework worsens DISTS and LPIPS across datasets.The naive framework directly optimizes the baseline with pseudo supervision; Spackle preserves it and trains augmented Gaussians for newly disoccluded regions.Source: Table 4 · p. 13, A Additional Quantitative Ablations · p. 13
• Disocclusion-guided initialization reduces auxiliary Gaussians compared with dense-grid initialization while maintaining rendering quality.The average number of initialized auxiliary Gaussians is reported per novel view.Source: Table 5 · p. 13, A Additional Quantitative Ablations · p. 13

B Additional Qualitative Analysis of the Naive Decoupled Framework

• Directly applying generated supervision to the baseline can improve some newly disoccluded regions while degrading already well-reconstructed visible regions.This is the qualitative failure mode illustrated for the naive decoupled framework under large viewpoint changes.Source: B Additional Qualitative Analysis of… · p. 13
• Spackle preserves the baseline 3DGS model and trains a residual 3DGS exclusively for unseen regions that the binary mask marks as poorly reconstructed.Source: B Additional Qualitative Analysis of… · p. 13

C Expanded Efficiency and Gaussian Overhead Analysis

• Spackle maintains moderate runtime overhead while adaptively adding Gaussian capacity as larger disparities expose more disoccluded regions.The efficiency analysis covers controlled large-view settings and reports merged Gaussians, runtime, FPS, and evaluated sample counts.Source: C Expanded Efficiency and Gaussian… · pp. 13–14
• Spackle adds 8.02M parameters (1.14%) to SHARP's 702.31M, for 710.32M total, while SHARP uses a fixed 1,179,648-Gaussian base.The table notes that a negative time overhead can reflect runtime measurement outliers.Source: Table 6 · p. 14
• The parameter overhead remains constant across settings because the same Spackle module is used for all views.Source: C Expanded Efficiency and Gaussian… · p. 15

D Pseudo Supervised Dataset Construction

• The training set samples 1,000 COCO 2017 unlabeled images and renders 20 controlled camera views per image, producing 20,000 training views.The 20 views include the original input view and rendered novel views.Source: D Pseudo Supervised Dataset Construction · p. 15
• Disocclusion masks mark pixels with opacity below a threshold as uncovered, and camera metadata pairs each pseudo target with its target viewpoint.Saved metadata includes trajectory type, camera positions, intrinsics, extrinsics, and resolution.Source: D Pseudo Supervised Dataset Construction · p. 15
• ProPainter completes masked trajectory frames with temporal consistency to create pseudo targets, but is used only during dataset construction, not inference.The authors select it as a non-diffusion alternative that is more controllable and efficient than iterative diffusion-based generation.Source: D Pseudo Supervised Dataset Construction · p. 15

E Training Details and Loss Weights

• Spackle alone is optimized while the pretrained base remains frozen; training uses a 1.6 × 10−4 learning rate, a 0.95 opacity threshold, and at least 64 auxiliary Gaussians per view.Source: E Training Details and Loss Weights · p. 15
• The four-term objective weights color, alpha, perceptual, and non-hole preservation losses at 10.0, 0.5, 1.0, and 5.0, respectively; preservation matches detached base output outside holes.Source: E Training Details and Loss Weights · p. 15

F Broader Impacts

• Spackle may benefit virtual and augmented reality, 3D content creation, and embodied intelligence, but can produce plausible incorrect scene content, so the authors frame it as a research tool, not verified evidence.Source: F Broader Impacts · p. 15

G Existing Assets and Licenses

• Training uses COCO 2017 unlabeled, SHARP [17], and ProPainter; evaluation uses Middlebury, Booster, WildRGBD, ETH3D, LLFF, and Tanks and Temples.Source: G Existing Assets and Licenses · p. 16