Source-linked AI summary
Per-View Gaussian Predictions Enable Training-Free Distractor Filtering in Feed-Forward 3DGS
Kangmin Seo, Jae-Pil Heo
TL;DR
Transient objects in casual multi-view captures can persist in feed-forward 3D Gaussian reconstructions and create artifacts in novel views, while existing robustness approaches may require model-specific adaptation. This paper introduces training-free filtering from a single frozen prediction, consistently reducing distractor artifacts across models and benchmarks while preserving clean-scene quality.
Problem
Feed-forward reconstruction commonly assumes static, view-consistent scenes, but distractors in only some views can persist and cause artifacts, while robustness may require backbone-specific adaptation.
Method
The method excludes view-associated Gaussians to propose inconsistent regions, verifies candidate removal against other views, and filters the final subsets without retraining or rerunning the frozen model.
Results
Across three reconstruction models and two distractor benchmarks, filtering consistently improves reconstruction quality, while clean-scene quality is largely preserved across four models.
Takeaways & Limitations
Distractor robustness can be added as a drop-in post-processing step using the reconstruction itself while leaving the trained model untouched.
Takeaways & Limitations
Low feature similarity can also indicate poorly reconstructed static content or regions unobserved by remaining views, requiring opacity and depth restrictions during proposal generation.
Abstract
from arXiv · showhide
Feed-forward 3D Gaussian Splatting reconstructs an explicit Gaussian representation from multiple input images in one network execution, making 3D reconstruction increasingly accessible for casual captures. However, such captures frequently contain transient objects that appear in only a subset of the views. Such content can be encoded into the per-view Gaussians associated with the inputs that observe it and remain in the combined representation despite being observed by no other input. As a result, it may produce blurred, duplicated, or floating artifacts in novel views. We introduce a training-free filtering procedure that exploits this per-view prediction structure. For each input, we exclude its associated Gaussians and render the same camera using the remaining representation, revealing content that is inconsistent with the other inputs. Feature similarity forms candidate regions, and rendering-based verification retains only candidates whose removal reduces reconstruction error in the other input views. The procedure operates on a single frozen prediction without retraining or scene-specific optimization. Across three reconstruction models and two distractor benchmarks, it consistently improves novel-view quality with varying numbers of input views. On clean scenes, evaluations across four models show that the original reconstructions are largely preserved.
Introduction
Feed-forward 3DGS can preserve transient, view-inconsistent objects through per-view Gaussian predictions, causing artifacts in novel views. The proposed training-free procedure repurposes this per-view structure to detect and filter distractors from a single frozen reconstruction.
- Transient objects appearing in only some input images can create blurry regions, duplicated appearance, and floating artifacts in novel views.
- Per-view Gaussian associations can preserve content from an individual input even when that content conflicts with the remaining observations.
- Excluding one input’s associated Gaussians exposes inconsistent content, while rendering-based removal tests identify candidates whose deletion improves agreement with other views.
- The method filters distractor-associated Gaussians after one frozen-model execution without retraining, learned masks, predefined categories, or rerunning the model.
- Experiments use DepthSplat, ReSplat, and YoNoSplat with RobustNeRF and NeRF On-the-Go across varying input-view counts, plus clean-scene preservation, ablation, and runtime analyses.
Related Work
Related work spans generalizable feed-forward Gaussian reconstruction and distractor handling through optimization-based, feature-based, transient-separation, and learned-mask approaches. In contrast, this method decouples filtering from reconstruction, operating once on a fixed-input Gaussian prediction without additional training or learned mask prediction.
- Feed-forward Gaussian reconstruction progressed from image-pair prediction to sparse posed views, learned depth estimation, less constrained inputs, recurrent refinement, and token-aligned prediction.
- Distractor-free synthesis has used optimization-based NeRF and 3DGS methods to model transients, suppress inconsistent observations, exploit uncertainty, or separate static and transient representations.
- Recent feed-forward methods add distractor-aware training, auxiliary mask supervision, or reference-image mask prediction and refinement before pruning.
- This method decouples distractor handling from reconstruction, requiring one fixed-input Gaussian prediction with no additional training, predefined categories, or learned mask prediction.Unlike DGGS's reported protocol, it assumes only a fixed input set rather than a scene image pool, executes reconstruction once, and filters the resulting Gaussian representation.
Method
The method filters transient distractors from a single frozen feed-forward Gaussian prediction by exploiting view-associated Gaussian subsets. It proposes inconsistent regions using feature, opacity, and depth cues, then retains removals only when rendering-based verification improves agreement with other input views.
- View-associated Gaussian representation: A feed-forward model predicts Gaussian subsets associated with input views in one network execution, enabling temporary exclusion and immediate rendering of any subset.The representation remains frozen, with no clean images, distractor masks, additional input-selection images, retraining, or scene-specific optimization.
- Proposal generation: For each input, excluding its associated Gaussians exposes content unlikely to be reproduced by the remaining views, and DINOv3 feature similarity identifies mismatched regions.Cosine feature similarity is used because it is less sensitive than direct RGB comparison to small appearance differences.
- Proposal generation: Opacity and depth conditions restrict proposals by rejecting largely empty regions and requiring exclusion to reveal a farther surface.Similarity boundaries τ1 < τ2 define two proposal ranges, which are converted into connected component masks and candidate Gaussian subsets using model-specific associations.
- Candidate verification: Each candidate is verified on other input cameras by measuring rendering-change-weighted reconstruction-error reduction outside proposal regions.The verification cameras are selected by projected-center visibility, and candidates with positive individual scores proceed to final Gaussian selection.
- Candidate verification: First-range candidates require only individual verification, whereas weaker second-range candidates undergo additional per-view and joint verification before retention.If the combined second-range verification fails, those candidates are discarded; selected Gaussians are removed by setting their opacity to zero.
Experiments
Experiments across distractor benchmarks and multiple feed-forward reconstruction models show that filtering consistently improves reconstruction quality while suppressing transient-object artifacts. Ablations, clean-scene evaluations, and runtime analysis further characterize the method’s mechanism, preservation behavior, and cost.
- Quantitative Results: Across all evaluated models, datasets, and input counts, filtering improves PSNR, SSIM, and LPIPS, with gains attributable to filtering alone.Each before-and-after pair uses identical images, model weights, and initial Gaussian predictions.
- Qualitative Results: Filtering suppresses blurred, duplicated, and floating artifacts from transient people and objects while largely preserving nearby static scene content.The qualitative comparisons use the same enlarged image regions across methods.
- Ablation Studies: Removing verification causes the largest ablation degradation, while DINOv3 similarity and dual proposal ranges provide further gains over simpler configurations.The ablation averages ten scenes from both datasets and three backbones in the 4-view setting.
- Comparison with In-the-Wild Feed-Forward Model: Filtering improves frozen AnySplat predictions across both distractor datasets and both four- and six-view settings, outperforming the released GenWildSplat on average.All methods use cameras estimated by AnySplat, including proposal generation and verification for the proposed method.
- Preservation on Clean Scenes: On four clean RobustNeRF scenes, filtering leaves original reconstructions nearly unchanged across four reconstruction models under both posed and estimated-pose settings.Clean-input proposal regions are often rejected during verification, unlike semantic masking that can remove static objects in predefined transient categories.
- Overhead Analysis: Filtering runs once after Gaussian prediction and produces a single filtered representation, so novel-view rendering requires no reselection or reconstruction; verification dominates runtime.Verification renders each candidate from multiple views, and its cost grows with the input count.
Conclusion
Per-view Gaussian predictions enable training-free distractor filtering in feed-forward 3DGS by exposing inconsistent content and verifying candidate removal through rendering. The procedure reduces distractor artifacts while preserving reconstruction quality on clean scenes.
- Method: Per-view prediction structure enables training-free distractor filtering by excluding each input’s Gaussian subset to expose content inconsistent with remaining observations.Rendering-based verification retains candidates whose removal improves reconstruction.
- Results: The resulting procedure consistently reduces distractor artifacts across multiple reconstruction models and distractor benchmarks while preserving reconstruction quality on clean scenes.