Source-linked AI summary
FreeView: Robust 3D Reconstruction from Uncurated Views
Junho Kim, Hyunyoung Choi, Seongwon Lee
TL;DR
Uncurated image collections may contain unrelated views, and reference-conditioned filtering needs a trustworthy reference that is not guaranteed to be available. FreeView selects a representative reference through collection consensus and adapts view rejection using visual and geometric evidence. Across multiple datasets, it reports high clean-view retention and distractor rejection with reliable reconstruction.
Problem
Reference-conditioned filtering needs a trustworthy reference, but uncurated collections do not guarantee that the selected input view belongs to the target scene.
Source: §I · p. 2Method
FreeView uses collection-level consensus to select a representative filtering reference, then combines visual and geometry-aware evidence for adaptive view rejection.
Source: §I · p. 2Results
Experiments across multiple datasets report high clean-view retention and distractor rejection, with robust filtering across distractor levels and reference conditions.
Source: §I · p. 2, B. Experimental Results · p. 7Takeaways & Limitations
Abstract
from arXiv · showhide
Reconstructing a coherent 3D scene from uncurated image collections remains challenging due to the presence of irrelevant or non-overlapping views. Recent work has shown that feed-forward reconstruction models contain internal view-consistency signals that support distractor rejection. However, reference-conditioned filtering relies on a curated clean reference, which is not guaranteed in practical image collections. To address this limitation, we introduce FreeView, a training-free framework that jointly selects a filtering reference and rejects distractor views from an uncurated collection. FreeView leverages collection-level consensus to identify a representative view that serves as a reliable reference for filtering. It then adapts view rejection to each input collection by combining complementary visual and geometric evidence. This approach preserves useful views under substantial appearance and viewpoint variations while rejecting distractors. FreeView integrates with both VGGT and pi^3, removing the need for a manually curated reference or manual calibration of view rejection. Extensive experiments across multiple datasets demonstrate high clean-view retention and distractor rejection rates, leading to reliable 3D reconstruction from uncurated views.
I. INTRODUCTION
Uncurated image collections can mix views of unrelated scenes, while reference-conditioned filtering depends on a trustworthy reference that may not be known in advance. FreeView selects a representative reference through collection-level consensus and adapts view rejection using visual and geometric evidence.
I. INTRODUCTION
- Reference-conditioned filtering can misclassify clean views when its reference is a distractor, so uncurated collections require the reference itself to be inferred.A distractor reference may make clean views appear inconsistent while giving unrelated images stronger support.Source: §I · p. 2
- FreeView combines consensus-based reference selection with adaptive visual and geometric rejection to curate views for reconstruction with VGGT or π3.The framework is training-free and requires neither a pre-verified clean reference nor manual calibration of rejection.Source: §I · p. 2
- Adaptive Dual-Cue Rejection retains views supported by either visual or geometry-aware evidence and rejects those unsupported by both.The decision adapts to each collection’s appearance, viewpoint, and cross-view agreement.Source: §I · p. 2
- Experiments across multiple datasets report high clean-view retention and distractor rejection, with reliable reconstruction from uncurated collections.FreeView is evaluated with two reconstruction backbones and across varying distractor levels and reference conditions.Source: §I · p. 2
A. Feed-Forward 3D Reconstruction
Feed-forward reconstruction advances geometric prediction, but uncurated inputs also require selecting which views should participate. FreeView addresses this input-set curation problem through a framework that selects a reference and filters views.
A. Feed-Forward 3D Reconstruction
- Permutation equivariance handles changes in input ordering, whereas view curation determines which observations participate in reconstruction.FreeView provides this curation framework for both VGGT and π3.Source: A. Feed-Forward 3D Reconstruction · p. 3
B. Robust View Filtering for 3D Reconstruction
- Reference-conditioned filtering can fail with an unrelated reference, and fixed rejection criteria may not adapt to collection-specific variation.FreeView infers a representative anchor from collection-level consensus, then rejects views using visual and backbone-derived geometric cues.Source: B. Robust View Filtering for 3D… · p. 3
C. Reference Selection and Visual Retrieval
- FreeView uses pairwise DINOv2 similarities to identify a collection-supported anchor, using visual similarity for reference selection and view retention alongside geometric evidence.This goal differs from retrieval’s task of finding images related to a query.Source: C. Reference Selection and Visual… · p. 3
III. METHODOLOGY
The preliminaries describe the outputs of feed-forward reconstruction backbones and reference-conditioned view filtering. The retained set depends on the chosen filtering reference.
III. METHODOLOGY
- VGGT and π3 predict camera parameters, per-pixel depth, 3D pointmaps, and confidence maps from multiple images.VGGT anchors reconstruction to the first input camera, whereas π3 uses permutation-equivariant reasoning in relative coordinate systems.Source: A. Preliminaries · p. 3
- Reference-conditioned filtering measures each view’s relevance to a reference using backbone representations and rejects views below a fixed threshold.The retained set is defined relative to the filtering reference.Source: A. Preliminaries · p. 3
- Because the retained set depends on the filtering reference, selecting that reference remains necessary even when a backbone does not use a fixed reconstruction reference.π3’s permutation-equivariant geometry prediction does not remove this filtering dependency.Source: A. Preliminaries · p. 3
B. Problem Formulation
An uncurated collection contains target-scene views and distractors, but their membership is unknown and no clean reference is provided. FreeView seeks a filtering reference and a retained subset for reconstruction.
B. Problem Formulation
- The collection is partitioned into target-scene views and distractors, whose membership is unknown at inference time.No clean reference is supplied.Source: B. Problem Formulation · p. 3
- The target scene is assumed to form a coherent group with stronger local visual support than competing distractor groups, without needing a majority.Source: B. Problem Formulation · p. 3
- FreeView aims to select a target-scene filtering reference and retain target views while excluding distractors for feed-forward reconstruction.The retained subset is used to estimate scene geometry and camera poses.Source: B. Problem Formulation · p. 3
C. Consensus Reference Selection
Consensus Reference Selection infers a filtering reference from mutual agreement among views, using local support so the target views need not dominate the full collection.
C. Consensus Reference Selection
- The filtering reference is inferred from mutual agreement because no input view is known beforehand to represent the target scene reliably.Source: C. Consensus Reference Selection · p. 4
- Each view’s consensus score uses support from its K most-similar neighbors rather than agreement with the entire collection.Source: C. Consensus Reference Selection · p. 4
- The method uses K = max(3, ⌊N/3⌋), which favors a coherent local subset without requiring target views to be a collection majority.Source: C. Consensus Reference Selection · p. 4
- After reference selection, VGGT places that view first and recomputes multi-view features, while π3 needs no reordering because it is permutation-equivariant.Appearance similarities remain unchanged because their descriptors are extracted independently for each image.Source: C. Consensus Reference Selection · p. 4
D. Adaptive Dual-Cue Rejection
Adaptive Dual-Cue Rejection combines appearance and backbone-derived geometric evidence, adapting thresholds to each collection and rejecting a view only when both cues indicate low relevance.
D. Adaptive Dual-Cue Rejection
- Appearance relevance reuses the DINOv2 similarity from reference selection, while geometry-aware relevance uses backbone features as a complementary signal.The backbone-derived features encode cross-view geometric consistency.Source: D. Adaptive Dual-Cue Rejection · p. 4
- Collection-specific appearance and geometry thresholds are set independently with Otsu’s criterion, which separates low- and high-relevance score groups.Source: D. Adaptive Dual-Cue Rejection · pp. 4, 6
- A cue’s rejection is suppressed when its resulting relevance groups have mean separation below γ = 0.04, a value fixed across experiments.Source: D. Adaptive Dual-Cue Rejection · p. 6
- A view is rejected only when both appearance and geometry-aware relevance fall below their respective thresholds, preserving views when cues disagree.Source: D. Adaptive Dual-Cue Rejection · p. 6
- The rejection rule runs for two rounds with the reference fixed and thresholds recomputed over the currently retained views each round.The final retained set goes to the selected reconstruction backbone.Source: D. Adaptive Dual-Cue Rejection · p. 6
A. Experimental Setup
The evaluation constructs controlled uncurated collections across three multi-view datasets and compares reference-selection settings, filtering metrics, and coverage-aware pose recall.
A. Experimental Setup
- Phototourism and On-the-Go collections use 30 clean views plus 10, 30, or 50 distractors; ETH3D uses 14 clean views plus 5, 14, or 30 distractors.Distractors are progressively added while the clean subset remains fixed.Source: A. Experimental Setup · p. 6
- The protocol compares Clean-only, clean-reference, distractor-reference, and automatic FreeView selection, using five paired reference choices per scene.MegaLoc and DINO thresholds are tuned per dataset on Clean Reference and held fixed for Distractor Reference evaluation.Source: A. Experimental Setup · p. 6
- Filtering is measured by clean-view keep rate and distractor rejection rate, while pose recall counts rejected clean views as failures.This coverage-aware protocol avoids evaluating pose accuracy only on retained views.Source: A. Experimental Setup · pp. 6–7
B. Experimental Results
FreeView maintains robust filtering and camera reconstruction across distractor levels and both VGGT and π3 backbones. Reference-conditioned methods can fail when given a distractor reference, while qualitative results show FreeView preserving target views and coherent reconstruction.
B. Experimental Results
- With distractor references, VGGT-based methods collapse in clean-view retention and pose recall, while FreeView achieves consistently strong filtering and camera reconstruction across noise levels.With clean references, retrieval-based methods and RobustVGGT effectively remove distractors and maintain strong pose recall.Source: B. Experimental Results · p. 7
- π3's permutation-equivariant prediction does not prevent RobustPi3 from degrading with a distractor reference; FreeView's consistent improvement is not specific to VGGT.The comparison also shows that π3-based filtering still depends on a designated reference.Source: B. Experimental Results · p. 7
- Across Phototourism, ETH3D, and On-the-Go, both FreeView variants remain robust from Small to Large distractor levels, avoiding catastrophic failures with incorrect references.The reported results show consistent reference selection and view filtering across VGGT and π3.Source: B. Experimental Results · p. 7
- Qualitatively, VGGT retains distractors, RobustVGGT depends on reference quality, and FreeView preserves clean views while suppressing distractors to recover coherent reconstruction.With a distractor reference, RobustVGGT rejects valid target views; with a clean reference, residual unrelated-landmark views still contaminate reconstruction.Source: B. Experimental Results · p. 7
C. Ablation and Analysis
FreeView combines consensus-based reference selection with adaptive dual-cue rejection and integrates with VGGT and π3. The ablations examine the components under Large noise, the neighborhood size for consensus, and how cue fusion balances clean-view retention against distractor rejection.
C. Ablation and Analysis
- Under Large noise with a distractor reference, reference selection alone substantially recovers performance across three datasets, while adaptive rejection adds dataset-dependent improvements.The ablations examine design choices under the benchmark's highest distractor level.Source: C. Ablation and Analysis · p. 7
- On Phototourism under Large noise, minimum-neighbor consensus lowers reference accuracy and clean-view retention; K = max(3, ⌊N/3⌋) performs most consistently.Performance improves as the neighborhood expands and remains stable over a broad intermediate range; N/3 is the default setting.Source: C. Ablation and Analysis · p. 7
- DINOv2 appearance performs particularly well on Phototourism and On-the-Go, while geometry remains competitive on ETH3D; conservative AND rejection preserves clean views better than OR rejection or averaging.The two cues capture different aspects of relevance, and OR rejection or direct averaging frequently discards valid target-scene views.Source: C. Ablation and Analysis · p. 7
V. CONCLUSIONS
- FreeView combines Consensus Reference Selection and Adaptive Dual-Cue Rejection, integrates with VGGT and π3, and achieves high clean-view retention and distractor rejection across datasets.The conclusion reports reliable reconstruction across varying distractor levels and reference conditions, without a pre-verified clean reference or manual rejection calibration.Source: §V · p. 7