Long-tail Internet photo reconstruction

1. Introduction

• Most Internet landmarks have only sparse, noisy images, unlike famous sites that are photographed from many angles and reconstructed accurately by standard SfM.The paper describes these sparsely represented scenes as the long-tail of online photo collections and says they are the norm.Source: §1 · pp. 1–2
• Classical methods struggle to find correspondences across sparse or wide-baseline views, while learned models trained on clean, dense captures often recover inconsistent geometry from long-tail imagery.The learned-model passage names DUSt3R and VGGT as examples.Source: §1 · p. 2
• MegaDepth-X is a clean, dense-depth-enhanced dataset, 7× larger than MegaDepth, paired with sampling that favors wide baselines and partial overlap over clustered views.The dataset is built from Internet photo reconstructions with depth refinement and manual checks against references such as maps and satellite imagery.Source: §1 · p. 2
• Fine-tuning with MegaDepth-X and sparsity-aware sampling improves robustness on long-tail collections, including doppelganger scenes where classical SfM and pretrained models often fail.The paper identifies the Calvaire de Plougonven example as one such scene.Source: §1 · p. 2

Feed-forward 3D reconstruction

• Feed-forward models predict 3D attributes directly from images and perform well on dense, well-conditioned scenes, but remain suboptimal on sparse, noisy Internet photos.The passage surveys DUSt3R, MASt3R, VGGT, Fast3R, FLARE, π3, ZipMap, and Scal3R.Source: §2 · p. 2

Long-tail challenges in 3D vision

• Long-tailed problems arise when common cases dominate data while rare cases are individually scarce, as in object recognition and safety-critical driving events.Source: §2 · p. 2
• MegaScenes contains unreconstructed or incorrectly reconstructed scenes; proposed remedies include stronger features and matchers, learned wide-baseline pose relations, and pruning false correspondences.The cited work on pruning false matches uses classifiers during structure-from-motion.Source: §2 · pp. 2–3
• Because available reconstructors do not reliably derive pseudo-ground-truth from sparse natural collections, the paper proposes subsampling well-conditioned collections to simulate long-tail views.This approach is presented as a way to balance training scene distributions for regression models.Source: §2 · p. 3

3. The MegaDepth-X Dataset

• MegaDepth-X is built from well-reconstructed MegaScenes scenes to provide clean, large-scale, depth-refined supervision where long-tail scenes often lack usable reconstructions.The passage identifies noise, dynamic content, and ambiguities as reliability challenges.Source: §3 · p. 3

3.1. Filtering and Disambiguation

• Candidate scenes with more than 100 registered images can still fail when dynamic content attracts matches or geographically distant lookalikes are registered together.These errors can produce incorrect camera poses and fragmented, inconsistent point clouds.Source: §3.1 · p. 3, Figure 2 · p. 3
• The pipeline excludes scenes dominated by crowds or moving objects, uses MASt3R-SfM with Doppelganger classification to prune suspicious edges, then manually checks reconstructions against maps and satellite imagery.Source: §3.1 · p. 3

3.2. Dense Depth Refinement

• MVS depth maps contain depth bleeding and noisy transient-object regions; MegaDepth refinement reduces some artifacts but still leaves bleeding and depends on a manually designated category list.The figure contrasts these residual artifacts with the additional filtering step.Source: §3.2 · p. 3, Figure 3 · p. 4
• Monocular depth-guided filtering aligns geometric depth to monocular predictions and removes pixels with excessive depth or gradient discrepancies, filtering bleeding and transient-object noise without manual category lists.The method uses MoGe2 depth predictions as ordinal priors.Source: §3.2 · p. 4

3.3. Dataset Statistics

• The final MegaDepth-X dataset contains 1,865 reconstructions totaling 440k images, with 127 scenes reserved for testing.The dataset began with 2,474 candidate scenes; 609 were filtered for dynamic content, reconstruction errors, or geometric inconsistencies.Source: §3.3 · p. 4

4.1. Defining Properties of Long-Tail Scenes

• Scenes with low registration rates (e.g., 20% registered) have 8% of cameras at degree two or less versus 3% in well-reconstructed head scenes; verified matches average 294.8 versus 395.3 in head scenes.Source: §4.1 · p. 4
• Sampling should provide viewpoint diversity and wide baselines while preserving enough within-component covisibility for local reconstructability.The stated requirements are diversity, sparsity, and local reconstructability; zero-overlap samples within a component can produce unstable training signals and difficult optimization.Source: §4.1 · pp. 4–5

4.2. Sparsity-Aware Sampling Strategy

• The task samples N views into at most Ncc connected components, avoiding naive subsampling that can produce zero-overlap sets or clusters biased toward dense regions.Components may be disconnected from one another, but each sampled component should retain sufficient internal covisibility.Source: §4.2 · p. 5
• The method detects viewpoint communities in a filtered view graph, then uses an approximate Steiner tree to connect representative views with necessary intermediate nodes.Edges with fewer than 50 feature matches are pruned before community detection; the connected subgraph spans the selected community representatives.Source: §4.2 · p. 5
• Greedy sampling prioritizes unseen communities and spatially distant views; search depth controls spread, with local-neighborhood sampling filling any remaining view slots.The algorithm ranks candidate neighbors lexicographically by community novelty and spatial distance, then repeats for D iterations; the figure contrasts D = 24 with D = 12 for N = 24 and Ncc = 1.Source: §4.2 · pp. 5–6, Figure 4 · p. 5

5.1. Experimental Setup

• The setup finetunes π3 and VGGT on MD-X with the proposed sampling strategy, and evaluates 24-image MD-X samples at easy and hard sparsity levels.Finetuning updates only the Alternating-Attention modules while keeping point-cloud and camera decoders frozen; easy uses D = 5, Ncc = 1, and hard uses D = 24, Ncc = 4.Source: §5.1 · p. 6, §5.2 · p. 6

5.2. Internet Photo Evaluation

• On MegaDepth-X, finetuning markedly improves π3 and VGGT over their pretrained baselines across camera-pose and point-map metrics, with larger gains observed in harder, sparser scenes.Source: §5.2 · p. 6, Table 1 · p. 6
• Unfiltered DIRTY training reduces accuracy and can underperform the pretrained model on point maps, while MIXED sampling achieves slightly better overall performance across difficulty levels.RANDOM sampling gives reasonable camera-pose accuracy but limited point-map improvement; DENSE is less effective in sparse scenes, and SPARSE alone does not yield the best trade-off.Source: §5.2 · p. 7, Table 2 · p. 7
• On MD-X and real long-tail scenes, the finetuned model produces denser, more coherent reconstructions than pretrained π3, including from very few views and in doppelganger cases.Figure 5 shows clearer gains in the harder setting; Figure 6 includes a scene where COLMAP registered zero images.Source: §5.2 · p. 7, Figure 5 · p. 7, Figure 6 · p. 8

5.3. Generalization to Standard Benchmarks

• On RealEstate10K and CO3Dv2, finetuning generally maintains both backbones’ relative-pose performance and modestly improves VGGT.The evaluation follows π3’s pose-sampling conventions.Source: §5.3 · p. 7, Table 3 · p. 7
• Point-map accuracy remains comparable on DTU, 7-Scenes, and NRGBD, but decreases on ETH3D, with a mild VGGT drop under sparse NRGBD evaluation.The authors attribute the ETH3D decrease likely to the domain gap between controlled datasets and Internet imagery.Source: §5.3 · pp. 7–8, Table 4 · p. 8, Table 5 · p. 8

6. Conclusion

• The dataset and sampling strategy improve recovery of consistent geometry from sparse, noisy, and ambiguous imagery, including doppelganger scenes where classical SfM and feed-forward models fail.The conclusion also reports maintained generalization across benchmarks.Source: §6 · p. 8
• The work presents a step toward robust, Internet-scale 3D reconstruction by defining and addressing the long-tail regime of Internet photo collections.Source: §6 · p. 8
• The dataset focuses on landmark-scale scenes, which represent only a small fraction of Internet photos.Source: §6 · p. 8

Visualization Webpage

• The project page adds sampling animations, pretrained-versus-finetuned reconstruction comparisons, point-cloud flythroughs, and qualitative results on real-world scenes.The comparisons include long-tail scenes where COLMAP registers zero images.Source: Visualization Webpage · p. 11

A.1. Data Processing

• The data-processing pipeline mitigates reconstruction errors in ambiguous scenes and filters background leakage and transient-object depths, prioritizing accurate depth maps over complete ones.Monocular depth guides filtering rather than warping MVS depth, because uncertain relative-depth predictions may add cross-view noise and inconsistency.Source: §A.1 · p. 11

A.2. Dataset Statistics

• MegaDepth-X contains substantially more scenes with broad camera-position coverage and diverse viewing directions than MegaDepth.The dataset comparison reports reconstruction statistics and camera-coverage metrics; azimuth coverage counts occupied bins across the full 360° range.Source: §A.2 · p. 11, Table 6 · p. 12

B.1. Greedy Sampling Algorithm

• Greedy sampling prioritizes previously unvisited camera communities, then selects farther camera positions to diversify viewpoints and increase baselines.Candidates are lexicographically ranked by community novelty and spatial distance.Source: §B.1 · pp. 11–12

B.2. Graph Partition

• The graph partition method assigns nodes through parallel round-robin breadth-first searches from randomly selected seed nodes.Source: §B.2 · p. 12

B.3. Graph Span vs. Search Depth

• The graph analysis measures full-graph coverage and sampled-subset sparsity; higher graph and Euclidean dispersion means sampled views are farther apart.Coverage includes k-hop reach and average nearest-sample distance; dispersion uses pairwise distances within the sampled set.Source: §B.3 · pp. 12–13
• Deeper greedy searches yield higher coverage of the full view graph and sparser, more widely distributed sampled subsets.The analysis uses 24 sampled views across 12 search depths, averaging eight runs for each depth on the 100 scenes with the most registered images.Source: §B.3 · p. 13, Figure 7 · p. 13

C.1. Training Setup

• Finetuning uses the original π3 and VGGT losses, updates only Alternating-Attention modules, and includes BlendedMVS and TartanAir training data.The point-cloud and camera decoders remain frozen during 100 epochs of finetuning.Source: §C.1 · p. 13

C.2. Additional Depth-Estimation Results

• The finetuned models maintain competitive depth-estimation performance across the reported monocular and video datasets.Evaluation uses Abs Rel and accuracy at δ < 1.25 on Sintel, Bonn, KITTI, and NYU-v2 for monocular depth, and Sintel, Bonn, and KITTI for video depth under scale and scale-and-shift alignment.Source: §C.2 · p. 13

• Fine-tuned π3 distinguishes visually similar structures that COLMAP and pretrained models merge, recovering geometry consistent with aerial references and global layout.Source: §C.3 · pp. 13–14, Figure 8 · p. 14
• The illustrated evaluation compares pretrained π3 with variants fine-tuned using different sampling strategies on doppelganger scenes.Source: §C.3 · p. 14, Figure 9 · p. 14
• Sparsity-aware fine-tuning, including mixed or sparse sampling, tends to improve disambiguation, while pretrained and dense-only models are less robust.Source: §C.3 · p. 14, Figure 9 · p. 14

C.4. Quantitative results on Long-tail scenes

• Across evaluated long-tail scenes, the model consistently reduces mean relative rotation and translation errors while producing more complete point clouds.Source: §C.4 · pp. 14–15, Figure 10 · p. 15

C.5. Limitations

• When image subsets depict disconnected regions without overlap, both pretrained and fine-tuned π3 may fuse them into one structure, and robust reasoning about those components remains challenging.Source: §C.5 · p. 15, Figure 11 · p. 16
• Replacing COLMAP with MASt3R-SfM and doppelganger++ separates similar landmark structures and registers more images in a low-texture scene.Source: Figure 12 · p. 16
• The filtered dense-depth strategy uses monocular ordering priors to mitigate depth bleeding, prioritizing accurate maps over complete ones.Source: Figure 13 · p. 16