PaperlayerView exact revision on arXiv

Source-linked AI summary

MVLoMa: Multi-View Local Feature Matching with Any Geometry Foundation Model

Xinyang Lu, Shengzhe Wei, Sheng Tang, Weiqi Zhang, Juntong Fang, Zesong Li

arXiv:2610.31321v1cs.CV

TL;DR

Pairwise matchers repeat centre-side work and cannot use evidence across the full group of overlapping images. MVLoMa reuses frozen multi-view geometry features in a star-graph matcher with geometry-aware supervision. It achieves the strongest macro-average relative-pose AUC among compared methods at all three thresholds on 25-view stars.

  • Problem

    Pairwise matchers duplicate reference-side computation and cannot exploit complementary geometric evidence across the remaining observations.

    Source: §1 · p. 1

  • Method

    MVLoMa reuses frozen multi-view geometry tokens in a star matcher that jointly produces centre–neighbour matches with visibility-filtered positives and geometric negative supervision.

    Source: §1 · pp. 1–2

  • Results

    MVLoMa leads macro-average relative-pose AUC at all three thresholds in the 25-view star-graph evaluation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The ablations indicate that gains depend on both the frozen representation and proposed supervision, while additional view context is useful but not uniformly beneficial.

    Source: §6 · p. 7

    Training uses 2–8 views versus 25-view evaluation, added observations are not uniformly beneficial, and interaction with complete global SfM systems is outside the evaluation scope.

    Source: §5 · p. 7

Abstract

from arXiv · show

Local feature matching is typically formulated for image pairs, although structure-from-motion and visual localization commonly match one reference image against many retrieved or covisible neighbours. Applying a pairwise matcher independently to these centre-neighbour edges duplicates reference-side computation and prevents the matcher from exploiting complementary multi-view geometric evidence from the remaining observations. To address these limitations, we introduce MVLoMa, a sparse star-graph matcher that extends LoMa from image pairs to multiple views. MVLoMa first uses a frozen multi-view geometry model to jointly describe all images in a star, then injects the resulting cross-view geometric context into local descriptors. It then matches the centre image to all neighbours in a single forward pass through per-view self-attention and bidirectional centre-neighbour cross-attention. For supervision, we derive positive correspondences only from depth-valid, mutually visible dense warps and explicitly penalize high-confidence matches that violate epipolar geometry. On 25-view star graphs from HyperSim, ScanNet++, Mip-NeRF 360, and Tanks and Temples, MVLoMa obtains the best macro-average relative-pose AUC among the compared methods at all three thresholds (58.65/85.41/90.99). Its matcher is 2.20 times faster than sequential evaluation of the 24 centre-neighbour edges. The same design supports four tested frozen geometry encoders -- DA3, $\pi^3$, VGGT, and VGGT-$\Omega$ -- with DA3-Giant performing best on all reported encoder-ablation metrics. Controlled ablations further isolate the effects of geometry-aware descriptors, visibility filtering, geometric negative supervision, and joint multi-view inference.

1 Introduction

MVLoMa addresses the mismatch between pairwise matching and multi-image geometric evidence by jointly matching a star while reusing frozen multi-view geometry features. It combines geometry-aware descriptors, star-structured inference, and visibility- and geometry-aware supervision.

1 Introduction

  • Pairwise matching repeats reference-side computation and decides matches without evidence from other neighbours, where a third view can resolve ambiguity.Existing groupwise or dense multi-view matchers are not designed for repeated sparse centre–neighbour assignments, while batching pairwise matchers does not allow communication between edges.Source: §1 · p. 1
  • MVLoMa reuses spatially aligned, cross-view-fused final-layer tokens from a frozen 3D foundation model as geometry-aware local descriptor inputs.A light trainable descriptor head replaces LoMa’s frozen DINOv2 stream, avoiding another frozen feature stream while keeping the foundation encoder fixed.Source: §1 · pp. 1–2
  • A star-structured matcher shares the centre representation across edges and alternates per-view self-attention with bidirectional centre–neighbour cross-attention.It produces conventional score matrices for each edge, preserving explicit pairwise outputs for robust pose estimation.Source: §1 · p. 2
  • Training uses positives only from depth-valid, mutually visible dense warps and penalizes predicted matches with epipolar error above 3 px.The geometric negative loss targets high-confidence but geometrically wrong matches that can survive mutual matching and harm RANSAC.Source: §1 · p. 2
  • MVLoMa leads macro-average relative-pose AUC at all three thresholds under the 25-view star-graph protocol.The evaluation compares the same centre–neighbour edges for every method; dense two-view RoMa remains stronger on Mip-NeRF 360 at the strictest threshold.Source: §1 · p. 2

2 Related Work

Prior matching work differs in correspondence density, representation, and use of multi-view context. MVLoMa moves contextual reasoning to star graphs while retaining sparse pairwise outputs and a specialized assignment head.

2.1 Sparse and Dense Feature Matching

  • Pairwise sparse and dense matchers differ in representation, but their assignment remains fundamentally pairwise.MVLoMa inherits LoMa’s local descriptor refinement and dual-softmax interface while shifting contextual reasoning from an image pair to a star graph.Source: §2.1 · p. 2

2.2 Multi-View Correspondence and Tracks

  • Unlike groupwise or track-based multi-view methods, MVLoMa uses centre–neighbour communication and preserves an individual score matrix and confidence for each edge.This retains the sparse matching and RANSAC interface without post-hoc track snapping.Source: §2.2 · p. 2

2.3 Multi-View Geometric Foundation Models

  • MVLoMa uses frozen last-layer, cross-view-fused tokens from a multi-view geometry model as input to a specialized sparse matcher.Unlike direct track decoding, this keeps the assignment head specialized and leaves the geometry encoder unchanged.Source: §2.3 · p. 2

2.4 Matching in Reconstruction Systems

  • MVLoMa targets the correspondence-module bottleneck in reconstruction systems as a matching primitive, not a replacement for global optimization.Optimization-based SfM still builds tracks and geometrically verifies pairwise matches.Source: §2.4 · p. 3

3 Method

MVLoMa addresses pairwise matching’s duplicated centre-side computation and inability to use evidence across neighbours by jointly representing and matching a star of images. It combines frozen multi-view geometry features, shared-centre star attention, and visibility-filtered and geometry-aware supervision.

3.1 Problem Formulation

  • MVLoMa formulates matching as simultaneous sparse correspondences between a centre image and every neighbour, avoiding independent pairwise calls that duplicate centre computation and exclude evidence from other edges.The frozen foundation features encode the complete star before shared-centre matching, with conventional pairwise assignment recovered at the output.Source: §3.1 · p. 3

3.2 Geometry-Aware Multi-View Descriptors

  • Frozen, spatially aligned foundation-model tokens already encode cross-view context, replacing LoMa’s frozen DINOv2-L stream while retaining its trainable VGG and convolutional-refiner structure.A frozen encoder must expose spatially aligned per-image tokens after multi-view fusion; DA3-Giant is the default, with π3, VGGT, and VGGT-Ω also evaluated.Source: §3.2 · p. 3

3.3 Star-Graph Attention and Assignment

  • Each matcher layer applies self-attention within views, then bidirectional centre–neighbour cross-attention; neighbours do not attend to one another, while the centre aggregates evidence across observations.Pairwise assignment remains separate per edge, but shared centre descriptors let one centre keypoint associate with multiple neighbours, forming candidate star tracks without separate track snapping.Source: §3.3 · p. 3, Figure 1 · p. 4

3.4 Visibility-Filtered Ground Truth and Geometric Negatives

  • Positive matches require valid source depth, in-image reprojection, and satisfied occlusion and target-depth checks; high-confidence geometric outliers are explicitly penalized using epipolar error.The geometric negatives are mined from final-layer mutually maximal matches above the confidence threshold, and the default epipolar threshold is 3 px.Source: §3.4 · p. 4

4 Experiments

Experiments compare matching on fixed centre–neighbour edges in 25-frame star graphs, then isolate supervision, descriptor, encoder, view-count, and computational effects. MVLoMa leads the reported macro-average at all three pose-AUC thresholds, with gains and boundaries that vary by dataset and setting.

4.1 Protocol and Implementation

  • Experiments use 25-frame sets with frame 0 as the centre and the same 24 centre–neighbour edges for every method; pose is estimated with PoseLib RANSAC.AUC@1°/5°/10° combines rotation and translation error; MV-RoMa uses 13-view groups because its 25-view configuration exhausts memory.Source: §4.1 · pp. 4–5, §4.2 · p. 5

4.2 Comparison to Matching Baselines

  • On Mip-NeRF 360, dense RoMa leads at AUC@1°, while MVLoMa is the strongest multi-view method at 5° and 10°; LoMa-L leads on Tanks and Temples.These comparisons position multi-view matching as complementary to dense two-view matching, rather than a universal replacement.Source: §4.2 · p. 5

4.3 Ablations

  • Removing geometric negative supervision lowers HyperSim AUC@1° from 52.84 to 51.04, while removing GT filtering lowers it to 51.93.Both components improve results on HyperSim and Mip-NeRF 360; filtering excludes invalid dense-warp positives, while the negative term targets high-confidence false matches.Source: §4.3 · p. 5, Table 2 · p. 6
  • DA3-G outperforms both DINOv2 variants in the controlled frozen-backbone ablation, exceeding DINOv2-G by 1.16, 11.44, 0.41, and 0.65 AUC@1° points across four test subsets.The comparison holds training data, resolution, star matcher, losses, and optimization configuration fixed.Source: §4.3 · p. 5, Table 3 · p. 6
  • DA3-G is best on all reported frozen-encoder metrics; on HyperSim AUC@1°, it exceeds the other encoders by 1.62–2.40 points.VGGT-Ω is close on MegaDepth-X and TerraSky3D, and the ablation does not establish that any single DA3-G property explains the difference.Source: §4.3 · p. 5, Table 4 · p. 7

4.2 Comparison to Matching Baselines

  • 58.65/85.41/90.99 are MVLoMa’s best macro-average AUC@1°/5°/10° scores among compared methods on 25-view star graphs.At 784 resolution, AUC@1° also improves over LoMa-L by 1.91 points on HyperSim and 0.76 points on ScanNet++.Source: §4.2 · p. 5, Table 1 · p. 6

4.3 Ablations

  • At 25 views, star matching reduces Transformer FLOPs by 37.0% and latency by 54.6% versus sequential pairwise evaluation, a 2.20× speedup.The matcher-only peak memory is higher: 979 MiB for star matching versus 176 MiB for sequential pairwise evaluation.Source: §4.3 · pp. 6–7, Table 6 · p. 7
  • At two inference views, macro-average AUC falls to 62.21/86.63/91.34 at 1°/5°/10°, approximately 0.8 points below the best multi-view result at each threshold.The fixed-edge comparison isolates loss of joint interaction; beyond two views, trends vary by dataset and threshold, so added views are not uniformly beneficial.Source: §4.3 · pp. 5–6, Figure 2 · p. 6, Table 5 · p. 7

5 Reproducibility and Limitations

MVLoMa’s reproducibility materials describe the implementation and promise release of the implementation and evaluation configuration upon publication. The stated scope includes dependence on frozen-feature quality, memory growth, and evaluation through relative pose estimation.

5 Reproducibility and Limitations

  • The implementation and evaluation configuration will be released upon publication.The main paper and Appendix A describe the architecture, loss, evaluation topology, RANSAC setting, keypoint budget, frozen and trainable components, and training schedules.Source: §5 · p. 7
  • MVLoMa depends on the geometric quality of frozen foundation features; textureless or uncertain regions can yield sparse or ambiguous matches.Source: §5 · p. 7
  • The 2–8-view training range is smaller than the 25-view evaluation star, and additional views are not uniformly beneficial.The view-count ablation shows useful extrapolation, but its results are non-monotonic.Source: §5 · p. 7
  • The centre–neighbour topology does not directly exchange information between neighbours, and memory grows with keypoints and views.Source: §5 · p. 7
  • The evaluation measures relative pose estimation; interaction with complete global SfM systems is outside the work’s scope.Source: §5 · p. 7

6 Conclusion

MVLoMa reuses frozen geometry-aware tokens and shares the centre representation across a star while preserving sparse pairwise outputs. Across four benchmarks, the 784 model leads macro-average AUC at all three thresholds and runs 2.20× faster than sequential pairwise evaluation.

6 Conclusion

  • The 784 model attains the strongest macro-average AUC among compared methods at all three thresholds across four benchmarks.Source: §6 · p. 7
  • The 25-view matcher is 2.20× faster than sequential pairwise evaluation.Source: §6 · p. 7
  • MVLoMa reuses frozen geometry-aware tokens and shares the centre representation across a star while preserving ordinary sparse pairwise outputs for pose estimation.Source: §6 · p. 7
  • Ablations show that gains depend on both the frozen representation and proposed supervision, while additional context is useful but not uniformly beneficial.Source: §6 · p. 7

A Implementation Details

The implementation uses frozen geometry features alongside trainable matching components, with 25-view evaluation and training that progresses from fixed four-view stars to variable view counts. Appendix details also identify the final DA3-G training-data mixture.

A Implementation Details

  • Evaluation covers all 24 centre–neighbour edges in each 25-frame set, while encoder and view-count ablations sample 2–8 views.Inference uses 4096 DaD keypoints per image; the foundation encoder remains frozen while the VGG stream, refiner, descriptor projection, and matcher are optimized jointly.Source: A Implementation Details · p. 9
  • Geometric negatives use a 3 px epipolar threshold and weight 1/(N −1).The fixed-view ground-truth and loss ablation uses LoMa-L initialization, four views, batch size one, and 100k iterations at resolution 784.Source: A Implementation Details · p. 9
  • Table 7 presents the final DA3-G training-data mixture; its caption defines MVS as multi-view stereo and identifies dataset-sampling and geometric negative-loss weights.The accompanying text names MegaDepth, BlendedMVS, and TartanAir among datasets used in addition to those cited in the main text.Source: Table 7 · p. 9, A Implementation Details · p. 9
  • Final experiments use 784 × 784 inputs, LoMa-L initialization, AdamW, batch size one, and 200k iterations on one NVIDIA A800 GPU.The first 100k iterations use fixed four-view stars, followed by 100k iterations with views sampled uniformly from 2–8.Source: A Implementation Details · p. 9

B Qualitative Visualizations

The appendix collects qualitative MVLoMa visualizations from development experiments, grouped by dataset. Each star visualization is shown beneath its original RGB views.

B Qualitative Visualizations

Related papers

Loading 2610.31321v1…