PaperlayerView exact revision on arXiv

Source-linked AI summary

LoMa: Local Feature Matching Revisited

David Nordström, Johan Edstedt, Georg Bökman, Jonathan Astermark, Anders Heyden, Viktor Larsson, Mårten Wadenbäck, Michael Felsberg, Fredrik Kahl

arXiv:2604.04931v2cs.CV

TL;DR

Local feature matching has lagged behind data-scaled vision methods, while common benchmarks often use relatively easy pairs and are approaching saturation. LoMa addresses both issues with scaled data, training, models, and compute, alongside the challenging manually annotated HardMatch benchmark. The resulting models achieve state-of-the-art performance on HardMatch and established benchmarks, while still struggling on some difficult HardMatch subgroups.

  • Problem

    Local feature matching has progressed more slowly than newer data-scaled approaches, and evaluations based on successful 3D reconstructions are mostly saturated.

  • Method

    LoMa combines large diverse training mixtures, modern training recipes, increased model capacity and compute, while HardMatch provides 1000 manually annotated challenging image pairs.

  • Results

    LoMa achieves state-of-the-art performance on HardMatch and established benchmarks, surpassing even dense matchers in the reported evaluations.

  • Takeaways & Limitations

    Large-scale data and modern training practices can substantially improve local feature matching across diverse benchmarks.

  • Takeaways & Limitations

    LoMa still struggles on HardMatch subgroups including Doppelgängers and extreme viewpoint changes, and descriptor training tends to overfit.

Abstract

from arXiv · show

Local feature matching has long been a fundamental component of 3D vision systems such as Structure-from-Motion (SfM), yet progress has lagged behind the rapid advances of modern data-driven approaches. The newer approaches, such as feed-forward reconstruction models, have benefited extensively from scaling dataset sizes, whereas local feature matching models are still only trained on a few mid-sized datasets. In this paper, we revisit local feature matching from a data-driven perspective. In our approach, which we call LoMa, we combine large and diverse data mixtures, modern training recipes, scaled model capacity, and scaled compute, resulting in remarkable gains in performance. Since current standard benchmarks mainly rely on collecting sparse views from successful 3D reconstructions, the evaluation of progress in feature matching has been limited to relatively easy image pairs. To address the resulting saturation of benchmarks, we collect 1000 highly challenging image pairs from internet data into a new dataset called HardMatch. Ground truth correspondences for HardMatch are obtained via manual annotation by the authors. In our extensive benchmarking suite, we find that LoMa makes outstanding progress across the board, outperforming the state-of-the-art method ALIKED+LightGlue by +18.6 mAA on HardMatch, +29.5 mAA on WxBS, +21.4 (1m, 10$^\circ$) on InLoc, +24.2 AUC on RUBIK, and +12.4 mAA on IMC 2022. We release our code and models publicly at https://github.com/davnords/LoMa.

1 Introduction

LoMa revisits local feature matching by scaling data, training practices, model capacity, and compute, while HardMatch addresses saturation in existing evaluations with challenging annotated pairs.

  • Local feature matching remains widely used in SfM, but newer detector-free and feed-forward approaches have benefited from much larger training datasets.
  • LoMa combines diverse training data, modern recipes, increased data scale, and larger model capacity to improve matching across many benchmarks.
  • Existing benchmarks derived from successful 3D reconstructions are nearing saturation, making robustness and generalization improvements difficult to observe.
  • HardMatch contains 1000 hand-labeled image pairs from 100 categories, organized into 9 challenging groups for more demanding evaluation.
  • LoMa achieves +18.6mAA over LightGlue on HardMatch and strong results across more than ten established matching and localization benchmarks.

2 Related Work

Related work spans sparse local matching, detector-free and dense matching, and benchmark protocols based on reconstructed scenes or manually labeled correspondences.

  • Local feature matching traditionally detects keypoints, describes local appearance, and matches descriptors, with learned models replacing handcrafted methods such as SIFT and ORB.
  • SuperGlue adds graph attention for global reasoning over local descriptors, while LightGlue improves speed through pruning, early stopping, and layer-wise supervision.
  • Detector-free and dense methods eliminate sparse keypoints or match every pixel, and have topped several matching benchmarks since DKM.
  • Common evaluation uses relative pose estimation or visual localization on sparse views from successful 3D reconstructions, leaving many benchmarks mostly saturated.
  • WxBS instead uses manually labeled challenging pairs and epipolar error, but its 37-pair size limits model comparison.

3 Training the LoMa Descriptor and Matcher

LoMa trains a DeDoDe-based descriptor and LightGlue-based matcher with diverse multi-dataset supervision, scaled model capacity, and modern training objectives. Its pipeline uses detector supervision, attention-based feature refinement, and mutual dual-softmax matching.

  • Architecture: LoMa combines a DeDoDe-based descriptor with a LightGlue-based matcher for two-view local feature matching.The descriptor and matcher are trained as separate neural networks within the standard detection, description, and matching pipeline.
  • Supervision: Ground-truth correspondences come from known relative poses and depth maps in the training datasets.DaD supervises both the descriptor and matcher, with the descriptor trained before the matcher while using 2048 keypoints per image.
  • Learning Objective: The descriptor is trained with a dual-softmax loss that encourages ground-truth matches to be maxima across both image dimensions.The loss uses an inverse-temperature hyperparameter and can be viewed as an InfoNCE-style objective over ground-truth correspondences.
  • Matching: The matcher refines descriptors through self- and cross-attention, applying dual-softmax and matchability losses at every layer.Layer-wise supervision supports a performance–speed trade-off at inference time; matchability labels indicate whether a keypoint has a ground-truth correspondence.
  • Inference: At inference, mutual row-and-column maxima in the dual-softmax matrix define correspondences, with matches below µ = 0.1 discarded.The released matcher variants differ primarily in embedding dimensionality, ranging from 256 to 1024.
  • Training Scale: LoMa scales training to 17 3D datasets with diverse wide-baseline and optical-flow data, unlike prior methods typically trained on MegaDepth.The released variants use progressively larger embedding dimensions, while all share nine alternating self- and cross-attention blocks.

4 HardMatch

HardMatch is a manually annotated benchmark designed to expose weaknesses hidden by saturated matching evaluations. It contains 1000 diverse image pairs organized into nine challenging groups and includes a human annotation baseline.

  • Benchmark: HardMatch is an extremely challenging image-matching benchmark divided into nine groups.The benchmark is intended to provide meaningful signal for future research and will be released under a permissive license.
  • Data Collection: The dataset begins with diverse Wikimedia Commons images drawn from 100 categories under permissive licenses.Candidate pairs are selected from a large online corpus designed to provide broad category diversity.
  • Data Collection: Difficult pairs are filtered using RoMa v2 confidence, then manually inspected and classified as matchable or unmatchable.Pairs with maximum confidence between 0.3 and 0.9 are selected until 10 pairs per category are identified.
  • Annotation: 1000 image pairs receive manual keypoint annotations, with 8 to 28 correspondences per pair.The pairs span diverse geographic and temporal conditions and are grouped into qualitative challenge categories.
  • Validation: Eight independent annotators provide a human baseline by matching random keypoints and measuring pixel-error distributions.Each annotator verifies 20 pairs, and the resulting error curve is reported in the evaluation.
  • Evaluation: Evaluation estimates a Fundamental matrix from matcher correspondences and reports PCK under multiple pixel-error thresholds.Each pair contributes equally to PCK regardless of its number of annotated keypoints.
  • Challenge Diversity: HardMatch includes extreme appearance changes such as aerial-to-ground views, century-scale time gaps, sketches, night–day transitions, seasons, and viewpoints up to 180 degrees.These variations are intended to test matching beyond conventional benchmark conditions.

5 Experiments

LoMa is evaluated across extreme matching, relative pose estimation, visual localization, additional benchmarks, efficiency, ablations, and scaling. It achieves strong results across these settings, while increased data scale and model capacity consistently improve performance.

  • 5.1 Extreme Matching: LoMa-G achieves 73.4 mean accuracy on WxBS, barely surpassing RoMa at 72.6 and outperforming other sparse matchers.WxBS contains 37 hand-labeled pairs with extreme viewpoint, illumination, and modality changes.
  • 5.1 Extreme Matching: 54.3 mAA@10px is LoMa's best HardMatch result, with Doppelgängers, large viewpoint changes, aerial photographs, and star constellations particularly challenging.HardMatch performance is approximately 20 points below LoMa's performance on WxBS.
  • 5.2 Relative Pose Estimation: LoMa-L improves AUC@5° by 8.4 and 12.9 on MegaDepth-1500 and ScanNet-1500, respectively, compared with other sparse matchers.The comparison covers relative pose estimation against sparse, dense, and feed-forward reconstruction methods.
  • 5.3 Visual Localization: LoMa-G achieves approximately a 20-point precision increase on Map-free and more than a 20-point increase over the second-best matcher on InLoc's narrowest DUC2 threshold.Visual localization is evaluated with the HLoc pipeline on InLoc and with VCRE<90px on Map-free.
  • 5.4 Additional Evaluations: LoMa-G improves RUBIK AUC at 10° and 20° by approximately 24 points and sets a new state of the art on IMC 2022.The IMC 2022 submission also beats the competition winner and RoMa.
  • 5.5 Ablations, Throughput, and Scaling: LoMa-B matches LightGlue's runtime while producing significantly more accurate matches, with early stopping enabling an accuracy–speed trade-off.The comparison uses different stopping layers for the matcher on an A100.
  • 5.5 Ablations, Throughput, and Scaling: Scaling training data and model size significantly reduces validation loss and produces consistent performance gains as data is cumulatively added.Ablations also report gains from changing descriptors and retraining, expanding matcher and descriptor data, longer training, and increasing embedding dimension.

6 Limitations

Despite strong empirical performance, LoMa retains limitations involving descriptor scaling, difficult matching cases, benchmark construction, and robustness scope.

  • LoMa still struggles on HardMatch subgroups such as Doppelgängers and extreme viewpoint changes.
  • HardMatch relies on human-annotated keypoints, while its Fundamental-matrix evaluation requires static scenes and perspective cameras.
  • Although HardMatch is more diverse than previous benchmarks, it retains geographic and temporal biases.
  • LoMa does not address robustness to in-plane rotations, leaving augmentation or architectural design as future work.

7 Conclusion

The paper concludes that combining large-scale data with modern practices substantially improves local feature matching. It introduces HardMatch and LoMa, which achieve state-of-the-art results and surpass dense matchers on established benchmarks.

  • Large-scale data combined with modern practices yields substantial performance gains in local feature matching.
  • HardMatch is a challenging evaluation dataset containing 1000 hand-labeled image pairs.
  • LoMa achieves state-of-the-art performance on HardMatch, IMC 2022, and WxBS, surpassing even dense matchers.

Supplementary Material

The supplementary material reports detector, keypoint, benchmark, evaluation, scaling, and throughput analyses that further characterize LoMa’s performance and operating conditions.

  • Detector and keypoint studies: LoMa is evaluated with single-detector and multi-detector training, including an ensemble of DeDoDe v2, DISK, ALIKED, and DaD keypoints.
  • Detector and keypoint studies: Performance degrades significantly below 2048 keypoints, while increasing from 2048 to 4096 yields only a slight LoMa benefit.
  • Additional benchmarks: LoMa-B128 achieves the highest score on HPatches, while LoMa struggles with the large in-plane rotations in SatAst.
  • Additional benchmarks: Oxford Day-and-Night supplementary results report night-query localization percentages at (0.25m, 2°), (0.5m, 5°), and (1m, 10°).
  • Benchmark and scaling analyses: The supplementary figures report RUBIK cumulative success rates, WxBS accuracy across thresholds, and speed for different model sizes and stopping layers.
  • Scaling analyses: Descriptor validation loss saturates around 50K steps and then slowly increases, so descriptor training is limited to 50K steps versus 250K for the matcher.

A.9 Map-free Visual Relocalization Benchmark

The map-free relocalization supplementary material specifies evaluation protocols, reports additional baselines, and documents HardMatch’s distribution, difficult groups, and LoMa-G match visualizations.

  • Benchmark evaluation: The official benchmark reports LoMa-G at AUC 92.5 and precision 75.7 for the additional map-free baselines.
  • Evaluation setup: LoMa evaluations use 784 × 784 internal resolution, 4096 DaD keypoints, and nine matcher layers.
  • Evaluation setup: The AUC protocol measures relative-pose error using the maximum rotational or translational error, with translation represented by angular difference.
  • Evaluation setup: The Kaggle protocol uses 200K MAGSAC iterations and mAA over ten uniformly spaced rotational and translational threshold pairs.
  • Correspondence evaluation: HardMatch evaluation estimates a Fundamental matrix from matches and measures ground-truth correspondence consistency, with an alternative direct-keypoint protocol for cases where Fundamental matrices are poorly defined.
  • HardMatch analysis: HardMatch spans images from the early twentieth century to the present with a global footprint, but most images are from early twenty-first-century Europe.
  • HardMatch analysis: HardMatch includes difficult Doppelgänger groups where all matchers fail, alongside visualizations of LoMa-G inliers and outliers.

C.5 Results by Category and Group

The HardMatch evaluation is organized by Wikimedia Commons categories and reports performance for different groups. The appendix also examines how matches change across matcher stopping layers.

  • HardMatch is sourced from 100 Wikimedia Commons categories, with test categories listed in Fig. 14.
  • Performance is reported as mAA@10px for different HardMatch groupings in Table 14.
  • Qualitative match detections are examined at different stopping layers to track refinement through the matcher.

E Visualizing a Training Batch

The appendix visualizes a random batch of 32 image pairs to show training-data diversity and examines how matching changes through matcher layers. HardMatch performance is also plotted across categories from easy to hard.

  • A randomly sampled training batch contains 32 image pairs and is plotted to illustrate the diversity of the training data.
  • LoMa's PCK@10px is plotted across HardMatch categories ordered from easy to hard.
  • The descriptor initially fails on one image pair, but the pair gradually becomes matchable as features pass through matcher layers.
Loading 2604.04931v2…