PaperlayerView exact revision on arXiv

Source-linked AI summary

MODS: Fast and Robust Method for Two-View Matching

Dmytro Mishkin, Jiri Matas, Michal Perdoch

arXiv:1503.02619v2cs.CV

TL;DR

Wide-baseline matching must recover geometry across difficult viewpoint changes without incurring the high cost of exhaustive robust methods. MODS combines complementary detectors, progressive on-demand view synthesis, and improved correspondence selection; experiments report broader problem coverage with speed comparable to standard matchers on easy cases. A reported correspondence-selection modification increases correct matches by 5-20% at no additional computational cost.

  • Problem

    Wide-baseline matching requires reliable geometry recovery across difficult image pairs while balancing robustness against computational cost.

  • Method

    MODS combines multiple detectors, progressively applies more powerful detectors and synthesized views on demand, and improves tentative correspondence selection.

  • Results

    MODS solves matching problems beyond the state of the art while remaining comparable in speed to standard wide-baseline matchers on easy problems.

  • Takeaways & Limitations

    MODS adjusts complexity to the problem and performs well on difficult two-view cases involving modality, acquisition-time, and lighting differences.

  • Takeaways & Limitations

    Homography selection receives no special adjustment in MODS, so the reported performance is a lower bound.

Abstract

from arXiv · show

A novel algorithm for wide-baseline matching called MODS - Matching On Demand with view Synthesis - is presented. The MODS algorithm is experimentally shown to solve a broader range of wide-baseline problems than the state of the art while being nearly as fast as standard matchers on simple problems. The apparent robustness vs. speed trade-off is finessed by the use of progressively more time-consuming feature detectors and by on-demand generation of synthesized images that is performed until a reliable estimate of geometry is obtained. We introduce an improved method for tentative correspondence selection, applicable both with and without view synthesis. A modification of the standard first to second nearest distance rule increases the number of correct matches by 5-20% at no additional computational cost. Performance of the MODS algorithm is evaluated on several standard publicly available datasets, and on a new set of geometrically challenging wide baseline problems that is made public together with the ground truth. Experiments show that the MODS outperforms the state-of-the-art in robustness and speed. Moreover, MODS performs well on other classes of difficult two-view problems like matching of images from different modalities, with wide temporal baseline or with significant lighting changes.

1. Introduction

MODS combines multiple detectors, progressive on-demand view synthesis, and improved correspondence selection to address difficult wide-baseline matching while limiting computational cost. It is evaluated against prior methods on public datasets and newly collected extreme-view problems.

  • MODS targets viewing-angle differences beyond ASIFT without significantly increasing computational costs over standard matchers.
  • Combining complementary detectors increases average matching speed and solves image pairs that no single detector can solve.
  • Progressively stronger detectors, descriptors, and synthesized views are applied only until sufficient support for two-view geometry is obtained.
  • MODS generalizes the first-to-second nearest-distance ratio for tentative correspondence selection across SIFT, LIOP, and MROGH descriptors.
  • The algorithm was optimized and evaluated for speed and reliable geometry recovery on public datasets and a new Extreme View Dataset.

Algorithm 1 The standard two view matching scheme

The standard two-view matching scheme detects and describes local features independently, generates tentative correspondences, and geometrically verifies them while estimating a homography or fundamental matrix.

  • The pipeline takes two images as input and returns a fundamental or homography matrix together with corresponding features.
  • Local features are detected and described independently in each image.
  • Tentative correspondences are generated using the second-closest ratio before geometric verification.
  • RANSAC verifies tentative correspondences while estimating H or F.

2. Related work

Related work ranges from fast but less tolerant standard matchers to view-synthesis methods that extend the handled viewpoint range at substantially higher computational cost.

  • Standard local matching reported viewpoint differences up to 60° for planar objects and 30° for non-planar 3D objects.
  • Synthetic views were introduced to improve repeated keypoint detection under affine deformations and support learned correspondence labels.
  • ASIFT synthesizes views for both images, matches every synthesized-view pair, filters duplicate and one-to-many correspondences, and verifies geometry with ORSA.
  • ASIFT handled viewpoint differences up to 80° for planar objects, with computational costs ranging from tens of seconds to a few minutes.

Algorithm 2 ASIFT

ASIFT generates synthetic views for both images, extracts and matches features across synthesized-view pairs, filters duplicate correspondences, and performs geometric verification with ORSA.

  • ASIFT outputs corresponding points and a fundamental matrix from two input images.
  • It independently generates synthetic views and detects and describes local features for each input image.
  • Tentative correspondences are generated for every pair of synthesized views using the second-closest ratio.
  • Correspondences are merged, reprojected to the original images, and filtered for duplicate, one-to-many, and many-to-one matches.
  • ORSA geometrically verifies the tentative correspondences while estimating the fundamental matrix.
  • ASIFT reaches viewpoint differences up to 80° for planar objects but requires tens of seconds to a few minutes.

3. The MODS algorithm

MODS repeats increasingly capable two-view matching procedures until it obtains enough geometrically verified correspondences. Its detector and view-synthesis sequence is experimentally chosen to solve challenging problems while keeping speed comparable to standard single-detector matchers.

  • MODS repeats matching procedures with different detectors and synthesized views until the required number of geometrically verified correspondences is found.It starts with fast, limited-invariance detectors and progresses to more robust, computationally costly configurations.
  • The selected detector and view-synthesis sequence was experimentally searched to solve challenging development problems while maintaining speed comparable to standard single-detector matchers.The optimization targeted correct two-view geometry recovery and speed.

Algorithm 3 MODS

Algorithm 3 generates and aggregates features from synthesized views, forms tentative matches with FGINN, removes duplicates, and verifies geometry with robust estimation and local affine-frame consistency. Its view synthesis models local affine changes through sampled scale, tilt, and rotation transformations.

  • Algorithm inputs and loop: MODS takes two images, a minimum match count, and a maximum iteration count, then outputs a fundamental or homography matrix and corresponding points.
  • View synthesis and features: Each iteration synthesizes views, detects and describes features, backprojects them to the original image, and adds them to general feature lists.The synthesized views are generated from scale, tilt, rotation, detector, and descriptor settings.
  • View synthesis and features: MODS combines complementary detectors and descriptors, using ORB with BRIEF and MSER or Hessian-Affine with RootSIFT.Features detected on synthesized views retain their originating view and are mapped into the original-image coordinates.
  • Tentative correspondences: FGINN compares a descriptor with the closest geometrically inconsistent descriptor rather than the second closest descriptor, using a default center-distance threshold of n = 10 pixels.This addresses repeated detections of the same feature across synthesized views.
  • Geometric verification: After tentative matching, MODS filters duplicate matches, estimates homography or epipolar geometry with DEGENSAC, and applies a local affine-frame consistency check.The LAF-check tests closest and farthest ellipse points; insufficient surviving correspondences trigger further view synthesis.

4. Implementation and parameter setup

The implementation study evaluates detector, descriptor, view-synthesis, and matching-strategy choices on synthetic viewpoint changes. It uses Oxford building images warped across sampled tilts and compares configurations by solvability, matching time, and correspondence quality.

  • Synthetic evaluation: A synthetic problem is considered successfully solved when at least n ≥50 ground-truth correspondences are recovered.The threshold is intentionally high because synthetic warping underestimates effects such as scene nonplanarity and illumination changes.
  • Configuration selection: The selected configurations maximize solved tilt difference for each case while reducing matching time through sparse view synthesis.View synthesis improves all detectors, but its benefits are not uniform across configurations.
  • FGINN validation: FGINN slightly outperforms standard SNN for SIFT and SURF, with a substantially larger difference for LIOP and MROGH.The comparison uses recall-precision curves over 26 image pairs and varying ratio thresholds.

5. Experiments

Experiments evaluated MODS against ASIFT and detector configurations on public datasets, including extreme viewpoint problems. MODS combined progressive configurations, view synthesis, and FGINN to improve robustness while retaining speed.

  • Experimental setup: MODS, ASIFT, and single-detector configurations were tested on seven public datasets using ground-truth geometric verification.An image pair was considered solved when at least 10 output correspondences were correct.
  • Detector configurations: No single detector solved all 15 Extreme View Dataset pairs; Hessian-Affine, MSER, Harris-Affine, and DoG solved 13, 13, 12, and 13 pairs, respectively.Knowing the suitable detector and configuration for each image could solve all pairs.
  • MODS performance: MODS solved all Extreme View Dataset pairs faster than a suitable configuration selected separately for each image pair.The comparison used more time-consuming MODS configurations.
  • MODS performance: The proposed MODS configuration was fast on easy Oxford problems and had little overhead on the harder Extreme View Dataset.The medium configuration without the first sparse synthesis step supported the value of progressive view synthesis.
  • Comparison with ASIFT: ASIFT matched only 6 image pairs and was slower than the identical DoG configuration with the same tilt-rotation set.The paper attributes this to eliminating one-to-many correspondences, the standard second-closest ratio, and brute-force matching.
  • Runtime: Detection and description consumed 40% and 35% of total time, while matching consumed 10% and synthesis was nearly linear in synthesized-view area.Without fast SIFT computation, SIFT description exceeded 50% of runtime.

5.2. MODS testing on a non-planar dataset

On the non-planar Turntable dataset, evaluation used ground-truth epipolar geometry and a median symmetrical epipolar-error criterion. Structured scenes showed little benefit from additional synthesis, while Hessian-Affine remained strongest and MODS was similar but slower.

  • Dataset and protocol: The evaluation used 35 Turntable image sequences containing planar-surface and low-textured general 3D objects.Reference and tested views were matched across viewpoint differences formed from 5° steps.
  • Dataset and protocol: Matching accuracy was measured with symmetrical epipolar error computed from the output correspondences and estimated geometry.The evaluation used the fundamental matrix and its associated ground-truth geometry.
  • Dataset and protocol: An image pair was correctly matched when its median symmetrical epipolar error under the ground-truth fundamental matrix was ≤6 pixels.Figure 11 and Table 8 reported the percentage and number of correctly matched sequences by viewpoint difference.
  • Results: For structured scenes, easy, medium, and hard configurations differed little because self-occlusion, rather than distorted-region detection, limited matching.Additional synthesized views therefore added few correspondences.
  • Results: Hessian-Affine outperformed other detectors on structured scenes, while MODS showed similar performance but was slower than Hessian-Affine.The experiments confirmed that Hessian-Affine could be used alone in these scenes.

5.3. MODS testing on other datasets

MODS was evaluated on extreme zoom, ultra-wide-baseline, and non-geometric-change datasets using correspondence-based correctness criteria. It performed strongly across several difficult settings, but modality-specific image characteristics favored an area-based method on MMS.

  • Extreme Zoom Dataset: The Extreme Zoom Dataset contains six image sets with increasing zoom levels and compares MODS with ASIFT, DualBootstrap, MSER, ORB, and Hessian-Affine.A pair was solved when at least 10 output correspondences were correct.
  • Ultra-wide-baseline dataset: The ultra-wide-baseline evaluation used aerial building pairs with large viewpoint differences, repeated structures, and illumination changes.A pair counted as correct only when the facade plane had at least 75% correct inliers; ground or roof homographies were incorrect.
  • Evaluation caveat: Reported ultra-wide-baseline performance is a lower bound because MODS used no special homography-selection adjustment.This scope caveat applies to the reported results for that dataset.
  • Other datasets: MODS was also evaluated on GDB-ICP, SymBench, and MMS for modality, photometric, temporal, and infrared-visible changes.The evaluation compared MODS with ASIFT, DualBootstrap, and detector-only baselines without view synthesis.
  • Other datasets: MODS was the fastest method and matched the most pairs on GDB-ICP and SymBench without problem-specific features.The primary criterion was finding sufficiently correct geometric transformations in reasonable time.
  • Other datasets: MODS-IR solved 83 out of 100 MMS pairs after lowering detector thresholds and adding orientation-restricted SIFT.MMS images produced few features and had short geometric baselines, favoring DualBootstrap.

6. Conclusions

MODS broadens solvable wide-baseline matching while remaining fast on simple problems by combining view synthesis, progressively applied detectors, and improved correspondence selection. Its evaluation reports stronger robustness and speed than prior approaches, including difficult nonstandard two-view conditions.

  • MODS solves a broader range of wide-baseline problems than the state of the art while remaining fast on simple problems.Its complexity adjusts to the problem at hand rather than applying the most expensive processing uniformly.
  • Progressively more time-consuming detectors and on-demand synthesized images are applied until a reliable geometry estimate is obtained.This design addresses the apparent robustness-versus-speed trade-off.
  • View synthesis with Hessian-Affine or MSER outperforms ASIFT, while ORB achieves similar results to ASIFT in orders of magnitude shorter time.These findings support view synthesis with both affine-covariant and very fast detectors.
  • 5-20% more correct matches are obtained by modifying the first-to-second nearest distance rule at no additional computational cost.The improved tentative-correspondence selection works with and without view synthesis.
  • MODS outperforms prior approaches in robustness and speed and also performs well on multimodal, temporally separated, and lighting-changed image pairs.Evaluation covered standard datasets and a new set of geometrically challenging wide-baseline problems.
Loading 1503.02619v2…