PaperlayerView exact revision on arXiv

Source-linked AI summary

Comprehensive empirical evaluation of feature extractors in computer vision

Murat Isik

arXiv:2610.39551v1cs.CVcs.LG

TL;DR

Feature extractors must be compared across image conditions to understand their robustness and computational efficiency. This study evaluates traditional methods across transformed images, finding that the strongest choice varies by condition and pipeline stage. FAST with ORB and BF is fastest, while ORB and AKAZE lead in specified transformation conditions.

  • Problem

    The study addresses which traditional feature extractors best balance speed and robustness across image transformations.

    Source: DISCUSSION · p. 15

  • Method

    The study evaluates traditional feature extractors on transformed images, comparing matching and computational performance across more than two million comparisons.

    Source: METHODS · p. 11, DISCUSSION · pp. 15, 18, CONCLUSIONS · p. 23

  • Results

    FAST with ORB and BF is fastest for feature extraction and matching; ORB is favored for affine transformations, while AKAZE performs strongly under blurring, rotation, scaling, and perspective distortions.

    Source: DISCUSSION · p. 21, CONCLUSIONS · p. 23

  • Takeaways & Limitations

    The preferred extractor combination depends on the expected image transformation and whether processing speed is critical.

    Source: CONCLUSIONS · p. 23

    The dataset may not capture the diversity of real-world conditions, and runtime results from a single hardware configuration may not generalize to other platforms.

    Source: CONCLUSIONS · p. 24

Abstract

from arXiv · show

Feature detection and matching are fundamental components in computer vision, underpinning a broad spectrum of applications. This study offers a comprehensive evaluation of traditional feature detections and descriptors, analyzing methods such as Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Binary Robust Independent Elementary Features (BRIEF), Oriented FAST and Rotated BRIEF (ORB), Binary Robust Invariant Scalable Keypoints (BRISK), KAZE, Accelerated KAZE (AKAZE), Fast Retina Keypoint (FREAK), Dense and Accurate Invariant Scalable descriptor for Yale (DAISY), Features from Accelerated Segment Test (FAST), and STAR. Each feature extractor was assessed based on its architectural design and complexity, focusing on how these factors influence computational efficiency and robustness under various transformations. Utilizing the Image Matching Challenge Photo Tourism 2020 dataset, which includes over 1.5 million images, the study identifies the FAST algorithm as the most efficient detector when paired with the ORB descriptor and Brute-Force (BF) matcher, offering the fastest feature extraction and matching process. ORB is notably effective on affine transformed and brightened images, while AKAZE excels in conditions involving blurring, fisheye distortion, image rotation, and perspective distortions. Through more than 2 million comparisons, the study highlights the feature extractors that demonstrate superior resilience across various conditions, including rotation, scaling, blurring, brightening, affine transformations, perspective distortions, fisheye distortion, and salt-and-pepper noise

INTRODUCTION

Feature extraction identifies informative image structures and represents them numerically for computer vision tasks. This study evaluates established detectors and descriptors for speed, robustness, and sensitivity to image transformations.

Feature extraction

  • Features are informative image structures, including corners, blobs, edges, junctions, and lines, related to image texture and shape.Feature extraction transforms pixel values around such structures into numerical representations for machine-learning algorithms.Source: INTRODUCTION · pp. 1–2
  • Detection identifies distinctive keypoints, description encodes local image information around them, and matching compares descriptors across images.Matching can identify corresponding points, such as window corners, in images of the same building taken from different angles.Source: INTRODUCTION · p. 2

Study scope

  • The study examines widely adopted methods including SIFT, SURF, BRIEF, ORB, BRISK, KAZE, AKAZE, FREAK, DAISY, FAST, and STAR.These methods address different aspects of feature matching, including scale and rotation invariance, computational efficiency, and robustness under challenging conditions.Source: INTRODUCTION · p. 2

Contribution

  • Traditional feature extractors also remain components of deep-learning preprocessing and benchmarks for evaluating advanced models.The study presents its findings as a reference for research and applications requiring robust and efficient feature extraction.Source: INTRODUCTION · pp. 2–3

Study scope

  • The study systematically evaluates 10 traditional feature extractors across transformations to compare speed, robustness, and sensitivity.The selected conditions include rotation, affine and perspective transformations, scaling, brightening, and blurring.Source: INTRODUCTION · p. 2

RELATED WORKS

Prior comparisons report differing strengths for traditional feature extractors across image conditions and tasks. This study positions its contribution as a quantitative comparison of 10 methods using a consistent dataset.

Prior evaluations

  • Tareen & Saleem (2018) found SIFT, SURF, and BRISK better on scaled images, AKAZE more effective on rotated images, and ORB and BRISK better suited to affine images.Chien et al. (2016) reported SURF's highest accuracy, ORB's computational efficiency, and a balance of accuracy and efficiency for AKAZE.Source: RELATED WORKS · p. 3
  • Andersson & Reyna Marquez (2016) reported SIFT's superior performance across their tests, while AKAZE matched KAZE accuracy with improved computational efficiency.Their dataset contained over 170 test images subjected to scaling, rotation, and illumination adjustments.Source: RELATED WORKS · p. 3
  • A 14-detector comparison by Tareen & Raza (2023) used 10 image pairs altered with conditions including noise, blur, occlusions, and extreme affine transformations.The study reports varied results across the derived images.Source: RELATED WORKS · p. 3
  • Prior studies report KAZE leading tested face-identification scenarios and a SIFT–SURF–ORB hybrid yielding optimal results under tested image variations.Aydin (2022) compared KAZE, SURF, and ORB; Bansal, Kumar & Kumar (2021) tested rotation, scaling, shearing, intensity variation, and noise.Source: RELATED WORKS · p. 3
  • The article compares 10 fundamental feature extractors and descriptors on a consistent dataset, producing quantitative results and examining differences from analogous studies.The authors note that prior evaluations often use few images or report only a few methods.Source: RELATED WORKS · p. 4

Feature-extractor designs

  • The related-work section describes SIFT as detecting multiscale extrema and assigning keypoint orientations from local image gradients.SURF uses integral images and Hessian-based keypoint refinement, with Haar wavelet responses for descriptors.Source: RELATED WORKS · p. 4
  • DAISY uses gradient histograms and circular weighting, while FREAK uses retina-like sampling and binary intensity comparisons.FAST detects corners from contiguous pixel intensities around a center but lacks keypoint orientation; STAR approximates a circular mask with overlapping squares.Source: RELATED WORKS · pp. 5–6
  • ORB combines FAST keypoint detection, Harris corner response, orientation assignment, and rotated BRIEF binary descriptors.BRISK uses scale space, non-maximum suppression, and a sampling pattern; KAZE uses nonlinear diffusion and multiscale features.Source: RELATED WORKS · p. 5

METHODS

The study evaluates traditional feature extractors on a transformed subset of a large photo-tourism dataset, comparing their detection, description, and matching performance. Its experimental design spans ten image transformations and two matching strategies.

METHODS

  • From a corpus of over 1.5 million images, 4,282 were selected to generate 196,972 variations, producing a 201,254-image benchmark subset.The source dataset contains images from 10 distinct locales.Source: METHODS · p. 7
  • The evaluation measured keypoint counts, detection and description time, and matching performance for transformed images against their originals.The procedure was applied across the algorithms under study.Source: METHODS · pp. 7–8
  • Blur variants used Gaussian and averaging kernels, while brightening used gamma correction from 0.4 to 1.6 in 0.4 increments.The blur procedure produced eight variations per image; gamma correction produced four.Source: METHODS · p. 9
  • Affine images used 15–30° increments; rotations used eight 40° steps, while scaling produced six variations from 20% to 300%.Affine transformations generated four variations per image.Source: METHODS · pp. 8–9
  • The suite also applied barrel fisheye distortion, horizontal and vertical perspective changes, and salt-and-pepper noise to the images.Perspective changes ranged from 5° to 25° in 5° increments; salt-and-pepper density ranged from 0.1 to 0.26.Source: METHODS · pp. 9–10
  • Cross-checked BF and kNN (k=2; ratio threshold 0.75) were compared, using Hamming distance for binary and L2 for floating-point descriptors.The ratio test filters kNN matches using the distance ratio between the best and second-best matches.Source: METHODS · p. 11

RESULTS

The study benchmarks feature extractors on a 201,254-image transformed subset across ten conditions. Its results section presents matching-accuracy comparisons and summarizes missing matches, extraction times, and matching performance.

RESULTS

  • The benchmark uses 4,282 images and 196,972 variations to compare baseline feature extractors across ten transformations and their strengths and weaknesses.The final comparative dataset contains 201,254 images.Source: RESULTS · p. 11
  • Tables 3–5 report absent matches, detection and description timing, and matched-feature counts, matcher time, and successful-match percentages.Table 4 also summarizes timing variability and observed minima and maxima, excluding zero values.Source: RESULTS · p. 11, Table 3 · p. 12, Table 4 · p. 12
  • Figures 4 and 5 depict matching performance under blurring and fisheye distortion, respectively.They show how feature extractors preserve accurate matches under those conditions.Source: RESULTS · p. 12, Figure 4 · p. 14, Figure 5 · p. 15
  • Figure 3 plots matching accuracy by feature extractor under affine transformation, enabling comparison of robustness across the tested methods.The figure’s x-axis identifies the feature extractor and its y-axis gives matching accuracy percentage.Source: RESULTS · pp. 11–12, Figure 3 · p. 13
  • Figure 6 covers horizontal and vertical perspective, while Figure 7 covers brightening and salt-and-pepper noise.The graphs compare matching performance across the respective conditions.Source: RESULTS · pp. 13–14
  • Figures 8 and 9 compare feature-extractor matching performance under image rotation and scaling, respectively.The figures show matching accuracy across the tested degrees of each transformation.Source: RESULTS · p. 15

DISCUSSION

The study compares conventional feature extractors across image transformations and large-scale matching operations. Results show that the strongest extractor depends on the transformation, while FAST+ORB is the fastest feature-extraction combination.

DISCUSSION

  • Across more than 2 million comparisons, STAR+FREAK most often produced no matches, while SURF, SIFT, and KAZE matched in nearly all images.Each extractor was assessed in 201,254 comparison operations using 4,282 original and 196,972 transformed images.Source: DISCUSSION · pp. 18–19
  • FAST was the fastest detector and FAST+ORB the fastest feature-extraction combination; BF matching was shortest with ORB, while kNN matching was fastest with STAR+BRIEF.KAZE consumed the most time; kNN produced fewer matches than BF and took longer to match.Source: DISCUSSION · pp. 19–21
  • Beyond 20 degrees of affine transformation, ORB achieved the best matching results; AKAZE led under blurring, followed by KAZE and SURF.STAR+DAISY led up to 20 degrees of affine transformation, while STAR+FAST had the lowest matching results under blurring.Source: DISCUSSION · p. 21
  • AKAZE, KAZE, SURF, and SIFT achieved approximately similar top matching results under barrel fisheye distortion.STAR+FAST had the lowest matching success in this condition.Source: DISCUSSION · p. 21
  • For vertical and horizontal perspective distortions, STAR+DAISY and AKAZE were the most resilient overall, while ORB led at 25 degrees.The best-performing combinations varied across distortion directions and degrees.Source: DISCUSSION · pp. 21–22
  • ORB had the highest matching rate across brightness levels, while BRISK performed best against salt-and-pepper noise, followed by FAST+ORB.Matching success declined for all extractors as brightness increased; sufficiently intense noise could produce anomalously high matches between non-identical images.Source: DISCUSSION · p. 22
  • AKAZE achieved the highest matching rate for rotated images, followed by SIFT, with its strongest results between 160 and 200 degrees.STAR+DAISY, STAR+FAST, and STAR+BRIEF had the lowest, approximately equal matching values.Source: DISCUSSION · p. 22
  • AKAZE achieved the best results against scaling; its success rate showed no significant increase beyond 220% enlargement.Matching rates for all extractors declined and converged as images were reduced in size.Source: DISCUSSION · p. 22

CONCLUSIONS

Across more than two million comparisons, the study identifies feature extractors suited to different image conditions. Its conclusions recommend specific detector–descriptor choices for speed and perspective distortion, and identify algorithms for other transformations.

CONCLUSIONS

  • The study’s more than two million comparisons assess feature extractors across rotation, scaling, blurring, brightening, affine and perspective transformations, fisheye distortion, and salt-and-pepper noise.Source: CONCLUSIONS · p. 23
  • For real-time tasks, FAST detection with ORB description and BF matching is recommended for the fastest feature extraction and matching procedure.ORB is also recommended for affine-transformed images.Source: CONCLUSIONS · p. 23
  • AKAZE demonstrated superior robustness against blurring, rotation, and scaling, making it suitable when image quality may be compromised.Source: CONCLUSIONS · p. 23
  • For barrel fisheye distortion, SIFT, SURF, KAZE, or AKAZE are recommended, having demonstrated similar matching accuracy.Source: CONCLUSIONS · p. 23
  • For horizontal or vertical perspective distortions, AKAZE or STAR as detector with DAISY as descriptor is recommended.The study reports these combinations as optimal for the stated conditions.Source: CONCLUSIONS · p. 23
  • ORB is recommended for substantial brightness variation, while BRISK is advised for substantial salt-and-pepper noise.Source: CONCLUSIONS · p. 23
  • The authors suggest using the findings to guide feature-extractor selection for application requirements, including real-time or resource-constrained settings.Future work could combine traditional feature extractors with deep-learning techniques and expand the dataset to more diverse images and conditions.Source: CONCLUSIONS · p. 23
  • The dataset may not capture the diversity of real-world conditions, and computational-performance results come from a single hardware configuration.Dataset transformations and image properties may have influenced outcomes; results may differ across hardware platforms.Source: CONCLUSIONS · p. 24

ADDITIONAL INFORMATION AND DECLARATIONS

The authors report no funding and no competing interests. The reproduced dataset and IMC-PT 2020 dataset are identified as available sources.

ADDITIONAL INFORMATION AND DECLARATIONS

Loading 2610.39551v1…