Source-linked AI summary
SHREC 2011: robust feature detection and description benchmark
E. Boyer, A. M. Bronstein, M. M. Bronstein, B. Bustos, T. Darom, R. Horaud, I. Hotz, Y. Keller, J. Keustermans, A. Kovnatsky, R. Litman, J. Reininghaus, I. Sipiran, D. Smeets, P. Suetens, D. Vandermeulen, A. Zaharescu, V. Zobel
TL;DR
SHREC’11 evaluates feature detectors and descriptors for shape retrieval under diverse transformation classes and strengths. It standardizes comparisons on transformed TOSCA meshes using repeatability, normalized descriptor distance, and ROC analysis, while including scale-aware and other mesh-specific methods. The benchmark is intended to reveal how algorithms cope with different transformations, with correspondence assumptions and ignored unmatched features defining important scope boundaries.
Problem
Feature-based methods are increasingly used in shape retrieval, but their detectors and descriptors need evaluation under varied transformations.
Method
The benchmark compares point, region, and dense feature methods on transformed TOSCA meshes using repeatability, normalized L2 distance, and ROC measures.
Results
The report evaluates detector repeatability and descriptor performance across transformation classes and strengths using tables and figures for point and region methods.
Takeaways & Limitations
The benchmark supports analysis of which transformation classes and strengths shape feature algorithms can handle.
Takeaways & Limitations
Detection evaluation assumes groundtruth dense correspondence to the null shape, and features without such correspondence are ignored.
Abstract
from arXiv · showhide
Feature-based approaches have recently become very popular in computer vision and image analysis applications, and are becoming a promising direction in shape retrieval. SHREC'11 robust feature detection and description benchmark simulates the feature detection and description stages of feature-based shape retrieval algorithms. The benchmark tests the performance of shape feature detectors and descriptors under a wide variety of transformations. The benchmark allows evaluating how algorithms cope with certain classes of transformations and strength of the transformations that can be dealt with. The present paper is a report of the SHREC'11 robust feature detection and description benchmark results.
1. Introduction
The benchmark brings feature detection and description evaluation into shape analysis, testing detectors and descriptors across transformation classes and strengths relevant to shape retrieval.
- Feature-based shape analysis is gaining popularity, while descriptor choice strongly affects the resulting bag-of-features representation.
- SHREC’11 simulates feature detection and description stages in feature-based shape retrieval algorithms.
- The benchmark evaluates detector and descriptor performance under varied transformations, including which transformation classes and strengths algorithms can handle.
2. Data
The benchmark uses triangular TOSCA human meshes and applies 11 transformation classes, each generally represented at five versions with increasing strength.
- The dataset contains TOSCA shapes represented as triangular meshes with approximately 10,000–50,000 vertices.
- The dataset covers one human shape class with simulated transformations, adding more transformation classes and greater challenge than SHREC 2010.
- Transformations are organized into 11 classes, with five versions per class and generally increasing strength from version 1 to 5.
3. Evaluation methodology
The evaluation separates point and region detection from point, region, and dense description, comparing transformed shapes with corresponding null shapes using repeatability and descriptor-quality measures.
- The benchmark evaluates point and region detectors alongside point, region, and dense descriptors on transformed shapes versus corresponding null shapes.
- Detector quality is measured by repeatability: the percentage of detected features whose corresponding locations or regions satisfy the matching criterion.
- Point descriptors use the closest corresponding feature point, while descriptor quality is evaluated with normalized L2 distance.
- ROC evaluation labels descriptor pairs by a distance threshold and compares true-positive rate against false-positive rate, with higher TPR preferred at fixed FPR.
- Features lacking groundtruth correspondence, such as regions corresponding to holes, are excluded from descriptor evaluation.
4. Feature detection methods
The benchmark includes mesh-adapted detectors based on Harris responses, scale spaces, maximally stable components, and a scale-aware Difference-of-Gaussians method.
- 4. Feature detection methods: Harris 3D fits smoothed quadratic neighborhood patches and derives vertex responses from their derivatives.
- 4. Feature detection methods: Mesh-DoG constructs a scale space by repeatedly convolving a mesh-associated scalar function and detecting extrema across scales.
- 4. Feature detection methods: Mesh SIFT selects local feature locations from extrema in difference scale spaces built from mesh curvature quantities.
- 4.1. Point features: Mesh-Scale DoG detects local maxima across location and scale, assigning each feature a support proportional to the filter width for scale invariance.
- 4. Feature detection methods: Shape MSER finds maximally stable 3D components using vertex- or edge-wise weighting functions.
5. Feature description methods
The benchmark includes descriptors that encode local mesh geometry through gradient histograms, scale-normalized spin images, depth-map SIFT, and tensor invariants.
- Mesh-HoG: Mesh-HoG computes 3-D gradient histograms over a geodesic support region after selecting a local rotation-invariant coordinate system.Its histogram captures coarse spatial structure and finer orientation information.
- Scale Invariant Spin Image: Scale Invariant Spin Image computes the Spin Image descriptor over the local scale estimated at each interest point.The authors report improved feature-point matching, particularly for significant partial matching between meshes.
- Local Depth SIFT: Local Depth SIFT creates a scale- and rotation-invariant descriptor by applying SIFT to a canonically oriented depth map around each interest point.The depth map uses distances to the tangent plane, while PCA supplies the dominant local angle.
- Generalized HKS: Generalized HKS extends the heat kernel signature to 1-forms and uses scalar invariants of the resulting second-order tensor field.The construction retains a time parameter t for comparing the descriptors.
- Detector evaluation: The benchmark reports Mesh DoG mean and Gaussian detector repeatability at ρ = 5, with average detected-point counts of 392 and 391, respectively.The supplied tables identify the algorithms, evaluation radius, and average numbers of detected points but not their cell values.
6. Results
Results are organized around detector repeatability and descriptor similarity across transformation classes and strengths, with separate analyses for point, region, and dense features.
- Repeatability evaluation: Tables 1–9 break down point-descriptor repeatability by transformation class and strength, while Figures 2–3 vary geodesic distance from 0 to 5.Higher repeatability indicates better performance.
- Point feature detectors: Point-detector tables report repeatability at ρ = 5 for Mesh-Scale DoG, Mesh SIFT, and Harris3D variants, with algorithm-specific detected-point counts.The supplied table passages identify scope and sample counts but do not provide repeatability values or a cross-method winner.
- Region feature detectors: Region-detector tables report repeatability at overlap ≥0.7 for Shape MSER variants, with average detected-region counts of 12.36, 8.85, and 9.25.The variants are EW 1/HKS, EW 1/CT, and VW HKS, respectively.