Source-linked AI summary
Feature Level Fusion of Face and Fingerprint Biometrics
Ajita Rattani, Dakshina Ranjan Kisku, Manuele Bicego, Massimo Tistarelli
TL;DR
The paper addresses the difficulty of fusing face and fingerprint features despite their incompatible representations and the dimensionality problem caused by concatenation. It constructs compatible pointsets, reduces their dimensions, and matches fused data using point-pattern or Delaunay methods. Experiments on chimeric and real databases report higher recognition accuracy for feature-level fusion than for individual modalities and score-level fusion.
Problem
Face and fingerprint feature-level fusion is difficult because the modalities have radically different representations, incompatible feature sets, and concatenation can cause dimensionality problems.
Method
The approach extracts SIFT face points and fingerprint minutiae, makes their pointsets compatible, reduces features before or after concatenation, and matches them using point-pattern matching or Delaunay triangulation.
Results
Feature-level fusion outperformed score-level fusion by 2.64% in recognition accuracy in one experiment and by 0.67% on the authors’ real multimodal database.
Takeaways & Limitations
The experiments support fusing independent face and fingerprint information at the feature level to improve recognition performance relative to score-level fusion.
Abstract
from arXiv · showhide
The aim of this paper is to study the fusion at feature extraction level for face and fingerprint biometrics. The proposed approach is based on the fusion of the two traits by extracting independent feature pointsets from the two modalities, and making the two pointsets compatible for concatenation. Moreover, to handle the problem of curse of dimensionality, the feature pointsets are properly reduced in dimension. Different feature reduction techniques are implemented, prior and after the feature pointsets fusion, and the results are duly recorded. The fused feature pointset for the database and the query face and fingerprint images are matched using techniques based on either the point pattern matching, or the Delaunay triangulation. Comparative experiments are conducted on chimeric and real databases, to assess the actual advantage of the fusion performed at the feature extraction level, in comparison to the matching score level.
I. INTRODUCTION
Feature-level fusion of face and fingerprint biometrics addresses an understudied problem caused by incompatible feature spaces and high-dimensional concatenation. The paper proposes compatible point-based representations and reports improved performance over score-level fusion on real and chimeric databases.
- Feature-level fusion is motivated by the richer source information retained in feature sets than in matching scores or output decisions.
- Feature-level fusion is relatively understudied compared with matching-score, rank, and decision-level fusion.
- Incompatible feature sets and unknown correspondence between feature spaces make feature-level fusion difficult to implement.
- Concatenated feature sets can create curse-of-dimensionality problems through noisy or redundant data and more complex matching.
- Face and fingerprint are widely used biometric traits, but their feature-level fusion had not previously been proposed because their image representations differ radically.
- The proposed approach makes face and fingerprint feature pointsets compatible for concatenation and compares feature-level fusion with score-level fusion.
A. Face Recognition based on Scale Invariant Feature Transform Features (SIFT)
The face module represents images using SIFT feature points that combine spatial, orientation, and keydescriptor information.
- The face recognition system extracts SIFT features from query and database face images.
- Each SIFT point contains spatial location, local orientation, and a 1x128 keydescriptor.
- SIFT provides invariance to image scaling, translation, and rotation, with partial invariance to illumination changes and affine or 3D projections.
B. Fingerprint Verification based on Minutiae matching technique
The fingerprint module uses minutiae-based processing and extracts feature points defined by spatial location and local orientation after image preprocessing.
- Fingerprint recognition uses a minutiae-based technique after normalization, Gabor filtering, binarization, and thinning.
- Rotation invariance is supported by estimating the fingerprint foreground slope from fitted boundary lines.
- The fingerprint module outputs minutiae points, each represented by spatial location and local orientation.
III. FEATURE LEVEL FUSION SCHEME
The feature-level fusion scheme concatenates feature points from different information sources to form a fused pointset with greater discrimination power than individual feature vectors.
- Feature-level fusion is implemented by concatenating feature points obtained from different information sources.
- The concatenated feature pointset is intended to provide greater discrimination power than the individual feature vectors.
A. Feature set compatibility and normalization
The fingerprint minutiae pointset is made compatible with face SIFT features by adding local descriptors and handling geometric invariances. Fingerprint descriptors use Gabor-filter responses around minutiae.
- A. Feature set compatibility and normalization: Fingerprint minutiae are made compatible with face SIFT features by adding rotation and translation invariance plus local keypoint descriptors.The descriptor carries local information around each minutia.
- A. Feature set compatibility and normalization: Gabor filtering uses eight orientations, eight scales, and two phases to produce a 1x128 keypoint descriptor.Rotation is handled during preprocessing, translation through image registration, and scale through sensor dpi.
B. Feature Reduction and Concatenation
Feature-level fusion concatenates normalized face and fingerprint pointsets, with dimensionality reduction available either before or after concatenation.
- B. Feature Reduction and Concatenation: The fused pointset is formed by concatenating normalized face and fingerprint feature pointsets.The concatenation combines the normalized SIFT-derived and minutiae-derived point representations.
- B. Feature Reduction and Concatenation: Feature reduction can be applied before concatenation to individual pointsets or after concatenation to the fused pointset.This addresses irrelevant or excessive features in alternative stages of the fusion pipeline.
C. Feature Reduction techniques
Three reduction strategies remove redundancy or retain highly distinctive regions before matching. They operate either on the fused set or separately on face and fingerprint pointsets.
- C. Feature Reduction techniques: K-means clusters the concatenated normalized pointset and retains one centroid per cluster, with averaged descriptors and PBM-selected cluster counts.Clusters use spatial and orientation information, and Euclidean distance is used for classification.
- C. Feature Reduction techniques: Neighbourhood elimination independently removes face and fingerprint points within experimentally chosen radii of 20 and 15 pixels, respectively.The reduction uses spatial proximity to remove nearby points.
- C. Feature Reduction techniques: Specific-region selection retains face points near the eyes, nose, and lower mouth and fingerprint points near the central core region.Selection radii are 85 pixels for face and 120 pixels for fingerprint; the core region is intended to reduce distortion effects.
- C. Feature Reduction techniques: The reduction methods aim to retain optimal pointsets by removing redundant nearby points or keeping points from highly distinctive regions.K-means and neighbourhood elimination target redundancy, whereas region selection targets discriminative locations.
D. Matching techniques
The system matches fused pointsets either by pairing compatible individual points or by comparing Delaunay-triangulated point triplets. Scores are based on matched pairs or corresponding triangles.
- D. Matching techniques: Point pattern matching pairs points when spatial, directional, and keypoint-descriptor distances fall within fixed thresholds.The thresholds are 4 pixels, 3°, and 6 pixels for spatial, direction, and descriptor distances, respectively.
- D. Matching techniques: One-to-one point correspondence selects the candidate with minimum descriptor distance among points satisfying the spatial and directional thresholds.Rotation, scale, and translation invariance remove the need to calculate alignment transformation parameters.
- D. Matching techniques: Point-pattern matching scores use the ratio of matched pairs to the total number of feature points in database and query sets.The ratio is computed for individual traits and the fused pointset.
- D. Matching techniques: Delaunay triangulation groups points into triplets after constructing a Voronoi-based neighborhood structure.The triangulation connects centers of neighboring Voronoi regions.
- D. Matching techniques: Each Delaunay triangle is represented using angles, maximum side length, vertex orientations, and two side-length ratios.Corresponding triangles are matched when their attributes differ by no more than a fixed threshold.
IV. EXPERIMENTAL RESULTS
Experiments evaluate monomodal and multimodal systems using point-pattern matching, feature reduction, and Delaunay triangulation on chimeric and real databases. Feature-level fusion and selected matching and reduction strategies improve recognition accuracy over comparison systems.
- Experimental setup: The experiments evaluate mono-modal and multimodal algorithms on chimeric and real multimodal databases using multiple matching and feature-reduction strategies.The chimeric database uses 50 individuals, while the real database uses 100 individuals with face and fingerprint images.
- Feature-level fusion: 1.64% higher recognition accuracy followed the introduction of fingerprint keydescriptors, while feature-level fusion exceeded score-level fusion by 2.64%.Normalized compatible feature pointsets were concatenated and reduced using k-means before matching.
- Feature reduction: The specific-region reduction strategy increased recognition accuracy by 0.31% and reduced FRR to 0%, whereas neighborhood removal did not increase accuracy.These comparisons were reported for point-pattern matching after feature reduction before concatenation.
- Delaunay triangulation: Delaunay triangulation improved face and fingerprint recognition accuracy by 5.05% and 0.82%, respectively.The comparison included monomodal modalities and multimodal fusion at matching-score and feature-extraction levels.
- Delaunay triangulation: Multimodal feature-level fusion with Delaunay triangulation increased recognition accuracy by 0.35% over other feature-level fusion experiments.Restricting points to specific regions before triangulation produced a further 0.44% increase.
- Cross-database comparison: The best strategy outperformed score-level fusion by 0.67% on the real multimodal database.The best configuration combined specific-region point restriction with Delaunay triangulation and was evaluated on both chimeric and real multimodal data.
V. CONCLUSION
The paper presents face–fingerprint integration at the feature-extraction level using compatible feature representations and reports improved accuracy from feature-set fusion. It also identifies the need for further validation on standard multimodal databases.
- Conclusion: The paper presents a multimodal biometric system integrating face and fingerprint traits at the feature-extraction level.The two traits are described as widely accepted biometrics, with multimodal systems also associated with ease of use, noise robustness, and low-cost hardware.
- Conclusion: Compatible feature extraction algorithms transform face and fingerprint images into comparable features that can be fused.The conclusion describes this as a novel approach for processing the two image types.
- Conclusion: Feature-level fusion improved accuracy compared with score-level fusion for information from independent or uncorrelated face and fingerprint sources.The paper characterizes the reported results as preliminary and inspiring.
- Limitations: Further experiments on standard multimodal databases are needed to better validate overall system performance.This is the paper’s stated boundary for validating the approach beyond the reported experiments.