Source-linked AI summary
Secure Face Matching Using Fully Homomorphic Encryption
Vishnu Naresh Boddeti
TL;DR
Face-template databases require protection because representations can leak private facial information, yet matching must remain useful. The paper uses fully homomorphic encryption for encrypted-domain matching and reduces its cost through batching and dimensionality reduction. Across benchmark datasets, the framework reports practical template size and matching time with minimal loss in matching performance.
Problem
Face-template databases can leak reconstructed facial images and soft attributes, creating a need to prevent information leakage while preserving matching performance.
Method
The paper secures face templates and probe features with fully homomorphic encryption and performs matching directly on encrypted data using batching and dimensionality reduction.
Results
Across benchmark datasets, encrypted matching can achieve 16 KB per template and 0.01 seconds per pair while exhibiting minimal loss in matching performance.
Takeaways & Limitations
Fully homomorphic encryption could be a viable practical solution for accurate face matching in the encrypted domain while helping prevent information leakage and preserve user privacy.
Takeaways & Limitations
Homomorphic multiplication remains computationally expensive, especially for high-dimensional face templates.
Abstract
from arXiv · showhide
Face recognition technology has demonstrated tremendous progress over the past few years, primarily due to advances in representation learning. As we witness the widespread adoption of these systems, it is imperative to consider the security of face representations. In this paper, we explore the practicality of using a fully homomorphic encryption based framework to secure a database of face templates. This framework is designed to preserve the privacy of users and prevent information leakage from the templates, while maintaining their utility through template matching directly in the encrypted domain. Additionally, we also explore a batching and dimensionality reduction scheme to trade-off face matching accuracy and computational complexity. Experiments on benchmark face datasets (LFW, IJB-A, IJB-B, CASIA) indicate that secure face matching can be practically feasible (16 KB template size and 0.01 sec per match pair for 512-dimensional features from SphereFace) while exhibiting minimal loss in matching performance.
1. Introduction
Face recognition’s wider deployment increases the security risks of storing face representations, motivating encrypted-domain matching that protects templates without sacrificing utility. The paper applies fully homomorphic encryption and introduces batching and dimensionality reduction to reduce its computational burden.
- Motivation: Face-template databases can expose users’ facial images and attributes, making information-leakage prevention while preserving matching performance the paper’s central aim.Access to representations has been shown to enable face reconstruction and prediction of age, gender, and ethnicity.
- Approach: Fully homomorphic encryption secures templates and probe features while allowing matching directly on encrypted data without decryption.The approach exploits the decomposition of Euclidean distance or cosine similarity into additions and multiplications supported by FHE.
- Challenge: 48.7 MB per 512-dimensional template and 12.8 seconds per match pair illustrate the computational barrier of straightforward FHE face matching.Homomorphic multiplication is especially costly for the high-dimensional vectors used in face recognition.
- Efficiency improvements: Batching performs multiple homomorphic multiplications for the cost of one, while dimensionality reduction trades matching accuracy against computational efficiency.These techniques target the computational requirements that otherwise make encrypted matching difficult to deploy.
- Reported efficiency: The proposed scheme reports 16 KB per encrypted template and 0.01 seconds per match pair, substantially reducing the stated FHE matching costs.The reported figures are for the paper’s efficient encrypted face-matching configuration.
2. Related Work
Prior biometric template-protection methods use cryptographic or pattern-recognition techniques, while face-matching systems have also explored partially and fully homomorphic encryption. The paper situates its approach among these methods and targets secure matching for learned face representations.
- Prior protection methods: Cryptographic biometric methods include biometric encryption, visual cryptography, fuzzy vaults, non-invertible transforms, cancelable biometrics, and key-binding systems.These approaches have been applied across fingerprint, iris, and face biometrics.
- Biometric template protection: Pattern-recognition-based protections can trade matching accuracy for security, whereas the paper presents FHE as avoiding that trade-off.The related-work passage characterizes earlier approaches as either losing accuracy or providing inadequate template protection.
- Prior FHE face matching: Earlier FHE face verification used Gabor features but incurred much higher costs, including 380 MB templates and 100 seconds per match pair.The cited system was based on an early computationally expensive FHE scheme.
- Face representations: The paper evaluates secure matching for learned representations from FaceNet and SphereFace, using 128- and 512-dimensional features.These models represent the deep neural network-based feature extraction setting targeted by the framework.
3. Approach
The approach secures face templates and probe representations with fully homomorphic encryption, enabling matching directly on encrypted data. It reduces the cost of encrypted matching through batching, dimensionality reduction, and an enrollment-authentication protocol.
- Problem setup: Face dissimilarity is computed from normalized representations using d scalar multiplications and d scalar additions.The approach uses the normalized inner product underlying cosine dissimilarity.
- Secure face matching: Fully homomorphic encryption secures stored templates and probe features while supporting face matching directly in the encrypted domain.The approach decomposes Euclidean distance or cosine similarity into additions and multiplications supported by FHE.
- Enrollment and authentication: The enrollment protocol encrypts each user’s extracted templates and stores them with the identity label and public key in the database.Users generate keys, capture enrollment images, extract representations, and transmit encrypted templates to the database.
- Enrollment and authentication: During authentication, the device encrypts the probe, the database performs any needed key switching and encrypted comparisons, and the device decrypts the returned scores.The protocol supports different keys for the probe and enrolled user templates without decrypting the probe at the database.
- Efficient homomorphic face matching: 16.5 MB per template and 0.6 seconds per match for a 512-dimensional template motivate batching and dimensionality reduction to reduce encrypted matching costs.Straightforward encrypted matching also requires d encrypted multiplications and d−1 encrypted additions, making high-dimensional representations expensive.
- Efficient homomorphic face matching: Batching evaluates multiple homomorphic multiplications within one block-level multiplication, while cyclic rotations and additions recover the packed sum needed for the inner product.Packing improves efficiency but prevents direct access to individual encrypted elements, requiring rotations and additions.
4. Experiments
The experiments evaluate encrypted face matching across benchmark datasets and representations, measuring accuracy, efficiency, and the effects of batching and dimensionality reduction. The results indicate substantial efficiency gains while preserving competitive matching performance.
- Efficiency: 0.59ms versus 38.64ms and 4KB versus 1MB show batching substantially reduces matching time and template memory for n=256, d=128.For n=1024, d=512, batching reduces time from 624ms to 11.42ms and memory from 16.5MB to 16KB.
- Efficiency: 16KB per template and 0.01 seconds per match pair demonstrate practical encrypted matching performance across benchmark experiments.The conclusion reports these requirements for the proposed framework, while the experiments compare its efficiency with prior encrypted matching.
- Trade-offs: The proposed scheme is over 1000x slower than real-domain matching but provides 128 to 192 bits of security for encrypted matching.The comparison is made against real-domain matching while the security range is reported for the FHE framework.
- Experimental setup: The experiments use FaceNet and SphereFace features of 128 and 512 dimensions, respectively, and evaluate TAR at FARs of 0.01%, 0.1%, and 1.0%.The study considers benchmark datasets including LFW, IJB-A, IJB-B, and CASIA.
- Accuracy and reduction: The highest-precision quantization scheme performs competitively with unencrypted features, while PCA-based reduction can remain comparable to original high-dimensional features.These choices trade representation precision or dimensionality against homomorphic matching efficiency.
5. Conclusion
The paper concludes that fully homomorphic encryption, combined with batching and dimensionality reduction, can support accurate face matching while protecting encrypted templates. Its reported efficiency reaches 16KB per template and 0.01 seconds per match pair.
- Conclusion: Fully homomorphic encryption can provide a viable practical solution for accurate face matching in the encrypted domain.The conclusion bases this assessment on experiments over multiple benchmark datasets.
- Conclusion: Batching performs multiple homomorphic multiplications in a single operation, while dimensionality reduction improves computational efficiency.Both mechanisms are presented as part of the framework for securing and matching face templates.
- Conclusion: 16KB of memory per encrypted template and 0.01 seconds per matching pair are the reported practical requirements.These figures are reported for the proposed framework.