PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy Preserving Face Recognition Utilizing Differential Privacy

M. A. P. Chamikara, P. Bertok, I. Khalil, D. Liu, S. Camtepe

arXiv:2005.10486v2cs.CRcs.DB

TL;DR

The paper addresses privacy risks from biometric processing on third-party servers, where facial data may be linked to sensitive information. It proposes PEEP, a local-differential-privacy protocol that perturbs facial data before recognition. PEEP reports an accuracy drop as low as 6 percent, from 85% to 79%, at ε = 8, while training and recognition use randomized data.

  • Problem

    Biometric facial data processed by third-party servers can expose identity-linked information and create privacy risks.

  • Method

    PEEP applies local differential privacy to perturb facial data and performs training and recognition using randomized data.

  • Results

    6 percent accuracy drop, from 85% to 79%, at ε = 8.

  • Takeaways & Limitations

    PEEP stores and processes randomized facial data, limiting the use of original images during model training and recognition.

  • Takeaways & Limitations

    Encryption-based privacy-preserving approaches can require trusted parties and have high computational complexity.

Abstract

from arXiv · show

Facial recognition technologies are implemented in many areas, including but not limited to, citizen surveillance, crime control, activity monitoring, and facial expression evaluation. However, processing biometric information is a resource-intensive task that often involves third-party servers, which can be accessed by adversaries with malicious intent. Biometric information delivered to untrusted third-party servers in an uncontrolled manner can be considered a significant privacy leak (i.e. uncontrolled information release) as biometrics can be correlated with sensitive data such as healthcare or financial records. In this paper, we propose a privacy-preserving technique for "controlled information release", where we disguise an original face image and prevent leakage of the biometric features while identifying a person. We introduce a new privacy-preserving face recognition protocol named PEEP (Privacy using EigEnface Perturbation) that utilizes local differential privacy. PEEP applies perturbation to Eigenfaces utilizing differential privacy and stores only the perturbed data in the third-party servers to run a standard Eigenface recognition algorithm. As a result, the trained model will not be vulnerable to privacy attacks such as membership inference and model memorization attacks. Our experiments show that PEEP exhibits a classification accuracy of around 70% - 90% under standard privacy settings.

1. Introduction

Face recognition creates privacy risks because biometric data can be linked to sensitive records and processed on accessible third-party servers. The paper proposes PEEP, which uses differential-privacy-based perturbation to support recognition while limiting access to original facial data.

  • Biometric facial images can be linked to sensitive health and financial records, creating significant privacy concerns.
  • Third-party servers are used for computationally intensive biometric processing but may be accessed by untrusted parties.
  • Encryption-based approaches can require trusted parties, incur high computational complexity, and expose original data to authorized parties.
  • Existing de-identification methods can introduce utility problems at large scale because of limitations in their privacy models.
  • The paper identifies five issues: preventing sensitive-data linkability, ensuring scalability, restricting biometric access, preventing cross-application linkability, and enabling revocation.
  • PEEP perturbs input and training data with local differential privacy, stores perturbed data, and allows privacy adjustment through the privacy budget.
  • At ε = 8, PEEP incurs as low as a 6 percent accuracy drop, from 85% to 79%.

2. Related Work

Prior privacy-preserving face-recognition methods use cryptography, perturbation, anonymization, or complementary camera techniques, but face-recognition privacy and scalability remain difficult challenges. Differential privacy offers a privacy model for limiting information leakage, while perturbation can reduce computational complexity at some utility cost.

  • Motivation: Biometric privacy is especially important because biometric data are generally non-revocable and can identify people within large populations.The related-work discussion frames privacy violation as a major threat in biometric data analysis.
  • Cryptographic approaches: Cryptographic face-recognition methods can require trusted parties or computationally expensive operations, limiting their suitability for large-scale scenarios.ZEYN requires O(log M) rounds and homomorphic computation; ANRA also has complexity problems in large-scale settings.
  • Semi-honest approaches: Semi-honest approaches such as cloud outsourcing, PE-MIU, and POR can retain privacy limitations because they introduce untrusted modules or lack a proper privacy definition.PE-MIU also faces scalability limits from exponential template comparisons, while POR inherits semi-honest-model issues.
  • Anonymization: k-anonymity-based de-identification can reduce accuracy and increase information leakage for high-dimensional data or large-scale surveillance.The cited discussion specifically identifies these concerns for k-same and related uses of k-anonymity.
  • Perturbation: Data perturbation generally has lower computational complexity than encryption, making it more efficient for large-scale data mining, but it may concede some privacy leakage.Perturbation preserves original data formats, which is identified as a possible source of leakage.
  • Differential privacy: Differential privacy was developed to provide stronger privacy guarantees than earlier models vulnerable to privacy attacks, using mechanisms such as Laplace and Gaussian noise.The passage identifies differential privacy as one of several privacy models and lists common mechanisms used to achieve it.

3. Foundations of Differential Privacy and Eigenface recognition

This section introduces differential privacy, local and global privacy models, sensitivity, Laplace noise, and eigenface-based dimensionality reduction. Eigenfaces represent images with principal components to focus recognition on essential features while improving speed and retaining high accuracy.

  • Differential privacy: Differential privacy bounds how much information an adversary can learn about an individual’s data presence in a database.The privacy budget ε indicates privacy loss, and higher ε corresponds to higher privacy loss.
  • Global vs. local differential privacy: Local differential privacy randomizes data before curator access, removing the need for a trusted curator but potentially producing noisier data.The text characterizes LDP as the untrusted-curator model and notes that noise is applied for individual-record privacy.
  • Sensitivity and Laplace mechanism: The Laplace mechanism adds noise with scale Δf/ε in the local setting, where Δf denotes function sensitivity.Sensitivity is the maximum influence one individual can have on a numeric query.
  • Eigenface recognition: Dimensionality reduction concentrates face recognition on essential image features while increasing algorithm speed and preserving high accuracy.The motivation includes avoiding slow training and testing caused by long one-dimensional vectors from high-resolution images.
  • Eigenface recognition: Eigenfaces use PCA and selected largest eigenvectors to project face images into reduced dimensions.The construction uses the covariance matrix and represents an m×n image in a k-dimensional eigenface using the largest k eigenvectors.

4. Our Approach: PEEP

PEEP is a privacy-preserving face-recognition approach that applies differential-privacy randomization to eigenfaces before data are stored or processed by untrusted servers. It randomizes training and testing inputs and trains recognition models solely on randomized data.

  • PEEP approach: PEEP applies differential privacy to eigenfaces, creating privacy-preserving versions of input face images.The approach assumes image-capture devices randomize faces before sending them to storage devices or servers.
  • PEEP approach: PEEP’s three primary steps are accepting original face images, generating eigenfaces, and adding Laplacian noise to randomize the images.The figure describes the same randomization flow over eigenfaces.
  • PEEP approach: PEEP trains the face-recognition model solely with randomized data, while the untrusted server stores only a privacy-preserving model.The supplied passages identify randomized data as the model’s training input and the privacy-preserving model as the server-held artifact.
  • Distributed eigenface generation: Distributed eigenface generation partitions input data and combines partition-level means and covariance matrices at a central computer.The central computer collects partition outputs to generate the global mean face and covariance matrix.
  • Distributed eigenface generation: The distributed procedure iteratively updates global covariance information across partitions and returns eigenvectors associated with the largest eigenvalues.It derives eigenvectors from the covariance representation and can run across distributed computing nodes within the local edge.

4.2. Generation of the principal components

PEEP normalizes input images, computes principal components from the covariance matrix, and selects a number of eigenvectors to form each eigenface. The choice of principal components trades feature representation against computational efficiency.

  • Image normalization: PEEP normalizes images to a predefined resolution, using 47 × 62 as the default resolution.Users can adjust the image width and height according to input sizes and edge-device computational power.
  • Principal-component generation: PEEP computes principal components from covariance-matrix eigenvectors and uses the largest nc eigenvectors to create an eigenface.The number of principal components, nc, is provided as an input.
  • Principal-component generation: Higher nc increases input-feature representation but lowers efficiency, so nc must balance recognition accuracy and computational efficiency.The text states that a suitable nc should provide high accuracy and high efficiency simultaneously.

4.3. Declaring the sensitivity before noise addition

PEEP defines sensitivity from differences between adjacent PCA-derived image vectors before adding Laplacian noise. Because normalized indices lie in [0,1], it selects sensitivity 1, giving noise scale 1/ε while noting that excessive noise can reduce utility.

  • Declaring the sensitivity before noise addition: PEEP reduces input images with PCA and treats the resulting PCA vectors as the objects to which noise is added.The vectors are formed from image inputs after dimensionality reduction.
  • Declaring the sensitivity before noise addition: Sensitivity is defined as the maximum difference between two adjacent PCA vectors, represented by Δf.The paper expresses this difference using flattened image vectors scaled within [0,1].
  • Declaring the sensitivity before noise addition: A Cartesian-distance alternative gives sensitivity as a maximum of √nc, where nc is the number of principal components.This alternative follows from considering Euclidean distance between normalized PCA vectors.
  • Declaring the sensitivity before noise addition: 1 is selected as the sensitivity because normalized PCA indices are bounded between 0 and 1.The paper states that sensitivity much greater than 1 would introduce substantial noise and drastically reduce utility under LDP.
  • Declaring the sensitivity before noise addition: The resulting Laplacian-noise scale is 1/ε, while further algebraic analysis is planned to improve the mechanism’s precision and flexibility.PEEP applies the noise to each PCA-vector index after defining the position and scale parameters.

4.4. Introducing Laplacian noise

PEEP creates private image representations by adding Laplacian noise to scaled PCA-vector indices. Users choose ε according to the desired privacy level, and the algorithm trains a classifier on the randomized vectors.

  • Introducing Laplacian noise: PEEP adds Laplacian noise to each index of the PCA vectors, using the index values as positions and 1/ε as the noise scale.The perturbed indices form private versions of the images.
  • Introducing Laplacian noise: The privacy parameter ε controls the trade-off: higher ε provides lower privacy, with 0 < ε ≤ 9 treated as an acceptable range.The implementation follows an upper limit of 9 for ε.
  • Introducing Laplacian noise: Algorithm 3 is presented as the procedure for conducting privacy-preserving face recognition with PEEP.The algorithm begins by determining image dimensions before generating the randomized training data.
  • Introducing Laplacian noise: The procedure generates flattened vectors, computes the first nc PCA components, and scales all vector indices between 0 and 1.These scaled vectors are the inputs to the perturbation step.
  • Introducing Laplacian noise: With sensitivity Δf = 1, PEEP perturbs the scaled indices, feeds the randomized vectors and targets to a classifier, and trains on the resulting data.The trained classifier is described as differentially private.

4.5. Algorithm for generating a differentially private face recognition model

The PEEP algorithm configures image resolution and the number of images per face, then extracts eigenfaces for classification. Its imthresh parameter directly influences accuracy by improving image representation between face classes.

  • Algorithm for generating a differentially private face recognition model: Algorithm 3 uses resolution parameters and imthresh to control input-image processing and the number of images considered per person.The parameters support the preparation of face classes for recognition.
  • Algorithm for generating a differentially private face recognition model: Each face represents a classification class, so the model requires a useful image representation to achieve good accuracy.The algorithm treats face recognition as an image-classification task.
  • Algorithm for generating a differentially private face recognition model: A higher imthresh value contributes to higher accuracy by providing better representation of images between face classes.The passage links this parameter to extracting eigenfaces that better represent the input images.
  • Algorithm for generating a differentially private face recognition model: The procedure includes a step ensuring that the selected number of PCA components does not exceed the permitted limit.This constraint appears as part of the algorithm’s component-selection logic.

4.6. Privacy preserving face recognition using PEEP

PEEP randomizes face inputs locally before training or testing and stores only perturbed representations on third-party servers. The paper argues that differential privacy and postprocessing invariance preserve privacy for the trained model, while empirical privacy evaluation is conducted separately.

  • Privacy preserving face recognition using PEEP: Each image undergoes PEEP randomization before training or testing, with Eigenface generation and randomization performed within local edge bounds.This design assumes input devices communicate with third-party servers only through PEEP.
  • Privacy preserving face recognition using PEEP: The face-recognition database stores only perturbed images, and the model is trained only on perturbed Eigenfaces.The paper identifies this storage and training arrangement as central to limiting exposure of biometric information.
  • Privacy preserving face recognition using PEEP: The paper states that untrusted server access will not permit loss of valuable biometric data to malicious third parties, while testing-data perturbation minimizes privacy leakage.The claim covers both stored training data and testing image inputs.
  • Privacy preserving face recognition using PEEP: Additional computations on an ε-DP outcome retain ε-DP under postprocessing invariance, which PEEP therefore inherits.This property is used to relate the privacy guarantee of perturbed data to the trained model.
  • Privacy preserving face recognition using PEEP: The paper concludes that the trained model satisfies the same privacy imposed by PEEP and that third-party-server privacy leakage is minimal.The authors indicate that empirical evidence is investigated in Section 5.

4.8. Datasets

The study evaluates PEEP using the lfw-funneled and CelebA face-image datasets, then examines eigenfaces before and after privacy perturbation. The experiments show that perturbation obscures visible biometric features, including under an extreme privacy-budget setting.

  • Datasets: The evaluation uses the lfw-funneled and CelebA face-image datasets, with 70% of the input data for training and 30% for testing.lfw-funneled contains gray images, whereas CelebA contains colored images.
  • Datasets: The lfw-funneled dataset has 13,233 gray images, restricted here to 1,140 images across five identity classes.The selected classes are Colin Powell, Donald Rumsfeld, George W Bush, Gerhard Schroeder, and Tony Blair.
  • Datasets: CelebA contains more than 200K celebrity images, with 10,177 identities, 202,599 face images, five landmark locations, and 40 binary attributes per image.
  • Eigenfaces and perturbation: Eigenfaces reduce dimensionality and retain essential image features, but they still expose important biometric features and can be reconstructed effectively.The paper therefore treats unperturbed eigenfaces as insufficient for privacy protection.
  • Eigenfaces and perturbation: At ε = 4, perturbed eigenfaces reveal no biometric features to the naked eye, while at ε = 100 they retain only mild visible facial features.The paper describes ε = 100 as outside its acceptable privacy-budget range of 0 < ε ≤9.

5. Results and Discussion

PEEP’s experiments evaluate recognition accuracy, parameter effects, reconstruction resistance, privacy properties, and efficiency. Recognition performance improves with greater privacy budgets, more images per class, and more PCA components, while reconstruction attacks do not recover identifying detail under tested settings.

  • Experimental setup: PEEP was evaluated with an MLPClassifier, alongside efficiency and privacy comparisons against the homomorphic-encryption methods ZEYN and ANRA.The experiments used scikit-learn’s MLPClassifier and compared PEEP with two cryptographic approaches.
  • Privacy-budget performance: Lower privacy budgets introduced heavier noise and made MLPClassifier training more difficult.The study intentionally avoided parameter tuning to isolate the effect of perturbation on model performance.
  • Privacy-budget performance: Increasing the privacy budget increased accuracy because higher budgets imposed less randomization on the eigenfaces, with reasonable accuracy for 4 < ε ≤ 8.The paper treats 0 < ε ≤ 9 as an acceptable privacy-budget range.
  • Effect of imthresh: Recognition performance improved with imthresh, the number of images per single face, because larger values provide more representation for each class.The experiment held ε at 8 and the number of PCA components at 128.
  • Effect of PCA components: Recognition performance increased sharply from 10 to 20 PCA components and then rose gradually, with smaller effects after 40 components.The paper attributes the early gains to the first 20–40 components representing significant input-image features.
  • Privacy and efficiency comparison: PEEP stores randomized data, avoids a trusted party, provides low linkability across randomized inputs, and was reported as more scalable and faster than ZEYN and ANRA.The reported runtime was around 0.006 seconds under the stated experimental settings.

6. Conclusions

The paper concludes that PEEP provides privacy-preserving face recognition through local differential-privacy perturbation, while retaining efficient recognition and configurable privacy. It also identifies future evaluation across other biometric algorithms and input domains.

  • PEEP is a novel privacy-preserving face-recognition mechanism based on data perturbation and differential privacy.The protocol is designed for privacy-preserving face recognition.
  • PEEP applies local randomization before facial images reach an untrusted server, requires no trusted party or secure channel, and forwards only randomized data.This design supports integration into resource-constrained devices.
  • Training and recognition use randomized facial data, and the paper reports no efficiency loss during face recognition.The conclusion attributes this outcome to performing both training and testing/recognition solely on randomized data.
  • Users can tune differential-privacy parameters according to domain requirements, making privacy configurable.The conclusion presents parameter adjustment as a property of the differentially private formulation.
  • Future work will examine PEEP with other biometric algorithms and domains, including fingerprint and iris recognition, with attention to effectiveness and sensitivity.
Loading 2005.10486v2…