PaperlayerView exact revision on arXiv

Source-linked AI summary

HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning

Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar, Heiko Ludwig

arXiv:1912.05897v1cs.CRcs.LG

TL;DR

Federated learning reduces direct data sharing but exchanged updates and trained models can still leak private information, while privacy-preserving methods often impose substantial communication and training costs. HybridAlpha uses differential privacy with a functional-encryption-based SMC protocol designed for efficient, dynamic participation. On MNIST CNN training, it reduces training time by 68% and data transfer volume by 92% on average while retaining model performance and privacy guarantees.

  • Problem

    Federated-learning parameters and trained models can reveal private information, while existing privacy-preserving approaches impose high training and communication costs.

  • Method

    HybridAlpha combines differential privacy with an SMC protocol based on multi-input functional encryption and supports changing participant groups.

  • Results

    68% lower training time and 92% lower data transfer volume on average, with the same model performance and privacy guarantees as existing solutions.

  • Takeaways & Limitations

    HybridAlpha supports privacy-preserving federated learning in settings sensitive to training time and communication overhead.

  • Takeaways & Limitations

    The framework assumes a trusted third-party authority and an honest-but-curious aggregator within its threat model.

Abstract

from arXiv · show

Federated learning has emerged as a promising approach for collaborative and privacy-preserving learning. Participants in a federated learning process cooperatively train a model by exchanging model parameters instead of the actual training data, which they might want to keep private. However, parameter interaction and the resulting model still might disclose information about the training data used. To address these privacy concerns, several approaches have been proposed based on differential privacy and secure multiparty computation (SMC), among others. They often result in large communication overhead and slow training time. In this paper, we propose HybridAlpha, an approach for privacy-preserving federated learning employing an SMC protocol based on functional encryption. This protocol is simple, efficient and resilient to participants dropping out. We evaluate our approach regarding the training time and data volume exchanged using a federated learning process to train a CNN on the MNIST data set. Evaluation against existing crypto-based SMC solutions shows that HybridAlpha can reduce the training time by 68% and data transfer volume by 92% on average while providing the same model performance and privacy guarantees as the existing solutions.

1 INTRODUCTION

Federated learning keeps training data local but can still leak private information through exchanged parameters and trained models. HybridAlpha addresses these risks with an efficient, dropout-resilient SMC approach based on functional encryption.

  • Federated learning lets participants train locally and exchange model parameters rather than their training data.
  • Parameter exchanges and trained models can still expose private information through inference, model extraction, and model inversion attacks.
  • Existing hybrid privacy-preserving approaches combine differential privacy with SMC but incur long training times, high transmission costs, and limited dropout support.
  • Privacy-preserving approaches can require many communication rounds, substantially increasing federated-learning training time.
  • HybridAlpha uses differential privacy and a multi-input functional-encryption-based SMC protocol designed for efficiency and changing participant groups.
  • The paper evaluates HybridAlpha on a CNN trained with MNIST and reports improved training time and communication cost with comparable model performance and privacy guarantees.

2 BACKGROUND AND PRELIMINARIES

The paper reviews privacy-preserving federated learning, differential privacy, SMC, and functional encryption as foundations for HybridAlpha. Its design uses multi-input functional encryption to aggregate participant inputs while addressing dynamic participation and privacy-preserving computation.

  • 2.1 Privacy Preserving Federated Learning: Federated learning keeps participant data local, but model updates and outputs can still reveal private information.
  • 2.1 Privacy Preserving Federated Learning: Dynamic participation requires handling participants joining or leaving without redoing training rounds or fully re-keying existing approaches.
  • 2.1 Privacy Preserving Federated Learning: HybridAlpha limits communication to one message per round to reduce training time relative to privacy-preserving approaches with more exchanges.
  • 2.2 Differential Privacy and Multiparty Computation: Differential privacy limits the effect of including one training instance on an algorithm’s output and commonly adds sensitivity-scaled noise.
  • 2.2 Differential Privacy and Multiparty Computation: SMC computes functions over multiple parties’ inputs without revealing their individual inputs, using techniques including homomorphic and functional encryption.
  • 2.2 Differential Privacy and Multiparty Computation: Using SMC can preserve the same differential-privacy guarantee while distributing noise across participants and reducing total injected noise.
  • 2.3 Functional Encryption: Functional encryption lets an external entity compute a specified function over ciphertexts without learning additional information about the plaintext data.
  • 2.3 Functional Encryption: HybridAlpha adapts multi-input functional encryption for an n-ary inner-product function, with participants encrypting inputs and aggregators decrypting the function result.

3 HYBRID-ALPHA FRAMEWORK

HybridAlpha is a privacy-preserving federated-learning framework that combines functional encryption, differential privacy, and inference controls. It supports changing participation while protecting against the specified aggregator and colluding-participant threats.

  • HybridAlpha uses functional encryption to implement a simple, efficient secure multiparty computation protocol for federated learning.The framework is designed for privacy-preserving training and supports participant groups that change during learning.
  • Threat Model: Under the stated threat model, the framework limits aggregator knowledge to the differential-privacy mechanism’s expected output and protects honest participants’ parameters from colluding participants.The model assumes an honest-but-curious aggregator, potentially colluding participants, and a trusted TPA; denial-of-service and malicious-update attacks are out of scope.
  • Non-adversarial setting: HybridAlpha provisions additional keys so new participants can join after training begins without requiring changes for existing participants.Joining participants obtain an individual public key from the TPA before participating.
  • Non-adversarial setting: The aggregator updates the global model by decrypting the weighted inner product of collected ciphertexts, with weights adjusted to the number of received responses.For averaging, the weighted vector assigns 1/n to each of n received responses; dropouts change n accordingly.
  • Non-adversarial setting: Participants retain local datasets, train local models, add differential-privacy noise, encrypt the resulting parameters, and send ciphertexts to the aggregator.The aggregator collects participant responses and uses a functional-encryption key for the selected weighted inner product.
  • Inference Prevention Module: The Inference Prevention Module filters functional-encryption key requests to prevent inference-enabling inner products by curious aggregators.It is collocated with the TPA and is intended to limit risks from aggregators and colluding participants.

4 EVALUATION

HybridAlpha is evaluated against TP-SMC and P-SMC through theoretical crypto-operation analysis and experiments on encrypted federated learning. Across these evaluations, it reduces communication-related costs while preserving model-quality and privacy trade-offs described for the compared approaches.

  • Baselines and setup: HybridAlpha is compared with TP-SMC and P-SMC, which use threshold-based homomorphic encryption and an additive homomorphic SMC approach, respectively.The comparison uses a CNN trained on MNIST and a unified implementation platform for benchmarking.
  • Theoretical comparison: m(n −1) and m(2t −1) fewer crypto-related operations are required by HybridAlpha than by P-SMC and TP-SMC, respectively.The reduction follows from not sending encrypted aggregated model updates back to participants for decryption.
  • Cryptographic benchmarks: Precision settings have no significant impact on the operation time of the evaluated cryptosystems, while six decimal digits were used for subsequent experiments.The number of decimal points affects trained-model accuracy, and six digits provided good model and training-time performance.
  • Cryptographic benchmarks: HybridAlpha’s encryption takes around 4 seconds and decryption around 30 seconds for 10 participants and 118110 gradients, compared with 35-second encryption for both baselines.Under the same setting, P-SMC decryption takes 31 seconds and TP-SMC decryption takes 88 seconds.
  • Scaling with participants: As participant numbers increase, HybridAlpha’s decryption time grows approximately linearly, whereas TP-SMC’s grows exponentially.Participant-side encryption and decryption remain stable, while aggregator-side ciphertext multiplication increases almost linearly across approaches.
  • Model quality: Combining differential privacy with SMC achieves higher F1-scores while protecting input and output privacy, whereas local differential privacy yields the lowest F1-score.Approaches without output privacy obtain higher F1-scores, illustrating the model-quality cost associated with protecting against inference on the model.
  • Efficiency comparison: HybridAlpha has training time similar to FL-no-privacy and the lowest transmission rate because it performs one encrypted-data communication round without subsequent ciphertext transmission.TP-SMC is slowest because it requires more communication rounds per global step.

5 SECURITY AND PRIVACY ANALYSIS

HybridAlpha’s security analysis combines functional encryption, differential privacy, and protections against inference by curious aggregators and colluding participants. The framework preserves output-model and aggregation privacy while supporting threshold-based dropout tolerance.

  • HybridAlpha’s security analysis evaluates the MIFE scheme, framework privacy guarantees, and prevention of inference attacks.
  • MIFE’s security construction remains unchanged from the original scheme, so HybridAlpha retains the security proved for that construction.
  • The framework ensures privacy of both the output model and the aggregation computation.
  • HybridAlpha uses differential privacy to provide the same output-model privacy guarantee established by prior work.
  • MIFE lets the aggregator obtain the average weight while hiding participants’ original model data.
  • At least t honest participants using distinct public keys prevent colluding participants from inferring private information from aggregator outputs and local models.
  • The threat model excludes DDoS attacks that interrupt the network or replace valid updates.

6 RELATED WORK

Related work addresses federated-learning privacy through differential privacy, trusted aggregators, and secure multiparty computation. Existing approaches differ in threat-model assumptions, cryptographic mechanisms, and support for horizontally versus vertically partitioned data.

  • Federated learning avoids transmitting raw data but remains vulnerable to inference attacks using exchanged updates or the final model.
  • Directly applying differential privacy in federated learning can produce poor model performance, motivating approaches tailored to FL.
  • Some prior methods rely on trusted aggregators and honest participants, unlike HybridAlpha.
  • Existing SMC approaches use SPDZ, threshold-based partially additive homomorphic encryption, or secret sharing with authenticated encryption.
  • Most federated-learning frameworks target horizontally partitioned data, while vertical-partition methods focus on entity resolution and simple models such as logistic regression.
  • Vertically partitioned data cases are outside this paper’s scope and deferred to future work.

7 CONCLUSION

HybridAlpha targets the training-time and communication costs of privacy-preserving federated learning while retaining privacy and model performance. The reported evaluation finds substantial average reductions in both costs.

  • 68% average training-time reduction and 92% average data-transfer-volume reduction are reported without sacrificing privacy guarantees or model performance.
Loading 1912.05897v1…