PaperlayerView exact revision on arXiv

Source-linked AI summary

FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System

Weizhao Jin, Yuhang Yao, Shanshan Han, Jiajun Gu, Carlee Joe-Wong, Srivatsan Ravi, Salman Avestimehr, Chaoyang He

arXiv:2303.10837v3cs.LGcs.CR

TL;DR

Federated learning keeps raw data on clients, yet plaintext updates can expose training data, while HE-based aggregation carries substantial overhead. FedML-HE selectively encrypts sensitive parameters to reduce that overhead; the optimized system reports reductions for ResNet-50 and BERT, supporting its potential for real-world deployment.

  • Problem

    Plaintext model updates in federated learning can expose sensitive training data, while HE-based federated learning faces substantial computation and communication overhead.

    Source: §1 · pp. 1–2

  • Method

    FedML-HE selectively encrypts the most privacy-sensitive parameters to reduce encrypted update size while providing customizable privacy preservation.

    Source: §1 · pp. 2–3, §2.4 · p. 5

  • Results

    The optimized system reports ~10x reduction for HE-federated training ResNet-50 and up to ~40x reduction for BERT.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The reported overhead reductions demonstrate the potential for real-world HE-based federated-learning deployments.

    Source: §1 · p. 3

    The security definition assumes a semi-honest adversary that follows the protocol while trying to learn information.

    Source: §2.2 · p. 3

Abstract

from arXiv · show

Federated Learning trains machine learning models on distributed devices by aggregating local model updates instead of local data. However, privacy concerns arise as the aggregated local models on the server may reveal sensitive personal information by inversion attacks. Privacy-preserving methods, such as homomorphic encryption (HE), then become necessary for FL training. Despite HE's privacy advantages, its applications suffer from impractical overheads, especially for foundation models. In this paper, we present FedML-HE, the first practical federated learning system with efficient HE-based secure model aggregation. FedML-HE proposes to selectively encrypt sensitive parameters, significantly reducing both computation and communication overheads during training while providing customizable privacy preservation. Our optimized system demonstrates considerable overhead reduction, particularly for large foundation models (e.g., ~10x reduction for ResNet-50, and up to ~40x reduction for BERT), demonstrating the potential for scalable HE-based FL deployment.

1 INTRODUCTION

Federated learning keeps raw data on clients, but plaintext model updates can still expose sensitive training data to reconstruction attacks. FedML-HE addresses the resulting privacy and overhead challenges with selective parameter encryption and reports substantial reductions in overhead for large models.

1 INTRODUCTION

  • Plaintext local models or updates can enable reconstruction attacks, especially when clients train on small local datasets.Federated learning keeps raw data on clients, but does not by itself prevent an adversarial server from exploiting shared updates.Source: §1 · p. 1
  • Differential privacy can degrade model performance through added noise, while secure aggregation adds synchronization steps and is sensitive to client dropout.HE preserves exact gradients but introduces substantial computation and communication overhead, particularly for large models.Source: §1 · pp. 1–2
  • O(n) computation and package-size overheads grow with model size, leaving unoptimized HE impractical for large models despite being faster than Nvidia FLARE.Figure 2 compares computation and communication overhead for naive FedML-HE, Nvidia FLARE, and plaintext aggregation; FLARE did not finish the BERT run on the 32GB machine.Source: §1 · p. 2, Figure 2 · p. 2
  • ~10x reduction for HE-federated training ResNet-50 and up to ~40x reduction for BERT are reported for the optimized system.The paper presents these results as evidence of potential for real-world HE-based federated-learning deployments.Source: §1 · p. 3
  • FedML-HE selectively encrypts the most privacy-sensitive parameters to reduce encrypted update size while allowing customizable privacy preservation.The proposed system also supports encryption key management and encrypted federated-learning deployment.Source: §1 · pp. 2–3

2 FEDML-HE SYSTEM DESIGN

FedML-HE combines encrypted federated aggregation with selective encryption guided by model sensitivity, and organizes the system across cryptographic, ML-bridging, and FL-orchestration layers.

2.1 System Overview

  • The workflow has three stages: key agreement, encryption-mask calculation, and encrypted federated learning with selectively encrypted local updates.Clients may use threshold key agreement or a trusted key authority.Source: §2.1 · p. 3

2.2 Threat Model

  • The threat model allows a semi-honest server or clients; collusion between the server and corrupted clients can expose benign clients’ models under shared-key setup.The paper defaults to single-key HE, while threshold or multi-key variants mitigate this collusion issue and are detailed in the appendix.Source: §2.2 · pp. 3–4

2.3 Algorithm for HE-Based Federated Aggregation

  • The aggregation algorithm uses weighted local models to produce a partially encrypted global model, with plaintext aggregation weights by default.The system uses one multiplicative depth of HE multiplication for weighting; optional local differential-privacy noise can be added after local training.Source: §2.3 · p. 4

2.4 Efficient Optimization by Selective Parameter Encryption

  • FedML-HE estimates parameter sensitivity, aggregates local sensitivity maps into a global privacy map, then uses selection ratio p to choose which parameters to encrypt.The selected mask controls which parameters are aggregated in encrypted form, allowing overhead and privacy guarantees to be balanced.Source: §2.4 · p. 5, Figure 4 · p. 5

2.5 Software Framework: Homomorphic Encryption In Federated Learning

  • The software framework separates HE functions, ML-to-crypto integration and optimization, and participant coordination into three layers.Its layered design allows HE libraries to be switched and further FL optimization techniques to be added.Source: Figure 6 · p. 6, §2.5 · p. 6

3 PRIVACY BY SELECTIVE PARAMETER ENCRYPTION

The section analyzes privacy for fully encrypted federated learning and then for selective parameter encryption. It uses simulation security and differential-privacy bounds to state the guarantees of these designs.

3.1 Proof of Base Protocol

  • The base-protocol proof argues that an ideal-world simulator's output is computationally indistinguishable from an adversary's view in the real execution.The stated privacy definition concerns a semi-honest adversary and simulation security.Source: §3.1 · pp. 6–7

3.2 Proof of Encrypted Learning by DP Theory

  • The Laplace mechanism uses sensitivity-calibrated noise, while the section claims homomorphic encryption gives encrypted parameters a 0-differential-privacy guarantee.The text says an adversary cannot retrieve sensitive information from encrypted parameters.Source: §3.2 · p. 7

3.3 Proof of Selective Parameter Selection

  • With parameter-selection ratio p, Laplace noise on all parameters satisfies J-DP, random encryption satisfies (1−p)J-DP, and sensitive-parameter selection satisfies (1−p)^2J-DP.The paper states that selective parameter encryption requires (1−p)^2 times less privacy budget than random selection.Source: §3.3 · p. 8

4 EVALUATION

The evaluation measures overhead reduction and attack defense from selective parameter encryption. Sensitivity-based selection is reported to defend against attacks with smaller encrypted parameter shares than random selection.

4.1 Experiment Setup

  • The evaluation covers models from different ML domains and sizes, uses PALISADE by default, and tests selection defense with gradient inversion on image and language tasks.The setup also implements the HE core with TenSEAL and PALISADE.Source: §4.1 · p. 8, §4.2.2 · p. 8

4.2.1 Optimized Overheads

  • At 10% encryption, computation and communication overheads are close to plaintext aggregation and nearly proportional to encrypted-parameter size.The figure compares overheads across model sizes and distinguishes selection-based 10% encryption from random 50% encryption.Source: §4.2.1 · p. 8, Figure 7 · p. 9
  • For a medium-sized model, HE shifts some training-cycle time toward aggregation-related steps; those overheads drop proportionally for smaller models.The comparison uses a single AWS region bandwidth and includes plaintext FL, HE without optimization, and optimized HE.Source: §4.2.1 · p. 8

4.2.2 Effectiveness of Selection Defense

  • According to the model privacy map, top-10% encryption selection alone defended against attacks, whereas random selection required encrypting 42.5% of parameters to begin protecting against attacks, with lower overall overhead for the same privacy protection.Source: §4.2.2 · p. 9
  • With a top-30% privacy-sensitive mask, the Bert experiment reports better defense against inversion attacks than randomly encrypting 75% of model parameters.The experiment used language samples from WikiText.Source: §4.2.2 · p. 9

5 RELATED WORK

The related work contrasts differential privacy, secure aggregation, and HE-based FL approaches. It presents FedML-HE's selective encryption as a response to overhead and deployment gaps in prior HE-FL work.

5 RELATED WORK

  • Privacy attacks include inference and data reconstruction, and plaintext access to local models is identified as a common source of exposure.The cited discussion notes that access to more fine-grained local models can increase attack success.Source: §5 · p. 9
  • Local differential privacy adds noise that can degrade model performance, while secure aggregation uses masks but faces synchronization and learner-dropout challenges.The paper contrasts these approaches with HE, which it describes as non-interactive, dropout-resilient, and causing negligible model-performance degradation.Source: §5 · p. 9
  • Prior HE-FL work either uses restricted schemes such as Paillier or generic aggregation implementations, while the HE-overhead increase remains unresolved.The passage associates restricted schemes with limits in extensibility, performance, or security guarantees.Source: §5 · p. 9
  • FedML-HE proposes selective encryption of the most privacy-sensitive parameters to reduce encrypted updates and overhead while retaining customizable privacy protection.The paper frames its optimization as making HE-based FL viable for practical deployments.Source: §5 · p. 10

6 CONCLUSION

The conclusion introduces FedML-HE as a practical HE-based FL system, alongside background on federated learning and homomorphic encryption, threshold key management, framework APIs, and deployment across cloud and edge devices.

6 CONCLUSION

  • FedML-HE is presented as a practical homomorphic-encryption-based federated-learning system with encryption key management and encrypted FL.Source: §6 · p. 10

A.1 Federated Learning

  • Federated learning keeps personal data on clients while the server aggregates their local model updates into a global model.In the typical setup, the server can see each client's model in plaintext.Source: §A.1 · p. 14

A.2 Homomorphic Encryption

  • Homomorphic encryption allows computation on encrypted data without revealing the underlying plaintext; FedML-HE uses CKKS for model parameters that are usually non-integers.CKKS supports approximate numbers.Source: §A.2 · pp. 14–15

B KEY MANAGEMENT AND THRESHOLD HE

  • The system assumes a trusted key-authority server and a potentially compromised aggregation server, with no collusion between them.A decentralized version is planned; threshold-HE FedAvg microbenchmarks use a two-party setup and estimated precision 36.Source: B KEY MANAGEMENT AND THRESHOLD HE · p. 15, Figure 12 · p. 15

C.1 Framework APIs

  • The paper points to a table of framework APIs related to homomorphic encryption.Source: §C.1 · p. 15

C.2 Deploy Anywhere: A Deployment Platform MLOps For Edges/Cloud

  • The deployment platform accepts Python server and client packages, builds a Docker image for the server, and lets users monitor training and system overheads through a web interface.Monitoring includes device status, progress, training time, communication time, CPU/GPU utilization, and memory utilization.Source: §C.2 · p. 15

D ADDITIONAL EXPERIMENTS

The additional experiments examine HE overheads across training scenarios and compare the framework with other open-source HE solutions. They also report gains from parameter-efficiency techniques in HE-based federated learning.

D ADDITIONAL EXPERIMENTS

D.1 Parameter Efficiency Techniques in HE-Based FL

  • Table 5 reports optimization gains from applying model parameter-efficiency solutions in HE-based federated learning.Source: §D.1 · p. 16

D.2 Results on Different Scales of Models

Across models from linear models to foundation models, HE computation and communication overheads generally grow with model size. The reported computation fold ratio is generally 5x–20x, while communication overhead can reach a common 15x.

D.2 Results on Different Scales of Models

  • HE computation and communication overheads generally increase with model size, across evaluations from linear models to foundation models such as ViT and BERT.Source: §D.2 · p. 16
  • Table 4 reports computation fold ratios generally 5x ∼20x and communication overhead commonly 15x relative to plaintext federated aggregation; small models tend to show a higher computational overhead ratio increase.Source: §D.2 · p. 16
  • For models smaller than 4096 numbers, communication overhead is significant because packing still requires a full ciphertext below the packing-batch size.Source: §D.2 · p. 17

D.3 Results on Different Cryptographic Parameters

The experiments find that larger packing batches generally reduce computation time and ciphertext size, while scaling bits have almost negligible overhead impact. They also examine accuracy changes and the effect of client counts on system costs.

D.3 Results on Different Cryptographic Parameters

  • Larger packing batch sizes generally yield faster computation and smaller ciphertext files, whereas the scaling factor has almost negligible overhead impact.Source: §D.3 · p. 17, Table 6 · p. 17
  • Higher scaling bits increase decrypted-value accuracy, which generally corresponds to encrypted aggregated models having test performance close to plaintext aggregated models.Source: §D.3 · p. 17
  • CKKS's approximate scheme and noise can produce positive or negative encrypted-model test-accuracy deltas, though the deltas are usually negative or nearly zero.Source: Table 6 · p. 17, §D.3 · p. 18

D.4 Impact from Number of Clients

  • As client numbers grow, added ciphertext inputs mainly increase server costs; client selection can remove clients when the server is overloaded without largely degrading model performance.Source: §D.4 · p. 18

D.5 Communication Cost on Different Bandwidths

FedML-HE is evaluated across intra-center, same-region inter-center, and cross-region communication settings. Its secure HE functionality has a much larger impact in low-bandwidth environments than in medium-to-high-bandwidth settings.

D.5 Communication Cost on Different Bandwidths

  • The evaluation examines how communication bandwidth affects communication costs and the FL training cycle across geo-located parties.Source: §D.5 · p. 18
  • The three bandwidth cases are Infiniband at 5 GB/s, Single AWS Region at 592 MB/s, and Multiple AWS Region at 15.6 MB/s.They represent intra-center, inter-center within one geo-region, and inter-center across geo-regions, respectively.Source: §D.5 · p. 18
  • HE has an enormous impact in low-bandwidth environments, while medium-to-high-bandwidth settings suffer limited impact during training cycles compared with Non-HE settings.The comparison uses three geo-distributed environments with different bandwidths.Source: §D.5 · p. 18

D.6 Different Encryption Selections

Table 7 reports overhead reductions across parameter-selection configurations for a Vision Transformer. Its caption specifies that the 10% configuration encrypts only 10% of parameters, with the rest aggregated in plaintext.

D.6 Different Encryption Selections

  • Table 7 reports overhead reductions for different parameter-selection configurations tested on a Vision Transformer.Source: §D.6 · p. 18, Table 7 · p. 19
  • In the “Enc w/ 10%” configuration, encrypted computation is performed only on 10% of parameters, while the remaining parameters use plaintext aggregation.The reported computation and communication results include the plaintext aggregation overhead.Source: Table 7 · p. 19

D.7 Comparison with Other FL-HE Frameworks

FedML-HE is compared with NVIDIA FLARE and IBMFL, using Microsoft SEAL-based implementations and an additional TenSEAL version of FedML-HE. In the CNN evaluation, the PALISADE-powered framework has the smallest computational overhead, while IBMFL has the smallest file serialization results.

D.7 Comparison with Other FL-HE Frameworks

  • The comparison covers NVIDIA FLARE and IBMFL, both of which use Microsoft SEAL as their underlying HE core.The paper also implements a TenSEAL version of FedML-HE for evaluation.Source: §D.7 · p. 19
  • FedML-HE (PALISADE) has the smallest computational overhead, while IBMFL has the smallest file serialization results in the CNN comparison.FedML-HE (PALISADE) ranks second in communication cost, attributed to HELayers’ efficient packing of Tile tensors.Source: §D.7 · p. 19
  • NVIDIA’s TenSEAL-based realization is faster than FedML-HE’s TenSEAL variant because it scales each learner’s parameters locally, avoiding a server-side ciphertext weighting multiplication.This setup is unsuitable when the central server does not want learners to learn its per-model weighting mechanism, which can reveal partial or full participant information.Source: §D.7 · p. 19
  • The frameworks are tested using a CNN with two convolutional and two fully connected layers and three clients.FedML-HE uses a key authority server, FLARE a security content manager, and IBMFL currently provides a local simulator.Source: Table 8 · p. 20
Loading 2303.10837v3…