Source-linked AI summary
PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
Fan Mo, Hamed Haddadi, Kleomenis Katevas, Eduard Marin, Diego Perino, Nicolas Kourtellis
TL;DR
Federated learning can leak sensitive information through model parameters and gradients despite keeping raw data local. PPFL uses client- and server-side TEEs with greedy layer-wise training to protect updates under TEE memory limits. It reports protection against three attack classes, comparable utility with fewer communication rounds, and modest client-side overhead.
Problem
Federated learning keeps raw data local, but adversaries can still extract sensitive information from model parameters and gradients.
Method
PPFL trains and aggregates each DNN layer inside client and server TEEs using greedy layer-wise training to accommodate limited trusted memory.
Results
PPFL protects against data reconstruction, property inference, and membership inference while achieving similar utility with 0.54× communication rounds and 1.002× network traffic versus end-to-end FL.
Takeaways & Limitations
PPFL provides comparable model utility and full privacy protection with client-side overheads up to ~15% CPU time, ~18% memory usage, and ~21% energy consumption.
Takeaways & Limitations
The evaluation uses training rounds sufficient to compare PPFL and regular FL at matched accuracy rather than to maximize CIFAR10 test accuracy.
Abstract
from arXiv · showhide
We propose and implement a Privacy-preserving Federated Learning ($PPFL$) framework for mobile systems to limit privacy leakages in federated learning. Leveraging the widespread presence of Trusted Execution Environments (TEEs) in high-end and mobile devices, we utilize TEEs on clients for local training, and on servers for secure aggregation, so that model/gradient updates are hidden from adversaries. Challenged by the limited memory size of current TEEs, we leverage greedy layer-wise training to train each model's layer inside the trusted area until its convergence. The performance evaluation of our implementation shows that $PPFL$ can significantly improve privacy while incurring small system overheads at the client-side. In particular, $PPFL$ can successfully defend the trained model against data reconstruction, property inference, and membership inference attacks. Furthermore, it can achieve comparable model utility with fewer communication rounds (0.54$\times$) and a similar amount of network traffic (1.002$\times$) compared to the standard federated learning of a complete model. This is achieved while only introducing up to ~15% CPU time, ~18% memory usage, and ~21% energy consumption overhead in $PPFL$'s client-side.
1 INTRODUCTION
Federated learning avoids centralizing user data, but model parameters and gradients can still leak sensitive information. PPFL uses TEEs and greedy layer-wise training to protect all layers while retaining practical utility and overhead.
- Motivation: FL trains a global model from locally updated parameters without requiring users to reveal their raw data.However, adversaries can retrieve sensitive information from model parameters themselves.
- Motivation: TEEs provide secure storage and near-native arbitrary code execution, but limited memory prevents placing all DNN layers inside the trusted area simultaneously.Prior approaches therefore protected only selected sensitive layers, leaving broader attack surfaces in FL.
- PPFL: PPFL protects model and gradient updates at both clients and servers by combining TEEs with greedy layer-wise training and aggregation.Each layer is trained inside the trusted area until convergence, overcoming TEE memory constraints.
- Evaluation: PPFL fully protects against data reconstruction, property inference, and membership inference attacks, degrading attack outcomes to random guessing.Examples include white-noise reconstructions and 50% precision scores.
- Evaluation: 0.54× communication rounds and 1.002× network traffic let PPFL achieve similar model utility to end-to-end FL.Client-side overhead reaches ~15% CPU time, ~18% memory usage, and ~21% energy consumption.
2 BACKGROUND AND RELATED WORK
The paper situates PPFL among FL privacy risks, TEE-based protections, and layer-wise training methods. Its key design insight is to train each layer inside a TEE, satisfying limited-memory constraints while protecting against multiple attack classes.
- Privacy risks: Data reconstruction attacks invert model gradients to recover original inputs and are especially effective on early layers or batches smaller than 8.This motivates protecting early-layer updates inside the trusted area.
- Privacy risks: Property inference attacks use classifiers trained on auxiliary-data gradients and can target both servers and clients.Property information may remain preserved in aggregated global-model snapshots observed by clients.
- Privacy risks: Membership inference attacks determine whether instances appeared in training data and can target the final model or its last layer.PPFL therefore supports keeping the last layer inside client TEEs after training.
- TEE-based protection: TEE-based ML can hide model parameters from untrusted software, but limited TEE memory has led prior work to protect only part of a DNN.PPFL instead keeps each trained layer inside TEEs during its training stage.
- Layer-wise training: Greedy layer-wise training starts with a shallow model, trains it to convergence, then adds and trains one new layer at a time.PPFL applies this strategy in FL and may build classifiers atop layers for prediction and loss computation.
3 THREAT MODEL AND ASSUMPTIONS
The threat model considers honest-but-curious clients and server owners seeking sensitive information from global models or gradients. PPFL assumes adequate TEE capacity, secure trust bootstrapping, key management, and server forwarding.
- Threat model: The standard FL setting has multiple clients train locally and send model parameters to a centralized server for aggregation.Adversaries seek sensitive information embedded in the resulting global model through reconstruction or inference attacks.
- Threat model: The adversaries are passive, honest-but-curious client users or server owners with access to model snapshots or updated gradients.They may have full control, including root privileges, over relevant untrusted software.
- Assumptions: PPFL assumes the server and enough participating clients have TEEs whose memory exceeds the DNN’s largest layer.If a layer does not fit, the design must use smaller layers or a smaller training batch size.
- Assumptions: PPFL assumes secure trust bootstrapping between server and client TEEs and mechanisms to update and revoke keys.Examples include modified SIGMA key exchange or attested TLS.
- Assumptions: A malicious server that refuses to forward data to its TEE affects availability, while the stated security and privacy properties remain intact.This denial-of-service case is outside the standard TEE threat model.
4 PPFL FRAMEWORK
PPFL keeps model layers and updates protected inside client and server TEEs while using greedy layer-wise training to overcome limited TEE memory. It trains each layer to convergence before proceeding, with secure aggregation and optional layer blocks supporting the federated workflow.
- 4.1 System Overview: PPFL trains each model layer to convergence inside TEEs before moving to the next layer, preserving privacy during layer updates.The framework uses greedy layer-wise training to address limited TEE memory.
- 4.1 System Overview: PPFL protects updated gradients from adversaries during training, while the final layer can remain inside client TEEs to reduce membership-inference risk.The design specifically treats the last layer as sensitive after convergence.
- 4.1 System Overview: Clients are selected according to whether their TEE memory can accommodate the model layer being trained.The server can reconstruct a model suitable for the available clients’ memory constraints.
- 4.1 System Overview: Client TEEs perform local layer training, while the server TEE securely aggregates participating clients’ layer parameters with FedAvg.Clients report completed layer parameters through secure channels before server-side aggregation.
- 4.2 Layer-wise Training and Aggregation: Grouping multiple layers into blocks can improve TEE-memory utilization and reduce communication rounds needed for block convergence.Blocks are trained inside client TEEs similarly to individual layers.
- 4.2 Layer-wise Training and Aggregation: Model-partitioned execution transfers activations between REEs and TEEs while executing the target layer and its classifier inside the TEE.Previous layers remain frozen during training of the target layer.
- 4.2 Layer-wise Training and Aggregation: PPFL’s training overhead comes from repeated forward passes through previous layers and repeated forward and backward passes through the classifier.These costs arise because earlier layers are frozen while successive layers are trained.
5 IMPLEMENTATION & EVALUATION SETUP
The PPFL prototype combines Arm TrustZone clients, Intel SGX servers, secure communication, and evaluations on CNNs using MNIST and CIFAR10. Evaluation measures privacy attacks, model performance, and client-side system costs.
- 5.1 PPFL Prototype: The prototype implements PPFL with Arm TrustZone on HiKey 960 clients and Intel SGX on an Intel NUC server.The client uses 16MiB TEE secure memory, while the server uses an SGX-enabled NUC.
- 5.1 PPFL Prototype: Secure channels use SSH and SCP, with data leaving TEEs encrypted using AES-CBC and 128-bit cryptographic keys.The implementation establishes secure communication between server and client components.
- 5.2 Evaluation Setup: The evaluation focuses on CNNs and uses MNIST and CIFAR10 to enable comparison with prior federated-learning methods.LeNet is tested on MNIST, while the other evaluated models use CIFAR10.
- 5.2 Evaluation Setup: Evaluation assesses privacy, model performance, and client-side system cost, including test accuracy, communication rounds, and total communication.The study also measures CPU execution time, memory usage, and energy consumption.
- 5.3 Performance Metrics: The privacy assessment applies data reconstruction, property inference, and membership inference attacks to PPFL and standard end-to-end FL models.Attack outcomes are measured using MSE, AUC, and precision, respectively.
- 5.3 Performance Metrics: The evaluation measures energy for one on-device training step and controls trials with continuous 2000-step runs after cooling periods.Energy is recorded with a Monsoon High Voltage Power Monitor.
6 EVALUATION RESULTS
PPFL protects against data reconstruction, property inference, and membership inference attacks while preserving comparable utility and modest client-side overhead. Its layer-wise design can reduce communication, but performance varies with model complexity, data heterogeneity, block size, and initialization.
- Privacy protection: PPFL reduces data reconstruction, property inference, and membership inference attacks to near-random outcomes by protecting intermediate gradients and the final layer inside client TEEs.The reported outcomes include noisy reconstructions, AUC of approximately 0.5 for property inference, and membership-inference precision of approximately 0.5.
- Communication cost: PPFL adds small communication overhead and can sometimes reduce communication while achieving performance comparable to end-to-end FL.The reduction is attributed to reaching good accuracy during early layer-wise training without training the complete model.
- Client-side cost: 3× or higher delay is introduced when completing layer-wise training because finalized layers are still processed during each forward pass.Training one layer costs a similar amount of time to end-to-end training of the complete model, so clients can contribute for only part of an FL round.
- Model utility: 74% and 60.78% test accuracy are reached by AlexNet on IID and Non-IID CIFAR10, while VGG9 reaches 74.60% and 38.35%, respectively.PPFL performance can match or surpass end-to-end FL toward the end of 50 rounds, although adding a new layer initially causes a drop because training restarts for that layer.
- Model utility: Non-IID VGG9 can fall to approximately 55% after the second layer, motivating early exits for clients with insufficient data or already-high accuracy.Two-layer blocks can recover performance for complex models and require half or less communication cost than one-layer blocks at the same target accuracy.
- Client-side cost: 14.6% CPU time, 18.31% memory usage, and 21.19% energy consumption are the maximum client-side TEE overheads reported for one training step.The comparison is against layer-wise training without TEEs, and PPFL remains comparable to end-to-end training when each layer is trained.
- Block training: 13.24% CPU time, 32.71% memory usage, and 14.47% energy consumption are the maximum overheads reported when training two-layer blocks.Combining layers does not always increase overhead; for VGG9, CPU and energy overhead decrease relative to one-layer training.
- Bootstrapping: Pre-trained feature extractors can help final PPFL models reach accuracy similar to centralized training, but adding more fully connected layers does not improve accuracy.The paper attributes this limitation to frozen transferred feature extractors that cannot fully capture variability in new data.
7 DISCUSSION & FUTURE WORK
PPFL’s evaluation identifies privacy protections, communication and client-cost trade-offs, and several directions for extending or accelerating the framework.
- PPFL protects gradient updates and exposes layers only after convergence, thwarting data reconstruction and property inference attacks while the last layer mitigates membership inference.
- 0.538× communication rounds and 1.002× communication achieve the same test accuracy as end-to-end FL.Layer-wise FL increases the total rounds needed to finish all layers, but can reach the same accuracy with fewer rounds overall.
- Up to ∼15% CPU time, ∼18% memory usage, and ∼21% energy consumption are incurred on clients, with most system cost coming from local training.
- Training 2-layer blocks decreases communication cost by at least half while slightly increasing system overhead for small models.
- Pre-trained models can bootstrap PPFL training, increasing ML utility and reducing communication and system overhead.
- Model Architectures: PPFL’s tested architecture is compatible with possible extensions to graph neural networks, LSTMs, and block-based training for models with skipping connections.ResNet layer-wise training has not been investigated, but including its shortcut inside a block could be attempted.
- Accelerating Local Training: PPFL currently uses only client CPUs and sequential layer training, leaving specialized processors and GPU-based TEEs as acceleration opportunities.
- Federated Learning Paradigms: PPFL was tested with FedAvg, while other FL paradigms may be compatible because PPFL does not alter hyper-parameter determination, loss functions, or aggregation.
8 CONCLUSION
The conclusion presents PPFL as a practical framework that protects client information with TEEs while preserving comparable utility and avoiding significant communication and system overhead.
- PPFL protects clients’ private information against known privacy-related attacks through greedy layer-wise training inside server and client TEEs.The prototype uses TrustZone for mobile-like clients and Intel SGX for servers.
- PPFL achieves comparable ML utility with regular end-to-end FL without significant communication and system overhead.
A.1 Transferring Public Datasets
PPFL uses server-held public data to initialize a model, while clients train only the final layers inside TEEs and upload those layers for secure aggregation. Public-data coverage and data heterogeneity determine whether this approach preserves or improves accuracy.
- Method: PPFL initializes a global model from server-held public data, then clients train and upload only the last layers inside the TEE.Only the trained final layers are sent for secure aggregation.
- Accuracy with public data: More server-held public data generally produces higher final global-model test accuracy, although this trend is not universal across data distributions.With complete training data, the process becomes centralized training.
- Accuracy with public data: In IID settings, training all layers with 0.1 public-data fraction can outperform end-to-end FL, whereas this advantage does not hold for Non-IID CIFAR10.The first layers trained on public data may fail to represent private-data features.
- Accuracy with public data: With insufficient public data, training only the last one or two layers can yield extremely low performance or training failure.The limitation arises when public-data-trained first layers inadequately represent client datasets.
- Layer selection: The number of trained last layers has little effect in IID cases but significantly affects test accuracy in Non-IID cases.IID public data can represent the complete private-data feature space more effectively.
- Overall finding: With public datasets, PPFL can guarantee privacy while achieving better performance than training all layers.This overall result depends on the public data and distribution conditions described above.