Source-linked AI summary
Toward verifiably private learning from federated data
Katharine Daly, Yu Xiao, Zachary Garrett, Brett McLarnon, Jianpeng Hou, Arun Ganesh, Yanxiang Zhang, Noriyuki Takahashi, Haicheng Sun, Yuanbo Zhang, Timon Van Overveldt, Daniel Ramage
TL;DR
Earlier federated-learning systems faced operational constraints from device-side computation and availability. This paper presents a TEE-based system that ties encrypted uploads to permitted server-side programs, and reports faster Gboard training with a smaller privacy budget and parity on key usability metrics.
Problem
Earlier systems required on-device gradient computation, which limited model size, and synchronization across devices with unpredictable availability and heterogeneous resources posed operational challenges.
Source: §1 · pp. 1–2Method
The system uploads encrypted device data tied to policies that permit only specific programs to process it later in system-provided TEEs.
Source: §3 · p. 2Results
3x faster training and a 3x smaller privacy budget enabled the TEE-trained English Gboard language model to maintain parity on key usability metrics against the production model.
Source: §7 · p. 10Takeaways & Limitations
The productionized system supports Gboard typing-decoder models with externally verifiable Differential Privacy guarantees.
Source: §7 · p. 10The pipeline operator can observe which uploads are used in each training round, limiting privacy amplification by sampling.
Source: §6.3 · p. 9
Abstract
from arXiv · showhide
Federated Learning (FL) allows devices with private data to collaborate in training a shared model. We present a next-generation FL system based on Trusted Execution Environments (TEEs) that addresses operational challenges associated with earlier systems and provides externally verifiable central Differential Privacy (DP) guarantees for the first time while offering a better privacy-utility tradeoff. In our system, devices upload data encrypted with keys managed by a TEE-hosted Key Management Service (KMS). The uploaded data is cryptographically tied to a policy limiting the set of Python programs that may later process the data in server-side TEEs. External parties may inspect public transparency logs to observe the set of workloads allowed by these policies. Our experimental results show that the new system improves device coverage and favorably shifts privacy-utility curves by enabling collected data to be integrated into the server-side workload at a schedule that optimizes DP guarantees and is unaffected by device availability. Our new system has been productionized, enabling models for the Android Keyboard (Gboard) to be trained faster and achieve better accuracy under smaller, now externally verifiable privacy budgets in comparison to models trained using the prior system.
1. Introduction
Federated learning lets clients collaborate while retaining control over data, workloads, and anonymization. The paper presents a TEE-based system aimed at strengthening privacy principles and easing operational constraints of earlier device-centered systems.
1. Introduction
- Federated learning is a setting where multiple clients collaborate on a machine-learning problem under a service provider’s coordination.Clients should retain control over their data, permitted workloads, and anonymization properties, with transparency and control for users whose data they manage.Source: §1 · p. 1
- The system uses TEEs to raise the bar on data minimization, data anonymization, transparency and control, and verifiability and auditability.These four principles were defined in earlier work.Source: §1 · p. 1
- Earlier systems’ on-device gradient computation limited model size, while fleet synchronization and heterogeneous resources created operational challenges.The new system permits devices to participate independently and moves most computation server-side.Source: §1 · pp. 1–2
2. Related Work
Prior work applied TEEs to server-side inference and federated learning, and introduced workload-agnostic infrastructure for verifiable processing. This paper describes its production deployment for arbitrary Python workloads.
2. Related Work
- Server-side TEEs also support privacy-sensitive, computationally expensive mobile inference, though systems vary in privacy, security, and external inspectability guarantees.The cited examples include Apple Private Cloud Compute, Google Private AI Compute, and Meta Private Processing; current-generation TEEs have known security limitations.Source: §2 · p. 2
- Earlier TEE-based FL systems include Papaya’s asynchronous server-side aggregation and OpenFL’s edge enclaves for protecting model IP and validating code.Papaya computes gradients on-device; OpenFL uses server-side TEE validation before incorporating updates.Source: §2 · p. 2
- A prior architecture enabled devices to upload data for future processing only by server-side TEE workloads known at upload time.A production deployment used it for LLM structured summarization and DP aggregation; this paper applies it to arbitrary Python programs, including ML training and evaluation.Source: §2 · p. 2
3. System Description
The system separates data upload from server-side processing, with a KMS linking devices to future TEE workloads. Its Python interface supports controlled data access, distributed execution, public workload verification, and recovery across executions.
3. System Description
- Devices upload encrypted data for later processing only by specific programs in system-provided TEEs, with workload and runtime binaries externally inspectable.The system also supports distributed execution, dynamic parameter loading, proprietary per-user processing, and failure recovery without impacting privacy guarantees.Source: §3 · pp. 2–3, Figure 1 · p. 3
3.1. Program API
- The trusted_program function runs in a root TEE and receives an ExternalHandle for uploaded-data access, runtime inputs, plaintext result release, and recovery information.The workload author is responsible for ensuring released results are appropriately anonymized.Source: §3.1 · pp. 3–4
3.2. Distributed Execution
- For compute-heavy workloads, the root TEE can delegate portions of a program to worker TEEs; Federated Language intrinsics identify work that can be parallelized.Program portions without those intrinsics run entirely on the root TEE.Source: §3.2 · p. 4
3.3. Verifiability and Transparency
- Before upload, devices can know the server-side workloads that may later process their data through a KMS-brokered chain of trust.The KMS uses a cluster of TEEs and maintains a rollback-protected database of keys and pipeline-specific state.Source: §3.3 · p. 4
3.3.1. Chain of Trust
- Devices validate that KMS software and access policies are recorded in public transparency logs, then encrypt uploads tied to policies restricting which TEE binaries receive decryption keys.For Python workloads, policies embed the program and root and worker binary hashes; auditors can inspect open-source logic and how runtime-delivered inputs are used to maintain DP guarantees.Source: §3.3.1 · pp. 4–5
3.3.2. Pipeline State
- KMS pipeline state links released results to encrypted recovery information, allowing a failed training program to resume without extra privacy budget.Recovery cannot use information from before the most recently released result, preventing re-release of earlier outputs that could expose additional information.Source: §3.3.2 · p. 5
4. System Privacy Comparison
Compared with the prior system, the TEE-based design uploads encrypted training data for server-side processing, improves workload visibility, and collects uploads before training so participation and DP parameters can be optimized at runtime.
4. System Privacy Comparison
- The comparison evaluates how the new design changes privacy considerations and creates headroom for stricter DP guarantees or improved utility.Some design changes directly improve the privacy principles introduced earlier.Source: §4 · p. 5
4.1. Data Minimization and Anonymization
- The new system uploads encrypted training data rather than on-device gradients, enabling a server-side DP loop after one client upload while keeping raw data inaccessible to the pipeline operator.A KMS-enforced time-to-live limits decryption to a timeframe on a best-effort basis.Source: §4.1 · p. 5
4.2. Transparency, Control, Verifiability, and Auditability
- Before upload, devices can inspect all future server-side workloads, and public transparency logs let third-party auditors see potentially active binaries without participating devices.The access-policy hashes describe reproducibly buildable TEE containers based on open-source code.Source: §4.2 · pp. 5–6
4.3. Privacy-Utility Curves
- Collecting uploads before training allows runtime optimization of MF-DP-FTRL parameters and equal device participation, favorably shifting privacy-utility curves.The system can achieve the maximum possible minSep for available uploads and target rounds, then derive a BLT and noise multiplier based on the target zCDP or vice versa.Source: §4.3 · p. 6
5. Experimental Results
The experiments compare the TEE system with the older baseline on device coverage and privacy–utility tradeoffs, and evaluate a TEE-trained English model in a Gboard A/B experiment.
5.1. Improved Device Coverage
- The baseline incorporated 8.5M of 35.5M available devices (23.9%) over 3,000 rounds and 38 days, while TEE training used all 17.8M uploads collected in about six days.Baseline losses included 20.5M devices never assigned work and 6.5M interrupted; TEE training start time trades waiting for uploads against the decryption-key TTL.Source: §5.1 · p. 6, Figure 2 · p. 7
5.2. Shifted Privacy-Utility Curves
- At 5,000 rounds, zCDP 0.232 required noise multiplier 5.16 in TEE versus 9.54 baseline; maxP/minSep were 3/1822 versus 8/561.The TEE’s two-stage design enabled on-demand access to uploads and optimized participation patterns; the curves show possible zCDP and noise-multiplier combinations.Source: §5.2 · p. 7, Figure 3 · p. 8
5.3. Live A/B Testing
- 3x smaller privacy budget (zCDP=0.215 vs zCDP=0.641, adjusted for the same MF-DP-FTRL mechanism) maintained neutral key usability metrics; training took 3 weeks versus 2 months.The usability metrics were Words Per Minute and Words Modified Ratio; each experiment arm included 3.5M devices.Source: §5.3 · p. 8
6. Future Work
Future work targets infrastructure scale, model-specific privacy–compute tuning, additional privacy-utility headroom, and uses beyond model training. The proposed directions include parallel server-side execution, upload obfuscation, and broader workloads, with larger models and optimal Gboard configurations still requiring further work.
6.1. Infrastructure Scaling
- Parallel root-to-worker TEE execution improves round times, while training beyond the tested 10M-parameter scale may require worker GPUs and root-worker communication fixes.The reported parallelism improvement was observed with as few as 14 machines.Source: §6.1 · p. 8
6.2. Hyperparameter Optimization
- At fixed work budget, increasing cohort size improves signal-to-noise ratio, but maxP transitions cause abrupt drops as more noise is required.The work budget is approximated by T×B; larger cohorts also have diminishing returns for per-step convergence, so cohort size should be bounded by compute and convergence goals.Source: §6.2 · p. 9, Figure 4 · p. 9
- The Gboard experiments tested only one configuration per model, and additional experiments are needed to find optimal hyperparameters under the privacy, compute, and convergence trade-offs.A Python program could explore configurations with early stopping and dynamic candidate selection instead of requiring multiple parallel server-side workloads.Source: §6.2 · p. 9
6.3. Additional Privacy-Utility Headroom
- Obfuscating which uploads are used in each round could improve sampling amplification, reducing the consumed privacy budget while keeping the noise multiplier constant.Proposed approaches include verifiable shuffling or having the workload over-select uploads so the operator cannot see the exact subset used.Source: §6.3 · pp. 9–10
6.4. Additional Use Cases
- The system's arbitrary-Python execution can support use cases beyond model training, including synthetic-data generation and inference.The authors are also exploring pipelines combining this system's data-processing TEEs with those from earlier work.Source: §6.4 · p. 10
7. Conclusion
The paper presents a TEE-based federated-learning system with externally verifiable Differential Privacy guarantees and improved privacy-utility trade-offs. In a Gboard typing-decoder A/B test, its English model trained faster and with a smaller privacy budget while maintaining parity on key usability metrics.
7. Conclusion
- 3× faster training and a 3× smaller privacy budget maintained parity on key usability metrics for the TEE-trained Gboard English typing-decoder model.The decoder supports autocorrection, word completion, and next-word prediction.Source: §7 · p. 10
- The two-stage upload-then-server-side-training approach lets collected data enter training on a schedule optimized for DP algorithms, producing more favorable privacy-utility curves.The schedule is not tied to device availability.Source: §7 · p. 10
- The system provides externally verifiable Differential Privacy guarantees while improving other aspects of federated-learning training relative to earlier systems.Source: §7 · p. 10
A. Supplemental Listings
The supplemental listings illustrate a Python workload that selects and processes uploads, releases noised results, and stores recovery information. They also show how the access policy restricts execution to specified programs and attested containers.
A. Supplemental Listings
- The example program selects available clients, parses their data, adds values to a cumulative sum, releases a noised sum, and stores encrypted recovery information.Source: A. Supplemental Listings · p. 13
- KMS pipeline state identifies the latest atomically released result and its associated encrypted recovery information, allowing execution to validate whether it is starting or resuming.Source: A. Supplemental Listings · p. 14
- The illustrated data-flow configuration allows reads of device uploads and intermediate recovery information, and outputs encrypted recovery information and unencrypted anonymized results.Source: A. Supplemental Listings · p. 14
- The root container permits only the specified program, which checks its computed zCDP against a maximum allowed value of 0.58.Source: A. Supplemental Listings · p. 14
- The access policy attaches the inlined Python program and hashes of permitted containers to uploaded data, while the KMS checks root-container reference measurements before providing decryption keys.The measurements span the firmware, kernel, system image, and root-container layers.Source: A. Supplemental Listings · p. 14