Source-linked AI summary
Private Computation Space: Experience with Trusted Multi-Cluster Federated Learning for Agriculture
Shuangyu Lei, Muhammad Salman Abid, Jacob Belding, Sam Mosher, Manushi B. Trivedi, Shivranjani Baruah, Liam Wickes-Do, Andrew Anderson, Braulio Dumba, Alyssa Whitcraft, Ritvik Sahajpal, Sijin Li, Kelly Robbins, Michael Gore, Margaret Frank, Steven Wolf, Liz Jones, Abraham Stroock, Kaitlin Gold, Hakim Weatherspoon
TL;DR
Agricultural ML adoption is constrained by farmer privacy concerns, scarce heterogeneous data, and fragile rural infrastructure. The paper introduces PCS, a trusted multi-cluster federated-learning system, and evaluates it in two deployments, improving worst-site accuracy while preserving privacy.
Problem
Farmer privacy concerns, scarce heterogeneous data, and fragile rural infrastructure complicate agricultural ML participation and limit the applicability of large-scale federated-learning strategies.
Method
PCS combines Kubernetes-based multi-cluster orchestration with asynchronous Federated Learning for sensitive agricultural data across farm clusters and public cloud.
Results
PCS achieved DSC 0.71 for nitrogen monitoring and R^2 0.84 for evapotranspiration prediction, improving worst single-site accuracy by 22.4% and 9.1%, respectively.
Takeaways & Limitations
PCS operated for six and ten months despite client dropouts and network disruptions, while TEE-secured aggregation added less than 2% end-to-end training-time overhead.
Takeaways & Limitations
TEE protection depends on secure underlying hardware and timely infrastructure-provider security updates because software-only exploits can breach TEE security.
Abstract
from arXiv · showhide
Artificial Intelligence has shown to help improve agricultural practices, yet adoption remains limited: 69% of U.S. farmers have privacy concerns with sharing their data, and these concerns must be addressed before adoption is widespread. While Federated Learning has been demonstrated to protect privacy at scale for other sectors, deploying a system for agriculture comes with its own set of challenges; the problem necessitates a system that can protect farmer data and identities while preserving model utility, runs on commodity hardware, and is resilient to fragile rural infrastructure. To address these concerns, we introduce the Private Computation Space (PCS), a deployed, open-source Machine Learning system to provision and process farmer data securely. We design a system tailored to an agricultural setting, with multi-cluster orchestration for reliability in rural areas with asynchronous Federated Learning (FL), Differential Privacy (DP), and Trusted Execution Environments (TEEs), to allow farms to participate in the framework while keeping their data private. We evaluate the system on two deployed workloads: monitoring nitrogen with living plant sensors in NY for six months and predicting evapotranspiration from weather stations in CA for ten months. Our evaluation finds a Dice Similarity Coefficient (DSC) of 0.71 and $R^2$ accuracy of 0.84 for the respective workloads, improving the worst single-site model accuracy by 22.4% and 9.1%, respectively, while preserving privacy.
1 Introduction
Agricultural ML adoption is constrained by farmer privacy concerns and rural deployment conditions. PCS addresses these constraints with trusted multi-cluster orchestration, asynchronous federated learning, Differential Privacy, and TEEs, and reports improved utility in two deployments.
- Motivation: 69% of U.S. farmers reported privacy concerns about sharing their data, while trust in private companies and government was also limited.The cited passage reports that 73% did not trust private companies and 58% did not trust the government.
- System: PCS combines Kubernetes-based multi-cluster orchestration with asynchronous Federated Learning for sensitive farmer data.The system runs workloads across on-premise farm clusters and the public cloud.
- Privacy design: Farm clusters keep data on-premises, while Differential Privacy and TEE-based cloud aggregation protect participation and model weights.The hybrid design provisions and processes farmer data without moving raw training data off client premises.
- Deployments: The deployments cover nitrogen monitoring across three New York farms for six months and evapotranspiration prediction across ten California weather stations for ten months.These workloads represent geographically distributed agricultural data and resource-management tasks.
- Results: DSC reached 0.71 for nitrogen monitoring and R^2 reached 0.84 for ET prediction, improving the worst single-site accuracy by 22.4% and 9.1%, respectively.PCS continued operating despite client dropouts and network disruptions, with TEE-secured aggregation adding less than 2% overhead relative to end-to-end training time.
2 Background
Digital Agriculture applies digital technologies to agricultural practice, including resource management and predictive monitoring. Its benefits depend on systems that preserve privacy and tolerate heterogeneous data, unreliable connectivity, and client dropouts; Federated Learning and privacy mechanisms address parts of this challenge.
- Digital Agriculture: Digital Agriculture applies digital methods and technologies to traditional agricultural practices, including precision management of water, fertilizer, and feed.The passage reports that 68% of large-scale crop-producing farms used yield or soil monitoring technologies in 2023.
- Requirements: Farmer-facing tools must use resources efficiently, predict disease, operate through unreliable connectivity, and keep grower data private.These functions connect agricultural utility with reliability and privacy requirements.
- Machine Learning: Agricultural ML extends sensor-based monitoring by predicting crop conditions ahead of time to support proactive maintenance and better crop health decisions.The supplied passage describes prediction as an extension of monitoring with precise irrigation, soil analysis, and plant-health sensors.
- Privacy: Privacy is a prominent Digital Agriculture challenge because sensor data can contain coordinates, location metadata, and embedded owner information.The passage identifies privacy, financial risk, and equitable participation as reasons farmers may seek control over this information.
- Federated Learning: Federated Learning trains locally and aggregates model updates, while asynchronous FL avoids waiting for all clients when dropouts or partitions occur.FedBuff aggregates K client updates per server step to reduce aggregation frequency while retaining tolerance of stragglers and partitions.
- Privacy mechanisms: Differential Privacy adds calibrated noise to limit what an attacker can learn about an individual record, with smaller epsilon providing stronger privacy.The privacy budget epsilon quantifies the guarantee.
3 Experience
PCS was deployed in two long-running agricultural workloads—nitrogen monitoring in New York and evapotranspiration prediction in California—to study accuracy, privacy, reliability, and overhead under real farm conditions.
- PCS ran nitrogen monitoring across New York farms for six months and evapotranspiration prediction across ten California weather stations for ten months.
- Central DP reduced nitrogen segmentation from DSC = 0.85 to DSC = 0.71 at 𝜖=10, providing a formal membership privacy guarantee.
- In ten months of live ET prediction, 9 of 10 stations achieved R^2 between 0.84 and 0.92; one station reached R^2 = 0.66 after lacking ground-truth measurements for seven months.
- Agricultural deployments must balance actionable accuracy, privacy, reliable operation under fragile infrastructure, and resource overhead.
- Heterogeneous, non-IID farm data is scarce, costly to collect, and often insufficient for each client to train a robust model independently.
4 System Design
PCS combines federated learning, trusted execution, differential privacy, and multi-cluster orchestration to keep data local, secure aggregation, protect participation, and tolerate unreliable rural connectivity.
- PCS keeps raw data on farm clusters through FL, aggregates model weights in a cloud TEE, and applies DP to protect client participation and released models.
- FL clients process data locally and share model weights with the server, so raw data never leaves the farm.
- The TEE-based Trusted Secure Server protects aggregation against model providers, infrastructure providers, and third parties.
- Remote attestation lets clients verify the trusted binary before sending updates, and TEE hosting with attestation meets the data and computation privacy goals.
- Central DP adds noise to aggregated results before the federated model is released, while local DP noises client weights before upload but typically reduces utility.
- Multi-cluster orchestration deploys client and server clusters, while synchronous and asynchronous FL support tolerance of stragglers, dropouts, and network disruptions.
5 Implementation
PCS implements its agricultural platform with KubeStellar-managed farm clusters, confidential virtual machines for aggregation, and client software that supports local training and monitoring.
- KubeStellar uses workload and inventory spaces to deploy workloads, aggregate status, and report across independent farm Kubernetes clusters.
- Farm clusters can continue operating during control-plane disconnections when a valid deployment is already present.
- Confidential VMs protect code and data from the virtual machine monitor and host using hardware-based TEE support.
- PCS integrates Confidential Containers to provision aggregation workloads in confidential VMs without modifying the server application.
- Long-running trusted aggregation amortizes confidential-VM boot and memory-acceptance overheads.
- The PCS client pulls models, trains on local data, can run concurrent training tasks, and reports metrics to the server.
6 Deployment
PCS was evaluated in two agricultural deployments using privacy-preserving training, asynchronous aggregation, and TEE-secured infrastructure. Across nitrogen monitoring and ET prediction, the system maintained strong accuracy, tolerated client availability problems, and added little aggregation overhead.
- Accuracy and privacy: Federated Learning matched centralized accuracy while keeping raw images or station data on-device in both workloads.Nitrogen achieved DSC 0.85 versus 0.86 centralized, while ET achieved R^2 0.88 versus 0.89 centralized.
- Accuracy and privacy: Central DP remained viable, reaching DSC 0.71 for nitrogen and R^2 0.84 at 𝜖=10.49 for ET, whereas LDP required larger budgets or underperformed local baselines.For nitrogen, LDP achieved only 0.55 at 𝜖=100; for ET, LDP reached 0.83 at 𝜖≈63.
- Async FL robustness: For nitrogen, CDP required K=3 with only three clients to converge above DSC 0.7 because smaller buffers left DP noise insufficiently averaged.The experiment replayed high-, medium-, and low-availability traces from a greenhouse client.
- Async FL robustness: K=8 was the ET operational sweet spot, bringing R^2 within 0.03 of the no-DP baseline while completing 80 aggregations within 168 rounds across availability tiers.With low availability, K=8 required 156 rounds, while K=10 failed to complete within the 168-round window.
- TEE overhead: TEE-secured aggregation added less than 2% overhead across both deployments and local epoch sizes, with client-side training amortizing server-side cost.The comparison used bare and TEE server configurations while clients ran on Raspberry Pi hardware.
7 Limitations
PCS has limitations involving its threat model, underlying TEE hardware, and the behavior of asynchronous aggregation under unequal client availability.
- PCS assumes an honest-but-curious threat model, leaving data poisoning and model poisoning outside its scope.
- TEE protection depends on secure hardware and timely security updates, because software-only exploits can breach AMD SEV-SNP-based Confidential Containers.
- FedBuff can theoretically bias aggregation toward more available clients under non-IID data, although neither deployment showed a final-accuracy effect.
8 Related Work
Related work shows that agricultural FL requires specialized privacy, connectivity, and orchestration support beyond general-purpose frameworks. PCS addresses these gaps by combining multi-cluster management, buffered asynchronous aggregation, Differential Privacy, and cloud-only TEE protection.
- Agricultural digital systems collect and process extensive data, making privacy a key requirement for farmer adoption.
- Prior agricultural FL studies report privacy-preserving, near-baseline results, but broad support for heterogeneous applications remains limited.
- Farm infrastructure must tolerate fragile rural connectivity and temporary unavailability through distributed or disconnected operation.
- Unlike general-purpose FL frameworks, PCS provides buffered asynchronous aggregation, multi-cluster orchestration, and partition recovery for agricultural deployments.
- PCS confines TEE-secured aggregation to the cloud, allowing farm clients to run on commodity Raspberry Pis.
- PCS uses Differential Privacy rather than Papaya FL’s masking approach, with an acceptable accuracy cost and protection against membership inference.
9 Conclusion
The paper presents PCS as a privacy-preserving FL system for agricultural resource-allocation workloads, designed for heterogeneous farms and fragile connectivity. Across two deployments, it improved worst single-site accuracy while supporting farmer privacy, with future work targeting broader use cases and national-scale deployment.
- PCS combines Kubernetes-based multi-cluster FL with Differential Privacy and synchronous or asynchronous training for agricultural deployments.
- PCS supports resilient participation by farm clusters despite fragile rural infrastructure through multi-cluster orchestration and asynchronous fallback protocols.
- 22.4% improvement in worst single-site model accuracy was achieved for nitrogen monitoring, while preserving privacy.
- 9.1% improvement in worst single-site model accuracy was achieved for evapotranspiration prediction, while preserving privacy.
- Future work will extend PCS to more agricultural use cases and scale it toward a national infrastructure-level deployment.