Source-linked AI summary
The Role of Cross-Silo Federated Learning in Facilitating Data Sharing in the Agri-Food Sector
Aiden Durrant, Milan Markovic, David Matthews, David May, Jessica Enright, Georgios Leontidis
TL;DR
Agri-food data owners may hesitate to share commercially sensitive data, limiting the pooled data available for machine learning. This paper evaluates cross-silo federated learning and model sharing for soybean yield prediction without exchanging raw data. The methods perform better than locally trained models and close to traditional machine-learning baselines in the reported setting.
Problem
Method
The study evaluates cross-silo federated learning and model sharing on decentralized data, with differential privacy methods used to address privacy concerns.
Source: §1 · pp. 2–3Results
5.32% and 6.68% reduction in RMSE for the image and tabular dataset respectively, with federated models also outperforming locally trained models.
Source: §8 · p. 19Takeaways & Limitations
The results provide empirical evidence that privacy-preserving collaborative training can perform close to traditional machine-learning counterparts in this setting.
Source: §9 · p. 20Real-world implementation, statistical heterogeneity, and social and political challenges remain future-work concerns.
Source: §8 · p. 19, §9 · p. 20
Abstract
from arXiv · showhide
Data sharing remains a major hindering factor when it comes to adopting emerging AI technologies in general, but particularly in the agri-food sector. Protectiveness of data is natural in this setting; data is a precious commodity for data owners, which if used properly can provide them with useful insights on operations and processes leading to a competitive advantage. Unfortunately, novel AI technologies often require large amounts of training data in order to perform well, something that in many scenarios is unrealistic. However, recent machine learning advances, e.g. federated learning and privacy-preserving technologies, can offer a solution to this issue via providing the infrastructure and underpinning technologies needed to use data from various sources to train models without ever sharing the raw data themselves. In this paper, we propose a technical solution based on federated learning that uses decentralized data, (i.e. data that are not exchanged or shared but remain with the owners) to develop a cross-silo machine learning model that facilitates data sharing across supply chains. We focus our data sharing proposition on improving production optimization through soybean yield prediction, and provide potential use-cases that such methods can assist in other problem settings. Our results demonstrate that our approach not only performs better than each of the models trained on an individual data source, but also that data sharing in the agri-food sector can be enabled via alternatives to data exchange, whilst also helping to adopt emerging machine learning technologies to boost productivity.
1. Introduction
The paper addresses reluctance to share commercially sensitive agri-food data by proposing decentralized machine-learning approaches for horizontal collaboration. It demonstrates the approach on soybean yield prediction and outlines additional production and distribution use cases.
1. Introduction
- The investigation targets horizontal sharing across supply-chain participants for optimization or regulatory purposes, rather than vertical traceability.It uses soybean yield prediction and open-source imaging and tabular datasets to demonstrate the approach.Source: §1 · pp. 1–2
- Data owners’ sensitivity and competitive concerns make collaboration difficult, while limiting access to analytics that could improve production optimization.The proposed goal is to improve on individual models without disclosing sensitive data.Source: §1 · p. 2
- The paper argues that privacy-preserving methods can support trustworthy collaboration while participants retain independent ownership of their data.It identifies differential privacy as its method for addressing inference-attack concerns in federated learning.Source: §1 · pp. 2–3
- Federated learning and model sharing are proposed to train across distributed agri-food datasets without direct data pooling.The paper also argues that differential privacy can address privacy and security concerns in this setting.Source: §1 · pp. 2–3
1.1. Use cases
- Soft-fruit growers could collaborate across farms to improve production analytics despite limited data-collection resources.The use case connects better estimates with reduced risks of fruit waste or shortages and their financial and environmental repercussions.Source: §1.1 · p. 3
- A fresh-food distributor could coordinate a shared model across clients to inform product improvements and food availability.The proposed use case also identifies potential reductions in transport costs and food waste.Source: §1.1 · p. 3
2. Related Work
The related work describes limited direct data sharing in agri-food despite interest in traceability and data analytics. It positions cross-silo federated learning and privacy-preserving methods as approaches for learning across independent organizations.
2. Related Work
- Direct sharing of raw data between agri-food organizations remains uncommon, while prior work has addressed traceability and regulatory challenges.The cited discussion contrasts this with domains where data sharing has supported research.Source: §2 · pp. 3–4
- Machine learning has been applied to yield prediction, crop disease detection, and production safety, but federated learning has not been widely applied in agri-food.The paper suggests these applications may be replicated or approximated without direct data pooling.Source: §2 · p. 4
- Cross-silo federated learning involves a few clients, each representing a larger data store such as a company or organization.This differs from cross-device settings involving millions of edge-device clients.Source: §2 · p. 4
- Differential privacy introduces uncertainty into released models to mask individual contributions, while fully homomorphic encryption permits computation on encrypted data.The discussion also identifies secure aggregation as a way to reduce risks from public attacks.Source: §2 · p. 4
3. Problem Setting
The study asks whether models can use data from independent agri-food silos without centralizing or explicitly sharing that data. It evaluates model sharing and federated learning on two soybean-yield datasets partitioned to simulate separate state-level data owners.
3. Problem Setting
- Direct pooling is considered impractical because of privacy concerns, distrust, and commercial sensitivity, including the risk of inferring sensitive financial information from yields.The problem setting asks whether distributed training can use multiple silos while preserving privacy through appropriate mechanisms.Source: §3 · p. 4
- The study compares model sharing, which transfers models rather than data, with federated learning across independent silos.Because real-world multi-organization data were limited, the authors simulate silos by partitioning two open-source datasets into subsets.Source: §3 · p. 5
3.1. Remote Sensing Yield Prediction
- Both benchmark datasets predict soybean yield at county level in the US corn-belt region, using remote sensing imagery or tabular weather and soil data.The remote-sensing dataset covers 11 states and uses sequences of 30 temporally spaced multispectral satellite images per county.Source: §3 · p. 5, §3.1 · p. 5
- For remote sensing, the dataset is split by US state into 11 silos, with image sequences mapped to average observed yield.The images are converted into per-band pixel-count histograms and concatenated as network inputs.Source: §3.1 · p. 5
3.2. Tabular Yield Prediction
- The tabular soybean-yield dataset covers nine US states and combines planting, weather, soil, and yield records organized by year and county.It spans 1980–2018 and is partitioned into nine state-level silos; the test year is 2018.Source: §3.2 · p. 6
4. Model Sharing
Model sharing lets participants train on separate data silos and combine model predictions without pooling raw data. The paper explores ensemble prediction, including geographic weighting, as a way to use knowledge from independent models.
4. Model Sharing
- Model sharing means each participant independently trains a model on its own data silo and distributes the trained model.The paper relates the concept to transfer learning and domain adaptation, while noting that fine-tuning many models can be impractical.Source: §4 · p. 7
- Ensemble predictors combine models trained on different participants’ data to leverage their features collectively for yield prediction.The paper motivates this approach as a way to form predictions under a holistic view of the data.Source: §4 · p. 7
- A simple ensemble averages all model predictions equally, so one poorly performing model is not the sole basis for a prediction.Each model is trained on a unique data subset with its own feature distribution.Source: §4 · p. 7
- Geographic weighting gives lower weight to predictions from models trained farther from the prediction location.The distance rank is shortest = 1 and longest = |K|; Figure 1 depicts local training followed by prediction aggregation.Source: §4 · p. 7, Figure 1 · p. 8
- Ensemble methods let participants use knowledge learned from independent models without sharing raw data, though model transfers can still raise privacy and trust concerns.The paper notes that inference attacks may extract raw data from shared models and considers differential privacy as a possible mitigation.Source: §4 · p. 8
5. Federated Learning
Cross-silo federated learning coordinates a shared model across a relatively small number of data silos while leaving raw data with participants. The paper discusses feature-distribution heterogeneity and reports that FedBN improves performance by retaining local batch-normalization parameters.
5. Federated Learning
- Federated learning coordinates a single model through a central server, sharing model updates rather than raw data.Clients train locally on their own data, return model weights for aggregation, and repeat this process across communication rounds.Source: §5 · pp. 8–9
- Cross-silo federated learning trains a shared model on data held by relatively few participants, typically organizations whose data are usually available.The paper distinguishes it from cross-device learning by participant quantity, data size, and availability.Source: §5 · p. 9
- A central communication server may create obstacles related to malicious information retrieval in the agri-food setting.The paper identifies data trusts and differential privacy as possible ways to address these concerns.Source: §5 · p. 10
5.1. Statistical Heterogeneity of Silos
- Feature shifts across silos can make local models misaligned and degrade the global model’s performance on the union of client data.The paper distinguishes covariate shift, where Pk(x) varies across clients, from concept shift, where Pk(x|y) varies while Pk(y) remains the same.Source: §5.1 · p. 11
- FedBN improves performance by excluding batch-normalization parameters from averaging and retaining them locally for each silo.The authors report empirical improvement in Table 3 and Figure 3; Table 3 compares aggregation methods by average RMSE across dataset modalities.Source: §5.1 · p. 11, Table 3 · p. 11
- FedBN assumes local models contain batch-normalization layers, and its suitability therefore depends on the model architecture.The paper’s setting satisfies this assumption because it replicates architectures with batch-normalization layers.Source: §5.1 · p. 11
- Methods for addressing statistical heterogeneity depend on the data and should be considered in practical federated-learning adoption.The paper also names FedProx and FedMA as approaches focused on label-distribution skew.Source: §5.1 · p. 11
6. Local Differential Privacy
The section places privacy protection at each client before model aggregation, avoiding reliance on a trusted server. This local approach trades predictive performance against privacy, with the trade-off adjustable through training and privacy settings.
6. Local Differential Privacy
- Differential privacy limits what an adversary can learn about original training data from model parameters by masking any individual user’s contribution.The formal definition considers a randomized mechanism on adjacent datasets and its possible outputs.Source: §6 · pp. 11–12
- Local differential privacy applies the randomized mechanism to each client’s dataset before its model leaves the client, unlike a central approach that trusts the server to apply it.This addresses the setting’s concern about trusting communication and the server.Source: §6 · p. 12
- DP-SGD adds noise locally, but the greater noise compared with a single server-side addition typically reduces performance and leads practitioners to use higher ε and δ.The authors aim to keep both ε and δ small for stronger privacy.Source: §6 · p. 12
- As the privacy budget increases and the privacy guarantee weakens, model performance increases in the authors’ experiments.The authors empirically choose the performance–privacy trade-off for their task and data.Source: §6 · pp. 12–13
- The privacy parameters are tuneable, and local differential privacy lets participants set their own privacy budgets and control their privacy–performance trade-off independently.The authors note that this flexibility suits agri-food tasks and goals that vary widely.Source: §6 · p. 13
7. Experimental Results
Across soybean yield prediction experiments, cross-silo federated learning outperformed local training and shared-model ensembles while performing close to pooled training. Model-sharing results varied with data modality and silo size, and differential-privacy performance depended on the selected privacy budget.
7.1. Model and Data Description
- The experiments predict county-level soybean yields from state-based silos using remote-sensing imagery and tabular weather, soil, and crop-management data.Traditional training pools the data; model-sharing ensembles and federated learning train from local state datasets.Source: §7.1 · p. 13, §7.1.1 · p. 14, §7.1.2 · p. 14
7.1.1. Imaging with remote sensing data
- The remote-sensing experiments replicated CNN and LSTM baselines, trained for up to 160 epochs, and evaluated seven prediction years using preceding years for training.Performance was measured as county-level RMSE in bushels per acre, averaged over three runs of three seeds and evaluated per state silo.Source: §7.1.1 · pp. 13–14
7.1.2. Tabular weather, soil and crop management data
- The tabular experiments adapted the CNN-RNN baseline and evaluated county-level predictions for 2016–2018, averaging RMSE over three runs of three seeds.The adaptation added batch normalization, removed MLP layers, and restricted CNN layers to a single time-step before the RNN.Source: §7.1.2 · p. 14
7.2. Model Sharing Performance
- Distance-weighted model-sharing ensembles reduced average RMSE by about 0.2 for image data and 1.5 for tabular data versus simple averaging.Across modalities, ensembles increased RMSE over local baselines by approximately 1.32% for image data and 9.09% for tabular data; tabular ensembles reduced local-test RMSE by 0.3.Source: §7.2 · pp. 14, 16
7.3. Cross-Silo Federated Learning Performance
- Federated learning aggregates local model updates into a global model evaluated on each silo’s test set, and it outperformed both local training and model-sharing ensembles.The authors report federated learning performed nearly identically to traditional training, with a 6.68% disparity.Source: §7.3 · p. 16
- Federated batch normalization was used to address statistical heterogeneity, and careful algorithm choice can improve performance and reduce time to convergence.The authors note that architecture, dataset, and task differences make these choices application-specific.Source: §7.3 · pp. 16–17
- Federated learning improved RMSE over local baselines by 22.75% for image data and 39.81% for tabular data, with a 6.68% disparity from traditional training.The comparison covers the CNN, LSTM, and CNN-RNN results reported across Tables 5–7.Source: §7.3 · p. 16, Table 5 · p. 17, Table 6 · p. 17
7.4. Differential Privacy Performance
- For remote sensing, federated LDP-SGD at ϵ = 8.0 and δ = 1 × 10−5 reduced RMSE by 19.85% (CNN) and 10.68% (LSTM); tabular federated learning at ϵ = 1.5 and δ = 1 × 10−7 reduced it by 19.76%.The remote-sensing ϵ was selected where the privacy-performance trade-off was roughly converging; the tabular ϵ was empirically selected as the initial point before convergence.Source: §7.4 · p. 18
8. Discussion
The discussion presents distributed learning as a way to perform soybean-yield optimization without raw-data exchange, with federated learning close to pooled baselines and better than local models. It also notes that coordination and participant understanding remain challenges for adoption.
Discussion
- The demonstrated soybean-yield task is one potential production-optimization use case; the authors also identify possible food-safety and sustainability applications tied to regulatory requirements.The discussion frames these as potential benefits of federations or consortia of independent actors.Source: §8 · pp. 18–19
- Across image and tabular datasets, federated learning achieved RMSE reductions of 5.32% and 6.68% from traditional baselines, respectively, and outperformed locally trained models.The discussion also reports that the shared-model ensemble was not ideal for remote imaging, attributing the deficit primarily to dataset size and over-parameterized models.Source: §8 · p. 19
- Privacy-preserving distributed training avoids raw-data sharing while allowing collaborative model training, and the authors note fully homomorphic encryption as another possible privacy measure.They describe privacy guarantees and protection against attempts to obtain training data from shared information as trust-relevant concerns.Source: §8 · p. 19
- Coordinating participant training and distributing trained models may still raise competitive-advantage concerns, and participant understanding is identified as a driver of changed perspectives.The authors distinguish these social and political adoption challenges from concerns about sharing raw data.Source: §8 · p. 19
9. Conclusion
The conclusion presents distributed machine-learning training as a response to commercial sensitivity and reluctance to share raw data in agri-food. It reports privacy guarantees and performance close to traditionally trained models, while identifying real-world implementation and remaining technical and social challenges as future work.
9. Conclusion
- Distributed machine-learning training is proposed to address commercial sensitivity and unwillingness to share raw data, barriers linked to transparency and mandatory requirements.The authors describe this as a way to overcome strong social barriers to data sharing.Source: §9 · p. 19
- The authors describe this as the first exploration of federated learning in the agri-food sector, with potential benefits for industry and consumers.Source: §9 · p. 20
- The proposed methods provide privacy guarantees and perform close to traditionally trained machine-learning counterparts.The authors support this conclusion with theoretical descriptions and empirical evidence.Source: §9 · p. 20
- Real-world implementation, statistical heterogeneity, social and political challenges, and a more concrete data-to-model pipeline remain future work.The proposed pipeline would address communication, central servers, and regulatory implications.Source: §9 · p. 20