Source-linked AI summary
An Accurate and Single-Communication Federated Inference Algorithm
Laura Montagnani, Anthony CC Coolen, Marianne A Jonker
TL;DR
Privacy rules and institutional policies limit sharing patient-level data, motivating a single-communication federated inference method that uses third-order approximations. The proposed estimator is designed to improve accuracy, with results close to pooled estimates.
Problem
Privacy regulations and institutional governance policies complicate sharing individual-level patient data across institutions, limiting collaborative analyses.
Method
The paper proposes a single-communication federated inference estimator based on a third-order approximation of local log-likelihood functions.
Results
The proposed method produces estimates very close to pooled estimates in the evaluated setting.
Takeaways & Limitations
Third-order federated inference supports collaborative analysis while retaining the accuracy target of pooled estimation.
Takeaways & Limitations
Iterative cyclic federated approaches can require additional complexity and costs.
Abstract
from arXiv · showhide
Joint analyses across multiple institutions are increasingly important in biomedical and epidemiological research, particularly for rare diseases where datasets are typical small. However, privacy regulations and institutional policies often prevent the sharing of individual-level patient data. In this paper we present an accurate and single-communication federated inference algorithm. Single-communication federated inference enables statistical analyses through a single exchange of summary statistics between participating centers and a coordinating server, preserving privacy while reducing communication and computational costs compared with iterative federated learning. We extend a recently proposed single-communication federated inference strategy that is based on second-order Taylor expansions by using third-order expansions to better approximate local log-likelihood functions. The proposed method is evaluated through simulation studies based on real data and compared with existing federated inference strategies. The simulation studies assess the performance of the proposed method, with a particular focus on scenarios involving small local sample sizes, where quadratic approximations may fail to capture skewness and other higher-order characteristics of the log-likelihood function. They demonstrate that incorporating higher-order information of the log-likelihood function improves the accuracy while preserving the privacy, communication efficiency, and scalability required for collaborative biomedical and epidemiological research.
Introduction
Multi-institutional biomedical research needs privacy-preserving joint analysis because rare-disease datasets are small and individual-level data cannot often be shared. This paper investigates third-order Taylor approximations to improve the accuracy of single-communication federated inference beyond quadratic methods.
- Introduction: Joint analyses can improve statistical accuracy and generalizability for rare diseases, but privacy regulations and institutional policies often prevent sharing individual-level patient data.
- Introduction: Cyclic federated approaches require multiple communication rounds, increasing computational, organizational, and coordination costs across participating centers.Efficiency is constrained by the slowest center, and adding a center requires further rounds involving all participants.
- Introduction: Single-communication methods instead combine summary statistics from participating centers while preserving patient privacy and reducing coordination requirements.
- Introduction: Small local samples can make quadratic approximations inadequate because skewness and other higher-order features may cause federated posteriors to deviate from pooled-data posteriors.The paper therefore explores third-order Taylor terms, producing cubic approximations intended to capture features ignored by the previous approach and improve single-communication federated inference.
- Introduction: The paper’s overarching goal is to determine whether higher-order approximations can produce more accurate single-communication federated inference.
Methods
The methods estimate a shared parameter without exchanging individual-level observations by aggregating local summary statistics, extending second-order federated inference with a third-order estimator. The approach also describes inference extensions, Bayesian translation, and optional second-round Newton–Raphson or third-order updates.
- Federated framework: Each center computes local analyses and sends selected summary statistics to a coordinating server, which combines them into an estimate approximating pooled-data inference.The framework assumes independent observations across centers and targets a finite-dimensional parameter θ.
- FI2 estimator: FI2 replaces each local log-likelihood with a second-order Taylor expansion around its local MLE and requires transmitting the local MLE and Hessian.The first-order term vanishes at the local MLE, and the estimator is justified when local MLEs lie near the pooled MLE.
- FI2 estimator: FI2 is asymptotically unbiased, Gaussian, and efficient as local sample sizes increase, implying no information loss from avoiding pooled individual-level data.The negative Hessian provides an estimate of the Fisher information matrix for inference.
- FI3 estimator: Because FI2 may be less accurate for small samples and fail to capture skewness, FI3 uses third-order Taylor expansions around local MLEs.Each center transmits its local MLE, Hessian, and third-order derivative tensor; the tensor has dimension d × d × d.
- Multi-round extensions: With an additional communication round, the server can use FI2 or FI3 as the initial value for a Newton–Raphson update, or construct FI3+ from a global third-order expansion.The latter requires centers to share gradients, Hessians, and third-order derivatives evaluated at FI3.
Simulation Studies
Simulation studies used pooled trauma-patient data to compare federated estimators against pooled-data estimates, assessing parameter-estimation and prediction accuracy. FI3 and FI3+ generally performed best, while ODAL2 was unstable, especially with smaller local centers, although additional communication reduced MSE.
- Simulation design: Outcomes were simulated from a homogeneous logistic-regression model using pooled-data parameter estimates as true values, then assigned to hospitals with fixed sample sizes.A second study split the data among more centers to evaluate multiple small-center settings.
- Evaluation criteria: Estimator accuracy was evaluated by mean square error relative to the pooled-data estimate, the gold standard for federated analysis.Prediction accuracy was assessed separately using a 90% training set and 10% test set.
- Parameter estimation: Among one-communication procedures, FI2 and FI3 clearly outperformed ODAL2 regardless of the lead center, while a second communication round reduced MSE further.ODAL2 performance depended on lead-center choice and improved with increasing lead-center sample size.
- Multiple small centers: Across configurations, FI3 outperformed FI2 and FI3+ outperformed FI2+, whereas ODAL2 showed considerable instability as the number of centers increased.Increasing the number of centers reduced local sample sizes and led to higher MSEs; concentrated observations could make ODAL2 outperform FI2 but not FI3 or FI2+.
- Prediction performance: The conclusions for predictions matched those for parameter estimation, while FI2 appeared less sensitive and prediction performance depended on the fitted linear predictor.The reported MSEθ,FI3 values were 0.104 (K = 6), 0.0475 (K = 8), 0.0866 (K = 10), and 0.1353 (K = 12) in Table 2, and 0.0027 in Table 1.
Discussion
The paper presents privacy-preserving federated inference algorithms requiring one or two communication rounds and applicable to any parametric model. Third-order Taylor approximations improve estimation accuracy by capturing local asymmetry overlooked by quadratic approximations, while extensions to heterogeneous and Bayesian settings remain future work.
- Contributions: The proposed privacy-preserving algorithms require one or two communication rounds and apply to any parametric model, with performance evaluated through logistic-regression simulations.The methods were developed for federated statistical inference and tested in simulation studies for logistic regression.
- Methodological advance: Third-order Taylor approximations improve estimation accuracy by incorporating information beyond the previously used second-order approximations.The third-order estimator extends one-shot methodology based on second-order approximations of local log-likelihood functions or Bayesian log-posteriors.
- Methodological advance: The proposed approach captures local asymmetry of the log-likelihood or log-posterior that quadratic approximations neglect, producing more accurate simulation estimates.This higher-order information is intended to improve estimation accuracy where quadratic approximations omit local asymmetry.
- Limitations and extensions: The methodology was developed under a homogeneous shared-model assumption but can be extended to heterogeneous settings analogously to a Bayesian Federated Inference framework.Extending ODAL2 to heterogeneous settings is described as less straightforward because its initial parameter estimate is computed at a designated lead center and communicated to other sites.
- Future work: Future work will translate the methodology to a Bayesian setting and implement it in the existing R package BFI for generalized linear and parametric survival models.The package was developed for Bayesian Federated Inference methodology described in the cited prior work.
Conclusions
Including the third-order Taylor term appears to improve aggregated-estimator accuracy, demonstrated for logistic regression across multiple settings. The methodology also naturally extends to Bayesian inference and heterogeneous federated learning settings.
- Including the third-order Taylor term in the log-likelihood approximation improves the estimation accuracy of the aggregated estimator.
- This accuracy improvement has been demonstrated for logistic regression models in multiple settings.
- The methodology naturally extends to Bayesian inference and can be generalized to heterogeneous federated learning settings.