PaperlayerView exact revision on arXiv

Source-linked AI summary

FedEHR-Agents: Federated Agentic Optimization for Automated EHR Modeling

Jun Bai, Ruilin Wang, Yue Li

arXiv:2608.27856v1cs.LGcs.AIcs.MA

TL;DR

Hospital-specific data and modeling environments constrain autonomous EHR agents, while conventional FL primarily shares prediction-model parameters rather than the modeling experience behind them. FedEHR-Agents federates refined prompts and structured evidence through evidence-guided aggregation and global meta-prompts; experiments report consistent gains across diverse tasks and robustness across federation scales and LLM backbones. The paper positions clinical modeling experience as a collaborative object beyond parameter-centric FL, while noting that experience sharing is privacy-aware rather than a formal guarantee against leakage.

  • Problem

    Hospital heterogeneity limits independently deployed agents, and conventional FL largely excludes the preprocessing, feature engineering, evaluation, and refinement experience behind clinical models.

  • Method

    FedEHR-Agents uses autonomous local EHR agents, historical memory, evaluation, TextGrad refinement, evidence-guided experience aggregation, and global meta-prompts.

  • Results

    Experiments on real-world multi-hospital benchmarks show effectiveness across diverse clinical prediction tasks and robustness across federation scales and LLM backbones.

  • Takeaways & Limitations

    Clinical modeling experience is a promising collaborative object beyond conventional parameter-centric federated learning.

  • Takeaways & Limitations

    Experience representations may indirectly encode local data characteristics, so experience sharing is privacy-aware rather than a formal guarantee against privacy leakage.

Abstract

from arXiv · show

Recent advances in large language models are enabling autonomous clinical agents to perform increasingly complex electronic health record (EHR) modeling workflows. However, agents deployed at individual hospitals remain constrained by institution-specific data and modeling environments, while direct cross-hospital collaboration is restricted by the sensitivity of patient-level EHR data. Although federated learning (FL) provides a natural foundation for privacy-preserving collaboration, existing approaches remain predominantly model-centric, limiting federation to prediction models or their updates while overlooking the richer modeling experience accumulated by autonomous agents. To address this limitation, we propose FedEHR-Agents, an experience-centric federated agentic optimization framework for automated EHR modeling. Each hospital deploys an autonomous clinical EHR agent that performs data preprocessing and model development while refining local clinical modeling experience through historical memory, task-specific evaluation, and TextGrad-based prompt refinement. The federated server performs evidence-guided experience aggregation to integrate reliable and complementary modeling experience across heterogeneous hospitals and distills the aggregated experience into global meta-prompts for subsequent local refinement. Extensive experiments on real-world multi-hospital EHR benchmarks demonstrate that FedEHR-Agents consistently outperforms local and federated baselines across diverse clinical prediction tasks and remains robust across different federation scales and LLM backbones. These results establish clinical modeling experience as a promising collaborative object beyond conventional parameter-centric FL and point toward federated autonomous clinical intelligence.

1 Introduction

FedEHR-Agents addresses the limits of hospital-isolated, model-centric federated learning by federating clinical modeling experience accumulated by autonomous EHR agents. The framework combines local refinement with evidence-guided aggregation and global meta-prompts, and experiments support its effectiveness across hospitals and clinical tasks.

  • Motivation: EHR agents can automate complex, multi-stage clinical modeling workflows beyond isolated prediction models.LLM agents use reasoning, planning, memory, reflection, and tool use to support autonomous modeling processes.
  • Motivation: Institutional heterogeneity limits independently deployed agents, while complementary preprocessing, feature engineering, and model-development experience could improve cross-hospital modeling.Hospitals differ in patient populations, workflows, measurements, and documentation systems.
  • Motivation: Conventional FL aggregates prediction-model parameters but leaves preprocessing, feature engineering, evaluation, and iterative refinement outside the federation loop.This model-centric paradigm captures final models rather than the broader decisions that produce them.
  • Motivation: Clinical modeling experience contains reusable strategies, decisions, feedback, and refinement traces that can be transferred without exchanging patient-level EHR data.Such experience is accumulated through reasoning-driven modeling decisions across heterogeneous hospitals.
  • FedEHR-Agents: FedEHR-Agents refines local agent experience, aggregates refined prompts and structured evidence, and distills them into global meta-prompts for subsequent execution.Local refinement uses historical memory, task-specific evaluation, and TextGrad-based prompt refinement.
  • Results: Experiments on real-world multi-hospital benchmarks show effectiveness across diverse prediction tasks and robustness across federation scales and LLM backbones.The results support transferable cross-hospital knowledge sharing beyond conventional model-centric federation.

2 Problem Formulation

The formulation models each hospital as an autonomous agent that locally develops EHR models while maintaining evolving clinical modeling experience. Federated rounds aggregate shareable experience, generate global guidance, and integrate it into subsequent local refinement to maximize task-specific utility.

  • 2.1 Federated Clinical Agentic Setting: Each hospital holds private longitudinal EHR data for a shared clinical prediction task while keeping patient-level records local.The setting includes static and temporal clinical observations across K distributed hospitals.
  • 2.1 Federated Clinical Agentic Setting: Unlike conventional FL, each hospital is represented by an autonomous clinical agent that performs the complete local EHR modeling workflow.The workflow includes preprocessing, feature engineering, model development, evaluation, and iterative refinement.
  • 2.1 Federated Clinical Agentic Setting: Local experience is refined across communication rounds through execution feedback, historical memory, and global meta-prompts while prediction models may remain heterogeneous.This preserves hospital-specific modeling experience while enabling cross-hospital benefit.
  • 2.2 Clinical Modeling Experience: Clinical modeling experience is the federated optimization object, encompassing preprocessing, feature engineering, model decisions, execution feedback, and historical optimization traces.It is an outcome of local reasoning and execution rather than a prediction-model parameter update.
  • 2.2 Clinical Modeling Experience: Experience comprises historical memory, structured modeling evidence, executable prompts, and preserved execution and refinement information.Only refined prompts and structured modeling evidence are communicated during aggregation; the full experience remains local.
  • 2.3 Federated Experience Optimization: Federated experience optimization iteratively refines local experience, aggregates shareable representations, generates global meta-prompts, and integrates them into local states.This replaces direct prediction-model optimization with experience-centered collaboration.
  • 2.3 Federated Experience Optimization: The server synthesizes meta-prompts that distill transferable preprocessing, feature-engineering, model-development, and optimization guidance.The generated guidance is derived from the globally aggregated experience representation.
  • 2.3 Federated Experience Optimization: The overall objective maximizes task-specific utility induced by experience across hospitals, measured by workflow quality and downstream predictive performance.Utility is optimized through local refinement, aggregation, meta-prompt generation, and experience integration.

3 Methodology

FedEHR-Agents uses autonomous hospital agents to perform end-to-end EHR modeling while federating clinical modeling experience rather than prediction-model parameters. Local execution, evaluation, memory, prompt refinement, evidence-guided aggregation, and global meta-prompts support iterative cross-hospital improvement.

  • Framework overview: Each hospital deploys an autonomous clinical EHR agent with data-preprocessing and model-development modules over private EHR data.The framework keeps patient-level data local while coordinating experience across hospitals.
  • Data preprocessing: The DP module generates and locally executes LLM-guided code for extraction, cleaning, temporal alignment, feature engineering, and feature selection, repairing errors when needed.Its resulting feature representation and selected-feature statistics support downstream modeling and later aggregation.
  • Model development: The MD module autonomously selects modeling strategies, training configurations, and hyperparameters, then generates, executes, and repairs model-development code locally.It produces prediction models together with validation performance, feature importance, model preferences, training statistics, and execution records.
  • Local refinement: Historical memory stores prior artifacts, execution logs, evaluation results, and refinement histories so agents can reuse effective decisions across communication rounds.The memory remains local and informs task-specific evaluation and subsequent prompt refinement.
  • Local refinement: TextGrad converts evaluator feedback and historical context into revised executable DP and MD prompts that incorporate newly acquired modeling experience.Prompts function as executable representations of clinical modeling experience rather than fixed manual instructions.
  • Federated aggregation: EGEA combines refined prompts with structured modeling evidence, assesses experience reliability and transferability, and distills complementary hospital experience into global DP and MD meta-prompts.The server broadcasts these meta-prompts to guide subsequent local experience refinement.

4 Evaluation

FedEHR-Agents is evaluated on four multi-hospital clinical prediction tasks using privacy-preserving experience aggregation, and it consistently outperforms independent and federated baselines. Additional analyses show improved feature efficiency, cross-hospital consistency, interpretability, component complementarity, scalability, and robustness across LLM backbones.

  • Experimental Setup: Experiments use the multi-center eICU dataset, treating hospitals as federated clients across Mortality-48h, ARF-4h, LOS>7d, and Sepsis tasks.Patient-level EHR data remain local throughout optimization; experiments generally use three hospitals and five communication rounds.
  • Overall Performance: FedEHR-Agents consistently achieves the best performance among decentralized methods across all four clinical prediction tasks.Compared with Local Agent, AUPRC rises from 0.218 to 0.236 on Mortality-48h, 0.171 to 0.194 on ARF-4h, 0.483 to 0.532 on LOS>7d, and 0.268 to 0.316 on Sepsis.
  • Overall Performance: Structured evidence aggregation improves performance over PromptAvg, with absolute AUPRC gains of 0.015, 0.011, 0.026, and 0.030 across the four tasks.The results indicate that structured modeling evidence provides more effective guidance than prompt aggregation alone.
  • Federated Experience Optimization: Performance improves rapidly early and stabilizes later while selected feature counts decline, indicating increasingly compact feature subsets without reduced predictive performance.The trajectory suggests progressively more informative feature selection during federated optimization.
  • Federated Experience Optimization: Cross-hospital Jaccard and SHAP similarities increase across communication rounds, indicating greater agreement in feature selection and feature importance.This pattern supports the emergence of more consistent and reusable modeling knowledge across distributed hospitals.
  • Interpretability: Shared mortality predictors remain clinically meaningful across hospitals, while their SHAP distributions differ with local populations, disease prevalence, and clinical practice.The analysis reports renal, metabolic, inflammatory, and respiratory-related predictors alongside hospital-specific modeling characteristics.
  • Ablation Study: Removing any component reduces predictive performance; disabling the DP Module lowers AUPRC from 0.236 to 0.219, the largest degradation.Replacing EGEA with prompt averaging lowers AUPRC to 0.221, while removing Historical Memory and the Evaluator lowers it to 0.223 and 0.226.
  • Scalability Analysis: Average AUPRC increases from 0.236 to 0.252 as federation size grows from 3 to 20 hospitals, while FedEHR-Agents remains above PromptAvg.The corresponding PromptAvg increase is 0.221 to 0.241, and scalability trends also appear across other tasks.

5 Discussion and Future Work

FedEHR-Agents preserves patient data locality while sharing refined prompts and structured modeling evidence across hospitals. The paper identifies privacy leakage risks and proposes future work on formal privacy, multimodal capability, and long-term deployment evaluation.

  • Discussion: Patient-level EHR records and locally developed prediction models remain within originating hospitals, while collaboration uses experience representations and local historical memory.Shared representations include refined prompts and structured modeling evidence.
  • Limitations: Experience representations may indirectly encode local data characteristics, so experience sharing is privacy-aware rather than a formal guarantee against privacy leakage.Potentially revealing signals include feature importances, validation statistics, and optimization patterns under strong inference or reconstruction attacks.
  • Future Work: Future work will integrate formal privacy protections, including differential privacy, secure aggregation, trusted execution environments, and locally deployed LLMs.These directions target experience-level updates and server-side aggregation while reducing external service exposure.
  • Future Work: Future research will extend the framework to multimodal clinical data and adaptive experience selection, aggregation, and continual evolution across heterogeneous institutions.The proposed modalities combine structured EHRs, medical imaging, and clinical text.
  • Future Work: Long-term evaluations are needed to assess sustained reliability and safety in real-world multi-agent collaboration and continual learning.The paper identifies deployment-oriented assessment as a future research direction.

Appendix

The appendix algorithm operationalizes FedEHR-Agents through repeated local module execution, TextGrad refinement, experience construction and aggregation, and global meta-prompt integration. Its stated objective is to return refined clinical modeling experience and local prediction models.

  • Algorithm 1: The algorithm returns refined clinical modeling experience together with local prediction models.The stated guarantee identifies both outputs.
  • Algorithm 1: Across communication rounds, hospitals run the DP and MD modules in parallel.The algorithm specifies parallel execution for all hospitals within each round.
  • Algorithm 1: The MD prompt is refined via TextGrad before each hospital builds its local MD experience representation.These are separate algorithmic steps in the local workflow.
  • Algorithm 1: The server aggregates MD experience and generates global DP and MD meta-prompts.The algorithm then makes those global meta-prompts available for subsequent hospital updates.
  • Algorithm 1: Hospitals integrate the global meta-prompts during later parallel processing.This closes the repeated federated optimization loop across communication rounds.

B.1 Scalability Across Additional Clinical Tasks

Across ARF-4h, LOS>7d, and Sepsis, larger federations generally improve performance, while FedEHR-Agents consistently outperforms its PromptAvg variant at every federation scale. The appendix reports average AUPRC across 3 to 20 participating hospitals.

  • B.1 Scalability Across Additional Clinical Tasks: FedEHR-Agents is evaluated against PromptAvg with average AUPRC reported as participating hospitals increase from 3 to 20.The analysis covers ARF-4h, LOS>7d, and Sepsis under the same experimental setting.
  • B.1 Scalability Across Additional Clinical Tasks: FedEHR-Agents AUPRC rises from 0.194 to 0.236 on ARF-4h, from 0.532 to 0.553 on LOS>7d, and from 0.316 to 0.341 on Sepsis as hospitals increase from 3 to 20.These are the reported endpoint values for the three tasks.
  • B.1 Scalability Across Additional Clinical Tasks: At 20 hospitals, FedEHR-Agents exceeds PromptAvg by 0.022, 0.020, and 0.023 AUPRC on ARF-4h, LOS>7d, and Sepsis, respectively.The comparison is reported for the largest federation scale.
  • B.1 Scalability Across Additional Clinical Tasks: Both methods generally benefit from increasing hospital participation across all three clinical prediction tasks.The reported trend is consistent with the Mortality-48h findings.

B.2 Ablation Across Additional Clinical Tasks

Ablations across ARF-4h, LOS>7d, and Sepsis show that every component contributes to the complete framework. The dominant component varies by task, with DP most influential on ARF-4h and EGEA on LOS>7d and Sepsis.

  • B.2 Ablation Across Additional Clinical Tasks: Removing any individual component consistently degrades FedEHR-Agents performance across ARF-4h, LOS>7d, and Sepsis.The complete framework achieves the best performance on all evaluated tasks.
  • B.2 Ablation Across Additional Clinical Tasks: Removing the DP Module decreases ARF-4h performance from 0.194 to 0.176, its largest reported task-specific impact.The DP Module contributes most strongly to ARF-4h.
  • B.2 Ablation Across Additional Clinical Tasks: Removing EGEA reduces performance from 0.532 to 0.506 on LOS>7d and from 0.316 to 0.286 on Sepsis.EGEA has the largest impact on those two tasks.
  • B.2 Ablation Across Additional Clinical Tasks: Removing the Evaluator causes consistent degradation across all three tasks, while Memory provides smaller but stable improvements.The ablation identifies complementary contributions across clinical prediction settings.

B.3 Robustness to Different LLM Backbones

FedEHR-Agents remains robust across GPT-4o, GPT-5 mini, and GPT-5, consistently outperforming FedEHR-Agents (PromptAvg) across the evaluated backbones.

  • FedEHR-Agents consistently outperforms FedEHR-Agents (PromptAvg) across GPT-4o, GPT-5 mini, and GPT-5.All other experimental settings are kept unchanged, and results are averaged across three participating hospitals.

C Dataset and Feature Configuration

The study uses hospital-partitioned eICU data and standardizes heterogeneous clinical tables into longitudinal event representations before agent-based feature construction. Initial feature spaces remain hospital-specific in size and composition.

  • Each participating hospital is treated as an independent federated client in experiments using the eICU Collaborative Research Database.The dataset includes demographics, intake/output records, laboratory measurements, medications, respiratory-care records, and vital signs.
  • Heterogeneous eICU tables are transformed into unified events (ID, t, variable_name, variable_value) while preserving temporal information.The standardized format supports temporal alignment, aggregation, missing-value handling, and feature selection across sources.
  • At communication round r, the DP module independently processes each standardized clinical source and constructs task-specific representations.Source-specific representations are subsequently aligned and concatenated along the feature dimension.
  • Hospitals 73, 264, and 420 contain 2,663, 3,006, and 2,636 patient samples, with initial feature spaces of 112, 98, and 108 features, respectively.The resulting feature dimensions are allowed to differ across hospitals.
  • Table C1 reports initial feature counts retained from different clinical sources for three representative hospitals on Mortality-48h.Nk denotes the number of patient samples.
  • Source-level feature composition varies across hospitals because of differences in available variables, measurement frequency, missingness, and recording practices.Laboratory measurements typically contribute many features, while medications, respiratory-care variables, vital signs, and demographics provide complementary information.

C.1 Clinical Feature Definitions for Mortality-48h

The Mortality-48h SHAP analysis uses standardized feature names mapped to clinical meanings and eICU source variables. The reported features cover multiple physiological and organ-support domains.

  • Most reported features map directly to laboratory, demographic, or vital-sign variables, while some are constructed during preprocessing.Feature names displayed in Fig. 5 are standardized names and may differ from original eICU naming conventions.
  • Table C2 defines the clinical features reported in the Mortality-48h SHAP analysis and maps them to eICU variables.The mappings provide clinical context for interpreting hospital-level SHAP distributions.
  • The feature definitions span renal, metabolic, acid–base, inflammatory, electrolyte, cardiovascular, respiratory, hematological, and organ-support dimensions.These definitions contextualize the hospital-level SHAP distributions in Fig. 5.

D Prompt Templates and Structured Outputs

FedEHR-Agents uses module-specific prompts and structured outputs to coordinate local preprocessing, modeling, evaluation, refinement, and federated experience aggregation. The server converts evidence-supported experience into transferable meta-prompts for later rounds.

  • D Prompt Templates and Structured Outputs: The framework uses module-specific prompts for data preprocessing, model development, local evaluation and refinement, and server-side experience aggregation.Templates specify roles, inputs, constraints, and expected outputs, with task- and hospital-specific information inserted dynamically.
  • D Prompt Templates and Structured Outputs: Representative structured outputs organize execution results, evaluator feedback, and modeling evidence for local prompt refinement and federated experience aggregation.Information communicated outside the local execution environment is restricted to modeling-related representations.
  • D.1 Data Preprocessing Prompts: The DP Module separates a general system prompt from source-specific instructions so preprocessing adapts to heterogeneous EHR sources.The general prompt defines common preprocessing requirements, while source-specific instructions guide feature construction and aggregation.
  • D.1 Data Preprocessing Prompts: The preprocessing prompt generates one executable Python program for a leakage-safe temporal representation within the specified prediction window.Inputs include task, target outcome, observation window, temporal resolution, source paths, and global or source-specific guidance.
  • D.1 Data Preprocessing Prompts: The standardized longitudinal input uses ID, t, variable_name, and variable_value, with temporal preprocessing preserving patient ordering and excluding target labels.Hourly representations use hour_index = t // 60, and patients without records receive an appropriate empty or default representation.
  • D.1 Data Preprocessing Prompts: Source-aware preprocessing constructs compact, interpretable features while handling repeated, missing, and irregular observations with source-appropriate strategies.The prompt also requires clinically meaningful temporal information and avoids excessive, unstable, or redundant feature generation.
  • D.1 Data Preprocessing Prompts: Data-leakage constraints prohibit using patient-level labels, future observations, or features that directly or indirectly encode the target outcome.The output has shape (N,T,Dsource), and saved tensor, flattened representation, and feature-name ordering must remain consistent.
  • D.4 Federated Experience Aggregation Prompts: EGEA integrates module-specific experiences according to supporting evidence and converts them into concise global meta-prompts for the next communication round.The meta-prompts provide transferable guidance without directly modifying a hospital’s local prompt.
Loading 2608.27856v1…