PaperlayerView exact revision on arXiv

Source-linked AI summary

Can AI Agents Interface Clinicians and Complex Models?

Giulio Colombini, Matteo Falcioni, Mario Massimo, Francesco Durazzi, Marianna Santoro, Paolo Tubertini, Mirko Degli Esposti, Armando Bazzani

arXiv:2610.39681v1cs.MAcs.AI

TL;DR

Clinicians and decision-makers face a steep learning curve in using predictive models developed in mathematical biology and related fields. EpidemIA is a training-free conversational multi-agent system that connects natural-language requests to data analysis and expert-written epidemiological models. In example use cases, it performed statistical analyses, visualizations, and dynamical-model predictions, while remaining a proof of concept.

  • Problem

    Clinicians and decision-makers face a methodological divide from model developers, and mathematical-model concepts are not a main focus of their training.

    Source: §1 · p. 2

  • Method

    EpidemIA is a training-free conversational MAS that routes natural-language requests to analysis and forecasting tools using expert-written models.

    Source: §1 · p. 3, §2.1 · pp. 5–6

  • Results

    EpidemIA demonstrated natural-language use of data analysis, visualizations, and epidemiological predictions, including a hypothetical influenza scenario with a 5% higher R0.

    Source: §3 · p. 8, §4 · pp. 8–9

  • Takeaways & Limitations

    EpidemIA is a proof-of-concept knowledge tool intended to help clinicians, decision-makers, and model researchers extract relevant information and combine their efforts.

    Source: §4 · p. 9

    Whole-system evaluation is constrained because reproducible testing requires a publicly available underlying dataset, which was unavailable for the data used.

    Source: §4 · p. 10

Abstract

from arXiv · show

Background: Contemporary healthcare institutions generate daily very large amounts of data, which contain highly useful information for the improvement of work organization, planning and overall efficiency. While models and techniques to extract these insights are well known and constantly developed in mathematical biology, complex systems physics and data science, the adoption of these techniques among Clinicians and Hospital Administrators remains limited, mainly due to language and methodological divides. Objective: In this article we implement an AI Multi Agent System (MAS) that allows Users to use predictive models and make statistical analyses via natural language conversations. Methods: We developed EpidemIA, a training-free MAS endowed with a modular suite of epidemiological models written by human experts, executable on a single-GPU workstation. To avoid relaying sensitive data to third parties, the system uses locally an open-weights LLM as backbone for its two subagents. Results: Using a dataset of Emergency Department visits provided to us by IRCCS Sant'Orsola (Bologna) we test the model on a few typical use cases, demonstrating the possibility of performing both simpler statistical analyses and visualizations as well as more complicated epidemiological predictions based on dynamical models, through simple queries made in natural language. Conclusions: While still being a proof-of-concept implementation, EpidemIA showcases the transformative capabilities of LLM based technologies as an enabling factor in the widespread adoption of complex dynamical models, which is at present hindered by their steep learning curve.

1 Introduction

Healthcare data analysis spans clinical, organizational, and population scales, but clinicians and decision-makers face a steep learning curve for predictive models. The introduction presents AI agents and multi-agent systems as a way to make complex analyses accessible through natural-language interaction.

1 Introduction

  • Dynamical epidemiological models support scenario simulation, but their use requires coding, rigorous data processing, and analytical expertise.Models can represent transmission and richer dynamics such as mobility, vector-borne disease, or distributed stage durations.Source: §1 · pp. 2–3
  • Clinicians and decision-makers face a methodological divide from model developers, since mathematical biology and complex-systems concepts are not a main focus of their training.This creates a steep learning curve for using predictive models.Source: §1 · p. 2
  • LLMs offer intuitive interaction but lack the structure for numerical or analytical tasks and cannot independently process external files.Agents extend LLM workflows with external tool calls, which can be iterated to handle complex tasks.Source: §1 · p. 2
  • Prior healthcare MAS applications have focused mainly on single-patient settings or organizing public-health data, with limited integration of clinical data access and data-informed forecasting.EpidemIA adopts MAS task decomposition for flexible sequences of analysis, simulation, visualization, and reporting.Source: §1 · p. 3

2 Methods

EpidemIA is a training-free conversational multi-agent system that connects users to data-analysis and epidemiological-modeling tools. Its design separates user interaction from computational work, uses expert-written modular models, and supports local deployment with an open-weight LLM.

2 Methods

  • EpidemIA is a training-free conversational MAS with two LLM-based subagents that exchange messages, share state, and interact with the file system.Its purpose is to help healthcare and organizational professionals use predictive models and data-science techniques.Source: §2 · p. 4

2.1 Agents and tools

  • The Supervisor handles user interaction and delegates computational requests to the Analyst, which explores and filters data before using analysis or prediction tools.The Analyst can run autonomous Python analyses, while predictive-model code is expert-written and cannot be modified by the agent; the Supervisor has no raw-data or computational access.Source: §2.1 · pp. 4–6, §2 · p. 4

2.2 Modular epidemiological model system

  • Expert-written epidemiological models and fitting metrics use modular interfaces, allowing the Analyst to discover available options and fit forecasts to observed data.The fit-and-forecast tool returns predicted incidence and fitted parameter values; the system includes SIR and SEIR models.Source: §2.2 · pp. 6–7, §2.1 · p. 6

2.3 Implementation details

  • The agents use a locally served open-weight LLM to keep sensitive documents within the infrastructure and avoid dependence on proprietary endpoints.The implementation uses Qwen3.8:27B on one RTX A6000 GPU; the architecture can use another instruction-following LLM with structured tool calling.Source: §2.3 · p. 7

3 Results

Four use cases on Sant’Orsola Emergency Department data illustrate statistical analysis, flu forecasting with SIR and SEIR, and a hypothetical change in R0.

3 Results

  • The system analyzed red-code Emergency Department visits for a specified month and produced a histogram.The data were filtered by emergency code and month before visualization.Source: §3 · p. 8
  • For generic daily flu forecasting, the Supervisor defaulted to SIR and the Analyst produced a forecast visualization.Retrospective observations after the forecast start were shown for comparison, although they would be unavailable in a real forecast.Source: §3 · p. 8
  • For a user-requested flu forecast, the Analyst used SEIR and generated a visualization.The workflow otherwise followed the generic forecasting case, with a different model and forecasting period.Source: §3 · p. 8
  • A follow-up scenario raised the data-inferred R0 by 5% and generated a new prediction alongside the observed data and previous prediction.The Analyst reused the SEIR case output, recalibrated parameters, and plotted the three curves.Source: §3 · p. 8

4 Discussion

EpidemIA is a training-free conversational MAS that gives non-specialists access to statistical analyses and predictive models through a natural-language interface. The authors describe it as a proof of concept, note limits to whole-system evaluation, and outline extensions to its models, data access, and evaluation.

4 Discussion

  • EpidemIA integrates expert-developed models and techniques with an intuitive interface as a knowledge tool, not a replacement for clinicians, decision-makers, or researchers.The system generates tool-call sequences from plain-English questions and returns reports with descriptive statistics, plots, and plain-language explanations; processing requires neither internet access nor external LLM APIs.Source: §4 · pp. 8–9
  • EpidemIA-like systems could help clinicians access data and support operational planning, while helping researchers diffuse and field-test modular models without directly handing off sensitive data.The authors say the system could reduce time spent on interpretative and implementation issues, without removing the need for greater interdisciplinarity.Source: §4 · pp. 9–10
  • Whole-system evaluation is the study’s main limitation: reproducible testing depends on a publicly available underlying dataset, which was unavailable for this work.The authors also identify model-agnostic procedural generation of question-answer pairs as a challenge and say hand compilation is time-consuming and data- and model-dependent.Source: §4 · p. 10

4.1 Next Steps

  • The authors propose extending predictive tools beyond epidemiology, including environmental and climate data that could inform forecasts of health hazards such as heatwaves.They suggest adding tools or agents to the modular MAS, with the potential to give emergency-department decision-makers lead time for staff and bed-capacity planning.Source: §4.1 · p. 10
  • Possible architecture extensions include a more capable LLM backbone or specialized agents for SQL queries against large hospital databases.Direct database access would require further privacy and data-preservation safeguards, such as local sandboxing and code security checks.Source: §4.1 · p. 10
  • For system-level evaluation, the authors identify LLM-as-a-judge as a promising approach, conditional on a practical way to generate question-answer pairs.They suggest independent LLM instances could help handle generic data formats.Source: §4.1 · p. 10

CRediT authorship contribution statement

The contribution statement assigns the project’s conceptualization, methods, software, analysis, investigation, writing, visualization, supervision, administration, and funding roles.

CRediT authorship contribution statement

A Full MAS answers for the examples

The section directs readers to the complete agent responses and the visualizations for the four use cases.

A Full MAS answers for the examples

Use case (a): Standard statistical analysis

The example reports descriptive statistics for daily red-code emergency-room accesses from December 2025 through January 2026, including their distribution and variation by day of week.

Use case (a): Standard statistical analysis

  • 560 red-code accesses across 62 days averaged 9.03 per day, with a median of 9, a standard deviation of 2.80, and an interquartile range of 7.25–11.0.The minimum was 2 and the maximum was 17.Source: Use case (a): Standard statistical… · p. 16
  • Weekday red-code accesses averaged about 9.1 per day, compared with about 8.1 on weekends.Thursday and Monday had the highest daily means (10.0 and 9.8); Sunday and Saturday had the lowest (8.0 and 8.25).Source: Use case (a): Standard statistical… · p. 16
  • The report describes daily access volume as relatively stable, with no dramatic spikes or sustained trends during the period.The maximum of 17 occurred on January 3, 2026, and the minimum of 2 on December 13, 2025.Source: Use case (a): Standard statistical… · p. 16

Use case (b): SIR model prediction

The SIR model forecast a declining flu-incidence trend for January–March 2026, but underestimated the observed early-January peak. The use case also produced fitted parameters and plots comparing predictions with real data.

Use case (b): SIR model prediction

  • The fitted SIR parameters were β = 0.698, μ = 0.569, I0 = 500, baseline = 2.315, and R0 = 1.226, with RMSE = 2.347.The detection rate f = 0.005 and population N = 400,000 were fixed.Source: Use case (b): SIR model prediction · p. 16
  • The SIR model predicted 13.6 cases/day on 1 January, versus 29 observed, then projected a decline to 2.3 cases/day by the end of March.Real data available through 8 March showed 0–5 cases/day by early March; the model captured the overall downward trend but underestimated the peak.Source: Use case (b): SIR model prediction · pp. 16–17
  • The system produced full-range and November–April zoomed plots overlaying the SIR prediction on real data.Source: Use case (b): SIR model prediction · p. 17

Use case (c): SEIR model prediction

The SEIR use case generated forecasts and visualizations for flu incidence, with a declining trend broadly consistent with observed data. A hypothetical 5% increase in R0 produced a higher, earlier peak and a concentrated hospital-capacity planning impact.

Use case (c): SEIR model prediction

  • The fitted SEIR parameters were β = 1.148, γ = 0.664, μ = 0.886, E0 = 800, I0 = 800, and R0 = 1.296, with RMSE = 2.16.The baseline was 0.0; detection fraction f = 0.005 and population N = 400,000 were fixed.Source: Use case (c): SEIR model prediction · p. 17
  • The SEIR model predicted a peak of about 24 cases/day around mid-December 2025, declining to below 0.05 cases/day by the end of March 2026.Observed data through 8 March showed 411 positive cases in the forecast window, a high of 29 on 1 January, and 0–5 cases/day by early March.Source: Use case (c): SEIR model prediction · p. 17
  • The system produced full-range and zoomed visualizations overlaying SEIR predictions on real data for comparison.Source: Use case (c): SEIR model prediction · p. 17
  • A 5% increase in R0 raised the peak by about 33% and shifted it 2 days earlier, with hospital capacity planning needing to account for roughly 32 additional cases/day during the mid-December peak week.By January 2026, the higher-transmission scenario showed slightly fewer cases as the epidemic depleted its susceptible pool faster.Source: Use case (c): SEIR model prediction · p. 18

Implementation patterns for models and metrics

The implementation patterns define how model modules expose fitting, simulation, and parameter information, and how metrics evaluate predicted incidence. Together, these interfaces support fitting models to observed data and generating dated incidence predictions.

Predictive models

  • Model definitions include parameter explanations, defaults, and fitting bounds; those bounds are treated as internal and need no separate interface for fitting and forecasting.Source: Predictive models · p. 19
  • Predictive models fit observed incidence data by minimizing a supplied metric, with optional caller-provided initial guesses and fixed parameter values.Defaults are used when custom initial guesses or fixed values are not supplied.Source: Predictive models · p. 19
  • The fit result records optimization status, the minimum metric value, fitted and fixed parameters, the gradient, and the model’s R0.R0 is computed using the fitted parameter values.Source: Predictive models · pp. 19–20
  • The model interface simulates daily incidence between requested dates from a parameter dictionary and returns the incidence values with their corresponding dates.At simulation time, fitted and a priori fixed parameters are not distinguished.Source: Predictive models · p. 20

Metrics

  • A metric function evaluates an incidence-predicting model against data using its parameters and simulation time span.The minimizer supplies fitted parameters, fixed parameters, the model function, and observed data to the metric.Source: Metrics · pp. 20–21
Loading 2610.39681v1…