Driving Epidemic Models with AI Agents: the Epydemix Agent Framework

1 Introduction

• Library-based epidemic modeling requires programming expertise, while graphical interfaces remove that requirement but constrain what users can specify and simulate.AI agents offer a natural-language interaction mode in which researchers can configure models, compare interventions, and interpret results without writing code for each analysis.Source: §1 · pp. 1–2
• Agents cannot reliably preserve Epydemix’s in-memory objects between tool calls or discover its functionality and extract summary measures without custom code.They may also specify plausible but invalid parameters or reference unsupported functions, creating risks when outputs inform intervention choices.Source: §1 · p. 2
• The framework adds discoverability, validated declarative scenarios, tested and reproducible execution, and inspectable simulation summaries to the Epydemix API.These capabilities let agents find supported models and parameters and retrieve needed quantities without reading source code or parsing raw outputs.Source: §1 · p. 2
• Declarative configurations can be inspected, edited, versioned, and shared, addressing reproducibility concerns highlighted by an audit that reproduced only a small fraction of sampled COVID-era studies.The introduction also notes omissions in reproducibility checklists.Source: §1 · p. 2
• The Epydemix Agent Framework implements these capabilities as an additive layer, leaving the library, existing notebooks, and scripts unchanged.Its stateless, file-based interface makes each operation a pure function of its input files.Source: §1 · p. 2
• The paper previews an end-to-end case study comparing four intervention strategies for a novel respiratory virus and an assessment spanning more than 50 agent sessions.The introduction does not report the assessment’s comparative results.Source: §1 · pp. 2–3

2.1 The Epydemix library

• Epydemix is an open-source Python package for developing and calibrating dynamic transmission models, with discrete-chain binomial simulation and support for inference, forecasting, and scenario projection.Its models are assembled from compartments and transitions through method calls.Source: §2.1 · p. 3

2.2 The Epydemix Agent Framework

• The framework guides an agent through Discover, Declare, Run, and Inspect, validating a YAML scenario before computation and enabling structured review of results.The workflow begins with a natural-language scenario description and uses a self-contained output bundle for results.Source: §2.2 · p. 3, Figure 1 · p. 4
• Machine-readable parameter registries support model discovery, while reusable YAML overlays and pre-run validation make scenario settings explicit and catch invalid values before computation.Registries describe accepted parameters and their defaults; overlays can inherit a base configuration and override selected fields.Source: §2.2 · p. 4
• Stateless CLI calls operate on files, and output bundles preserve simulation data and provenance while inspection returns compact summaries and comparisons.Bundles include trajectories, calibration samples when applicable, figures, metadata, and source-configuration records; inspection reports quantities such as quantiles, peak timing, attack rates, and pairwise differences.Source: §2.2 · p. 5
• The declarative interface covers common compartmental workflows but not custom transition functions, non-standard age groups, or post hoc transformations; these remain available through the direct API without the structured interface’s safeguards.The agent is instructed to use the CLI by default and fall back to direct API calls when an operation cannot be expressed declaratively.Source: §2.2 · p. 5

2.3 Experimental setup

• The reported agent sessions used Claude Code with Claude Sonnet 5, Epydemix 1.3.1, and agent framework version 1.Sessions were run on July 1, 2026.Source: §2.3 · p. 6
• The framework condition provided the CLI and agent contract, while the direct API condition used the same repository with those removed and retained library documentation and examples.The direct API condition was intended to represent a researcher using a well-documented library with a coding agent, not an absence of tooling.Source: §2.3 · p. 7
• The multi-task evaluation covered five modeling tasks, each repeated five times per condition, for 50 sessions with identical starting conditions and no shared artifacts.Tasks were SEIRHD scenario comparison, SIR calibration, calibrate-then-project, school-closure sweep, and measles coverage sweep.Source: §2.3 · p. 7
• For each task, the evaluation extracted wall-clock and model-request time, turns, output tokens, and cost from the agent runtime.Task definitions and per-session values were provided in Supplementary Information, Additional Tasks S5.Source: §2.3 · p. 7
• The vaccination case study used four runs per condition, separating read-only planning from execution, with no configuration writes or simulations during planning.In execute mode, the agent carried out its plan step by step.Source: §2.3 · p. 7
• For each task and metric, framework-to-direct-API median ratios below one indicated lower resource use; 96.8% distribution-free intervals were computed without multiple-comparison correction.For the case study, the authors reported coefficients of variation across four runs separately for planning and execution; costs were provider-billed US dollars at session time and were not stable estimates.Source: §2.3 · p. 7
• Case-study reporting used coefficients of variation across four runs for planning and execution separately, and treated session-time billed costs as comparison-only estimates.Costs were reported in US dollars and should not be interpreted as stable estimates.Source: §2.3 · p. 7

3.1 Case study: vaccination scenarios

• Across four framework instances, the agent consistently selected an SEIR model with vaccination and hospitalization and derived transmission from Italy’s contact-matrix spectral radius rather than using a preset rate.For the leaky-vaccine model, it computed attack rate from susceptible-to-exposed and vaccinated-to-exposed flows because the built-in metric did not apply.Source: §3.1 · p. 9
• The four runs differed in vaccination rates, stochastic realizations, start dates, and combined-strategy contact-reduction settings, but preserved the ordering Slow rollout < Combined < Rapid rollout for vaccination rates.Source: §3.1 · p. 9
• Combined rollout stayed below capacity but had a 26.3% attack rate as transmission resumed after contact reductions ended; rapid rollout outperformed it on every metric in this run.The combined strategy used half the vaccination rate of rapid rollout; the comparison reflects the specific rates and contact-reduction assumptions used.Source: §3.1 · pp. 9–10
• Baseline occupancy peaked at roughly 95,000 beds; slow rollout lowered the peak by about 13,500 beds but still breached the 55,000-bed capacity for 29 days, while rapid rollout peaked below 3,500 beds.Figure 3 shows median occupancy across 100 stochastic simulations, with 90% intervals.Source: §3.1 · p. 9, Figure 3 · p. 11

3.2.1 The vaccination case study

• Mean total cost was $2.98 with the framework versus $4.89 with direct API use, while mean runtime was 582 s versus 835 s; the framework also showed lower run-to-run variation.Both conditions reached similar qualitative epidemiological conclusions, identifying rollout speed as the main factor associated with outcomes and using the same strategies to keep occupancy below capacity.Source: §3.2.1 · pp. 10–11
• In the vaccination case study, cost variation was 0.10 during planning and 0.15 during execution, versus 0.19 and 0.35 for the direct API; runtime variation was 0.15 and 0.17, versus 0.30 and 0.40.Lower coefficients of variation indicate greater run-to-run consistency.Source: Table 1 · p. 11, §3.2.1 · p. 11

3.2.2 Additional modeling tasks

• SIR calibration and the Measles coverage sweep showed the same resource-reduction pattern, consistent with the framework reducing exploratory setup and output generation.The structured interface and explicit contract helped agents spend fewer turns discovering how to configure and run models and generate less text.Source: §3.2.2 · p. 12
• For Calibrate → Project, the framework used 0.53× the wall-clock time, 0.45× the output tokens, and 0.52× the cost of the direct API, with confidence intervals excluding parity for every metric.This task showed the largest difference across the additional modeling tasks.Source: §3.2.2 · p. 12, Table 2 · p. 13
• For the School-closure sweep, the framework condition required 1.41× API time, 1.32× output tokens, and 1.81× cost relative to the direct API condition; the task swept 18 intervention start dates.Source: §3.2.2 · p. 12
• For the 18-point school-closure sweep, framework bundles preserved each point’s configuration, outputs, provenance, and random seed, enabling point-by-point reproduction.The resource increase was spent on recording each sweep step as an independent, auditable artifact rather than representing a performance regression.Source: §3.2.2 · p. 13
• The direct API sweep was not unreproducible, but its result schemas varied across runs and it retained raw per-point trajectories in only three of five cases.The paper frames the framework’s higher resource use in this task as a reproducibility trade-off.Source: §3.2.2 · p. 13

• Across tasks, framework agents generally used fewer turns, output tokens, and compute than direct-API agents; in the vaccination case study both reached the same qualitative conclusions, but the framework was cheaper, faster, and more consistent.Source: §4 · p. 13
• Registry validation catches invalid configurations before simulation, reducing exploratory input-checking and shifting case-study effort from planning toward execution.Source: §4 · p. 13
• The School-closure sweep cost more because 18 points ran as independent bundles, but each preserved configuration, seed, and provenance; direct API outputs did so in 3/5 runs and schemas differed in all five.This makes reproducibility a property of interface consistency rather than the agent’s choices, though whether the extra cost is worthwhile depends on the task.Source: §4 · p. 14
• The implementation chose a stateless, file-based CLI, whose inputs and outputs are inspectable and versionable and whose commands serve humans or agents without a separate server.Source: §4 · p. 14
• The evaluation is limited by using a single agentic coding tool and a limited set of tasks.Source: §4 · p. 14
• A community benchmark should define representative infectious-disease tasks and correctness and reproducibility measures, quantify contract failure modes, compare models and tools, assess generalization, and test open-weight models against closed ones.Source: §4 · p. 14

Declaration of AI and AI-assisted technologies use

• The authors used Claude Code agent sessions with Claude Sonnet 5 under a Claude Pro subscription and report one experiment’s prompt and output verbatim.The terminal sessions used Claude Code version v2.1.198.Source: Declaration of AI and AI-assisted… · p. 15
• AI tools supported proofreading, text editing, and structural formatting; the authors reviewed and verified all generated suggestions and retain responsibility for the final content.The stated aims were to improve grammatical clarity and readability and align the text with journal style guidelines.Source: Declaration of AI and AI-assisted… · p. 15

Competing interests

• The authors declare no competing interests.Source: Competing interests · p. 15

Data availability

• The open-source framework is released under GPL-3.0 in the epydemix repository’s agent-framework branch, which includes the agent contract, CLI source code, and documentation.Source: Data availability · p. 15
• The case-study prompt and agent output appear verbatim in the main text, while the agent plan and step-by-step reproduction guide are in the Supplementary Information.The supplementary materials are SI Plan S1 and SI Guide S4.Source: Data availability · p. 15

2 Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern

• The Laboratory for the Modeling of Biological and Socio-technical Systems is affiliated with Northeastern University’s Network Science Institute in Boston, Massachusetts.Source: §2 · p. 18

SI Plan S1: Agent plan (Claude Code, plan mode)

• The study compares no vaccination, slow rollout, rapid rollout, and moderate vaccination with temporary NPIs using hospital-capacity and attack-rate outcomes.The combined strategy applies 70% school and 30% work contact reductions for eight weeks; the proposed comparison includes peak occupancy, peak date, days over 55,000 beds, and attack rate.Source: SI Plan S1: Agent plan (Claude Code,… · pp. 18, 19, 22
• The plan requires deriving transmission_rate from the target R0 and Italy’s contact-matrix spectral radius, then validating each scenario configuration before running it.The specified discovery workflow identifies a spectral radius of approximately 13.8988 and a population total of 59,435,140.Source: SI Plan S1: Agent plan (Claude Code,… · pp. 18, 19, 23
• Vaccination is modeled as a constant per-capita S→V rate, while vaccine efficacy reduces breakthrough transmission through a V→Exposed transition.The plan therefore calls for calculating attack rate from both S_to_E_total and V_to_E_total flows because the built-in metric is unsuitable for this vaccination model.Source: SI Plan S1: Agent plan (Claude Code,… · pp. 19, 22
• The simulations span 2024-01-01 to 2025-05-01 at dt 0.5, with 100 simulations per scenario and a comparison table and hospital-occupancy figure planned.The figure overlays scenario medians with 90% confidence-interval bands and a 55,000-bed capacity line.Source: SI Plan S1: Agent plan (Claude Code,… · pp. 18, 19, 22, 23
• The combined-strategy NPI window is set roughly four weeks before through four weeks after the no-vaccination median hospitalization peak.This makes the intervention window depend on inspecting the baseline run, with the window clipped to the simulation horizon.Source: SI Plan S1: Agent plan (Claude Code,… · p. 21

SI Bundle S2: Structure of the .epx output bundle

• Each run, calibrate, or project operation writes a self-contained .epx directory containing a manifest, configuration, simulation data, and a comparison figure.The example baseline bundle includes compartment counts, transition counts, per-simulation parameters, and hospital_capacity_comparison.png.Source: SI Plan S1: Agent plan (Claude Code,… · p. 23
• The Parquet files store compartment counts, daily transitions, and per-simulation parameter values; transition flows can also support derived quantities such as leaky-vaccine attack rate.That attack rate sums Susceptible_to_Exposed and Vaccinated_to_Exposed flows over the simulation horizon.Source: SI Plan S1: Agent plan (Claude Code,… · pp. 23–24
• The manifest records model and run metadata, file schemas, usage hints, and provenance, allowing interpretation without opening the Parquet files.It includes the Epydemix version, model and compartments, demographic groups, simulation window and realizations, parameter values, and the source configuration path.Source: SI Plan S1: Agent plan (Claude Code,… · p. 24
• A verbatim copy of the originating declarative configuration travels inside the bundle, preserving the exact setup even if the source file changes or disappears.The copy includes the model, population, parameters, simulation, and initial-condition blocks.Source: SI Plan S1: Agent plan (Claude Code,… · p. 24

Process comparison

• Without-framework planning took 504s versus 240s and cost $2.73 versus $1.43; execution was nearly identical in duration (331s versus 342s) but cost about 40% more without the framework.Source: Process comparison · p. 25

Content comparison

• No-vaccination and slow rollout breached the 55,000-bed capacity line for roughly a month, whereas rapid rollout and the combined strategy cleared it in every run.When vaccination was the only intervention, rollout speed was the dominant lever; no-vaccination peak occupancy stayed tightly clustered across all eight runs (94,755–99,079; mean 97,441 ± 2,202).Source: Content comparison · p. 25
• Slow-rollout mean peak occupancy was 81,277 vs. 80,587 beds, and all eight runs breached capacity, making it insufficient on its own in either condition.The passage describes the cross-condition difference as only a few thousand beds and the breach as lasting roughly a month.Source: Content comparison · pp. 25–26, Table 2 · p. 26
• For rapid rollout, rate means were essentially identical between with- and without-framework conditions; every run cleared capacity with 0 days over, though with-framework runs showed wider variance from one conservative-rate outlier.Source: Content comparison · p. 26
• Combined strategies cleared capacity in every run and were more consistent than rapid rollout; with-framework runs had lower mean peak occupancy and attack rate.The chosen school/work reductions and intervention windows did not change the finding that the combined strategy cleared capacity in all eight runs.Source: Table 4 · p. 26, Content comparison · pp. 26–27

Caveats and summary

• The comparison leaves the main-text picture unchanged: baseline dynamics converge tightly, the without-framework condition is costlier and more planning-heavy, and vaccination-rate ambiguity drives disagreement over rapid rollout alone.The comparison used four runs per condition; the passage notes that a single differently worded rate choice can flip a condition’s headline outcome.Source: Caveats and summary · p. 27

Agent run

• Any comparable terminal-based agent can use the CLI and AGENT.md; launch it from the repository root, provide the case-study prompt, and it can configure, validate, simulate, and query results.Starting in the repository root lets the agent discover AGENT.md, which describes how to use the framework.Source: SI Guide S4: Step-by-step guide to… · pp. 27–29

Installation and verification

• Install Epydemix with its agent extra, preferably in a fresh virtual environment, then verify the CLI lists available model types before involving an agent.The guide recommends a virtual environment to avoid dependency conflicts.Source: SI Guide S4: Step-by-step guide to… · p. 28

Outputs and inspection

• Afterward, the agent reports a chat summary and saves configuration files, scenario bundles, and an analysis script, while the same CLI lets people inspect or compare bundles directly.The comparison figure is saved inside the baseline bundle’s figures folder.Source: SI Guide S4: Step-by-step guide to… · p. 29

Reproducibility

• Differences across reproductions can arise from stochastic simulations and from agents choosing different vaccination rates, NPI timing or combinations, or initial conditions under the qualitative prompt.The prompt specifies rollout speed and intervention magnitude qualitatively, so these choices may vary while remaining consistent with its intent.Source: SI Guide S4: Step-by-step guide to… · p. 30
• Rapid rollout and the combined strategy avoid hospital-capacity breach, while slow rollout and no vaccination breach it, a qualitative pattern expected to reproduce reliably.Small numerical differences are expected because simulations are stochastic and agents may choose different values consistent with the qualitatively specified prompt.Source: SI Guide S4: Step-by-step guide to… · p. 30

Setup

• The comparison comprised 50 sessions: five tasks run five times each under baseline and framework conditions, with stochastic agent behavior motivating repeated runs.Baseline provided the library, documentation, and examples; framework added its AGENT.md contract and command-line tooling.Source: Setup · p. 30

Agent harness

• Each task used a non-interactive Claude Code session with one initial prompt; delegation, web access, and customization mechanisms were disabled to measure a single agent’s work.The harness allowed shell commands and custom Python execution while withholding shortcuts and external documentation access.Source: Agent harness · pp. 30–31

Isolation of task runs

• Sessions used fresh workspaces, and filesystem-path checks found no isolation violations across the 50 runs.Earlier results, scripts, and transcripts were inaccessible to later sessions.Source: Isolation of task runs · p. 31

Execution and metrics

• The 50 runs consumed 4.6 hours of agent wall-clock time and 84 million tokens; recorded metrics also included request time, turns, errors, and token categories.Token usage was split into input, output, cache-read, and cache-creation counts.Source: Execution and metrics · pp. 31–32

Results: baseline vs framework

• Table 7 reports per-task median execution metrics and observed minimum–maximum ranges for five runs in each condition, but the supplied text gives no cell values.The surrounding text identifies the conditions as baseline and framework.Source: Table 7 · p. 32, Results: baseline vs framework · p. 32

Task prompts

• The two conditions differed only in their orientation preamble; task bodies and closing instructions were shared.The baseline preamble directed the agent to consult the repository, while the framework preamble instructed it to read AGENT.md.Source: Task prompts · p. 33

Task bodies

• The task set included custom SEIRHD scenarios with no, early, or late intervention and SIR calibration against synthetic infection data using a deliberately wide transmission-rate prior.The SEIRHD prompt specified a 40% transmission reduction for the intervention scenarios; the SIR task used a prior over [0.1, 0.8].Source: Task bodies · pp. 33–35
• The calibration sensitivity task asks whether jointly estimating transmission_rate and recovery_rate worsens estimates relative to true values, using uniform priors and 300 particles over 8 generations.The specified prior ranges are [0.1, 0.8] for transmission_rate and [0.03, 0.3] for recovery_rate; the task requests inspection of the joint posterior.Source: Task bodies · pp. 35–36
• The task sets transmission_rate to 0.12, then overrides it to 0.03 from April 15 through June 1 and asks for baseline-versus-intervention projection comparisons, including peak and final-attack effects.Source: Task bodies · pp. 36–37
• T4 evaluates the optimal timing of a three-week school closure in New York State, removing all school-layer contacts for 21 consecutive days and sweeping start dates weekly from 1 December 2026 through 1 April 2027.Source: Task bodies · pp. 37–38