PaperlayerView exact revision on arXiv

Source-linked AI summary

Manual binding and semi-automated interface generation for efficient simulation backends with user-friendly frontends: A~demonstration for epidemic use cases

Maximilian Betz, Daniel-Michel Richter, Daniel Abele, René Schmieding, Henrik Zunker, Martin J. Kühn

arXiv:2610.40343v1cs.SE

TL;DR

Python and R are widely used in infectious-disease modeling, while compiled implementations can be substantially faster and manual bindings add work. The paper presents MEmilio’s Python interface and a modular AST-based generator, reporting minimal to no overhead and performance close to its C++ baseline.

  • Problem

    Python and R offer low-barrier modeling, but compiled C or C++ can be much faster, while implementing bindings adds work for backend developers.

    Source: §1 · p. 2

  • Method

    The paper binds MEmilio’s C++ backend to Python and uses AST analysis, an intermediate representation, and templated generation to create binding code.

    Source: §6 · p. 22

  • Results

    The holistic Python coupling has minimal to no overhead and yields almost the same performance as the C++ baseline.

    Source: §6 · p. 22

  • Takeaways & Limitations

    The Python interface lets users configure and run MEmilio models and couple them with Python frameworks for sensitivity analysis or parameter estimation.

    Source: §6 · p. 22

    The paper omits further technical details and generalizations, such as those used in agent-based models, and refers readers to MEmilio’s code base and documentation.

    Source: §2 · p. 5

Abstract

from arXiv · show

Challenges and chances of mathematical software are directly entangled with the programming language used. Depending on the background of developers and users, conflicts between different goals may arise. While several, e.g., scripting languages allow a low barrier and user-friendly entry to mathematical modeling and simulation, corresponding implementations will most likely not allow timely outcomes for detailed simulations. In infectious disease modeling, while several large-scale software frameworks rely on C or C++ implementations, most models are implemented in Python or R. While the latter allow for fast prototyping and more interactive development, it has been shown for several applications that C or C++ implementations outperform plain Python or R implementations by factors between one and three orders of magnitude. In particular, in times of pandemics, the urgent need for swift estimation of the system's dynamics makes the use of large-scale models in these noncompiled languages prohibitive. In order to create synergies and close the gap between scientists from the application and the computational domain, binding libraries allow to execute entire, compiled software frameworks from languages such as Python. However, with iterative model development, extension, or addition of new models, implementing bindings can represent another relevant barrier to developers of the C or C++ backends. In this paper, we show how a large-scale software framework with a multitude of infectious disease models can be efficiently exposed to Python. In addition, we suggest a semi-automated way to generate interface code by analyzing generic C++ code through abstract syntax trees, enabling a modularized approach working with intermediate representations.

1 Introduction

Python and R make infectious-disease modeling accessible, while compiled C and C++ can deliver much faster simulations. The paper presents flexible Python bindings and semi-automated generation to bridge that gap and reduce binding work.

1 Introduction

  • Compiled C or C++ implementations have outperformed Python by factors around 50, 100, or more, while repeated simulations can make timely epidemic modeling difficult.The cited comparison also reports fully bound C++ running ODE-based formulations 1–2 orders of magnitude faster than using compiled routines only for solving the ODE system.Source: §1 · p. 2
  • Flexible Python interfaces make compiled backends accessible to Python users, but manually binding large code bases adds substantial work and language-specific complexity.Existing approaches include BoostPython, pybind11, and nanobind; the paper notes that binding large code bases can cost less than creating entirely new implementations.Source: §1 · p. 2
  • Binding generation has two complementary routes: descriptions can generate interfaces alongside clearly separated new modules, while AST analysis can generate bindings for existing compiled code.The paper identifies unified descriptions and domain-specific languages, including SBML, as examples of the first route.Source: §1 · pp. 2–3
  • The paper proposes a modular AST-to-intermediate-representation pipeline for generating Python bindings and evaluates code completeness and Python runtime performance.It also describes a scanner and handlers connected through an intermediate representation to a code generator.Source: §1 · p. 3

2 Modular and generic implementations for infectious disease models

MEmilio uses modular, recurring structures across infectious-disease models as a basis for binding and reuse. The section illustrates those structures with an age-stratified SEIR model and limits its technical coverage.

2 Modular and generic implementations for infectious disease models

  • Unified patterns and modularity can reduce redundant backend code and substantially lower manual binding overhead, while recurring patterns are prerequisites for functional binding-code automation.MEmilio is presented as an existing modular framework with patterns unified across modules.Source: §2 · p. 3
  • An age-stratified ODE-SEIR model uses four infection states and three forward flows, with population and parameter structures supporting model definition.Its parameter type can distinguish floating-point precision or carry automatic-differentiation values; aggregated population counts can also use uncertain values.Source: §2 · pp. 3–4
  • Reusable population structures support stratification by an arbitrary number of categories, and age-resolved models can be instantiated at compile time with a chosen number of age groups.The section omits further technical details and generalizations, including those used in agent-based models.Source: §2 · pp. 4–5

3 From C++ backend to a practitioner-targeted Python interface

MEmilio’s Python interface aims to give Python users flexible, familiar access to its C++ backend. Manual bindings and semi-automated generation serve different needs, while ongoing backend development makes manual maintenance burdensome.

3 From C++ backend to a practitioner-targeted Python interface

  • A flexible Python interface offers users more freedom than binding command-line executables, but requires developers to bridge language concepts and understand the backend’s public API.The cited challenges include C++ ownership, references, object lifetimes, templates, and deciding how backend elements should appear in Python.Source: §3 · p. 5

3.1 Development of the MEmilio Python interface

  • The interface targets low-barrier, line-by-line Python use and lets users adapt parameters and scenarios or couple established packages for sensitivity analysis and parameter estimation.Bindings are partly manual and partly generated: existing ABM and equation-based models use pre-implemented bindings, while generation targets new equation-based models and small structural extensions.Source: §3.1 · p. 6
  • The Python interface exposes selected functionality rather than all C++ template possibilities, including simulation calls for double precision while some technical helpers remain C++ only.The C++ and Python ODE-SEIR calls are similar; when no integrator is specified, both select the default Runge-Kutta Cash-Karp5(4) integrator.Source: §3.1 · pp. 6–7

3.2 Challenges for new models and structures

  • As MEmilio evolves, changes to the C++ library can require corresponding binding adaptations, making manual interface maintenance an ongoing and potentially growing task.Interface tests compare outcomes with identical C++ backend tests, though the authors acknowledge a marginally higher chance of errors on the Python interface side.Source: §3.2 · p. 8, §3.1 · p. 8

4 Semi-automated interface generation through ASTs of existing code

The paper proposes semi-automated binding generation for existing C++ code, using AST analysis and templates to speed implementation. Its modular pipeline separates extraction, intermediate representation, and code generation.

4 Semi-automated interface generation through ASTs of existing code

  • The generator is intended to accelerate binding implementation and avoid typical typo or copy-and-paste errors, with processing taking seconds to minutes on laptop infrastructure.The approach uses local AST analysis and supports model-based or more general structure-based generation through different target-code templates.Source: §4 · p. 9

4.1 Concept and workflow

  • The pipeline extracts structures from C++ targets into ASTs, scans them into a consolidated intermediate representation, then applies templates to produce binding code.The ASTHandler can process targets in parallel; the Scanner extracts selected user-defined structures, and the Generator writes output to dedicated files.Source: §4.1 · p. 9
  • Separating AST construction, analysis, and code generation allows individual stages to be maintained, extended, or replaced independently.The intermediate representation is described as a stable interface between analysis and code generation.Source: §4.1 · p. 9
  • The workflow diagram distinguishes pipeline components from inputs and outputs, and data structures from logic classes or active components.Blue boxes denote components, orange boxes inputs and outputs, trapezoids data structures, and rounded boxes logic classes or active components.Source: Figure 1 · p. 10

4.2 Extracting structural information

The framework builds ASTs in the correct source and compiler context, then organizes them for inspection and analysis. A configurable Scanner extracts selected structures, from whole models or individually named functions.

4.2.1 Transforming source code and organizing ASTs.

  • ASTHandler provides a unified interface for creating ASTs and managing them for later analysis or transformation.AST construction also uses configuration for source files, output directories, and compiler settings so dependencies resolve in the correct build context.Source: §4.2.1 · pp. 10–11
  • Unique node identifiers and source line numbers enable precise lookup and selective extraction or visualization of AST subtrees.This supports focused examination of deeply nested constructs while suppressing irrelevant structures.Source: §4.2.1 · p. 11
  • The Parameters constructor AST exposes its parameter, base-class initialization, member initialization, and constructor body as a hierarchy of cursor kinds.Its empty body appears as a COMPOUND_STMT with no further information.Source: §4.2.1 · p. 11, Figure 2 · p. 12
  • The reduced simulate AST has 54 nodes and represents template parameters, return type, five arguments, and control flow including simulation setup, an optional integrator, advancement to tmax, and returned data.The subtree can be selected by the function's node identifier, 754637.Source: §4.2.1 · pp. 12–13, Figure 3 · p. 12

4.2.2 Identifying relevant structures from ASTs.

  • The Scanner traverses ASTs and collects structures specified by a configuration dictionary, which can select entire model components or add individual functions such as simulate.This combines automatic model-level selection with user-defined additions when model-level deduction is insufficient.Source: §4.2.2 · pp. 13–14

4.3 Organization of information in Intermediate Representation

The Intermediate Representation (IR) separates AST scanning from binding generation by organizing extracted information into a structured model. It stores declaration metadata and model-specific generation context for the Generator.

4.3 Organization of information in Intermediate Representation

  • The IR organizes and preserves extracted information so the Generator can work from the structured model without revisiting the original ASTs.It is the data model between scanning and generation.Source: §4.3 · p. 14
  • Each binding_type_info entry records declaration type, name, cursor kind, and namespace, with function return and argument details and declaration-context properties where relevant.Class declarations also collect their associated member functions.Source: §4.3 · pp. 14–15
  • The model-level IR combines model and generation settings with structural collections and found_bindings, which links to the declarations identified during scanning.Its fields include model class, namespace, scalar type, initialization, population groups, and include information.Source: §4.3 · p. 15

4.4 Interface generation

Binding generation uses templates and the IR to turn extracted structures into Python binding code. Model-specific templates fill placeholders from model data, while structural generation selects specialized templates and routines for general C++ structures.

4.4.1 Model-specific generation.

  • The model-specific template defines the PYBIND11_MODULE structure, with placeholders replaced by functions that use the IR to produce C++ names or complete code segments.The template remains model-independent while generated bindings specialize to each model's structures; FlowModel bindings expose the class and an additional flow-simulation function.Source: §4.4.1 · pp. 15–16

4.4.2 Generalization to structural C++ level.

  • Structural generation supports selected C++ classes, templated classes, and functions, but requires choosing a suitable template for each extracted element and more user-specific input.It can be used independently of or alongside model-specific generation.Source: §4.4.2 · p. 16
  • A coordination component routes IR entries by general type and finer attributes to specialized routines for classes, class-like structures, and functions.The selected routines handle structural properties and can use adaptive variants for overloaded or otherwise non-directly exposable functions.Source: §4.4.2 · pp. 16–17
  • AST analysis, the Intermediate Representation, and structure-based generation together form a complete transformation pipeline that outputs binding code.Separating template selection and processing supports expansion to previously unconsidered structures, including new C++ or binding-library functionality.Source: §4.4.2 · pp. 17–18

5 Results

Python bindings add little runtime cost to the C++ epidemic simulation, support coupling with sensitivity-analysis tools, and allow new model bindings to be generated in under a minute on average.

5.1 Runtime overhead of the Python bindings

  • At 1000 nodes, binding overhead is 0.0008 %–0.0038 % for Δ𝑡= 0.1 and 0.0033 %–0.0137 % for Δ𝑡= 0.5; the compared runtimes are shown in Figs. 5 and 6.At 10 nodes, overhead is about 3 μs; it falls below 1 % at 30 nodes for Δ𝑡= 0.1 or 100 nodes for Δ𝑡= 0.5. The C++ and Python results were bitwise identical in validation runs.Source: Figure 5 · p. 19, §5.1 · pp. 19–20, Figure 6 · p. 20

5.2 Coupling with established Python frameworks

  • OpenTURNS sensitivity analysis identifies contact transmission probability as the most influential parameter, while severe and critical case shares matter little for infections but more for deaths.The analysis used 1,000 samples from 12 independent parameter distributions and measured infections and deaths after 30 days; Fig. 7 presents the Sobol’ indices.Source: §5.2 · pp. 20–21, Figure 7 · p. 21

5.3 Generating bindings to a new model

  • The binding-generation workflow averaged 52 seconds for SEIR and 53 seconds for SECIR, with AST creation taking the largest share of time.Figure 8 shows the workflow stages; scanning took about four seconds and code generation was negligible.Source: Figure 8 · p. 21, §5.3 · p. 22

6 Conclusion

The paper presents a flexible Python interface to MEmilio’s C++ backend and a semi-automated process for generating bindings. The approach retains near-C++ performance and supports extensions of backend models in Python.

6 Conclusion

  • The Python interface exposes MEmilio’s major functionalities, including scenario setup, model stratification, parameter manipulation, ensemble simulation, and coupling to Python frameworks.The interface is based on pybind11 and translates framework functionality into a Pythonic form.Source: §6 · p. 22
  • The holistic Python coupling approach has minimal to no overhead and yields almost the same performance as the C++ baseline.The authors connect this interface to reducing barriers for Python developers and users of mathematical epidemiological modeling.Source: §6 · p. 22
  • The three-stage binding process uses ASTs, a configurable Scanner, an Intermediate Representation, and a Generator to create Python binding code for backend extensions.The process takes less than one minute on a single Linux workstation core; extensions require adjusting the general configuration, not the generation process.Source: §6 · p. 22
  • Future work will explore generating C++ models and Python bindings together from existing C++ code and user-defined model generalizations or extensions, including vector-borne diseases.Source: §6 · p. 23

Binding text for the simulation function

The simulation-binding generator formats binding code from an intermediate representation. It emits a standard simulation binding when simulation support is enabled and adds flow-simulation bindings for flow models.

Binding text for the simulation function

Parameters, explanations, and average value for sensitivity analysis

The table presents parameters, explanations, and average values for wild-type SARS-CoV-2, based on [33].

Parameters, explanations, and average value for sensitivity analysis

Loading 2610.40343v1…