PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Informed Multi-Agent Coordination for Hospital Patient Flow Optimization

Guoqing Zhang, Rafik Hadfi, Takayuki Ito

arXiv:2609.37022v1cs.AIcs.MA

TL;DR

Static BCMP models rely on stationary routing assumptions that do not capture state-dependent blocking, while hospital departments make decentralized decisions under coupled constraints. The paper embeds empirical BCMP topology and capacity boundaries in decentralized multi-agent reinforcement learning, with agents coordinating through localized action fingerprints. It reports an order-of-magnitude reduction in cumulative patient delay versus static analytical approximations, alongside better performance than heuristic dispatchers and independent MARL baselines.

  • Problem

    Stationary BCMP routing assumptions fail under capacity-driven blocking, while hospital departments operate with local observations and resources under coupled constraints.

    Source: §2.1 · p. 4, §3.1 · p. 5

  • Method

    The framework embeds empirical BCMP topology and capacity boundaries as physical priors for decentralized agents that coordinate routing and service scaling through localized action fingerprints.

    Source: §1 · p. 3

  • Results

    The approach reduces cumulative patient delay by an order of magnitude versus static analytical approximations and outperforms heuristic dispatchers and independent MARL baselines.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The paper uses BCMP structure to connect queueing-based physical boundaries with cooperative multi-agent control of hospital patient flow.

    Source: §1 · p. 3, §3.3 · p. 7

    Baseline service capacities rely on historical MIMIC-IV averages and aggregated clinical stages, omitting real-time clinician workload, nursing fatigue, and workplace stress.

    Source: Limitations and Future Research Horizons · p. 16

Abstract

from arXiv · show

Efficient patient flow coordination across autonomous hospital departments is critical for mitigating overcrowding and balancing resource utilization. While classical queueing theory, specifically open Baskett--Chandy--Muntz--Palacios (BCMP) networks, provides an interpretable mathematical topology for healthcare operations, analytical models rely on stationary assumptions and fixed routing matrices that degrade under state-dependent real-world dynamics. Conversely, centralized reinforcement learning approaches struggle to accommodate the decentralized structure of hospital governance, where individual clinical departments function with localized observations, heterogeneous resources, and divergent operational objectives. In this paper, we present a Multi-Agent Systems (MAS) framework titled \emph{Physics-Informed Multi-Agent Coordination}, which embeds empirically calibrated BCMP queueing topologies as physical priors within a decentralized multi-agent reinforcement learning architecture. Formulated as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) under coupled resource constraints, our method enables autonomous departmental agents to cooperatively negotiate patient routing and dynamic service scaling. To mitigate environmental non-stationarity without inducing excessive communication overhead, agents exchange localized action fingerprints along network edges and optimize a spatially decomposed reward structure. Empirical evaluations driven by real-world MIMIC-IV patient trajectories indicate that this cooperative multi-agent approach substantially reduces cumulative system delay compared to static Markovian approximations, heuristic dispatching, and independent multi-agent baselines, while maintaining clinical safety constraints.

1 Introduction

Hospital patient flow must coordinate autonomous departments with distinct local resources and objectives, while static queueing assumptions fail under state-dependent blocking. The paper combines empirical BCMP structure with decentralized multi-agent coordination and reports lower delay than static approximations, heuristic dispatchers, and independent MARL baselines.

1 Introduction

  • Finite beds and saturation make fixed-routing BCMP assumptions unreliable, while centralized control conflicts with departments’ local observations, resources, and objectives.Blocked transfers can accumulate upstream queues; centralized control also creates communication overhead and organizational friction.Source: §1 · p. 2
  • The framework uses empirical BCMP topology and capacity boundaries as physical priors while departmental MARL agents negotiate patient routing and capacity scaling.The BCMP network is calibrated on MIMIC-IV trajectories.Source: §1 · p. 3
  • Agents share smoothed policy representations along topological edges to address partial observability and non-stationarity without global communication overhead.The protocol uses localized action fingerprints between adjacent agents.Source: §1 · p. 3
  • Historical clinician transition trajectories inform an online action mask that excludes medically unfeasible patient routes.The mask is intended to maintain clinical protocol integrity and prevent hazardous routing.Source: §1 · p. 3
  • The approach reduces cumulative patient delay by an order of magnitude versus static analytical approximations and outperforms heuristic dispatchers and independent MARL baselines.The authors also report that cooperative action decomposition helps suppress non-linear congestion cascades.Source: §1 · p. 3

2 Related Work

Prior BCMP models provide structural clarity but depend on stationary assumptions that break under capacity blocking. Healthcare MARL offers decentralized decision-making, yet the paper identifies a gap in combining it with queueing structure and clinical safety constraints.

2.1 BCMP Queueing Networks in Healthcare Operations

  • Classical BCMP models represent multi-stage patient trajectories but stationary transition assumptions fail when finite-capacity downstream units block transfers and upstream queues grow.Under these state-dependent dynamics, product-form independence no longer holds, limiting stationary models for real-time intervention.Source: §2.1 · pp. 3–4

2.2 Multi-Agent Systems and Collaborative Control

  • Healthcare MARL commonly treats clinical systems as black-box simulators, leaving queueing mechanics and physical capacity limits outside the learned model.The cited discussion connects uncommunicated agent updates with localized traps and oscillating downstream bottlenecks.Source: §2.2 · p. 4
  • The paper positions empirical BCMP topology as domain physics for guiding cooperation, bounding negotiation, and supporting verifiable clinical safety.The stated gap is deployment of collaborative MARL in safety-critical healthcare networks.Source: §2.2 · p. 4

3 Physics Prior: BCMP Queueing Topology and Constraints

The paper repurposes BCMP network structure as a physical prior for multi-agent control rather than relying on its stationary formulas for dynamic dispatch. The prior encodes patient pathways, workload, capacity limits, and the architecture for departmental coordination.

3 Physics Prior: BCMP Queueing Topology and Constraints

  • The framework uses open BCMP network architecture as a structural boundary for multi-agent interaction rather than as a steady-state solver.The network topology is calibrated on MIMIC-IV trajectories and defines resource-capacity boundaries for coordination.Source: §3 · p. 5, Figure 1 · p. 6

3.1 Network Topology and Traffic Conservation

  • Class-specific external arrivals and transition probabilities determine each department’s effective arrival rates under traffic conservation.Patient classes are based on acuity and DRG characteristics, and the network includes clinical nodes such as the ED, surgery, ICUs, and general medicine wards.Source: §3.1 · p. 5

3.2 Physical Capacity Bounds and Stability Redlines

  • Each department’s offered traffic must remain below its physical capacity, with baseline capacities estimated using QED calculations and targeted delay thresholds.The capacity calculation combines Erlang-C delay verification with the Halfin–Whitt square-root staffing rule; targets are ϵ_ED = 0.01 and ϵ_ward = 0.05.Source: §3.2 · p. 5
  • The architecture figure pairs a hospital-wide BCMP topology with local controllers that use telemetry and neighbor fingerprints to adjust discharge acceleration and transfer routing.An expert prior action mask filters medically unfeasible transitions from exploration.Source: Figure 1 · p. 6

3.3 The Breakdown of Stationary Product Forms

  • Finite-capacity blocking makes routing state-dependent and breaks stationary product-form analysis, but the network graph, baseline service rates, and utilization limits remain useful physical priors.When downstream capacity is full, patients queue upstream until beds are available.Source: §3.3 · p. 7

4 Multi-Agent Coordination Framework

The framework assigns autonomous agents to departments because local decisions and resources are decentralized yet coupled across hospital pathways. Agents use localized observations, bounded service and routing actions, neighbor communication, and clinically filtered choices to coordinate patient flow.

4.1 Why Multi-Agent Over Centralized RL?

  • Assigning agents to departments preserves administrative boundaries and local decision-making while supporting routing negotiation across coupled resource constraints.Localized interventions such as discharge acceleration can affect upstream congestion when a downstream unit rejects transfers.Source: §4.1 · p. 8

4.2 Dec-POMDP Formulation and Theoretical Rationale

4.3 Topology-Aware Communication and Cooperative Negotiation

  • Agents exchange action fingerprints with adjacent topological neighbors to provide policy-shift feedback and support cooperative routing negotiation.In the paper’s example, downstream units can restrict non-critical admissions while step-down wards accelerate discharge to accommodate transfers.Source: Action Fingerprint Communication… · pp. 9–10

4.4 Expert-Guided Action Masking and Topological Safety

  • An expert mask derived from MIMIC-IV attending protocols removes clinically impermissible destinations before action sampling.The paper says this confines exploration to safe clinical envelopes and reduces policy variance.Source: §4.4 · p. 10

5 Reward Decomposition and Multi-Agent Credit Assignment

The reward architecture balances departmental efficiency with neighborhood coordination and hospital-wide flow, addressing the limits of purely global or purely local incentives. Its three layers assign credit at local, neighboring, and system scales.

5 Reward Decomposition and Multi-Agent Credit Assignment

  • Three reward layers assign credit across local throughput and resource costs, neighborhood spillover, and hospital-wide delay change.The hierarchy is intended to suppress cross-ward spillover while guiding macroscopic convergence.Source: §5 · pp. 10–11

5.1 Local Domain Objective (Rlocal,i)

  • The local objective penalizes waiting queues, utilization above 85%, premature-discharge risk, and staffing costs, while rewarding successful queue reduction after acceleration.The local credit bonus applies when active acceleration reduces the local waiting list.Source: §5.1 · p. 11

5.2 Neighbor Collaborative Objective (Rneighbor,i)

  • The neighborhood objective uses a 0.9 spatial discount to discourage relieving a ward by shifting bottlenecked queues to connected departments.Averaged feedback from connected peers supports receptive transition pathways and coordinated admission timing without global communication bandwidth.Source: §5.2 · p. 11

5.3 Global Systemic Objective (Rglobal)

  • The global objective uses the first-order change in total network delay to provide continuous, zero-mean credit signals that reward system-level delay contraction.It is designed to anchor agents to hospital-wide flow fluidity.Source: §5.3 · pp. 11–12

6 Experiments and Empirical Evaluation

Experiments use a MIMIC-IV-calibrated simulation to compare cooperative MA2C with analytical, heuristic, and independent-agent baselines, then test its components and pathway safety. Cooperative MA2C substantially reduces cumulative delay across execution modes while learned policies reduce both monitored pathway anomalies relative to historical records.

6.1 Experimental Configuration and EHR Calibration

  • A DES calibrated on approximately 60,000 MIMIC-IV admissions simulated a 1.25 demand surge; MA2C trained for 250 episodes of 4,000 simulated hours.Calibration parameterized arrival intensities, service distributions, and transition matrices from the inpatient cohort.Source: §6.1 · p. 12

6.2 Comparative Analysis against Multi-Agent Baselines

  • 2081.2 ± 571.7 hours (stochastic) and 3974.0 ± 481.0 hours (greedy): Cooperative MA2C achieves an approximate 56-fold reduction versus static Markovian approximations and outperforms independent MARL in both modes.Greedy dispatch also reduces delay versus the static baseline, while IA2C is highly sensitive to stochastic versus deterministic execution.Source: §6.2 · pp. 12–13, Table 1 · p. 13

6.3 Structural Ablation Studies

  • Ablations show higher delay when neighbor fingerprints or expert masking are removed, severe divergence without cooperative routing, and increased delay without dynamic acceleration.Removing fingerprints raises delay from 2,835.2 to 3,024.5 hours; disabling routing yields 226,231.1 hours and a 0.368% premature discharge rate. Removing acceleration yields 3,990.8 hours; removing masking yields 3,108.9 hours and a 0.190% premature discharge rate.Source: Table 2 · p. 14, §6.3 · p. 14

6.4 Clinical Pathway Safety and Anomaly Verification

  • All learned policies reduce ping-pong transfers and premature ICU discharges versus historical rates; MA2C records 1.727% and 1.736% ping-pong and 0.214% and 0.211% premature discharges across its two modes.Historical rates are 7.375% for ping-pong transfers and 2.585% for premature ICU discharges; the two MA2C values are greedy and stochastic, respectively.Source: Table 3 · p. 14, §6.4 · p. 15

7 Discussion & Conclusion

The discussion links nonlinear queue growth near capacity to localized coordination, reporting reduced simulation delay from timed interventions while noting limits in historical capacity assumptions.

7 Discussion & Conclusion

  • Queue growth is nonlinear: utilization at ρi ≤0.7 is described as a stable plateau, while utilization approaching 1− drives superlinear queue and delay escalation.The passage also notes that minor arrival perturbations in this congestive regime can trigger inter-departmental blocking cascades.Source: §7 · p. 15, Figure 2 · p. 16
  • Timed cooperative interventions achieve a 56-fold reduction in simulation delay (2 081.2 hours).The policy combines slight service acceleration (Mean α = 0.075) with cooperative patient routing diversion.Source: §7 · p. 16
  • Localized service acceleration and patient diversion lower utilization below the nonlinear inflection threshold, restoring the linear-delay regime without global staffing additions.The interventions are coordinated through localized departmental actors and topological action-fingerprint communication.Source: §7 · p. 16
  • Baseline service capacities use historical MIMIC-IV averages and aggregated clinical stages, overlooking real-time workload, nursing fatigue, and workplace stress.The authors identify this as a simulation-to-reality gap and propose future work coupling macroscopic routing with microscopic behavioral simulation.Source: Limitations and Future Research Horizons · p. 16
Loading 2609.37022v1…