PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Trustworthy AI-Enabled Emergency Medical Dispatch: Uncertainty-Aware Ambulance Travel Time Estimation

Muhammad Adil Abid, Johan Holmgren, Fabian Lorig, Jesper Petersson

arXiv:2610.31944v1cs.LG

TL;DR

Ambulance travel-time point predictions do not express individual uncertainty, and real-world trip logs can contain artefacts that distort the prediction task. The paper combines domain-informed curation, point prediction, and conformal calibration, evaluated on two years of Skåne trip data. CQR maintained 88.9–91.6% empirical coverage across distance quartiles, while fixed-margin strategies undercovered the longest trips.

  • Problem

    Ambulance travel-time approaches typically provide point predictions without indicating individual uncertainty, despite trips with similar expected times having different uncertainty.

    Source: §1 · pp. 2–3

  • Method

    The methodology curates ambulance trip data, trains a point-prediction model, and applies conformal calibration to produce prediction intervals.

    Source: §1 · p. 3

  • Results

    CQR maintained 88.9–91.6% empirical coverage across trip distance quartiles, while fixed-margin strategies had approximately 83% coverage for the longest trips.

    Source: §6 · p. 15

  • Takeaways & Limitations

    Across the evaluated distance quartiles, CQR provided the most stable empirical coverage among the evaluated strategies.

    Source: §6 · p. 15

    Future work will incorporate richer contextual information, including real-time traffic, weather, and incident-specific characteristics.

    Source: §6 · p. 15

Abstract

from arXiv · show

Emergency medical dispatch increasingly relies on AI-enabled decision support systems when making ambulance dispatch decisions. In this process, accurate ambulance travel time estimates are essential for timely resource allocation and prehospital care. However, existing ambulance travel time estimation approaches typically produce only single point predictions. Consequently, they do not indicate how reliable an individual estimate is or how much uncertainty it carries. This limitation is particularly problematic in safety-critical emergency medical service dispatch operations, where trips with similar expected travel times may exhibit substantially different levels of uncertainty. Furthermore, real-world ambulance trip logs may contain domain-specific artefacts, such as co-located healthcare facilities and physically implausible travel speeds. These artefacts can degrade point prediction accuracy and affect the prediction task on which subsequent uncertainty estimation depends. To address these challenges, we propose an uncertainty-aware methodology for ambulance travel time estimation and evaluate it through a retrospective real-world scenario study using two years of ambulance trip data from the Sk{\aa}ne region in Sweden. The proposed methodology moves beyond average-case point predictions by generating context-adaptive prediction intervals with stable empirical coverage across the evaluated operationally important contexts. The resulting prediction intervals provide explicit information about predictive uncertainty that can support more transparent and risk-sensitive ambulance dispatch decision-making.

1 Introduction

Ambulance travel time estimates support high-stakes dispatch decisions, but point predictions omit uncertainty and trip records can contain operational artefacts. The paper proposes a curated prediction workflow with conformal intervals and evaluates it using two years of Skåne ambulance data.

1 Introduction

  • Ambulance travel time estimation informs unit assignment and how quickly prehospital care may reach patients, especially in high-priority incidents.The introduction notes that delays in such incidents may have clinical consequences.Source: §1 · pp. 1–2
  • Point predictions omit individual-estimate uncertainty, while similar expected travel times can have substantially different uncertainty in safety-critical dispatch.Uncertainty information may help dispatchers assess whether an estimate is reliable or whether to consider additional operational information.Source: §1 · p. 2
  • Co-located facilities and trips with implausible speeds can distort the prediction task, motivating domain-informed curation before model development.The stated criteria remove trips between facilities sharing coordinates and trips with implausible average speeds.Source: §1 · pp. 2–3
  • The proposed methodology combines domain-informed curation, feature preparation, point prediction, and conformal calibration to assess reliability beyond point-prediction accuracy.The paper identifies deterministic prediction, operational data quality, and subgroup reliability as limitations it addresses.Source: §1 · p. 3
  • The retrospective Skåne study uses two years of ambulance trips and compares three conformal calibration strategies across operationally relevant contexts.The evaluation examines empirical coverage as well as point-prediction performance.Source: §1 · p. 3

2 Related Work

Prior ambulance travel-time research has increasingly used data-driven models, but these approaches generally return deterministic predictions and assume records are free of operational artefacts. The paper addresses gaps in uncertainty estimation and data curation for ambulance travel-time prediction.

2 Related Work

  • Machine-learning methods using GPS trajectories and spatiotemporal features improve prediction accuracy, but almost exclusively produce deterministic point predictions and generally assume artefact-free records.Earlier approaches include lookup tables, fixed-speed assumptions, probabilistic models, and commercial navigation services.Source: §2 · p. 3
  • Conformal prediction offers distribution-free intervals with finite-sample marginal coverage under exchangeability, but its ambulance travel-time application remains largely unexplored.The broader travel-time literature also considers Bayesian neural networks, quantile regression, and ensemble-based intervals.Source: §2 · p. 4
  • The paper responds by combining operational data curation with calibrated prediction intervals in a retrospective real-world scenario study.Source: §2 · p. 4

3 Methodology for Uncertainty-Aware Ambulance Travel Time Estimation

The methodology builds ambulance travel-time intervals through domain-informed curation, dispatch-time feature preparation, point prediction, and conformal calibration. It permits different prediction models and considers global, quantile-based, and priority-stratified calibration.

3.1 Domain-Informed Data Curation

  • Domain-informed curation removes trips between facilities sharing coordinates and trips whose implied average speeds fall outside a setting-specific plausible range.The criteria are set a priori from EMS operational knowledge rather than predictive performance.Source: §3.1 · p. 4

3.2 Feature Engineering and Preprocessing

  • Feature preparation represents dispatch-time spatial, temporal, operational, and route information for the selected prediction model.Preprocessing transformations are fitted using training data only to avoid information leakage.Source: §3.2 · pp. 4–5

3.3 Point Prediction Model

  • Point prediction is a supervised regression stage that can use different candidate models selected on data independent of the final test set.The methodology is not tied to a particular regression algorithm.Source: §3.3 · p. 5

3.4 Uncertainty-Aware Ambulance Travel Time Estimation

  • Conformal calibration complements a point estimate with an interval, with finite-sample marginal coverage guarantees under exchangeability.Global split conformal uses a finite-sample-adjusted quantile of absolute calibration errors as its margin.Source: §3.4 · pp. 5–6
  • The methodology evaluates global split conformal, CQR, and priority-stratified Mondrian calibration, with CQR intervals based on conditional quantiles.Global split conformal uses one margin for every prediction, whereas Mondrian calibration uses a separate quantile for each priority group.Source: §3.4 · pp. 6–7

4 Real-World Scenario Study and Results

The study instantiates the methodology using 2020–2021 ambulance records from Skåne, Sweden, and examines travel-time variability across operational contexts. Its exploratory analysis shows that variability differs by context and that aggregate interval coverage can conceal poor coverage for rare routes.

4.1 Scenario Description

  • The retrospective Skåne study examines data curation, point prediction, and prediction-interval calibration using ambulance trips recorded during 2020–2021.The dataset contains operational trip information; the scenario features include origin and destination areas, priority, day and hour, route distance, and a precomputed Google travel-time estimate.Source: §4.1 · p. 7, §4.2 · p. 8

4.2 Dataset

  • Curation reduced the dataset from 22,360 to 20,738 operationally valid trips by addressing co-located facilities and trips outside a plausible 5–150 km/h speed range.Cancelled trips and incomplete mission segments were also excluded.Source: §4.2 · p. 8

4.3 Exploratory Data Analysis

  • Very rare routes had a coefficient of variation (CV) of 0.592, versus 0.306–0.425 for routes observed more than 100 times, indicating higher relative variability.Across 234 groups containing at least 20 trips sharing the same route and hour, the median within-group standard deviation was approximately 7 min.Source: §4.3 · pp. 8–9
  • A global 95% reference interval covered 96.5% of trips overall, but only 69.9% of rare-route trips and 44.2% of very rare-route trips.Travel times were strongly right-skewed: half of trips took at most 31 min, while 5% exceeded 70 min and 1% exceeded 94 min; some long trips may reflect delayed event registration.Source: §4.3 · pp. 8–9

4.4 Experimental Setup and Evaluation Protocol

The evaluation compares candidate point-prediction models and assesses conformal calibration on independent data partitions. It also examines temporal variability and measures single-trip inference latency.

4.4 Experimental Setup and Evaluation Protocol

  • Point-prediction candidates were evaluated with five-fold cross-validation on 85% of the data, then tested once on an independent 15% split using MAE, RMSE, MAPE, and R^2.Candidates included polynomial linear regression, random forest, two CatBoost variants, and an ANN.Source: §4.4 · p. 9
  • Uncertainty evaluation used 70% training, 15% calibration, and 15% test data, with calibration assessed primarily at 90% nominal coverage and also from 50% to 99%.Results were assessed across repeated random partitions; Fig. 3 maps travel-time standard deviation by hour and weekday, with darker cells indicating greater variability.Source: §4.4 · p. 10, Figure 3 · p. 10
  • Average end-to-end inference latency was approximately 2 ms per trip on the stated hardware.The measurement includes feature preprocessing, point prediction, and calibrated prediction-interval generation.Source: §4.4 · p. 10

4.5 Effect of Domain-Informed Data Curation on Point Prediction Performance

The study compares progressively curated prediction tasks to assess how domain-informed data cleaning affects point-prediction performance. The operationally valid formulation performed best overall against the naive baseline.

4.5 Effect of Domain-Informed Data Curation on Point Prediction Performance

  • The controlled comparison used the same ANN architecture, input features, preprocessing, and evaluation protocol across naive, spatially disambiguated, and operationally valid formulations.The final formulation additionally excluded trips with implied average speeds outside 5–150 km/h.Source: §4.5 · pp. 10–11
  • Operationally valid curation reduced MAPE by 17.0%, MAE by 8.5%, and RMSE by 16.0%, while raising R^2 from 0.62 to 0.74 versus the naive baseline.The reported improvements concern point prediction and do not by themselves establish improved uncertainty calibration.Source: Table 3 · p. 11, §4.5 · p. 11
  • The intermediate spatially disambiguated formulation lowered MAE and RMSE and increased R^2, although MAPE rose slightly; overall curation removed 7.3% of original records.The authors note that removing degenerate short trips may change the distribution used to calculate percentage errors.Source: §4.5 · p. 11

4.6 Point Prediction Performance

Among the evaluated point-prediction models, the ANN achieved the best overall performance and was selected for subsequent uncertainty-estimation experiments in this scenario.

4.6 Point Prediction Performance

  • The ANN achieved MAPE 10.35%, MAE 4.42 min, RMSE 9.12 min, and R^2 0.74, with CatBoost performing similarly.Polynomial linear regression produced substantially larger prediction errors; the ANN was selected for subsequent uncertainty-estimation experiments in this scenario.Source: §4.6 · p. 11

4.7 Evaluation of the Proposed Uncertainty-Aware Methodology

At nominal 90% coverage, the three conformal strategies achieved empirical coverage close to target, while the Gaussian baseline produced wider intervals. The conformal methods also provide finite-sample marginal coverage guarantees under exchangeability.

Q1. Do the conformal calibration strategies achieve empirical coverage close to the desired nominal coverage?

  • The Gaussian baseline achieved 93.2% empirical coverage but produced the widest intervals, with a mean width of 24.7 min.Its coverage exceeded the nominal target, but its intervals were wider than those from the conformal methods.Source: §4.7 · pp. 11–12
  • 89.1% empirical coverage for global split conformal, 90.5% for CQR, and 89.0% for priority-stratified Mondrian conformal were all close to the nominal 90% target.The conformal procedures additionally provide finite-sample marginal coverage guarantees under the exchangeability assumption.Source: §4.7 · p. 12

Q2. How well calibrated are the prediction intervals across different nominal coverage levels?

The three conformal strategies closely tracked ideal calibration across nominal coverage levels from 50% to 99%. Across distance quartiles, CQR maintained more stable empirical coverage than the fixed-margin strategies.

Q2. How well calibrated are the prediction intervals across different nominal coverage levels?

  • Across nominal levels from 50% to 99%, all three conformal strategies closely followed the ideal calibration diagonal.The evaluation also found mean empirical coverages of 89.8%–89.9% at nominal 90% across twenty random partitions.Source: Q2. How well calibrated are the… · p. 12, Figure 4 · p. 12

Q3. How stable is empirical coverage across operationally relevant contexts?

  • Across dispatch priorities, all three strategies achieved empirical coverage between approximately 86% and 92%.Source: Q2. How well calibrated are the… · p. 12
  • CQR maintained 88.9%–91.6% empirical coverage across trip-distance quartiles, while allowing interval widths to increase for trips with greater predictive uncertainty.Global split and priority-stratified Mondrian conformal showed overcoverage for shorter trips and undercoverage for longer trips, reaching approximately 83% for the longest-distance quartile.Source: Q2. How well calibrated are the… · p. 13
  • Satisfactory marginal coverage does not necessarily imply equally stable empirical coverage across operational subgroups.Source: Q2. How well calibrated are the… · p. 13

5 Discussion

The discussion frames calibrated prediction intervals as a way to make uncertainty explicit for dispatch support, while retaining human oversight. Its evidence is retrospective and regional, and the method cannot account for rare events outside training conditions.

Implications for AI-Enabled Dispatch Systems

  • Calibrated prediction intervals make predictive uncertainty explicit and allow empirical coverage to be evaluated against a predefined target as operational outcomes become available.The authors suggest calibration guarantees and empirical coverage across operational contexts could complement conventional accuracy metrics for safety-critical dispatch support.Source: §5 · p. 13

Operational Self-Sufficiency

  • The methodology supports self-hosted inference using dispatch-available features and precomputed routing attributes, without external routing services at prediction time.Adding real-time traffic or other contextual inputs would reintroduce external dependencies, trading richer context against operational self-sufficiency.Source: §5 · pp. 13–14

Limitations and Human-in-the-Loop Governance

  • The methodology cannot account for rare or unprecedented events outside training conditions, such as major road closures or simultaneous casualty incidents.The authors position uncertainty estimates as support for discretionary human review rather than automated dispatch decisions.Source: §5 · p. 14
  • The retrospective evaluation covered the Skåne region and 2020–2021; the authors identify external validation and prospective human-centered evaluation as next steps.Source: §5 · p. 14

6 Conclusions and Future Work

The retrospective Skåne study found that domain-informed data curation improved point prediction performance and that CQR had the most stable distance-quartile coverage among evaluated strategies. The authors identify richer contextual information as future work.

6 Conclusions and Future Work

  • The evaluation used a retrospective real-world scenario study with two years of ambulance trip data from the Skåne region in Sweden.Source: §6 · pp. 14–15
  • R² increased from 0.62 to 0.74 after domain-informed data curation, while fixed-margin strategies reached approximately 83% coverage for the longest trips.CQR was most stable across distance quartiles, maintaining 88.9–91.6% coverage at the cost of moderately wider intervals.Source: §6 · p. 15
  • Future work will incorporate richer contextual information, including real-time traffic, weather, and incident-specific characteristics.Source: §6 · p. 15
Loading 2610.31944v1…