Mind the Long Tail: Understanding the Difficulty of Delay Detection in Business Processes

Keyvan Amiri Elyasi; Lukas Kirchdorfer; Heiner Stuckenschmidt

Source-linked AI summary

Mind the Long Tail: Understanding the Difficulty of Delay Detection in Business Processes

Keyvan Amiri Elyasi, Lukas Kirchdorfer, Heiner Stuckenschmidt

arXiv:2608.14367v1Aug 14, 2026cs.LGcs.AI

TL;DR

Delay detection in predictive process monitoring remains difficult to assess beyond aggregate performance, especially for rare cases with large delays. Across 14 event logs, the paper analyzes this difficulty and finds that uncertainty-aware models substantially improve delayed-case identification, while imbalance-focused approaches yield limited benefits.

Problem
Evidence on delay detection in remaining-time prediction is limited, particularly for the strongly skewed, long-tailed regression targets in this domain.
Method
The paper analyzes delay-detection difficulty across 14 event logs and evaluates imbalanced-regression and uncertainty-aware modeling approaches.
Results
Remaining times are strongly right-skewed, models struggle on rare high-delay cases, uncertainty increases with delay magnitude, and uncertainty-aware models substantially improve delay detection.
Takeaways & Limitations
The findings identify uncertainty-aware modeling as a promising direction for improving delayed-case identification in predictive process monitoring.
Takeaways & Limitations
The underlying sources of predictive uncertainty remain unresolved, potentially involving process variability, missing context, or concept drift.

Abstract

from arXiv · show

The early detection of delayed cases in business processes is a critical capability for organizations. Predictive process monitoring (PPM) supports this task by using historical event logs to predict the remaining time of ongoing cases, enabling timely interventions to avoid missed deadlines and service level violations. Although remaining time prediction has advanced considerably through sophisticated deep learning architectures, little is known about the intrinsic difficulty of delay detection itself. Since performance is typically assessed using aggregate metrics, prior work provides limited insight into how models perform across the target distribution, especially on the operationally most critical cases with large delays. In this paper, we address this gap by analyzing the difficulty of delay detection. Across 14 event logs, we show that remaining times are typically strongly right-skewed, with only a small fraction of cases exhibiting large delays. Existing models capture the mode of this distribution well but perform poorly on high-delay cases. We further uncover pronounced heteroscedasticity, showing that predictive uncertainty increases with delay magnitude. Based on these findings, we evaluate approaches to mitigate the imbalance problem, but find only limited benefits, suggesting that the key underlying problem may not be imbalance but higher uncertainty associated with delayed cases. We show that this correlation can be exploited to substantially improve the identification of delayed cases. Overall, our work provides new insights into the sources of difficulty in delay detection and identifies uncertainty-aware modeling as a promising direction for future PPM research.

1 Introduction

Business-process delays can cause serious operational and customer consequences, making early identification of delay-prone cases an essential goal of predictive process monitoring. This paper addresses the limited understanding of delay-detection difficulty by showing that rare, highly delayed cases are difficult to predict and that uncertainty-aware models can improve their identification.

Introduction: Execution delays can cause missed deadlines, service-level violations, higher operational costs, and diminished customer satisfaction.These consequences motivate the need for early identification of delay-prone cases.
Introduction: Remaining time prediction uses observed case prefixes and historical event logs to estimate completion time and support timely intervention.It provides an actionable signal for identifying potentially delayed cases.
Introduction: Aggregate metrics such as mean absolute error can obscure model failures on rare delayed cases, despite their operational importance.Recent progress includes LSTMs, Transformers, and graph neural networks, but these approaches are typically evaluated with aggregate metrics.
Introduction: Remaining-time distributions are typically strongly right-skewed, with most cases completing within an expected timeframe and a small number exhibiting extreme delays.The paper investigates whether existing models can reliably detect these rare cases and the intrinsic difficulty of doing so.
Introduction: Existing approaches capture the distributional mode well but fail to predict cases with large delays accurately, while predictive uncertainty increases with remaining-time magnitude.The paper studies whether this relationship can improve delayed-case identification.
Introduction: Uncertainty-aware models can substantially improve the identification of delayed cases.This is presented as a central result of the paper’s contribution.

2 Motivation and Analysis

Delay detection is difficult because remaining-time distributions are strongly right-skewed: models capture the typical cases but miss rare, high-delay cases. Tail cases are also more uncertain and harder to predict, reflecting both data scarcity and increased noise.

Illustration: The ground-truth remaining-time distribution is strongly right-skewed, with most observations near 10 days and only a few extending to 40–100 days.Positive skewness indicates a long right tail, with larger values representing stronger asymmetry.
Illustration: The standard LSTM compresses predictions into a narrower range, never predicting above 35 days and missing high-delay cases.A case requiring 80 days could receive a prediction of only 30–35 days.
Illustration: Prediction error increases substantially toward the right tail, making predictions ineffective for timely delay identification and intervention.The model’s compressed predictions produce especially large errors for long-running cases.
Cross-log analysis: In 10 out of 14 event logs, both absolute prediction error and prediction-interval width positively correlate with true remaining time.Moderate-to-strong correlations in several logs indicate that long-running cases are substantially noisier than short-running ones.
Cross-log analysis: Tail difficulty reflects both target imbalance and increased noise: long-delay cases are rare, substantially more variable, and harder to predict.The heteroscedastic pattern is clear but not universal across all event logs.

3 Problem Setting and Experimental Setup

The paper formulates remaining-time prediction as supervised regression from event-log prefixes and evaluates it using a temporal split across 14 event logs. Experiments use a data-aware LSTM and region-based nMAE normalized against a constant median predictor.

Problem Definition: Event logs contain chronological traces of process cases, while prefixes represent partial execution up to an event and form the model inputs.Each event includes a case identifier, activity, timestamp, and optional attribute-value map; the resulting dataset is D = {(x_i, y_i)}^N_i=1.
Problem Definition: Remaining-time prediction maps feature inputs x ∈ R^d to targets y ∈ R through an encoder–head regression model trained with supervised learning.The encoder produces a latent representation z = g(x, θ), and the head predicts remaining time as ŷ = h(z, ψ).
Experimental Setup: 14 event logs are evaluated using a temporal hold-out split of 64% training, 16% validation, and 20% test data.The setup uses a data-aware LSTM that predicts remaining time from event prefixes encoded as feature-vector sequences.
Experimental Setup: The target distribution is divided into few-shot, medium-shot, and many-shot regions comprising the highest 10%, next 30%, and remaining 60% of target values.These regions support comparisons across different target-value frequencies and cycle-time scales.
Experimental Setup: nMAE normalizes regional mean absolute error by the mean absolute deviation from the training-set median, making 1 the constant median predictor baseline.This enables comparison of prediction difficulty and model performance across regions and datasets on a common scale.

4 Assessing the Effectiveness of Imbalanced Regression

Imbalanced-regression methods provide limited overall benefit for remaining-time prediction: reshaping the training distribution can improve rare delayed cases while worsening common cases, and algorithm-level gains similarly involve trade-offs.

Data-level approaches: Data-level methods resample frequent and rare target regions, but prefix structure makes naive interpolation liable to create unrealistic event sequences.Oversampling may interpolate neighboring observations or perturb rare samples, yet left-padded prefixes depend on feature positions, lengths, and admissible process behavior.
Data-level approaches: Across BPIC20PTC configurations, SMOGN shifted probability mass toward higher remaining times without improving performance, with MAE between 8.1 and 9.4 days.Increasing oversampling of rare cases and undersampling of frequent cases increased delayed-case density but did not improve predictive performance.
Data-level approaches: 8.0% lower few-shot nMAE from SMOGN was outweighed by 20.5% and 31.4% higher medium- and many-region errors, producing 17.5% higher overall nMAE.Results are averaged over five seeds per log across all 14 event logs and compared with the same LSTM trained on original data.
Algorithm-level approaches: Algorithm-level methods modify the regression objective through cost-sensitive weighting, distribution-balanced objectives, error-aware losses, or relevance-based error measures.The approaches assign greater weight to underrepresented or highly relevant target regions and reduce the contribution of low-relevance errors.
Algorithm-level approaches: SERA ranked best in the few-shot region at 1.00 and significantly outperformed Vanilla, CSW, and EAL, whereas Vanilla ranked best in the many-shot region at 1.64.SERA underperformed relative to other approaches in many- and medium-shot regions, and its gains did not translate to overall performance.
Algorithm-level approaches: Algorithm-level improvements were not consistent in the long tail: SERA could aid high-delay identification, but gains came with worse many-shot and overall accuracy.Feature and label distribution smoothing produced only marginal and inconsistent effects.

5 Assessing the Effectiveness of Uncertainty Modeling

The section evaluates survival-based uncertainty modeling for delay detection and finds that uncertainty-aware classification substantially improves identification of delayed cases while preserving precision. Benefits persist under stricter delay definitions and are strongest at early stages of execution.

Modeling Uncertainty via Survival Analysis: Survival analysis models a distribution over remaining times instead of a single estimate, enabling expected-time, prediction-interval, and uncertainty extraction.The remaining-time horizon is discretized into K quantile-based intervals, with hazards trained using the negative log-likelihood of observed event times.
Delay Detection via Uncertainty Modeling: The uncertainty-aware classifier combines distributional summaries, uncertainty measures, tail mass, and temporal context to predict whether a case exceeds a delay threshold.For the main evaluation, delayed cases exceed the q = 0.8 quantile of training and validation cycle times.
Results: Uncertainty-aware modeling substantially increases recall over the point-estimate baseline, which misses most delayed cases, while maintaining competitive precision.For example, BPIC17W precision rises from 0.48 to 0.77, with no statistically significant overall precision difference (p = 0.17).
Results: F1-score consistently improves across datasets, and PR-AUC improves in nearly all cases, indicating better balance and ranking of delayed cases.The PR-AUC improvements are statistically significant according to the reported Wilcoxon test.
Earliness Analysis: The uncertainty-aware model outperforms the baseline across all prefix ratios, with the most pronounced recall gains at early execution stages.Performance is evaluated for prefixes covering up to 20%, 40%, 60%, and 80% of each case’s full execution trace.

6 Prior Work

Prior PPM research addresses imbalance mainly through classification methods, while imbalanced regression in remaining-time prediction remains largely unexplored. Emerging uncertainty-aware approaches provide complementary information but have not examined imbalance or its interaction with predictive uncertainty.

Imbalanced learning in PPM: Data imbalance affects multiple PPM tasks, including next activity prediction, outcome prediction, and anomaly detection, where rare operationally critical events are underrepresented.Examples include exceptional activities, negative outcomes, and anomalies.
Imbalanced learning in PPM: Existing PPM work primarily applies imbalanced-classification techniques, including cost-sensitive learning, focal-style losses, and oversampling or resampling.Approaches span data-level methods, algorithm-level methods, and hybrid combinations.
Imbalanced learning in PPM: Imbalance in regression tasks such as remaining-time prediction has not been systematically addressed despite strongly skewed, long-tailed target distributions.Prior work focuses on classification settings rather than regression.
Uncertainty-aware modeling: Uncertainty-aware remaining-time models quantify information beyond point estimates but do not explicitly address imbalance or its interaction with predictive uncertainty.This work responds by studying imbalanced regression and uncertainty-aware modeling as complementary perspectives for delay detection.

7 Conclusion

The paper finds that delay detection is intrinsically difficult because remaining times are right-skewed and uncertainty rises for rare, high-delay cases. It identifies uncertainty-aware modeling as promising while highlighting unresolved uncertainty sources and information loss from binary reformulation.

Main findings: Remaining times are typically strongly right-skewed, with existing models performing well on normal cases but struggling on rare, operationally critical high-delay cases.The analysis characterizes this as target imbalance in the delay-detection problem.
Main findings: Predictive uncertainty increases with delay magnitude, revealing pronounced heteroscedasticity in the distribution’s tail.The paper reports that exploiting predictive uncertainty can substantially improve delay detection.
Limitations and future work: The sources of predictive uncertainty remain unresolved and may include process variability, missing context, or concept drift.Future research could pursue richer uncertainty modeling, additional contextual information, and online adaptation in changing process environments.
Limitations and future work: Reformulating remaining-time prediction as binary delay detection loses information and makes results dependent on the chosen threshold.This limitation motivates further work on delay-detection methods beyond threshold-dependent binary reformulation.