Source-linked AI summary
Efficient Online Continual Foundation Model Fine-Tuning for Predictive Process Monitoring
Sjoerd van Straten, Marwan Hassani
TL;DR
PPM must adapt to shifting process distributions, but existing online methods often train task-specific networks from scratch, leaving a cold-start problem. COMPASS continually fine-tunes a foundation model with autonomous drift detection and a unified orthogonal knowledge subspace, outperforming competitors across nine event streams, with gains up to 19% on recurrent and complex streams.
Problem
Existing online continual PPM methods often train compact task-specific networks from scratch, while foundation-model continual fine-tuning for process mining remains unexplored.
Method
COMPASS detects task boundaries from stabilized loss and continually fine-tunes a foundation model using a unified subspace containing pre-trained and task-specific directions.
Results
COMPASS outperforms five competitors and baselines across nine event streams, with gains of up to 19% on recurrent drifts and complex, long-running cases.
Takeaways & Limitations
The results support foundation-model continual fine-tuning as an effective approach for next activity prediction under diverse process concept drifts.
Takeaways & Limitations
The study focuses on next activity prediction and does not empirically evaluate extension to other PPM tasks.
Abstract
from arXiv · showhide
Predictive Process Monitoring (PPM) models are increasingly deployed in dynamic environments where concept drift causes the underlying process distribution to shift over time. While recent work has moved toward online continual learning, existing methods train compact, task-specific networks entirely from scratch, leaving a persistent cold-start problem. Foundation Models (FMs) offer a compelling solution to this problem, but their continual fine-tuning in the process mining domain remains unexplored. We propose COMPASS (Continual Online foundation Model-based PPM with Adaptive SubSpaces), the first framework for online continual fine-tuning of FMs for PPM. COMPASS adapts loss-plateau drift detection to autonomously identify task boundaries in event streams and maintains a unified knowledge subspace including both pre-trained and task-specific directions. We evaluate our approach on nine event streams covering synthetic and real-world concept drift scenarios, across task-free and task-aware settings with multiple backbones and with consistent hyperparameter tuning across all methods. Our approach outperforms three SOTA non-FM competitors and two update strategy baselines, with particularly strong gains on streams exhibiting recurrent drift and complex, long-running cases, while incurring acceptable computational overhead compared to the non-FM competitors.
1 Introduction
PPM operates in dynamic organizational environments where concept drift challenges online adaptation, while existing methods face cold-start and stability-plasticity limitations. COMPASS addresses these gaps by continually fine-tuning foundation models with adaptive drift detection and a unified knowledge subspace.
- Concept drift changes process distributions over time because of regulations, seasonality, or resource availability.
- Existing online methods often train task-specific networks from scratch, creating a persistent cold-start problem and risking catastrophic forgetting.
- Long-running, interleaved cases and organizational changes make process-stream drift harder to detect autonomously than visually distinct image-stream task shifts.
- Foundation models can transfer sequence-modeling capabilities to process traces, but no process-mining foundation models exist and vanilla LoRA does not preserve prior knowledge.
- COMPASS detects stabilized loss, expands a unified subspace containing pre-trained and task-specific directions, and reinitializes adapters in the orthogonal residual space.
- The framework is evaluated on nine event streams across synthetic and real-world drifts, task-free and task-aware settings, multiple backbones, and several competitors and baselines.
2 Preliminaries and Problem Definition
The paper formalizes next activity prediction in non-stationary event streams and frames continual fine-tuning as balancing current-task plasticity with backward and forward stability. In task-free learning, task boundaries must be inferred from the stream.
- A process trace is a finite sequence of activity labels, and next activity prediction uses a prefix to predict its following ground-truth activity.
- Online event streams are timestamp-ordered into contiguous, non-overlapping windows, while concept drift permits the joint distribution P(X, Y) to change over time.
- The model follows test-then-train evaluation and minimizes cross-entropy loss over each event window.
- Parameter-efficient fine-tuning updates a low-dimensional increment ∆W while leaving the pre-trained base weights W0 nominally frozen.
- Continual learning must adapt to the current task while minimizing degradation on previous tasks and preserving general sequence understanding in W0.
- The task-free setting requires unknown task boundaries to be inferred directly from the event stream.
3 Related Work
Prior PPM work progressed from periodic offline retraining toward online adaptation, but existing approaches generally rebuild task-specific models from scratch. Foundation-model continual fine-tuning introduces parameter-efficient adaptation while leaving knowledge-preservation challenges to address.
- Early PPM maintenance methods used periodic, batch-oriented updates and showed that drift-aware incremental learning suits evolving processes.
- These earlier methods trained task-specific models from scratch, foregoing pre-trained structural knowledge and sequence understanding.
- Online PPM methods adapt to streaming data, but DynaTrainCDD retrains a single dense-layer backbone from scratch at detected drift points and does not address catastrophic forgetting.
- LoRA freezes base weights and injects trainable low-rank matrices, reducing trainable parameters by orders of magnitude without additional inference latency.
- Continual LoRA research extends parameter-efficient fine-tuning toward task-free online adaptation, motivating foundation-model approaches for PPM.
4 Method
COMPASS combines LoRA-based online fine-tuning with plateau-based drift detection and adaptive knowledge-subspace management. It preserves pre-trained and previously learned directions while initializing new adapters in residual subspaces to balance stability and plasticity.
- Framework overview: COMPASS adapts foundation models for next activity prediction through LoRA fine-tuning, autonomous plateau-based drift detection, knowledge-subspace expansion, and adapter reinitialization.These stages are presented as the framework’s main functional pipeline.
- LoRA fine-tuning: The model predicts each event window before updating its weights, while frozen base weights and low-rank adapters enable parameter-efficient online adaptation.Cross-entropy loss is optimized over each window, reducing trainable parameters by orders of magnitude.
- Drift detection: When the rolling variance of recent window losses falls below τ, COMPASS treats the loss plateau as a captured task distribution and triggers knowledge consolidation.A cooldown period of C windows prevents redundant triggers within the same task.
- Knowledge subspace expansion: The unified subspace M′_t combines fixed pre-trained directions Wp with task-specific directions Mt accumulated across observed tasks.Its columns are capped at maximum rank R to bound memory growth.
- Knowledge subspace expansion: New task-specific directions are isolated by projecting activations orthogonally to Wp and Mt−1, then retaining dominant residual directions selected by an energy threshold.The retained directions are appended to Mt and re-orthogonalized before forming the unified subspace.
- Adapter reinitialization: At consolidation, the LoRA adapter is reinitialized orthogonally to prior knowledge while approximating the new task’s gradient, and subsequent updates remain orthogonal to Wp.This construction is intended to prevent overwriting learned process knowledge and preserve pre-trained sequence understanding.
5 Experimental Setup
The evaluation covers nine event streams with synthetic recurrent drifts and real-world BPI Challenge data, comparing COMPASS with update strategies and three non-foundation-model continual-learning competitors. Experiments include multiple backbones, task-free and task-aware settings, validation-based hyperparameter tuning, and online test-then-train evaluation.
- Datasets and metrics: Nine event streams comprise five synthetic recurrent-drift logs and four real-world streams from the BPI Challenges.The synthetic streams are IRO5000, ORI5000, ROI5000, OIR5000, and RIO5000.
- Baselines: COMPASS is compared with DoNothing, γ = LastDrift, DynaTrainCDD, TFCLPM, and CNAPwP.The baselines include two update strategies and three state-of-the-art non-FM competitors.
- Implementation details: Experiments use Tiny-LLM 4 with 10M parameters and DistilGPT2 with 82M parameters, including task-aware oracle boundaries and task-free plateau detection.DoNothing and γ = LastDrift use the DistilGPT2 backbone for strategy comparisons.
- Evaluation protocol: The first 15% of each dataset is used for validation, while the remaining 85% is reserved for online evaluation under a test-then-train protocol.A paired two-sided Student’s t-test with α = 0.05 compares the best COMPASS configuration with the best competing baseline per experiment.
6 Evaluation Results
Across nine event streams, COMPASS achieves strong next activity prediction accuracy, especially under recurrent and complex drift, while maintaining acceptable computational overhead. Its full subspace design is necessary for the best average accuracy, and task-free detection matches task-aware performance.
- Average Accuracy: COMPASS DistilGPT2 achieves the highest accuracy on seven of nine evaluated logs, with a +19% gain over CNAPwP on recurrent BPI15-REC.The strongest gains occur on streams with recurrent drift and complex, long-running cases.
- Accuracy at Given Index: COMPASS maintains higher accuracy from the start and recovers faster after concept drift than the baselines on BPI15-REC and ORI5000.Task-free and task-aware variants follow nearly identical trajectories.
- Computational Efficiency: COMPASS DistilGPT2 incurs moderate computational cost because of its larger backbone and SVD-based consolidation, while remaining within practical runtime bounds.Periodic full retraining can have comparable or higher runtime while underperforming COMPASS in accuracy.
- Computational Efficiency: Memory usage remains stable after the initial Wp extraction because the rank cap bounds subspace growth regardless of the number of encountered tasks.The one-time peak is caused by SVD-based extraction of Wp.
- Ablation Study: The full Wp and Mt combination reaches .802 average accuracy in both task-free and task-aware settings, whereas removing both reduces accuracy to .735 and .758.Wp alone slightly lowers performance to .721/.747, indicating that the two protections are complementary.
- Ablation Study: The task-free and task-aware variants achieve identical average accuracy, showing no meaningful performance penalty from plateau-based boundary detection.This result supports autonomous task-boundary detection relative to oracle boundaries.
7 Conclusion and Future Work
COMPASS addresses cold-start and stability-plasticity challenges by continually fine-tuning a foundation model within an orthogonal knowledge subspace. Across nine streams, it outperforms competitors with gains up to 19%, while its limitations include untested semantic transfer and evaluation restricted to next activity prediction.
- Conclusion: COMPASS continuously fine-tunes a foundation model within a growing orthogonal knowledge subspace to preserve pretrained and previously learned process behaviors.The framework is evaluated across nine event streams and outperforms five competitors and baselines.
- Conclusion: COMPASS achieves gains of up to 19% on streams with recurrent drifts and complex, long-running cases.The task-free variant matches oracle performance across all evaluated datasets.
- Limitations and Future Work: Semantic transfer from language pretraining to integer-tokenized activities remains untested, and evaluation covers next activity prediction only.Future work includes adaptive hyperparameter tuning, multimodal inputs, and richer event attributes.