Source-linked AI summary
TSPFN: A Temporal Tabular Foundation Model for Physiological Time Series Classification
Jérémie Stym-Popper, Clément Rambour, Federica Granese, Nicolas Thome, Olivier Bernard
TL;DR
Physiological time-series classification remains difficult in low- to medium-data regimes, while TabPFN does not model the temporal and inter-channel structure central to these signals. TSPFN redesigns TabPFN with structured temporal and channel representations, and experiments report stronger performance than tabular and specialized deep baselines across diverse physiological benchmarks.
Problem
Low- to medium-data physiological time-series classification is challenged by limited labels, heterogeneous dynamics, and poor generalization across datasets, modalities, and clinical settings.
Method
TSPFN redesigns TabPFN with structured temporal representations, channel-wise positional embeddings, and pretraining on nearly 140,000 real-world physiological time series.
Results
TSPFN significantly outperforms XGBoost, TabPFN, TCN, MiniRocket, and LaBram on average across five low-data datasets, with AUROC comparison versus MiniRocket yielding p = 3.7 × 10−5.
Takeaways & Limitations
TSPFN generalizes effectively across unseen physiological datasets despite heterogeneous pretraining signals and supports in-context classification in small- and medium-data regimes.
Abstract
from arXiv · showhide
Designing models that generalize effectively in low- to medium-data regimes remains a primary challenge in medical machine learning, particularly for physiological time-series classification. While tabular foundation models such as TabPFN offer an attractive alternative to conventional fine-tuning through in-context learning, they are not designed to capture the temporal dependencies inherent to physiological signals. ~In this paper, we introduce TSPFN, a foundation model that redesigns TabPFN's architecture for time series data. TSPFN integrates structured temporal representations and positional embeddings to capture intra-sample temporal and channel dependencies. To fully leverage its spatio-temporal design, the model is pretrained on 140,000 real-world physiological time series across multiple medical domains. This yields a unified, generalizable framework capable of learning the specificities of medical time series. Experiments across diverse physiological benchmarks demonstrate that TSPFN consistently outperforms standard tabular baselines and TabPFN, and achieves superior cross-domain generalization compared to specialized deep time-series models. All our experiments, ablation studies, and pre-processing scheme are publicly available at https://github.com/Jeremstym/TSPFN
1 Introduction
Physiological time-series classification is difficult in low- to medium-data settings because clinical recordings vary across subjects, temporal dynamics, and datasets. TSPFN adapts tabular foundation modeling to these temporal and channel structures and is evaluated across diverse physiological tasks.
- Limited labeled recordings, heterogeneous temporal dynamics, and strong inter-subject variability hinder robust physiological time-series classification.
- Task-specific models trained on small or moderately sized cohorts often fail to generalize across datasets, modalities, or clinical settings.
- TabPFN supports in-context learning without gradient-based fine-tuning, but permutation-invariant features and synthetic pretraining limit its applicability to medical time series.
- TSPFN revisits TabPFN with structured input representations and channel-wise positional embeddings to model temporal dependencies in multivariate physiological time series.
- The study builds a unified pretraining collection of approximately 140,000 physiological samples and benchmarks TSPFN across five diverse datasets.
2 Background
Prior-Data Fitted Networks are transformer-based models pretrained on task distributions and used for Bayesian-style prediction through in-context learning.
- PFNs map labeled context examples to predictive distributions for unseen query samples without parameter updates at test time.
- PFNs are pretrained on synthetic datasets sampled from a prior distribution over tasks.
3 Method: TSPFN
TSPFN adapts TabPFN to physiological time series by replacing unordered feature treatment with explicit temporal and channel organization. Its embeddings model temporal order, channel identity, and dependencies within and across channels.
- Real-world physiological data are used because generating realistic synthetic time series with labels is difficult for this domain.
- TSPFN restructures each sample into a tabular row representing patient-specific time series and labels, while accommodating variable lengths and channel counts within Fmax = 500 features.
- Channel-wise RoPE injects temporal positions into query and key representations before attention, allowing relative temporal offsets to be modeled across variable-length sequences.
- Learnable channel identity embeddings are added to representations across each channel’s time steps to preserve channel identity and model inter-channel dependencies.
- The pretraining objective splits balanced support and query sets, masks query labels, and trains predictions for the masked labels end-to-end.
4 Experiments
Experiments evaluate TSPFN on heterogeneous physiological data under limited-data clinical conditions. The pretraining corpus spans EEG, ECG, and ICU signals, while evaluation uses five datasets with stratified cross-validation.
- Training: TSPFN is initialized from TabPFN weights and trained with AdamW, early stopping, and convergence typically reached after approximately 25 epochs.
- Pretraining: Approximately 140,000 samples from four public datasets provide heterogeneous EEG, ECG, and ICU waveforms for pretraining.
- Evaluation: Evaluation covers five datasets spanning ICU, EEG, and ECG, with limited data, class imbalance, and high intra-class variability, using stratified 5-fold cross-validation.
- Evaluation: The benchmark includes vital-sign, EEG, and ECG tasks with dataset-specific windows, channel selections, and class structures.
5 Results and ablation studies
TSPFN consistently outperforms diverse time-series classification baselines in low-data settings and maintains top cross-domain AUC across support-set sizes. Ablations attribute gains over TabPFN to physiological pretraining and temporal, channel-wise positional encoding.
- Comparative performance analysis: TSPFN significantly outperforms XGBoost, TabPFN, TCN, MiniRocket, and LaBram on average using 100 support samples.A DeLong test against MiniRocket in AUROC gives p = 3.7 × 10−5.
- Cross-domain generalization: TSPFN achieves top-1 AUC on average across support-set scales up to 1,000 samples, with consistent cross-domain performance.Baselines show greater variability across the benchmark suite.
- Cross-domain generalization: Specialized models can win on individual datasets but degrade across domains, whereas TSPFN remains stable and competitive across physiological datasets.MiniRocket may outperform TSPFN on eICU but degrades on ECG, while LaBRam performs worst across modalities and scales.
- Ablation studies: Adding RoPE and channel-wise positional embeddings further improves performance by 6.4 points on ESR and 8.3 points on ECG5000 versus the tabular baseline.The ECG5000 gain is reported coupled with RoPE.
6 Conclusion
The conclusion presents TSPFN as a PFN-based model for physiological time-series classification in small- and medium-data regimes. It reports effective fine-tuning-free generalization across EEG, ECG, and ICU tasks, surpassing specialized methods in several settings.
- 6 Conclusion: TSPFN targets physiological time-series classification for datasets with fewer than 1,000 samples.Its design addresses limitations involving temporal dependencies, sequential position, and multichannel signals.
- 6 Conclusion: TSPFN combines structured multi-scale inputs, channel-wise RoPE, and channel identity embeddings to model temporal and inter-channel dependencies.The architecture builds on TabPFN.
- 6 Conclusion: Across EEG, ECG, and ICU datasets, TSPFN generalizes to unseen tasks without fine-tuning and surpasses specialized state-of-the-art methods in several settings.The conclusion characterizes this as effective generalization despite heterogeneous physiological pretraining.