From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting

Motivation

• A unified framework for univariate and covariate-informed forecasting remains desirable because many real-world systems are influenced by external factors.The introduction notes that current time-series foundation models are typically designed around purely sequential inputs, making covariate use challenging.Source: §1 · pp. 1–2
• The paper asks whether TabPFN-v2 can perform probabilistic time-series forecasting when temporal structure is expressed through features.Tabular foundation models offer a potential unified representation for tabular and time-series forecasting without specialized sequence architectures.Source: §1 · p. 2

Contributions

• TabPFN-TS treats forecasting as tabular regression, pairing each time step's temporal features and optional covariates with its observed target.TabPFN-v2 predicts future rows in a single forward pass, without time-series-specific pretraining or fine-tuning.Source: §1 · p. 2
• TabPFN-TS achieves state-of-the-art covariate-informed forecasting and competitive univariate accuracy across GIFT-Eval and fev-bench.The 11M-parameter method pairs lightweight temporal features with pretrained TabPFN-v2.Source: §1 · p. 2
• The compact feature scheme encodes time progression, multi-scale seasonality, and covariates for tabular models.Source: §1 · p. 2
• The authors provide analyses of temporal-structure use, featurization choices, and performance across tabular foundation-model backbones.Source: §1 · p. 2
• The authors present tabular foundation models paired with temporal features as an efficient, extensible alternative bridging tabular and time-series learning.Source: §1 · p. 2

Time Series Forecasting

• Forecasting models the conditional distribution of future targets from observed history and covariates, with univariate forecasting as the no-covariate case.Source: §2 · pp. 2–3

Time Series Foundation Models

• Time-series foundation models seek zero-shot forecasts across domains through pretraining on collections of real-world series.Examples include Chronos and TimesFM for univariate forecasting, Toto for multivariate forecasting, and TiRex and Sundial with other architectures or objectives.Source: §2 · p. 3
• ChronosX and COSMIC incorporate covariates through modular adapters and in-context conditioning, respectively.Source: §2 · p. 3

TabPFN-v2

• TabPFN is pretrained on synthetic regression and classification tasks to transfer learned feature–target patterns to unseen problems.At inference it conditions on feature-target examples and predicts targets for new inputs.Source: §2 · p. 3
• TabPFN represents predictive distributions as discretized densities and is trained directly on that probabilistic representation.TabPFN-v2 adds architectural and synthetic-data-generator improvements and native support for missing values, outliers, and uninformative features.Source: §2 · p. 3

Positioning

• TabPFN-TS complements sequence-based forecasters by using TabPFN-v2 with temporal features to predict the full horizon in one non-autoregressive pass.Source: §2 · p. 3

3 Methodology

• The method recasts multi-step forecasting as tabular regression, representing the series as a table for TabPFN-v2.Source: §3 · p. 3

3.1 From Time Series to Tabular Data

• The first C observations form the historical context and Xtrain with ytrain, while the remaining H timestamps form Xtest for forecasting.Figure 3.1 depicts TabPFN-v2 conditioning on the historical feature-target context to predict targets at future feature rows.Source: §3.1 · p. 3, Figure 3.1 · p. 4
• ChronosX and COSMIC are excluded from the comparison because they were not publicly available when the paper was written.Source: §3.1 · p. 3
• Timestamp features include a running index, eight cyclic calendar components plus calendar year, and automatically detected top-k seasonal periodicities.The automatic extraction detrends the series, applies a Hann window and zero-padding, removes the zero-frequency component, and selects the largest spectral peaks.Source: §3.1 · p. 4
• Time-varying covariates are supported only when their values are known for future timestamps, such as holidays rather than weather.Source: §3.1 · p. 4
• Feature vectors are stacked into a matrix without lagged or autoregressive features, avoiding reliance on past predictions in multi-step inference.The paper notes that lagged features would make inference much slower in its non-autoregressive setup.Source: §3.1 · p. 5

3.2 Point and Probabilistic Forecasting with TabPFN-v2

• TabPFN-v2 receives the historical feature-target pairs and predicts a posterior predictive distribution for each future input.The method uses the mean for squared-error evaluations, the median for absolute-error evaluations, and quantiles for probabilistic metrics and uncertainty bands.Source: §3.2 · p. 5

4.1 Univariate Forecasting

• On GIFT-Eval, TabPFN-TS ranks closely behind leading foundation models on WQL and MASE, matching substantially larger TimesFM-2.0 and Chronos-Bolt despite being over two orders of magnitude smaller.The evaluation covers a benchmark with 23 datasets and 97 distinct tasks; scores are aggregated across datasets and Figure 4.1 shows 95% confidence intervals.Source: §4.1 · p. 6, Figure 4.1 · p. 6

4.2 Covariate-Informed Forecasting

• The covariate evaluation uses 28 tasks with known covariates in both historical and future horizons, excluding two tasks that exceeded the six-hour evaluation limit.The model requires future covariate values to condition forecasts; the comparison follows fev-bench’s pretrained-model protocol and reports MASE and WQL.Source: §4.2 · p. 7
• Across 28 fev-bench tasks with known dynamic covariates, TabPFN-TS achieves the strongest overall performance, outperforming models including Toto-1.0.It is the only compared model that directly incorporates covariate inputs; the authors argue this formulation integrates covariates without architectural changes or fine-tuning.Source: §4.2 · pp. 7–8, Figure 4.2 · p. 7

4 Experiments

• Across noise, trend, and seasonal structures, TabPFN-v2 handles noise and periodicity but cannot extrapolate linear or exponential trends beyond the observed target range.Figure 5.1 presents the qualitative comparison across these structures.Source: Figure 5.1 · p. 8

5.2 How TabPFN-TS Interprets Temporal Structure

• With sin(t) alone, TabPFN-v2 reconstructs odd but not even harmonics; adding cos(t) gives an injective phase embedding that supports harmonic reconstruction, though accuracy deteriorates at higher frequencies.The experiments suggest that TabPFN-TS interpolates effectively in expressive phase space, behaving like a learned kernel or nearest-neighbor regressor.Source: Figure 5.2 · p. 9, §5.2 · p. 10

5.1 Qualitative Analysis

• On controlled series, TabPFN-TS handles noise and represented seasonality reliably, but fails to extrapolate linear or exponential trends beyond the observed target range.For noise, it predicts the sample mean with predictive intervals closely aligned to Gaussian quantiles.Source: §5.1 · p. 9

5.3 How TabPFN-TS Utilizes Covariates

• Covariates improve MASE and WQL for most synthetic classes, especially pulses, ramps, random walks, and composite signals; linear-trend cases are the main exception.In the exception, future targets fall outside the context distribution and the model does not extrapolate linear growth.Source: Figure 5.4 · p. 11, §5.3 · p. 11

5.4 Choice of Tabular Backbone

• TabDPT-TS yields meaningful zero-shot forecasts but trails TabPFN-TS; its test covers 78 of 97 GIFT-Eval tasks, and the pretraining explanation remains unverified.The authors hypothesize that TabPFN-v2’s broader synthetic-task pretraining better aligns with forecasting, but say systematic analysis is needed.Source: §5.4 · pp. 11–12

5.5 Impact of Temporal Featurization

• Combining index, calendar, and automatic seasonal features gives the best results: the index encodes progression, while calendar and automatic seasonal features add complementary periodic structure.The framework can also incorporate domain-specific periodic features when that knowledge is available.Source: §5.5 · p. 12

6.1 Extrapolation Beyond the Conditioning Range

• TabPFN-TS struggles to extrapolate future targets outside the observed range because its pretraining primarily optimizes conditional interpolation.When future targets fall outside the observed span, predictions remain within or close to that span; the limitation stems from the pretraining objective rather than the forecasting formulation.Source: §6.1 · pp. 12–13

6.2 Inference Time

• Roughly 30× more inference time than Chronos-Bolt-Base and TimesFM-2.0 is required under equal hardware assumptions.Figure 6.1 reports per-task times normalized to one GPU; TabPFN-TS runs across four T4s while the comparison models run on one T4.Source: Figure 6.1 · p. 13, §6.2 · p. 13
• Inference overhead arises because each series requires a separate forward pass and TabPFN-v2 retains full attention with O(L^2) complexity in sequence length L.The formulation treats each series as an independent query instance.Source: §6.2 · p. 13
• Batching, architectural refinement, and sequence-length optimization remain directions for improving inference efficiency.Source: §6.2 · p. 13

• An 11M-parameter TabPFN-TS combines TabPFN-v2 with lightweight temporal featurization and achieves strong univariate and state-of-the-art covariate-informed forecasting performance without time-series-specific pretraining or fine-tuning.The formulation adapts a tabular foundation model pretrained solely on synthetic, non-temporal data.Source: §7 · p. 13
• Ablations find that combining monotonic progression with complementary periodicities improves performance, while TabPFN-TS excels at interpolation but struggles with systematic extrapolation.Replacing its backbone with TabDPT also produces meaningful zero-shot forecasts.Source: §7 · p. 13
• The tabular formulation unifies univariate and covariate-aware forecasting, treats inputs as general features, and supports probabilistic forecasting without modifying the backbone architecture.Source: §7 · p. 14
• Addressing trend extrapolation beyond the conditioning range may require pretraining objectives combining interpolation, controlled extrapolation, and explicit treatment of out-of-support targets.Source: §7 · p. 14

• The implementation includes a year column alongside sine and cosine embeddings for seasonal features.Source: §A.1 · p. 16
• The seasonal feature set spans second-of-minute, minute-of-hour, hour-of-day, day-of-week, day-of-month, day-of-year, week-of-year, and month-of-year cycles.Their listed natural periods are 60, 60, 24, 7, 30.5, 365, 52, and 12, respectively.Source: §A.1 · p. 16
• For each seasonal feature, the implementation obtains an integer cycle index and uses the feature’s period in constructing its sine and cosine embeddings.Source: §A.1 · p. 16

• The algorithm selects up to k dominant periods by identifying local spectral peaks, converting frequencies to integer periods, and removing duplicate and zero periods.Local peaks are larger than their immediate neighbors; the input specifies k as the maximum number of periods.Source: §A.2 · p. 17
• Linear regression detrends the input time series before spectral analysis.The trend coefficients are found using least squares.Source: §A.2 · p. 17
• The detrended series is Hann-windowed and symmetrically zero-padded to length 2N before computing its fast Fourier transform.Source: §A.2 · p. 17

A.3 TabPFN-TS Configuration

• Missing time points are removed from conditioning, which is capped at the most recent 4096 observations; target values receive no extra normalization.TabPFN-v2 performs internal z-normalization during inference.Source: §A.3 · p. 18
• All experiments use the tabpfn-v2-regression-2noar4o2.ckpt3 checkpoint, without further hyperparameter tuning or checkpoint selection.The fixed choice showed stable behavior in preliminary point and probabilistic forecasting evaluations.Source: §A.3 · p. 18
• Automatic Seasonal Features retain the top k = 5 periodicities; a more extensive study of this parameter is left for future work.The authors report accuracy was relatively stable across reasonable k values in preliminary checks.Source: §A.3 · p. 18

A.4 GIFT-Eval Benchmark Datasets and Corresponding Statistics

• GIFT-Eval comprises 97 tasks spanning datasets, prediction horizons, and sampling frequencies; asterisked large datasets are excluded from ablations.The benchmark table provides dataset statistics and marks the datasets excluded from ablation studies.Source: §A.4 · p. 19, Table 1 · p. 19

A.5 Additional Results on GIFT-Eval

• Example TabPFN-TS predictions are shown for randomly selected short-, medium-, and long-term GIFT-Eval tasks.Figures A.1–A.3 provide the respective visualizations.Source: §A.5 · p. 20, Figure A.1 · p. 20, Figure A.2 · p. 21

A.6 Covariate-Informed Forecasting Evaluation on fev-bench

• The fev-bench evaluation uses tasks with covariates in both conditioning and prediction windows, listed in Table 2.Detailed task characteristics are referenced in the original benchmark paper and documentation.Source: §A.6 · p. 22, Table 2 · p. 22

A.7 Additional Results for Harmonic Reconstruction Experiments

• For {sin(t), cos(t)}, low- and mid-frequency reconstruction and 3–10-sinusoid composites are accurate, while sMAPE rises as frequency n grows.The passages attribute high-frequency degradation to reduced phase separability at the available sampling density.Source: Figure A.4 · p. 23, Figure A.5 · p. 23, Figure A.6 · p. 24

A.8 Synthetic Data Generation for Time Series with Covariates

• The synthetic-series figure distinguishes six classes and shows ground truth, predictions with and without covariates, uncertainty, and the train/test split.The classes include additive, multiplicative × LinearTrend, Pulses, Ramps, RandomWalk, and Composite.Source: Figure A.7 · p. 25

A.9 Ablation: Context Length vs. Accuracy

• Across 1024, 2048, 4096, and 10,000-point contexts, performance improves overall but gains diminish beyond 4096; MASE keeps improving while WQL increases slightly at 10,000.The longest context matches the largest dataset size used during TabPFN-v2 pretraining.Source: §A.9 · p. 26, Figure A.8 · p. 26