Source-linked AI summary
Chronos-2: From Univariate to Universal Forecasting
Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, Michael Bohlke-Schneider
TL;DR
Existing pretrained forecasting models largely focus on univariate data, leaving multivariate and covariate-informed tasks insufficiently supported for real-world use. Chronos-2 addresses this gap with group-attention-based in-context learning and synthetic multivariate training, achieving state-of-the-art results across three benchmarks, especially on covariate-informed tasks.
Problem
Most pretrained forecasting models are limited to univariate data, while practical forecasting often requires multivariate dependencies and covariates.
Method
Chronos-2 uses group attention for in-context information sharing across related series, trained on synthetic data imposing multivariate structures on univariate series.
Results
Chronos-2 achieves state-of-the-art performance across three benchmarks and outperforms baselines across task categories, with the largest gains on covariate-informed tasks.
Takeaways & Limitations
Chronos-2 provides a general-purpose zero-shot forecasting model for univariate, multivariate, and covariate-informed tasks.
Abstract
from arXiv · showhide
Pretrained time series models have enabled inference-only forecasting systems that produce accurate predictions without task-specific training. However, existing approaches largely focus on univariate forecasting, limiting their applicability in real-world scenarios where multivariate data and covariates play a crucial role. We present Chronos-2, a pretrained model capable of handling univariate, multivariate, and covariate-informed forecasting tasks in a zero-shot manner. Chronos-2 employs a group attention mechanism that facilitates in-context learning (ICL) through efficient information sharing across multiple time series within a group, which may represent sets of related series, variates of a multivariate series, or targets and covariates in a forecasting task. These general capabilities are achieved through training on synthetic datasets that impose diverse multivariate structures on univariate series. Chronos-2 delivers state-of-the-art performance across three comprehensive benchmarks: fev-bench, GIFT-Eval, and Chronos Benchmark II. On fev-bench, which emphasizes multivariate and covariate-informed forecasting, Chronos-2's universal ICL capabilities lead to substantial improvements over existing models. On tasks involving covariates, it consistently outperforms baselines by a wide margin. Case studies in the energy and retail domains further highlight its practical advantages. The in-context learning capabilities of Chronos-2 establish it as a general-purpose forecasting model that can be used "as is" in real-world forecasting pipelines.
1 Introduction
Pretrained forecasting models simplify zero-shot prediction but largely remain limited to univariate data, motivating Chronos-2’s unified handling of univariate, multivariate, and covariate-informed tasks. Chronos-2 uses group attention and synthetic training structures to support these capabilities, achieving state-of-the-art benchmark performance.
- Motivation: Most pretrained forecasting models focus on univariate data, limiting their applicability to multivariate and covariate-informed production tasks.Real-world examples include jointly evolving cloud infrastructure metrics and energy patterns influenced by weather.
- Contribution: Chronos-2 performs univariate, multivariate, and covariate-informed forecasting zero-shot, including tasks with past-only or known future numeric and categorical covariates.Its cross-learning capability also shares information across univariate series in a batch.
- Architecture: Group attention exchanges information among related series, multivariate variates, or targets and covariates without concatenating them into a longer context.It operates across the batch axis and is designed to scale with the number of variates.
- Training: Chronos-2’s capabilities are enabled by synthetic time series data that impose diverse multivariate structures on base univariate series.This training approach addresses the scarcity of high-quality multivariate and covariate-informed pretraining data.
- Results: Chronos-2 achieves state-of-the-art performance across fev-bench, GIFT-Eval, and Chronos Benchmark II.On fev-bench, it outperforms baselines across univariate, multivariate, and covariate-informed categories, with the largest gains on covariate-informed tasks.
2 Background and Related Work
Time series forecasting predicts future observations from historical values, optionally using covariates and estimating either point forecasts or predictive distributions. Pretrained models enable zero-shot transfer, but most remain univariate, whereas Chronos-2 uses group attention to unify diverse forecasting setups without task-specific adaptation.
- Background: Forecasting predicts the next H time steps from historical observations, optionally conditioning on historical and future covariates.Probabilistic forecasting estimates P(Y_T+1:T+H | Y_1:T, X_1:T+H) to represent uncertainty.
- Background: Zero-shot forecasting generates predictions for previously unseen datasets without additional training, adaptation, or fine-tuning.This setting distinguishes pretrained forecasting from local or dataset-specific model fitting.
- Related Work: Earlier forecasting methods included local models with series-specific parameters and global models sharing parameters across series within a dataset.Examples of local methods include ARIMA, Exponential Smoothing, and Theta.
- Related Work: Pretrained forecasting models extend transfer-learning ideas to zero-shot generalization across diverse datasets, borrowing concepts from large language models while training on time series.Early work adapted language models directly; later approaches primarily borrowed architectural concepts.
- Related Work: Most pretrained models treat multivariate dimensions independently and ignore covariates, with Moirai-1 and Toto as notable exceptions.The supplied passage specifically notes that Moirai-1 incorporates multivariate inputs but internally flattens them.
- Chronos-2: Group attention unifies univariate, multivariate, and covariate-informed forecasting over related series without architectural changes or task-specific adaptations.It generalizes cross-attention and cross-learning ideas while accommodating diverse group definitions.
3 The Chronos-2 Model
Chronos-2 is an encoder-only transformer that constructs target–covariate inputs, processes temporal and groupwise information, and produces multi-step quantile forecasts. Its group attention enables in-context learning across appropriately grouped series while supporting heterogeneous forecasting tasks.
- 3.1 Scaling and Tokenization: Chronos-2 constructs historical target–covariate inputs and future inputs with known covariates retained while targets and past-only covariates are masked as missing.Categorical covariates are converted to real-valued representations before concatenation.
- 3.1 Scaling and Tokenization: The tokenization pipeline standardizes inputs, applies an sinh^-1 transformation, and concatenates normalized historical and future values.The transformation stabilizes variance and reduces outlier influence.
- 3.1 Scaling and Tokenization: Chronos-2 splits each dimension and its time-index and observation-mask features into patches, then maps concatenated patches into transformer embeddings.The mask identifies missing historical values and future-known covariates; padding handles lengths not divisible by the patch size.
- 3.2 Architecture: The model alternates temporal self-attention with group attention, which exchanges information across related series sharing a group at each patch index.Groups can represent individual series, related items, multivariate variates, or targets and covariates; attention is restricted within groups.
- 3.2 Architecture: A quantile head produces direct multi-step forecasts for target dimensions, predicting 21 quantiles from 0.01 through 0.99.The model supports efficient long-horizon prediction by producing forecasts for multiple target patches in one forward pass.
- 3.3 Training: Training batches combine univariate, multivariate, past-only-covariate, and known-covariate tasks, using quantile regression evaluated only on target dimensions.A second training stage extends context length to 8192 and increases sampled output patches for long-context and long-horizon forecasting.
4 Training Data
Chronos-2 relies extensively on synthetic data because available large-scale time-series corpora are primarily univariate. Multivariatizers impose diverse cross-series dependencies to create training tasks with multivariate targets and covariates.
- 4 Training Data: Chronos-2 uses synthetic data extensively because large-scale time-series datasets primarily contain univariate series.Select datasets from the Chronos and GIFT-Eval corpora are also included in the training corpus.
- 4 Training Data: Multivariatizers sample multiple series from base univariate generators and impose dependencies to create multivariate dynamics.The base generators include autoregressive, exponential-smoothing, TSI, and KernelSynth models.
- 4 Training Data: Cotemporaneous multivariatizers create same-time linear or nonlinear relationships, whereas sequential multivariatizers induce temporal dependencies such as lead–lag effects and cointegration.These two classes provide complementary multivariate structures for training.
- 4 Training Data: Generated multivariate series support both tasks predicting all variates and tasks where randomly selected variates serve as known covariates.This construction exposes the model to both multivariate forecasting and covariate-informed forecasting settings.
5 Experiments
Chronos-2 is evaluated across three comprehensive forecasting benchmarks, with results showing strong performance across univariate, multivariate, and covariate-informed tasks. Experiments also examine in-context learning, domain-specific covariate use, model size, synthetic-only training, and long-context post-training.
- 5.1 Benchmark Results: Chronos-2 surpasses baselines by a statistically significant margin on fev-bench pairwise win rates and skill scores.The confidence intervals exclude 50% for win rates and 0% for skill scores against every baseline.
- 5.1 Benchmark Results: Chronos-2 outperforms existing pretrained models across fev-bench, GIFT-Eval, and Chronos Benchmark II.The reported comparisons use win rate and skill score across probabilistic and point forecasting metrics.
- 5.2 Improvements with In-context Learning: In-context learning improves univariate skill scores, especially on Chronos Benchmark II tasks with short contexts.The analysis attributes this gain to cross-learning from related time series in the batch.
- 5.2 Improvements with In-context Learning: In-context learning yields modest gains on multivariate fev-bench tasks but produces the largest gains on tasks with covariates.With ICL, Chronos-2 exploits covariates that univariate inference ignores and outperforms baselines by a large margin.
- 5.3 Domain Case Studies: Energy and retail case studies show more accurate forecasts when Chronos-2 uses dynamic or known covariates through ICL.The examples involve energy prices, load and renewable-generation forecasts, store sales, footfall, promotions, and holidays.
- 5.4 Ablation Studies: The 28M-parameter model trails the base model by as little as 1% points in skill score on GIFT-Eval while offering nearly 2× faster inference.This trade-off supports low-resource or speed-prioritized deployments.
- 5.4 Ablation Studies: Extending context length from 2,048 to 8,192 time steps improves performance, particularly on GIFT-Eval’s high-frequency datasets with long seasonal periods.The comparison is between Chronos-2-2K and the post-trained extended-context variant.
6 Discussion
Chronos-2 extends zero-shot forecasting across univariate, multivariate, and covariate-informed tasks, with especially strong gains on covariate-informed tasks. Its results also underscore synthetic data and flexible grouping as important foundations and opportunities for future applications.
- Chronos-2 handles univariate, multivariate, and covariate-informed forecasting tasks in a zero-shot manner across three comprehensive benchmarks.
- Chronos-2 substantially surpasses prior foundation models on covariate-informed tasks, where the performance gap is particularly large.
- Chronos-2’s capabilities beyond univariate forecasting rely entirely on synthetic data, while synthetic-only training performs only slightly worse than mixed real-and-synthetic training.
- Flexible group attention could support retrieval-augmented forecasting by grouping series with sparse metadata or dense embeddings, especially in small-data or cold-start settings.
A Training Data
Table 6 identifies the real univariate datasets used to pretrain Chronos-2.
- Table 6 lists the real univariate datasets used for Chronos-2 pretraining.
- The table concerns real datasets rather than the synthetic data used to provide capabilities beyond univariate forecasting.
- The listed datasets define the real univariate component of Chronos-2’s pretraining data.
B Additional Results
The additional results organize Chronos-2’s forecasting evaluations by benchmark subset, metric, model comparison, and domain case study. They include univariate, multivariate, and covariate-focused views across fev-bench, GIFT-Eval, Chronos Benchmark II, energy, and retail data.
- Figure 9 presents Chronos-2’s univariate point-forecasting results and ICL improvements across fev-bench, GIFT-Eval, and Chronos Benchmark II.
- Tables 7–9 report fev-bench average win rates and skill scores for MASE, WQL, and WAPE, respectively, with higher values better for both metrics.
- Figure 10 shows univariate point-forecasting results and ICL gains on the multivariate and covariates subsets of fev-bench.
- Figure 11 compares Chronos-2 with baselines on dynamic-covariate tasks from energy and retail, using WAPE for retail.
- Tables 10 and 11 identify fev-bench dynamic-covariate datasets used in the energy and retail case studies.