PaperlayerView exact revision on arXiv

Source-linked AI summary

Aurora-X: Built for Extreme Time Series Forecasting

Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng, Peng Chen, Yang Shu, Jilin Hu, Bin Yang

arXiv:2609.31038v1cs.LG

TL;DR

TSFMs have underexplored training potential and limited architectural versatility for general-purpose forecasting. Aurora-X combines a progressive curriculum with a unified architecture, and the paper reports state-of-the-art forecasting performance across five benchmarks against pretrained TSFMs and task-specific supervised models.

  • Problem

    TSFMs' training potential remains underexplored, and existing architectures have yet to combine several practical forecasting capabilities within one model.

    Source: §1 · p. 2

  • Method

    Aurora-X combines channel-independent pretraining, capability-oriented midtraining, variable-resolution post-training, pattern-guided sparse experts, and an implicit quantile head in a unified architecture.

    Source: §1 · p. 2

  • Results

    Aurora-X demonstrates state-of-the-art forecasting performance against pretrained TSFMs on GIFT-Eval, TIME, and FEV-Bench, and supervised models on TFB and DAG-Bench.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Variable-resolution post-training lets Aurora-X adjust historical coverage to the token budget at inference with fixed model weights.

    Source: §1 · p. 2

Abstract

from arXiv · show

Time series foundation models (TSFMs) enable cross-domain forecasting, but their development as general-purpose forecasters remains constrained by underexplored training potential and limited architectural versatility. To address these challenges, we introduce Aurora-X, a billion-scale TSFM with a progressive curriculum and a unified architecture. We first use channel-independent pretraining to learn temporal patterns, then introduce cross-variable dependencies, varied context and horizon lengths, and future covariates if available during midtraining. Variable-resolution post-training further enables an adjustable temporal span per token at inference. With fixed model weights, this supports longer histories under a fixed token budget or fewer tokens for the same history, enabling test-time scaling. With a versatile architecture, Aurora-X supports cross-variable modeling, covariate conditioning, and parallel decoding of future patches for probabilistic forecasting. These are supported by a novel pattern-guided mixture-of-experts that expands model capacity through sparse activation and uses shallow patch similarities to constrain deep-layer routing, guiding expert specialization across heterogeneous time series. Furthermore, we propose an implicit quantile network head that predicts arbitrary quantiles to characterize predictive distributions, enhancing probabilistic forecasting flexibility. Comprehensive experiments on GIFT-Eval, TIME, FEV-Bench, TFB, and DAG-Bench demonstrate state-of-the-art forecasting performance against pretrained TSFMs and task-specific supervised models.

1 INTRODUCTION

Aurora-X combines staged training with a versatile architecture to address gaps in TSFM training flexibility and forecasting capabilities. The paper reports state-of-the-art performance across five benchmarks against pretrained TSFMs and task-specific supervised models.

1 INTRODUCTION

  • Existing TSFMs have yet to combine cross-variable modeling, future-covariate conditioning, flexible uncertainty characterization, and parallel decoding in one architecture, and few support test-time scaling.The paper identifies underexplored alignment of training stages with data characteristics, task difficulty, and target capabilities as a related gap.Source: §1 · p. 2
  • Aurora-X uses channel-independent pretraining, midtraining with joint variables, varied context and horizon lengths, and future covariates, followed by variable-resolution post-training.At inference, adjustable temporal span per token supports longer histories under a fixed token budget or fewer tokens for the same history.Source: §1 · p. 2
  • Its versatile architecture supports cross-variable and covariate conditioning, long input histories, and parallel future-patch decoding, while an implicit quantile head predicts arbitrary quantiles.The head characterizes predictive distributions; shallow patch similarities constrain deep-layer MoE routing to guide expert specialization across heterogeneous series.Source: §1 · pp. 2–3
  • Aurora-X demonstrates state-of-the-art forecasting performance against pretrained TSFMs on GIFT-Eval, TIME, and FEV-Bench, and supervised models on TFB and DAG-Bench, with favorable computational cost and inference latency.The evaluations cover five benchmarks.Source: §1 · p. 3

2 RELATED WORK

Related TSFMs address cross-domain transfer, multivariate forecasting, uncertainty estimation, and inference adaptation through varied approaches. Aurora-X combines arbitrary-level marginal quantiles and parallel patch decoding with guided expert routing and staged training.

2.1 TIME SERIES FOUNDATION MODELS

  • Chronos-2 and TiRex-2 support multivariate forecasting with known future covariates, while Baguan-TS integrates covariates into sequence-native in-context learning.Moirai uses any-variate attention to accommodate tasks with differing variable counts and combinations.Source: §2.1 · p. 3
  • Aurora-X predicts marginal quantiles at arbitrary continuous levels and decodes future patches in parallel, unlike fixed-grid quantile regressors and successive autoregressive output blocks.The related-work comparison also notes serial hidden-state computation within Timer-S1's supported horizon.Source: §2.1 · p. 3

2.2 TRAINING AND INFERENCE ADAPTATION

  • Aurora-X uses learned shallow patch similarities to regularize deep-layer expert routing, complementing load balancing and guiding specialization across heterogeneous time series.The passage contrasts this with sparse experts, scale-specific pathways, and clustering encoded patches in other approaches.Source: §2.2 · p. 3
  • Aurora-X progressively develops temporal modeling, cross-variable dependencies, and covariate conditioning, then makes temporal resolution controllable at inference to match historical coverage to the context-token budget.The passage situates this among approaches that vary sampling or flow-step budgets, use retrieval and ensembles, or adapt temporal resolution.Source: §2.2 · p. 3

3 AURORA-X

Aurora-X combines a temporal and cross-variable backbone with pattern-guided sparse experts, an arbitrary-quantile prediction head, and a staged curriculum that supports variable-resolution inference.

3.1 UNIFIED BACKBONE

  • The shared backbone embeds masked, normalized patches, models temporal dependencies within variables and aligned cross-variable groups, and sends future states to a quantile head.Figure 2 depicts a 1.05B-parameter model with L = 12 blocks.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 PATTERN-GUIDED MIXTURE OF EXPERTS

  • Shallow similarities between normalized patches regularize deep expert assignments, while each layer routes tokens to 4 of 16 experts plus a shared expert.The pattern-guided objective favors expert sharing above the weighted mean similarity and discourages it below.Source: §3.2 · p. 5

3.3 IMPLICIT QUANTILE HEAD

  • The IQN conditions shared future states on continuous quantile levels, predicts arbitrary quantiles in parallel, and trains sampled levels with masked pinball loss.At inference, one backbone evaluation provides future states for parallel decoding of all patch–quantile pairs.Source: §3.3 · p. 6

3.4 CURRICULUM AND VARIABLE-RESOLUTION POST-TRAINING

  • Training progresses from singleton-variable pretraining to multivariate groups and longer contexts and horizons, then varied lengths, available future covariates, and multiscale post-training.Each curriculum stage continues from the preceding checkpoint.Source: §3.4 · p. 6
  • Post-training resamples each kP-point block to P values, so k > 1 reduces token count or extends history at a fixed context budget; inference restores requested resolution with fixed weights.The inference process maps resampled inputs to forecasts and returns them to the requested resolution.Source: §3.4 · p. 7, Figure 3 · p. 7

4 EXPERIMENTS

Aurora-X achieves the lowest relative MASE among compared TSFMs on three benchmark collections. The experiments also examine architecture choices and show gains across the progressive training curriculum.

4 EXPERIMENTS

  • The experiments cover five benchmarks and compare Aurora-X with pretrained TSFMs and task-specific supervised models.The section presents GIFT-Eval, TIME, and FEV-Bench results, with TFB and DAG-Bench comparisons reported in Appendix A.8.Source: §4 · p. 7, Figure 4 · p. 7

4.1 GENERAL FORECASTING PERFORMANCE

  • Aurora-X achieves the lowest relative MASE among compared TSFMs on GIFT-Eval, TIME, and FEV-Bench.Its reductions range from 0.3% over Toto-2.0 (2.5B) on TIME to 8.0% over TiRex on GIFT-Eval.Source: Figure 5 · p. 8, §4.1 · p. 8

4.2 MODEL ANALYSIS

  • The MoE comparison examines expert specialization and forecasting accuracy across Aurora-X, Time-MoE, Moirai-MoE, and Timer-S1 implementations.It inspects routing at layer 6 of the 12-layer backbone across five temporal scenarios and evaluates relative MASE on three benchmarks.Source: §4.2 · pp. 8–9, Figure 6 · p. 8, Figure 7 · p. 9
  • The forecasting-head comparison tests IQN, fixed quantile regression, flow matching, and a mixture distribution head using median forecasts and relative MASE.The backbone, pretraining data, and evaluation horizons are shared.Source: §4.2 · p. 9
  • 13.1%, 19.0%, and 20.4% are the relative MASE reductions from Pre to Post on GIFT-Eval, TIME, and FEV-Bench, respectively.Relative MASE decreases at every training-stage transition, and post-training reduces it a further 2.1–2.8% relative to M3.Source: §4.2 · p. 9

5 CONCLUSION

Aurora-X combines a capability-oriented curriculum with pattern-guided MoE routing and an implicit quantile head. The conclusion reports strong performance across five forecasting benchmarks, with favorable computational cost and inference latency.

5 CONCLUSION

  • Figure 8 reports relative MASE for four forecasting heads, while Figure 9 traces relative MASE across training stages.The captions identify the comparisons but do not state their numerical outcomes.Source: Figure 8 · p. 9, Figure 9 · p. 9
  • Aurora-X combines pattern-guided MoE routing, an implicit quantile head, and a capability-oriented curriculum.Variable-resolution post-training enables context-token budget control and longer historical coverage.Source: §5 · p. 9
  • The paper reports strong performance across five forecasting benchmarks together with favorable computational cost and inference latency.The conclusion presents Aurora-X as an out-of-the-box tool for decision intelligence.Source: §5 · p. 9

A.1 BENCHMARKS AND EVALUATION PROTOCOL

The appendix specifies five benchmark settings, comparison sources, Aurora-X’s configuration, and its training data and staged curriculum. Evaluation follows benchmark-defined tasks, splits, horizons, and metrics, with input settings reflecting covariate availability.

A.1 BENCHMARKS AND EVALUATION PROTOCOL

  • The five benchmarks cover general, multivariate, and future-covariate forecasting, and evaluation follows benchmark-defined tasks, splits, horizons, and metrics.Relative MASE is a geometric mean of configuration-wise ratios to Seasonal Naive; TFB and DAG-Bench report arithmetic horizon averages of MSE and MAE.Source: §A.1 · p. 14

A.2 BASELINES AND RESULT SOURCES

  • Comparisons draw on pretrained TSFMs, fourteen supervised TFB baselines, and ten supervised DAG-Bench baselines.The main MASE figure excludes benchmark-specific fine-tuning, ensembles, and tabular foundation-model adaptations; appendix rankings include expanded model selections.Source: §A.2 · pp. 14–15

A.3 MODEL CONFIGURATION

  • Aurora-X has approximately one billion parameters, a 12-block backbone, and a native patch length of 48.Each MoE layer has one shared expert and 16 routed experts, with four routed experts selected per token.Source: §A.3 · p. 15, Table 2 · p. 16

A.4 TRAINING DATA AND FIVE-STAGE CURRICULUM

  • Pretraining and midtraining use the heterogeneous TimeBench corpus; post-training uses a one-billion-original-time-point subset from LightGTS and ROSE corpora.The subset size refers to source observations before multi-scale augmentation.Source: §A.4 · p. 15
  • The curriculum begins with channel-independent temporal modeling, then introduces joint variables and longer contexts and horizons.Pretraining and midtraining proceed through Pre and stages M1–M3, each continuing from the preceding checkpoint.Source: §A.4 · pp. 16–17, Table 3 · p. 16
  • M3 introduces known future covariates, and post-training develops variable-resolution forecasting through temporal-span enumeration and resampling.At inference, variable resolution adjusts historical coverage and context-token use with fixed model weights.Source: §A.4 · p. 17

A.5 ARCHITECTURE COMPARISON PROTOCOL

The architecture comparisons hold key training and evaluation conditions constant while varying modules. Routing analyses compare input-dependent expert preferences across temporal scenarios and through the model’s layers.

A.5 ARCHITECTURE COMPARISON PROTOCOL

  • Architecture studies use a shared Aurora-X pretraining corpus and matched backbone dimensions, training exposure, and evaluation configurations.The MoE variants use 16 routed experts with top-4 selection and matched expert width; head comparisons change only the forecasting head and its objective.Source: §A.5 · p. 17
  • Five TFB-based temporal scenarios are used to compare input-dependent expert routing; rows show normalized selection frequency and columns index experts.Each sequence receives equal weight, and expert indices are local to each variant.Source: §A.5 · pp. 17–18, Table 4 · p. 17
  • Forecasting variants are compared with common benchmark configurations and relative MASE aggregation.Head comparisons use a common quantile grid containing the median; distributional heads estimate grid quantiles from their predictive distributions.Source: §A.5 · p. 18
  • Aurora-X has distinct scenario routing profiles at every layer, while the other three implementations’ scenario differences weaken from layer 4 onward.The panels span all 12 layers; expert indices are local to each layer and variant.Source: §A.5 · p. 18, Figure 10 · p. 19

A.6 PERFORMANCE ACROSS TRAINING STAGES

Aurora-X improves across successive training stages and reports strong point and probabilistic forecasting results, including on multivariate and covariate-aware benchmarks.

A.6 PERFORMANCE ACROSS TRAINING STAGES

  • Relative MASE decreases at every training stage on all three benchmarks, with post-training reductions versus M3 of 2.3% on GIFT-Eval, 2.1% on TIME, and 2.8% on FEV-Bench.From pretraining to post-training, relative MASE falls by 13.1%, 19.0%, and 20.4%, respectively.Source: Figure 11 · p. 20, §A.6 · p. 20

A.7 EXPANDED PROBABILISTIC FORECASTING RANKINGS

  • Aurora-X records relative WQL of 0.4665 on GIFT-Eval and relative SQL of 0.5168 on FEV-Bench, the lowest among the twelve displayed FEV-Bench models.GIFT-Eval’s ranking includes 43 models and checkpoints; the fine-tuned Toto-2.0 (2.5B) checkpoint scores 0.4634 relative WQL.Source: §A.7 · p. 20, Figure 12 · p. 21, Figure 13 · p. 22

A.8 MULTIVARIATE AND COVARIATE FORECASTING

  • Aurora-X reduces average MSE by 9.0% and MAE by 3.6% versus DUET on TFB, and by 11.5% and 7.7% versus DAG on DAG-Bench.It ranks first in MSE on seven of eight TFB datasets and nine of twelve DAG-Bench datasets, and in MAE on five and six, respectively.Source: Table 5 · p. 23, Table 6 · p. 23, §A.8 · p. 23

A.9 COMPLETE MULTIVARIATE FORECASTING RESULTS

The complete TFB results report Aurora-X and supervised baselines across four forecast horizons, with separate model-group tables and within-table rankings.

A.9 COMPLETE MULTIVARIATE FORECASTING RESULTS

  • Each TFB table ranks its displayed models separately, including all ties at the reported precision.The full tables use two groups of models; Table 5 summarizes seven recent advanced supervised baselines.Source: §A.9 · p. 24, Table 7 · p. 24, Table 8 · p. 25
  • The complete results follow the source-selection procedure described for the TFB comparisons.The supplied passages do not state the full tables’ individual metric values.Source: §A.9 · p. 24
  • The full TFB results report MSE and MAE at forecast horizons of 96, 192, 336, and 720 steps.Tables 7 and 8 divide the fourteen supervised baselines into two groups, with Aurora-X first in each.Source: Table 7 · p. 24, §A.9 · p. 24, Table 8 · p. 25

A.10 COMPLETE COVARIATE FORECASTING RESULTS

The appendix reports Aurora-X’s covariate-aware forecasting results, evaluates two context-token allocation modes, and measures complete-forecast inference costs.

A.10 COMPLETE COVARIATE FORECASTING RESULTS

A.11 TOKEN BUDGET AND EFFECTIVE CONTEXT LENGTH

  • The TFB token-budget study considers compressing a fixed 2,880-point history into fewer context tokens and extending historical coverage with ten context tokens.The five temporal spans allocate 60, 30, 15, 9, and 6 context-token slots for fixed-history inputs, or expose 480 to 4,800 historical points with ten tokens.Source: Figure 14 · p. 28, §A.11 · p. 28

A.12 INFERENCE EFFICIENCY

  • At native token length 48, Aurora-X uses 76.5% fewer MACs than Toto-2.0-1B and achieves a measured 2.35× speedup.At length 192, MACs fall 71.0% versus length 48, but latency improves only 1.05×; this efficiency sweep does not evaluate the corresponding accuracy change.Source: Figure 15 · p. 30, Figure 16 · p. 30, §A.12 · p. 30

A.13 FORECASTING WITH ARBITRARY QUANTILE QUERIES

The appendix illustrates arbitrary quantile queries and fixed-window forecasts, then analyzes resolution adaptation and its relationship to LightGTS projection resizing.

A.13 FORECASTING WITH ARBITRARY QUANTILE QUERIES

  • The IQN head directly returns queried quantile levels, illustrated with five random levels across eight forecasting windows.The central forecast summary averages predictions at twenty levels and is neither the median nor an exact integral over the predictive distribution; the displayed cases are selected using test targets.Source: §A.13 · p. 31

A.14 ADDITIONAL FORECASTING VISUALIZATIONS

B RESOLUTION ADAPTATION AND TRAINING CONSISTENCY

  • Resolution adaptation changes the temporal span represented by each token while retaining a pretrained backbone.The section compares explicit patch resampling with projection resizing under matched patch boundaries, normalization, and auxiliary inputs.Source: B RESOLUTION ADAPTATION AND TRAINING… · p. 38

B.1 EQUIVALENT PLACEMENT OF RESOLUTION OPERATORS

  • Resampling an input or output patch is exactly equivalent to composing the corresponding resolution map with its value projection.The stated equivalence extends through the nonlinear modules when the linear resolution maps are placed at their boundaries.Source: §B.1 · p. 38

B.2 RELATIONSHIP TO LIGHTGTS FLEX-RESIZE

  • LightGTS Flex-resize implements interpolation-based upsampling or downsampling implicitly in projection weights, using a pseudoinverse-derived reconstruction.Its reconstruction weights need not equal ordinary interpolation weights.Source: §B.2 · p. 38
  • Exact equivalence to direct resampling concerns placement of a matched operator; approximation to LightGTS’s operator depends on the discrepancy in Equation (19).When the upsampling map has full column rank, the pseudoinverse round trip projects onto the recoverable native-signal subspace.Source: §B.2 · p. 39

B.3 CONSISTENCY OF MULTI-SCALE TRAINING AND INFERENCE

Multi-scale training and inference reuse a shared canonical-grid forecaster, with resolution changes expressed through paired augmentation and output resampling. This alignment depends on consistent preprocessing, forecast-origin alignment, validity rules, and native-grid time features.

B.3 CONSISTENCY OF MULTI-SCALE TRAINING AND INFERENCE

  • Training and inference both call the same native-grid forecaster on canonical inputs, with inference applying the matching output-resampling operator.The training objective uses masked pinball loss over valid target positions.Source: §B.3 · p. 39
  • Preprocessing normalizes and applies arcsinh using history statistics before resampling, without renormalizing augmented arrays.History blocks align to the forecast origin; future blocks start at the first forecast step, and targets are transformed separately using history statistics. Training and inference require identical validity rules and native-grid time features.Source: §B.3 · p. 39
  • Paired augmentation applies the same temporal granularity to each history–target pair, aligning histories, covariates, and targets across scales.The data pipeline can batch multiple resolutions with the same patch shape, embedding, quantile head, and loss, without requiring a detected period.Source: §B.3 · p. 39
  • The paired construction retains a fixed model interface while adapting temporal resolution for heterogeneous sequences.It supports a reusable augmentation pipeline, with scales enumerated without detecting a period.Source: §B.3 · p. 39

C THEORETICAL ANALYSIS OF SHALLOW-PATTERN CLUSTERING

The analysis characterizes how shallow-pattern similarities shape expert sharing in Aurora-X, and distinguishes the objective’s preferences from guarantees about routing optimization. It also separates pattern clustering from projection diversity and load balancing.

C THEORETICAL ANALYSIS OF SHALLOW-PATTERN CLUSTERING

  • The pattern loss links normalized, overlapping expert assignments to cluster coherence, while projection regularization encourages distinct feature directions.Its gradient favors sharing among sufficiently similar patches.Source: C THEORETICAL ANALYSIS OF… · p. 40

C.1 NORMALIZED CO-ASSIGNMENT AND KERNEL CLUSTERING

  • Under unit-norm projected patches, the loss measures weighted within-expert kernel dispersion and constrains the share of co-assignment weight on dissimilar pairs.Experts are weighted by their numbers of co-assigned ordered pairs, and assignments may overlap.Source: §C.1 · p. 40

C.2 PATTERN-DEPENDENT LEARNING SIGNALS FOR ROUTING

  • The pattern loss favors overlap between more-similar-than-average patches, and its straight-through surrogate passes this preference to router logits.The learned pattern projection also receives a signal to increase similarity among co-assigned patches; the similarity matrix comes from shallow patches rather than deep hidden-state distinctions.Source: §C.2 · p. 41

C.3 WHEN PATTERN GROUPING IS PREFERRED TO COLLAPSED ROUTING

  • With enough experts for disjoint group assignments, pattern-separated routing has lower loss than collapsed routing when mean within-group similarity exceeds mean between-group similarity.This objective preference does not ensure discrete-routing optimization escapes collapse; identical assignments can have zero routing gradient in the stated two-patch example.Source: §C.3 · pp. 41–42

C.4 PROJECTION DIVERSITY AND THE ROLE OF LOAD BALANCING

  • The orthogonality penalty discourages repeated projection directions, while load balancing acts on marginal expert use and shallow-pattern clustering guides which patches share experts.For m > 1, a penalty below 1/[m(m −1)] implies full row rank, but distinct inputs may still have the same normalized projection.Source: §C.4 · p. 42

D ADDITIONAL RELATED WORK

  • Related work spans multivariate and covariate conditioning, irregular or incomplete observations, uncertainty estimation, task adaptation, and expert specialization.The section positions Aurora-X’s staged training and multiple temporal resolutions within these broader forecasting directions.Source: D ADDITIONAL RELATED WORK · p. 43

Related papers

Loading 2609.31038v1…