QiYao-I: A Manifold Based Foundation Model for Irregular Multivariate Time Series Forecasting

1 INTRODUCTION

• Existing foundation models rely on regular temporal structures and aligned observations, making direct transfer to irregular series challenging.In irregular data, equal-duration intervals may contain different numbers of sampled points, and variables can be observed at distinct asynchronous timestamps.Source: §1 · pp. 1–2
• QiYao-I jointly models irregular temporal dependencies and asynchronous cross-variable interactions in a unified foundation model framework.The paper reports superior forecasting performance and strong generalization in zero-shot and few-shot settings.Source: §1 · p. 3
• Sampling-conditioned temporal manifold attention distinguishes the same time interval under different sampling structures.It maps timestamps into a learnable manifold and injects distance-based biases into attention.Source: §1 · pp. 2–3
• Frequency-guided dynamic variable interaction selects patches across asynchronously observed variables and weights contributions by frequency similarity.The model selects temporally nearby patches and uses local and global frequency-pattern similarities to determine aggregation weights.Source: §1 · pp. 2–3

2 RELATED WORKS

• End-to-end irregular time-series methods model uneven or asynchronous observations from several perspectives, but most are task-specific and not designed for large-scale pretraining or zero-shot transfer.The passage identifies recurrent, continuous-time, set-, graph-, attention-, and patch-based approaches.Source: §2 · p. 3
• QiYao-I is built as a foundation model for irregular multivariate series and directly models irregular timestamps in a learnable temporal manifold.The related-work discussion contrasts it with task-specific end-to-end models and regular-input foundation models.Source: §2 · p. 3
• Most time-series foundation models assume regular sampling and may require resampling or interpolation for irregular inputs, while local sampling structure remains underexplored.MIRA introduces continuous-time modeling for irregular medical time series, but the passage says local sampling structure remains underexplored.Source: §2 · p. 3

3.1 FRAMEWORK OVERVIEW

• QiYao-I combines temporal manifold attention, irregular spectral representations, and frequency-guided cross-variable interaction to model irregular temporal dependencies and asynchronous variable interactions.At future timestamps, it fuses temporal states with extrapolated frequency representations to produce forecasts.Source: §3.1 · pp. 3–4

3.2 SAMPLING-CONDITIONED TEMPORAL MANIFOLD ATTENTION

• Sampling-conditioned temporal manifold attention maps timestamps into a learnable space and turns pairwise manifold distances into attention biases.Its bias bandwidth adapts to local sampling structure, allowing the same distance to receive different penalties under different observation patterns.Source: §3.2 · pp. 4–5

3.3 LEARNABLE IRREGULAR SPECTRAL REPRESENTATION

• Learnable sinusoidal bases evaluated at actual timestamps extract frequency representations directly from irregular observations without resampling.The model aggregates historical spectra and rotates them to future query timestamps, assuming historical spectral components remain relevant over the forecast horizon.Source: §3.3 · p. 6

3.4 FREQUENCY-GUIDED CROSS-VARIABLE INTERACTION

• Frequency-guided cross-variable interaction selects nearby patches from other variables and weights their contributions using local and global frequency similarities.It ranks patches by interval distance, then gates the resulting message into future-token states.Source: §3.4 · pp. 6–7

3.5 FORECASTING OBJECTIVE AND PRE-TRAINING

• For each valid future query, the fused temporal-spectral state produces quantile predictions, with the median mapped back to the original scale at inference.The prediction quantile set includes 0.5.Source: §3.5 · p. 7

4.1 EXPERIMENTS SETUP

• Experiments use four pretraining sources and eight open-source downstream datasets across seven domains, comparing foundation models and, for few-shot evaluation, end-to-end irregular forecasters.The authors report no test-data leakage.Source: §4.1 · pp. 7–8

4.2.1 COMPARED TO OTHER FOUNDATION MODELS

• QiYao-I achieves the best or second-best average results on most datasets, with performance and robustness across data scales, prediction ranges, and time-series distributions.The comparison covers regular-input and irregular-input foundation models.Source: §4.2.1 · p. 8, Table 1 · p. 8

4.2.2 COMPARED TO OTHER END TO END IRREGULAR MODELS

• Using 10% of downstream training data, QiYao-I achieves 10 best results among 16 metrics against 15 end-to-end irregular models trained on full datasets.It records the best MSE and MAE simultaneously on CESNET, APTC, ClusterTrace, and Seabirds.Source: Table 2 · p. 8, §4.2.2 · p. 9

4.3 MODEL ANALYSIS

• Ablations show that temporal manifold bias outperforms standard RoPE and CT-RoPE, while removing cross-variable interaction or using dense interaction degrades performance.The studies cover FNSPID, Seabirds, and Human Activity; Table 3 defines “Full” as dense interaction and “Aware” as selective frequency-guided interaction.Source: §4.3 · p. 9, Table 3 · p. 9
• For identical endpoint timestamps, temporal range, and valid-observation count, different local sampling structures yield different attention for the same endpoint pair.The illustrated pair is q = tL = 544.8 and k = tR = 575.3.Source: §4.3 · p. 9, Figure 3 · p. 9

• QiYao-I uses sampling-conditioned temporal manifold attention and frequency-guided patch interaction to capture irregular temporal dependencies and asynchronous cross-variable relations.Source: §5 · p. 9
• The authors used generative AI tools for language polishing and say the AI-assisted text was reviewed and verified.Source: AI USE STATEMENT · p. 10
• Experiments use existing pretraining and downstream datasets, without human subjects or intentional collection of personally identifiable information.The authors state that they are not aware of direct ethical risks from the methodology itself.Source: ETHICS STATEMENT · p. 10
• The authors report that the model architecture, experimental settings, dataset details, implementation, configurations, and evaluation protocols are provided for reproducibility.They state that source code is available and necessary scripts will be released.Source: REPRODUCIBILITY STATEMENT · p. 10

• Pretraining draws from GIFT-Eval Pretrain, GIFT-Eval training data, the Chronos corpus, and a processed irregular corpus, with respective sampling probabilities of 0.30, 0.25, 0.20, and 0.25.The stream samples records and windows without loading the full corpus into GPU memory.Source: §A.1 · p. 15
• Historical windows retain their native sampling or undergo point deletion, block deletion, sparse subsampling, or timestamp resampling.Timestamp resampling interpolates only between adjacent originally valid historical observations and does not use future targets.Source: §A.1 · p. 15, Table 4 · p. 15
• Each variable in a native multivariate record is augmented independently, producing asynchronous views while preserving the original variable grouping.Source: §A.1 · p. 15

A.2 DOWNSTREAM BENCHMARKS

• Eight datasets span climate, activity, cybersecurity, engineering, finance, social events, and nature, with varied sampling patterns and missing rates.Data are standardized; datasets with sufficient records are split at record level, while some datasets with few long records use chronological splits and rolling windows.Source: §A.2 · p. 15, Table 5 · p. 15
• Dataset-specific lookback and prediction lengths represent real-time ranges, and evaluation excludes invalid future targets while applying the same split and target mask across models.Normalization statistics exclude test data; USHCN uses z-score normalization and other datasets use masked min–max normalization.Source: §A.2 · p. 16, Table 6 · p. 16

A.3 BASELINES AND INPUT ADAPTATION

• QiYao-I and MIRA receive irregular timestamps and masks directly, while regular-input foundation models receive linearly interpolated histories; few-shot baselines use full training splits versus QiYao-I’s 10%.The same valid future targets are used for scoring, and rankings should be interpreted with the difference in training-data access in mind.Source: §A.3 · p. 17

A.4 MODEL AND OPTIMIZATION SETTINGS

• Pre-training proceeds from univariate windows to multivariate records, while few-shot adaptation uses a fixed 10% subset of each downstream training split.Both pre-training stages use AdamW with peak learning rate 6 × 10^-5; the reported checkpoint uses history length 576 and prediction length 192.Source: §A.4 · pp. 17–18

A.5 EVALUATION METRICS AND AGGREGATION

• Errors are computed per variable over valid future targets, then Macro-MSE averages across variables with at least one valid target.Dataset AVG rows are arithmetic means of the two prediction lengths, calculated separately for MSE and MAE.Source: §A.5 · pp. 18–19

B.1 EFFICIENCY ANALYSIS

• 8.07 s inference time and 4.08 GiB peak GPU allocation accompany QiYao-I’s lowest ClusterTrace macro MSE; Toto-2 is faster and uses less memory but has higher MSE.The evaluation uses 37 test windows in 10 batches of up to four.Source: Figure 4 · p. 20, §B.1 · p. 20

B.2 ANALYSIS OF TOPK SELECTION

• On Human Activity, both errors decrease as K rises from 1 to 16; on USHCN, both improve through K = 8, then rise slightly at K = 16.The results support selective rather than unrestricted cross-variable interaction because the optimal neighbor-set size depends on relevant information.Source: §B.2 · p. 20

B.3 ANALYSIS OF INTERPOLATION METHOD

• Linear interpolation supplies regular-input baselines with gridded histories without extra training, while a separate comparison evaluates Chronos-2, Toto2, and MOIRAI2 with BayOTIDE-interpolated inputs.Table 8 reports zero-shot results averaged over two prediction lengths on CESNET, FNSPID, and Human Activity; QiYao-I directly models irregular observations.Source: §B.3 · p. 20, Table 8 · p. 21

B.4 SAMPLING-CONDITIONED BANDWIDTH

• The full model has lower MSE and MAE than fixed-bandwidth and no-bias interventions across evaluated prediction lengths on ClusterTrace and USHCN.Fixed bandwidths show larger performance gaps on ClusterTrace than USHCN; removing manifold bias also increases errors on both datasets.Source: §B.4 · p. 22, Table 9 · p. 22
• Adaptive bandwidth and bias respond to changed sampling structure in all 24 layer-head combinations, while fixed-bandwidth endpoint bias remains unchanged.The paired inputs share endpoints and point counts, so the response distinguishes sampling patterns beyond endpoint interval and observation count.Source: §B.4 · p. 22, Table 10 · p. 22

B.5 FREQUENCY-BASED ROUTING WITH FIXED NEIGHBORS

• Combined frequency routing reduces average MSE versus uniform aggregation by 0.39% on Human Activity and 1.09% on FNSPID, with task-dependent benefits.Local-only routing is slightly better on Human Activity, while global-only is slightly better on FNSPID; compared settings use identical temporal neighbors and validity masks.Source: §B.5 · pp. 22–23, Table 11 · p. 23

B.6 PRE-TRAINED VERSUS RANDOM INITIALIZATION

• Pre-training lowers mean MSE and MAE on CESNET and ClusterTrace, but results are mixed on USHCN across prediction lengths.Across the two lengths, USHCN shows 0.78% lower MSE but 0.18% higher MAE; the comparison uses the same fixed 10% training subset and three seeds.Source: §B.6 · pp. 23–24, Table 12 · p. 23

B.7 LIMITATIONS

• Evaluation uses median point forecasts with MSE and MAE, which do not establish predictive-quantile calibration; the pre-training corpus is underexplored.The authors say a controlled comparison of naturally irregular data, synthetic data, and their combination could clarify their respective contributions to cross-dataset generalization.Source: §B.7 · p. 24

C MORE RELATED WORKS

• Related work spans forecasting, missing and irregular observations, exogenous information, probabilistic forecasting, and foundation models.For irregular multivariate series, the discussion highlights asynchronously observed variables and dynamic patch-level interactions that select relevant source patches for each target patch.Source: C MORE RELATED WORKS · pp. 24–25

D FULL RESULTS

• Four zero-shot tables report per-horizon MSE and MAE results for CESNET, APTC, ClusterTrace, Seabirds, Human Activity, FNSPID, GDELT, and USHCN.The captions mark best and second-best values; the USHCN ranking includes only models with verified per-horizon results.Source: Table 13 · p. 26, Table 14 · p. 26, Table 15 · p. 27
• The captions specify that lower MSE and MAE are better and that few-shot table scales match the main few-shot table.They identify highlighted rankings but do not state the underlying values or which model wins.Source: Table 13 · p. 26, Table 17 · p. 28
• Four few-shot tables report per-horizon MSE and MAE for the same eight datasets, with QiYao-I using 10% training and end-to-end baselines using full-shot training.The captions describe best and second-best highlighting, with ties at reported precision highlighted equally.Source: Table 17 · p. 28, Table 18 · p. 28, Table 19 · p. 29