Rethinking Agentic Model Orchestration for Time Series Forecasting

1 Introduction

• Different TSFMs excel in different forecasting settings, so no single model consistently performs best across datasets and horizons.The introduction describes this performance heterogeneity as motivation for orchestration.Source: §1 · p. 1
• AMO leaderboard gains do not isolate orchestration because systems use different candidate pools, whose strength and complementarity affect performance and available headroom.The same orchestrator can perform differently with stronger candidates, while complementary members leave more headroom than redundant ones.Source: §1 · pp. 1–2
• FairAMO compares orchestrators using identical candidate forecasts and specified historical evidence, then varies pool composition, size, and evidence budget.Orchestrators commit to selection or combination weights before test labels are revealed, and forecasts are scored under the official GIFT-Eval protocol.Source: §1 · p. 2

Taxonomy

• The taxonomy distinguishes LLM-mediated, feature-conditioned, and historical-error-driven orchestration by decision mechanism, with multiple mechanisms possible in one system.The distinction is about how the decision is formed, not merely whether training uses a loss function.Source: §2 · p. 2

Reproducibility

• In artifacts inspected in September 2026, some methods exposed inference artifacts without complete orchestration training code, while others exposed code for deriving and applying decisions.The authors reused or adapted available repository components to reproduce methods.Source: §2 · p. 2

3 Methodology

• All candidate TSFMs run once, and their cached forecasts form a bank of training backtests, optional validation backtests, and test forecasts.Training and validation backtests include ground truth; test ground truth is withheld.Source: §3 · p. 2, Figure 2 · p. 3
• Before test labels are revealed, each orchestrator uses the permitted recent training backtests and optional validation evidence to commit to a model selection or combination weights.The strategy is frozen for test evaluation; rerunning, reweighting, or inspecting test scores after commitment is not permitted.Source: §3 · p. 3, Figure 2 · p. 3
• The evaluation includes reference, external, and re-implemented orchestrators, including simple selection and weighting baselines and published methods.The reference group includes a non-deployable oracle that selects the best member per input window using test labels.Source: §3 · p. 3

RQ1

• 0.658 MASE from TSOrchestra SLSQP improves on TiRex2's 0.697 by 0.039 (5.6%), while the window-level oracle reaches 0.580.The strongest deployable orchestrator still leaves two thirds of the distance to the oracle uncovered.Source: §4 · p. 3

RQ3

• 0.670–0.690 MASE spans the ten size-3 subsets; pools with TiRex2 realize 28.5% of headroom on average versus 13.4% without it.The best single TSFM scores across subsets differ by only approximately 0.006, indicating that composition sensitivity tracks complementarity rather than candidate strength.Source: Figure 5 · p. 4, §4 · p. 4

RQ2

• 0.699→0.697 MASE for the best single TSFM contrasts with the oracle's 0.647→0.580 and Argmin Router's 0.689→0.662 as pool size grows from two to five.Argmin's headroom realization rises from 18.4% to 29.8%, but the third member adds 5.1 points and the fourth and fifth about 3 points each.Source: Figure 3 · p. 4, §4 · p. 4
• 0.672 with validation only, 0.661 after three training backtests, and 0.664 at 100 describe Argmin Router's evidence-budget pattern; SLSQP peaks at 0.658 with ten backtests.Larger evidence budgets bring no further gains and can degrade performance.Source: Figure 4 · p. 4, §4 · p. 4

RQ3

• Although Toto2-313m is the weakest single model at 0.703, adding it improves five of six eligible size-2 pools; Chronos2+Toto2-313m realizes only 3.1% despite 0.048 headroom.The results favor pool construction by differences in error profile rather than single-model ranking.Source: §4 · p. 4

Conclusion

• AMO improves accuracy, but its gains are bounded by the candidate pool, depend on complementarity rather than pool size, and historical evidence quickly saturates.These are the central conclusions under FairAMO.Source: §5 · p. 4
• The strongest methods are simple selection and weighting rules, leaving the contribution of LLM-mediated orchestration unclear.The conclusion identifies finer-grained commitments and context beyond backtests as directions for future work.Source: §5 · p. 4
• A single decision per config limits realization of the available window-level headroom.The authors identify finer-grained commitments as a possible direction for addressing this limit.Source: §5 · p. 5

• All reported numbers use the official GIFT-Eval scorer on its official test split, making scores directly comparable with the public leaderboard.Source: A Metrics · p. 6
• MASE scales point-forecast absolute error by seasonal-naive error, while CRPS scores predictive-distribution calibration; lower is better for both.MASE below one beats the seasonal-naive baseline.Source: A Metrics · p. 6
• A config pairs a dataset with a forecast term, and each window is a rolling origin; the orchestrator's decision stays fixed across a config's windows.Source: A Metrics · p. 6
• Dev-Selected Best reaches 0.702 MASE versus 0.697 for the best single TSFM; the window-level oracle is a non-deployable per-window ceiling.The best-single baseline requires test labels, whereas Dev-Selected Best uses permitted evidence.Source: A Metrics · p. 6
• Headroom H(P) is best-member MASE minus oracle MASE; gain G is best-member minus method MASE, and HR=G/H compares recovered headroom across pools.Source: A Metrics · pp. 6–7
• The val+B_n budget combines validation backtests with the n most recent training backtests; unless stated otherwise, results use val+B10.Source: A Metrics · p. 7

• Table B1 reports main-pool val+B10 results, grouped as reference, external, and re-implemented methods; G is gain over the best single TSFM, while HR is the fraction of 0.117 headroom realized (G/0.117); bold marks the best deployable result.The oracle row may switch members per window and is a non-deployable ceiling.Source: Table B1 · p. 7
• Table B2 averages results over all subsets at each pool size and reports oracle ceiling, headroom H, and the Argmin Router's realized fraction HR.Bold marks the largest headroom and best Argmin result.Source: Table B2 · p. 7
• Table B3 sweeps evidence budgets on the main pool, reporting MASE and marking the best budget for each method that uses training backtests.Rows for methods that do not use training backtests remain constant.Source: Table B3 · p. 8
• Table B4 reports Argmin Router results for every size-2 and size-3 subset at val+B10, sorted by HR within each subset size.It includes each subset's best single TSFM and headroom H; bold marks the highest HR within each size.Source: Table B4 · p. 8