PaperlayerView exact revision on arXiv

Source-linked AI summary

MetaCaster: Meta-Harness-Optimized Agent for End-to-End Few-Shot Learning of Lightweight Time Series Forecasters

ChengAo Shen, Wenchao Yu, Fangyu Wu, Dongjin Song, Hanghang Tong, Dongsheng Luo, Wei Cheng, Haifeng Chen, Jingchao Ni

arXiv:2608.23473v1cs.LGcs.AI

TL;DR

Lightweight time-series forecasters are more economical than large models but usually need substantial data, limiting few-shot deployment. MetaCaster uses agents to generate task-specific data and train or select specialized forecasters, achieving data and computational efficiency with high-quality performance across broad experiments.

  • Problem

    Lightweight forecasters offer resource efficiency but typically require substantial training data, creating a few-shot learning gap for scarce or privacy-sensitive time series.

  • Method

    MetaCaster uses a meta-harness-optimized multi-agent framework to generate domain-compliant data, then train and select specialized lightweight forecasters from a few examples and textual context.

  • Results

    Across 18 datasets, MetaCaster demonstrates data and computational efficiency while maintaining high-quality forecasting performance against 14 baselines.

  • Takeaways & Limitations

    The results support an agent-as-engineer paradigm in which agents prepare efficient, task-specific lightweight forecasters rather than directly producing forecasts.

  • Takeaways & Limitations

    MetaCaster does not address the extreme zero-shot setting, where lacking reference time series can make generated data unreliable.

Abstract

from arXiv · show

Time series forecasting (TSF) is evolving toward multimodal and agentic settings, yet using foundation models remains uneconomical in resource-constrained scenarios, where compact, specialized forecasters are more desirable. However, lightweight forecasters typically require substantial training data, limiting their use in domains with scarce, slowly accumulated, or privacy-sensitive time series. To address this dilemma, we investigate the challenging problem of few-shot learning for lightweight forecasters. We propose MetaCaster, a meta-harness-optimized multi-agent framework that uses agentic data generation to automatically train specialized lightweight forecasters from only a few examples and textual contexts. Our work highlights a new TSF paradigm in which agents act not as forecasters but as intermediary engineers that prepare efficient, task-specific forecasters for deployment. Experiments on 18 datasets, 23 state-of-the-art lightweight forecasters, and 14 baselines demonstrate that MetaCaster achieves both data efficiency and computational efficiency while maintaining high-quality TSF performance.

1 Introduction

Few-shot learning for lightweight time-series forecasters is motivated by the cost, delay, and infeasibility of collecting large datasets, especially in privacy-sensitive domains. METACASTER addresses this challenge with agents that generate task-specific training data and prepare specialized forecasters rather than directly forecasting.

  • Motivation: Dedicated numerical models remain competitive with LLM-centered forecasting, whose effectiveness is limited by the modality gap between language tokens and continuous time-series values.The passage names DLinear and PatchTST as examples of specialized numerical models.
  • Motivation: Large training datasets can delay deployment and may be infeasible in privacy-sensitive healthcare and finance applications.These constraints motivate learning lightweight forecasters from only a few samples.
  • METACASTER: METACASTER uses two agents to prepare a specialized forecaster for a target task from a few-shot support set Dsup and contextual description C.Its agents act as intermediaries instead of directly generating forecasts; MGAGENT generates a sufficient dataset from Dsup and C.
  • METACASTER: METACASTER generates time series specifically to improve forecasting performance, aligning data generation with forecaster quality.The authors distinguish this objective from existing time-series generation models focused on data simulation and describe the framework as the first of its kind to make this alignment.
  • Contributions: 20+ state-of-the-art lightweight forecasters are compiled into LT-LIB, a unified library.The passage lists few-shot learning for lightweight forecasters and efficient inference as contributions alongside the library.

2 Related Work

Prior LLM-based time-series forecasting work primarily treats LLMs or agents as forecasters, while relatively fewer approaches use agents as engineers through harnesses. Existing standalone data generators simulate data properties rather than directly optimizing forecasting performance, leaving a gap addressed later in the paper.

3 The Proposed Method

MetaCaster addresses few-shot learning for non-pre-trained lightweight forecasters by generating domain-specific training data from limited examples and textual context. Its meta-harness framework coordinates a Meta-Generator, Forecaster Trainer, and Harness Proposer to optimize data generation for efficient deployment.

  • Problem Formulation: The task is to train a library of small, non-pre-trained forecasters from a K-shot support set and textual domain context, despite scarce data.The target is optimal forecasting performance on a test set, with difficulty increasing as K becomes small.
  • MetaCaster Overview: MetaCaster generates a dataset with N′ ≫ K examples from the support set and context, then splits it for forecaster training and validation.The generated data is intended to provide sufficient training examples despite the limited original data.
  • MetaCaster Overview: Unlike realism-focused generators, MetaCaster optimizes generated data so forecasters trained on it perform comparably to those trained on same-sized real target-domain data.This objective is designed to avoid mismatches that can produce biased forecasters.
  • Meta-Generator: MGAGENT uses an LLM to analyze few-shot series, create a domain-specific TS-Generator program, and perform quality checks before accepting generated data.The LLM orchestrates tools through planning, reasoning, and coding rather than directly generating multivariate time series.
  • Harness Proposer: HPAGENT optimizes MGAGENT’s trainable Harness parameters θ across multiple epochs, using long-term memory to support rollback and select the best θ.The LLM is frozen, while the system prompt and skills serve as θ and are edited in an outer optimization loop.
  • Forecaster Trainer and Deployment: FTAGENT trains the lightweight forecasters through parallelized hyperparameter-search jobs, while the resulting Harnesses and LT-LIB support low-cost downstream deployment and inference.LT-LIB contains 23 SOTA lightweight forecasters with a unified interface, and FTAGENT monitors, repairs, and recovers training jobs without human intervention.

4 Experiments

Experiments on 18 datasets evaluate MetaCaster across in-domain and out-of-domain settings, showing strong few-shot forecasting, generalization, and efficiency. Ablations and optimization traces indicate that forecasting-oriented harness optimization and textual context are central to its performance.

  • Experimental Setup: MetaCaster is evaluated on 18 datasets spanning 9 domains, with disjoint in-domain and out-of-domain test corpora and K ∈{10, 30, 50} few-shot samples.Datasets are chronologically split into 80%/10%/10% train/validation/test sets, using look-back window T = 336 and prediction horizon H = 192.
  • Main Results: MetaCaster outperforms generation and augmentation baselines in most cases, improves with larger K, remains competitive at K = 10, and generally outperforms baselines on OOD datasets.When K ≥30, its performance approaches or surpasses the full training reference, suggesting optimized data can improve training over raw data.
  • Main Results: MetaCaster produces more high-quality forecasters with lower variance and generalizes better to held-out forecasters than the baselines.TimeScientist lacks data generation, struggles to train generalizable forecasters, and does not scale with K.
  • Efficiency: 103× lower latency and 105× fewer parameters are achieved by MetaCaster than TSFMs at comparable performance on the Solar dataset under K = 30.MetaCaster selects MixLinear with 243 parameters at runtime and incurs no further agent overhead after deployment.
  • Ablation Study: Forecasting-oriented optimization outperforms direct distribution alignment, while contextual cues are crucial for guiding domain-relevant time-series generation.Different LLMs yield comparable results in many cases; GPT-5.4 is adopted for consistent performance because GPT-5.3-Codex is unstable on ETTm1 and USbirths.
  • Further Analysis: MetaCaster converges quickly over 8 harness-optimization epochs and selects the final harness from epoch 5.A diagnosis of broken inter-variate correlation led to generator updates that produced more correlated and few-shot-compliant series in epoch 5 than epoch 4.

5 Conclusion

MetaCaster studies few-shot learning for lightweight forecasters through a meta-harness-optimized multi-agent framework. It automates the end-to-end process of time series generation, forecaster training, parameter tuning, and model selection, establishing groundwork for the Agent-as-Engineer paradigm.

  • 5 Conclusion: MetaCaster investigates few-shot learning for lightweight forecasters with a novel meta-harness-optimized multi-agent approach.The framework is presented as an end-to-end solution for the studied problem.
  • 5 Conclusion: The framework automates time series generation, forecaster training, parameter tuning, and model selection.These components form MetaCaster’s automated end-to-end pipeline.
  • 5 Conclusion: Experiments validate MetaCaster’s effectiveness and establish groundwork for the Agent-as-Engineer paradigm in agentic time series forecasting.The conclusion frames these results as a basis for further exploration of this paradigm.

Limitations · A Algorithm · 19 end

MetaCaster targets few-shot improvement of lightweight time series forecasters for rapid deployment, while acknowledging limits in zero-shot generalization, dataset and library coverage, and the scope of its meta-harness optimization algorithm. The appendix specifies the algorithm’s inputs and optimization loop.

  • Limitations: MetaCaster addresses few-shot improvement of lightweight forecasters for rapid deployment, reducing delays from costly data acquisition and privacy constraints.The approach leverages AI agents’ knowledge and reasoning capabilities.
  • Limitations: Zero-shot forecasting is not addressed because, without reference series, agents lack statistical grounding and may generate unreliable data.The passage identifies this as a common limitation of time series generation-based models.
  • Limitations: Experiments cover 18 datasets across a limited domain range, and future work will extend coverage toward modern TSFMs’ full pre-training corpora.The stated goal is to further improve Harness optimization.
  • Limitations: LT-LIB is not exhaustive because new lightweight forecasters continue to emerge, so future work will continuously update the library.The current library contains collected state-of-the-art lightweight forecasters.
  • A Algorithm: Algorithm 1 summarizes MetaCaster’s meta-harness optimization algorithm, using notation consistent with Section 3.The algorithm is presented as the appendix’s formal description of meta-harness optimization.
  • A Algorithm: The algorithm initializes m=1, LT-LIB F = {f1, ..., fL}, epoch count E, batch size B, and outputs optimized harness θ∗.These are the listed algorithm inputs and output.
  • A Algorithm: The optimization loop runs for i = 1, ..., E and processes a K-shot support set through MGagent.The support-set step is marked as running MGagent according to Eq. (2).

B Datasets and Baselines … C.1 Forecaster Pool

MetaCaster is evaluated under a leakage-controlled few-shot forecasting setup spanning 18 diverse datasets, with standardized data processing and support-set construction. The study compares against synthesis, augmentation, foundation, and agent-driven baselines while providing a pool of 23 lightweight forecasters across four architectural families.

  • B.1 Datasets: To prevent leakage, training and evaluation use disjoint GIFT-Eval collections, with dataset identifiers, source URLs, and benchmark names removed before agent access.Runtime audits further confirmed that MGAGENT issued no external web requests and touched no real test data; synthesis used only the few-shot support set and de-identified context.
  • B.1 Datasets: Training uses 8 datasets across 6 domains, while testing uses 10 datasets split into 7 IND and 3 OOD datasets.The OOD set includes M4, a mixed-domain daily subset probing ambiguous or absent domain information.
  • B.1 Datasets: The 18 datasets span six orders of magnitude in sampling frequency, variate counts D ∈ {1, 2, 5, 7}, and series lengths from 1.2k to 7.4M steps.This coverage exposes univariate, small-multichannel, and widely varying temporal-scale regimes.
  • C Implementation Details: Every method follows an 80/10/10 chronological split, per-channel train-based z-score normalization, and windows with look-back T = 336 and horizon H = 192.Main experiments use K ∈ {10, 30, 50} support windows, while scaling studies use K ∈ {10, 20, 30, 50, 100}; textual domain context is supplied when supported.
  • B.2 Baselines: Baselines include deep generative synthesisers, classical augmentation procedures, the no-synthesis Dm sup floor, and the full-data Dm tr oracle.The study also compares foundation forecasters and TimeScientist, a local-Qwen2.5 multi-agent pipeline that composes and tunes classical predictors per test window.
  • C.1 Forecaster Pool: The lightweight pool contains 23 forecasters grouped into four families and exposed through a uniform LT-LIB training interface.FTAGENT can launch any model using the same forecaster, dataset, and hyperparameter call signature.
  • C.1 Forecaster Pool: The linear family spans direct, decomposed, normalized, cross-correlated, mixed-domain, and MLP-based forecasters, including MixLinear’s O(n^2) to O(n) parameter reduction on downsampled length.Representative models include Vanilla Linear, DLinear, NLinear, RLinear, CrossLinear, MixLinear, TSMixer, LightTS, PatchMLP, xPatch, CMoS, and PatchTSMixer.

C.2 Training Environment

The experiments run on a single node with four RTX 6000 Ada GPUs and a specified Python/PyTorch software stack, while all three MetaCaster agents use OpenAI GPT-5.4 via the API.

  • Hardware: All experiments run on a single node equipped with 4×NVIDIA RTX 6000 Ada GPUs, each with 48 GB of memory.
  • Software stack: The software stack uses Python 3.12.8, PyTorch 2.5.1, CUDA 12.4, cuDNN 9.1, and listed scientific-computing and visualization libraries.The listed libraries include NumPy 2.1.3, Pandas 2.2.3, SciPy 1.15.1, scikit-learn 1.6.1, Matplotlib 3.10.0, and torchvision 0.20.1.
  • Software stack: All three MetaCaster agents—HPAGENT, MGAGENT, and FTAGENT—are driven by OpenAI GPT-5.4 through the OpenAI API.

C.3 Compute Budget · D System Prompts

MetaCaster separates computation into one-time harness optimisation, per-dataset generation and forecaster training, and lightweight per-query inference without LLM calls. Its prompts operationalize HPAGENT’s harness evolution, MGAGENT’s validated synthetic-data generation, and FTAGENT’s parallel forecaster training and selection.

  • C.3 Compute Budget: Harness optimisation runs once per project, with HPAGENT proposing edits, MGAGENT generating candidate data, and FTAGENT scoring trained forecasters.The resulting H⋆ is reused for every subsequent deployment.
  • C.3 Compute Budget: At deployment, MGAGENT generates synthetic data and FTAGENT trains LT-LIB forecasters, while only MGAGENT invokes the LLM.Downstream forecaster training proceeds without an LLM call.
  • C.3 Compute Budget: Per-query inference uses only the deployed lightweight forecaster’s forward pass, requiring a few milliseconds per look-back window on either GPU or CPU.No LLM call lies on the inference path.
  • D System Prompts: The system prompts are rendered with runtime values supplied as <placeholder>, while FTAGENT’s prompt is hand-authored and HPAGENT optimizes MGAGENT’s prompt slot.MGAGENT is prohibited from training forecasters, and FTAGENT must use only registered factory architectures without modifying dataset.npy or test.npy.
  • D.1 Harness Proposer (HPAGENT): HPAGENT’s prompt directs an epoch-by-epoch propose–evaluate–accept loop that edits the harness library rather than generating data.The harness begins empty, and HPAGENT authors validated synthesis skills for MGAGENT to load at deployment.
  • D.1 Harness Proposer (HPAGENT): HPAGENT compares candidates with downstream hinge, distribution-metric, and diagnostic-figure evidence when deciding whether an epoch is a new best.The decision rule prioritizes reduced median downstream hinge and majority-dataset distribution-metric improvement, using visual evidence as a tie-break.
  • D.2 Meta-Generator (MGAGENT): MGAGENT analyzes few-shot data into a fingerprint, routes it to synthesis skills, and validates outputs against shape, distribution, dependence, coverage, and diversity gates.On failure, it retries with a different recipe; successful output is saved as dataset.npy.
  • D.3 Forecaster Trainer (FTAGENT): FTAGENT enumerates forecaster, hyperparameter, and split jobs across the 23-model LT-LIB pool, supervises execution, and returns the lowest-validation-MSE forecaster.It monitors losses and resources, reallocates GPUs, resolves errors, and recovers interrupted jobs without human intervention.

E Supplementary Results

The supplementary results provide MAE counterparts and compare MetaCaster with foundation forecasters and TimeScientist under a shared lightweight-forecaster pool. MetaCaster outperforms TimeScientist on 26 of 30 cells in that shared-pool comparison.

  • MAE counterpart of the main table: Table 7 reports per-dataset MAE for the same nine methods compared using MSE in Table 1.M4 uses instance-normalised MAE.
  • Foundation models on all benchmarks: Foundation forecasters VisionTS, Chronos, Moirai, Time-LLM, and TimeScientist use 91 M–7 B-parameter backbones invoked at every test window.Figure 7 compares MetaCaster with these foundation methods across all 10 evaluation datasets, with bubble area proportional to log10(params).
  • Comparison with TimeScientist under the shared forecaster pool: 26 of 30 cells favor MetaCaster over TimeScientist when both select lightweight forecasters from the shared LT-LIB pool.The comparison uses the per-dataset MSE of each method’s best selected forecaster.

F Further Study

Further studies show that METACASTER remains strong when deployment selects only top-performing forecasters, scales positively with more few-shot inputs, and is robust across seeds and forecasters. Its synthetic windows also align well with real series in both qualitative behavior and distributional structure.

  • Top-K deployment: METACASTER retains the lead across most per-dataset top-K comparisons when methods are evaluated using their 1, 3, or 5 best forecasters.The comparison uses the 20 main-pool forecasters at nfew = 30, reflecting deployment of a single forecaster per dataset.
  • Few-shot input scaling: METACASTER’s Relative MSE improves as nfew grows on most datasets across both IND and OOD subsets.The scaling study evaluates nfew ∈ {10, 20, 30, 50, 100}.
  • Multi-seed sensitivity: METACASTER is seed-insensitive on most datasets when evaluated across three independent LLM-sampling seeds at nfew = 30.Table 9 reports per-dataset MSE as mean ± std across the three seeds.
  • Per-forecaster distributions: METACASTER has lower Relative MSE than every non-Full baseline at every quartile in both IND and OOD, with the narrowest non-trivial distribution.The distributions span 20 × {7, 3} forecaster-dataset cells.
  • Synthetic sample quality: METACASTER-generated windows track dominant periods and local fluctuations, while overlapping real manifolds on most datasets in separate t-SNE analyses.The qualitative comparison covers four representative datasets, and the t-SNE analysis covers 10 evaluation datasets.
Loading 2608.23473v1…