PaperlayerView exact revision on arXiv

Source-linked AI summary

Fracast-0: Fractal Weight Sharing for a Time Series Foundation Model with Only 85K Parameters

Tianxiang Zhan, Huanyao Zhang, Yuanpeng He

arXiv:2609.32209v1cs.AI

TL;DR

Time series foundation models need breadth, probabilistic output, and multiple temporal scales without parameter growth at every scale. Fracast-0 reuses operators across scales, and on GIFT-Eval it is the smallest evaluated checkpoint while remaining non-dominated. The results support cross-scale weight reuse as a practical route to compression, within the tested scope.

  • Problem

    A general probabilistic forecaster must preserve multi-domain breadth and multiple temporal scales without parameter count growing as each scale receives a separate representation.

    Source: §1 · p. 1

  • Method

    Fracast-0 uses a parameter-free seasonal detector and shared encoder and decoder blocks across dilation ladders, with pretraining across six corpora.

    Source: §1 · p. 2

  • Results

    On 97 GIFT-Eval configurations, Fracast-0 is the smallest of 28 evaluated checkpoints and remains non-dominated, with MASE 0.808 and WQL 0.564.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support cross-scale weight reuse as a practical route to a smaller general forecaster while preserving the breadth, quantile output, and multiscale setting studied.

    Source: §1 · p. 2

    Evidence is limited to one Fracast-0 architecture and one benchmark, and large-scale ablations were not run because compute was limited.

    Source: §5 · p. 10

Abstract

from arXiv · show

Time series foundation models must preserve multi-domain breadth, probabilistic output, and multiple temporal scales, but parameter count grows when each scale receives a separate representation. We introduce Fracast-0, a probabilistic forecasting foundation model that exploits temporal self-similarity to reuse one operator across scales. A parameter-free detector extracts significant seasonal structure. The encoder applies a shared local block along a geometric dilation ladder with scale conditioning, while the decoder combines context-gathered states with an explicit seasonal future state and reuses a second block along another ladder before emitting nine quantiles. Pretraining across six corpora preserves multi-domain breadth within 85,001 parameters. On 97 GIFT-Eval configurations without per-dataset fine-tuning, Fracast-0 is the smallest of 28 evaluated checkpoints and remains non-dominated in the aggregate parameter-accuracy plane with MASE 0.808 and WQL 0.564. It uses 42.0% fewer parameters than TinyCast, whose MASE and WQL are 4.2% and 3.3% lower. These results support cross-scale weight reuse as a practical route to further time series foundation model compression.

1 Introduction

Fracast-0 asks whether cross-scale weight reuse can preserve breadth, probabilistic output, and multiple temporal scales with fewer parameters. It answers with a shared-operator design and remains non-dominated on GIFT-Eval.

1 Introduction

  • The design question is whether a general probabilistic forecaster can preserve multi-domain evidence at smaller scale by reusing operators across scales.The motivation is to preserve breadth, probabilistic output, and multiple temporal scales without parameter growth as scale count increases.Source: §1 · p. 1
  • Figure 1 contrasts separate encoder and decoder parameter sets at every scale with Fracast-0's reused operators, whose stored count does not grow with scale rungs.This makes the paper's cross-scale parameter-sharing choice concrete.Source: §1 · p. 1, Figure 1 · p. 2
  • Fracast-0 uses a parameter-free seasonal detector, a shared encoder block across a geometric dilation ladder, and a second shared decoder block before emitting nine quantiles.Pretraining across six corpora supplies breadth within an 85K-parameter budget.Source: §1 · p. 2
  • On 97 GIFT-Eval configurations, Fracast-0 is the smallest of 28 evaluated checkpoints and remains non-dominated with MASE 0.808 and WQL 0.564.TinyCast uses 42.0% more parameters and improves both aggregate metrics by 4.2% and 3.3%.Source: §1 · p. 2

2 Related Works

Related work spans self-similar and multiscale models as well as time series foundation models. Fracast-0 shares parameters inside its forecasting block, against a broader design space of representations, training regimes, and benchmarks.

Self-similar and multiscale models

  • Fracast-0 is closest to Scaleformer, which shares weights while iteratively refining predictions across scales; Fracast-0 instead shares parameters inside the forecasting block itself.Its encoder and future-state decoder each reuse one local update along dilation ladders, with scale conditioning distinguishing rungs.Source: §2 · p. 2
  • Prior self-similar approaches include FractalNet's recursive topology, WaveNet's dilated receptive fields, ALBERT's tied depth weights, and time-series models such as TimesNet, TimeMixer, and FlowState.These works apply self-similarity or multiscale structure in different ways.Source: §2 · p. 2

Time series foundation models and GIFT-Eval

  • Foundation models differ in representation and modeling choices, including patching, numeric tokenization, arbitrary variates and horizons, recurrent state, streaming, and heterogeneous training.The related work also covers compact or adaptive models, output scaling, and training-regime choices.Source: §2 · pp. 2–3
  • GIFT-Eval compares 23 datasets across domains and sampling frequencies, reporting MASE and WQL across short, medium, and long horizons without per-dataset fine-tuning.The paper places Fracast-0 at the 85K point on the parameter-accuracy frontier.Source: §2 · p. 3

3 Methodology

Fracast-0 turns a univariate context window into quantile forecasts through preprocessing, an encoder, a decoder, and a training objective designed for rollout. Channels are forecast independently with shared parameters.

3 Methodology

  • The three-stage pipeline maps normalized context to an encoder hidden state and then to horizon forecasts, with the same parameters used to forecast channels independently.Figure 2 shows the overall path and expands the encoder and decoder in separate panels.Source: §3 · p. 3, Figure 2 · p. 3

3.1 Preprocessing

  • Preprocessing normalizes a masked univariate window, projects value, coverage, and recency features, and adds a periodic prior from significant periods detected without learned parameters.WindowMinMax uses each window's range; retained periods contribute sine and cosine channels, and the periodic prior is projected into the model width.Source: §3.1 · pp. 3–4

3.2 Encoder

  • The encoder applies one local block across a geometric dilation ladder, using log-scale FiLM conditioning so the same operator serves all ten scales.The block uses a causal depthwise-separable convolution and feed-forward path; its 23,232 parameters are shared across the ladder.Source: §3.2 · pp. 4–5

3.3 Decoder

  • The decoder combines a context-gathering path with a causal future-state path initialized from a seasonal reference, then projects their sum to horizon quantiles.The seasonal fill uses phase bins, while the future-state path reuses a decoder-specific block along dilations 1, 2, 4, 8, 16, and 32.Source: §3.3 · p. 5

3.4 Training

  • Training uses pinball loss for quantile forecasts and a commitment term that penalizes positions where the median is less accurate than the seasonal-copy reference.The commitment term is disabled when the seasonal copy is not better over the window; training also uses scheduled sampling across future chunks.Source: §3.4 · p. 6

4 Experiments

Experiments evaluate Fracast-0 on GIFT-Eval without per-dataset fine-tuning and compare its forecasting accuracy and local CPU deployment profile. The results assess its parameter budget alongside forecasting and deployment trade-offs.

4 Experiments

  • The GIFT-Eval benchmark covers 23 datasets, seven domains, and ten sampling frequencies; Fracast-0 is pretrained on six public corpora without using the GIFT-Eval train split.Evaluation follows the official test protocol across 97 configurations, with nine quantile levels and no per-dataset fine-tuning.Source: §4 · p. 6
  • Reusing one encoder block across dilation scales reduces dilated-convolution parameters by 59.9%; the periodic prior gives the largest single reduction, at 96.5%.Further savings come from the future-state path, while FiLM conditioning and a wider gather head add parameters.Source: §4 · p. 6, Table 1 · p. 7

4.1 Forecasting accuracy

  • 85K-parameter Fracast-0 is non-dominated on GIFT-Eval, with 0.808 MASE and 0.564 WQL; 147K-parameter TinyCast scores 0.774 MASE and 0.545 WQL.The comparison aggregates 97 configurations without per-dataset fine-tuning.Source: §4.1 · pp. 6–7, Table 2 · p. 8, Figure 4 · p. 8

4.2 Deployment efficiency

  • 23.9 ms CPU latency and 389 MiB resident memory make Fracast-0 faster and lighter than TinyCast in the fixed local batch-1 test, but not the fastest model overall.The benchmark uses a fixed random-input workload; resident memory is measured after inference.Source: §4.2 · pp. 8–9, Figure 6 · p. 9
  • Memory-constrained local scoring favors Fracast-0 for many daily and hourly series, while sub-hourly forecasts and extended rollouts may benefit from added capacity.Deployment choices should weigh horizon, frequency, and local resource budget rather than parameter count alone.Source: §4.2 · p. 9

5 Conclusion

Fracast-0 reuses blocks across dilation ladders and remains non-dominated on 97 GIFT-Eval configurations. The results support cross-scale weight reuse as a compression route, while the evidence is limited to one architecture and one benchmark.

5 Conclusion

  • Fracast-0 reuses one block along each dilation ladder to exploit temporal self-similarity.The results support this as a viable compression path, not a general law.Source: §5 · pp. 9–10
  • 85K parameters place Fracast-0 among non-dominated checkpoints across 97 GIFT-Eval configurations, with MASE 0.808 and WQL 0.564.It uses 42.0% fewer parameters than TinyCast and has 42.3% lower batch-1 CPU latency.Source: §5 · pp. 9–10
  • Evidence is limited to one Fracast-0 architecture and one benchmark, and large-scale ablations were not run because compute was limited.Future work should test more stocks, horizons, and regimes, and examine which weights to merge and whether depth still helps.Source: §5 · p. 10

A.1 Training and Evaluation Protocol

Fracast-0 is pretrained with a specified model and optimization setup, then evaluated on 97 GIFT-Eval configurations without per-dataset fine-tuning. Evaluation uses nine quantiles and Seasonal Naive-normalized geometric-mean scores.

A.1 Training and Evaluation Protocol

  • The training setup uses 85,001 parameters, width 64, ten dilations from 1 to 512, a 2,048-step context, and a 48-step horizon.The model emits nine quantiles.Source: §A.1 · p. 12
  • Pretraining uses AdamW with β2 = 0.95, weight decay 0.01, gradient clipping 1.0, and a peak learning rate of 3 × 10^-3.The released checkpoint runs 40,075 steps, about 164M windows, under bf16 autocast.Source: §A.1 · p. 12
  • GIFT-Eval evaluation covers 97 released test configurations without per-dataset fine-tuning, grouped into 55 short, 21 medium, and 21 long members.Each evaluation unit is defined by dataset, sampling frequency, and forecast term.Source: §A.1 · p. 12
  • MASE uses the median forecast and WQL uses nine quantiles; both are Seasonal Naive-normalized, and aggregate scores are geometric means giving each configuration equal weight.Lower normalized scores are better.Source: §A.1 · p. 12

A.2 Qualitative short-horizon forecasts

Figure 7 compares short-horizon forecasts on six selected GIFT-Eval windows, spanning varied daily and hourly signals. Fracast-0 has the lowest panel-title nMAE in each, with the clearest advantage under abrupt load changes.

A.2 Qualitative short-horizon forecasts

  • Fracast-0 has the lowest panel-title nMAE in all six selected windows, but the margins differ sharply.Its relative gains over TinyCast and TimesFM-3 are 71.6% and 81.3% on Electricity, versus 5.3% and 2.7% on KDD Cup.Source: §A.2 · p. 12
  • The forecasts preserve dominant periodic timing on ETTh1, Solar, and Jena Weather, while remaining baselines track similar levels or cycles elsewhere.M4 Daily and KDD Cup have nearly overlapping medians, so their small score differences are not visually decisive separations.Source: §A.2 · p. 12
  • Figure 7’s shaded band is Fracast-0’s 10–90% quantile interval, and each panel title reports normalized MAE for its selected window.The six selected cases do not estimate an aggregate win rate.Source: Figure 7 · p. 12, §A.2 · p. 12

A.3 Official By Domain comparison

The official seven-domain comparison reports MASE for ten checkpoints. Fracast-0 is below Seasonal Naive across domains but leads none; TinyCast is better on six domains, with Sales the exception.

A.3 Official By Domain comparison

  • Fracast-0 remains below Seasonal Naive but does not lead any of the seven domains.TinyCast is better on six domains, with Sales as the exception.Source: §A.3 · p. 13
  • TiRex-2, Chronos-2, TimesFM-3, Toto-2.5B, and Falcon-2.0 provide the strongest domain-level references.The comparison covers ten checkpoints at GIFT-Eval Space.Source: §A.3 · p. 13
  • Table 3 presents the official GIFT-Eval by-domain comparison, with each cell reporting MASE.Source: Table 3 · p. 13, §A.3 · p. 13

A.4 Official By Frequency comparison

Fracast-0’s frequency results show its largest shortfalls at sub-hourly frequencies, while it improves over TinyCast on longer-period groups.

A.4 Official By Frequency comparison

  • Fracast-0 is worse than TinyCast at Secondly and Minutely by 25.8% and 9.2%, but better on Monthly, Quarterly, and Yearly by 3.4%, 6.8%, and 12.4%.The frequency comparison reports MASE for eight groups.Source: §A.4 · p. 13, Table 4 · p. 13
  • Fracast-0 is below Seasonal Naive in seven frequency groups; only Secondly is above 1.0, at 1.141.Source: §A.4 · p. 13, Table 4 · p. 13
  • On Monthly, Quarterly, and Yearly groups, Fracast-0 improves over TinyCast but remains behind TimesFM-3 and Falcon-2.0.Source: §A.4 · p. 13, Table 4 · p. 13

A.5 Official By Variate Type comparison

Fracast-0 has higher MASE on multivariate than univariate inputs and trails TinyCast in both groups; the authors also describe AI assistance and review practices.

A.5 Official By Variate Type comparison

  • MASE is 0.831 on multivariate inputs and 0.790 on univariate inputs, a 0.041 absolute gap and 5.2% relative increase.Every checkpoint in the table has higher multivariate MASE.Source: §A.5 · p. 13, Table 5 · p. 14
  • Fracast-0 trails TinyCast in both input groups and leads neither, so its accuracy gap is not specific to one input type.Source: §A.5 · p. 13, Table 5 · p. 14

B AI use statement

  • Generative AI assisted literature work, research framing, methods, coding, data processing, results interpretation, and manuscript and research-artifact preparation.Source: B AI use statement · pp. 13–14
  • The authors say AI did not generate datasets or prove claims; they reviewed its contributions, manually checked equations, tested code, reproduced results, and retained final decisions.Source: B AI use statement · p. 14

Related papers

Loading 2609.32209v1…