PaperlayerView exact revision on arXiv

Source-linked AI summary

The Nixtlaverse: An Open-Source Ecosystem for Forecasting

Olivier Sprangers, Max Mergenthaler Canseco, Marco Peixeiro, Saul Caballero Ramirez, Mariana Menchero García, Jing-Qiang Goh, Han Wang, Nikhil Gupta, Rogelio Melo, Senbong Gee, Cristian Challu

arXiv:2609.39741v1cs.LG

TL;DR

Forecasting software must either hide model-family differences behind one estimator interface or require users to redo preparation and evaluation across separate packages. The Nixtlaverse instead shares panel and keyed-output contracts while retaining specialized engines; M5 use cases show these contracts supporting cross-engine workflows while leaving behavioral semantics and scalability boundaries visible.

  • Problem

    Forecasting model families differ in fitted state, training, and parallelism, while separate packages can force users to repeat data preparation and evaluation.

    Source: §1 · p. 1

  • Method

    The Nixtlaverse standardizes long-panel inputs and keyed forecast outputs, retains specialized engines, and lets downstream evaluation and reconciliation consume forecasts.

    Source: §1 · p. 2, §3.3 · p. 4

  • Results

    Shared keyed outputs supported common evaluation and reconciliation across resident and external engines, while dense reconciliation was not computationally tractable at the full hierarchy.

    Source: §6 · p. 13, §7 · p. 14

  • Takeaways & Limitations

    The case study supports narrow shared keys for cross-engine workflows rather than a common behavioral interface, with multi-repository coordination remaining a cost.

    Source: §7 · p. 13, §8 · p. 14

    Evidence is limited to a small set of untuned methods on one dataset and ecosystem, with point forecasts only.

    Source: §7.1 · p. 14

Abstract

from arXiv · show

Large forecasting applications often combine statistical, machine-learning, and neural models. These families solve the same problem but differ in fitted state, training procedures, and how they parallelize work. Forecasting software must therefore either hide these differences behind a single estimator interface, or keep the families in separate packages, forcing users to rewrite data preparation and evaluation for every package. We present the Nixtlaverse, an ecosystem of open-source Python libraries for time series forecasting, as a case study of a third design: all libraries share the same long-format panel data and keyed forecast outputs, while every model family keeps its own specialized implementation. We demonstrate this design through three use cases on the public M5 competition data. First, we evaluate statistical, machine-learning, and neural models, and an external engine from a separate ecosystem, in a single rolling-origin evaluation with per-series and hierarchy-weighted metrics. Second, we profile runtime and peak memory from 100 to 30,490 series and locate each family's bottleneck: statistical fitting scales approximately linearly in the number of series, feature construction dominates machine-learning memory, and neural training time is nearly independent of panel size under a fixed training budget. Third, we reconcile the forecasts of multiple engines, including the external one, over all 42,840 series of the M5 hierarchy, with sparse reconciliation where dense implementations exhausted memory. These use cases establish the costs, boundaries, and utility of shared data and output contracts. The Nixtlaverse has seen substantial public distribution, scholarly reuse, and adoption through other forecasting frameworks, and is released under permissive open-source licenses with public datasets, reproducible examples, and verifiable benchmark artifacts.

1. Introduction

Forecasting model families differ in state, training, and parallelism, complicating a common software interface. The Nixtlaverse shares panel and forecast-output contracts while keeping model-family implementations specialized, then examines that design through evaluation, scaling, and reconciliation use cases.

1. Introduction

  • The Nixtlaverse shares panel and forecast-output contracts while retaining engine-specific semantics and specialized implementations.Its three commitments also place scalability inside each model family and define downstream operations over keyed forecasts.Source: §1 · pp. 1–2
  • Its seven libraries provide specialized forecasting engines and shared tools for grouped operations, evaluation, and reconciliation.The ecosystem includes StatsForecast, MLForecast, NeuralForecast, CoreForecast, UtilsForecast, HierarchicalForecast, and DatasetsForecast.Source: §1 · p. 2
  • Three M5 use cases test the design through cross-family evaluation, resource profiling from 100 to 30,490 series, and reconciliation across 42,840 hierarchy series.The evaluation also includes an engine external to the ecosystem.Source: §1 · p. 2
  • The descriptive contribution inventories licenses and public infrastructure and examines release-coordination and reproducibility obligations in a multi-repository design.The contribution list also identifies compatibility testing and versioned research artifacts as part of the ecosystem-level analysis.Source: §1 · p. 2

2. Related Work

Related forecasting frameworks vary in model exposure and data representation, while statistical, machine-learning, and neural methods retain distinct requirements. This paper situates the Nixtlaverse among those designs and adds ecosystem-level analysis of shared contracts, scaling, and downstream interoperability.

2. Related Work

  • Prior frameworks range from indexed tsibble/model tables (Wang et al., 2020; O’Hara-Wild et al., 2026b,a) to unified interfaces such as Darts (Herzen et al., 2022), sktime (Löning et al., 2019), and AutoGluon–TimeSeries.The reviewed landscape also includes local automatic modeling, decomposable models, probabilistic and neural components, and pretrained forecasting workflows.Source: §2 · p. 2
  • Statistical, machine-learning, and neural methods differ in parameterization and data, computational, and tuning requirements; this paper measures where those differences surface in software.Its contribution is not to establish that the families differ, but to measure the resulting software boundaries and enable repetition through a released artifact.Source: §2 · p. 2
  • Forecasting frameworks vary independently in data representation and model exposure; the Nixtlaverse uses generic dataframes, separate engines, and keyed outputs for shared evaluation and reconciliation.Generic dataframes offer flexibility across dataframe systems but leave frequency and alignment checks to users.Source: §2 · p. 3
  • Prior reconciliation attaches to fitted-model tables in fabletools (O’Hara-Wild et al., 2026b), whereas FoReco uses positionally aligned matrices (Girolimetto and Di Fonzo, 2026); this paper uses explicit series keys.The paper tests whether one output representation supports common evaluation and reconciliation across model families.Source: §2 · p. 3
  • Beyond prior package descriptions, this paper adds ecosystem-level contract boundaries, phase profiles, full-hierarchy reconciliation, an external-engine test, and a verifiable benchmark artifact.The reconciliation analysis includes feasibility limits.Source: §2 · p. 3

3. The Nixtlaverse

The Nixtlaverse shares long-panel inputs and keyed forecast outputs across specialized forecasting libraries, while leaving model-family behavior and scaling strategies engine-specific. Its common contracts support reusable downstream workflows, but the multi-repository design requires manual coordination and its reported adoption indicators have limits.

3.1. Design requirements

  • The long-panel workflow preserves keys and temporal information without requiring model families to share fitted state or a scaling strategy.Local statistical, global machine-learning, and neural methods differ in state, feature construction, training loops, hardware preferences, and natural parallelism.Source: §3.1 · p. 3

3.2. Panel representation

  • A long panel uses series, timestamp, and target columns, with optional covariates, supporting unequal histories and partitioning across dataframe systems.Explicit keys allow forecasts to join observations, but do not themselves establish future covariate availability or prevent all alignment errors.Source: §3.2 · p. 3

3.3. The contract

  • The contract accepts keyed dataframes with required series, time, and target fields, while covariate semantics and additional validation remain engine-specific.Downstream requirements can extend beyond the output frame, such as in-sample fitted values for some reconcilers.Source: §3.3 · pp. 3–4

3.4. Nixtla’s forecasting libraries

  • StatsForecast, MLForecast, and NeuralForecast retain family-specific implementations while accepting long panels and returning keyed forecast dataframes.Their distinct approaches include compiled local statistical models, temporal features for regressors, and sampled windows for neural architectures.Source: §3.4 · p. 4

3.5. Shared components

  • UtilsForecast supplies evaluation and panel utilities, DatasetsForecast supplies benchmark loaders, HierarchicalForecast reconciles forecasts, and CoreForecast supplies grouped operations.The shared components operate on or consume the common panel and forecast-output contracts.Source: §3.5 · p. 4

3.6. Scaling strategies

  • Scalability is implemented per family: statistical work partitions by series, machine-learning work materializes panel features, and neural work batches temporal windows.The same differences shape distribution strategies, so preparation, fitting, prediction, evaluation, and reconciliation are measured separately.Source: §3.6 · p. 4

3.7. Open-source implementation and coordination

  • The multi-repository design preserves model-family independence but requires manual alignment of conventions, defaults, schemas, and dependencies across releases.Cross-library tests cover only exercised combinations; Table 1 records versions, licenses, public project assets, and the benchmark’s pinned dependency closure.Source: §3.7 · p. 4, Table 1 · p. 5

3.8. Observable reach and downstream adoption

  • The ecosystem shows public distribution, scholarly reuse, and downstream embedding, including through independently maintained forecasting frameworks.AutoGluon–TimeSeries relies on StatsForecast for local statistical models and MLForecast for tabular forecasters; download counts indicate distribution activity, not distinct users.Source: §3.8 · p. 6

4. Use case 1: five models from four engines

This use case tests shared panel and keyed-output contracts across three Nixtlaverse engines and an external engine, while leaving model construction and behavioral defaults engine-specific. The merged evaluation makes visible that per-series and hierarchy-weighted metrics can rank the same forecasts differently.

4. Use case 1: five models from four engines

  • The evaluation uses five fixed, untuned configurations on 30,490 M5 bottom-level series, with three 28-day forecast origins and per-series MASE and RMSE plus final-origin WRMSSE.The selected methods are Seasonal Naive and AutoETS, LightGBM, NHITS, and external Croston from sktime; this is a representative set, not a benchmark of every model.Source: §4 · p. 7

4.1. One panel, three engines

  • A long panel and outputs keyed by series, timestamp, and origin can be joined and scored together, while constructors retain family-specific requirements and estimators remain non-interchangeable.The engines also differ in how they specify forecast horizon.Source: §4.1 · p. 7
  • Origin spacing and refitting policy must be specified explicitly because default argument values are not synchronized across libraries.Shared keys make outputs combinable but do not synchronize behavioral defaults.Source: §4.1 · p. 7
  • External Croston forecasts pass through the same merge and evaluation call after a 121-line adapter emits the keyed frame.The test covers one method from one external engine, not the cost of mapping every external engine.Source: §4.1 · p. 9, §4.2 · p. 9

4.2. Benefits of output-level interoperability

  • NHITS has the best per-series MASE but worst hierarchical WRMSSE, while approximately unbiased AutoETS shows the opposite pattern.Across origins, NHITS forecasts 66–70% of observed demand and produces 11.7–16.7% negative forecasts, with bottom-level errors accumulating into downward bias at sales-weighted upper levels.Source: §4.2 · p. 9
  • WRMSSE requires a separate hierarchy-join and competition-evaluator workflow, and per-series ranking alone could favor forecasts totaling only 70% of observed demand.Output interoperability standardizes exchanged data but does not choose the metric or weighting scheme for model selection.Source: §4.2 · p. 9
  • Clipping negative forecasts changes WRMSSE only from 0.673 to 0.670 for AutoETS and 1.888 to 1.885 for NHITS, without changing rankings.Additional NHITS seeds keep mean MASE within 0.002 and WRMSSE between 1.732 and 1.888, still far from the next engine.Source: §4.2 · p. 9
  • The comparison exercises target lags and calendar features, not static- or future-covariate declarations, and its fixed configurations do not rank underlying methods.The external-engine test establishes admission through an adapter written by the authors, not that every external engine maps as cheaply.Source: §4.2 · p. 9

5. Use case 2: scaling from a pilot to the full panel

This use case profiles phase-level runtime and peak memory as representative M5 panels grow from 100 to 30,490 series. It finds different bottlenecks across statistical, machine-learning, and neural engines, with scaling behavior conditioned by parallelization, retained state, and the fixed neural training budget.

5. Use case 2: scaling from a pilot to the full panel

  • Phase-specific measurements inform resource provisioning because bottlenecks and natural parallelization units differ by model family.The use case considers which phase will dominate, memory needs, worker counts, and when a distributed backend may justify its overhead.Source: §5 · p. 9
  • Nested, compositionally sampled subsets span 100–30,490 series, with a 28-day holdout and five fresh-process repetitions measured by phase using medians.The subsets profile resource use; comparative accuracy is reported on the complete panel.Source: §5 · pp. 9, 11, Figure 3 · p. 10
  • Statistical fitting scales approximately linearly with series count, machine-learning estimator fitting dominates its phase, and fixed-budget neural training is weakly dependent on panel size.Statistical fitting rises from 3.0 seconds at 100 series to 691 seconds at full scale; neural training takes 8.8–16.2 seconds on one GPU under the fixed 1,000-step budget.Source: Figure 3 · p. 10, §5 · p. 11
  • 28 seconds makes the fixed-budget neural engine fastest at full scale versus 47 seconds for machine learning; peak memory is 15.2 GiB for the statistical portfolio, 7.0 GiB for machine learning, and 4.1 GiB for neural training.The statistical memory peak is 13.8 GiB for AutoETS alone; feature-table materialization drives machine-learning memory, while streamed window batches keep neural training lower.Source: Figure 3 · p. 10, §5 · p. 11
  • The single-worker fitting exponent is 0.99, while four and twenty workers yield exponents of 0.95 and 0.84, respectively.Speedups are 3.4 and 6.9 times at 100 series versus 3.75 and 9.8 times at 1,000; per-worker overhead amortizes over larger panels.Source: Table 4 · p. 11, §5 · p. 11
  • Adding SeasonalNaive to AutoETS costs 1.6–2.0 GiB of retained state at full scale without measurable time, because fitted state includes in-sample values, residuals, and stored series.The retained-state cost grows with observations rather than series count, despite SeasonalNaive having seven parameters per series.Source: §5 · p. 11
  • Ray takes 897 seconds and 19.2 GiB versus 738 seconds and 3.1 GiB for local multiprocessing; the streaming interface peaks at 3.1 GiB, versus 15.2 GiB for fit-then-predict.Ray's total includes 32 seconds for one-time data conversion; these backend and workflow contrasts were not crossed in a complete 2 × 2 design.Source: Table 5 · p. 12, §5 · p. 12
  • The study does not compare family-specific strategies with a family-agnostic scaling primitive, and absolute runtimes and GPU results remain hardware- and driver-dependent.The reported findings concern dominant phases under these configurations; an unfixed neural budget would recouple training time to panel size.Source: §5 · p. 12

6. Use case 3: coherent forecasts over the complete hierarchy

Across the complete M5 hierarchy, shared keyed forecast outputs let multiple engines use the same reconciliation implementation, but reconciliation guarantees coherence—not accuracy gains or computational feasibility. Sparse methods made some configurations feasible, while memory and scope constraints remained.

6. Use case 3: coherent forecasts over the complete hierarchy

  • Base forecasts violated aggregation constraints by up to 1,051 units for AutoETS, 18,071 for LightGBM, and 204 for the external engine; reconciled outputs were coherent by construction.Upper levels were produced by summing the reconciled bottom level.Source: §6 · p. 12, Table 6 · p. 13
  • Both resident engines’ keyed forecast dataframes entered the same reconciliation implementation, decoupling reconciliation from model fitting.The external engine also used the sparse BottomUp reconciler across all 42,840 series, with zero aggregation residual afterward and marginal accuracy changes.Source: §6 · pp. 12–13
  • Departmental MASE for LightGBM rose from 1.149 to 5.694 under BottomUp, while variance-weighted MinT improved every reported level.Reconciliation redistributes error rather than consistently reducing it; the result depends on both base forecasts and reconciliation method.Source: §6 · pp. 12–13, Table 6 · p. 13
  • Hierarchy construction took 258 seconds and peaked at 28.5 GiB; a dense summing matrix required 9.7 GiB, and attempted dense BottomUp and MinT-shrink runs were terminated.These measurements concern the 42,840-series M5 hierarchy.Source: §6 · p. 13, Table 6 · p. 13
  • Sparse MinT still peaked at 43.6 GiB and was terminated with both engines’ intermediates resident, but completed in 179 and 197 seconds when run one engine at a time.Aggregated data and the 66-million-row fitted frame remained resident because the output contract passed the complete frame to the variance estimator.Source: §6 · p. 13
  • Sparse MinT is not guaranteed to match the dense estimator, whose memory exhaustion prevented bounding the difference; variance-weighted reconciliation was not demonstrated for external producers lacking keyed in-sample fitted values.The observed feasibility boundary is a single threshold crossing, not a general comparison of workflow or algorithm choices.Source: §6 · pp. 12–13

7. Discussion

The discussion argues for sharing keyed data contracts while keeping model-family behavior and scalability specialized. It also narrows the evidence to particular methods, data, software, and point-forecast workflows, and identifies settings where the architecture may be less useful.

7. Discussion

  • Shared series, time, and origin keys support common evaluation across model families, but do not standardize feature availability, losses, refitting defaults, horizon placement, or probabilistic semantics.The supported interoperability claim is about exchange, not behavioral semantics.Source: §7 · p. 13
  • Scaling behavior belongs within model families, but capacity planning must account for both fitted-state residency and backend overhead.Local memory rose from 3.1 to 15.2 GiB with fit-then-predict, while Ray object-store and worker copies raised streaming memory from 3.1 to 19.2 GiB; these contrasts were not measured on a common design.Source: §7 · p. 13
  • Keyed outputs let downstream evaluation and reconciliation use multiple engines, including an external engine routed through the workflows with a 121-line adapter.The tested contract does not ensure coherent base forecasts, accuracy gains from reconciliation, or dense-method tractability.Source: §7 · p. 14
  • The commitments are most useful when several engines evolve independently, multiple model families operate in production, and downstream workflows must support them.A dedicated time-series object, a unified estimator for automated search, or a single-family production stack may make the shared-contract design less advantageous.Source: §7 · p. 14

7.1. Limitations

  • Evidence from a small set of untuned methods, one dataset, one ecosystem, and fixed software versions neither ranks model families nor establishes general efficiency.Results depend on dataset size, seasonal patterns, covariates, hardware, and model settings; M5’s daily, intermittent, hierarchical patterns may not transfer to smoother or shallower panels.Source: §7.1 · p. 14
  • The study cannot compare its engines with the unified-estimator alternative, and whether keyed contracts are minimal or sufficient for every downstream operation remains untested.Some operations may require fitted models rather than their outputs.Source: §7.1 · p. 14
  • Evaluation covers point forecasts only; the infrastructure inventory does not measure governance quality, contributor diversity, maintenance responsiveness, or long-term sustainability.The authors also state that this is not an independent evaluation of the software under study.Source: §7.1 · p. 14

8. Conclusion and Future Work

The conclusion presents the Nixtlaverse as a case study in shared data and output contracts with specialized model-family implementations, and identifies both measured operational boundaries and future work.

8. Conclusion and Future Work

  • The Nixtlaverse's design commitments are to share panel and output contracts, implement scalability within each model family, and define downstream operations over keyed forecasts.Its multi-repository implementation preserves specialized dependencies and interfaces, while making compatibility testing and versioned research artifacts part of the coordination burden.Source: §8 · p. 14
  • Explicit input, origin, and output keys supported evaluation of all three model families and one external engine on the complete M5 panel without per-engine reshaping.The constructors and behavioral semantics remained distinct; whether these boundaries generalize awaits replication in another ecosystem or a domain with different intermittency and hierarchy depth.Source: §8 · p. 14
  • For the per-series engine, peak memory was 3.1 GiB with local streaming, 15.2 GiB with local fit-then-predict, and 19.2 GiB with Ray streaming.The measurements identify fitted-state residency and distributed-execution overhead as separate budgeting mechanisms; a dense double-precision M5 summing matrix requires 9.7 GiB before reconciliation.Source: §8 · p. 14
  • Per-series parallelization does not transfer to global models, so one parallelization control describes an interface rather than a shared scaling behavior.The conclusion states this finding for the measured workload.Source: §8 · p. 15
  • The proposed next step is a single workspace with a shared dependency lockfile, allowing cross-package development and testing while keeping packages separately installable.The proposal aims to reduce manual coordination overhead while preserving modularity.Source: §8 · p. 15

CRediT author statement

The CRediT statement assigns Olivier Sprangers a broad set of project roles and lists software and review contributions for the other named authors.

CRediT author statement

  • Olivier Sprangers is credited with conceptualization, data curation, formal analysis, investigation, methodology, project administration, software, validation, visualization, and writing.His writing roles include original drafting and review and editing.Source: CRediT author statement · p. 15
  • Max Mergenthaler Canseco, Marco Peixeiro, Saul Caballero Ramirez, Mariana Menchero García, Jing-Qiang (JQ) Goh, Han Wang, Nikhil Gupta, Rogelio Melo, Senbong Gee, and Cristian Challu are credited with software and writing–review and editing.Source: CRediT author statement · p. 15

Funding

The authors report that the research received no specific grant from funding agencies.

Funding

  • The research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.Source: Funding · p. 15

Declaration of competing interest

The authors disclose their employment at Nixtla, the company developing the software examined in the article.

Declaration of competing interest

Data and code availability

The artifact provides code and materials for reproducing the reported results, but does not redistribute the M5 source files.

Data and code availability

Appendix A.3. Evaluation protocol

The appendix documents the M5 data, model settings, evaluation and resource-measurement protocols, hierarchical workflow, external-engine tests, and reproducibility materials. The experiments use fixed configurations and hardware-dependent measurements.

Appendix A. Experimental protocol

  • The artifact contains benchmark code and specifications for reproducing the use cases described in Sections 4–6.Source: Appendix A · p. 18

Appendix A.1. Datasets

  • The M5 bottom-level panel has 30,490 series and 47,649,940 observations; its twelve-level hierarchy has 42,840 series and 67,687,697 observations.Both use the 28-day competition horizon; the bottom-level panel supports evaluation and scaling, while the hierarchy supports reconciliation.Source: Appendix A.1 · p. 18

Appendix A.2. Method configurations

  • Configurations use documented defaults or prespecified settings rather than method-specific hyperparameter optimization.All engines use a 28-day horizon, daily frequency, and fixed seed per repetition; NHITS has a 1,000-step training budget and context length 56.Source: Appendix A.2 · p. 18
  • Evaluation uses three non-overlapping 28-day origins, re-estimating each model at every origin and reporting per-series-and-origin mean MASE and RMSE.WRMSSE is reported only at the final origin because competition weights are defined for a specific evaluation window; intervals use 2,000 whole-series bootstrap resamples and condition on one neural training realization.Source: Appendix A.2 · p. 18

Appendix A.4. Resource-measurement protocol

  • Resource profiling measures preparation, fitting, prediction, and evaluation time and peak memory across seeded nested panel subsets.Panel-size configurations have five repetitions and worker-count configurations three; reconciliation and distributed-backend configurations have one measured run, and parallel runs may include thread oversubscription.Source: Appendix A.4 · pp. 18–19

Appendix A.5. Hierarchical workflow

  • Hierarchical reconciliation compares AutoETS and LightGBM base forecasts with fixed BottomUp and MinTraceSparse configurations on a single final-origin 28-day holdout.MinTraceSparse uses a diagonal residual-based forecast-error variance approximation without a non-negativity constraint; results include level-wise MASE, runtime, and maximum aggregation residual.Source: Appendix A.5 · p. 19

Appendix A.6. External-engine setup

  • The external-engine tests use Croston from sktime in a separate environment, exchanging only the long panel and keyed forecast dataframe across the boundary.Its forecasts enter the common evaluation and sparse bottom-up reconciliation workflows; variance-weighted reconciliation is omitted because it requires producer-side fitted values.Source: Appendix A.6 · p. 19

Appendix A.7. Reproducibility

  • The artifact records version-pinned dependencies, data hashes, configurations, raw outcomes, and result-to-command evidence; reported runtime and GPU outcomes depend on hardware and drivers.Source: Appendix A.7 · p. 19
Loading 2609.39741v1…