PaperlayerView exact revision on arXiv

Source-linked AI summary

FITS: Modeling Time Series with $10k$ Parameters

Zhijian Xu, Ailing Zeng, Qiang Xu

arXiv:2307.03756v3cs.LG

TL;DR

Resource-constrained edge devices motivate models that reduce the computational and memory costs of time-series analysis. FITS reframes forecasting and reconstruction as complex-frequency interpolation with a complex-valued linear layer and low-pass filtering, achieving comparable performance to state-of-the-art models with about 10k parameters. Its compactness supports deployment or training on edge devices, although binary event datasets can favor time-domain anomaly-detection models.

  • Problem

    Recent time-series models often have computational and memory costs that make them unsuitable for resource-constrained edge devices, while the compactness of frequency-domain representations remains underused.

  • Method

    FITS performs forecasting and reconstruction through interpolation in the complex frequency domain using a complex-valued linear layer, rFFT, and low-pass filtering.

  • Results

    FITS achieves performance comparable to state-of-the-art models while using approximately 10k parameters.

  • Takeaways & Limitations

    FITS is positioned as a low-cost model suitable for deployment or training on resource-constrained edge devices.

  • Takeaways & Limitations

    FITS performs comparatively worse on SMAP and MSL, where binary event data may not be effectively captured by its frequency-domain representation.

Abstract

from arXiv · show

In this paper, we introduce FITS, a lightweight yet powerful model for time series analysis. Unlike existing models that directly process raw time-domain data, FITS operates on the principle that time series can be manipulated through interpolation in the complex frequency domain. By discarding high-frequency components with negligible impact on time series data, FITS achieves performance comparable to state-of-the-art models for time series forecasting and anomaly detection tasks, while having a remarkably compact size of only approximately $10k$ parameters. Such a lightweight model can be easily trained and deployed in edge devices, creating opportunities for various applications. The code is available in: \url{https://github.com/VEWOXIC/FITS}

1 INTRODUCTION

FITS addresses the mismatch between resource-intensive time-series models and constrained edge devices by interpolating in the complex frequency domain. Its compact architecture and low-pass filtering support comparable performance with roughly 10k parameters.

  • Resource-constrained edge devices make the computational and memory costs of recent time-series models problematic.
  • Existing frequency-domain methods use spectral information but do not fully exploit complex numbers to capture both amplitude and phase.
  • FITS reframes forecasting and reconstruction as interpolation exercises in the complex frequency domain.For forecasting, it extends a look-back window; for reconstruction, it interpolates the frequency representation of a downsampled segment.
  • A complex-valued linear layer learns amplitude scaling and phase shift for frequency-domain interpolation.
  • rFFT maps inputs into the complex frequency domain, after which interpolated frequencies are transformed back into elongated time-domain segments for downstream tasks.The design supports time-domain tasks including forecasting and anomaly detection.
  • Fewer than 10k parameters make FITS 50 times more compact than DLinear and approximately 10,000 times smaller than other mainstream models in most scenarios.A low-pass filter provides a compact representation while preserving essential information.

2 RELATED WORK AND MOTIVATION

Prior frequency-aware approaches improve temporal modeling but remain dependent on engineered dominant-period selection and can be inefficient. FITS instead motivates compact complex-frequency manipulation by exploiting the signal structure of sinusoidal components.

  • Frequency-aware models: FNet, FEDFormer, FiLM, and TimesNet incorporate frequency information to capture temporal dependencies, periodicity, or long-term patterns.
  • Frequency-aware models: TimesNet identifies high-energy frequencies with FFT and reshapes the one-dimensional series into two-dimensional images according to their periods.
  • Limitations of prior work: These approaches rely on feature engineering to identify dominant periods, while energy-based selection may retain mainly a dominant period and its harmonics.The passage also characterizes these methodologies as inefficient and prone to overfitting.
  • Motivation for FITS: A time series can be decomposed without information loss into sinusoidal components defined by frequency, initial phase, and amplitude.Forecasting each component requires applying a phase bias for the time shift before recombining the shifted waves.
  • Motivation for FITS: Frequency-domain manipulation offers a more compact, information-rich alternative to cumbersome time-domain forecasting of individual sinusoidal components.The resulting forecast preserves the look-back window’s frequency characteristics while maintaining semantic consistency with the forecast horizon.

3 METHOD

FITS represents time series in the complex frequency domain, where frequency components encode amplitude and phase, and uses interpolation to extend or reconstruct signals. Its pipeline combines rFFT, low-pass filtering, complex-valued linear interpolation, zero-padding, irFFT, and reversible normalization, with cutoff selection based on harmonic content and weight sharing for multivariate data.

  • Preliminary: FFT and Complex Frequency Domain: Complex frequency representations encode each frequency component with amplitude and phase, while time shifts appear as corresponding phase shifts.A frequency component is represented as a complex number, visualized by amplitude as vector length and phase as vector angle; shifting time preserves amplitude and changes phase linearly.
  • FITS Pipeline: FITS projects normalized time-series segments with rFFT, interpolates their complex frequency representation using one complex-valued linear layer, and maps the result back with irFFT.The forecasting pipeline then reverses normalization, while reconstruction uses the same pipeline with reconstruction supervision.
  • FITS Pipeline: For forecasting, FITS generates the look-back window and horizon and can use both forecast and backcast supervision; reconstruction instead upsamples a downsampled segment.Combining backcast and forecast supervision improves performance in certain scenarios according to the ablation study.
  • Key Mechanisms of FITS: The interpolation rate η equals the ratio of output length L_o to input length L_i and projects frequency band 1∼f to 1∼ηf.The interpolated representation is zero-padded to the desired output length before irFFT; with LPF, the layer input length corresponds to the cutoff frequency.
  • Key Mechanisms of FITS: The low-pass filter removes components above a cutoff frequency to compress the representation while retaining essential information.FITS selects the cutoff using harmonics of the dominant frequency, aiming to preserve signal structure and periodicity while suppressing noise and irrelevant high-frequency content.
  • Key Mechanisms of FITS: FITS shares weights across multivariate channels, relying on common base frequencies for channels from the same physical system.Channels with different base frequencies can instead be clustered according to the paper’s stated approach.

4 EXPERIMENTS FOR FORECASTING

FITS is evaluated on long- and short-term forecasting using public datasets and standard baselines, with experiments emphasizing its competitive accuracy and efficiency. Longer look-back windows generally help, while low cutoff frequencies preserve efficiency with near-state-of-the-art performance.

  • Forecasting performance: FITS consistently achieves comparable or superior performance across the reported long-term forecasting experiments.The evaluation uses long-term forecasting datasets including Traffic, Electricity, Weather, and ETT.
  • Efficiency: 4.5K to 16K parameters allow FITS to match or exceed larger forecasting models, while DLinear has 139.7K parameters.The comparison includes models ranging from thousands to hundreds of millions of parameters.
  • Hyperparameter effects: Longer look-back windows generally improve performance, whereas increasing the cutoff frequency produces only marginal gains.The cutoff-frequency increase also raises computational-resource requirements.
  • Efficiency: 720-step look-back windows with the second-harmonic cutoff achieve state-of-the-art performance at around 10k parameters.Reducing the look-back window to 360 yields close-to-state-of-the-art performance with fewer than 5k parameters under the same cutoff choice.
  • Hyperparameter effects: A low cutoff frequency combined with a longer look-back window provides near-state-of-the-art performance at minimal computational cost.This configuration is presented as a practical balance between accuracy and resource use.

FOR ANOMALY DETECTION

FITS applies self-supervised frequency interpolation to reconstruct downsampled time-series segments for anomaly detection. It achieves strong detection results with very small computational requirements, although performance is weaker on binary-event datasets.

  • Method: FITS downsamples each input segment and trains frequency interpolation to reconstruct the original segment.Anomalies are identified when reconstruction loss exceeds a threshold selected using validation-set F1 score.
  • Evaluation: FITS is evaluated on five benchmark datasets using Precision, Recall, and F1-score against established anomaly-detection models.The datasets are SMD, PSM, SWaT, MSL, and SMAP.
  • Results: 99.95% F1-score on SMD and 98.9% on SWaT demonstrate FITS’s strongest reported anomaly-detection results.On these datasets, FITS is described as achieving nearly perfect performance and outperforming several comparison models.
  • Limitation: FITS performs comparatively worse on SMAP and MSL, where binary-event data may be poorly captured by its frequency-domain representation.The passage states that time-domain modeling is preferable for these datasets and that THOC and Omni Anomaly achieve higher F1-scores.
  • Efficiency: 1-4K parameters and 10-137K MACs underlie the reported anomaly-detection results.These measurements emphasize FITS’s low computational and memory requirements.
  • Results: 100% F1 score on the synthetic dataset shows that FITS detects anomalies introducing unexpected frequency components.The dataset combines a single-frequency sinusoid with injected anomaly patterns that are difficult to identify in the time domain.
  • Practical implication: Sub-millisecond inference enables FITS to act as a rapid first-response filter before a specialized anomaly-detection algorithm.The paper describes this pairing as combining rapid responsiveness with more detailed detection.

6 CONCLUSIONS AND FUTURE WORK

The paper concludes that FITS offers a low-cost alternative for time-series analysis, combining approximately 10k parameters with performance comparable to much larger state-of-the-art models. Future work targets broader real-world evaluation, improved interpretability, and larger complex-valued frequency-domain networks.

  • Conclusion: FITS uses 10k parameters while achieving performance comparable to state-of-the-art models that are often several orders of magnitude larger.The authors characterize FITS as a low-cost model for time-series analysis.
  • Future work: Future work will evaluate FITS in more real-world scenarios and improve its interpretability.The authors also plan to explore large-scale complex-valued neural networks such as complex-valued Transformers.

A PIPELINE FOR RECONSTRUCTION

The reconstruction pipeline downsamples a time-series segment, interpolates its frequency representation, and reconstructs a full-length output for comparison with the original signal.

  • Input preparation: The model input x is obtained from a time-series segment y through equidistant sampling at downsample rate η.This downsampled segment is the input to FITS’s reconstruction process.
  • Frequency interpolation: FITS performs frequency interpolation to generate an upsampled output with the same length as the original segment y.The pipeline maps the sampled input back to the original temporal length.
  • Training objective: Reconstruction loss compares the original segment y with the upsampled output.This comparison supplies the reconstruction objective for the pipeline.

B DETAILS OF FORECASTING DATASETS

The appendix reports additional forecasting evaluations, parameter studies, and dataset-specific behavior for FITS. Results are generally favorable with compact models, but performance depends on dataset diversity, look-back windows, distribution shifts, and frequency content.

  • Appendix scope: The forecasting appendix covers dataset statistics, parameter counts, and results across ETTh, ETTm, Traffic, Weather, Electricity, and M4 settings.It includes tables for dataset statistics and long-term forecasting results, with additional retrieved comparisons for transformer-based models.
  • Additional comparisons: FITS outperforms N-HiTS and N-BEATS in most reported MSE cases while maintaining a compact model size.These comparison results are retrieved from the N-HiTS paper and were considered for inclusion in the main results.
  • M4 forecasting: FITS shows suboptimal SMAPE results on M4, where heterogeneous series, short look-back windows, and larger competing models create challenging conditions.The M4 series span different domains and periodicities, while the look-back windows are only twice the forecasting horizon.
  • Look-back sensitivity: Distribution shift on ETTh1 and ETTm1 can make longer look-back windows harmful by introducing information from a shifted distribution.On ETTh1, FITS achieves state-of-the-art performance at a look-back window of 360 rather than 720.
  • ETT datasets: FITS consistently achieves state-of-the-art performance on ETTm2 with under 10k parameters.The appendix reports parameter counts and results under different settings for ETTm1 and ETTm2.
  • Traffic: 50k parameters yield comparable performance on Traffic despite its large number of channels.The passage contrasts FITS’s parameter requirement with the larger parameter needs of many models on this dataset.
  • Electricity: Electricity is sensitive to the cutoff frequency because its significant multi-periodicity requires retaining high-frequency components.If those components are removed, FITS does not learn the corresponding information.

F ANOMALY DETECTION RESULTS ON SYNTHETIC DATASET

The synthetic anomaly-detection dataset uses 4,000 time steps, with 2,500 for training and 1,500 for testing, and evaluates FITS across four reconstruction windows.

  • 4,000 time steps are generated, with 2,500 used for training and 1,500 for testing.
  • 5% outliers are injected into each channel using different outlier types.
  • FITS is evaluated with four reconstruction windows labeled FITS-winxxx.
  • The results are compared with values retrieved from Table 17 of Lai et al. (2021).

G DATASETS VISUALIZATION ON ANOMALY DETECTION

The anomaly-detection datasets contain varied signal types: PSM and SMD are largely analog, SWaT includes binary event channels, and SMAP and MSL are mostly binary and difficult for FITS to learn in the frequency domain. The reported evaluation also includes cross-dataset results, model-complexity measurements, and a critical-difference analysis of forecasting performance.

  • Dataset visualizations: PSM and SMD mostly contain analog-valued channels, with PSM exhibiting strong periodicity.
  • Dataset visualizations: SWaT includes binary event-valued channels.
  • Evaluation results: The evaluation materials include results on five datasets and a synthetic dataset, alongside Electricity-dataset results.
  • Dataset visualizations: SMAP and MSL are mostly binary event-valued datasets that are hard for FITS to learn in the frequency representation.
  • Model complexity: FITS is evaluated with fixed sliding windows of 200 and 400, no frequency filter, and a downsample rate of 4.
  • Forecasting comparison: FITS ranks at the top of the critical-difference plot without intersecting other lines, indicating statistically significant MSE differences under alpha=0.05.
Loading 2307.03756v3…