Kolmogorov-Arnold Networks (KANs) for Time Series Analysis

Cristian J. Vaca-Rubio; Luis Blanco; Roberto Pereira; Màrius Caus

Source-linked AI summary

Kolmogorov-Arnold Networks (KANs) for Time Series Analysis

Cristian J. Vaca-Rubio, Luis Blanco, Roberto Pereira, Màrius Caus

arXiv:2405.08790v2May 14, 2024eess.SPcs.AIcs.LG

TL;DR

Time series forecasting requires accurate prediction from historical observations, and the paper investigates whether KANs can provide a practical alternative to conventional MLPs. It applies spline-based KANs to real-world satellite traffic forecasting and reports better forecasting performance with fewer parameters, while examining KAN-specific parameter effects. The study also notes that continual learning was not assessed and that broader comparisons with advanced architectures remain needed.

Problem
Time series forecasting must predict future values from historical data, motivating evaluation of adaptive neural architectures for real-world forecasting.
Method
The paper applies KANs with trainable spline functions to supervised satellite traffic forecasting and analyzes node counts and grid sizes.
Results
KANs outperformed MLPs on satellite traffic forecasting, achieving lower error metrics with fewer parameters; the four-depth KAN used 109k parameters versus 329k for the four-depth MLP.
Takeaways & Limitations
KANs appear to be a reasonable alternative to traditional MLPs for traffic management because they combine forecasting effectiveness with parameter efficiency.
Takeaways & Limitations
Continual learning was not assessed, and broader comparisons with established LSTM, GRU, and CNN architectures require further investigation.

Abstract

from arXiv · show

This paper introduces a novel application of Kolmogorov-Arnold Networks (KANs) to time series forecasting, leveraging their adaptive activation functions for enhanced predictive modeling. Inspired by the Kolmogorov-Arnold representation theorem, KANs replace traditional linear weights with spline-parametrized univariate functions, allowing them to learn activation patterns dynamically. We demonstrate that KANs outperforms conventional Multi-Layer Perceptrons (MLPs) in a real-world satellite traffic forecasting task, providing more accurate results with considerably fewer number of learnable parameters. We also provide an ablation study of KAN-specific parameters impact on performance. The proposed approach opens new avenues for adaptive forecasting models, emphasizing the potential of KANs as a powerful tool in predictive analytics.

I. INTRODUCTION

Time series forecasting supports decisions across many domains, but traditional statistical and neural approaches have limitations that motivate evaluating KANs as a potential alternative to MLPs in satellite traffic forecasting.

Motivation: Time series forecasting predicts future values from observed data and supports decision-making in finance, economics, medicine, meteorology, and biology.Contemporary ML methods seek to learn temporal patterns from data rather than relying only on domain-specific parametric models.
Prior approaches: Traditional forecasting methods include statistical approaches such as ARIMA, which combines autoregression, integration, and moving averages.SARIMA extends ARIMA to handle seasonality, while exponential smoothing addresses trends and seasonality.
Prior approaches: ML and DL models broaden forecasting capabilities by capturing complex nonlinear relationships in large datasets.MLPs, LSTMs, GRUs, and CNNs have all been applied to forecasting, with recurrent models designed to preserve information over long periods.
Motivation: MLPs can exhibit poor parameter scaling with depth and often lack interpretability.These limitations provide context for exploring alternative neural architectures.
Paper contribution: KANs are evaluated as a prospective alternative to MLPs for real-world satellite traffic forecasting, emphasizing parameter efficiency and forecasting performance.The paper notes that broader comparisons with established LSTM, GRU, and CNN architectures require further investigation.

II. PROBLEM STATEMENT

The paper formulates traffic forecasting as predicting future time-series values from a historical context window and evaluates point forecasts over the prediction range.

Problem formulation: Traffic forecasting represents the series at time t by y_t and predicts its future values from historical observations.The formulation targets forecasts over a specified future interval.
Problem formulation: The historical range [t_0 − c, t_0 − 1] is the context length, while [t_0, T] is the prediction length.The forecast begins at t_0 and continues through T.
Forecast objective: The approach generates point forecasts for each time step in the prediction length.The stated objective is accurate and reliable forecasting across that interval.

A. Kolmogorov-Arnold representation background

KANs are grounded in the Kolmogorov-Arnold representation theorem, which expresses multivariate continuous functions through compositions of univariate functions.

Theoretical basis: KANs rely on the Kolmogorov-Arnold representation theorem rather than the universal approximation theorem underlying MLPs.The theorem is also called the Kolmogorov-Arnold superposition theorem.
Theoretical basis: Any multivariate continuous function on a bounded domain can be represented as a finite composition of simpler one-variable continuous functions.The representation uses inner and outer univariate functions.
Implications: The theorem’s practical appeal is that learning a high-dimensional function can be reduced to learning a polynomial number of one-dimensional functions.The paper notes that these functions may be nonsmooth and therefore difficult to learn in practice.

B. Kolmogorov-Arnold network background

KANs replace conventional node activations and linear weights with trainable univariate spline functions arranged across network layers. Their differentiable compositional structure supports backpropagation while exposing both architectural and spline-resolution degrees of freedom.

Spline parameterization: B-splines approximate the one-dimensional inner functions, with smoothness controlled by spline order k and grid intervals G.The paper notes k = 3 as a common order and describes G as the number of spline segments.
Network architecture: KANs place activation functions on edges and perform simple summation at nodes, extending a basic two-layer construction into wider and deeper architectures.The deeper architecture addresses the limited approximation capacity of a simplistic two-layer network.
KAN layer structure: KAN layers use trainable univariate functions ϕi,j arranged as a matrix mapping Nin inputs to Nout outputs.The functions are spline-parametrized and constitute the network’s learnable transformations.
Training: All KAN operations are differentiable, so the networks can be trained with backpropagation.The architecture combines spline-based univariate approximation with compositional network structure.
Complexity: Increasing either the number of layers L or grid dimension G increases the number of parameters and network complexity.These variables provide external architectural and internal spline-related degrees of freedom.

C. KAN time series forecasting network

The forecasting model learns a mapping from a historical context window to the future prediction window using a two-layer KAN. Its inner and output function layers transform the context into the requested future time steps.

Problem formulation: Traffic forecasting is formulated as supervised learning that approximates yt0:T from historical inputs xt0−c:t0−1.The input-output training pairs cover the context and prediction lengths.
Architecture: The proposed two-layer KAN uses Ni input nodes, n intermediate nodes, and No output nodes corresponding to the time-series windows.The input layer is not counted as a KAN layer in the paper’s notation.
Forecast generation: The output functions generate No predicted values for the future traffic sequence using only the context-length observations.The network forecasts every time step in the prediction length.

III. EXPERIMENTAL SETUP

The experiments evaluate KAN and MLP architectures on normalized, real satellite traffic data across six beams. Models use a one-week context to predict one day, with comparable-depth architectures and held-out test series.

Dataset: The dataset contains normalized, aggregated traffic from a GEO satellite communication system, measured hourly in Mbps over approximately one month.Anonymous clients have more than 500 connected users, and the data are collected per satellite beam.
Model comparison: The study compares KAN and MLP architectures for satellite traffic forecasting across six beam areas.The comparison targets forecasting efficacy and parameter efficiency using models with similar depths but different architectures.
Forecast horizon: Each experiment uses 168 hours of context to predict 24 hours, corresponding to one week of history and one day of forecasts.The setup defines c = 168 and T = 24.
Experimental scope: The KAN–MLP comparison is presented as a fair baseline because KANs are still in their infancy.The setup deliberately does not compare against more complex architectures such as LSTMs.
Training protocol: Training uses two weeks plus one day of data, while testing uses one week plus one day of previously unseen series.All networks are trained for 500 epochs with Adam at learning rate 0.001, minimizing MAE over the prediction length.

A. Performance analysis

Across three satellite beams, 4-depth KAN forecasts generally track real traffic more closely than 4-depth MLP forecasts, especially during rapid or highly variable changes. Quantitative comparisons identify the 4-depth KAN as the best-performing and more parameter-efficient architecture.

4-depth KAN forecasts better approximate real satellite traffic than 4-depth MLP forecasts across the evaluated beams.The comparison uses test-set prediction lengths for different beams.
KAN accurately tracks rapid traffic changes in Figure 3a, whereas MLP forecasts sometimes moderately over- or under-predict the final forecast segment.This difference is most evident during sudden changes in traffic volume.
KAN adjusts more quickly during fast-changing conditions in Figure 3b, while MLP exhibits a lag during the last 6 hours of the prediction length.Figure 3c further reports robust KAN performance under more variable, intense, and higher-volume traffic conditions.
The 4-depth KAN is the best-performing architecture across MSE, RMSE, MAE, and MAPE comparisons.Lower error values indicate more accurate predictions and greater consistency across traffic volumes.
109k parameters make KAN (4-depth) substantially smaller than MLP (4-depth) at 329k and MLP (3-depth) at 238k.The reported results also state that adding 16k KAN parameters significantly improves performance, unlike adding 91k MLP parameters.
KANs achieve lower error rates with considerably fewer parameters than traditional MLP networks.The authors relate this combination of accuracy and compactness to modeling complex nonlinear traffic patterns.

B. KANs parameter-specific analysis

The ablation study evaluates how KAN node counts and spline grid sizes affect training loss in satellite traffic forecasting. Higher node counts generally improve performance, while larger grids help mainly when paired with more nodes, with n = 20 and G = 20 performing best but potentially requiring more computation.

Experimental configurations: The study evaluates 2-depth KANs [168, n, 24] with n ∈{5, 10, 20} and spline grids G ∈{5, 10, 20} using k = 3 B-splines.The configurations were analyzed during training time.
Node-count effects: Increasing the number of nodes generally produces lower loss values, with n = 20 outperforming smaller node counts across all grid sizes.The authors associate higher node counts with better capture of complex traffic patterns.
Grid-size effects: Larger grid sizes improve performance when node counts are n ∈{10, 20}, but show the opposite effect when n = 5.The authors suggest larger grids provide a more detailed spline basis when node counts are sufficiently high.
Best configuration: The n = 20, G = 20 configuration achieves the best performance among the tested settings.The authors attribute this to greater flexibility and learning capacity for modeling intricate traffic dependencies.
Practical trade-offs: Higher node counts and larger grids can improve KAN performance but may increase computational demands and training time.The paper recommends balancing parameter settings against practical computational constraints and notes that continual learning was not assessed.

V. CONCLUSION

The paper analyzes KANs and MLPs for satellite traffic forecasting, finding that KANs provide better forecasting performance and parameter efficiency. It also examines how KAN node counts and grid sizes affect performance, while positioning KANs as an alternative for traffic management.

Conclusion: KANs consistently outperform MLPs on satellite traffic forecasting through lower error metrics and better results with lower computational resources.The conclusion reports both forecasting and resource-efficiency advantages.
Conclusion: Optimizing KAN node counts and grid sizes affects model performance in traffic forecasting.The paper specifically explores the impact of these KAN parameters.
Conclusion: The findings present KANs as a reasonable alternative to traditional MLPs in traffic management.This conclusion follows the reported accuracy and efficiency results.