fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions

1 Introduction

• ReLU returns 0 for negative inputs, a limitation associated with dead neurons, motivating the study of alternative activation functions.Adaptive activations can learn their parameters during training and may mitigate limitations of fixed functions.Source: §1 · pp. 1–2
• Orthogonal-polynomial activations have been used in neural networks, while fractional orthogonal functions extend polynomial spaces to fractional functions and have also been applied in neural networks.Prior fractional approaches include B-spline, wavelet, Bernoulli, Jacobi, Legendre, and Lagrange functions.Source: §1 · pp. 2–4
• The paper introduces a fractional Jacobi neural block for KANs, with Jacobi activation parameters tuned during training and fractional order explored to improve accuracy.The block is intended for integration into deep or KAN architectures.Source: §1 · p. 4
• The evaluation spans synthetic regression, MNIST classification, Fashion-MNIST denoising, IMDB sentiment analysis, and Lane-Emden, Burgers, and fractional-delay differential equations.The paper also describes the organization of its background, methodology, experiments, and conclusion.Source: §1 · p. 4

2.1 Multi-Layer Perceptron

• MLPs combine weighted basis-function approximations with nonlinear activations; differentiability supports backpropagation, while nonlinearity enables complex pattern learning.The text also identifies boundedness and computational efficiency as desirable activation-function properties.Source: §2.1 · pp. 4–5

2.2 Kolmogorov-Arnold Networks

• KANs represent multivariate functions through compositions of univariate functions and addition, but variants still face inner-function non-smoothness and time complexity.Chebyshev KANs provide an orthogonal-polynomial alternative but retain polynomial limitations.Source: §2.2 · pp. 5–6

2.3 Jacobi polynomials

• Jacobi polynomials sit within the hypergeometric-function hierarchy, and choosing their parameters yields other familiar orthogonal polynomials.Their derivatives can be expressed using Jacobi polynomials themselves; shifting the domain and extending to fractional order yields shifted fractional Jacobi functions.Source: §2.3 · pp. 7–8
• Jacobi functions are presented as activation candidates because they are nonlinear and differentiable, have readily computed derivatives, and confine outputs to a predetermined range.The text links these properties to simpler backpropagation calculations and network stability.Source: §2.3 · p. 8

• The fractional Jacobi neural block (fJNB) uses trainable α, β, and γ, allowing the network to tune Jacobi parameters during training.Source: §3 · pp. 8–9
• Because Jacobi inputs must lie in a bounded interval, the design uses a bounded activation rather than batch normalization, selecting sigmoid’s (0,1) range to support fractional powers.Source: §3 · p. 8
• fKAN extends KAN with fractional univariate functions, choosing fractional Jacobi functions for their flexibility, ease of calculation, and adaptability.Source: §3 · p. 9
• ELU constrains α and β to values greater than −1, while sigmoid keeps the fractional order γ between zero and one to ensure it is positive and avoid potential instability from unbounded alternatives.Source: §3 · p. 9
• Because Jacobi polynomials are nonzero except at roots, fKAN omits the outer summation used for local basis functions and uses a linear Φ with trainable weights.Source: §3 · p. 9
• The broader formulation sets Q as the maximum degree, makes the outer summation input-independent, and lets Φ fuse fJNB outputs through concatenation or attention.Source: §3 · p. 9

4.1 Deep learning tasks

• In synthetic regression, fKAN is reported more accurate than common activation functions and KAN in both single-layer and multilayer networks.The multilayer comparison reports higher variance than other activations, while KAN has the lowest variance and increased time complexity as layers increase.Source: Regression task · pp. 10–11
• The trainable Jacobi parameters converge to optimal values in about 250 iterations, while CPU evaluation is roughly two times slower than Sigmoid and three times slower than ReLU.The timing comparison evaluated random matrices of shape (1000, 1000).Source: Regression task · p. 11
• On MNIST, the fractional Jacobi neural block outperforms the alternative activations in the CNN classification experiments.The comparison is reported in Table 2 and Figure 6.Source: Image classification · p. 12, Table 2 · p. 13
• The denoising experiment maps Gaussian-noised Fashion MNIST images to clean targets, while the IMDB sentiment experiment evaluates a 1D CNN using binary-classification metrics.The denoising noise factor is 0.3; the IMDB metrics are accuracy, precision, recall, and ROC-AUC.Source: Image denoising · pp. 12–13, Sentiment Analysis · pp. 13–14

4.2 Physics-informed Deep Learning tasks

• For physics-informed tasks, the network minimizes a differential-equation residual, using a Caputo operational matrix to approximate fractional derivatives and a concatenated multi-fJNB architecture.The matrix method uses an L1 finite-difference scheme.Source: §4.2 · pp. 14–15
• For Lane–Emden equations with m = 0 through 5, the paper reports significant accuracy improvements over a similar neural-network approach, assessing accuracy by the first predicted root.Table 5 compares predicted first roots with exact roots and the prior approach.Source: Ordinary differential equations · p. 15, Table 5 · p. 16
• The Burgers PDE simulations show accurate predictions for two parameter choices, whereas the fractional-delay experiment reports residual-error fluctuations linked to architectures with multiple fJNBs.The Burgers comparison uses predicted and exact solutions; Figure 11 reports the fractional-delay solution and residuals.Source: Partial differential equations · p. 16, Figure 10 · p. 17, Fractional differential equations · p. 18

• fKAN extends the KAN framework by using fractional-order orthogonal Jacobi polynomials as basis functions for approximation.Source: §5 · p. 18
• The proposed basis function is nonlinear and has simple, straightforward derivatives, while its fractional order is adaptively tuned during training.Source: §5 · p. 18
• Experiments across deep-learning tasks, including MLP and 1D/2D-CNN architectures, report fKAN's superior performance over traditional KAN and other activation functions.Source: §5 · p. 18
• Compared with simpler activation functions, fKAN has increased time complexity and, relative to KAN, reduced interpretability because its basis functions are global.The paper suggests exploring local basis variants such as fractional B-splines.Source: §5 · p. 18