PaperlayerView exact revision on arXiv

Source-linked AI summary

RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basis

Amirhosein Azarpour

arXiv:2609.01729v1cs.LGcs.AI

TL;DR

KAN variants learn edge functions but keep their basis family fixed before training. RecKAN learns a second-order recurrence that generates the basis; it achieves the highest reported classification accuracy and lowest ETTh1 MSE among the compared KAN variants, and its CNN heads outperform matched MLP heads on three datasets.

  • Problem

    Existing KAN variants fix the basis family before training and learn only combination weights, making basis selection a practitioner decision.

    Source: §1 · p. 2

  • Method

    RecKAN learns the five coefficients of a second-order polynomial recurrence that generates its basis and includes several classical polynomial families as special cases.

    Source: §5 · p. 14

  • Results

    RecKAN reports the highest test accuracy on all classification tasks and lowest ETTh1 MSE among compared KAN variants, while CNN–RecKAN heads outperform matched MLP heads on Fashion-MNIST, CIFAR-10, and SVHN.

    Source: §5 · p. 14

  • Takeaways & Limitations

    Learning the basis-generating recurrence offers an alternative to selecting a fixed polynomial basis in advance.

    Source: §5 · p. 14

    One shared set of five recurrence coefficients may limit expressiveness when layers, channels, or input regions require different univariate representations.

    Source: §6 · p. 14

Abstract

from arXiv · show

Kolmogorov--Arnold Networks (KANs) replace the fixed scalar weights of a standard network with learnable univariate functions on each edge, but existing variants still fix the \emph{basis} that those functions are built from: B-splines, Chebyshev polynomials, wavelets, or Jacobi polynomials, and learn only the combination weights over it. We introduce RecKAN, which instead defines the basis itself by a second order polynomial recurrence, $R_{n+1}(x) = (ax^2+bx+c)R_n(x) + (dx+e)R_{n-1}(x)$, whose five coefficients are learned jointly with the network. We show this recurrence recovers several classical polynomial families including both kinds of Chebyshev polynomials, Fibonacci, Pell, and Jacobsthal polynomials as special cases, and prove that its degree grows linearly in $n$ exactly on the sub-family containing all of them, giving a concrete sense in which the learned basis can move beyond any fixed classical choice. Across multiple benchmark datasets spanning image, text, biomedical time series classification, and time series forecasting, RecKAN outperforms three parameter-matched KAN baselines (Chebyshev, Jacobi, and spline based) on all classification tasks and achieves the lowest MSE on the ETTh1 forecasting benchmark. Additionally, when used as a classifier head with a convolutional backbone, RecKAN achieves higher accuracy than standard MLP heads on Fashion MNIST, CIFAR-10, and SVHN. On a synthetic function fitting benchmark it tracks a sharply oscillatory target that a parameter comparable MLP under fits. We further show that the learned recurrence coefficients are interpretable: on the task requiring the most local structure, training moves the basis away from the linear degree growth regime that contains every classical family we identify, consistent with our theoretical analysis of what that structural shift enables.

1 Introduction

KAN variants learn edge functions but retain a basis selected before training. RecKAN learns the recurrence that generates the basis, and the introduction outlines matched-baseline evaluations, CNN-head comparisons, and analysis of learned recurrence parameters.

1 Introduction

  • KANs replace scalar edge weights with learnable univariate functions, and prior work reports parameter-efficiency and interpretability benefits relative to MLPs.In simple cases, edge functions can be inspected as closed-form expressions.Source: §1 · pp. 1–2
  • Existing KAN variants replace spline bases with alternatives such as Chebyshev, wavelet, or fractional Jacobi polynomials, but still fix the basis family before training.Only the linear-combination weights over that fixed basis are learned, leaving basis selection to the practitioner or trial and error.Source: §1 · p. 2
  • RecKAN learns the five coefficients of a second-order polynomial recurrence that generates the basis, rather than choosing a family in advance.The recurrence includes Chebyshev, Fibonacci, Pell, and Jacobsthal polynomial families as special cases and can move beyond them.Source: §1 · p. 2
  • RecKAN classifier heads consistently improve over standard MLP heads on three image datasets, while the study also analyzes the learned recurrence parameters against its theoretical characterization.The contributions describe comparisons on Fashion-MNIST, CIFAR-10, and SVHN.Source: §1 · p. 2
  • The proposed benchmark comparison matches parameter budgets across image, text, biomedical time-series classification, and forecasting tasks against three KAN baselines.The baselines are Chebyshev-, Jacobi-, and spline-based KANs.Source: §1 · p. 2

2 Related Work

Related work expands KAN architectures and applications while often retaining a basis family fixed by design. RecKAN instead learns the recurrence that generates its basis.

2 Related Work

  • KAN research spans theoretical expressiveness, benchmarking, surveys, and architectural variants for convolutional, graph, temporal, segmentation, and parameter-efficient settings.Applications include computer vision, sensor-data processing, and permutation-invariant function representation.Source: §2 · pp. 2–3
  • Physics-informed KAN work includes PDE and power-system applications, with KINN reporting gains over MLP-based PINNs across several problem settings.Related approaches include Navier–Stokes models and hybrid encoder-decoder KAN-GRU architectures.Source: §2 · p. 3
  • Fixed-basis KAN variants use polynomial, wavelet, radial-basis, ReLU, or Fourier families, while training adjusts coefficients or parameters within the chosen family.A comparative study surveys eighteen polynomial families as candidate KAN bases [29].Source: §2 · p. 3
  • RecKAN differs from those variants by learning the second-order recurrence that generates its basis, while containing several classical recurrence families as special cases.The cited classical families include Chebyshev, Fibonacci, Pell, and Jacobsthal polynomials [11].Source: §2 · p. 3

3 Method

RecKAN replaces fixed KAN basis families with a recursively generated basis whose shared coefficients are learned alongside the network. Its recurrence can recover selected classical families, while its degree growth, layer-specific combination weights, and stability controls define how the basis is used and trained.

3.1 Preliminaries: Kolmogorov–Arnold Networks

  • KANs place learnable univariate functions on edges, and RecKAN learns the recurrence that generates their basis instead of fixing a basis family in advance.Earlier variants fix families such as B-splines, Chebyshev polynomials, wavelets, or fractional Jacobi polynomials and learn only their combination weights.Source: §3.1 · pp. 3–4

3.2 A Learnable Recursive Polynomial Basis

  • The basis uses a second-order recurrence with five coefficients shared across recursion steps, so its parameter count stays fixed as order K changes.The degree bound is at most 2(n−1) when a ≠ 0 and n−1 when a = 0; equality requires no leading-term cancellation.Source: §3.2 · p. 4

3.3 Recovering Classical Polynomial Families

  • The recurrence recovers both kinds of Chebyshev polynomials, while its degree-independent coefficients restrict which classical families can appear as special cases.The first- and second-kind Chebyshev sequences arise under different initial conditions; the cited table lists additional recovered families.Source: §3.3 · pp. 4–5, Table 1 · p. 5

3.4 Edge Functions and the RecKAN Layer

  • Each RecKAN edge function is a learned linear combination of the generated basis values, with separate combination weights for each input-output edge.A layer with d_in inputs, d_out outputs, and order K has d_in d_out (K + 1) combination weights.Source: §3.4 · p. 5

3.5 Network Architecture and Parameter Sharing

  • A RecKAN network composes layers with their own combination tensors while sharing one set of five recurrence coefficients across every layer.Pre-activations are mapped through tanh before entering the recurrence.Source: §3.5 · pp. 5–6, Figure 2 · p. 6, Figure 1 · p. 6

3.6 Numerical Stability and Training Scheme

  • Training controls recurrence growth with bounded coefficients, batch-based magnitude normalization, bounded inputs, and separate recurrence-parameter optimization with optional warm-up and gradient clipping.Normalization uses one scalar per basis index, detached from gradient computation; tanh bounds recurrence inputs.Source: §3.6 · pp. 6–7

4 Experiments and Results

Across parameter-matched KAN benchmarks, RecKAN achieves the highest mean test accuracy on all four classification datasets and the lowest ETTh1 forecasting MSE; its CNN classifier head also outperforms MLP heads on three image datasets. On synthetic targets, RecKAN fits local high-frequency detail, while the 1D run learns a recurrence with greater degree capacity.

4.1 Experimental Setup

  • Comparisons use closely matched KAN parameter counts and equal training budgets; synthetic function-fitting comparisons use parameter-comparable MLPs.KAN baselines include ChebyKAN, a Jacobi-polynomial variant, and a B-spline variant.Source: §4.1 · p. 8

4.2 Function Approximation

  • At epoch 33,000, 1D training MSE is 0.0391 for 149-parameter RecKAN versus 0.1583 for a 385-parameter MLP, whose fit misses rapid local oscillations.RecKAN tracks both the oscillatory and smooth regions.Source: Figure 3 · p. 8, §4.2 · pp. 8–9
  • At epoch 3,001, 2D training MSE is 0.113 for 1,445-parameter RecKAN versus 0.200 for a 2,305-parameter MLP; RecKAN recovers more local high-frequency structure.Both models reproduce broad variations in the noisy target surface.Source: Figure 4 · p. 9, §4.2 · p. 9

4.3 Analysis of the Learned Recurrence Parameters

  • The 2D recurrence stays near its initialization, but the 1D run learns a=0.335, raising the degree bound at order K=8 from seven to fourteen; the proposed fit connection is descriptive, not causal.In 2D, a=0.049 remains near zero; the paper says controlled ablations would be needed to establish causality.Source: Table 4 · p. 10, §4.3 · p. 10

4.4 Benchmark Classification and Forecasting

  • RecKAN achieves the highest mean test accuracy on all four classification datasets and the lowest ETTh1 mean test MSE: 0.011172 versus 0.011879 for SplineKAN, 0.033786 for JacobiKAN, and 0.036781 for ChebyKAN.On MNIST, CIFAR-10, AG News, and ECG5000, the strongest-baseline margins are 0.20, 0.89, 0.13, and 0.08 percentage points, respectively.Source: Table 5 · p. 11, §4.4 · pp. 11–12

4.5 RecKAN Classifier Heads for Convolutional Networks

  • CNN–RecKAN exceeds CNN–MLP test accuracy on Fashion-MNIST (93.357% vs 92.840%), CIFAR-10 (88.503% vs 88.117%), and SVHN (96.150% vs 96.070%) with 40 extra parameters.The models use identical convolutional backbones and closely matched parameter counts.Source: §4.5 · pp. 12–13, Table 7 · p. 13

5 Conclusion

RecKAN learns the recurrence that generates its polynomial basis, recovering familiar polynomial families while allowing bases beyond predefined choices. It outperforms the compared KAN variants on classification and ETTh1 forecasting benchmarks, and its CNN head also outperforms matched MLP heads.

5 Conclusion

  • RecKAN learns a second-order recurrence to generate its basis, recovering Chebyshev, Fibonacci, Pell, and Jacobsthal polynomial sequences while permitting bases beyond predefined families.Source: §5 · p. 14
  • Activating the quadratic recurrence coefficient raises maximum degree growth from one to two degrees per step, making higher-degree basis elements possible at fixed order.The learned coefficients make this capacity mechanism observable in the basis.Source: §5 · p. 14
  • RecKAN achieves the highest reported test accuracy on all four classification benchmarks and the lowest MSE on ETTh1 among compared Chebyshev, Jacobi, and spline-based KAN variants.The comparisons use closely matched parameter budgets.Source: §5 · p. 14
  • With identical convolutional feature extractors and nearly identical parameter counts, CNN–RecKAN achieves higher reported test accuracy than matched MLP heads on Fashion-MNIST, CIFAR-10, and SVHN.Source: §5 · p. 14

6 Limitations and Future Work

The discussion identifies limits in how RecKAN shares and parameterizes its recurrence, in the stabilization scheme, and in interpreting learned coefficients. It outlines extensions and analyses that could address these boundaries.

6 Limitations and Future Work

  • One shared set of five recurrence coefficients may limit expressiveness when different layers, channels, or input regions need substantially different univariate representations.The authors suggest studying layer-, group-, or edge-specific recurrences while preserving numerical stability and interpretability.Source: §6 · p. 14
  • The second-order recurrence does not cover all classical orthogonal polynomial families whose recurrence coefficients vary with basis index.Controlled index-dependent or higher-order recurrences, rational recurrences, and mixtures with non-polynomial bases are proposed as possible extensions.Source: §6 · p. 14
  • Stabilization relies on bounded recurrence coefficients and inputs, basis normalization, and separate optimization control for recurrence parameters.How these components interact with basis order, network depth, normalization design, and data modalities remains open.Source: §6 · p. 14
  • Further analysis of optimization geometry and numerical properties may clarify when particular recurrence structures are preferable.Source: §6 · p. 15
  • The learned coefficients’ task-level interpretation remains descriptive, motivating links to spectral properties, effective degrees, and structural regularizers.Source: §6 · p. 15

A Additional Visualizations of the Recursive Basis

The appendix visualizes learned recurrence coefficients and the resulting bases across experiments. Its plots compare coefficient shifts, degree growth, and learned bases with classical polynomial families.

A Additional Visualizations of the Recursive Basis

  • The visualizations cover recursive basis functions and learned coefficients from experiments discussed in Section 4.3, supplementing the main text with views of task-specific basis adaptation.Source: A Additional Visualizations of the… · p. 17
  • Figure 5 compares learned recurrence coefficients across experiments and shows deviation from the initial Chebyshev-like configuration, particularly in image benchmarks.Source: Figure 5 · p. 18
  • Figure 6 plots maximum polynomial degree against basis index; the SVHN learned coefficients reach degree 14, compared with degree 7 at initialization.Source: Figure 6 · p. 18
  • Figure 7 compares learned bases with Chebyshev, Fibonacci, Pell, and Jacobsthal families, illustrating recovery of known families or movement beyond them.Source: Figure 7 · p. 19

B Complete Proof of Proposition 1 (Degree Growth)

The proof establishes degree bounds for the recursive basis in two cases: with a nonzero quadratic coefficient, degree grows by at most two per step; with that coefficient zero, it grows by at most one. The bounds are attained when dominant terms do not cancel.

B Complete Proof of Proposition 1 (Degree Growth)

  • For a ≠ 0, deg Rn ≤ 2(n − 1); for a = 0, deg Rn ≤ n − 1, with equality in either case when no leading-term cancellation occurs.The proof establishes the two bounds by induction.Source: B Complete Proof of Proposition 1… · p. 20
  • The degree bounds are achieved only when dominant terms in the recurrence do not cancel.The leading-coefficient conditions determine whether equality holds in each case.Source: B Complete Proof of Proposition 1… · p. 21
  • The quadratic term ax^2 permits up to two degrees of growth per recursion step, while a = 0 permits at most one degree per step.The proof frames the degree result in terms of the degree contributions from the recurrence terms.Source: B Complete Proof of Proposition 1… · p. 21
  • At basis order K = 8, the maximum degree is 7 when a = 0 and 14 when a ≠ 0.Activating the quadratic term therefore permits higher-degree basis functions without increasing K.Source: B Complete Proof of Proposition 1… · p. 21
Loading 2609.01729v1…