PaperlayerView exact revision on arXiv

Source-linked AI summary

SW-KAN: Kolmogorov-Arnold Networks with Stieltjes-Wigert q-Orthogonal Polynomials

Amirhosein Azarpour, Seyyed Moein Kazemi

arXiv:2610.00050v1cs.LGcs.AI

TL;DR

Polynomial KANs still face a mismatch between unbounded network inputs and the domains of orthogonal polynomial bases. SW-KAN combines Stieltjes-Wigert polynomials with stable exponential-of-tanh mapping and reports favorable results across classification and 2D function approximation.

  • Problem

    Polynomial KANs lack a systematically evaluated, numerically stable strategy for mapping unbounded real inputs to orthogonal-polynomial supports; bounded-domain mappings can saturate and attenuate gradients.

    Source: §1 · pp. 2–3, §2.3 · p. 5

  • Method

    SW-KAN uses Stieltjes-Wigert q-orthogonal edge polynomials, exponential-of-tanh domain mapping, and three-term recurrence evaluation.

    Source: §1 · p. 3

  • Results

    SW-KAN's results span MNIST, constrained Fashion-MNIST, and 2D approximation; on MNIST, its large configuration reaches 98.24% test accuracy with 219,907 parameters, beating all 18 polynomial baselines at respective budgets.

    Source: §5.1 · p. 19

  • Takeaways & Limitations

    The reported results support SW-KAN as an alternative for resource-constrained classification and low-parameter 2D function approximation.

    Source: §5.1 · p. 19

    Evaluation remains limited to small-scale, low-to-moderate-dimensional tasks; scalability on larger, higher-dimensional problems remains to be assessed.

    Source: §5.2 · p. 20

Abstract

from arXiv · show

Kolmogorov-Arnold Networks (KANs) represent a paradigmatic shift in deep learning by replacing fixed node activations with learnable univariate functions on edges, offering enhanced interpretability and parameter efficiency. While recent polynomial-based KAN variants have addressed the computational overhead of original B-spline implementations, they introduce a fundamental yet underexplored challenge: the domain mismatch between unbounded real-valued inputs and the bounded or semi-infinite support of orthogonal polynomial bases. To address this limitation, we propose the Stieltjes-Wigert Kolmogorov-Arnold Network (SW-KAN), a novel architecture that employs Stieltjes-Wigert q-orthogonal polynomials defined on the semi-infinite domain (0, infinity). We introduce a smooth exponential-of-tanh mapping that stably bridges the domain gap while preserving well-conditioned gradients, and leverage a numerically stable three-term recurrence that evaluates polynomial expansions in O(N) operations without special-function calls. Through comprehensive experiments spanning image classification and continuous function approximation, we demonstrate that SW-KAN achieves superior accuracy-efficiency trade-offs across diverse tasks. The log-normal weight structure and learnable q-parameter of Stieltjes-Wigert polynomials provide a distinct inductive bias that enables robust performance under resource-constrained conditions, including reduced feature dimensionality and limited training data. The proposed architecture not only outperforms established polynomial KAN baselines on standard benchmarks but also exhibits strong representational capacity for approximating complex multivariate functions with remarkably few parameters, making it a compelling alternative for efficient function approximation and classification in resource-constrained settings.

1. Introduction

The introduction identifies a domain mismatch in polynomial KANs and proposes SW-KAN, which pairs Stieltjes-Wigert polynomials with stable domain mapping and recurrence-based evaluation. It reports competitive results on MNIST, constrained Fashion-MNIST, and continuous function approximation.

  • Polynomial KANs reduce B-spline costs but still face a domain mismatch: real-valued layer inputs can exceed the bounded supports of common bases, and tanh normalization can attenuate tail gradients.Source: §1 · pp. 2–3
  • SW-KAN replaces B-spline edge activations with Stieltjes-Wigert q-orthogonal polynomials, whose log-normal weight is supported on the semi-infinite domain (0, ∞).Source: §1 · p. 3
  • An exponential-of-tanh mapping sends real inputs into the polynomials' positive support while preserving well-conditioned gradients.Source: §1 · p. 3
  • A three-term recurrence evaluates the degree-N expansion in O(N) operations without special-function calls.Source: §1 · p. 3
  • Reported advantages generalize beyond a single benchmark and task type; the study also evaluates continuous multivariate function approximation.Source: §1 · p. 3

2. Related Work

The related work traces KANs from approximation-theoretic foundations to edge-based architectures and polynomial alternatives. It identifies domain mismatch and the absence of a stable bridge to semi-infinite supports as an unresolved challenge addressed by SW-KAN.

2.1 Theoretical Foundations and the Limits of Multi-Layer Perceptrons

  • MLPs have universal approximation guarantees but fixed node nonlinearities can demand many parameters; KART instead represents continuous multivariate functions through univariate functions.Source: §2.1 · pp. 3–4

2.2 Kolmogorov-Arnold Networks: Activations on Edges

  • KANs place learnable univariate functions on edges, supporting parameter-efficient approximation and mathematical interpretability; later variants extend the framework to multiplication nodes and symbolic discovery [2] [4].Source: §2.2 · p. 4

2.3 Polynomial KAN Variants: Efficiency Gains and Persistent Bottlenecks

  • Polynomial KANs reduce B-spline recursion and grid-management costs, but no basis consistently optimizes parameter-accuracy trade-offs, and bounded-domain mappings can saturate or destabilize gradients [5] [6].B-spline evaluation and grid extension also complicate parallelization and mini-batch training [16] [17].Source: §2.3 · pp. 4–5

2.4 q-Orthogonal Polynomials and the Semi-Infinite Domain Opportunity

  • Stieltjes-Wigert polynomials combine log-normal orthogonality on (0, ∞), O(N) recurrence evaluation, and a tunable q parameter; SW-KAN adds stable mapping from real inputs to that support.Source: §2.4 · pp. 5–6

3.1 Kolmogorov–Arnold Theorem

This section presents the Kolmogorov-Arnold Representation Theorem as a foundation for KANs. It outlines the theorem's univariate decomposition, its historical limitations, and the edge-function architecture that made the framework practical.

3.1 Kolmogorov–Arnold Theorem

  • KART provides a theoretical foundation for Kolmogorov-Arnold Networks.Source: §3.1 · p. 6

3.1.1 Statement of the Theorem

  • Any continuous function on [0, 1]^n can be represented exactly using continuous univariate functions and summation, without multiplying input variables.Source: §3.1.1 · p. 6

3.1.2 Significance and Limitations

  • The theorem was long considered impractical for machine learning because constructive algorithms were lacking and inner functions could be non-smooth; later work connected it to neural-network realizations [24, 23, 25, 26].Source: §3.1.2 · pp. 6–7

3.1.3 From KART to KAN

  • KANs place learnable univariate functions on edges and sum them at nodes [2]; the modular framework permits polynomial bases such as Stieltjes-Wigert in place of B-splines [2] [6].Source: §3.1.3 · p. 7

3.2 Stieltjes–Wigert Polynomials

The section introduces Stieltjes-Wigert polynomials and their q-dependent forms, then describes recurrence relations for efficient evaluation. Their log-normal connection and recurrence structure distinguish this q-orthogonal family.

3.2 Stieltjes–Wigert Polynomials

  • Stieltjes-Wigert polynomials are q-orthogonal, connected to the log-normal distribution, and have an indeterminate moment problem.Source: §3.2 · p. 7

3.2.1 Definition and Equations

  • The polynomials are defined for 0 < q < 1 and x in (0, ∞).Source: §3.2.1 · p. 7

3.2.2 Hypergeometric Form

  • The section presents a basic-hypergeometric 1φ1 form and identifies the q-binomial coefficient; q is related to a weight-function parameter k > 0.Source: §3.2.2 · p. 7

3.2.2.1 Explicit Low-Degree Forms

  • The section lists the first few Stieltjes-Wigert polynomials in explicit low-degree forms.Source: §3.2.2.1 · p. 8

3.2.3 Recurrence Relations

  • A three-term recurrence relation provides a central route for computing Stieltjes-Wigert polynomials.Source: §3.2.3 · p. 8

3.2.3.1 Monic form.

  • In monic form, recurrence coefficients involve powers of q, enabling evaluation of the first N basis functions in O(N) operations without special-function calls.Source: §3.2.3.1 · p. 8

3.2.4 Orthogonality

The Stieltjes–Wigert polynomials are orthogonal on (0, ∞), with equivalent discrete and continuous weight descriptions. Their log-normal weight has specified moments, yet the moment problem is indeterminate.

3.2.4 Orthogonality

  • Stieltjes–Wigert polynomials are orthogonal on (0, ∞), and the passage notes that multiple distinct positive measures share their moment sequence.The text attributes this indeterminacy to failure of Carleman’s condition.Source: §3.2.4 · p. 8, §3.2.4.2 · p. 9

3.2.4.1 Discrete Weight; 3.2.4.2 Log-Normal Weight

  • The text presents an orthogonality relation with respect to a discrete measure involving a q-Pochhammer theta product and an equivalent relation for a continuous log-normal weight.The continuous-weight description introduces q = e−1/(2k2).Source: §3.2.4.2 · p. 9, §3.2.4.1 · p. 9

3.2.4.2 Log-Normal Weight

  • The log-normal weight has normalization constant hn > 0, and its moments are given as q−n(n+1)/2 for n = 0, 1, 2, ... .Source: §3.2.4.2 · p. 9

3.3 Justification for the Stieltjes–Wigert Basis vs. Classical Alternatives

The section motivates the Stieltjes–Wigert basis through its log-normal weight, learnable q-deformation, and indeterminate moment problem. It presents these properties as relevant to activation distributions, representational adaptability, and regularization.

3.3 Justification for the Stieltjes–Wigert Basis vs. Classical Alternatives

  • The log-normal weight is described as better suited to skewed, heavy-tailed neural-network activations and as mitigating premature saturation for large-magnitude inputs.Source: §3.3 · p. 9
  • The learnable q parameter tunes basis-root spacing and oscillatory frequency, allowing local representation resolution to adapt to the empirical data distribution during gradient descent.Source: §3.3 · p. 9
  • The indeterminate moment problem is presented as an implicit regularizer that provides a smoother optimization landscape and reduces overfitting risk from high-degree classical polynomials on unbounded domains.Source: §3.3 · pp. 9–10

3.4 Domain Handling

The section addresses the mismatch between real-valued network inputs and the Stieltjes–Wigert support (0, ∞) using a smooth, invertible mapping. Its chosen mapping bounds transformed inputs to a range motivated by the polynomials’ roots.

3.4 Domain Handling

  • The mapping bridges the mismatch between real-valued layer inputs and the Stieltjes–Wigert polynomials’ natural support, (0, ∞).Source: §3.4 · p. 10
  • The exponential-of-tanh mapping sends real inputs smoothly and monotonically into (e−γ, eγ), a positive bounded interval intended to support stable gradients and avoid numerical overflow.The experimental configuration sets γ = 2 and ι = 1, giving the interval (e−2, e2) ≈ (0.135, 7.389).Source: §3.4 · p. 10
  • Figure 1 presents Stieltjes–Wigert polynomials S0–S5 for different values of q.Source: §3.4 · p. 10, Figure 1 · p. 11

3.5 Stieltjes–Wigert KAN

SW-KAN parameterizes learnable KAN edge activations with Stieltjes–Wigert polynomials and maps inputs to their positive support. It evaluates the expansions using a stable orthonormal recurrence and a layer-shared learnable q parameter.

3.5 Stieltjes–Wigert KAN; 3.5.1 Edge Activation via Stieltjes–Wigert Polynomials

  • SW-KAN uses Stieltjes–Wigert polynomial expansions as learnable univariate edge activations within a KAN layer.Source: §3.5 · p. 11, §3.5.1 · p. 11

3.5.1 Edge Activation via Stieltjes–Wigert Polynomials

  • Each edge has learnable expansion coefficients, while a smooth mapping sends inputs to the positive polynomial domain and one learnable q is shared across each layer.Source: §3.5.1 · pp. 11–12

3.5.2 Recurrence-Based Evaluation

  • Iterating the three-term recurrence computes polynomial values through degree N in O(N) arithmetic operations and powers of q, without special-function calls.The paper says this makes SW-KAN cheaper than variants using heavier hypergeometric families such as Askey–Wilson polynomials [5].Source: §3.5.2 · p. 12

3.5.3 Numerically Stable Evaluation via an Orthonormal Recurrence

  • The monic recurrence can produce rapidly growing coefficients and inf/NaN activations, so SW-KAN instead evaluates a mathematically equivalent orthonormal recurrence.Source: §3.5.3 · p. 12
  • The orthonormal recurrence keeps Tn of order O(1) over the working domain, and replacing Sn with Tn preserves the same degree-N polynomial subspace.Source: §3.5.3 · pp. 12–13

3.5.4 Learnable Parameters

  • A layer uses one shared q parameter rather than one per edge, keeping q-deformation overhead to one scalar per layer.The layer’s expansion coefficients contribute nℓnℓ+1(N + 1) parameters.Source: §3.5.4 · p. 13

3.5.5 Network Computation

  • A SW-KAN layer maps each input component to the polynomial domain, evaluates the Stieltjes–Wigert expansion on every edge, and combines edge outputs at each neuron.Stacking L such layers yields the full network, which can be trained with standard stochastic gradient methods.Source: §3.5.5 · pp. 13–14

3.6 Theoretical Computational Complexity and Polynomial Degree

SW-KAN’s recurrence gives a linear-in-degree basis evaluation without B-spline knot management, while a low polynomial degree balances representational capacity and generalization.

3.6 Theoretical Computational Complexity and Polynomial Degree

  • The Stieltjes–Wigert recurrence evaluates degree-N bases in O(N) operations per edge, versus O(kG) for degree-k B-splines over G knots.Recurrence coefficients depend only on q and can be updated in O(1) time; B-splines also require dynamic grid extension.Source: §3.6 · p. 14
  • At 105,856 parameters, the medium SW-KAN uses N = 3 to capture cubic nonlinearity while limiting high-degree oscillatory artifacts.The passage describes cubic nonlinearity as allowing up to two inflection points and identifies the avoided artifacts as Runge’s phenomenon.Source: §3.6 · p. 14

4. Experiments and Results

Experiments assess SW-KAN on MNIST, resource-constrained Fashion-MNIST, and continuous 2D function approximation. It outperforms B-spline KAN on Fashion-MNIST and approximates three surfaces with low reported test errors.

4. Experiments and Results

  • The experiments evaluate accuracy, model size, and training time on MNIST, then test performance on Fashion-MNIST and continuous multivariate functions.The stated goal is to assess trade-offs against polynomial-KAN and B-spline-KAN baselines across these tasks.Source: §4 · p. 14

4.1 Digit Classification on MNIST

  • MNIST uses 60,000 training and 10,000 test images, flattened to 784 features and normalized to [0, 1].The experiments use the standard split without data augmentation.Source: §4.1 · pp. 14–15

4.1.1 Experimental Setup

  • The MNIST SW-KAN uses three hidden KAN layers with learnable Stieltjes-Wigert edge expansions; medium and large configurations have 105,856 and 219,907 parameters.The parameter budget is controlled by maximum polynomial degree and channels per edge; the layer’s q parameter is trainable and shared across its edges.Source: §4.1.1 · p. 15

4.1.2 Baselines

  • MNIST comparisons cover 18 polynomial-KAN baselines, with Table 1 listing test accuracy, parameter count, and training time.The baselines use a similar KAN backbone and differ mainly in their polynomial basis.Source: §4.1.2 · p. 15, Table 1 · p. 16

4.2 Fashion-MNIST Classification

  • 83.10% vs 82.65% test accuracy, with 32% fewer parameters, favors SW-KAN under 10,000-sample, 40-feature Fashion-MNIST conditions.Both models converge within approximately 15 epochs; B-spline KAN reaches 83.13% validation accuracy at epoch 9, while SW-KAN reaches 82.60% at epoch 12.Source: Figure 3 · p. 17, §4.2 · pp. 17–18

4.3 Function Approximation on 2D Surfaces

  • Test MSEs are 2.5 × 10^-5 for f1, 2.9 × 10^-5 for f2, and 3.5 × 10^-3 for f3, using 389 parameters.The reported error plots show errors distributed across the domain; the model captures smooth sinusoidal surfaces and multi-scale oscillatory structures.Source: Figure 4 · p. 18, §4.3 · p. 19

5. Conclusion and Discussion

The conclusion presents SW-KAN as a q-orthogonal-polynomial KAN with an exponential-of-tanh input map and recurrence-based evaluation. Its experiments report a superior accuracy-efficiency trade-off across classification and function approximation, while identifying scale and design questions for future work.

5. Conclusion and Discussion

  • SW-KAN replaces B-spline edge activations with Stieltjes–Wigert q-orthogonal polynomials and uses a stable three-term recurrence to evaluate expansions in O(N) operations without special-function calls.Source: §5 · p. 19

5.1 Discussion of Experimental Findings

  • Across classification and function approximation, SW-KAN achieves a superior accuracy-efficiency trade-off compared with existing polynomial-based KANs.Source: §5.1 · p. 19
  • On MNIST, SW-KAN’s medium configuration reaches 97.53% test accuracy with 105,856 parameters, exceeding Vieta–Pell-KAN’s 97.49% at the same parameter count.Source: §5.1 · p. 19
  • On Fashion-MNIST, SW-KAN reaches 83.10% test accuracy with 14,634 parameters, versus 82.65% with 21,658 parameters for B-spline KAN, using 40 PCA features and 10,000 samples.Source: §5.1 · p. 19
  • With 389 parameters, SW-KAN achieves test MSEs of 2.5×10^-5–2.9×10^-5 for smoother 2D surfaces and 3.5 × 10^-3 for the more oscillatory target.Source: §5.1 · p. 19
  • The exponential-of-tanh map projects inputs into a positive interval away from zero and extreme tails; learnable q and log-normal weights are associated with a smoother, better-conditioned loss landscape.Source: §5.1 · p. 20

5.2 Limitations and Future Work

  • Validation remains limited to small-scale, low-to-moderate-dimensional tasks; larger, higher-dimensional evaluations are needed to assess scalability.Source: §5.2 · p. 20
  • Mapping parameters were fixed and q was shared layer-wise; the effects of finer q parameterizations and the network implications of moment indeterminacy remain open.Source: §5.2 · p. 20
Loading 2610.00050v1…