PaperlayerView exact revision on arXiv

Source-linked AI summary

Kolmogorov-Arnold Networks: Curse of Dimensionality and Structural Bias

Yunhao Ni, Yuxin Guo, Jie Luo, Wenjun Wu, Lei Huang

arXiv:2610.21480v1cs.LG

TL;DR

The paper asks whether KANs’ favorable dimensional scaling is an architectural advantage or depends on restricted target structure. It proves a worst-case lower bound and compares KANs with MLPs across constructed and trained targets. The findings show that favorable scaling is not a general KAN-specific escape from the curse of dimensionality.

  • Problem

    KANs have been reported to attain dimension-independent approximation exponents on targets with sufficiently smooth representations, leaving the worst-case parameter cost for standard spline-KANs on broad Sobolev classes to be quantified.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    The paper proves a Sobolev-class lower bound for arbitrary-depth standard spline-KANs and compares KANs and MLPs on scaling-law, controlled ridge-profile, and MNIST random-label tasks.

    Source: §1 · p. 2, §5 · p. 7, §5.2 · p. 8

  • Results

    Standard spline-KANs require at least exponentially many spline coefficients below a fixed accuracy threshold on isotropic Sobolev balls; empirically, KANs favor decoupled ridge components, while interacting targets often favor MLPs at tight budgets.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Favorable dimensional dependence on low-arity targets is not exclusive to KANs, and the empirical ranking varies with target structure and budget.

    Source: §6 · p. 9

    The theorem covers standard spline-only KANs with arbitrary edge-wise knot locations, not variants with SiLU base branches, multiplication nodes, or non-spline edges.

    Source: §6 · p. 9

Abstract

from arXiv · show

Kolmogorov-Arnold networks (KANs) have a dimension-independent approximation exponent when the target admits a sufficiently smooth representation by univariate edge functions. We ask whether this favorable scaling belongs to the architecture or to the assumed target class. For arbitrary-depth standard spline-KANs, we prove a worst-case lower bound on isotropic Sobolev balls: below a dimension-independent accuracy threshold, the required number of spline coefficients grows at least exponentially with input dimension. Existing composition-matched MLP results, by contrast, show that favorable exponents on low-arity targets are not exclusive to KANs. We also revisit empirical comparisons. On four scaling-law targets, spline-KAN approximants constructed from known decompositions transfer exactly to MLPs with the cubic ReLU activation $\sigma(x)=(\max\{0,x\})^3$, although their full dense parameter counts are larger. In controlled ridge-profile scans, KANs have a strong advantage when each nonlinear component depends on one input coordinate. When a component couples multiple input coordinates, this advantage may narrow sharply. At tight budgets, interacting targets often favor MLPs. In the random-label test on MNIST, ReLU-MLPs fit the randomized training labels more efficiently than spline-only KANs. Together, these results locate favorable scaling in target structure and a compatible inductive bias, not a general KAN-specific escape from the curse of dimensionality.

1 INTRODUCTION

KANs’ favorable approximation scaling is conditional on smooth, low-dimensional target structure, so the paper asks whether it reflects the architecture or the assumed target class. It combines a worst-case lower bound with finite-budget comparisons to examine that distinction.

1 INTRODUCTION

  • KANs’ dimension-independent exponent assumes a sufficiently smooth Kolmogorov–Arnold representation, and the reported P^-4 scaling experiment used five targets known to admit such representations.The comparison reported by Liu et al. (2025b) found the tested MLPs scaled more slowly and plateaued under the same vanilla LBFGS protocol.Source: §1 · p. 1
  • For arbitrary-depth standard spline-KANs, approximating isotropic Sobolev balls below a dimension-independent accuracy threshold requires at least exponentially many spline coefficients in the input dimension.The paper contrasts this worst-case result with rates on structurally restricted targets.Source: §1 · p. 2
  • The paper argues that favorable dimensional dependence reflects target structure rather than a general KAN approximation advantage.It also notes that composition-matched MLP theory gives favorable rates on low-arity targets.Source: §1 · p. 2
  • On four scaling-law targets, constructed spline-KAN approximants transfer to ReLU3 MLPs with essentially the same test error, though dense parameter counts may be larger.The transfer uses known target decompositions, separating representational possibility from finite-budget training behavior.Source: §1 · p. 2

2 RELATED WORK

Prior work reports favorable approximation rates both for KANs under smooth representation assumptions and for MLPs matched to low-arity target structure. Broad Sobolev classes retain dimension-dependent complexity, motivating a worst-case lower bound for standard spline-KANs.

2.1 FAVORABLE RATES UNDER STRUCTURAL ASSUMPTIONS

  • On structurally restricted targets, KAN rates can have dimension-independent exponents under smooth representation assumptions, while composition-matched MLPs also attain exponents governed by local arity.The MLP results include binary-tree and directed-acyclic-graph classes under compatible architecture and smoothness assumptions.Source: §2.1 · pp. 2–3

2.2 APPROXIMATION LIMITS ON BROAD FUNCTION CLASSES

  • For broad Sobolev classes, existing approximation bounds retain explicit dimension dependence, while prior KAN upper bounds alone do not establish that the dependence is unavoidable.A worst-case lower bound is needed to establish a curse-of-dimensionality requirement for KANs.Source: §2.2 · p. 3
  • The paper’s lower bound targets arbitrary-depth standard spline-KANs using Sobolev bumps, a KAN-to-MLP representation, dense parameter counting, and a VC-dimension bound.This complements Hu et al. (2026), whose lower bound treats two-layer B-spline KANs through a direct VC-dimension argument.Source: §2.2 · p. 3

2.3 SUBSEQUENT KAN DEVELOPMENTS AND LIMITATIONS

  • Later KAN variants and comparisons broaden the literature, but reported practical results depend on task and protocol; this paper further tests target-structure effects with ridge profiles and MNIST.The cited comparisons include MLP wins outside symbolic formula representation and lower B-spline KAN accuracy and efficiency on tested scientific machine-learning problems.Source: §2.3 · p. 3

3 PRELIMINARIES

The preliminaries define the approximation problem and model families used to transfer a lower bound from ReLUk MLPs to standard spline-KANs. They also establish an exact edge-level spline realization that supports the full parameter-count argument.

3 PRELIMINARIES

  • The complexity question is how many parameters a fixed architecture needs to approximate every target in the specified function class to a prescribed uniform accuracy.Parameter values may depend on the target function.Source: §3 · pp. 3–4

3.1 ReLUk-MLPS

  • A ReLUk-MLP composes affine maps with coordinate-wise truncated-power activations, where σk(x)=(max{x,0})^k.Its trainable-parameter count includes weights and biases across affine layers.Source: §3.1 · p. 4

3.2 SPLINE-KANS

  • A standard spline-KAN sums learnable degree-k spline edge functions at receiving nodes, with G+k trainable coefficients per edge.The theorem’s model boundary allows arbitrary knot locations but excludes SiLU base branches and non-spline variants.Source: §3.2 · pp. 4–5

3.3 EXACT SPLINE REALIZATION BY ReLUk

  • Each degree-k spline edge has an exact one-hidden-layer ReLUk representation of width O_k(G+k), which enables counting affine parameters in the full MLP construction.The shifts are edge knots, and the representation coefficients depend linearly on the spline coefficients.Source: §3.3 · p. 5

4 WORST-CASE COD OF SPLINE-KANS

The lower-bound argument transfers spline-KANs to ReLUk-MLPs, establishes Sobolev approximation lower bounds for those MLPs, and transfers the bounds back to spline-KANs. It shows that standard spline-KANs retain worst-case curse-of-dimensionality behavior on isotropic Sobolev classes.

4.1 REPRESENTATION BRIDGE

  • A spline-KAN is represented exactly by a ReLUk-MLP, and the parameter-count bound tracks all affine weights and biases.The representation permits knot locations to differ across edges.Source: §4.1 · pp. 5–6

4.2 A SOBOLEV LOWER BOUND FOR ReLUk-MLPS

  • The Sobolev lower bound uses smooth bumps encoding binary labels, then combines a VC-dimension lower bound with upper bounds for fixed-degree piecewise-polynomial networks.The argument adapts Yarotsky (2017, Theorem 4).Source: §4.2 · p. 6

4.3 STANDARD SPLINE-KANS SUFFER FROM COD

  • For fixed accuracy below a dimension-independent threshold, standard spline-KANs require a number of spline coefficients that grows at least exponentially with input dimension.The theorem treats arbitrary-depth standard spline-KANs on isotropic Sobolev balls.Source: §4.3 · pp. 6–7
  • The proof applies the ReLUk-MLP approximation lower bound to the exact representation and transfers its parameter-cost requirement back to spline-KANs.The theorem includes a separate logarithmic-depth case.Source: §4.3 · pp. 6–7
  • Together with composition-matched MLP results, the lower bound suggests that comparable approximation capabilities on structured targets are not unique to KANs.The section proceeds to experimental comparisons of the architectures.Source: §4.3 · p. 7

5 EMPIRICAL BOUNDARIES BETWEEN KANS AND MLPS

The experiments distinguish what the architectures can represent from what finite-budget training finds. KANs do well on decoupled ridge components, while interacting targets and random-label fitting expose settings where MLPs can compare favorably.

5.1 REVISITING THE KAN–MLP SCALING-LAW COMPARISON

  • On four scaling-law targets, known decompositions yield spline-KAN approximants that transfer exactly at step zero to ReLU3 MLPs representing the same functions.Validation selects an unmodified MLP for two targets and a lower-error fine-tuned MLP for the Bessel and exp-sine targets; MLP curves use larger full dense parameter counts.Source: §5.1 · p. 7, Figure 1 · p. 8

5.2 INTERACTION SUPPORT AND THE FINITE-BUDGET BOUNDARY

  • At moderate budgets, square-ridge targets with one active coordinate per component favor KANs; interactions sharply contract that advantage, tight budgets generally favor MLPs, and the largest tested budget favors KANs.The main-text scan uses d = 16 and compares selected test-MSE ratios across budgets and support sizes.Source: §5.2 · p. 8, Figure 2 · p. 9

5.3 MNIST CLASSIFICATION AND RANDOM-LABEL TEST UNDER MATCHED BUDGETS

  • On MNIST, both families reach high standard-label test accuracy, but ReLU-MLPs use fewer parameters and fit random labels more efficiently than spline-only KANs.For standard labels the reported metric is test accuracy; for random labels it is final training accuracy.Source: §5.3 · p. 9

6 CONCLUSION

The paper concludes that worst-case dimensional scaling remains for standard spline-KANs, while favorable behavior on structured targets depends on target structure and compatible inductive bias. Its empirical comparisons also identify boundaries in which KANs and MLPs fare better.

6 CONCLUSION

  • Arbitrary-depth standard spline-KANs require exponentially many spline coefficients to approximate isotropic Sobolev balls below a fixed accuracy threshold; favorable low-arity exponents are not exclusive to KANs.The conclusion places this beside composition-matched MLP results.Source: §6 · p. 9
  • Constructed spline-KAN approximants transfer to MLPs, while ridge scans favor KANs on decoupled components and often favor MLPs on interacting targets at tight budgets.The ridge boundary shifts with target nonlinearity and budget; the random-label MNIST test probes fitting beyond structured targets.Source: §6 · p. 9
  • The theorem covers spline-only standard KANs with arbitrary edge-wise knot locations, not variants with SiLU base branches, multiplication nodes, or non-spline edges.Source: §6 · p. 9

A.1 B-SPLINE CONVENTION

  • Under the stated B-spline convention, B_r,k is a degree-k piecewise polynomial supported on [τ_r, τ_(r+k+1)] and is C^(k−1) across simple interior knots.Source: §A.1 · p. 12

A.2 SCOPE RELATIVE TO PRACTICAL AND NON-SPLINE KANS

The practical KAN edge includes a trainable SiLU base branch alongside its spline, but that branch is outside the spline-only analysis. The scope also covers relocated spline knots, not other KAN basis families.

A.2 SCOPE RELATIVE TO PRACTICAL AND NON-SPLINE KANS

  • A practical edge adds a trainable SiLU base-function branch to its B-spline expansion, but the branch is excluded from the spline-only model analyzed here.The branch is distinct from a layer-wise residual or skip connection.Source: §A.2 · p. 12
  • Grid updates only relocate knots, leaving each edge a degree-k spline with G intervals covered by the representation bridge.Thus arbitrary edge-wise knot locations remain within the stated spline scope.Source: §A.2 · p. 12
  • Fourier, rational, wavelet, and polynomial-basis KANs lie outside the analysis and require separate representation or lower-bound arguments.The cited examples include Fourier KANs (Zhang et al., 2025), rational KANs, wavelet KANs (Bozorgasl & Chen, 2024; Seydi et al., 2024), and polynomial-basis variants (Sidharth S. S. et al., 2024; Seydi, 2024).Source: §A.2 · p. 12

B PROOFS AND SUPPORTING RESULTS

The parameter-count lemma converts each spline-KAN layer into shifted ReLU^k units and composes those realizations into an exact ReLU^k MLP. Its width and depth bounds then support a dense parameter count.

B PROOFS AND SUPPORTING RESULTS

  • Each KAN layer is realized by at most C_k a_l H shifted ReLU^k units, followed by an affine summation, and the stacked construction exactly realizes g.Here H = G + k and a_l = d_l d_{l+1}.Source: §B.1 · p. 12
  • The constructed MLP has hidden width at most C_k w^2(G + k) and depth at most C_k L.The bounds follow from M_l ≤ C_k d_l d_{l+1}H and the architecture's layer count.Source: §B.1 · p. 12
  • The dense parameter count sums the first, intermediate, and final affine-map parameters, then bounds each summand by w^4 for a coarser bound.The affine-map counts depend on adjacent hidden widths and the input and output dimensions.Source: §B.1 · p. 12

B.2 PROOF OF THE ReLUk VC-DIMENSION LOWER BOUND

The proof builds separated Sobolev bumps on a grid so that approximating every function forces the network to shatter the grid. A VC-dimension bound then yields a worst-case parameter lower bound.

B.2 PROOF OF THE ReLUk VC-DIMENSION LOWER BOUND

  • For fixed architecture and width W, the thresholded ReLU^k network class has VCdim at most e c_3 L^2 W ln(2W), with constants depending only on k.This activation-specific estimate is the only activation-specific input used below.Source: §B.2 · p. 13
  • A grid of separated points and disjoint smooth bumps encodes arbitrary Boolean labels, so any network approximating every constructed function must shatter the grid.At most one bump and its derivatives are nonzero at each point; thresholding outputs at 3ϵ/2 separates the two labels.Source: §B.2 · p. 13
  • For fixed d, n, and k, decreasing the constant extends the bound to the remaining ϵ in [ϵ0, 1).The dimension-uniform threshold ϵ0 is specified separately in the theorem statement.Source: §B.2 · p. 13
  • For sufficiently small c2 > 0, W ≥ c2 ϵ^(-d/n) ln^(-2p-1)(1/ϵ); decreasing c2 further makes this hold for every ϵ in (0, 1/2), proving part (b).Source: §B.2 · p. 13

B.3 PROOF OF THE SPLINE-KAN COD THEOREM

The theorem proof transfers spline-KANs into a ReLU^k architecture to obtain a worst-case lower bound. The experimental sections distinguish exact representational transfer, controlled scan protocols, and target- and activation-dependent empirical crossovers.

B.3 PROOF OF THE SPLINE-KAN COD THEOREM

  • The spline-KAN function class transfers into a ReLU^k MLP with architecture width and depth independent of knot values, so the MLP inherits any uniform approximation capability of that KAN.The proof applies the MLP lower bound after bounding the transferred architecture's parameter count.Source: §B.3 · p. 14

C EXPERIMENTAL DETAILS

  • Reported parameter counts include every trainable scalar used in the forward pass, while all reported KANs are spline-only and omit residual connections.Ridge-profile active edges count G + k coefficients plus a scale; MNIST edges count only G + k coefficients because the scale is fixed.Source: C EXPERIMENTAL DETAILS · p. 14

C.1 CONSTRUCTIVE SCALING PROTOCOL AND SOURCE COMPARISON

  • Constructed spline-KAN interpolants are assembled as dense ReLU3 MLPs, but their full parameter counts include all affine entries and are not a matched-parameter comparison.The four target constructions use cubic interpolation of known scalar functions; the 100-dimensional target's domain is unspecified in Liu et al. (2025b), and the nested target's formula and topology are inconsistent there.Source: §C.1 · p. 14, Figure 3 · p. 15

C.2 CONTROLLED RIDGE-PROFILE PROTOCOL

  • The ridge-profile scan uses normalized random coefficients and directions, fresh online training samples, and fixed evaluation sets; checkpoint selection and seed pooling target the best fit within a fixed search budget.The scan contains 45 target cells per profile and 324 architecture–learning-rate configurations per cell and seed.Source: §C.2 · pp. 15–16

C.3 TARGET PROFILE AND ACTIVATION SENSITIVITY

  • Across the four target profiles, KANs have a strong regime at s = 1, while the crossover at larger s depends on target and budget; square and SiLU do not show a universal MLP win.For sine at d = 64, some dense cells leave both models near the unit-variance baseline, so near-unit error ratios do not indicate successful fitting.Source: §C.3 · p. 16
  • Removing the sine-matched activation leaves KAN lower-error in 20 of 24 cells at P = 10^4 for d ∈ {4, 16}, but the sparse-versus-dense support gap remains large.The KAN sparse-target advantage also persists with a fixed ReLU comparator, while activation matching changes where the crossover occurs.Source: §C.3 · p. 16

C.3.1 EXACT REPRESENTATION AND OPTIMIZATION DIAGNOSTIC

The section separates an exact sine-MLP representation from what training discovers, then checks selected-model error ratios against absolute-error baselines.

C.3.1 EXACT REPRESENTATION AND OPTIMIZATION DIAGNOSTIC

  • The sine-profile target has an exact one-hidden-layer sine-MLP representation, and its dense implementation uses R(d + 2) + 1 trainable parameters.The construction uses dR input weights, R hidden biases, R output weights, and one output bias.Source: §C.3.1 · p. 16
  • The complete square-ridge scan reports test-MSE ratios for d, R ∈ {4, 16, 64} using Figure 2’s training-probe selection rule.Figure 4’s middle row is shown in the main text.Source: Figure 4 · p. 17
  • At d = R = 64, explicit weights reached test MSE 1.96 × 10−13 without training, versus 0.542 for the default-initialized s = 1 model.Providing teacher ridge directions reduced error, while matching only the first-layer weight norm with random directions did not resolve the high error.Source: §C.3.1 · p. 21
  • Removing sine activation expands the KAN-favored region, but denser supports narrow that advantage, including with a fixed ReLU comparator.With ReLU, increasing support again shifts the comparison toward MLPs.Source: Figure 8 · p. 21, Figure 9 · p. 22
  • At moderate and large budgets, most selected models beat the MSE-one zero-predictor baseline, and training-probe and independent-test curves closely track.The absolute errors show that the ratios generally do not compare two models stuck near the trivial predictor.Source: §C.3.1 · p. 21, Figure 11 · p. 23

C.4 MNIST CLASSIFICATION AND RANDOM-LABEL TEST

The MNIST comparison evaluates standard labels and independently randomized training labels across scanned model budgets. MLPs achieve a higher normal-label test frontier and fit random labels more effectively than the scanned spline-only KANs.

C.4 MNIST CLASSIFICATION AND RANDOM-LABEL TEST

  • Each seed uses all 60,000 official MNIST training images; random-label conditions assign each image an independent uniform label from 0 to 9.Normal-label evaluation uses the official 10,000-image test partition, and both model families receive the same images and labels within each condition and seed.Source: §C.4 · p. 24
  • For normal labels, test accuracy is read from the optimizer and learning rate selected by final training accuracy; random-label fitting is measured by training accuracy.Figure 12 reports three-seed architecture medians after optimizer and learning-rate selection, with the best available value shown within each parameter budget.Source: §C.4 · p. 24, Figure 12 · p. 24
  • Across scanned budgets, the MLP test envelope rises faster and reaches higher accuracy on normal labels; with random labels, wider MLPs approach 100% training accuracy while KANs remain below 20%.Figure 12 shows full-budget frontiers, and Figure 13 shows the depth–width scan, where the random-label fitting gap persists across tested depths.Source: §C.4 · p. 24, Figure 12 · p. 24, Figure 13 · p. 25
  • With 6,000 training images, the MLP eventually fits all randomized labels, while the best spline-only KAN plateaus below 100% training accuracy.Random-label fitting is easier at this sample size, but the qualitative separation persists.Source: §C.4 · p. 24, Figure 14 · p. 25

Related papers

Loading 2610.21480v1…