KAN: Kolmogorov-Arnold Networks

• MLPs are foundational nonlinear-function approximators, yet they consume substantial transformer parameters and are comparatively difficult to interpret without post-analysis tools.Source: §1 · p. 2
• KANs replace MLP node activations and linear weight matrices with learnable univariate spline functions on edges, while nodes only sum incoming signals.Despite spline-valued parameters, KANs usually permit much smaller computation graphs than MLPs.Source: §1 · p. 2
• KANs generalize Kolmogorov-Arnold representations to arbitrary widths and depths, using modern training methods and experiments to assess their potential for AI + Science.Earlier work largely focused on depth-2 width-(2n + 1) representations, with limited use of modern techniques such as back propagation.Source: §1 · p. 2
• KANs combine MLP-like external feature learning with spline-like internal univariate optimization, addressing the complementary weaknesses of splines and MLPs under the curse of dimensionality.Splines are accurate and locally adjustable for low-dimensional functions but suffer from compositional curse of dimensionality, whereas MLPs learn features more effectively but approximate univariate functions less accurately.Source: §1 · p. 2
• KANs can outperform MLPs by a large margin on high-dimensional functions with compositional structure because they learn both the composition and its univariate functions.Splines can fail for large N because of curse of dimensionality, while MLPs may inefficiently approximate exponential and sine functions with ReLU activations.Source: §1 · p. 3
• The paper evaluates KANs for accuracy and interpretability through data fitting, Poisson-equation solving, and scientific-law discovery in knot theory and Anderson localization.The planned experiments examine scaling laws, PDE solving, and KANs as collaborators for rediscovering mathematical and physical laws.Source: §1 · p. 3

2.1 Kolmogorov-Arnold Representation theorem

• KANs are motivated by smooth, sparse compositional functions, despite the original theorem’s univariate functions potentially being nonsmooth or fractal and therefore difficult to learn.The approach generalizes beyond the theorem’s fixed two-layer form because practical scientific and machine-learning functions may have exploitable structure.Source: §2.1 · p. 4

2.2 KAN architecture

• KANs parameterize univariate functions with B-splines, placing learnable activations on edges and summation on nodes instead of separating weights and nonlinearities.The original representation becomes a two-layer KAN with hidden width 2n + 1, while deeper networks stack generalized KAN layers.Source: §2.2 · pp. 4–6

2.3 KAN’s Approximation Abilities and Scaling Laws

• Under smooth Kolmogorov–Arnold representations, finite-grid KANs achieve approximation rates independent of dimension, thereby potentially beating the curse of dimensionality.The constant in the bound still depends on the representation and therefore on dimension; for m = 0, the result yields an L∞ accuracy and RMSE bound with scaling exponent k + 1.Source: §2.3 · pp. 7–8
• With cubic splines, KANs have scaling exponent α = 4, which the paper identifies as larger than those of the compared theories and reports as empirically achievable.The exponent follows α = k + 1 for spline order k = 3, while excessively large k may create oscillatory functions and optimization problems.Source: §2.3 · pp. 8–9

2.4 For accuracy: Grid Extension

• Grid extension increases KAN accuracy by refining spline grids after initial training, allowing larger models to inherit a smaller model’s parameters without retraining from scratch.Fine-graining produces staircase-like loss drops; in the toy example, the [2, 5, 1] KAN’s test loss followed a U-shape and reached its observed interpolation threshold near G ∼50.Source: §2.4 · pp. 9–10, Figure 2.3 · p. 9
• Smaller KANs can generalize better when their architecture matches the function structure: a [2, 1, 1] KAN achieved lower test losses than the [5] KAN in the synthetic task.The smaller model also delayed the interpolation threshold, motivating later regularization and pruning to discover minimal architectures.Source: §2.4 · p. 10
• KANs show better scaling laws than MLPs for data fitting and PDE solving, with roughly test RMSE ∝G^-3 versus the theoretical G^-4 rate.Using the square root of median squared losses yields scaling closer to G^-4; training time also scales favorably with grid points, except at G = 1000, where LBFGS line search becomes significantly slower.Source: §2.4 · p. 10

2.5 For Interpretability: Simplifying KANs and Making them interactive

• KAN interpretability is improved by starting with an oversized network, applying sparsity regularization, pruning, and then simplifying or interactively modifying the resulting structure.This approach addresses the unknown KAN shape required by a dataset and produces more interpretable pruned networks.Source: §2.5 · p. 11

2.5.1 Simplification techniques

• KAN sparsification combines prediction loss with L1 and entropy regularization, then prunes node-level subnetworks whose incoming and outgoing scores are not both greater than θ = 10−2, while suspected symbolic activations receive affine fitting.Source: §2.5.1 · pp. 11–12

2.5.2 A toy example: how humans can interact with KANs

• In a toy interaction, pruning a [2, 5, 1] KAN produced [2, 1, 1], after which symbolic identification and affine retraining recovered the true formula.The recovered output was 1.0e1.0y2+1.0sin(3.14x), and KANs provide continuous, user-controllable function-space search unlike brittle symbolic regression.Source: §2.5.2 · p. 13

3.1 Toy datasets

• KANs show better parameter–RMSE scaling than MLPs on five toy functions, nearly saturating the predicted α = 4 scaling law while MLPs plateau quickly.The comparison uses LBFGS in a vanilla setup; deeper KANs are more expressive than shallower ones, and the study leaves advanced training comparisons for future work.Source: §3.1 · p. 14, Figure 3.1 · p. 14

3.2 Special functions

• Across 15 special functions with unknown KAN shapes, KANs consistently achieve lower losses at equal parameter counts and better Pareto frontiers than MLPs.Auto-discovered compact representations can decompose high-dimensional lookup tables into several one-dimensional lookup tables with few additions at inference.Source: §3.2 · p. 15, Figure 3.2 · p. 15

3.3 Feynman datasets

• On Feynman equations, MLPs and KANs perform comparably on average, but auto-discovered pruned KANs are usually smaller than human-constructed ones.For relativistic velocity composition, auto-discovery finds a 2-layer KAN corresponding to the rapidity addition trick, versus a 5-layer human construction.Source: §3.3 · p. 17

3.4 Solving partial differential equations

• A 2-layer width-10 KAN is 100 times more accurate than a 4-layer width-100 MLP (10^-7 vs 10^-5 MSE) and 100 times more parameter efficient (10^2 vs 10^4 parameters).Under identical PINN hyperparameters, KANs also achieve smaller errors and steeper scaling laws, but train approximately 10x slower; the symbolic ground truth may favor KANs.Source: §3.4 · p. 18, Figure 3.3 · p. 18

3.5 Continual Learning

• KANs avoid catastrophic forgetting in a sequential five-peak regression task because local spline bases update nearby coefficients, whereas global MLP activations alter distant regions.These are preliminary results from an extremely simple example, and generalization to realistic high-dimensional continual-learning settings remains unclear.Source: §3.5 · p. 19, Figure 3.4 · p. 19

4.1 Supervised toy datasets

• KANs reveal compositional structures in symbolic formulas and learn correct univariate functions, including arithmetic identities for xy and x/y, digit-recognition spikes, and the Bessel function J0(20x) numerically.For xy, computation leverages 2xy = (x + y)^2 − (x^2 + y^2); for x/y, it leverages x/y = exp(logx − logy).Source: §4.1 · p. 20

4.2 Unsupervised toy dataset

• KANs turn unsupervised relation discovery into supervised classification by contrasting real feature vectors with feature-corrupted vectors and enforcing a nonzero relation through a Gaussian final activation.On a synthetic six-dimensional task, different random seeds reveal dependencies among (x1, x2, x3) or (x4, x5), but the preliminary method remains randomness-dependent.Source: §4.2 · pp. 21–22

4.3 Application to Mathematics: Knot Theory

• In knot theory, KANs recover signature dependence with 81.6% test accuracy using approximately 200 parameters, versus 78% for DeepMind’s approximately 3 × 10^5-parameter MLP.KAN visualizations identify µr as the dominant variable, while µi and λ contribute less; µr alone reaches 65.0% accuracy versus 43.8% for µi alone, and a µr–λ symbolic formula reaches 77.8%.Source: §4.3 · pp. 23–24, Table 3 · p. 23
• KANs automatically generate a Pareto frontier of knot-signature formulas and, in unsupervised mode, rediscover three known relations among knot invariants without manually partitioning inputs and outputs.The discovered relations include V = µrλ and the known upper-bound relation 2r ≤ gr; the unsupervised results are reliable but do not yet establish new mathematics.Source: §4.3 · pp. 24–25

4.4 Application to Physics: Anderson localization

• KANs are applied to quasiperiodic tight-binding data to extract Anderson-localization mobility edges in Mosaic, generalized Aubry-André, and modified Aubry-André models.The framework is intended to combine symbolic interpretability with human collaboration: users can propose activation-function assumptions or other hypotheses, which KANs then test efficiently.Source: §4.4 · pp. 26–27
• A [2, 1] KAN recovered the mobility edge while providing qualitative understanding and quantitative results matching the ground truth.The user encoded sigmoid classification and a linear dependence for λ through fix_symbolic before training.Source: §4.4 · p. 27
• 98.9% accuracy yielded an already symbolic GAAM-KAN formula after restricting activations to linear or quadratic functions, approaching the true answer after simplifying terms.The symbolic network slightly improved over the preceding 98.7% model and its formula was reported in Table 5.Source: §4.4 · pp. 28–29
• KANs support an interactive discovery process in which users inspect learned activations, formulate hypotheses, and refine symbolic expressions without pre-specifying a symbolic library.More manual operations generally simplify formulas at a slight accuracy cost, creating a tunable simplicity–accuracy tradeoff.Source: §4.4 · pp. 29–30
• For MAAM, enlarging the KAN raised accuracy from around 75% to 98.4%, while pruning and manually snapping activations preserved 97.7% accuracy.Automatic symbolic regression achieved 97.1%; specifying cosh or quadratic dependence produced 96.9% or 95.4%, respectively.Source: §4.4 · pp. 29–30

5 Related works

• KANs generalize earlier Kolmogorov-Arnold networks to arbitrary widths and depths, addressing their limited expressive power and artificial toy-experiment focus.The paper also relates KANs to generalized additive models, graph neural networks, and kernel machines, while highlighting their potential as foundation models for AI + Science.Source: §5 · p. 30
• A smooth Kolmogorov-Arnold representation lets a high-dimensional function scale as a 1D function, yielding the fastest scaling exponent proposed in this work.Experiments achieve this fast neural scaling law on synthetic datasets, but whether it extends to complex tasks and is found in practice remains open.Source: §5 · pp. 30–31
• KANs pursue active mechanistic interpretability by making both the architecture and training method interpretable by design.This contrasts with passive mechanistic interpretability, which primarily analyzes networks trained with standard methods.Source: §5 · p. 31
• Learnable activations have been parameterized with differentiable or discrete methods using polynomials, splines, sigmoid linear units, and neural networks; KANs use B-splines.The paper also introduces preliminary learnable activation networks whose properties lie between KANs and MLPs, with results deferred to Appendix B.Source: §5 · p. 31
• KANs differ from symbolic-regression methods by continuously learning activation functions before symbolic snapping rather than manually fixing them.Related approaches include genetic algorithms, neural-network methods, physics-inspired methods, and reinforcement learning-based methods.Source: §5 · p. 31

Physics-Informed Neural Networks (PINNs) and Physics-Informed Neural Operators

• KANs can replace MLPs for imposing PDE losses in PDE-solving architectures, with potential applicability across Deep Ritz, PINNs, Fourier neural operators, PINOs, and DeepONet.These methods address PDE solving or operator learning by learning solution maps.Source: Physics-Informed Neural Networks… · p. 31
• AI for mathematics has been applied to knot recognition, knot-invariant prediction, and uncovering relations among invariants, while broader work addresses mathematical and theoretical-physics datasets and rigorous ML-derived results.Examples include detecting unknots or ribbon knots and studying data-science applications in mathematics and theoretical physics.Source: Physics-Informed Neural Networks… · p. 31

6 Discussion

• KAN accuracy may improve through underexplored architecture and training choices, including radial basis functions, other local kernels, and adaptive grid strategies.The paper notes that spline activations are not the only possible choice and that adaptive grids could improve performance.Source: §6 · p. 32
• KAN efficiency suffers because distinct activation functions limit batch computation, while multi-head grouping can interpolate between MLP-style sharing and KAN-style specialization.Within each group, activation functions share parameters, offering a possible efficiency compromise.Source: §6 · p. 32
• Hybrid models can isolate KANs’ two changes by combining learnable node activations with MLP-like placement or fixed edge activations with KAN-like placement.Appendix B studies learnable activations on nodes, while another possible model uses fixed activations on edges.Source: §6 · p. 32
• Spline locality enables adaptive KAN design and training, including multi-level training inspired by multigrid methods and domain-dependent bases inspired by multiscale methods [115] [116] [117].These approaches are proposed to enhance both accuracy and efficiency.Source: §6 · p. 32
• KANs are proposed for science tasks such as Navier–Stokes equations and density functional theory, as well as machine-learning architectures such as transformers through “kansformers.”The application agenda extends existing evidence in physical-equation fitting and PDE solving to regression- and PDE-based tasks more broadly.Source: §6 · p. 32
• Interpretable functions could let KANs support AI–scientist collaboration by communicating in science’s language of functions and adapting to scientists’ domain-specific inductive biases.The discussion favors AI that assists scientists over fully automated AI scientists because human preferences about simplicity and interpretability are difficult to quantify and vary across fields.Source: §6 · pp. 32–33

Final takeaway: Should I use KANs or MLPs?

• 10x slower training than MLPs is KANs’ main bottleneck, but the paper treats this as an engineering issue and recommends KANs when interpretability or accuracy outweighs speed.For fast training, the paper recommends MLPs; for small-scale AI + Science problems where slow training is acceptable, it suggests trying KANs.Source: Final takeaway: Should I use KANs or… · p. 33

• Table 6 summarizes common KAN functionalities that users may find useful.Source: Table 6 · p. 42, A KAN Functionalities · p. 42

B.1 Architecture

• LANs make activation functions learnable while keeping them on nodes, unlike KANs’ edge-based learnable activations, isolating the two architectural changes.LANs are otherwise nearly standard MLPs, with spline-parameterized activations.Source: §B.1 · p. 42
• For width N, depth L, and grid size G, LANs use N^2L + NLG parameters, with spline overhead typically small because G ≪ N.The N^2L term comes from weight matrices, while NLG comes from spline activations.Source: §B.1 · p. 42
• LANs can be initialized from pretrained MLPs and fine-tuned by learning their activation functions, including for improving SIREN.The SIREN application is presented in Section B.3.Source: §B.1 · p. 42
• LANs are conceptually simpler than KANs because they retain standard MLP-like structure and change only the activation functions.KANs additionally move activations from nodes to edges.Source: §B.1 · p. 42
• LAN activation parameters scale as N with width, compared with N^2 for KANs, so LANs scale better.LANs place learnable activations on nodes, whereas KANs place them on edges.Source: §B.1 · p. 42
• LANs seem less interpretable than KANs because their weight matrices remain difficult to interpret, as in MLPs.This limitation follows from retaining MLP-like weight matrices.Source: §B.1 · p. 42
• LANs seem less accurate than KANs but more accurate than MLPs, and spline-parameterized activations admit grid extension.The accuracy comparison is stated qualitatively, without reported numerical values.Source: §B.1 · p. 42

Interpretability results

• LANs appear much less interpretable than KANs on the same synthetic examples where KANs are perfectly interpretable.Figure B.2 reports these preliminary interpretability results.Source: §B.2 · p. 42, Figure B.2 · p. 43
• Weight matrices are less readily interpretable than learnable activation functions and introduce too many degrees of freedom.These degrees of freedom leave LAN learnable activation functions too unconstrained.Source: §B.2 · p. 43
• The preliminary LAN results imply that eliminating linear weight matrices by placing learnable activations on edges, as in KANs, is necessary for interpretability.The inference is based on the contrast between LANs with weight matrices and KANs with edge-based learnable activations.Source: §B.2 · pp. 42–43

B.3 Fitting Images (LAN)

• LANs represent images as 2D functions f(x, y), using activations composed of a base function plus a trainable spline.Because smooth initialization alone does not capture SIREN’s high-frequency features, the base function is set to sine functions.Source: §B.3 · pp. 43–44
• Higher PSNR: LAN outperforms the same-shaped MLP because its learnable activations can be fine-tuned for image representation.Both networks use shape [2,128,128,128,128,128,1].Source: Figure B.3 · p. 44, §B.3 · p. 44
• 4%: The LAN’s additional parameter increase is approximately G/N = 5/128 ≈4% over the original parameters.The experiments use G = 5.Source: §B.3 · p. 44
• MLP initialization: A LAN initialized from an MLP can be further fine-tuned to obtain better PSNR.Training uses Adam for 5000 steps at learning rate 10^-3 followed by 5000 steps at 10^-4.Source: §B.3 · p. 44

C Dependence on hyperparameters

• Entropy penalty reduces the number of active activation functions and prevents duplicate functions, supporting more interpretable graphs.The desired number of active activation functions is as small as possible, ideally three.Source: C Dependence on hyperparameters · p. 44
• Random seeds can change the results, and an unlucky seed may leave the pruned network in an undesirable state.Source: C Dependence on hyperparameters · p. 44
• The overall penalty strength λ effectively controls network sparsity.Source: C Dependence on hyperparameters · p. 44
• A grid number G that is too small makes activation functions less expressive, encouraging ensembling that complicates interpretation.Source: C Dependence on hyperparameters · p. 44
• The piecewise polynomial order k has only a subtle effect on interpretability and, like random seeds, shows no visible pattern in this toy example.Source: C Dependence on hyperparameters · p. 44

D Feynman KANs

• Figure D.1 compares the Pareto frontiers of KANs and MLPs across the Feynman datasets.The comparison covers each Feynman dataset.Source: D Feynman KANs · p. 44
• Figure D.3 visualizes minimal KANs for each Feynman equation-fitting task under test RMSE < 10^-2.Minimality is evaluated subject to the stated test-RMSE constraint.Source: D Feynman KANs · p. 44
• Figure D.2 visualizes the best KAN for each Feynman equation-fitting task using the lowest test RMSE loss.The best models are selected by test RMSE rather than by the minimality constraint.Source: D Feynman KANs · p. 44

E Remark on grid size

• On uniform training grids, both PDE and regression tasks show a sudden training-loss increase when grid size becomes large.The increase corresponds to a sudden drop in performance.Source: E Remark on grid size · p. 45
• The performance drop occurs when grid size becomes comparable to the number of different training points in one spatial direction.This condition is reported for both PDE and regression tasks.Source: E Remark on grid size · p. 45
• The observed large-grid degradation may result from the implementation of B-splines in higher dimensions and requires further investigation.The proposed explanation is explicitly tentative.Source: E Remark on grid size · p. 45

F KANs for special functions

• Best KANs are selected by having the lowest test RMSE loss.Source: F KANs for special functions · p. 45, Figure F.1 · p. 49
• For each special-function fitting task, Figures F.2 and F.1 visualize minimal and best KANs, respectively.The results extend the special-function dataset experiments from Section 3.2.Source: F KANs for special functions · p. 45, Figure F.1 · p. 49, Figure F.2 · p. 50
• Minimal KANs are selected under the constraint test RMSE < 10−2.Source: F KANs for special functions · p. 45, Figure F.2 · p. 50