PaperlayerView exact revision on arXiv

Source-linked AI summary

ER-KANs: Efficient and Robust Kolmogorov-Arnold Networks for Data-Scarce Scientific Machine Learning

Harshil Lodhiya

arXiv:2608.14773v1cs.LGcs.AI

TL;DR

Efficient-KANs have been evaluated mostly on clean data, leaving their behavior under noisy, data-scarce conditions insufficiently characterized. ER-KAN addresses this gap with shared Gaussian RBF bases, curriculum noise injection, and adaptive regularisation, achieving a 1.4× noise degradation ratio versus 7.9× for vanilla KAN and 10.6× for ChebyKAN while all models fail on Burgers’.

  • Problem

    Efficient-KANs are benchmarked mostly on clean data, limiting evidence about architecture behavior under noisy, data-scarce scientific machine learning conditions.

  • Method

    ER-KAN is a 595-parameter KAN variant combining shared Gaussian RBF bases, curriculum noise injection, and adaptive regularisation.

  • Results

    ER-KAN showed a 1.4× noise degradation ratio across eight analytic functions, versus 7.9× for vanilla KAN and 10.6× for ChebyKAN.

  • Takeaways & Limitations

    ER-KAN offers stable, predictable behavior as noise grows, making it relevant to physical-science and engineering settings where measurement noise is common.

  • Takeaways & Limitations

    All models fail to converge on Burgers’ equation with ν =0.01/π.

Abstract

from arXiv · show

The efficient-KAN literature---covering Chebyshev, wavelet, and radial-basis-function variants of the original Kolmogorov-Arnold Network---has been benchmarked almost entirely on clean data. We show that this choice conceals a large capability difference between architectures: ChebyKAN's test MSE (evaluated against clean ground truth) increases by a factor of 10.6x when training data is corrupted with sigma=0.1 noise, versus 7.9x for vanilla KAN, 1.7x for a standard MLP, and just 1.4x for our proposed ER-KAN. ER-KAN combines three design choices targeting the noisy, data-scarce setting: shared Gaussian RBF bases across all edges in a layer (providing locality and efficient parameterisation), curriculum noise injection during training (explicitly teaching noise robustness), and entropy-weighted adaptive regularisation (preventing overfitting at small N). The result is a 595-parameter network that matches MLP accuracy at moderate noise while degrading far more gracefully as noise grows. We evaluate on eight analytic functions (N in {50, 200, 500}, sigma in {0, 0.03, 0.1}), on a damped harmonic oscillator physics-informed neural network where ER-KAN achieves 4.2x lower solution MSE than MLP, and on a Burgers' equation PINN where all models fail to converge---a genuine limitation we report rather than suppress. We introduce the noise degradation ratio as a simple complementary metric and recommend it become a standard reporting requirement for efficient-KAN papers.

1 Introduction

ER-KAN targets noisy, data-scarce scientific machine learning, where basis choice strongly affects robustness. The paper introduces a lightweight architecture, a noise degradation ratio, and experiments spanning analytic functions and PINNs.

  • Motivation: Scientific machine learning commonly involves 5–20% target corruption and scarce data, making basis-function choice consequential.Sources include sensor noise, simulation discretisation error, experimental uncertainty, and expensive simulation runs.
  • Central finding: 10.6×: Chebyshev models amplify σ=0.1 noise despite achieving the best clean-data geomean RMSE of 0.025 at N =50.The clean-data comparison range for other models is 0.030–0.132.
  • Central finding: 1.4×: ER-KAN’s Gaussian RBF basis degrades less than vanilla B-spline KAN at 7.9× and remains close to a parameter-matched MLP at 1.7×.The paper presents ER-KAN as the best among KAN-family models by this degradation measure.
  • Method: 595 params: ER-KAN combines shared Gaussian RBF bases, curriculum noise injection, and adaptive regularisation.These design choices define the proposed lightweight KAN variant.
  • Evaluation: The noise degradation ratio DR complements clean-data RMSE and reveals a 7-fold capability difference.The evaluation includes 8 functions, 3 sample sizes, 3 noise levels, and 5 seeds per model, while PINNs show opposite outcomes: oscillator success and Burgers’ failure.

2 Background and Related Work

KANs replace MLP edge operations with learnable univariate functions, while efficient variants use cheaper basis functions such as Gaussian RBFs, Chebyshev polynomials, wavelets, and reflective linear bases. Prior work covers KAN-based PINNs and general noise-robustness methods, but does not compare KAN bases through noise degradation.

  • KAN foundations: KANs place learnable univariate functions on network edges using linear combinations of cubic B-spline basis functions.This replaces the standard MLP dot-product-plus-activation operation.
  • Efficient KAN variants: Efficient KAN variants replace expensive per-sample, per-edge, per-layer B-spline evaluation with analytically cheaper bases.FastKAN uses fixed Gaussian RBF centres, while ChebyKAN, WaveKAN, and FasterKAN use Chebyshev, wavelet, and reflective linear bases, respectively.
  • Efficient KAN variants: FastKAN assigns Gaussian RBF centre sets per edge, whereas ER-KAN shares one centre set across each layer, halving parameter count and reducing noise sensitivity.The passage identifies FastKAN as the closest prior to ER-KAN and attributes the noise-sensitivity comparison to an ablation.
  • Noise sensitivity: ChebyKAN is fast and accurate on clean data, but its derivative bound |T′_d(x)| ≤ d^2 implies perturbation amplification proportional to d^2.The stated bound connects polynomial degree with sensitivity to input perturbations.
  • PINNs and robustness: PINNs encode governing PDEs as residual losses without labeled solution data, while prior KAN-PINN results are problem-dependent and spectral bias remains an MLP failure mode.The paper’s experiments confirm that problem dependence.
  • Noise sensitivity: Noise robustness has commonly been studied through data augmentation or input dropout, but prior work has not compared KAN basis functions using noise degradation.The paper positions its degradation-ratio comparison as addressing this gap.

3 ER-KAN Architecture

ER-KAN uses shared Gaussian RBF bases, curriculum noise injection, and entropy-weighted regularisation to target efficient and noise-robust learning. Its shared-basis design is the main effective component, while entropy regularisation serves only as a safeguard in this setting.

  • Noise sensitivity: For default σg = 0.1, the RBF sensitivity bound is approximately 3.7, an order of magnitude smaller than the degree-8 Chebyshev bound.The analytic prediction is confirmed by empirical degradation ratios in Table 2.
  • Shared RBF basis: Shared Gaussian RBF centres across all edges give each edge G scalar weights, and removing sharing worsens the geomean MSE ratio to 1.54× the full model.The centres are uniformly placed in [−1, 1], with trainable widths initialised to 1/G.
  • Curriculum noise injection: Quadratically decayed curriculum noise transitions training from aggressive augmentation to clean-data fine-tuning, while removing it increases geomean MSE by 7%.The base noise scale matches the expected noise level.
  • Adaptive regularisation: Entropy-weighted ℓ1 regularisation encourages sparse activations on small datasets but has no measurable effect here, yielding a 1.00× ratio.The authors retain it as a regularisation safeguard rather than claiming it contributes to performance.
  • Efficiency: The 595-parameter ER-KAN trains 2.7× faster than vanilla KAN per epoch and has 8× lower inference latency, while training 2× slower than MLP per epoch.Its inference latency is 1.76 vs 1.07 µs, described as negligible in practice.

4 Experimental Setup

The experiments span eight analytic functions under controlled sample sizes and training-label noise, plus damped-oscillator and Burgers’ equation PINNs. Four models are compared using clean test labels, with noise sensitivity summarized by a multiplicative degradation ratio.

  • Analytic-function suite: Eight analytic functions cover oscillatory, algebraic, smooth-exponential, and discontinuous-like behaviors.The suite includes one- and two-dimensional functions such as sin(πx), the Runge function, |x|, exponential forms, a step function, and sin(πx1) cos(πx2).
  • Evaluation protocol: 360 runs per model result from 8 functions, 3 sample counts, 3 noise levels, and 5 seeds.Training inputs are uniform on [−1, 1]d, test inputs form a fixed 2,000-point grid, and only training labels are corrupted; test labels remain clean.
  • Evaluation protocol: The noise degradation ratio is the multiplicative increase in clean-test MSE caused by σ-noisy rather than clean training.Its geometric mean is reported across the eight functions at fixed N and σ; DR ≈1 indicates insensitivity, whereas DR ≫1 indicates compounded qualitative impact.
  • Physics-informed benchmarks: The damped harmonic oscillator PINN uses 200 collocation points, loss weights 1:10:5, and 5,000 training epochs.The equation uses ζ = 0.15 and ω = 2.0 over t ∈[0, 10], with initial position and velocity conditions.
  • Physics-informed benchmarks: The Burgers’ equation PINN uses 2,500 interior collocation points, loss weights 1:20:20, and 20,000 training epochs.Reference solutions use SciPy RK45 on a 512 × 201 grid with rtol=10−9 and atol=10−11.
  • Model comparisons: Four models are compared: ER-KAN (595 params), ChebyKAN (320 params), vanilla KAN (801 params), and MLP (4,353 params for 1D, 8,577 for PINN).All models share the outer training loop, while only ER-KAN uses curriculum noise augmentation.

5 Results

Results show that clean-data rankings favor ChebyKAN and vanilla KAN, but substantially understate ER-KAN’s robustness to noisy, data-scarce training. ER-KAN also improves oscillator-PINN accuracy, while all models fail on Burgers’ equation.

  • Analytic-function benchmark: At N = 50 and σ = 0, ChebyKAN leads geomean RMSE at 0.025, followed by vanilla KAN at 0.030, MLP at 0.119, and ER-KAN at 0.132.This clean-data advantage persists across sample sizes, but clean rankings are misleading under noise.
  • Noise robustness: At σ = 0.1, ChebyKAN degrades 10.6× and vanilla KAN 7.9×, whereas ER-KAN changes only 20%, from RMSE 0.132 to 0.158.At σ = 0.03, ChebyKAN degrades 2.6× versus ER-KAN’s 1.04×; Chebyshev derivatives can amplify perturbations up to d2 = 64, versus approximately 3.7 for Gaussian RBFs.
  • Analytic-function benchmark: On the 2D quadratic at N = 50 and σ = 0.1, ER-KAN achieves RMSE 0.071 versus 0.240 for vanilla KAN, 0.172 for ChebyKAN, and 0.140 for MLP.ER-KAN also achieves RMSE 0.265 versus MLP’s 0.415 on the 2D sinusoidal function, while its degradation-ratio advantage is strongest in data-scarce settings.
  • Efficiency and ablations: ER-KAN matches MLP’s accuracy with geomean MSE ratio 1.01 at 0.61× its training speed, while running 2.7× faster than vanilla KAN and 4.6× faster at inference.Basis sharing is the dominant component: removing it increases geomean MSE by 54%, whereas curriculum noise injection improves it by 7% and adaptive regularisation has no measurable effect.
  • Physics-informed benchmarks: On the oscillator PINN, ER-KAN achieves best-solution MSE 1.17 × 10−7, a 4.2× improvement over MLP’s 4.91 × 10−7.All models reached the residual target of 10−4 on every seed, so solution-MSE differences reflect accuracy rather than convergence failures.
  • Physics-informed benchmarks: On Burgers’ equation with ν = 0.01/π, none of the models converge; MLP has 10.3% mean L2 relative error, versus 19.5–26.7% for KAN variants.The reported limitation is attributed to Adam with 20,000 epochs being insufficient and smooth bases being mismatched to the near-discontinuous shock.

6 Discussion

The discussion identifies noise degradation ratio as a robustness complement to RMSE, attributes ER-KAN’s resilience primarily to shared basis functions, and explains its strengths and Burgers’ failure mode. It recommends reporting DR alongside RMSE in future efficient-KAN evaluations.

  • Noise degradation ratio: DR measures proportional sensitivity rather than absolute error, becoming decisive for deployment across noise conditions and beyond σ = 0.1.ChebyKAN can retain lower absolute RMSE at σ = 0.1 despite ER-KAN’s flatter degradation trajectory.
  • Comparative performance: ER-KAN is non-inferior to MLP in 5 scarcity/noise cells, with a geomean ratio of 1.01, and is 1.97× faster than efficient-kan.Its ratio against efficient-kan is 1.44.
  • Mechanisms: Basis sharing is the dominant robustness factor, with its removal causing a +54% change versus −7% for removing curriculum noise.Shared centres regularise edges by forcing a common latent representation instead of independent noise fitting.
  • Mechanisms: Shared Gaussian RBF bases may better model smooth interaction terms in noisy 2D functions than Chebyshev or B-spline bases.The proposed explanation is that localised RBF activations reduce cross-term interference during multivariate composition.
  • Limitations: Burgers’ fails because a shock layer concentrates residuals in a small region, while smooth KAN bases struggle to approximate the near-discontinuity.The discussion identifies adaptive sampling or second-order methods as needed remedies and gives MLP a minor representational advantage.
  • Recommendation: The paper recommends reporting DR alongside RMSE because one additional noisy-training evaluation reveals robustness that clean-data RMSE can hide.This recommendation is intended as a standard reporting requirement for future efficient-KAN papers.

7 Limitations and Future Work

ER-KAN has clear limitations in clean-data accuracy, absolute accuracy at tested noise levels, dimensionality, and shock-dominated PDEs. Future work includes locating the noise crossover, adapting RBF centres, improving Burgers’ training, and clarifying the role of entropy-weighted regularisation.

  • Performance limitations: ER-KAN does not match polynomial-basis KANs on clean data, so low-noise deployment should prefer ChebyKAN or vanilla KAN.This limitation applies when both training and test conditions are low-noise.
  • Performance limitations: At σ =0.1, ChebyKAN’s absolute RMSE is lower than ER-KAN’s for most 1D functions.The crossover where ER-KAN’s stability advantage dominates in absolute terms occurs above the tested noise levels and remains to be characterised.
  • PDE limitations: All models fail on Burgers’ with ν =0.01/π.Coupling ER-KAN with adaptive collocation or L-BFGS is identified as an open direction.
  • Architecture directions: Fixed uniformly spaced RBF centres may limit clean-data performance.Adaptive placement could concentrate centres where functions vary rapidly without sacrificing noise robustness.
  • Scalability limitations: The highest-dimensional experiment is 2D, and for d > 5 the shared-basis approach may need modification to avoid the curse of dimensionality.Higher-dimensional behaviour is therefore not established by the reported experiments.
  • Regularisation directions: The entropy-weighted penalty had no measurable effect.Understanding when it contributes, if ever, and whether another regularisation design would help remains an open question.

8 Conclusion

ER-KAN is presented as a 595-parameter KAN variant combining shared Gaussian RBF bases, curriculum noise injection, and adaptive regularisation. Its central contribution is stable noise robustness, quantified by a 1.4× noise degradation ratio versus 7.9× for vanilla KAN and 10.6× for ChebyKAN, alongside a recommendation to report this metric routinely.

  • Core contribution: 1.4× noise degradation ratio—versus 7.9× for vanilla KAN and 10.6× for ChebyKAN—was measured across eight analytic functions with σ=0.1 noise at N=50.ER-KAN uses 595 parameters and combines shared Gaussian RBF bases, curriculum noise injection, and adaptive regularisation.
  • Ablation: Basis sharing consistently dominates the ablation, curriculum noise provides a smaller consistent gain, and adaptive regularisation remains flat across noise–N regimes.The ablation compares geometric-mean MSE ratios against the full ER-KAN.
  • Scope and limitation: Stable, predictable behaviour as noise grows is ER-KAN’s stated advantage for physical-science and engineering settings where measurement noise is common.The paper explicitly does not claim ER-KAN is the best efficient KAN in every setting.
  • Evaluation recommendation: The noise degradation ratio is proposed as a simple addition to function-approximation evaluations and recommended as standard reporting practice.The authors argue that it reveals robustness properties currently invisible in the literature.
  • Limitations: No model reached the <1% target on Burgers’ PINN, with MLP identified as the least bad model.This result is reported as a genuine limitation rather than suppressed.

A Full Scarcity–Noise Sweep · B ODE Surrogate Results · C Additional Figures

The supplied material identifies an ODE surrogate sample-efficiency analysis but provides no substantive findings for the full scarcity–noise sweep or additional figures. The paper context nevertheless frames ER-KAN as efficient and robust under noisy, data-scarce scientific machine learning settings.

  • A Full Scarcity–Noise Sweep: The full scarcity–noise sweep is named as subsection A, but no numerical results or methodological details are supplied here.The available passage contains only an ODE surrogate figure caption.
  • B ODE Surrogate Results: Figure 16 presents ODE surrogate sample efficiency curves.The caption identifies the figure’s subject without reporting curve values or model rankings.
  • B ODE Surrogate Results: The supplied passage does not state the ODE surrogate models, sample sizes, noise levels, or performance metrics.Consequently, no quantitative comparison can be recovered from the available caption.
  • B ODE Surrogate Results: The ODE analysis is explicitly visualized through curves rather than summarized by a reported scalar result in the supplied passage.Figure 16 is the only evidence node provided for this merged section block.

D Reproducibility

The study reports a CPU-only Apple M3 Pro implementation with five controlled random seeds, specified architecture and optimization settings, and supplementary scripts for reproducibility.

  • Hardware: All main experiments ran on an Apple M3 Pro CPU under macOS, with no GPU used for comparisons.MPS timings are reported only for completeness.
  • Software: The implementations used PyTorch 2.x, SciPy, and NumPy, with ER-KAN, ChebyKAN, and vanilla KAN written from scratch without external KAN libraries.The code was included in the experiment scripts.
  • Seeds: 5 seeds (0–4) controlled weight initialisation and data sampling, with torch.manual_seed and np.random.seed set globally before each run.This procedure was applied across all experiments.
  • Hyperparameters: ER-KAN used G = 8, hidden dimension 32, two layers, σbase = 0.1, λ = 10−4, cosine-decayed Adam learning rate 10−3 →10−5, batch 64, and patience 500.The reported baselines used ChebyKAN degree 8, MLP hidden dimension 64, and vanilla KAN G = 5 B-spline knots; PINN loss weights were also specified.
  • Code: All scripts are included in the supplementary material.This provides the experiment code for independent reproduction.
Loading 2608.14773v1…