PaperlayerView exact revision on arXiv

Source-linked AI summary

Sample-Efficiency of Kolmogorov-Arnold Networks

Kevin Riehl, Shaimaa K. El-Baklish, Fan Wu, Anastasios Kouvelas

arXiv:2610.10627v1cs.LG

TL;DR

RL control can require costly environment interactions, and evidence about KAN sample efficiency in RL has been limited and mixed. This paper compares KANs with MLPs in physics-equation regression and PPO-based Gymnasium control. It reports similar performance with 40% fewer samples using KANs, with gains persisting qualitatively under tested reward-noise levels.

  • Problem

    RL control requires many environment interactions, while systematic evidence about KAN sample efficiency in RL control under comparable budgets is lacking.

    Source: §1 · pp. 1–2

  • Method

    The study compares parameter-matched KANs and MLPs on Feynman physics-equation regression and PPO control in Gymnasium environments.

    Source: §1 · p. 2, §3.1 · p. 3, §3.2 · p. 3

  • Results

    KANs achieve similar performance with about 40% fewer samples in Gymnasium, while Feynman mean NRMSE improves from 0.127 to 0.060 at 1000 samples for the best KAN family versus MLPs.

    Source: §4 · p. 4

  • Takeaways & Limitations

    KANs may be useful for low-data control problems with smooth, low-dimensional structure, especially during early and middle training when samples are costly.

    Source: §5 · p. 4

    The size of the Gymnasium advantage varies by environment, and both architectures approach similar return levels after many training episodes.

    Source: §4 · p. 4

Abstract

from arXiv · show

Deep reinforcement learning has achieved substantial performance gains over classical control approaches. Yet, a central challenge to learning in real-world applications is acquiring costly samples. Kolmogorov-Arnold Networks are a recently proposed architecture that can learn physical relationships in control problems effectively, with significantly higher parameter efficiency and interpretability when compared to Multi-Layer-Perceptron architectures. In this work, we systematically study sample-efficiency using computational experiments, covering the Feynman dataset and the Gymnasium RL benchmark. The results show that similar performance can be achieved with 40% fewer samples using the Kolmogorov-Arnold architecture, and that relative performance improvements up to 50% occur during the training process. The observed gains are robust to varying levels of noise in rewards. These results highlight the potential of the Kolmogorov-Arnold architectures for more sample-efficient reinforcement learning. Code: https://github.com/DerKevinRiehl/neurips26_kan_training

1 Introduction

Sample efficiency remains a central challenge in RL control because useful controllers require costly environment interactions. The paper compares KANs with MLPs to assess whether KANs can improve sample efficiency in RL-based control.

1 Introduction

  • RL control often requires many costly environment interactions to learn a useful controller, motivating function approximators that use fewer samples while remaining interpretable.Each control sample is a transition generated by running a policy in a simulator, digital twin, or physical system.Source: §1 · p. 1
  • KANs replace fixed node activations and scalar weights with learnable univariate functions on network edges, offering parameter efficiency and interpretability.The architecture is motivated by applications that need both data efficiency and model transparency.Source: §1 · p. 2
  • Prior findings on KAN sample efficiency are mixed, and systematic comparisons under comparable RL interaction budgets are lacking.Some low-data comparisons find MLPs with individualized learnable activations outperform KANs.Source: §1 · p. 2
  • Across the Feynman dataset and Gymnasium RL benchmark, the study reports similar performance with 40% fewer samples and reward gains up to 50% during training using KANs.The comparison is against MLPs.Source: §1 · p. 2

2 Related Works

Prior work suggests KANs can be parameter-efficient and effective in structured settings, but findings on sample efficiency are mixed. Existing RL studies chiefly emphasize parameter efficiency and interpretability rather than controlled sample-efficiency comparisons.

Kolmogorov–Arnold networks and learning guarantees

  • KAN approximation and sample-complexity guarantees for smooth function classes rely on assumptions that differ from RL control, where targets are noisy, bootstrapped, and policy-induced.KANs place learnable univariate functions on edges rather than fixed nonlinearities on nodes.Source: §2 · p. 2

Low-data and fixed-sample empirical comparisons

  • Low-data comparisons find mixed results: MLPs with individualized learnable activations can outperform KANs, and KANs do not dominate across function types or stabilized test loss.KANs can converge quickly in some cases.Source: §2 · p. 2

Structured and expensive-data settings

  • Structured-data studies report KANs reducing expensive high-fidelity data needs or matching and surpassing an MLP autoencoder with less data, but these are not online RL studies.The reported fault-detection comparison varies training-set sizes.Source: §2 · pp. 2–3

KANs in RL and control

  • RL studies report comparable performance with fewer parameters and interpretable policies, but mainly emphasize parameter efficiency and interpretability rather than controlled sample-efficiency comparisons.Reported applications include PPO control, CartPole, radio-access-network control, and load balancing.Source: §2 · p. 3

KANs for robotic policies and world models

  • KAN components show varied results across policy learning and world models, including cases where direct replacement of visual encoders or actor-critic modules can harm sample efficiency.FastKAN is reported as promising for reward and continue prediction in KAN-Dreamer.Source: §2 · p. 3

3 Experiments

The experiments compare KANs and MLPs in a supervised physics-regression benchmark and in PPO-based Classic Control environments, including tests with noisy rewards. Models are matched by parameter count, and sample counts to reach target performance assess efficiency.

3.1 Feynman Sample-Efficiency Benchmark

  • The Feynman benchmark tests sample efficiency on independently sampled physics-equation regression tasks, using normalized root mean squared error (NRMSE) as the primary metric.Training-set sizes vary while test-set size is fixed.Source: §3.1 · p. 3

3.2 RL Gymnasium Benchmark

  • The Gymnasium experiment uses PPO on five Classic Control environments with state-variable inputs and comparable learning curves and convergence limits.The environments are Acrobot-v1, CartPole-v1, MountainCar-v0, MountainCarContinuous-v0, and Pendulum-v1.Source: §3.2 · p. 3

3.3 RL Gymnasium Benchmark With Noised Rewards

  • A third experiment tests whether KAN sample-efficiency gains over MLPs persist across varying Gaussian reward-noise levels.Source: §3.3 · p. 3

3.4 Comparison and Sample-Efficiency Assessment

  • KAN and MLP models are matched by parameter count, and sample counts needed to reach a target performance measure sample efficiency across ReLU and Sigmoid MLPs and B-Spline and RBF KANs.Grid search explores model complexities and training parameters.Source: §3.4 · p. 4

4 Results

KANs show lower Feynman benchmark NRMSE than MLPs across the reported sample counts, with larger relative improvement at 1000 samples. In Gymnasium control, KANs reach similar performance with fewer samples, though the advantage varies by environment and narrows later in training.

4 Results

  • Gaussian reward noise increases variance, but the KAN–MLP comparison retains the same qualitative shape across tested noise scales.Source: §4 · p. 4
  • Mean NRMSE falls from 0.200 to 0.163 at 50 samples and from 0.127 to 0.060 at 1000 samples for the best KAN family versus MLPs, with relative improvement rising from 13.9% to 49.3%.KANs reach several MLP error levels with substantially fewer samples once the target error is below the early-data regime.Source: Figure 1 · p. 4, §4 · p. 4
  • About 40% fewer samples achieve similar performance with KANs in Gymnasium; the advantage peaks at about 41.6% at 2250 episodes and narrows to about 7.5% at the final checkpoint on average.Both architectures approach similar return levels after many training episodes.Source: Figure 2 · p. 4, §4 · p. 4

5 Discussion

KANs improve sample efficiency on tasks with smooth, low-dimensional structure, with the strongest gains early and midway through training. With many samples, final returns are often close across architectures.

5 Discussion

  • KANs improve sample efficiency when tasks contain smooth, low-dimensional structure.The gains are strongest early and midway through training, when simulator calls, measurements, or hardware trials are expensive.Source: §5 · p. 4
  • With many samples, final returns are often close across KAN and MLP architectures.The effect is not uniform across tasks.Source: §5 · p. 4
  • When inference cost is the bottleneck, the paper suggests training with KANs during scarce-sample learning and distilling the policy into a cheaper MLP.Source: §5 · p. 4

A.1 Feynman Benchmark Details

The Feynman experiments use a fixed set of sampled analytic equations, compare KANs and MLPs across training sizes and seeds, and control model-comparison settings to assess sample efficiency.

A.1.1 Dataset Construction

  • The unit-free Feynman subset contains 27 implemented equations with finite train and test samples, including univariate and multivariate cases.Fixed train pools and test sets are sampled per equation; invalid or non-finite samples are rejected before normalization for all models.Source: §A.1.1 · p. 8

A.1.2 Sample-Efficiency Protocol

  • For each equation, Ntest = 1000; experiments repeat over 20 random seeds for each training size, equation, and model family, with each seed determining the test draw.Inputs are standardized using the training set’s mean and standard deviation; the test set is not used for model selection or normalization.Source: §A.1.2 · p. 8

A.1.3 Model Comparisons

  • KAN and MLP regressors share optimizer, loss, update-step limit, batch size, and seeds, with parameter-matched MLPs as a secondary comparison.MLPs use ReLU and sigmoid activations; parameter counts and wall-clock training time are reported, and parameter ratios are given when exact matching is impossible.Source: §A.1.3 · p. 8

A.1.4 KAN Implementations

The appendix describes KAN backends, regression metrics and aggregate sample-efficiency summaries, then specifies the Gymnasium environments and model sweep.

A.1.4 KAN Implementations

  • The KAN implementation wrapper exposes EfficientKAN with B-spline edges, FastKAN with Gaussian RBF edges, and BSRBF-KAN with hybrid edges.Implementation choices are fixed across sample budgets, and reported results specify the backend.Source: §A.1.4 · p. 8

A.1.5 Metrics

  • RMSE and MAE retain each equation’s physical output scale, while NRMSE and relative ℓ2 error support cross-equation comparisons and R2 reports explained variance.Source: §A.1.5 · p. 9

A.1.6 Aggregate Reporting

  • Aggregate sample-efficiency summaries include NRMSE curves over training size, relative improvement and sample reduction against the best MLP curve, and error-sample-curve area.The curve uses a logarithmic training-size axis; the threshold Nτ is the smallest training size whose median NRMSE falls below target τ, or is reported as greater than 1,000 if none does.Source: §A.1.6 · p. 9

A.2.1 Benchmark and Agents

  • The Gymnasium benchmark uses five Classic Control environments with low-dimensional state vectors and supports discrete and continuous action spaces.Discrete policies are categorical, while continuous policies use diagonal Gaussian distributions with tanh squashing.Source: §A.2.1 · p. 9

A.2.2 Model Sweep

  • The agent sweep compares full KAN actor-critic systems using B-spline or Gaussian RBF KANs with ReLU or sigmoid MLP actor-critic systems.The KAN sweep includes hidden dimensions [2],,,,, and; the MLP sweep includes,, [64],, and [64, 64].Source: §A.2.2 · p. 10

A.2.3 PPO Training and Evaluation

The Gymnasium protocol trains agents with PPO and evaluates paired-seed learning curves. A noisy-reward variant perturbs training feedback while retaining clean deterministic evaluation.

A.2.3 PPO Training and Evaluation

  • Training collects complete episodes, estimates advantages with γ = 0.99 and λ = 0.95, and applies clipped PPO updates with Adam.Updates follow batches of five episodes; the result file contains ten paired seeds and learning curves up to 6,000 episodes.Source: §A.2.3 · p. 10
  • Evaluation uses deterministic actions across five episodes per checkpoint, selecting the maximum-logit action for discrete policies and mean action for continuous policies.Training seeds are paired across model families.Source: §A.2.3 · p. 10

A.3.1 Reward Perturbation

  • The noisy benchmark adds Gaussian noise to rewards during training only, then evaluates deterministically on clean rewards.The noise sweep uses α values from 0 to 10 in raw reward units; it tests robustness to corrupted training feedback, not stochastic evaluation.Source: §A.3.1 · p. 10

B Detailed Results

Three figures identify experiment designs for the Feynman benchmark, DeepRL on Gymnasium, and sample efficiency under reward noise.

B Detailed Results

  • Figure 3 presents the Feynman benchmark experiment design.Source: Figure 3 · p. 11
  • Figure 4 presents the DeepRL Gymnasium experiment design.Source: Figure 4 · p. 12
  • Figure 5 presents the sample-efficiency experiment for the noised Gymnasium benchmark.Source: Figure 5 · p. 12
Loading 2610.10627v1…