PaperlayerView exact revision on arXiv

Source-linked AI summary

KAN-Robust-Bench: A Benchmark for Evaluating the Robustness of Kolmogorov-Arnold Networks

Mohammad Meymani, Roozbeh Razavi-Far

arXiv:2608.21488v1cs.LGcs.AI

TL;DR

Adversarial evasion attacks can fool machine-learning models with subtle perturbations, motivating stronger evidence about KAN robustness. The paper evaluates defended and undefended KAN architectures using certified robustness analysis and strong white-box attacks, finding adversarial training most effective against gradient-based attacks while randomized smoothing is competitive against ℓ2-norm C&W attacks. The study also finds that architecture choices affect certified and empirical robustness.

  • Problem

    The paper addresses limited evidence about how different KAN architectures and defenses withstand adversarial evasion attacks, which can fool models with subtle perturbations.

  • Method

    The study evaluates KAN models using randomized smoothing and interval bound propagation for certification, plus FGSM, PGD, and C&W attacks across datasets and defenses.

  • Results

    Adversarial training was most effective against gradient-based attacks, while randomized smoothing showed competitive performance against ℓ2-norm C&W attacks.

  • Takeaways & Limitations

    Robustness varies with both defense strategy and KAN architecture, so architecture choice is relevant to certified and empirical robustness.

  • Takeaways & Limitations

    Future work is needed to study certified robustness with additional techniques and more diverse KAN architectures and defenses.

Abstract

from arXiv · show

While machine learning models have demonstrated strong performance in many domains, these models have shown profound vulnerabilities when they are exposed to adversarial threats. While adversarial attacks fall into various categories, the most prominent category in research studies is evasion. In evasion attacks, the adversary generates perturbed versions of samples, which might not be observable by human eyes. These samples generally fool the machine learning models with high confidence. This phenomenon poses a significant security violation against machine learning models. In this paper, we investigate the certified and empirical robustness of various Kolmogorov-Arnold network architectures against strong evasion attacks. At first, we provide the mathematical foundations for randomized smoothing and interval bound propagation, and report the $\ell_2$-certified robustness of the models under randomized smoothing. After that, we systematically evaluate the robustness of various defended and undefended KAN models under FGSM, PGD, and C&W attacks in order to find out the optimal defense strategies and architectures.

1. Introduction

The paper examines robustness within KAN architectures against white-box evasion attacks, combining empirical evaluation with certified robustness analysis across defenses, datasets, and architectures.

  • KAN foundations: KANs use compositions of continuous univariate functions and learnable Splines as alternatives to MLP models.KANs can outperform MLPs on some tasks with fewer parameters, but generally train more slowly.
  • Adversarial robustness: Adversarial attacks exploit ML vulnerabilities, while defenses aim to secure models throughout the ML lifecycle.The paper focuses on evasion attacks, which create subtle perturbations that can fool models with high confidence.
  • Study scope: The study evaluates state-of-the-art KAN computer-vision architectures against strong white-box evasion attacks.The evaluation targets vulnerabilities during inference, when attackers perturb inputs without changing training data or model parameters.
  • Study scope: Adversarial training, randomized smoothing, and interval bound propagation are compared across KAN architectures on CIFAR-10 and SVHN.The study reports both empirical and certified robustness, including certified accuracy and certified radius under randomized smoothing.
  • Method overview: The paper develops mathematical foundations for randomized smoothing and interval bound propagation before presenting certified and empirical results.The workflow selects a dataset, architecture, and training strategy, then evaluates models against diverse adversarial attacks.

2. Related Works

Prior work has examined KAN robustness against noise and adversarial attacks, often through comparisons with MLPs or CNNs. This paper instead studies robustness variation within the KAN family across architectures, datasets, attacks, and defenses.

  • KAN architectures: KAN-Mixers replace MLP-Mixer layers with KAN layers while retaining token- and channel-mixing for spatial and feature-level relationships.KAN-Mixers include patch embedding, mixer blocks, adaptive average pooling, and a classification head.
  • KAN architectures: KANICE combines interactive convolution blocks, convolutional layers, normalization, pooling, and KAN linear layers in a hybrid architecture.Its interactive convolution blocks extract spatial features before later convolutional and KAN processing.
  • KAN architectures: PoolKANNeXt combines pooling-based feature mixing, dual-activation KAN blocks, and a ConvNeXt-style hierarchical architecture.Its stages include stem, main, downsampling, and output phases.
  • Prior robustness findings: Earlier studies reported mixed robustness findings for KANs relative to MLPs, including greater robustness in some settings and greater vulnerability in others.Results varied across datasets, attacks, and model variants.
  • Prior robustness findings: Prior evaluations also found that robustness depends on architecture and that randomized smoothing and adversarial training can improve robustness.Some studies reported convolutional KANs outperforming comparable CNNs, while adversarially trained KANs outperformed a certified approach in another comparison.
  • Research gap: This study addresses the within-family gap by evaluating KAN-Mixers, KANICE, and PoolKANNeXt under common adversarial conditions.It considers CIFAR-10 and SVHN, multiple defense strategies, and a wide range of attacks.

3. Problem Settings

The problem setting defines white-box gradient-based and optimization-based attacks alongside defenses that modify training or smooth and certify model predictions. The study evaluates how these mechanisms affect robustness and accuracy.

  • Attacks: FGSM is a one-step white-box attack that generates an adversarial sample by moving toward the loss gradient.The perturbation depends on the model parameters, input, label, loss gradient, and perturbation size.
  • Attacks: PGD is a multi-step attack that iteratively refines a random perturbation to cause misclassification.Projection keeps the adversarial input within predefined perturbation bounds.
  • Attacks: C&W minimizes a distance term plus an attack objective while constraining the adversarial sample to [0, 1]^n.The objective balances perturbation size against attack success through λ.
  • Defenses: Adversarial training learns from benign and adversarial samples, but commonly involves a robustness-accuracy trade-off.It can augment training data with adversarial samples or define an objective that learns both representations.
  • Defenses: Randomized smoothing adds Gaussian noise to inputs and aggregates predictions from multiple noisy samples using Monte Carlo voting.This produces a smoothed model for robustness evaluation and certification.
  • Defenses: IBP propagates input perturbation bounds through the network and combines clean and IBP losses during training.The coefficient λ controls the trade-off between robustness and accuracy.

4. Methodology

The methodology evaluates KAN models under diverse white-box evasion attacks using adversarial training, randomized smoothing, and interval bound propagation. It combines empirical attack testing with certified robustness analysis based on randomized smoothing and interval bounds.

  • Experimental process: The experimental process selects a dataset, KAN architecture, and training strategy before evaluating models against diverse adversarial attacks.The threat model assumes full attacker knowledge and restricts perturbations to test-time, norm-bounded regions.
  • Certified defenses: Randomized smoothing injects Gaussian noise into inputs and certifies unchanged predictions within an ℓ2 radius determined by the top two class probabilities.The smoothed classifier selects the class with the highest probability under noisy inputs; Monte Carlo sampling estimates class probabilities during inference.
  • Certified defenses: Interval Bound Propagation certifies robustness by propagating perturbation intervals through linear transformations and learnable spline functions across KAN layers.The resulting output bounds certify robustness when the true-class lower bound exceeds the upper bounds of competing classes, under ℓ∞ perturbations.
  • Defense mechanisms: Adversarial training mixes benign and PGD-generated examples, using an 8/255 perturbation bound, 2/255 step size, and 10 iterations.The mixed batch trains the model on both benign and adversarial distributions.
  • Defense mechanisms: IBP-based training ramps perturbation size and the clean-to-robust loss trade-off across warm-up, ramp-up, and robust-training intervals.The ramp-up reaches λ = 0.5 and ε = 4/255 before training uses the maximum predefined values.
  • Threat model and attacks: Empirical robustness is tested with ℓ∞ FGSM, standard and enhanced PGD variants, and ℓ2 C&W attacks under specified perturbation and iteration settings.PGD uses an 8/255 bound and 2/255 step size; multi-step PGD uses three restarts, while adaptive PGD uses three EoT samples.

5. Experimental Setups and Results

The experiments compare three KAN vision architectures and several defenses across certified robustness and strong empirical attacks on CIFAR-10 and SVHN. Adversarial training generally provides the strongest empirical protection, while randomized smoothing is especially competitive against C&W attacks.

  • Experimental setup: The study evaluates KAN-Mixers, KANICE, and PoolKANNeXt on CIFAR-10 and SVHN, averaging empirical results over five runs.The architectures were selected for state-of-the-art computer-vision performance, enabling comparison of accuracy and robustness.
  • Certified robustness: PoolKANNeXt shows the strongest certified robustness on CIFAR-10, while PoolKANNeXt and KANICE are competitive on SVHN.Certified accuracy measures samples meeting a radius threshold, while mean radius measures average certified radius.
  • FGSM results: 86.65% SA is achieved by KANICE-IBP on CIFAR-10, while PoolKANNeXt-RS reaches 86.37%, showing certified defenses need not reduce clean accuracy.These are the highest reported CIFAR-10 standard-accuracy values in the cited comparison.
  • FGSM results: 45.56% RA at ϵ = 0.09 is maintained by PoolKANNeXt-AT, compared with 7.54% for its plain counterpart under FGSM.KANICE-AT also retains 56.22% RA at the strongest perturbation, exceeding KANICE-RS at 35.41% and KANICE-IBP at 31.85%.
  • FGSM results: 67.76% RA at ϵ = 0.09 is maintained by KAN-Mixers-AT on SVHN alongside 95.74% SA, outperforming the other defense mechanisms.SVHN generally has higher absolute robustness than CIFAR-10, although robust accuracy declines as perturbation size increases.
  • PGD and C&W results: 54.6% RA is maintained by PoolKANNeXt-AT across CIFAR-10 PGD configurations, while corresponding undefended models achieve less than 8% RA.Adversarial training consistently provides the strongest protection against gradient-based attacks, whereas randomized smoothing and IBP offer more limited empirical gains under FGSM and PGD.
  • PGD and C&W results: 72.58% robustness is achieved by KAN-Mixers-RS on SVHN under C&W, slightly exceeding the plain and adversarially trained variants.On CIFAR-10, PoolKANNeXt-AT reaches the highest C&W robustness at 31.17%, but several RS and plain configurations are comparable.
  • Cross-architecture comparison: Robustness remains architecture-dependent, with defense effectiveness varying among KAN-Mixers, KANICE, and PoolKANNeXt.Aggregated distributions show adversarial training generally separated from other defenses, randomized smoothing often second strongest, and IBP more moderate, particularly on SVHN.

6. Conclusion

The evaluation finds adversarial training most effective against gradient-based attacks, while randomized smoothing is competitive against ℓ2-norm C&W attacks. Architecture choices affect both certified and empirical robustness, motivating broader certification and architecture studies.

  • Adversarial training proved the most effective defense strategy against gradient-based attacks, while randomized smoothing performed competitively against ℓ2-norm C&W attacks.
  • KAN architecture choices directly affect both certified and empirical robustness.
  • Future work will examine additional certification techniques, more diverse KAN architectures, and broader defense sets to investigate model strengths and limitations.
Loading 2608.21488v1…