KanAdapter: A Kolmogorov-Arnold Network-based Plug-and-Play Module for Efficient Fine-tuning of Foundation Speech Models

• Speech SSL models achieve strong performance across speaker verification, speech emotion recognition, and deepfake detection after pre-training on massive unlabeled data.Source: §1 · p. 1
• Fully fine-tuning large-scale speech SSL models is computationally prohibitive for practical deployment.Source: §1 · p. 1
• PEFT methods inject small trainable modules into frozen backbones instead of updating all parameters.Source: §1 · p. 1
• MLP-based adapters use fixed activation functions that limit function-approximation capacity relative to more flexible alternatives.Source: §1 · p. 1
• KANs replace fixed MLP activations with learnable univariate mappings on network edges, offering more flexible approximation and improved resistance to catastrophic forgetting.Source: §1 · p. 1
• KanAdapter integrates GR-KAN modules as plug-in adapters within frozen SSL backbones for expressive, parameter-efficient adaptation.Source: §1 · p. 1

• KanAdapter is presented as the first work exploring KAN-based networks for efficient fine-tuning of speech foundation models.Source: §1 · p. 2
• Experiments on speaker verification, speech emotion recognition, and deepfake detection show KanAdapter outperforming existing fine-tuning approaches with significantly fewer trainable parameters.Source: §1 · p. 2
• Continual learning evaluations show substantially stronger resistance to catastrophic forgetting than full fine-tuning and existing adapter-based methods.Source: §1 · p. 2

KAN foundations

• KANs use learnable edge-wise activations and have been reported to offer superior continual learning relative to MLPs through localized basis functions.Source: §2 · p. 2

KANs in Computer Vision

• Vision studies apply KANs to classification heads, adapters, and Transformer feed-forward replacements across image and video benchmarks.Source: §2 · p. 2

KANs in Speech Processing

• Speech studies use GR-KAN projectors, KAN-integrated Conformers, and systematic KAN analyses for speech deepfake detection and spoken language understanding.Source: §2 · p. 2

PEFT

• PEFT includes bottleneck adapters, low-rank weight updates through LoRA, and prompt tuning, while its application to speech SSL models remains underdeveloped.Source: §2 · p. 2

Kolmogorov-Arnold Networks

• KANs construct multivariate mappings from compositions of learnable univariate functions, providing the theoretical basis for edge-wise nonlinear transformations.The representation theorem states that continuous multivariate functions can be represented by finite compositions of univariate continuous functions.Source: §3.1 · pp. 2–3
• KAN layers place learnable univariate functions on network edges instead of fixed activations at conventional MLP nodes.A full KAN stacks such layers, with each layer represented by a matrix of learnable univariate functions.Source: §3.2 · p. 3
• Each KAN edge can use a SiLU pathway combined with a learnable B-spline, enabling flexible, data-driven nonlinearities unlike fixed shared MLP activations.The B-spline coefficients are trainable, while the SiLU component provides a smooth residual pathway.Source: §3.2 · p. 3

Group-Rational KAN

• Vanilla KANs face GPU inefficiency, parameter overhead, and unstable default initialization when applied to large-scale models.These challenges arise from recursive B-splines, unique edge activations, and variance-violating initialization.Source: §3.3 · p. 3
• GR-KAN replaces B-splines with rational functions, requiring 21 FLOPs per evaluation versus 204 FLOPs for an equivalent B-spline at m=5 and n=4.Horner evaluation maps rational functions to parallel hardware operations.Source: §3.3 · p. 3
• Grouped parameter sharing reduces unique activation functions from din × dout to g, keeping GR-KAN near standard-MLP parameter complexity with constant overhead.Rational coefficients are shared within input-channel groups while each edge retains its own scalar weight.Source: §3.3 · pp. 3–4
• GR-KAN initializes rational coefficients to approximate known activations and scales weights using estimated variance, supporting stable gradient flow.The scheme fits identity or Swish-like functions before initializing w ∼N(0, α/din).Source: §3.3 · p. 4
• Together, rational functions, grouped sharing, and variance-preserving initialization make GR-KAN an efficient, stable, expressive alternative to vanilla KAN and MLP.These properties motivate using GR-KAN as KanAdapter’s core component.Source: §3.3 · pp. 3–4

Architecture

• KanAdapter is a lightweight plug-and-play module for parameter-efficient fine-tuning of speech SSL models.It is designed as an additional trainable branch within Transformer encoder blocks.Source: §4.1 · p. 4, §4 · p. 4
• The parallel bottleneck design preserves each Transformer block’s frozen MLP branch while adding a trainable KanAdapter branch.This follows the structural idea of AdaptFormer while retaining the original backbone pathway.Source: §4.1 · p. 4
• KanAdapter uses down- and up-projections around a GR-KAN transformation instead of a standard MLP intermediate nonlinearity.The GR-KAN layer supplies learnable rational activations within the low-rank bottleneck.Source: §4.1 · p. 4

Fine-tuning

• During fine-tuning, only the bottleneck projections, GR-KAN coefficients, and scaling factor are updated while the SSL backbone remains entirely frozen.Task-specific adaptation therefore occurs exclusively through lightweight KanAdapter branches.Source: §4.1 · p. 4

Speaker Verification

• KanAdapter achieves competitive speaker verification with 9M trainable parameters, reducing parameters by 97.5% while closely approaching full fine-tuning.It attains EERs of 0.52%, 0.50%, and 1.93% on Vox1-O, Vox1-E, and Vox1-H, versus full fine-tuning at 0.49%, 0.44%, and 1.58%.Source: §5.3.1 · p. 5

Speech Emotion Recognition

• KanAdapter delivers the strongest parameter-efficient speech emotion recognition results, using 16M parameters and reducing trainable parameters by 95.2% versus full fine-tuning.On the development set it achieves F1-Macro 0.5127 and F1-Micro 0.3436; on the test set, F1-Macro 0.3290 and F1-Micro 0.3180.Source: §5.3.2 · p. 6

Deepfake Detection

• KanAdapter reduces deepfake-detection trainable parameters from 319M to 17M, a 94.7% reduction, while remaining closest to full fine-tuning across every evaluation set.It achieves 0.34% EER on LA19 and outperforms LoRA and AdaptFormer on cross-corpus LA21 and DF21.Source: §5.3.3 · p. 6
• On the ITW out-of-domain benchmark, KanAdapter achieves 7.96% EER, outperforming LoRA, AdaptFormer, and full fine-tuning despite using a fraction of the parameters.The reported EERs are 10.94% for LoRA, 10.21% for AdaptFormer, and 8.34% for full fine-tuning.Source: §5.3.3 · p. 6

Continual Learning

• KanAdapter achieves the best continual-learning performance across all four evaluation sets and reduces LA19 EER by 83.6% over full fine-tuning and 82.6% over AdaptFormer.The paper attributes this retention to localized rational basis functions whose updates affect specific input regions, reducing interference with previously acquired knowledge.Source: §5.3.4 · pp. 6–7

6.1 Effect of Bottleneck Dimension ˆd

• KanAdapter’s bottleneck dimension controls trainable parameter count and trades adapter capacity against regularization.The study varies ˆd across 512, 256, 128, and 64 on deepfake detection.Source: §6.1 · p. 7
• 22.18% to 18.40%: LA5 EER drops as ˆd decreases from 512 to 128.The improvement is consistent across all evaluation sets and supports ˆd = 128 as the default.Source: §6.1 · p. 7
• At ˆd = 64, performance slightly degrades because the excessively small bottleneck restricts representational capacity.The results indicate a capacity limit beyond the best-performing bottleneck size.Source: §6.1 · p. 7

6.2 Generalizability Across SSL Backbones

• Across WavLM Large, UniSpeech-SAT, and mHuBERT-147, KanAdapter reduces trainable parameters by 94.9%, 91.0%, and 91.1%, respectively, while approaching full fine-tuning performance.The evaluation reports EER on LA21 and DF21 across three additional SSL backbones.Source: §6.2 · pp. 7–8
• On WavLM Large, KanAdapter reaches 4.19% and 11.76% EER on LA21 and DF21, within 0.12% and 0.51% of full fine-tuning.WavLM Large has 24 Transformer encoder layers and 333M parameters.Source: §6.2 · p. 8
• Performance gaps widen on the 12-layer UniSpeech-SAT and mHuBERT-147 models, although KanAdapter remains viable at a fraction of the parameter budget.The ablation therefore identifies large, deep SSL backbones as the most favorable setting.Source: §6.2 · p. 8

Conclusion

• KanAdapter is a lightweight, plug-and-play parameter-efficient fine-tuning framework built by replacing MLP adapter bottlenecks with GR-KAN modules.The design targets expressive task-specific adaptation while preserving deployment efficiency.Source: §7 · p. 8
• Across speaker verification, speech emotion recognition, and deepfake detection, KanAdapter reduces trainable parameters by up to 97.5% while remaining highly competitive with full fine-tuning.It also substantially outperforms AdaptFormer under comparable parameter budgets.Source: §7 · p. 8
• Continual-learning evaluations report EER reductions of up to 83.6% on previously learned tasks, attributed to localized grouped rational activations.The reported result concerns resistance to catastrophic forgetting.Source: §7 · p. 8

Future Work

• The study leaves applications to image, video, and large language models for future work.These extensions are motivated by KanAdapter’s plug-and-play nature and the continual-learning properties of KAN-based architectures.Source: §7 · p. 8

Limitations

• The improved expressiveness of GR-KAN over MLP adapters is demonstrated through downstream performance rather than targeted representation-level analyses.The paper does not isolate the contribution of learnable rational activations through spectral or frequency-band studies.Source: Limitations · p. 9
• Benefits are most pronounced on large, deep SSL backbones, while gaps to full fine-tuning widen for shallower models.UniSpeech-SAT and mHuBERT-147 are given as examples.Source: Limitations · p. 9
• Evaluation focuses on speech classification tasks, leaving sequence generation, other modalities, and large language models unverified.The paper specifically identifies ASR and broader modalities as unsupported evaluation areas.Source: Limitations · p. 9
• Continual learning is evaluated only in a two-stage ASVspoof2019-to-ASVspoof5 setting, so longer task sequences and broader benchmarks remain needed.The limitation concerns fully characterizing resistance to catastrophic forgetting.Source: Limitations · p. 9