Source-linked AI summary
Evaluating the Effectiveness of SechKAN on 1D Data
Hoang-Thang Ta
TL;DR
The effectiveness of SechKAN on 1D classification had not been evaluated across diverse datasets and architectural settings. The paper evaluates this sech-basis KAN with a 1D projection against several model families on three datasets. SechKAN is competitive across them and achieves the highest test accuracy on Crop, while its effectiveness varies with dataset characteristics.
Problem
SechKAN’s performance on 1D classification remained unexplored, particularly across diverse datasets and architectural settings.
Source: §1 · p. 2Method
The study evaluates SechKAN, which combines hyperbolic secant basis functions with a 1D projection, against several model families on three 1D classification datasets.
Source: §1 · p. 2, §3.2 · p. 4Results
SechKAN achieves competitive performance across the three datasets and the highest test accuracy on Crop among the evaluated models.
Source: §4.2 · pp. 7–9, §6 · p. 11Takeaways & Limitations
SechKAN’s effectiveness varies with dataset characteristics, and the reported results cover only the evaluated 1D classification setting.
Source: §5 · p. 10, §6 · p. 11The study covers only three 1D classification datasets, a limited model set, and no extensive search of SechKAN hyperparameters.
Source: §5 · p. 10
Abstract
from arXiv · showhide
The connection between the Kolmogorov-Arnold representation theorem (KART) and neural network design has led to the development of Kolmogorov-Arnold Networks (KANs), with applications ranging from STEM problems to AI tasks. In this paper, we investigate the effectiveness of a KAN variant, SechKAN, which relies on hyperbolic secant (sech) functions as basis functions, with a 1D projection to reduce the number of parameters to a level comparable to MLPs. We evaluate SechKAN on three 1D classification datasets: UCI Human Activity Recognition (UCI HAR), ElectricDevices, and Crop, and compare it with several effective networks, including EfficientKAN, MLP, CNN1D, ResNet1D, and DSCNN1D, using approximately comparable parameter budgets. The results indicate that SechKAN achieves competitive performance across the three datasets, with particularly strong performance on Crop. Ablation studies further show that grid size and normalization affect performance, suggesting that SechKAN's effectiveness depends on the dataset and architectural choices. Our source code and experimental implementation are publicly available at: https://github.com/hoangthangta/SechKAN_1D.
1 Introduction
KANs offer an alternative to MLPs, whose linear transformations may not adapt well to different data types. This paper evaluates SechKAN on 1D classification, where its performance had not yet been explored, across three datasets and several model families.
1 Introduction
- KANs provide an alternative to MLPs, whose linear transformations may not adapt well to different data types.KANs have been applied to problems involving functions and relationships in mathematics, physics, and other fields.Source: §1 · p. 1
- SechKAN uses hyperbolic secant functions as basis functions and a 1D projection to make layer parameter counts comparable to MLP layers.This design follows feature extraction using sech functions.Source: §1 · pp. 1–2
- SechKAN’s performance on 1D data remained unexplored despite its design for one-dimensional inputs.Earlier evaluations covered function approximation, PDE surrogate modeling, and image classification.Source: §1 · p. 2
- The study evaluates SechKAN on UCI HAR, ElectricDevices, and Crop against KAN-, fully connected, and convolutional models.EfficientKAN is included as a representative KAN baseline, while CNN-based models are selected for their local-pattern extraction capabilities.Source: §1 · p. 2
- The evaluation also tests how grid size and data normalization affect performance.Grid size is the number of basis functions.Source: §1 · p. 2
2 Related Work
Related work describes KANs as a KART-based alternative to MLPs, with applications across tasks and a range of basis functions. Other research addresses KAN parameter costs, while CNN variants provide established 1D classification baselines.
2 Related Work
- KANs use learnable univariate functions instead of fixed activation functions and are based on KART.KART had also inspired spline-based neural architectures [6].Source: §2 · p. 2
- KANs have been applied to differential equation solving, time series forecasting [8], and computer vision.Alternative basis functions studied include modified B-splines, polynomials [22], radial basis functions [16], Fourier bases [27], wavelets [4], rational and fractional Jacobi functions, and customized activations.Source: §2 · p. 2
- Parameter-reduction approaches include Group KAN, which uses weight sharing to reduce parameters and computational costs, and PRKAN, which targets model sizes comparable to MLPs [25].Other approaches include GS-KAN, which generates edge functions from a shared learnable parent function [7], and LeanKAN, a compact alternative to AddKAN and MultKAN.Source: §2 · pp. 2–3
- SechKAN uses hyperbolic secant basis functions and a 1D projection as a parameter-reduction approach.It is described alongside other parameter-efficient KAN variants.Source: §2 · p. 3
- CNN1D extracts local features with one-dimensional convolution [11], ResNet1D adds residual connections, and DSCNN1D uses depthwise separable convolutions to reduce computational cost and parameters [18].These architectures serve as established convolutional baselines for 1D classification.Source: §2 · p. 3
3 Methodology
KART motivates representing multivariate functions through one-dimensional functions, which KANs implement as learnable edge functions. SechKAN uses sech basis responses and a grid projection, with normalization and parallel projections in its layer design.
3.1 KART and KAN
- KART states that continuous multivariate functions on a bounded domain can be represented using one-dimensional functions, providing the theoretical basis for KANs.KANs apply this decomposition through learnable one-dimensional edge functions rather than the fixed activations and linear weights of MLPs.Source: §3.1 · p. 3
- Each KAN layer uses a function matrix of learnable functions connecting its input nodes to output nodes.The original KAN constructs each learnable function from a base function and a spline function; its base function is SiLU, and its spline uses learnable coefficients and B-spline basis functions.Source: §3.1 · pp. 3–4
3.2 SechKAN
- A SechKAN layer applies sech basis functions, grid projection, optional normalization, feature projection, and a parallel base-projection branch whose output is added to the main output.If the skip projection branch is disabled, the layer returns the main output directly.Source: §3.2 · p. 4, Figure 1 · p. 5
- Sech functions are smooth, bounded, bell-shaped, exponentially decaying, and infinitely differentiable, supporting smooth gradient propagation and localized representations.Saturation can lead to small gradients for inputs far from the center.Source: §3.2 · p. 4
- A SechKAN layer flattens its input to shape (B, din) and produces an output of shape (B, dout).For image inputs, the data dimension is D = C × W × H.Source: §3.2 · p. 5
- SechKAN maps each input feature to G sech-basis responses, aggregates them by grid projection, and uses learnable centers initialized uniformly across the specified interval.The centers are jointly optimized during training, while G remains fixed.Source: §3.2 · p. 5
- SechKAN uses a learnable width parameter shared by all sech basis functions within a layer.The width is parameterized through an unconstrained variable, and the parameterization ensures positive width and improves numerical stability during optimization.Source: §3.2 · p. 6
4 Experiments
The experiments compare SechKAN with KAN, MLP, and convolutional baselines on three 1D classification datasets. SechKAN is competitive overall, while ablations show that grid size and normalization choices affect its performance.
4.1 Datasets and Training Configurations
- The evaluation covers UCI HAR, ElectricDevices, and Crop, with approximately comparable parameter budgets and main-experiment results averaged over five runs.Ablation results are averaged over three runs.Source: §4.1 · pp. 6–7
4.2 Results
- SechKAN is competitive across all three datasets: close to the best KAN and comparable to CNN1D on UCI HAR, mixed on ElectricDevices, and highest on Crop.On ElectricDevices, ResNet1D has the highest test accuracy; SechKAN outperforms CNN1D and MLP and is comparable to DSCNN1D.Source: §4.2 · pp. 7–9, Table 2 · p. 8
- SechKAN combines high accuracy with relatively low training time on UCI HAR and Crop, but its accuracy–time advantage varies by dataset.On Crop it has the highest test accuracy with relatively low training time; deeper convolutional models show substantially higher training times in some cases.Source: Figure 2 · p. 8, §4.2 · p. 9
4.3 Ablation studies
- The best grid size varies by dataset—16 for Crop and 4 for ElectricDevices and UCI HAR—while BatchNorm at Norm1 gives the highest normalized accuracy on all three.Increasing grid size does not always improve performance; no normalization generally gives the lowest performance, and Norm2 results are less consistent.Source: Figure 3 · p. 9, §4.3 · p. 10, Figure 4 · p. 10
- The ablations cover only a limited set of configurations, so broader experiments on SechKAN hyperparameters and architectural choices are needed.The ablations were conducted after the main comparison and were not used to select the reported configurations.Source: §4.3 · pp. 9–10
5 Limitations
The study's evidence is limited in dataset, task, hyperparameter, and baseline coverage, constraining how broadly its results can be generalized.
5 Limitations
- Evaluation on only three 1D classification datasets does not establish generalization to other domains or to 2D and 3D data.The datasets vary in input dimensions, sample sizes, and class counts but do not span a wide range of domains.Source: §5 · p. 10
- The study evaluates classification only, leaving regression and forecasting untested.Source: §5 · p. 10
- The limited hyperparameter search and baseline set leave selected settings' optimality and broader effectiveness uncertain.The comparison includes one MLP, one KAN baseline, and three CNN-based models.Source: §5 · p. 10
6 Conclusion
SechKAN achieves competitive performance across three 1D classification datasets, with the highest test accuracy on Crop and relatively low training time. Results vary by dataset, and ablations show that grid size and normalization configuration affect performance.
6 Conclusion
- SechKAN is competitive across three 1D classification datasets, achieves the highest test accuracy on Crop, and maintains relatively low training time.Its performance varies across datasets, so effectiveness depends on task characteristics.Source: §6 · p. 11
- Grid size and normalization configuration affect SechKAN performance, and the best configurations vary across datasets.Source: §6 · p. 11
- The evaluation covers only three 1D classification datasets and a limited set of baselines; future work will broaden datasets, tasks, and architectural configurations.Proposed extensions include regression and forecasting.Source: §6 · p. 11