PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Dual-Model Collaborative Random Vector Functional Link Network

A. Quadir, A. Rahaman, Mushir Akhtar, M. Tanveer

arXiv:2608.13628v1cs.LG

TL;DR

Conventional RVFL networks remain inadequate for heavily noisy, corrupted, or imbalanced classification data. KRPRVFL combines RVFL efficiency with KRP-based robust learning and collaboration, achieving superior and statistically significant benchmark performance.

  • Problem

    Conventional RVFL variants remain inadequate for classification tasks involving heavily contaminated, noisy, corrupted, or imbalanced data.

  • Method

    KRPRVFL replaces least squares with a KRP-based loss and adds collaborative learning and kernel mapping to suppress unreliable samples and model nonlinear relationships.

  • Results

    85.20% average accuracy versus 81.55% for RVFL across 37 UCI and KEEL datasets, with statistically significant improvements over all baselines.

  • Takeaways & Limitations

    KRPRVFL is an efficient, scalable, and robust classification model for noisy, corrupted, and imbalanced data, particularly on small to medium datasets.

  • Takeaways & Limitations

    KRPRVFL performance depends on kernel and risk-sensitive parameters and is best suited for small to medium datasets.

Abstract

from arXiv · show

Random vector functional link (RVFL) networks are lightweight and fast neural models that offer efficient training and strong generalization through randomized hidden-layer weights and direct input-output connections. However, conventional RVFL models are sensitive to noisy labels, outliers, and imbalanced data, which limits their performance in real-world applications. To address these challenges, we propose the kernel risk-sensitive mean p-power based RVFL (KRPRVFL) model, which integrates the computational efficiency of RVFL with the robustness of the kernel risk-sensitive mean p-power (KRP) criterion. By replacing the standard least-squares objective with a KRP-based loss, KRPRVFL adaptively reduces the influence of corrupted or unreliable samples during training, resulting in improved stability and generalization. Additionally, a collaborative learning mechanism is introduced to enable adaptive interaction among model components, further enhancing robustness in complex and noisy environments. The proposed framework also leverages kernel-induced feature mapping to capture nonlinear relationships without requiring explicit hidden-layer selection, maintaining both efficiency and scalability. Extensive experiments on UCI and KEEL benchmark datasets demonstrate that KRPRVFL consistently outperforms baseline models in terms of accuracy, robustness, and statistical significance, highlighting its effectiveness as a fast, scalable, and reliable solution for challenging classification tasks.

I. INTRODUCTION

RVFL networks provide a lightweight, rapidly trained alternative to computationally expensive deep models, but conventional RVFL remains vulnerable to noisy, imbalanced, and corrupted data. KRPRVFL addresses these limitations by replacing least squares with a risk-sensitive KRP loss and combining RVFL efficiency with robust kernel-based learning.

  • RVFL motivation: RVFL uses randomly generated fixed hidden-layer weights and direct input-output connections to enable lightweight, rapid learning with few tunable parameters.The direct connections provide an implicit form of regularization.
  • Limitations: Conventional RVFL treats training samples uniformly, making it vulnerable to noise and outliers, while existing variants remain inadequate for heavily contaminated or noisy classification data.These limitations motivate a more robust RVFL formulation for real-world applications.
  • Robust criterion: The KRP criterion combines kernel-based nonlinear mapping with risk sensitivity to improve resilience to noise, outliers, and data imbalance.It is effective in reproducing kernel Hilbert spaces for robust learning.
  • Proposed method: KRPRVFL replaces the standard least-squares objective with a risk-sensitive KRP-based loss that adaptively suppresses outliers and mislabeled samples.The model integrates RVFL efficiency with the robustness of the kernel risk-sensitive mean p-power criterion.
  • Experimental evidence: KRPRVFL consistently outperforms baseline models in accuracy and statistical significance on UCI and KEEL benchmark datasets.The reported experiments support its effectiveness for noisy and challenging classification tasks.

II. RELATED WORK

The related-work section establishes the paper’s notation and introduces the RVFL model.

  • The section first defines the notations used throughout the paper and then overviews the RVFL model.

A. Notations

The notation defines the binary classification training dataset, its feature and label dimensions, and the corresponding sample matrices. It also specifies the transpose operator used throughout the formulation.

  • Dataset and dimensions: The training set is X = {(x_i, y_i) | i = 1, 2, ..., n}, with x_i ∈ R^1×m and y_i ∈ {+1, −1}.Here, n denotes the total number of training samples and m the number of features.
  • Operators: The transpose operator is denoted by (·)^T.
  • Sample matrices: The input and output sample matrices are X = [x_1^T, x_2^T, ..., x_n^T]^T and Y = [y_1^T, y_2^T, ..., y_n^T]^T, respectively.

B. Random Vector Functional Link (RVFL) Network

RVFL is a single-layer feedforward network that combines randomly initialized hidden-layer transformations with direct input-to-output shortcut connections. Its output weights are trained through regularized least squares, with a closed-form solution selected according to the feature and sample dimensions.

  • Architecture: RVFL uses random, fixed input-to-hidden weights and shortcut connections from inputs to outputs to enhance generalization.The hidden-layer output matrix H1 contains outputs from N hidden nodes.
  • Architecture: The hidden-layer matrix H1 is generated from randomly initialized weights W1, bias matrix b1, and activation function ϕ.W1 ∈ Rm×N and b1 ∈ Rn×N.
  • Output formulation: RVFL concatenates the input matrix X with hidden-layer outputs H1 to form H2, which is used to predict output ˆY through output weights β.The combined feature matrix is defined as H2 = [X; H1], with β ∈ R(m+N)×1.
  • Training: Training solves a regularized least-squares optimization problem, where C > 0 controls regularization and the closed-form β solution depends on H2’s dimensions relative to sample count n.The formulation also uses an identity matrix I of appropriate size.

III. THE PROPOSED ROBUST DUAL-MODEL COLLABORATIVE RANDOM VECTOR FUNCTIONAL LINK NETWORK

The proposed KRPRVFL replaces least-squares output-weight learning with a risk-sensitive KRP criterion in an RKHS, suppressing mislabeled and outlier samples. Regularization and sample-adaptive weighting yield an iterative, complexity-aware optimization procedure for robust RVFL learning.

  • Robust KRP objective: KRPRVFL replaces the conventional least-squares objective with the KRP criterion to suppress mislabeled samples and improve robustness under label noise.Conventional RVFL output weights are highly sensitive to corrupted labels, whereas KRP-based learning reduces unreliable-sample influence.
  • Kernel-induced representation: The KRP metric operates in an RKHS, where kernel-induced feature mapping captures dependencies and reduces outlier impact through risk-sensitive deviation measurement.The mapping is implicitly defined by a Mercer kernel, and the framework uses a Gaussian Mercer kernel with bandwidth σ > 0.
  • Kernel risk-sensitive learning: The learning task minimizes empirical KRP loss between true labels and RVFL outputs, with predictions formed from hidden-layer features and output weight matrix W.The model combines the RVFL hidden-layer feature matrix with the output weights to produce sample predictions.
  • Regularized optimization: Regularization controls model complexity, while a diagonal weighting matrix assigns each training sample an influence determined by its contribution to the overall loss.The sample-adaptive weights support robust optimization by reflecting relative sample influence during training.
  • Iterative solution: The output weights are updated iteratively through a fixed-point mapping, using alternative matrix formulations selected according to sample size and feature dimension until convergence.The preferred formulation uses (17) when n > L + N and ((18)) when n < L + N; iterations stop when ∥Wt − Wt−1∥2 < τ or the maximum iteration count is reached.

IV. EXPERIMENTAL RESULTS

The experimental evaluation validates KRPRVFL on widely used UCI and KEEL benchmark datasets through comparative studies with representative learning models, including experiments with added label noise.

  • IV. EXPERIMENTAL RESULTS: Experiments use widely used benchmark datasets from the UCI and KEEL repositories.The passage identifies UCI and KEEL as the dataset sources.
  • IV. EXPERIMENTAL RESULTS: KRPRVFL is evaluated through comparative studies with several representative learning models.The evaluation is designed to assess the proposed approach comprehensively.
  • IV. EXPERIMENTAL RESULTS: The comparison includes RVFL, ELM, GB-RVFL, GE-GB-RVFL, CRVFL, and ACRVFL.These models are cited as representative baselines in the experimental evaluation.
  • IV. EXPERIMENTAL RESULTS: Experiments with added label noise are included in the evaluation.The passage introduces label-noise experiments as part of the validation design.

A. Experimental Setup

The experiments use a standardized computing environment and evaluation protocol, with grid-searched hyperparameters and broad searches over regularization, risk sensitivity, power, hidden nodes, and activation functions.

  • Experiments run on Windows 11 with an Intel Xeon Gold 6226R (2.90 GHz), 256GB RAM, and Python 3.11.
  • Datasets use a 70:30 training-testing split, with hyperparameters tuned by grid search and five-fold crossvalidation.
  • Regularization coefficients span D = {10−5, 10−4, . . . , 105}, while µ ∈[1, 10], p ∈[20, 21, ..., 210], and hidden nodes N range from 3 to 203.
  • Nine activation functions are tested as part of the experimental configuration.

B. Evaluation on UCI and KEEL Datasets

Across 37 UCI and KEEL benchmark datasets, KRPRVFL achieved an average classification accuracy of 85.20% and the lowest average rank among seven evaluated models. Friedman and Nemenyi analyses confirmed statistically significant performance differences and improvements over all baselines.

  • Dataset Evaluation: KRPRVFL achieved an average classification accuracy of 85.20% across 37 UCI and KEEL benchmark datasets.Performance was assessed using classification accuracy (Acc) against six baseline models.
  • Ranking Analysis: KRPRVFL attained the lowest average rank at 2.03, compared with 3.88–4.95 for the six baseline models.The baseline average ranks were RVFL 3.88, ELM 4.34, GB-RVFL 4.07, GE-GB-RVFL 4.38, CRVFL 4.95, and ACRVFL 4.36.
  • Statistical Testing: 8.35 exceeded the 5% critical value of 2.1407, so the Friedman test rejected equal performance among the seven models.The analysis used n = 37 datasets and g = 7 competing models, yielding χ2_F = 41.807 and F_F = 8.35.
  • Post Hoc Analysis: The Nemenyi test found statistically significant improvements of KRPRVFL over RVFL, ELM, GB-RVFL, GE-GB-RVFL, CRVFL, and ACRVFL.The critical difference was 1.4811, while KRPRVFL’s average-rank differences versus the baselines ranged from 1.85 to 2.92.

V. CONCLUSION

The conclusion presents KRPRVFL as a robust, collaborative extension of RVFL that achieves strong classification performance on UCI and KEEL benchmarks. It also identifies parameter and dataset-scale limitations and outlines directions for future development.

  • Contributions: KRPRVFL integrates the kernel risk-sensitive mean p-power criterion with collaborative learning as a robust extension of RVFL.The model is designed to improve robustness and adaptability in complex data scenarios.
  • Empirical findings: KRPRVFL consistently achieves superior classification performance versus existing RVFL variants and baseline models on UCI and KEEL benchmark datasets.The reported advantage is emphasized particularly for noisy-label settings.
  • Limitations: KRPRVFL is computationally efficient and scalable, but its performance depends on kernel and risk-sensitive parameters and is best suited for small to medium datasets.These constraints define the current applicability of the proposed framework.
  • Future work: Future work will address large-scale and streaming data, adaptive kernel selection, multi-task and multiview learning, and deep RVFL integration for hierarchical features.These directions aim to extend the model’s scope and representation capabilities.

SUPPLEMENTARY MATERIAL … 2) Sensitivity analysis of hyperparameters G and N:

The supplementary experiments assess KRPRVFL under controlled label noise and examine sensitivity to activation, structural, and robustness-related hyperparameters. Results indicate robust performance and identifiable tuning regions across benchmark datasets.

  • S.I. EVALUATION ON UCI AND KEEL DATASETS WITH ADDED LABEL NOISE: Controlled label noise was injected at 5%, 10%, 20%, 30%, and 40% into UCI and KEEL benchmark datasets.The evaluation targets realistic learning scenarios where training data may contain label noise.
  • S.II. SENSITIVITY ANALYSIS: The sensitivity analysis evaluates how activation functions, network-structure parameters, and robustness-related parameters affect KRPRVFL classification performance.The analysis is conducted across different datasets to assess robustness and stability.
  • 1) Sensitivity analysis of the activation function (Act):: On cleve, accuracy increases sharply from SELU to ReLU, while other activations produce moderate fluctuations and overall stable performance.On ecoli-0-1-4-6 vs 5, accuracy remains consistently high across almost all tested activation functions.
  • 2) Sensitivity analysis of hyperparameters G and N:: Higher accuracy generally occurs when D and N take moderate values, whereas very small configurations cause noticeable degradation on cleve.The figure examines the combined influence of feature dimension D and enhancement-node count N.
  • 2) Sensitivity analysis of hyperparameters G and N:: Accuracy on conn bench sonar mines rocks is particularly sensitive to N, with performance improving as the enhancement-node setting changes.The supplied passage introduces increased sensitivity to N but does not provide the complete trend.
  • S.I. EVALUATION ON UCI AND KEEL DATASETS WITH ADDED LABEL NOISE: Table S.I compares KRPRVFL with RVFL, ELM, GB-RVFL, GE-GB-RVFL, CRVFL, and ACRVFL under noisy UCI and KEEL conditions.The supplied table excerpt reports, for example, cleve average Acc values of 73.47 for RVFL and 74.64 for KRPRVFL†.
  • 2) Sensitivity analysis of hyperparameters G and N:: Intermediate D and N settings yield the best performance on haberman survival, while overly small values of either parameter clearly reduce accuracy.KRPRVFL therefore maintains stable and competitive performance across a broad but defined D-and-N region.
  • 2) Sensitivity analysis of hyperparameters G and N:: For conn bench sonar mines rocks, moderate p and intermediate µ provide more stable and higher accuracy than small p settings.Figure S.3 evaluates the joint effects of the power parameter p and risk-sensitive parameter µ.
Loading 2608.13628v1…