PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Broad Learning System with Wave Loss for Classification under Data Uncertainty

Mushir Akhtar, A. Varshney, A. Quadir, A. Rahaman, M. Tanveer, Mohd. Arshad

arXiv:2608.29983v1cs.LG

TL;DR

BLS remains vulnerable to noise and outliers because it relies on squared-error loss, motivating a robust yet efficient alternative. Wave-BLS replaces that loss with wave loss and uses inversion-free NAG optimization; across benchmark and robustness evaluations, it reports stronger performance and statistically significant improvements over several baselines.

  • Problem

    BLS relies on squared-error loss, making it sensitive to noise and outliers, while existing robust variants can add architectural complexity or computational overhead.

  • Method

    Wave-BLS integrates asymmetric, bounded, and smooth wave loss into BLS and solves the resulting optimization with a NAG-based scheme without matrix inversion.

  • Results

    86.7448% average accuracy is reported for Wave-BLS on UCI benchmarks, and its best average rank of 2.75 shows significant improvements over several competing methods.

  • Takeaways & Limitations

    Wave-BLS provides a robust broad-learning alternative with higher classification performance and stronger stability across datasets and contamination settings.

Abstract

from arXiv · show

Broad Learning System (BLS) offers an efficient alternative to deep architectures by enabling fast learning through randomized feature mapping and closed-form solutions. However, its reliance on squared error loss makes it highly sensitive to noise, outliers, and corrupted labels, limiting its reliability in real-world scenarios. To address this limitation, we propose Wave-BLS, a robust broad learning framework that integrates the wave loss function, which is asymmetric, bounded, and smooth, enabling controlled penalization of large errors. The proposed formulation replaces the standard least-squares objective with a wave-loss-based optimization problem, solved efficiently using a Nesterov accelerated gradient (NAG)-based scheme without requiring matrix inversion, thereby improving scalability. Extensive experiments on 30 UCI benchmark datasets demonstrate that Wave-BLS consistently outperforms classical BLS and several robust variants. Statistical validation using Friedman and Nemenyi post-hoc tests confirms the significance of the observed improvements. Furthermore, robustness evaluations under controlled noise and outlier injection reveal that Wave-BLS exhibits substantially slower performance degradation compared to BLS, even in challenging contamination settings. These results establish Wave-BLS as a stable and robust alternative to existing broad learning models for learning under data uncertainty.

I. INTRODUCTION

BLS improves the efficiency and expressiveness of randomized broad learning, but squared-error optimization and matrix inversion limit robustness and scalability. Wave-BLS addresses these issues by integrating a bounded, smooth, asymmetric wave loss and inversion-free NAG optimization.

  • BLS expands network width with randomized feature and enhancement nodes, retaining closed-form learning, incremental learning, and effectiveness in limited-data regimes.
  • Squared-error loss makes BLS highly sensitive to noise and outliers, while matrix inversion introduces cubic-time complexity that limits scalability.
  • Existing BLS variants improve robustness through regularization, fuzzy inference, kernelization, or collaborative correction, often with greater architectural complexity or computational overhead.
  • Wave-BLS replaces least-squares loss with wave loss, reformulates optimization without matrix inversion, and uses NAG for stable updates and fast convergence.
  • The study evaluates Wave-BLS on 30 UCI datasets, applies Friedman and Nemenyi tests, and tests robustness under controlled noise and outlier contamination.

A. Notations

BLS combines randomized feature and enhancement representations before learning output weights. Its architecture uses a flat, width-expanding design with feature, enhancement, and output layers, while the conventional formulation solves regularized least squares.

  • BLS is a flat architecture with feature, enhancement, and output layers that expands width by incrementally adding nodes.
  • Feature Layer: The feature layer generates q windows of p randomized feature nodes from the input matrix X using nonlinear activation functions.
  • Enhancement Layer: The enhancement layer applies random nonlinear transformations to feature output Z through s windows containing r nodes.
  • Output Layer: The combined representation A concatenates feature output Z and enhancement output H before output-weight learning.
  • Output Layer: Standard BLS learns output weights W by solving a regularized least-squares problem with regularization parameter C and an identity matrix I.

III. PROPOSED WORK

Wave-BLS integrates the wave loss into BLS to control the influence of large errors. Its asymmetric, bounded, and smooth properties support robust learning and gradient-based optimization without explicit matrix inversion.

  • Wave-BLS is a broad learning system that replaces the conventional least-squares objective with wave-loss optimization.
  • Wave Loss Function: The wave loss is asymmetric, allowing unequal penalization of overestimation and underestimation through shape parameter a.
  • Wave Loss Function: The wave loss is upper-bounded by 1/λ, limiting the influence of large errors and reducing excessive outlier domination.
  • Wave Loss Function: The wave loss is continuously differentiable, enabling efficient and stable gradient-based optimization.

B. Formulation & Optimization: Broad Learning System with Wave Loss Function (Wave-BLS)

Wave-BLS replaces BLS’s conventional squared-error objective with aggregated wave loss and optimizes the resulting formulation without explicit matrix inversion. The method uses NAG-based updates, with regularization and a decaying learning rate supporting parameter optimization.

  • Wave-BLS formulation: Wave-BLS replaces conventional squared error with wave loss aggregated over training samples and classes.The wave loss is described as asymmetric, bounded, and smooth.
  • Objective function: The objective combines a regularization term controlling model complexity with the aggregated wave loss over training samples.The regularization parameter C controls the trade-off between model complexity and empirical risk.
  • Optimization: Wave-BLS avoids explicit matrix inversion during output-weight estimation, improving the scalability of optimization.This design addresses the computational bottleneck associated with conventional BLS parameter estimation.
  • Optimization: Nesterov accelerated gradient computes look-ahead gradients and uses momentum-based updates for efficient and stable optimization.The algorithm computes gradients with respect to the output weight matrix W and updates model parameters iteratively.
  • Optimization: An exponential decay schedule updates the learning rate as µ_t = µ_0 e^−ηt to balance convergence speed and stability.The initial learning rate is µ_0, while η controls the decay rate.
  • Optimization: Algorithm 1 returns the output weight matrix W after iteration until a maximum-iteration or convergence-tolerance condition is reached.The procedure takes the representation matrix A, target matrix Y, regularization parameter C, and optimization settings as inputs.

IV. EXPERIMENTS AND DISCUSSION

The experiments evaluate Wave-BLS against multiple randomized, broad-learning, fuzzy, kernelized, and wave-loss baselines on 30 UCI benchmark datasets under clean and noisy conditions.

  • Experimental setup: Wave-BLS is evaluated on 30 UCI benchmark datasets against eight baseline methods under clean and noisy conditions.The baselines include RVFL, RVFLwoDL, BLS, Wave-RVFL, NF-BLS, F-BLS, IF-BLS, and KRP-BLS.

A. Performance Evaluation

Across 30 UCI benchmark datasets, Wave-BLS achieves the strongest average classification performance and cross-dataset stability among the evaluated models. The evaluation also applies statistical testing to assess whether rank differences are significant.

  • Evaluation design: The comparison covers average accuracy and average rank for Wave-BLS and competing methods over 30 UCI benchmark datasets.Table I reports the aggregate performance comparison, with bold marking the best result and underlining the second-best.
  • Average performance: Wave-BLS achieves the lowest average rank and highest average accuracy among the compared methods.These results indicate stronger predictive performance and better cross-dataset stability than existing baselines.
  • Statistical validation: Friedman and Nemenyi post-hoc tests are used to assess the statistical significance of differences in average ranks.Detailed statistical results are reported in the supplementary material.

B. Robustness Analysis under Noise and Outliers

Controlled contamination experiments show that Wave-BLS is more robust than BLS under increasing outlier and label-noise levels. Its accuracy degrades more slowly and remains higher on average across the tested settings.

  • Performance under contamination: Wave-BLS consistently outperforms BLS once outliers or noise are introduced, including on a dataset where BLS is slightly better on clean data.The experiments contaminate 5%, 10%, 15%, and 20% of training samples with outliers and label noise.
  • Performance under contamination: Wave-BLS shows slower performance degradation as contamination increases under both outlier and noise perturbations.The wave loss bounds the influence of corrupted samples and mitigates extreme deviations.
  • Aggregate robustness: Wave-BLS achieves higher average accuracy and lower average rank across both datasets and contamination types.The evaluation uses the blood and horse colic datasets as complementary robustness cases.

V. CONCLUSIONS

Wave-BLS replaces BLS least-squares training with a bounded, asymmetric wave loss and NAG-based optimization without matrix inversion. Experiments and statistical tests report consistently higher performance and improved robustness under contamination.

  • Contribution: Wave-BLS integrates the wave loss into BLS to address sensitivity to noise and outliers.The loss replaces the least-squares error formulation while preserving the flat architecture.
  • Optimization: NAG-based optimization eliminates matrix inversion while retaining the flat and efficient BLS architecture.The proposed formulation uses a bounded and asymmetric loss.
  • Empirical findings: Wave-BLS achieves consistently higher classification performance than baseline variants, supported by Friedman and Nemenyi statistical tests.The conclusion also reports robustness studies under controlled noise and outlier injection.
  • Robustness: Under contamination, standard BLS degrades rapidly, whereas Wave-BLS provides stronger robustness according to the controlled robustness studies.The conclusion notes that BLS may remain competitive on clean data.

S.I. COMPUTATIONAL COMPLEXITY

Wave-BLS avoids the matrix inversion used by conventional BLS and instead performs iterative gradient and NAG updates. Its per-iteration cost is dominated by full-training-set gradient computation, with total cost scaling across the iteration budget.

  • Per-iteration cost: Gradient computation over all m samples and nout output dimensions dominates each Wave-BLS iteration.Each sample contributes a weighted multiplication involving the transformed feature vector A(i, :).
  • Per-iteration cost: NAG look-ahead, velocity, and parameter updates operate on matrices of size (pq + rs) × nout and contribute lower-order cost.These operations follow the gradient evaluation in each iteration.
  • Overall complexity: Over Imax iterations, Wave-BLS has total complexity O(Imax m nout (pq + rs)).The total cost follows from repeating the gradient-dominated update process.
  • Scalability: Conventional BLS requires matrix inversion, whereas Wave-BLS avoids that step to improve scalability.The comparison identifies matrix inversion as the conventional BLS computational burden.

S.II. EXPERIMENTAL SETUP

The evaluation uses controlled hyperparameter settings, benchmark comparisons, and nonparametric statistical tests across 30 UCI datasets. Wave-BLS is reported as statistically superior to several competing methods.

  • Evaluation design: Five-fold cross-validation and grid-search hyperparameter tuning are used to support fair comparisons across models.Each dataset is partitioned into five mutually exclusive folds, with training performed on four folds for each hyperparameter configuration.
  • Statistical validation: 11.1518 exceeds the 5% critical value 1.9784, so the Friedman test rejects equivalent performance across the evaluated models.The study evaluates K = 9 models on N = 30 datasets.
  • Statistical validation: Wave-BLS achieves the best average rank of 2.75 and significantly improves over RVFL, RVFLwoDL, BLS, NF-BLS, F-BLS, and KRP-BLS.Its rank differences versus Wave-RVFL and IF-BLS do not exceed the critical threshold, indicating comparable performance with those methods.
  • Statistical validation: The Nemenyi post-hoc procedure tests whether pairwise average-rank differences exceed a critical difference before declaring models significantly different.At α = 0.1, the reported critical difference is C.D. = 2.0188.
  • Evaluation design: Experiments compare Wave-BLS with eight randomized, broad-learning, and robust baseline models on 30 UCI benchmark datasets.The evaluated methods include RVFL, RVFLwoDL, BLS, Wave-RVFL, NF-BLS, F-BLS, IF-BLS, and KRP-BLS.

S.IV. SENSITIVITY ANALYSIS OF WAVE LOSS HYPERPARAMETERS

Sensitivity analysis on the breast cancer dataset examines the bounding parameter λ, the shape parameter a, and their joint effect on classification accuracy. Wave-BLS is broadly stable, but poor parameter choices can reduce performance.

  • Bounding parameter λ: Intermediate λ values improve accuracy, whereas very small or large values degrade performance on the breast cancer dataset.λ controls wave-loss saturation behavior and regulates the influence of large errors.
  • Shape parameter a: Wave-BLS maintains high accuracy across broad negative and positive ranges of a, with sharp degradation occurring only at a specific intermediate value.Accuracy rapidly recovers for neighboring values when other hyperparameters are fixed.
  • Joint sensitivity: The joint a–λ sensitivity surface contains a broad high-accuracy region alongside localized valleys from unfavorable parameter combinations.The results indicate that extremely precise tuning is unnecessary for competitive performance on this dataset.
  • Overall assessment: Careful selection of λ and a remains essential because suboptimal wave-loss parameters can cause noticeable performance degradation.The analysis finds reasonable stability across wide parameter ranges but not complete insensitivity.
Loading 2608.29983v1…