PaperlayerView exact revision on arXiv

Source-linked AI summary

GBTSVR: Twin Support Vector Regression via Granular Ball Computing

A. Quadir, M. Tanveer

arXiv:2610.20053v1cs.LG

TL;DR

Point-wise regression, including TSVR, remains vulnerable to noise, outliers, redundancy, and latent heterogeneity, despite TSVR’s use of nonparallel bounds. The paper replaces individual samples with granular-ball learning units and adds a quantile extension; across diverse evaluations, both models consistently perform strongly against state-of-the-art methods.

  • Problem

    Point-wise regression remains sensitive to noise, outliers, and redundancy, and TSVR can overlook latent heterogeneity while modeling asymmetric errors.

    Source: §1 · p. 4

  • Method

    GBTSVR and its GB-TSVQR extension use granular balls instead of individual samples, with the latter incorporating quantile regression.

    Source: §1 · p. 5, §2 · p. 6

  • Results

    GBTSVR and GB-TSVQR demonstrate superior or highly competitive RMSE, contamination robustness, and training efficiency across diverse experimental settings.

    Source: §6 · p. 24

  • Takeaways & Limitations

    Granular-ball-level learning reduces sensitivity to noisy and extreme samples while preserving essential data characteristics.

    Source: §6 · p. 24

    GB-TSVQR specifies strictly positive regularization parameters and restricts its quantile parameter τ to (0, 1).

    Source: §4 · p. 13

Abstract

from arXiv · show

Regression models used in real-world applications are often challenged by noise, outliers, data heterogeneity, and asymmetric error characteristics, which severely limit the effectiveness of conventional point-wise learning models. While twin support vector regression (TSVR) partially addresses these issues through the use of nonparallel regression bounds, it still relies on individual samples as basic learning units. As a result, TSVR remains highly sensitive to noisy and contaminated observations and fails to leverage the underlying structural and regional information inherent in complex datasets, leading to suboptimal robustness and generalization performance. To address these limitations, we introduce granular ball twin support vector regression (GBTSVR) and its quantile extension, granular ball twin support vector quantile regression (GB-TSVQR). The proposed models integrate granular ball computing with TSVR, replacing individual data points with compact, structure-preserving granular balls as fundamental learning units. This region-level representation effectively suppresses noise and outliers while preserving intrinsic data geometry, thereby enhancing robustness, stability, and generalization. GBTSVR constructs nonparallel lower- and upper-bound regression functions over granular balls, significantly reducing sensitivity to sample-level perturbations. Building upon this framework, GB-TSVQR further incorporates quantile regression principles to simultaneously capture heterogeneous trends and asymmetric error distributions across different quantile levels. Extensive experiments on synthetic datasets, time-series benchmarks, and outlier-contaminated data demonstrate that the proposed GBTSVR and GB-TSVQR consistently outperform state-of-the-art regression models.

1. Introduction

Conventional regression models learn from individual samples, leaving them vulnerable to noise, outliers, redundancy, and data heterogeneity; TSVR models asymmetry with nonparallel bounds but still uses raw samples. The paper introduces granular-ball versions of TSVR, including a quantile extension, and reports strong performance across diverse regression benchmarks.

1. Introduction

  • Point-wise regression remains sensitive to noise, outliers, and redundancy, while TSVR can still miss latent heterogeneity despite modeling asymmetric errors.Point-wise learning also degrades stability under contamination and adds computational overhead, especially for large, noisy datasets.Source: §1 · p. 4
  • GBTSVR replaces individual samples with granular balls that preserve local data geometry, mitigating noise and outliers while reducing redundancy.Granular-ball computing represents data as compact, high-purity hyperspherical regions that aggregate similar samples.Source: §1 · pp. 5–6
  • GB-TSVQR extends GBTSVR with quantile regression, jointly modeling heterogeneous and asymmetric distributional characteristics across quantile levels.Its twin-regressor formulation uses reduced-size optimization problems solved by dual coordinate descent.Source: §1 · pp. 5–6
  • GBTSVR and GB-TSVQR consistently outperform state-of-the-art regression models in accuracy, robustness, and stability across diverse noise settings.Experiments include synthetic datasets, time-series datasets, and benchmarks with injected outliers.Source: §1 · p. 6

2. Granular-Ball–Based Regression Framework

The framework replaces point-wise regression inputs with compact granular balls, then partitions the data into coherent regions using a regression-oriented purity criterion.

2. Granular-Ball–Based Regression Framework

  • Regression operates on compact, structured granules formed from the original dataset rather than directly on individual samples.The framework first transforms the dataset into granules and then performs regression at the granule level.Source: §2 · p. 6
  • Purity is the proportion of samples sharing a ball’s dominant label, and threshold τ enforces homogeneity within each ball.For regression, continuous responses are discretized into intervals that supply the labels used to calculate purity.Source: §2 · pp. 7–8
  • Granule generation balances ball compactness against the number of balls while requiring each ball to meet the purity threshold.The optimization weights compactness and granularity; balls below the purity threshold are split.Source: §2 · p. 8
  • Construction proceeds coarse to fine, beginning with one ball and splitting it until all balls satisfy the purity criterion.The resulting granules are described as preserving the data’s intrinsic structure.Source: §2 · pp. 8–9

3. The Proposed Granular Ball Twin Support Vector Regression

GBTSVR learns two nonparallel regression bounds from granular-ball representations. Its objectives fit the bounds to ball centers while penalizing violations, and the final estimate averages the two functions.

3. The Proposed Granular Ball Twin Support Vector Regression

  • GBTSVR constructs two nonparallel regression functions from granular-ball representations to capture local structure while suppressing noise and outlier influence.Each ball is represented by its center, radius, and associated statistical information.Source: §3 · p. 9
  • The objective fits the lower and upper ε-insensitive bounds to granular-ball centers, with slack variables measuring violations of their margin requirements.Cumulative slack penalties balance fitting error with model regularization.Source: §3 · pp. 10–11
  • KKT conditions yield dual formulations for the two bound problems.The paper derives a dual for each of the lower- and upper-bound objectives.Source: §3 · pp. 11–12
  • Adding regularization term δI addresses the computational cost and possible numerical instability of directly inverting G^TG.The regularized computation is described as facilitating an efficient solution.Source: §3 · pp. 11–12
  • The final regression estimate is the average of the lower- and upper-bound functions after their parameters are computed.The two functions respectively define the lower and upper ε-insensitive boundaries.Source: §3 · pp. 10, 12

4. The Proposed Granular Ball Twin Support Vector Quantile Regression

GB-TSVQR combines granular ball computing with TSVR to address heterogeneous and asymmetric data while improving robustness and training efficiency. Its quantile formulation uses nonparallel lower and upper bounds to represent quantile-dependent characteristics.

4. The Proposed Granular Ball Twin Support Vector Quantile Regression

  • GB-TSVQR integrates granular ball computing with TSVR to improve robustness to noise and outliers while avoiding large matrix inversions.The paper describes the resulting training process as more computationally efficient and scalable for complex, large-scale datasets.Source: §4 · pp. 12–13
  • As τ varies from 0 to 1, the regression function shifts upward across training samples to model heterogeneous, quantile-specific structures.The formulation uses τ to mitigate the impact of training errors.Source: §4 · pp. 13, 18
  • For a fixed τ and dataset, two QPPs learn ε1-insensitive lower and ε2-insensitive upper bounds, producing a nonparallel, asymmetric regression band.The optimization problems construct either the lower or upper bounding regression function.Source: §4 · pp. 13–14
  • The dual coordinate descent method solves the optimization problems by updating one coordinate at a time while holding the others fixed.The updates are subject to box constraints for α and γ.Source: §4 · pp. 15–16
  • After optimizing the dual variables, the final regression function is defined as the average of the lower and upper bounds.The bounds are expressed using the optimal dual variables α and γ.Source: §4 · pp. 16–17

5. Experimental Results

GBTSVR and GB-TSVQR are evaluated against representative regression baselines on UCI and synthetic datasets. The reported results show strong UCI performance and robustness across diverse noise and outlier conditions.

5. Experimental Results

  • The evaluation compares GBTSVR and GB-TSVQR with representative regression models on UCI benchmarks and synthetic datasets using RMSE and other reported error metrics.The UCI comparison includes RMSE, MAE, Pos Error, and Neg Error; RMSE is the primary metric.Source: §5 · p. 18, §5.1 · p. 18, Table 1 · p. 19

5.1. Experimental Evaluation on UCI Datasets

  • GB-TSVQR achieves the lowest RMSE on 13 of 18 UCI datasets, while GBTSVR ranks second on 12, with statistical tests favoring both over most baselines.The Nemenyi analysis reports exceptions for GBTSVR versus SVR and L1-TSVQR, and for GB-TSVQR versus SVR.Source: §5.1 · pp. 18, 21, Table 1 · p. 19

5.2. Experimental Evaluation on Synthetic Datasets

  • Across 15 synthetic datasets with Gaussian, uniform, or Student’s t noise, both proposed models perform best in most cases and remain competitive in the rest.Each dataset contains 500 samples; the sets combine five generated functions with three noise types.Source: §5.2 · pp. 21–23, Table 3 · p. 22
  • With 10% of samples corrupted by multiplying one randomly selected feature by 10, GB-TSVQR has the lowest RMSE in most cases and GBTSVR remains competitive.Both models perform strongly across the tested noise types and generating functions.Source: Table 4 · p. 22, §5.2 · p. 23

6. Conclusion

The paper presents GBTSVR and GB-TSVQR as granular-ball extensions of TSVR and evaluates them across benchmark, synthetic, outlier-contaminated, and time-series settings. The authors report competitive performance and identify broader regression settings for future work.

6. Conclusion

  • The experiments span 18 UCI benchmarks, synthetic datasets under varied noise and injected outliers, and real-world time-series datasets.The evaluations include statistical significance tests.Source: §6 · pp. 23–24
  • Across the evaluated settings, the models are reported to be superior or highly competitive in RMSE, robustness to contamination, and training efficiency.The authors attribute reduced sensitivity to noisy and extreme samples to granular-ball learning while preserving essential data characteristics.Source: §6 · pp. 23–24
  • Future work will extend the framework to multivariate and high-dimensional regression, online and streaming settings, and deep hybrids combining granular representations with neural networks.Source: §6 · p. 24

CRediT authorship contribution statement

The contribution statement assigns conceptualization to both authors, with distinct operational and supervisory responsibilities.

CRediT authorship contribution statement

S.1. Related Work

The section introduces prior work on granular-ball computing and the mathematical formulation of TSVR.

S.1. Related Work

  • The section examines prior work related to granular-ball computing and the mathematical formulation of TSVR.Source: §S.1 · p. 30

S.1.1. Granular-ball Computing

Granular-ball computing uses hierarchical, region-level representations to capture data structure before refining detail. Each ball represents a localized region, and the approach has been applied across several learning tasks.

S.1.1. Granular-ball Computing

  • Multigranularity cognitive computing analyzes data by first capturing high-level structural patterns and then refining them into finer representations.The strategy is described as a coarse-to-fine framework for understanding complex data distributions.Source: §S.1.1 · p. 30
  • Granular ball computing groups samples into compact balls that characterize local regions through center, radius, density, and purity.These region-level abstractions replace point-wise representations and are described as reducing computational overhead while enhancing robustness to noise, outliers, and redundancy.Source: §S.1.1 · p. 30
  • Granular-ball approaches have been applied to clustering, classification, sampling, rough sets, and three-way decision frameworks.The passages describe cluster-structure identification, resilience to noisy or mislabeled samples, representative subset selection, and decision-boundary approximation, [28],,.Source: §S.1.1 · pp. 30–31

S.1.2. Twin Support Vector Regression

TSVR learns lower and upper regression bounds through two independent optimization problems. The experiments compare granular-ball models with baselines on time-series datasets, where the proposed models show strong predictive performance.

S.1.2. Twin Support Vector Regression

  • TSVR solves two smaller independent quadratic programs to learn lower and upper bounding regressors.D1 and D2 are regularization parameters, while ϵ1 and ϵ2 define the insensitive zones.Source: §S.1.2 · p. 31

S.2. Experimental setup

  • Experiments use random 70/30 training/testing splits, five-fold cross-validation, and grid search over D1 = D2 = {10^-5, 10^-4, . . . , 10^5}.The models are evaluated using MAE, RMSE, negative error, and positive error.Source: §S.2 · p. 32

S.3. Experiments on Time-series Datasets

  • GB-TSVQR achieves the best RMSE on most time-series datasets and the lowest average RMSE, while GBTSVR also has lower average RMSE than baseline models.The reported comparison covers RMSE, MAE, positive error, and negative error.Source: Table S.1 · p. 33, §S.3 · pp. 33–34
Loading 2610.20053v1…