Source-linked AI summary
Active Learning for Regression Using Greedy Sampling
Dongrui Wu, Chin-Teng Lin, Jian Huang
TL;DR
Regression often needs many labeled samples, although unlabeled inputs may be abundant and expensive to label. The paper proposes GSy and iGS greedy-sampling approaches that select samples for output-space or joint input-and-output diversity, and evaluates them on benchmark and EEG tasks. Across these evaluations, the approaches were effective and robust, with iGS generally performing best among the compared methods.
Problem
Regression models often need substantial labeled data, while real-world inputs may be plentiful and costly to label.
Method
The paper proposes GSy, which increases output-space diversity, and iGS, which balances diversity in input and output spaces through iterative greedy sampling.
Results
Across 12 benchmark datasets and 15 EEG-based drowsiness subjects, the proposed approaches were effective and robust, with iGS outperforming both GSx and GSy on average.
Takeaways & Limitations
Selecting informative unlabeled samples can support accurate regression models using a small labeled sample set, with iGS combining input- and output-space diversity.
Takeaways & Limitations
GSy’s feature selection or weighting may be unreliable because it is evaluated using a regression function built from very few labeled samples.
Abstract
from arXiv · showhide
Regression problems are pervasive in real-world applications. Generally a substantial amount of labeled samples are needed to build a regression model with good generalization ability. However, many times it is relatively easy to collect a large number of unlabeled samples, but time-consuming or expensive to label them. Active learning for regression (ALR) is a methodology to reduce the number of labeled samples, by selecting the most beneficial ones to label, instead of random selection. This paper proposes two new ALR approaches based on greedy sampling (GS). The first approach (GSy) selects new samples to increase the diversity in the output space, and the second (iGS) selects new samples to increase the diversity in both input and output spaces. Extensive experiments on 12 UCI and CMU StatLib datasets from various domains, and on 15 subjects on EEG-based driver drowsiness estimation, verified their effectiveness and robustness.
1. Introduction
Regression models require labeled samples, but labeling can be costly even when inputs are easy to collect. Active learning for regression addresses this by sequentially selecting beneficial unlabeled samples, and this paper proposes two new approaches evaluated across benchmark and EEG datasets.
- Motivation: Regression estimates an output from one or more input features and generally benefits from more labeled training samples.Labeled samples contain known dependent and independent variable values.
- Motivation: In applications such as EEG-based drowsiness estimation, collecting many inputs is relatively easy, but obtaining ground-truth labels is difficult.The paper identifies EEG trials as plentiful while drowsiness labels are not easy to obtain.
- Active Learning for Regression: Active learning selects the most beneficial unlabeled samples to label instead of selecting samples randomly.It is presented alongside regularization and transfer learning as a direction for regression with few labeled samples.
- Active Learning for Regression: Sequential pool-based ALR chooses samples from a given unlabeled pool so a model trained on them estimates the remaining samples accurately.The selection proceeds sequentially rather than all at once.
- Contribution: Because sequential pool-based ALR has relatively few existing approaches, the paper proposes two new methods inspired by greedy sampling.The methods are evaluated on 12 UCI and CMU StatLib datasets and 15 EEG-based driver-drowsiness subjects.
2. Greedy Sampling ALR Approaches
The paper introduces GSy and iGS as greedy-sampling approaches for active learning in regression. GSy targets output-space diversity, while iGS combines input- and output-space diversity to address unreliable feature weighting.
- GSx: GSx selects a representative first sample, then iteratively chooses samples farthest from previously selected samples in input space.The initial sample is closest to the pool centroid, and later selections maximize the minimum distance to labeled samples.
- GSy: GSy selects initial samples with GSx, then chooses unlabeled samples farthest from existing outputs to increase output-space diversity.GSy requires K0 labeled samples to construct an initial regression model and uses estimated outputs for unlabeled samples.
- GSy: GSy favors samples that help refine coefficients for predictors to which the current regression model is more sensitive.When |s1| > |s2|, the procedure selects x3; when |s1| < |s2|, it selects x4.
- GSy limitation: GSy’s feature selection or weighting may be unreliable because predictor sensitivities are estimated from very few labeled samples.Although output-space diversity implicitly considers feature importance, the resulting weighting can be inaccurate.
- iGS: iGS combines GSx and GSy to retain feature selection or weighting while reducing the risk of catastrophic failure when that weighting is misleading.It uses GSx to select the first K0 samples, then integrates input- and output-space distances for subsequent selection.
3. Experiments on UCI and CMU StatLib Datasets
Experiments evaluated six sample-selection algorithms on 12 UCI and CMU StatLib datasets using repeated pool-based active learning for regression. The proposed GSy and iGS approaches were effective, with iGS generally achieving the strongest performance.
- Experimental setup: All six algorithms trained ridge-regression models from labeled samples, using regularized loss with λ = 0.01.Ridge regression was selected because labeled-sample counts were very small.
- Evaluation measures: Performance was measured after each iteration using RMSE and CC, with RMSE treated as the primary measure and CC as secondary.The pool-wide evaluation used true labels for selected samples and model estimates for remaining unlabeled samples.
- Experimental results: As K increased, all six algorithms generally achieved smaller RMSE and larger CC, while iGS achieved the smallest RMSE and largest CC on most datasets.The results were averaged over 100 runs and displayed across the 12 datasets.
- Experimental results: Across normalized AUCs, the average performance ranking was iGS > GSy > GSx > EMCM ≈ QBC > BL.For RMSE, smaller normalized AUC was better; for CC, larger normalized AUC was better.
- Statistical analysis: All five ALR approaches had statistically significantly better RMSE and CC than BL, while iGS had significantly better RMSE than all approaches except GSy.GSy also significantly exceeded QBC and EMCM on both measures and GSx on RMSE; iGS significantly exceeded all approaches except GSx on CC.
4. Experiments on EEG-Based Driver Drowsiness Estimation
The EEG-based driver-drowsiness experiment evaluated six algorithms on 15 subjects using data derived from controlled driving experiments. Across subjects, iGS generally performed best, while all five active-learning approaches outperformed the baseline.
- Experimental setup: The experiment used EEG data from 15 subjects after restricting each recording to the first 3,600 seconds and excluding one incorrectly recorded subject.Sixteen healthy subjects participated initially; one subject’s data was unusable.
- Experimental setup: Drowsiness was predicted every 10 seconds from theta-band EEG power features extracted from 30-second signal epochs.Features were computed across EEG channels after preprocessing and converted to decibel values.
- Experimental results: As K increased, all six algorithms achieved smaller RMSE and larger CC, and iGS achieved the best metric values for most subjects.Results were averaged over 100 runs.
- Experimental results: Each active-learning approach achieved smaller RMSE than BL on all 15 subjects and larger CC than BL on at least 14 subjects.The average performance ranking was iGS > GSy ≈ GSx > EMCM > QBC > BL.
- Statistical analysis: Non-parametric comparisons found all five active-learning approaches significantly better than BL on both RMSE and CC, while iGS had significantly better RMSE than every other approach.The reported pairwise tests used AUCs for RMSE and CC.
- Statistical analysis: The EEG findings were generally consistent with the 12-dataset experiments, supporting the effectiveness and robustness of the proposed approaches, particularly iGS.On average, iGS achieved the best performance among the six algorithms.
5. Conclusions and Future Research
The paper concludes that its two greedy-sampling approaches were effective and robust across benchmark regression datasets and EEG-based drowsiness estimation. Future work will extend the methods to classification and investigate combining transfer learning with active learning for regression.
- Conclusions: The proposed GSy and iGS approaches were effective and robust across 12 benchmark datasets and 15 EEG-based driver-drowsiness subjects.iGS considers diversity in both input and output spaces and outperformed several existing active-learning approaches.
- Future research: Future research will extend GSx, GSy, and iGS from regression to classification.The paper also identifies transfer learning plus active learning for regression as a future direction.
- Future research: The authors plan to study how to integrate transfer learning and active learning for regression problems with insufficient labeled training data.The current paper uses regularization together with active learning.
end
The section fragments describe greedy-sampling operations for constructing and updating a regression model, alongside tables summarizing datasets, algorithm ranks, and statistical comparisons. They do not provide enough continuous prose for a fuller interpretation.
- Sampling procedure: The procedure initializes a labeled sample set, constructs a regression model, selects an unlabeled sample using dxy, and moves it into the selected set.The fragments also specify querying the selected sample for its label and updating f(x).
- Evaluation summaries: Tables 2–5 cover algorithm ranks and non-parametric comparison p-values for the benchmark datasets and the 15 EEG subjects.The supplied captions identify the table topics but do not include their numerical entries.