PaperlayerView exact revision on arXiv

Source-linked AI summary

Not All Unlabeled Data Are Equal: Unlabeled Data Selection for Semi-Supervised QSAR Modeling with Large Chemical Repositories

Oliver Majer, Saso Dzeroski, Jurica Levatic

arXiv:2610.22025v1cs.LG

TL;DR

QSAR modeling faces scarce, costly activity labels despite abundant unlabeled molecules in chemical repositories, and the value of those repositories and realistic unlabeled-data selection remain unclear. This study tests three SSL algorithms and four selection strategies across ChEMBL datasets. SSL configurations outperform supervised baselines on every dataset, with more reliable gains generally coming from small sets of chemically distant molecules.

  • Problem

    QSAR labels are costly and scarce despite abundant unlabeled molecules in chemical repositories, and the benefits of these repositories and selection strategies in realistic SSL settings remain unclear.

    Source: §1 · p. 2

  • Method

    The study evaluates self-training, co-training, and label propagation with four unlabeled-data selection strategies on 10 ChEMBL QSAR datasets and a pool exceeding 400,000 molecules.

    Source: §1 · p. 3

  • Results

    Across 10 QSAR datasets, at least one SSL configuration outperformed the supervised baseline on every dataset; the most robust improvements generally came from self-training with furthest selection and up to 250 unlabeled molecules.

    Source: §5 · p. 13

  • Takeaways & Limitations

    Relatively small sets of chemically distant molecules often yield more reliable improvements than large unlabeled collections, suggesting composition may matter more than size.

    Source: §5 · p. 13

    Choosing the best SSL method for a given dataset remains open, and the study uses fixed hyperparameters and relatively simple selection strategies.

    Source: §4.1 · p. 10, §5 · p. 13

Abstract

from arXiv · show

Semi-supervised learning (SSL) is particularly attractive in domains where labels are expensive to obtain, while unlabeled examples are readily available. Quantitative Structure-Activity Relationship (QSAR) modeling, which predicts molecular biological activity from chemical structure, is one such domain. Obtaining activity measurements requires costly wet-lab experiments, resulting in small labeled datasets, while chemical repositories such as ChEMBL contain millions of unlabeled molecules. Despite this apparent suitability, the potential of large chemical databases to improve QSAR models via SSL remains largely unexplored. Furthermore, it is unclear how unlabeled data should be selected in realistic SSL settings with genuine unlabeled data. We conduct a systematic study of unlabeled-data selection using 10 QSAR datasets extracted from ChEMBL and a pool of more than 400,000 unlabeled molecules. To this end, we evaluate three established SSL algorithms (self-training, co-training, and label propagation) and formulate four unlabeled-data selection strategies based on random, nearest-neighbor, furthest-neighbor, and mixed sampling. Our results show that unlabeled data from large chemical repositories can improve QSAR models, with at least one SSL configuration outperforming the supervised baseline on each of the 10 datasets. Moreover, the most reliable improvements are generally achieved by selecting a relatively small number of chemically distant molecules. This suggests that the composition of the unlabeled set may be more important than its size, challenging the common premise that SSL necessitates an abundance of unlabeled data.

1 Introduction

QSAR modeling has scarce labeled data but access to abundant unlabeled chemical repositories, making SSL attractive while leaving the value of large repositories and realistic unlabeled-data selection unclear. The study evaluates SSL methods and selection strategies, reporting that small sets of chemically distant molecules yield more reliable improvements.

1 Introduction

  • SSL uses both labeled and unlabeled examples to improve predictive performance, unlike supervised learning, which relies only on labeled examples.The introduction situates SSL among methods used when labels are expensive or time-consuming to obtain.Source: §1 · pp. 1–2
  • QSAR labels are costly to obtain, leaving datasets small, while repositories such as ChEMBL contain millions of compounds, most unlabeled for a specific task.QSAR predicts biological activity from molecular structure, and limited labels can constrain predictive performance.Source: §1 · p. 2
  • How to select unlabeled data from large, genuinely unlabeled pools—and how selection affects SSL performance—remains unclear.Many prior studies simulate unlabeled data by hiding labels in a small same-distribution sample, unlike the real-world setting of scarce labels and potentially mismatched abundant data.Source: §1 · pp. 2–3
  • The study evaluates self-training, co-training regression, and label propagation across 10 ChEMBL QSAR datasets and more than 400,000 unlabeled molecules.It compares random, nearest-neighbour, furthest-neighbour, and mixed selection across different unlabeled-set sizes.Source: §1 · p. 3
  • Selecting a relatively small number of chemically distant molecules yields more reliable SSL improvements and may reduce training costs.The introduction says this challenges the premise that SSL benefits primarily from large unlabeled sets.Source: §1 · p. 3

2 Methods

The study applies three semi-supervised algorithms to QSAR regression and compares four ways to select a fixed-size unlabeled subset from a large pool. The methods use random-forest regression and select examples according to prediction confidence, agreement between views, graph propagation, or chemical distance.

2.1 Semi-supervised algorithms

  • All three methods use random-forest regression as the base learner, which provides predictions and estimates of prediction uncertainty.Source: §2.1 · p. 4
  • Self-training repeatedly adds unlabeled molecules whose random-forest predictions have sufficiently low variance across trees, then retrains the model.The threshold controls how conservative selection is: a smaller threshold admits fewer, more certain examples.Source: §2.1 · p. 4
  • Co-training uses two random-forest regressors trained on disjoint fingerprint halves; each adds examples that the two views predict with the least disagreement.The selected examples are pseudo-labeled across views, and both regressors are retrained on the enlarged labeled sets.Source: §2.1 · pp. 4–5
  • Label propagation propagates labels over a Tanimoto-similarity graph, then trains a random-forest regressor on labeled data augmented with selected pseudo-labels.The graph retains each node’s k nearest neighbors; pseudo-labels are retained only when propagated values differ from the labeled-target mean by a fixed margin.Source: §2.1 · p. 5

2.2 Unlabeled data selection strategies

  • For each selection strategy, the study varies the unlabeled-set budget from 100 to 5,000 molecules and measures chemical distance using 1024-bit fingerprints.The distance is one minus Tanimoto similarity.Source: §2.2 · pp. 5–6
  • Random sampling ignores labeled-set structure; Nearest and Furthest select chemically similar or dissimilar neighbors, while Mixed splits each labeled molecule’s quota between both.Nearest and Furthest use a shared quota scheme and randomized processing order; Mixed assigns any odd remaining slot to the nearest half.Source: §2.2 · p. 6

3 Experimental design

The experiments use 10 ChEMBL QSAR datasets and an external pool of 415,654 molecules to evaluate semi-supervised configurations against a supervised random-forest baseline. Evaluation uses fixed cross-validation splits and varies the method, selection strategy, and unlabeled-set size.

3.1 Data

  • Ten randomly selected datasets span 30 to 525 labeled molecules, covering small, intermediate, and comparatively large QSAR tasks.Each dataset represents curated numeric pXC50 bioactivity values for a protein target.Source: §3.1 · p. 7
  • The external unlabeled pool contains 415,654 unique molecules drawn from 2,970 QSAR datasets, excluding molecules labeled for the target under study.The molecules use 1024-bit FCFP4 fingerprints.Source: §3.1 · p. 7

3.2 Evaluation strategy

  • Each dataset is evaluated across three SSL methods, four selection strategies, and six unlabeled-set sizes, with a supervised random-forest baseline for comparison.The six sizes are m ∈ {100, 250, 500, 1000, 2500, 5000}; Table 2 reports each dataset’s lowest-RMSE configuration and its improvement over the baseline.Source: §3.2 · p. 8, Table 2 · p. 9
  • Fixed-seed cross-validation uses 10 folds for datasets with at least 100 labeled molecules and 5 folds for datasets with 30–99 molecules.Identical folds across conditions create paired comparisons with the labeled-data partition held constant.Source: §3.2 · p. 8
  • In each fold, SSL trains on the training fold plus selected external molecules, while both SSL and baseline models are tested on the same held-out fold.Selection uses only the training fold for distance calculations, excludes target-dataset molecules from the external pool, and is repeated independently per fold.Source: §3.2 · p. 8
  • Performance is measured by test-fold RMSE and summarized as percentage improvement over the supervised baseline, with positive values indicating better SSL performance.Method-specific hyperparameters use fixed defaults rather than per-dataset tuning, keeping the comparison focused on data selection.Source: §3.2 · pp. 8–9

4 Results and Discussion

Across 10 ChEMBL QSAR datasets, at least one SSL configuration outperformed the supervised baseline on each dataset. Aggregate results indicate that selection strategy and unlabeled-set size often matter more than the SSL method alone.

4.1 Can Large Chemical Repositories Improve QSAR Models via SSL?

  • At least one SSL configuration beat the supervised random forest baseline on every dataset, with improvements ranging from 0.5% to 28%.The largest gains occurred on datasets with 75 or fewer labeled molecules.Source: §4.1 · pp. 9–10
  • Performance varied substantially across datasets and configurations; label propagation had the largest gains on five datasets but also showed substantial degradations on several.The best-performing method varied by dataset, and some datasets improved under many configurations while others improved under only a few.Source: §4.1 · p. 10
  • The best-performing configurations do not provide a practical way to choose an appropriate SSL setup for a given dataset.Reliable configuration selection and methods that avoid degradation were outside the study’s scope.Source: §4.1 · p. 10

4.2 How Does Unlabeled Data Selection Affect SSL Performance?

  • Self-training generally improved with Random or Furthest selection, while co-training and label propagation improved only for specified small-set strategy combinations.Label propagation favored Nearest and degraded with Furthest; the authors suggest that distant molecules may distort its similarity graph, whereas pseudo-label methods may benefit from broader chemical-space coverage.Source: §4.2 · pp. 10–11
  • Furthest co-training at m = 250 reached a 70% win rate, while Furthest self-training was more robust and label propagation had 40–50% win rates with Nearest or Random.Co-training’s high win rate was less consistent because some datasets showed substantial degradation.Source: Figure 2 · p. 11, §4.2 · p. 12
  • Furthest selection and smaller unlabeled sets ranked best overall, with m = 100 outperforming m = 5000; selection and size differences exceeded those between SSL methods.Self-training had the best average rank among methods, while Nearest ranked worst among selection strategies.Source: Table 3 · p. 12, §4.2 · p. 12
  • Across methods, increasing the unlabeled-set size did not consistently improve performance, and the best results often came from relatively small sets.The authors conclude that unlabeled-set composition may matter more than size.Source: §4.2 · p. 12

5 Conclusions

Across 10 QSAR datasets, ChEMBL-derived unlabeled molecules improved predictive performance, and the most robust gains generally came from self-training with Furthest selection and a small unlabeled set. The conclusions emphasize selecting chemically distant molecules and note limitations in dataset scope and selection strategies.

5 Conclusions

  • Across 10 QSAR datasets, at least one SSL configuration outperformed the supervised baseline on every dataset; the most robust gains generally came from self-training with Furthest selection and up to 250 molecules.Benefits depended on the method, unlabeled-set composition, and number of examples.Source: §5 · p. 13
  • Selecting relatively few chemically distant molecules often yielded more reliable improvements than using large collections, suggesting set composition may matter more than size.Careful selection may achieve better results while reducing computational costs.Source: §5 · p. 13
  • The study is limited by its small number of datasets, single application domain, fixed SSL hyperparameters, and relatively simple unlabeled-data selection strategies.The authors identify broader datasets and domains, additional methods and hyperparameter settings, and more sophisticated selection approaches as future work.Source: §5 · p. 13
Loading 2610.22025v1…