PaperlayerView exact revision on arXiv

Source-linked AI summary

TopU-LBVS: A Realistic Multi Target Benchmark for Ligand Based Virtual Screening

Surbhi Kumar, Yuhe Zhou, Varun Shiralkar, Niu Huang, Baris Coskunuzer

arXiv:2609.29740v1cs.LGcs.AI

TL;DR

Existing LBVS benchmarks can make realistic hard-negative screening difficult to assess because they often rely on random negatives, limited target coverage, or nonstandard evaluation. TopU-LBVS builds a multi-target benchmark with fixed protocols and a paired decoy control; across protocols, learned models do not consistently outperform fingerprint baselines, and random-decoy evaluation substantially overestimates enrichment.

  • Problem

    Existing LBVS benchmarks often use random or easy negatives, limited target coverage, and nonstandard protocols, leaving performance on similar hard-negative libraries unclear.

    Source: §1 · pp. 1–2

  • Method

    TopU-LBVS builds 93 target-specific libraries with property-matched, structurally similar decoys and provides full, low-data, and mini protocols, including a paired random-decoy control.

    Source: §1 · p. 2, §4 · p. 5

  • Results

    Across protocols, learned molecular models do not consistently outperform strong fingerprint baselines; in the mini control, mean EF@1% is nearly 4× lower with TopU decoys than with random ChEMBL∗ decoys.

    Source: §6.4 · p. 9, §7 · p. 10

  • Takeaways & Limitations

    TopU-LBVS makes regime-dependent performance failures visible and measurable under hard-negative screening conditions.

    Source: §6.4 · p. 9

    The benchmark is retrospective and ligand-based, with no 3D binding-site information, docking scores, or prospective experimental validation.

    Source: §6.4 · p. 9

Abstract

from arXiv · show

Ligand-based virtual screening (LBVS) is a practical first-pass tool in early-stage drug discovery, but existing benchmarks can overestimate performance through random negatives, easy decoys, limited target coverage, and non-standardized evaluation protocols. We introduce TopU-LBVS, a multi-target benchmark for LBVS under hard-negative screening conditions. Starting from curated ChEMBL~35 bioactivity data, TopU-LBVS covers 93 protein targets across 7 protein classes and constructs target-specific screening libraries with property-matched, structurally similar decoys at a fixed 1:40 active-to-decoy ratio. Libraries contain roughly 400 to 10,000 compounds and are designed to reduce simple physicochemical and nearest-neighbor fingerprint shortcuts. TopU-LBVS provides three fixed protocols. TopU-LBVS-full evaluates ChEMBL$^\ast \rightarrow$ TopU generalization across all 93 targets. TopU-LBVS-low evaluates low-data TopU $\rightarrow$ TopU learning within the hard-negative distribution. TopU-LBVS-mini provides a compact seven-target protocol with a paired random-decoy control that changes only the test decoys, enabling low-cost development and direct measurement of the gap between random ChEMBL$^\ast$ and TopU decoys. Across ten reference baselines spanning fingerprint methods, molecular GNNs, fingerprint hybrids, and modern molecular models, performance under random-decoy evaluation degrades sharply under hard-negative screening. We release data, fixed splits, evaluation code, and baseline implementations for reproducible comparison of future LBVS and molecular representation learning methods. Code and data are available at https://github.com/topu-benchmark/topu-lbvs and https://huggingface.co/datasets/topu-benchmark/topu-lbvs.

1 INTRODUCTION

Existing LBVS benchmarks can simplify screening with random negatives, uneven target coverage, and nonstandard protocols, leaving performance on property-matched, structurally similar negatives uncertain. TopU-LBVS provides three fixed protocols and a paired random-decoy control to evaluate hard-negative screening across targets.

1 INTRODUCTION

  • Existing benchmarks often use random negatives, easy decoys, limited target coverage, or nonstandard splits, leaving transfer to hard-negative libraries unclear.Random or scaffold-agnostic splits can also place close analogues across training and test sets, producing optimistic performance estimates.Source: §1 · pp. 1–2
  • TopU-LBVS-full tests ChEMBL∗-to-TopU generalization, low tests low-data learning within TopU, and mini offers a compact seven-target protocol.The full and low protocols each cover all 93 targets.Source: §1 · p. 2
  • The mini protocol holds training, validation, test actives, and test ratio fixed while replacing TopU decoys with random ChEMBL∗ decoys.This paired comparison measures the effect of test-decoy construction.Source: §1 · p. 2
  • TopU-LBVS provides 93 target-specific tasks across 7 protein classes, pairing actives with property-matched, structurally similar decoys at a fixed 1:40 ratio.The construction is intended to reduce simple property and nearest-neighbor shortcuts.Source: §1 · p. 2
  • The benchmark releases a curated ChEMBL 35 resource, ten reference baselines, fixed splits, implementations, and evaluation code.The baselines span fingerprint methods, molecular GNNs, hybrids, and modern molecular models; early enrichment is the primary metric.Source: §1 · p. 2

2 BACKGROUND AND EXISTING LBVS BENCHMARKS

Existing LBVS resources address parts of realistic screening or standardized learning, but do not jointly offer broad target coverage, hard negatives, screening-relevant imbalance, and fixed supervised protocols. TopU-LBVS combines these elements with low-data and compact paired-control protocols.

2 BACKGROUND AND EXISTING LBVS BENCHMARKS

  • A useful learning-based LBVS benchmark combines challenging negatives, target diversity, screening-relevant imbalance, and fixed reproducible protocols.These properties are intended to support direct method comparison across ligand chemistries and protein families.Source: §2 · p. 2
  • MUV and LIT-PCBA provide challenging or experimentally confirmed negatives, but cover 17 and 15 targets respectively and lack canonical training-based protocols.FS-Mol supplies fixed few-shot splits but uses approximately balanced ratios and random inactives; CACHE targets prospective structure-based hit-finding.Source: §2 · p. 3
  • Existing resources do not jointly provide broad multi-class coverage, explicit hard-negative libraries, screening-relevant imbalance, fixed supervised protocols, and accessible low-cost evaluation.TopU-LBVS adds 93 targets, a low-data protocol, a seven-target mini benchmark, and a paired random-decoy control.Source: §2 · p. 3

3 THE TOPU–LBVS BENCHMARK

TopU-LBVS uses a diverse 93-target panel, curates ChEMBL 35 into a cleaned activity resource, and constructs hard-negative screening libraries with a fixed active-to-decoy ratio.

3.1 TARGET PANEL AND PROTEIN CLASSES

  • The benchmark spans 93 targets across seven protein classes, selected for biological diversity, ligand-discovery relevance, and assay coverage.When full-protocol target-specific inactives are insufficient, training negatives may be augmented from the same protein class, without affecting TopU test libraries.Source: §3.1 · p. 4

3.2 CHEMBL∗ CURATION

  • ChEMBL 35 is curated into ChEMBL∗ by standardizing compounds, resolving duplicates, binarizing activity, and filtering records and molecules.Conflicting active/inactive annotations are excluded from the decoy pool, and final TopU decoys are not recorded active for their target under the benchmark criterion.Source: §3.2 · p. 4

3.3 TOPU CONSTRUCTION AND STATISTICS

  • TopU libraries use a fixed 1:40 active-to-decoy ratio and are constructed by optimizing candidate decoys to reduce activity-label distinguishability across physicochemical descriptors and MFP3 features.A Random Forest serves as a post-construction hardness audit; libraries range from several hundred to more than ten thousand compounds.Source: §3.3 · p. 4

4 TASKS AND EVALUATION SETTINGS

TopU-LBVS defines fixed target-specific screening tasks across 93 targets, using TopU hard-negative libraries and protocols for historical-to-library generalization, low-data learning, and compact evaluation. The protocols specify distinct data splits and evaluation metrics, with TopU-LBVS-mini adding a paired random-decoy control.

4 TASKS AND EVALUATION SETTINGS

  • The benchmark defines 93 target-specific ranking tasks across seven protein classes, with libraries of roughly 400 to more than 10,000 compounds and three fixed evaluation protocols.Each library uses property-matched, structurally challenging decoys at a 1:40 active-to-decoy ratio.Source: §4 · pp. 4–5

4.1 TOPU-LBVS-FULL: CHEMBL∗→TOPU EVALUATION

  • TopU-LBVS-full tests historical ChEMBL∗SAR generalization on disjoint TopU libraries, using a 1:10 training pool, 15% stratified validation split, and EF@1% as the primary test metric.When target-specific training inactives are insufficient, negatives are augmented from other targets in the same protein class.Source: §4.1 · p. 5

4.2 TOPU-LBVS-LOW: LOW-DATA TOPU →TOPU EVALUATION

  • TopU-LBVS-low evaluates limited-label learning within the TopU hard-negative distribution, using active-count-dependent splits and EF@10% as its primary metric.Targets with fewer than 50 actives use a 6:2:2 split; those with at least 50 use 7:1:2, with actives split by Bemis–Murcko scaffold.Source: §4.2 · pp. 5–6

4.3 TOPU-LBVS-MINI: COMPACT EVALUATION AND RANDOM-DECOY CONTROL

  • TopU-LBVS-mini selects one target per protein class with the largest TopU active count, follows the full protocol, and provides a compact seven-target evaluation.Its paired control preserves training, validation, test actives, and the 1:40 test ratio while replacing TopU test decoys with random ChEMBL∗ decoys; EF@1% is primary.Source: §4.3 · p. 6

5 BASELINES

The benchmark compares classical fingerprint methods, molecular graph models and hybrids, and modern molecular models under common target-specific evaluation and model-selection procedures.

Evaluation and training

  • Ten reference baselines are evaluated with target-specific scores, validation PR-AUC for model selection, and each protocol’s designated primary test metric.Learned models use class-weighted binary cross-entropy, and results are aggregated over three seeds on fixed splits.Source: §5 · pp. 6–7

Classical fingerprint baselines

  • Classical references include Morgan-RF and Tanimoto-NN, both based on 2048-bit ECFP4-style Morgan fingerprints.Morgan-RF trains a Random Forest; Tanimoto-NN ranks compounds by maximum similarity to training actives.Source: §5 · pp. 6–7

GNN and hybrid baselines

  • The graph and hybrid baselines pair GIN, GAT, and GPS with fingerprint-augmented variants to test learned graph signal alongside Morgan fingerprints.Each hybrid combines a learned graph embedding with a learned projection of the Morgan fingerprint before classification.Source: §5 · p. 7

Modern molecular baselines

  • Modern molecular baselines comprise D-MPNN/Chemprop and MolFormer, a pretrained SMILES Transformer fine-tuned separately for each target.The passage identifies D-MPNN/Chemprop as a directed message-passing model for molecular property prediction.Source: §5 · p. 7

6 EXPERIMENTAL RESULTS

Across the three protocols, evaluated learned models do not reliably outperform strong fingerprint baselines under hard-negative screening, and random-decoy evaluation substantially overestimates enrichment.

6 EXPERIMENTAL RESULTS

  • The three protocols report per-target rankings averaged by protein class and overall, using EF@1% for full and mini and EF@10% for low.PR-AUC is used for validation and reported as a secondary metric.Source: §6 · p. 7

6.1 TOPU-LBVS-FULL

  • Morgan-RF achieves the best overall EF@1% (9.18) in TopU-LBVS-full, exceeding the best pure GNN, GIN (4.47), and D-MPNN (7.38).Tanimoto-NN reaches 7.58, and fingerprint-augmented hybrids improve over pure GNNs but remain about 2.5 EF@1% points below Morgan-RF.Source: §6.1 · p. 7, Table 3 · p. 8

6.2 TOPU-LBVS-LOW

  • MolFormer obtains the best overall EF@10% (1.12) in TopU-LBVS-low, but Tier 1 targets average only 0.80 across baselines.Random ranking corresponds to EF=1, and Morgan-RF and Tanimoto-NN reach 0.63 and 0.50, respectively.Source: Table 4 · p. 8, §6.2 · p. 8
  • Fingerprint augmentation does not consistently help in low-data screening: GIN+FP reaches 0.84 versus GIN's 1.03, while GPS+FP reaches 0.85 versus GPS's 1.05.TopU-LBVS-low-mini adds a compact 13-target subset covering one Tier 1 and one Tier 2 target per protein class where available.Source: §6.2 · p. 8

6.3 TOPU-LBVS-MINI: RANDOM-DECOY CONTROL

  • Mean EF@1% falls from 21.2 on random decoys to 5.5 on TopU decoys across ten models and seven targets.Random-decoy EF@1% exceeds TopU EF@1% in 67 of 70 paired model-target comparisons.Source: §6.3 · p. 9, Table 5 · p. 9

6.4 DISCUSSION, LIMITATIONS, AND RELEASE

  • Across protocols, learned molecular models do not reliably outperform strong fingerprint baselines under hard-negative screening, while random-decoy evaluation substantially overestimates enrichment.In the low-data setting, learned models become more competitive, but performance remains close to random enrichment on the most data-limited tasks.Source: §6.4 · p. 9
  • TopU-LBVS is retrospective and ligand-based, excludes 3D binding-site information and prospective validation, and may retain assay noise in ChEMBL-derived inactive labels.Its targets are not class-balanced and favor families with richer public assay coverage; Tier 1 low-data estimates can also have high variance.Source: §6.4 · pp. 9–10
  • The release includes data, fixed splits, evaluation code, and implementations of all ten baselines.Future benchmark versions will be separately versioned to preserve comparability with the current release.Source: §6.4 · p. 10

7 CONCLUSION

TopU-LBVS provides a multi-target benchmark for testing ligand-based virtual screening under hard-negative conditions. Across ten baselines, performance can degrade sharply against hard decoys, and learned molecular models do not consistently outperform fingerprint baselines.

7 CONCLUSION

  • Mean EF@1% is nearly 4× lower with TopU decoys than with random ChEMBL∗ decoys in the mini control.The conclusion also reports that performance under random-decoy evaluation can degrade sharply under hard-negative screening.Source: §7 · p. 10
  • TopU-LBVS provides 93 target-specific tasks, fixed 1:40 active-to-decoy libraries, and three evaluation protocols built from curated ChEMBL 35 data.The protocols cover ChEMBL∗→TopU generalization, low-data TopU→TopU learning, and a compact mini benchmark with a paired random-decoy control.Source: §7 · p. 10
  • Evaluated learned molecular models do not consistently outperform strong fingerprint baselines under these hard-negative conditions.The benchmark releases curated data, fixed splits, evaluation code, and reference baselines for reproducible future comparisons.Source: §7 · p. 10

ETHICS STATEMENT

The paper describes a computational benchmark built from curated ChEMBL data and target-specific libraries of hard decoys. Its construction standardizes records and selects property-matched, structurally similar negatives at a fixed active-to-decoy ratio.

ETHICS STATEMENT

  • The work uses public ChEMBL bioactivity data and includes no human participants, personal information, animal experiments, or new wet-lab experiments.The benchmark is intended for methodological research, not experimentally validated binding predictions or clinical or therapeutic guidance.Source: ETHICS STATEMENT · p. 10

A.1 CHEMBL∗CURATION

  • ChEMBL∗ is curated to remove assay artifacts, standardize molecules, and produce consistent target-specific active and inactive data.The pipeline collects records for canonical potency endpoints, standardizes molecular representations, and filters problematic compounds.Source: §A.1 · p. 14
  • Compounds are binarized using a pChEMBL threshold greater than 4.9, and conflicting labels for a standardized molecule are resolved as active.ChEMBL∗ is the source pool for TopU construction and historical SAR training data.Source: §A.1 · p. 14

A.2 TOPU CONSTRUCTION AND STATISTICS

  • TopU selects hard decoys from each target’s ChEMBL∗ inactive pool to reduce physicochemical and nearest-neighbor shortcuts.A modified dynamic genetic algorithm minimizes feature-label mutual information while a Random Forest validator enforces cross-validation AUROC below 0.6; libraries use a fixed 1:40 active-to-decoy ratio.Source: §A.2 · p. 15
  • TopU libraries vary from a few hundred to more than ten thousand compounds per target, reflecting differences in assay coverage and surviving actives.The selection process uses fewer TopU actives than the cleaned pool and only a selected subset of available inactives.Source: §A.2 · p. 15
  • TopU is presented as a target-specific library for testing LBVS under focused hard-negative screening rather than random-negative evaluation.Its decoys are selected to avoid simple physicochemical separation while retaining structural similarity to actives.Source: §A.2 · p. 15

A.3 LABEL, LEAKAGE, AND REPRESENTATION ROBUSTNESS AUDITS

The benchmark audits inactive-label consistency, training-negative overlap, and the robustness of hard-negative difficulty across molecular representations. It also specifies compact low-data protocols, a diverse target panel, and reference baselines spanning fingerprints, graph models, hybrids, and pretrained models.

A.3 LABEL, LEAKAGE, AND REPRESENTATION ROBUSTNESS AUDITS

  • All 224,440 final TopU decoys lack recorded activity against their own target under the benchmark criterion, although unmeasured activity cannot be ruled out.The audit removed conflict-bearing compounds from the inactive pool; same-class negative borrowing affects training pools for 15 targets and is never used in TopU test libraries.Source: §A.3 · pp. 15–16
  • Most targets remain difficult under alternative molecular representations, but the audit tests existing libraries rather than independently re-mining them.The tested alternatives include ECFP4, ECFP6, RDKit fingerprints, MACCS keys, and physicochemical descriptors.Source: §A.3 · p. 16

A.4 TOPU-LBVS-LOW-MINI: A COMPACT LOW-DATA BENCHMARK

  • TopU-LBVS-low-mini offers a lower-cost entry point to the low-data hard-negative protocol while preserving its splits, metrics, ratio, and evaluation code.It contains 13 targets: one Tier 1 and one Tier 2 per protein class where available; Tier 2 targets match TopU-LBVS-mini, and CYP has no Tier 1 target.Source: §A.4 · pp. 16–17, Table 6 · p. 16

A.5 TARGET PANEL AND PROTEIN CLASSES

  • The 93-target panel spans seven protein classes but is intentionally not class-balanced, with kinases, GPCRs, and other enzymes contributing the most tasks.TopU libraries have a median of 53 actives and a median total size of 2,173 compounds per target.Source: §A.5 · p. 17

B BASELINE MODELS

  • The reference baselines cover classical similarity methods, graph neural networks, fingerprint hybrids, and modern pretrained molecular models.The authors frame established fingerprint methods as an interpretable reference point that complex learned models should at least match or exceed.Source: B BASELINE MODELS · p. 17, §B.1 · p. 18

B.1 CATEGORY I: CLASSICAL SIMILARITY-BASED BASELINES

  • Morgan-RF ranks with an active-class probability, while Tanimoto-NN ranks by maximum similarity to training actives without learning.Tanimoto-NN is highlighted as a practical reference for judging whether learned models add value over similarity search.Source: §B.1 · pp. 19–20

B.2 CATEGORY II: GRAPH NEURAL NETWORKS AND HYBRID MODELS

  • The graph-model suite includes GIN, GAT, and GPS, plus late-fusion variants that concatenate graph and Morgan-fingerprint embeddings for classification.The GNNs represent molecules as graphs with atom and bond features; the hybrids combine learned graph embeddings with fingerprint embeddings.Source: §B.2 · pp. 20–21

B.3 CATEGORY III: MODERN PRETRAINED MODELS

  • D-MPNN and MolFormer represent two modern molecular-model baselines evaluated through task-specific fine-tuning.D-MPNN uses directed-edge message passing and physicochemical descriptors; MolFormer is a pretrained chemical language model.Source: §B.3 · p. 21

C HYPERPARAMETERS AND IMPLEMENTATION DETAILS

The appendix documents model configurations, shared training procedures, and computing requirements for the benchmark baselines. It also identifies released split files, scripts, configurations, and random seeds for reproducibility.

C HYPERPARAMETERS AND IMPLEMENTATION DETAILS

C.1 CLASSICAL BASELINES

  • Morgan-RF uses 2048-bit Morgan fingerprints and validation PR-AUC for tree-count stopping; Tanimoto-NN ranks by maximum similarity to training actives.Tanimoto-NN has no model selection or hyperparameter tuning, and its reported seed variance reflects split-induced variance.Source: §C.1 · p. 22

C.2 GNN AND HYBRID MODELS

  • GNNs use class-weighted binary cross-entropy, Adam, and validation PR-AUC for checkpoint selection and early stopping.GIN, GAT, and GPS share five-layer, 256-wide backbones; hybrid variants concatenate a 32-dimensional graph embedding with a 32-dimensional fingerprint embedding.Source: §C.2 · pp. 22–23

C.3 MODERN PRETRAINED MODELS

  • D-MPNN combines directed-edge message passing with 200-dimensional RDKit descriptors, while MolFormer is fine-tuned end-to-end from a released checkpoint.Both use validation PR-AUC for early stopping; MolFormer uses a maximum sequence length of 202 tokens and a batch size of 32.Source: §C.3 · p. 23

C.4 COMPUTATIONAL RESOURCES

  • Classical fingerprint baselines take under a minute per target, while MolFormer requires approximately 15–90 minutes per target on an A100 80 GB GPU.The full benchmark is estimated at 2,000–3,000 GPU-hours, dominated by D-MPNN and MolFormer fine-tuning.Source: §C.4 · p. 23

IMPLEMENTATION NOTES

  • The implementation notes identify the software stack and state that split files, configurations, scripts, and random seeds are released.The baseline code uses PyTorch for neural models, scikit-learn and NumPy for classical methods, RDKit for fingerprints, and Chemprop v2 for D-MPNN.Source: IMPLEMENTATION NOTES · p. 24

D METRIC DEFINITIONS

The benchmark scores target-specific compound rankings with complementary early-enrichment and area-under-curve metrics, then aggregates results across targets and seeds. It evaluates historical ChEMBL∗-to-TopU screening and low-data learning within TopU.

D.1 ENRICHMENT FACTOR AT x PERCENT (EF@X%)

  • EF@x% measures active enrichment among the top-ranked fraction of compounds, with values above 1 indicating enrichment over random ranking.EF@1% and EF@10% are used; EF@10% is more stable for smaller libraries.Source: §D.1 · p. 24

D.2 PRECISION RECALL AUC (PR AUC)

  • PR-AUC is the area under the precision–recall curve and is more informative than ROC-AUC under the strong class imbalance relevant to LBVS.The curve is traced by sweeping the decision threshold; its area is approximated with stepwise interpolation.Source: §D.2 · pp. 24–25

D.3 RECEIVER OPERATING CHARACTERISTIC AUC (ROC AUC)

  • ROC-AUC summarizes the curve of true-positive rate versus false-positive rate, but can remain high when early enrichment is modest.It is treated as a secondary legacy metric and reported in the appendix.Source: §D.3 · p. 25

D.4 BEDROC

  • BEDROC is a rank-based metric that exponentially weights early-retrieved actives, with larger α placing more weight on the top of the ranking.The metric is used as an additional early-enrichment summary in the appendix.Source: §D.4 · p. 25

D.5 AGGREGATION ACROSS TARGETS AND SEEDS

  • Metrics are averaged over three training seeds per target, then summarized across targets so larger test sets do not dominate.Main tables report means and standard deviations; appendix results also include medians, interquartile ranges, and protein-class averages.Source: §D.5 · p. 25

E.1 PER TARGET LBVS TASK

  • Each target defines a separate ranking task, with scores aggregated across targets; the panel spans seven protein classes and libraries from hundreds to over ten thousand compounds.Targets labeled “Miscellaneous” in ChEMBL were excluded because their class coherence was insufficient for meaningful grouping.Source: §E.1 · p. 26

E.2 SETTING 1: CHEMBL∗TO TOPU (MAIN BENCHMARK)

  • Setting 1 trains on ChEMBL∗ after excluding target-specific TopU compounds, then tests on the full TopU library using EF@1% as the primary metric.Training and validation use a 1:10 active-to-decoy ratio; the TopU test library uses 1:40, and all models share fixed splits.Source: §E.2 · p. 26

E.3 SETTING 2: TOPU ONLY (FEW SHOT)

  • Setting 2 trains, validates, and tests within each target’s TopU library to assess few-shot learning under the hard-negative distribution.Scaffold-based splits use 6:2:2 or 7:1:2 active allocations by tier; EF@10% is primary because EF@1% is unstable for the smallest libraries.Source: §E.3 · p. 26, Table 10 · p. 27

E.4 SETTING 3: CHEMBL∗ONLY (ABLATION)

Setting 3 contrasts a ChEMBL∗-only random-negative evaluation with the harder TopU test libraries in Setting 1. Its controlled construction holds actives and training data constant while changing test inactives.

E.4 SETTING 3: CHEMBL∗ONLY (ABLATION)

  • Setting 3 is a seven-target ablation using ChEMBL∗ training and test data with randomly sampled negatives and no TopU decoy curation.The targets represent one protein class each and were selected as those with the largest number of TopU actives in their class.Source: §E.4 · p. 27, Table 11 · p. 27
  • Training and validation match Setting 1 in size, split, and active-to-decoy ratio, with TopU compounds excluded from the training pool.The remaining ChEMBL∗ compounds use the same 85%/15% stratified split and seed 2026.Source: §E.4 · p. 27
  • The test comparison preserves the same actives and 1:40 ratio but changes inactives from focused TopU hard decoys to a shuffled mixture of ChEMBL∗background inactives and TopU compounds.TopU compounds are typically under 10% of Setting 3 test inactives, making the reported Setting 1-versus-3 comparison conservative; EF@1% is the primary test metric in both settings.Source: §E.4 · p. 27, Table 12 · p. 28

F LIMITATIONS AND SCOPE

TopU-LBVS prioritizes controlled stress-testing of ligand-based screening under hard-negative conditions, with scope boundaries that limit how broadly its results should be interpreted. The authors position it as a complement to existing benchmarks and a retrospective evaluation resource, not deployment evidence.

F LIMITATIONS AND SCOPE

  • The benchmark covers 2D ligand-based representations and a fixed ChEMBL 35 target panel, not protein-structure, docking, 3D, or hybrid ligand–protein screening.Conclusions may not directly generalize to underrepresented target families or proprietary chemical spaces.Source: F LIMITATIONS AND SCOPE · p. 28
  • Its intentionally adversarial decoys may overstate difficulty for early exploratory screening with more diverse libraries.Labels are binarized from curated public bioactivity data using fixed thresholds, so assay context, experimental uncertainty, and graded potency are not captured.Source: F LIMITATIONS AND SCOPE · p. 28

G BROADER IMPACTS

  • The benchmark is retrospective and does not establish that any model is ready for clinical or experimental deployment.The authors describe its primary impact as improving scientific rigor and reproducibility in molecular machine-learning benchmarks.Source: G BROADER IMPACTS · p. 28

H ADDITIONAL RESULTS

The appendix organizes per-target and per-model results for the TopU-LBVS protocols, with primary-metric tables serving as the reference for comparisons. It also supplies full metric tables for the full and few-shot settings.

H.1 PART I: PRIMARY-METRIC PER-TARGET RESULTS

  • The appendix provides per-target primary-metric tables for the three protocols as canonical references for comparing new methods with baselines.The index in Table 13 directs readers to the per-target result tables and their locations.Source: H ADDITIONAL RESULTS · pp. 28–29, Table 13 · p. 29
  • Tables 14–15 report TopU-LBVS-full EF@1% across 93 targets and 10 models, while Tables 16–17 report few-shot EF@10% grouped by protein class and tier.Tables 18–19 give mini-protocol results under TopU hard decoys and random ChEMBL∗ decoys, allowing per-target and per-model comparison of decoy difficulty.Source: §H.1 · p. 29

H.2 PART II: FULL PER-MODEL METRICS, TOPU-LBVS-FULL

  • The full-setting per-model tables report seven metrics for each of ten baselines across all 93 targets under ChEMBL∗→TopU.The tables are split into pairs per model, following the main-paper class ordering.Source: §H.2 · p. 29

H.3 PART III: FULL PER-MODEL METRICS, TOPU-LBVS-FEW

  • The few-shot per-model tables report all seven evaluation metrics under TopU→TopU, with targets grouped by class and tiers indicating the active-count split strategy.Tier 1 uses a 6:2:2 split for fewer than 50 actives; Tier 2 uses 7:1:2 for at least 50.Source: §H.3 · p. 32, Table 17 · p. 33

LogAUC

  • Table 31 reports part-two results for the TopU-LBVS-full GIN+FP hybrid, with EF, PR-AUC, ROC-AUC, BEDROC, and LogAUC among the reported metrics.The caption states that values are mean ± standard deviation over three random seeds and that PR-AUC, ROC-AUC, BEDROC, and LogAUC use a 0–100 scale.Source: Table 31 · p. 47

Related papers

Loading 2609.29740v1…