PaperlayerView exact revision on arXiv

Source-linked AI summary

Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development

Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W. Coley, Cao Xiao, Jimeng Sun, Marinka Zitnik

arXiv:2102.09548v2cs.LGcs.CYq-bio.BMq-bio.QM

TL;DR

Therapeutics machine learning needs meaningful learning tasks and carefully curated datasets, but existing resources do not systematically cover the field. TDC unifies AI-ready datasets, tasks, tools, evaluation strategies, and leaderboards across therapeutics; experiments show that prevailing methods still struggle with realistic distribution shifts and generalization challenges.

  • Problem

    Therapeutics machine learning lacks systematically formulated learning tasks and carefully curated, AI-ready datasets spanning drug discovery and development.

  • Method

    TDC integrates 66 datasets across 22 learning tasks with data functions, realistic splits, evaluation strategies, molecule-generation oracles, leaderboards, and a Python package.

  • Results

    Across 24 benchmarks, domain-specific methods often perform better than or comparably to state-of-the-art models, while key therapeutics challenges remain unsolved.

  • Takeaways & Limitations

    TDC provides a shared framework for domain and machine-learning scientists to develop, validate, and compare therapeutics models.

  • Takeaways & Limitations

    Drug–drug interaction models must extrapolate to unobserved combinations and account for dosage, which can substantially affect outcomes.

Abstract

from arXiv · show

Therapeutics machine learning is an emerging field with incredible opportunities for innovatiaon and impact. However, advancement in this field requires formulation of meaningful learning tasks and careful curation of datasets. Here, we introduce Therapeutics Data Commons (TDC), the first unifying platform to systematically access and evaluate machine learning across the entire range of therapeutics. To date, TDC includes 66 AI-ready datasets spread across 22 learning tasks and spanning the discovery and development of safe and effective medicines. TDC also provides an ecosystem of tools and community resources, including 33 data functions and types of meaningful data splits, 23 strategies for systematic model evaluation, 17 molecule generation oracles, and 29 public leaderboards. All resources are integrated and accessible via an open Python library. We carry out extensive experiments on selected datasets, demonstrating that even the strongest algorithms fall short of solving key therapeutics challenges, including real dataset distributional shifts, multi-scale modeling of heterogeneous data, and robust generalization to novel data points. We envision that TDC can facilitate algorithmic and scientific advances and considerably accelerate machine-learning model development, validation and transition into biomedical and clinical implementation. TDC is an open-science initiative available at https://tdcommons.ai.

1 Introduction

Therapeutics machine learning could accelerate drug discovery and development, but progress is hindered by fragmented, non-AI-ready data and difficult real-world learning settings. TDC addresses these challenges with an integrated platform of datasets, tasks, evaluation tools, and community resources.

  • Drug development takes 13–15 years and US$2–3 billion on average, while approved drugs per dollar spent have remained flat or decreased.
  • Limited AI-ready datasets and standardized knowledge representations make therapeutic questions difficult to formulate as solvable machine-learning tasks.
  • TDC provides 66 datasets across 22 learning tasks, plus evaluation strategies, molecule-generation oracles, data processors, and 29 leaderboards through a Python package.
  • TDC spans therapeutic products and development stages while supporting realistic data splits, evaluation metrics, and performance leaderboards for real-world model assessment.
  • Evaluation across 24 benchmarks finds therapeutics tasks challenging: domain-specific methods often match or outperform state-of-the-art models, exposing opportunities for algorithmic innovation.
  • TDC benchmarks support study of low-resource learning, heterogeneous modalities, distribution shifts, and robust generalization to unseen data.

2 Related Work

TDC unifies therapeutics datasets and learning tasks across drug discovery and development. Unlike repositories or model-focused toolkits, it emphasizes broad, systematic benchmarking across relevant modalities and therapeutic questions.

  • TDC is presented as the first unifying platform of datasets and learning tasks for drug discovery and development.
  • Existing repositories curate specialized resources such as binding affinities, bioassays, therapeutic targets, and biological networks.
  • MoleculeNet and TAPE provide focused benchmark coverage, whereas TDC broadly covers modalities relevant to therapeutics.
  • Unlike model-driven toolkits such as DeepChem, DeepPurpose, OpenChem, and ChemML, TDC focuses on datasets and related infrastructure.

3 Overview of TDC

TDC combines meaningful therapeutics learning tasks and curated datasets with tools, community resources, and leaderboards. Its modular ecosystem supports processing, model development, evaluation, and fair comparison under realistic settings.

  • TDC has three components: task-linked datasets, tools and community resources, and leaderboards for fair model comparison and benchmarking.
  • AI-ready datasets and learning tasks: The platform covers tasks from wet-lab target identification through biomedical product manufacturing and is updated with antibody and gene-editing datasets.
  • AI-ready datasets and learning tasks: TDC curates and processes rich feature representations for biomedical entities to provide machine-learning-ready datasets.
  • Tools and community resources: Its evaluation toolkit includes 23 strategies and five split types designed to assess performance and generalization under real-world learning settings.
  • Leaderboards: TDC provides 17 molecule-generation oracles and 29 leaderboards grouped around therapeutic questions with carefully selected datasets, splits, and metrics.
  • Tools and community resources: TDC supplies data-processing functions for format conversion, visualization, binarization, balancing, unit conversion, querying, and molecule filtering.

4 Organization of TDC

TDC organizes therapeutics machine learning hierarchically into problems, learning tasks, and datasets, spanning diverse therapeutic products and discovery pipelines. It provides dataset metadata, recommended evaluation practices, and resources for benchmarking and implementation.

  • Tiered and modular design: TDC uses three tiers: broad machine-learning problems, learning tasks, and datasets.The first tier contains single-instance prediction, multi-instance prediction, and generation.
  • Diverse learning tasks: The platform currently covers 22 learning tasks spanning small molecules, antibodies, peptides, microRNAs, and gene editing.Tasks map to pipelines including target discovery, activity modeling, efficacy and safety, and manufacturing.
  • Machine-learning-ready datasets: Each task is instantiated through multiple datasets with splits designed to simulate generalization needed for production and clinical implementation.Examples include generalization to unseen compounds and resolving patient responses to polytherapy.
  • Machine-learning-ready datasets: TDC organizes datasets and tasks with recommended splits, metrics, and units to support systematic benchmarking across therapeutics machine-learning problems.Table 2 identifies dataset size, feature type, task type, metric, and recommended split.

5 Single-Instance Learning Tasks in TDC

TDC’s single-instance tasks predict properties of individual biomedical entities, with ADME serving as a major small-molecule application. The associated datasets cover absorption, distribution, metabolism, and related physicochemical properties using scaffold-aware evaluation.

  • ADME property prediction: ADME prediction estimates absorption, distribution, metabolism, and excretion properties from a drug candidate’s structure.These properties describe how a small-molecule drug travels through and exits the body.
  • ADME property prediction: Poor ADME profiles are the most prominent reason for clinical-trial failure, making early accurate profiling necessary for small-molecule development.The passage identifies early ADME profiling as a necessary condition for successful development.
  • ADME property prediction: ADME models should generalize to structurally distant unseen drugs as drug structures evolve over time.Scaffold splits approximate this requirement by separating structurally distant training and test molecules.
  • ADME datasets: The listed datasets generally recommend scaffold splits, with AUROC for classification and MAE for numeric endpoints.Examples include Caco-2 permeability evaluated by MAE in cm/s and HIA evaluated by AUROC.

5.2 single_pred.Tox: Toxicity Prediction

TDC’s toxicity task predicts diverse adverse effects of drug molecules, motivated by toxicity-driven attrition and the value of early screening. Its datasets span acute, cardiac, genetic, hepatic, dermatological, carcinogenic, and clinical toxicity endpoints.

  • Toxicity prediction: Toxicity prediction aims to estimate multiple types of adverse effects that drug molecules may cause in humans.The task concerns toxicity toward human organisms.
  • Toxicity prediction: Approximately 70% of toxicity-related attrition occurs preclinically, supporting early toxicity prediction during development.The passage states that preclinical toxicities are strongly predictive of toxicities in humans.
  • Toxicity prediction: Toxicity models are expected to generalize to novel drugs with small structural similarity to existing drugs.Drug structures of interest evolve over time, creating a need for extrapolation beyond the existing drug set.
  • Toxicity datasets: The toxicity datasets generally recommend scaffold splits and use MAE for LD50 regression or AUROC for classification endpoints.LD50 contains 7,385 drugs, while hERG contains 648 drugs and is evaluated with AUROC.
  • High-throughput screening: High-throughput screening predicts experimental assay values from small-molecule structures, but screening large libraries remains time- and cost-limited.Machine learning can narrow chemical space to candidates for smaller-scale screening.

5.4 single_pred.QM: Quantum Mechanics

TDC’s quantum-mechanics task targets molecular properties whose ab initio calculation is computationally expensive, using machine learning to approximate them more efficiently. Its datasets include electronic, energetic, geometric, and thermodynamic measurements, while related tasks address synthesis yield and antibody engineering.

  • Quantum mechanics: Quantum-mechanics datasets support learning molecular properties that are expensive to calculate directly with ab initio many-body methods.Approximate quantum calculations trade accuracy against computational speed, motivating machine-learning models.
  • Quantum mechanics: A well-trained model can describe potential energy surfaces quickly enough to enable more accurate and longer molecular simulations.Such simulations can reveal molecular-level biological processes and protein-target function.
  • Quantum mechanics: Quantum-mechanics models must extrapolate from calculated compounds to unseen or structurally diverse compounds.This is stated as the generalization requirement for the task.
  • Quantum-mechanics datasets: QM7b contains 7,211 molecules with 14 properties, including polarizability, HOMO and LUMO eigenvalues, and excitation energies.The molecules use a 3D Coulomb matrix representation and the dataset recommends a random split.
  • Quantum-mechanics datasets: QM8 contains 21,786 small organic molecules with electronic spectra and excited-state energies computed by multiple quantum-mechanical methods.The dataset uses 3D Coulomb matrices, random splits, and MAE in eV.
  • Quantum-mechanics datasets: QM9 contains 133,885 stable CHONF molecules annotated with geometric, energetic, electronic, and thermodynamic properties.Its labels include geometries, harmonic frequencies, dipole moments, polarizabilities, and thermodynamic energies.

5.7 single_pred.Epitope: Epitope Prediction

TDC frames epitope prediction as classifying active and non-active sites from antigen protein sequences, with generalization to unseen pathogen antigens. The section also situates related antibody developability and gene-editing prediction tasks within therapeutic efficacy and safety.

  • Epitope Prediction: Epitope prediction classifies active and non-active sites from antigen protein sequences.The task concerns epitopes, regions recognized by antibodies that can trigger adaptive immune responses.
  • Epitope Prediction: Potential epitope identification supports vaccine design, antibody development, and understanding of the immune system.
  • Epitope Prediction: Models are expected to generalize to unseen pathogen antigens with diverse structures and functions.
  • Antibody Developability: Antibody developability prediction uses amino acid sequences to identify characteristics that can prevent therapeutic development.The listed negative characteristics include immunogenicity, instability, self-association, high viscosity, polyspecificity, and poor expression.
  • Gene Editing: CRISPR-Cas9 repair-outcome prediction targets therapeutic genome editing, with expected generalization across cell lines and reagent delivery methods.The dataset contains outcomes from knockout experiments on primary CD4+ T cells from 15 donors.

6 Multi-Instance Learning Tasks in TDC

TDC’s multi-instance learning tasks cover heterogeneous biomedical relationships, including drug-target, drug-drug, protein-protein, gene-disease, and drug-response prediction. These tasks emphasize extrapolation to unseen entities or combinations while supporting therapeutic discovery and safety applications.

  • Drug-Target Interaction: Drug-target interaction prediction estimates activity from compound structural information and protein amino acid sequences.Accurate affinity prediction can reduce high-throughput screening and enlarge the search space for candidate compounds.
  • Drug-Target Interaction: Models must extrapolate across unseen compounds, proteins, and compound-protein pairs while maintaining performance across disease and target groups.
  • Drug-Drug Interaction: Drug-drug interaction tasks predict interactions or side effects from pairs of drugs, addressing adverse outcomes associated with polypharmacy.The DrugBank_DDI dataset contains 191,808 pairs across 1,706 drugs and 86 interaction types.
  • Protein-Protein Interaction: Protein-protein interaction prediction uses paired amino acid sequences to infer activity where wet-lab measurements are expensive and many interactions remain unknown.
  • Gene-Disease Association: Gene-disease association prediction models links between genes and diseases to help identify potential therapeutic targets.TDC’s DisGeNET dataset integrates curated association sources and maps genes to amino acid sequences.
  • Drug Response: Drug-response prediction estimates response rates from a drug and a cell line’s genomics profile, reflecting variation across patients and cancer cell lines.

7 Generative Learning Tasks in TDC

TDC’s generative learning tasks address molecule generation and reaction prediction for discovery and manufacturing. They pair desired molecular properties or reaction products with expectations of novelty, synthetic feasibility, and generalization beyond training examples.

  • Molecule Generation: Molecule generation learns molecular characteristics evaluated by oracle functions and produces novel candidates with desirable chemical properties.
  • Molecule Generation: Generating molecules with predefined optimal properties can expand beyond existing libraries, where high-throughput screening is restricted.
  • Molecule Generation: Generated molecules are expected to combine superior properties with synthesizability and low off-target effects across structurally diverse drugs.
  • Datasets: TDC includes MOSES, ZINC, and ChEMBL molecular datasets, with dataset sizes ranging from 249,455 to 1,961,462 molecules.MOSES contains 1,936,962 molecules; ZINC contains 249,455; and ChEMBL contains 1,961,462.
  • Retrosynthesis: Retrosynthesis prediction recursively transforms target molecules into simpler precursor molecules until commercially available starting materials are identified.The task supports synthetic-route design for novel molecules and is associated with manufacturing.
  • Reaction Outcome Prediction: Reaction outcome prediction generates reaction products from reactants and serves as the reverse process of retrosynthesis prediction.The task aims to automate a time-consuming and expensive process that traditionally relies on experimentation and experienced chemists.

8 TDC Data Functions

TDC provides integrated functions for evaluating models, constructing meaningful data splits, processing datasets, and scoring generated molecules. These utilities operationalize task-specific benchmarking, including realistic generalization tests and more substantive molecule-generation evaluation.

  • Model Evaluation: TDC provides evaluators with established performance measures and additional metrics for biology and chemistry.
  • Model Evaluation: Regression, binary classification, multi-class classification, multi-label classification, and token-level classification each receive task-appropriate metrics.Examples include MSE, MAE, R2, AUROC, AUPRC, Micro-F1, Macro-F1, Cohen’s Kappa, and Avg-AUROC.
  • Molecule Evaluation: Molecule-generation metrics assess distributional properties through diversity, KL divergence, FCD, novelty, validity, and uniqueness.These metrics respectively capture molecular dissimilarity, distributional similarity or diversity, reference-set distance, training-set exclusion, structural validity, and duplicate generation.
  • Data Splits: Random, scaffold, cold-start, and combinatorial splits provide increasingly structured tests of molecular, entity-level, and combination-level generalization.Cold-start splits separate one entity type before moving associated pairs, while combinatorial splits create disjoint drug combinations.
  • Molecule Generation Oracles: TDC wraps 17 molecule-generation oracles, including GuacaMol, heuristic, docking, and retrosynthetic-pathway-based evaluations.The platform adds more realistic oracles because simplistic objectives can be easy to optimize or yield unrealistic molecules.

9 TDC’s Tools, Libraries, and Resources

TDC provides an accessible ecosystem of Python tools, APIs, documentation, collaboration infrastructure, and evaluation resources for machine-learning workflows.

  • Ecosystem: TDC offers flexible tools, libraries, and community resources that support workflows from model building and training through deployment.These resources are designed for both beginners and experts.
  • Accessibility and collaboration: TDC can be installed through the Python Package Index and is publicly hosted on GitHub for collaboration and development.The project provides source-control support for bug fixes and new functionality.
  • API design: A three-layer API organizes access to learning tasks and datasets through the modular Problem–Learning Task–Data Set structure.The hierarchy allows new tasks and datasets to be incorporated readily.
  • Dataset access: Users specify a problem class, task, and dataset, after which TDC retrieves a processed machine-learning-ready dataset and associated data object.The data object provides utility functions for operations such as generating train, validation, and test splits.
  • Evaluation and maintenance: TDC provides one-line access to evaluators, benchmark groups, molecule-generation oracles, and benchmark evaluation.The package also supplies documentation, tutorials, compatible environments, and continuous integration for usability and sustainability.

10 TDC Leaderboards and Experiments on Selected Datasets

TDC benchmarks evaluate therapeutics models under realistic discovery and development settings, including scaffold, temporal, and limited-oracle constraints. Across ADMET, DTI, and molecular generation tasks, results expose substantial weaknesses in current methods and motivate realistic benchmarking.

  • Benchmarking framework: TDC benchmarks and leaderboards support reproducible systematic model development and evaluation across therapeutic questions.The experiments and evaluation procedures are publicly accessible for reproduction.
  • ADMET benchmark: The ADMET benchmark contains 22 datasets and uses scaffold splitting to test generalization to structurally distant, unseen drugs.It covers endpoints including CYP metabolism, half-life, clearance, and off-target effects.
  • ADMET results: 13 endpoints favored pretrained GIN strategies, while RDKit2D descriptors achieved the best results on five endpoints and SMILES CNN on one.Performance varied by feature type; on TDC.CYP3A4-S, SMILES CNN was 8.7%-14.9% better than graph-based methods.
  • DTI domain generalization: Out-of-distribution DTI performance degraded by 33.9% to 43.6% across methods despite in-distribution performance reaching 0.7 PCC.MMD and CORAL performed best, but standard training had similar performance to current domain-generalization methods.
  • Docking generation: Under limited oracle budgets, nearly all molecule-generation methods failed to outperform best-in-data until 5,000 calls, when Graph-GA and LSTM surpassed it.Graph-GA achieved -14.811 and LSTM -13.017 at 5,000 oracle calls.

11 Conclusion and Future Directions

TDC is an open-science platform that supports therapeutics machine-learning research through curated datasets, evaluation tools, leaderboards, and community resources. Future development will expand its coverage, tooling, competitions, and participation to support algorithmic and scientific innovation.

  • TDC provides AI-ready datasets, systematic model-development and evaluation strategies, tools, leaderboards, and community resources for therapeutics research.
  • New Learning Tasks and Datasets: TDC’s continued development will add learning tasks and datasets for emerging products and pipelines, including ADCs, PROTACs, clinical trial design, drug delivery, and postmarketing safety.
  • New ML Tools: Additional data functions, tools, libraries, and community resources are planned for future TDC releases.
  • New Leaderboards and Competitions: New leaderboards will target therapeutics tasks that interest the community and could benefit from advanced machine learning.
  • TDC is an open-science initiative that welcomes contributions from the research community.
Loading 2102.09548v2…