PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery

Hakime Öztürk, Arzucan Özgür, Philippe Schwaller, Teodoro Laino, Elif Ozkirimli

arXiv:2002.06053v1q-bio.BMcs.CLcs.LGstat.ML

TL;DR

Predicting highly specific protein–compound interactions remains difficult, and computational approaches can be constrained by resources or 3D information. This review surveys NLP applications to biochemical text, including molecular generation and druglikeness-related findings.

  • Problem

    Highly specific protein–compound interactions remain difficult to predict, while some computational approaches require extensive resources or are limited by 3D information.

    Source: §1 · p. 2

  • Method

    The review outlines how NLP methods have influenced bioinformatics and cheminformatics and surveys text-based approaches to exploring chemical space.

    Source: §1 · p. 4, §2.4 · p. 17

  • Results

    The review describes LSTM-based de novo molecule generation and reports that SD-VAE provided better discriminative features for druglikeness.

    Source: §2.4 · pp. 18–19

  • Takeaways & Limitations

    Text-generation models provide a way to explore chemical space by learning distributions over biochemical text representations.

    Source: §2.4 · pp. 17–18

    Biological and chemical interpretability and explainability remain major challenges for AI-based approaches.

    Source: §1 · pp. 2, 4

Abstract

from arXiv · show

Text-based representations of chemicals and proteins can be thought of as unstructured languages codified by humans to describe domain-specific knowledge. Advances in natural language processing (NLP) methodologies in the processing of spoken languages accelerated the application of NLP to elucidate hidden knowledge in textual representations of these biochemical entities and then use it to construct models to predict molecular properties or to design novel molecules. This review outlines the impact made by these advances on drug discovery and aims to further the dialogue between medicinal chemists and computer scientists.

1. Introduction

The review examines how NLP can use unstructured biochemical text to explore chemical space for drug discovery. It frames these methods against challenges in predicting protein–compound interactions and limitations of existing computational approaches.

1. Introduction

  • Protein–compound interactions remain difficult to predict, while computational strategies help narrow compounds for experimental testing.Structure-based approaches include docking and molecular dynamics; ligand-based approaches include QSAR and molecular similarity.Source: §1 · p. 2
  • Text-based biochemical representations are more readily available, and NLP methods can bring latent knowledge in textual data to the fore.The review describes proteins, chemicals, and natural language as distinct biochemical language categories.Source: §1 · pp. 2–3
  • Biological and chemical interpretability and explainability remain major AI challenges, while data access, interoperability, and reusability matter for cross-disciplinary model sharing.The review also notes that some machine-learning and deep-learning approaches require extensive computational resources or are limited by 3D information availability.Source: §1 · pp. 2–4
  • The review surveys NLP's influence on bioinformatics and cheminformatics, focusing on unstructured biochemical text and its use in exploring chemical space for drug discovery.It also highlights an interdisciplinary approach integrating computer science and natural sciences.Source: §1 · pp. 3–4

1.1. NLP Basics

  • The review follows NLP's word, sentence, and context levels to organize biochemical applications, including understanding text and generating meaningful sequences.NLP's levels cover meaningful units, sentence grammar and syntax, and global sentence meaning.Source: §1.1 · p. 4
  • It also summarizes the machine-learning and deep-learning architectures discussed throughout the review.Source: §1.1 · p. 4

2. Biochemical Language Processing

This section reviews biochemical sequences and text-based representations of chemicals and proteins as language-like data, including tokenization, similarity analysis, classification, and molecular or protein modeling. It also surveys NLP-based generation and translation methods for exploring chemical space and adapting molecules, alongside reported applications and representation trade-offs.

2.1. Textual Chemical Data

  • The review focuses on 1D chemical representations for NLP, while noting that 2D and 3D forms are also used in machine-learning approaches.Chemical structures can be represented in 1D, 2D, or 3D; 1D textual forms are the focus here.Source: §2.1 · p. 5
  • Text-based SMILES describes molecular structures and reactions; strings from traversing a compound’s 2D graph provide more information than its chemical formula, and its textual form takes 50% to 70% less space than other representation methods.Source: §2.1 · p. 6
  • DeepSMILES was reported as more effective than SMILES at identifying undetectable patterns in drug–target binding-affinity prediction [31].The passage does not provide a quantitative comparison.Source: §2.1 · pp. 6–7
  • In molecule generation, DeepSMILES was suggested to limit SMILES-based models because its grammar-sensitive syntax can produce longer sequences [32].The comparison included canonical and randomized SMILES.Source: §2.1 · p. 7
  • SELFIES uses symbol sequences built on semantically constrained graphs, and was compared with SMILES and DeepSMILES for validity under random mutations on QM9 [33].The supplied passage reports that the evaluation favored a representation but does not identify it.Source: §2.1 · p. 7
  • A translation model converted InChI to canonical SMILES but failed in the reverse direction; Winter et al. attributed the difficulty to InChI’s complex syntax.The passage also notes that SMILES’ less complex structure makes it easier to use in molecular generation and translation-based representation building.Source: §2.1 · p. 7
  • SMARTS uses specialized symbols and logic operators for substructure (pattern) searches on SMILES strings, including database queries and rule-dictionary creation such as RECAP and BRICS for fragment extraction,,.Source: §2.1 · p. 8
  • Chemical-reaction classification systems learned directly from SMILES text using neural-network transforms.Source: §2.1 · p. 8

2.2. Identification of Words/Tokens

  • Because biochemical words are not explicitly known, distinct approaches are needed to extract syntactically and semantically meaningful units from chemical and protein sequences.The review frames identifying these units as a way to make bio- and cheminformatics problems easier to model.Source: §2.2 · p. 8
  • SMILES-based chemical similarity matched 2D-based similarity while running faster, and longest common subsequences performed similarly to 4-mers.The SMILES-based approach did not require 2D or 3D information.Source: §2.2 · pp. 9–10
  • For proteins, 4-mers performed best among k-mers, segmented sequences beat whole-sequence input, and BPE bio-words improved accuracy over 3-mers.The segmented inputs were long k-mers with amino-acid gaps; conserved subsequences were highlighted.Source: §2.2 · pp. 9, 11
  • Authors argued MCSs (fragments) can be described as words of chemical language; a study found these words followed Zipf’s Law across molecule subsets, and drug “words” were shorter than natural-product “words”.Source: §2.2 · p. 10
  • MDL-based protein words compressed less-conserved residues less, and the authors emphasized incorporating hydrophilic and hydrophobic amino-acid knowledge into grammar building.In MDL, a protein can be assigned to the family in which its sequence is compressed most.Source: §2.2 · p. 10
  • Yu et al. suggested that domains displayed syntactic and semantic features, but only a few multi-domain proteins have more than six domains, limiting use of domains as words to build sentences.Domains and motifs have also been used as words in drug-discovery tasks, including drug-target interaction affinity prediction [63]; studies showed that motifs and domains together contribute to prediction.Source: §2.2 · p. 11
  • SMARTS-defined substructures form binary fingerprints, while 8-mers, MACCS, and ECFP6 ligand representations showed similar performance in protein-family clustering.Source: §2.2 · p. 12
  • A comprehensive evaluation of word-extraction techniques was lacking beyond Wang et al.’s comparison of BPE-based words and k-mers for PPI prediction.Source: §2.2 · p. 12

2.3. Text representation

  • SMIfp achieved results comparable to a representation incorporating polar-group, topological, atom, and bond information in recovering active compounds.SMIfp represents molecules using frequencies of unique SMILES symbols for ligand-based virtual screening.Source: §2.3 · p. 13
  • TF-IDF-weighted LINGOs showed similar performance to graph-based chemical similarity for drug-target interaction prediction, and higher-scoring bonds had a higher probability of breaking.Source: §2.3 · p. 14
  • One-hot encoding is fast to build but can produce sparse, high-dimensional vectors when vocabularies are large.Source: §2.3 · p. 14
  • Unlike one-hot vectors, which assign zero similarity to distinct words, distributed representations use context so words occurring in similar contexts have similar vectors.Word2Vec builds context-based vectors; Skip-Gram predicts context words from a center word, while CBOW predicts a target from context words.Source: §2.3 · pp. 14–15
  • Word2Vec embeddings of protein 3-mers supported protein-class determination and PPI prediction, with 3-mers sharing biophysical and biochemical properties clustering together.Source: §2.3 · p. 15
  • Word2Vec embeddings of PFAM protein domains showed that semantically similar domains share similar Gene Ontology terms.The study treated each domain as a word and investigated embeddings through inherited GO annotations.Source: §2.3 · pp. 15–16
  • SMILESVec ligand-based protein representations performed comparably to sequence-based representations in protein family/superfamily clustering.SMILESVec embeds chemical 8-mers from SMILES, and proteins are represented by averaging vectors of their interacting ligands.Source: §2.3 · p. 16
  • Mol2Vec outperformed the simple Morgan fingerprint for solubility prediction and performed comparably to graph-based chemical representation.Mol2Vec learns representations from substructure identifiers extracted via ECFP.Source: §2.3 · p. 16
  • Distributed fragment-based ligand representations outperformed fragment-based binary vectors in ring-system clustering and performed comparably in Tetrahymena toxicity prediction.The fragments were extracted from SMILES using a graph-traversing algorithm.Source: §2.3 · p. 16
  • CheMixNet, combining SMILES-trained deep-learning representations with MACCS fingerprints, outperformed single-data representations such as SMILES2Vec and Chemception [90].The comparison investigated chemical-property prediction [89].Source: §2.3 · p. 17

2.4. Text generation

  • Text-generation models learn from SMILES to propose novel molecules with properties resembling existing compounds, which can then be assessed for properties or target binding.RNNs face challenges producing valid SMILES and may miss long-range dependencies; the non-context-free nature of SMILES also complicates VAE generation.Source: §2.4 · pp. 17–18
  • LSTM models pioneered de novo generation of valid novel SMILES, and transfer learning generated structurally similar leads for targets with few known ligands [94].LSTM models address the long-term dependency challenge of matching rings and branch closures.Source: §2.4 · p. 18
  • Target-focused methods generated target-specific molecules by predicting bioactivity, filtering inactive candidates, and retraining LSTMs; reinforcement learning biased models toward desired properties, while fine-tuning active target-focused libraries led to synthesis of novel compounds.Source: §2.4 · p. 18
  • A grammar variational autoencoder defined SMILES grammar rules to address invalid strings and generate grammatically valid sequences [103].Syntactic validity alone does not ensure chemical validity.Source: §2.4 · p. 19
  • SD-VAE generated syntactically and semantically valid SMILES, and its latent representations provided better druglikeness-discriminative features than those of VAE and GVAE.Its syntax-directed decoder was designed to address the gap between syntactic and chemical validity.Source: §2.4 · p. 19
  • Seq2seq models using SMILES for reaction prediction could compete with graph neural network-based models.The review also describes seq2seq approaches to predict products from precursors and, in reverse, reactants from products and reaction classes.Source: §2.4 · p. 21
  • In ligand-based virtual screening, InChI-to-SMILES latent representations statistically significantly improved on fingerprint methods such as ECFP.The translation task was used to learn representations integrating the molecule’s semantic meaning.Source: §2.4 · p. 21
  • An RNN encoder-decoder translating protein-sequence words into GO identifiers achieved performance comparable to state-of-the-art protein-function methods.The authors argued that more meaningful, biologically interpretable “words” could improve performance.Source: §2.4 · p. 21
  • The atom-mapping-independent Molecular Transformer outperformed other algorithms, including a two-step convolutional graph neural network, on commonly used reaction-prediction benchmarks.The review describes this as the first adoption of the Transformer architecture in cheminformatics.Source: §2.4 · p. 22
  • The proposed text-generation systems either outperformed state-of-the-art systems or obtained comparable results.Source: §2.4 · p. 22

3. Future Perspectives

The future perspectives section considers challenges in evaluating, reproducing, and interpreting bio/cheminformatics amid expanding biochemical data, computational power, and NLP methods, alongside opportunities for drug discovery. It presents NLP and machine-learning advances as promising for long-standing problems and encourages interdisciplinary uptake and communication.

3.1. Challenges

  • Different datasets and evaluation metrics complicate model assessment, and comprehensive benchmarking platforms for drug-discovery tools remain lacking.The review identifies universal benchmarks and metrics as a major need.Source: §3.1 · p. 23
  • Despite dataset and code sharing, reproducing other groups’ methods remains difficult.The review points to FAIR data principles, accessible automated workflows, and platform-free solutions as ways to support reproducibility.Source: §3.1 · p. 23
  • Biased sampling limits exploration: interaction data favor common-disease targets and known-actives-like molecules, while rare-disease data and inactive compounds are scarce and measurements may not reproduce across labs or conditions.The review describes these sampling and data-quality problems as sources of bias.Source: §3.1 · pp. 23–24
  • Because black-box predictions are hard to interpret, explainability is important for medicinal chemists’ use of findings and experts’ assessment of model reliability.An attentive pooling network is one approach used for drug-target interaction prediction.Source: §3.1 · p. 24

3.2. Opportunities

  • AlphaFold’s success in CASP highlights the potential of context-focused models to capture dependencies between residues distant in sequence but close in 3D.The review links this potential to advances in machine- and deep-learning systems and protein-folding prediction.Source: §3.2 · p. 25
  • Attention-based and pretrained language models, encoder-decoder architectures, Topic-Guided VAE [91], and graph transformers [148] offer routes to molecule generation and broader bio/cheminformatics applications.Source: §3.2 · p. 25
  • Multi-task learning with shared-parameter DNNs can draw information across related tasks, while transferability and domain-general NLP models may support generalization.Source: §3.2 · p. 26
  • Multi-task learning with contextual word representations may help address interwoven drug-discovery tasks, including chemical-property prediction and molecule generation.The review frames these tasks as opportunities for applying NLP methodologies.Source: §3.2 · p. 26
  • The review describes NLP as already integrated into biochemical-text processing, with AI-based methods both driving and benefiting from NLP.The applications discussed offer a broad perspective on this integration.Source: §3.2 · p. 26
  • Recent NLP and machine-learning advances are providing promising results for long-standing bio/cheminformatics problems.The review summarizes NLP’s impact in the field to encourage researchers to take advantage of these advances.Source: §3.2 · p. 26
  • Establishing a common vocabulary around shared goals can improve communication among researchers from different backgrounds and domains.The review presents itself as an attempt to facilitate this interdisciplinary conversation.Source: §3.2 · p. 26

Related papers

Loading 2002.06053v1…