Source-linked AI summary
MolTrans: Molecular Interaction Transformer for Drug Target Interaction Prediction
Kexin Huang, Cao Xiao, Lucas Glass, Jimeng Sun
TL;DR
DTI prediction is costly and remains limited by whole-molecule representations that overlook relevant substructures and by restricted use of labeled data. MolTrans combines substructure mining, explicit interaction modeling, and transformer-based use of unlabeled biomedical data, achieving improved predictive performance and explainability. Its reported scope is bounded by limitations identified in prior approaches and explicit fingerprinting.
Problem
DTI prediction is costly, while existing approaches overlook interaction-relevant substructures and massive unlabeled biomedical data.
Method
MolTrans mines drug and protein substructures, models their interactions explicitly, and uses transformers with massive unlabeled biomedical data.
Results
MolTrans achieves competitive DTI prediction under all settings with improved explainability and robustly improved performance over state-of-the-art baselines.
Takeaways & Limitations
MolTrans provides a DTI prediction approach that combines predictive performance with interpretable substructure-level interaction analysis.
Takeaways & Limitations
Similarity-based methods are not generally applicable to different protein classes, while explicit substructure fingerprints are described as explainable due to their hashing nature.
Abstract
from arXiv · showhide
Drug target interaction (DTI) prediction is a foundational task for in silico drug discovery, which is costly and time-consuming due to the need of experimental search over large drug compound space. Recent years have witnessed promising progress for deep learning in DTI predictions. However, the following challenges are still open: (1) the sole data-driven molecular representation learning approaches ignore the sub-structural nature of DTI, thus produce results that are less accurate and difficult to explain; (2) existing methods focus on limited labeled data while ignoring the value of massive unlabelled molecular data. We propose a Molecular Interaction Transformer (MolTrans) to address these limitations via: (1) knowledge inspired sub-structural pattern mining algorithm and interaction modeling module for more accurate and interpretable DTI prediction; (2) an augmented transformer encoder to better extract and capture the semantic relations among substructures extracted from massive unlabeled biomedical data. We evaluate MolTrans on real world data and show it improved DTI prediction performance compared to state-of-the-art baselines.
1. Introduction
MolTrans addresses two open DTI-prediction challenges: whole-molecule representations that overlook interaction-relevant substructures and methods that underuse massive unlabeled biomedical data. It combines substructure mining, explicit interaction modeling, and transformers, achieving robustly improved performance over state-of-the-art baselines.
- Motivation: DTI prediction supports in silico drug discovery, which is costly and time-consuming because experimental searches span large compound spaces.Researchers may search over 97M possible compounds in a candidate database.
- Open challenges: Existing methods often model whole drug and protein structures, although DTI involves only relevant substructures.Whole-structure representations can introduce noise and make it difficult to identify contributing drug and protein substructures.
- Open challenges: Previous approaches focus on limited labeled datasets while overlooking millions of unlabeled biomedical sequences and lack architectures designed for massive-data integration.Their scope may be limited to several thousand drugs and proteins.
- Approach: MolTrans uses Frequent Consecutive Sub-sequence mining to extract fit-sized drug and protein substructures, then models their interactions explicitly.Its interaction map provides a tractable path for examining which substructure combinations relate to outcomes.
- Approach: MolTrans mines millions of unlabeled drug and protein sequences and augments their representations with transformers to capture relations among extracted substructures.The resulting substructures are reported as higher quality than those learned from small training datasets alone.
- Evaluation: MolTrans shows robustly improved predictive performance over state-of-the-art baselines across realistic settings, including unseen drug or target problems and scarce training data.The evaluation compares methods in drug repositioning and limited-training-data settings.
2. Related Works
Related DTI methods include similarity-based, feature-based, and deep learning approaches. MolTrans differs by combining knowledge-driven architecture, interpretability, and external drug and target data.
- Similarity-based methods: Similarity-based methods exploit known drug-target similarities to infer new interactions but are not generally applicable across different protein classes.Kernel regression and matrix factorization are examples of these approaches.
- Feature-based methods: Feature-based methods feed numerical drug and protein descriptors into downstream prediction models.Examples include ECFP and PubChem for drugs, and CTD and PSC for proteins.
- Deep learning methods: Deep learning methods have improved DTI prediction by capturing complex nonlinear signals.Reported architectures include deep neural, belief, and convolutional neural networks.
- MolTrans: MolTrans differs through knowledge-driven architecture, emphasis on interpretability, and use of external drug and target data alongside interaction data.These design choices extend beyond predictive performance alone and may support medical-chemist decision making.
3. Method
MolTrans formulates DTI prediction as classifying drug–protein pairs and combines explicit sub-structure mining, contextual encoding, and interaction modeling. It uses massive unlabeled biomedical data to mine interpretable sub-structures and predicts interaction probabilities from pairwise and neighborhood interactions.
- 3.1. Problem Definition: DTI prediction learns a function mapping drug–protein pairs to interaction probabilities.Drugs are represented as SMILES atom-and-bond token sequences, while proteins are represented as sequences of 23 amino-acid tokens.
- 3.2.2. Augmented Transformer Embedding Module: Learnable content and positional embeddings are contextualized by separate transformer encoders for drug and protein sub-structure sequences.Self-attention modifies individual embeddings using contextual sub-structural information and chemical relationships among sub-structures.
- 3.2.3. Interaction Prediction Module: MolTrans forms a sub-structure interaction map, applies CNNs to capture higher-order neighborhood interactions, and decodes the result into a DTI probability.Dot products provide scalar pairwise interaction intensities, allowing the map to show which drug–target sub-structure pairs contribute to the outcome; all modules train end-to-end with binary classification loss.
- 3.2.1. Frequent Consecutive Sub-sequence Mining Module: MolTrans applies Frequent Consecutive Sub-sequence mining to identify recurring explicit sub-structures across drug and protein databases.FCS generates hierarchical frequent sub-sequences and converts each input drug and target into explicit sub-structure sequences.
- 3.2.1. Frequent Consecutive Sub-sequence Mining Module: FCS produces more explainable encodings than hashed fingerprints by decomposing molecules into discrete, moderate-size sub-structure partitions.The resulting explicit partitions provide hints about which sub-structures contribute to outcomes, unlike hashing-based ECFP fingerprints.
- 3.2.1. Frequent Consecutive Sub-sequence Mining Module: 560,823 unique protein sequences and 1,870,461 drug SMILES strings provide unlabeled data for mining sub-structures.The paper reports that larger sequence pools can identify useful sub-structures whose frequencies fall below thresholds in small datasets.
4. Experiment
MolTrans is evaluated on DTI prediction using BIOSNAP data, multiple baselines, realistic unseen-entity and scarce-data settings, interpretability examples, and ablations. Across these experiments, it achieves competitive or superior performance, remains robust with missing data, and produces interaction maps aligned with known interactions.
- Experimental Setup: The experiments use the MINER DTI dataset from BIOSNAP, with 4,503 drugs, 2,182 protein targets, and 15,138 interaction pairs.Drug and protein sequences are obtained from Uniprot and DrugBank, and missing sequence or SMILES entries are filtered out.
- Q1–Q2: Predictive Performance: MolTrans outperforms the predictive baselines in the standard DTI prediction setting and is competitive with state-of-the-art deep-learning baselines for unseen drugs and proteins.The unseen drug/protein results are reported in Figure 2; GNN-CPI is omitted from the unseen-drug setting because it performs worse than the other methods.
- Q2: Unseen Drug and Target Setting: Similarity-based KronRLS varies across settings because its performance is susceptible to the data properties of each setting.In the unseen-drug setting, one-layer logistic regression outperforms multi-layer DNN but remains below more sophisticated state-of-the-art deep models.
- Q3: Scarce Training Data: MolTrans is the most robust method as the missing fraction increases from 70% to 95%, while DeepDTI and DeepConv-DTI decline.Its scarce-data performance is attributed to transferable embeddings from relatively abundant substructures.
- Q4: Model Understanding: Interaction maps highlight chemically and biologically relevant substructures, including nitrogen oxide–KNWV and hydroxamic-acid–HDAC2 matches reported in previous studies.The Epha4–Dasatinib example also identifies protein motifs overlapping with the previously described complex.
- Q5: Ablation Study: Ablation results indicate that the CNN, transformer, and interaction modules contribute to final performance, while massive unlabeled data enriches inputs and improves performance.The FCS fingerprint alone has strong predictive performance, and replacing it with other popular fingerprints preserves similarly strong performance.
5. Conclusion
MolTrans is an end-to-end, biologically inspired deep-learning framework for modeling drug–target interactions in realistic drug-discovery settings. It achieves competitive DTI prediction performance with improved explainability.
- MolTrans is introduced as an end-to-end, biologically inspired deep-learning framework that models the DTI process.
- The framework is tested under realistic drug-discovery conditions against state-of-the-art baselines.
- MolTrans demonstrates competitive performance in accurately predicting DTI under all settings while improving explainability.