Source-linked AI summary
Boosting Docking-based Virtual Screening with Deep Learning
Janaina Cruz Pereira, Ernesto Raul Caffarena, Cicero dos Santos
TL;DR
Docking-based virtual screening needs better ways to distinguish active ligands from decoys without laborious human feature engineering. DeepVS learns representations from docking outputs using atom and amino-acid embeddings and atom-context convolutions, outperforming docking programs and achieving an AUC ROC of 0.81 on DUD with AutodockVina1.1.2. The approach also reports state-of-the-art virtual-screening performance and uses outputs from two docking programs in evaluation.
Problem
Docking-based virtual screening relies on manually defined features, while manual feature identification is laborious, complex, and difficult to apply at large scale.
Method
DeepVS uses docking outputs and learned atom and amino-acid embeddings, modeling compounds as atom contexts processed by a convolutional layer.
Results
DeepVS outperformed docking programs on AUC ROC and enrichment factor, achieving an AUC ROC of 0.81 with AutodockVina1.1.2 on DUD.
Takeaways & Limitations
DeepVS is presented as an attractive docking-based virtual-screening approach because it requires no human-defined features and achieves good results from a single docking program.
Takeaways & Limitations
AutodockVina1.1.2 experiments considered only the program’s single best-ranked pose rather than multiple output poses.
Abstract
from arXiv · showhide
In this work, we propose a deep learning approach to improve docking-based virtual screening. The introduced deep neural network, DeepVS, uses the output of a docking program and learns how to extract relevant features from basic data such as atom and residues types obtained from protein-ligand complexes. Our approach introduces the use of atom and amino acid embeddings and implements an effective way of creating distributed vector representations of protein-ligand complexes by modeling the compound as a set of atom contexts that is further processed by a convolutional layer. One of the main advantages of the proposed method is that it does not require feature engineering. We evaluate DeepVS on the Directory of Useful Decoys (DUD), using the output of two docking programs: AutodockVina1.1.2 and Dock6.6. Using a strict evaluation with leave-one-out cross-validation, DeepVS outperforms the docking programs in both AUC ROC and enrichment factor. Moreover, using the output of AutodockVina1.1.2, DeepVS achieves an AUC ROC of 0.81, which, to the best of our knowledge, is the best AUC reported so far for virtual screening using the 40 receptors from DUD.
Introduction
Drug discovery is costly and difficult to scale, motivating computational virtual screening and machine-learning methods. DeepVS addresses limitations of docking-based screening by learning representations directly from docking outputs and achieves strong benchmark performance.
- Motivation: Drug discovery is time-consuming, expensive, and difficult to scale across the many molecules considered during development.
- Motivation: High-throughput screening is costly and has a low success rate, making it inaccessible to academia and motivating lower-cost computational alternatives.
- Virtual screening: Virtual screening filters molecules by predicted activity, while structure-based methods use receptor structure to explore ligand–residue interactions.
- Docking-based screening: Docking-based virtual screening searches ligand placements and scores them with mathematical functions, creating a central methodological hurdle.
- Machine learning: Traditional machine-learning approaches rely on human-engineered features extracted from docking output, limiting how data are presented to the model.
- DeepVS: DeepVS uses deep convolutional networks to learn relevant features from docking results and protein–compound data without feature engineering.
- Evaluation: On DUD’s 40 receptors, DeepVS outperformed docking programs on AUC ROC and enrichment factor and achieved a state-of-the-art AUC of 0.81.
DeepVS
DeepVS represents protein-compound complexes through local atom contexts, learned embeddings, convolutional processing, and ligand-versus-decoy scoring. Its architecture converts basic structural information into a fixed-length complex representation that supports variable-sized compounds.
- Atom contexts: DeepVS extracts distances, neighboring atom types, partial charges, and associated residues from each compound atom’s local context.Each context includes the kc closest compound atoms and kp closest protein atoms.
- Scoring: The network applies hidden and output layers to the complex representation and produces a normalized score for distinguishing ligands from decoys.The feedforward process constructs context vectors, applies convolution and max pooling, then returns a normalized score.
- Embeddings: DeepVS converts atom types, distances, charges, and amino acid types into learned real-valued embedding vectors.Each basic feature has a corresponding embedding matrix used through lookup operations.
- Embeddings: Each atom-context representation concatenates embeddings for atom type, distance, charge, and amino acid features.The resulting vector is denoted za = {zatm; zdist; zchrg; zamino}.
- Convolutional representation: A convolutional layer transforms every atom-context vector into abstract features and summarizes them into a fixed-length complex representation r.Max pooling across atom contexts enables processing complexes with different numbers of atoms.
Experimental Setup
DeepVS was evaluated on the DUD benchmark using docking outputs from Dock 6.6 and AutoDock Vina 1.1.2. Performance assessment used strict leave-one-out cross-validation across 40 receptors, with similar receptors excluded from training.
- Dataset: The original DUD partial charges were corrected because they made ligand-decoy discrimination trivial.The benchmark was selected to enable comparison with previously published virtual-screening and scoring-function methods.
- Dataset: The DUD benchmark contains 40 receptors, 2,950 annotated ligands, and 95,316 decoys across six biological groups.Each annotated ligand is paired with 36 decoys selected to mimic associated physical properties.
- Docking: Dock 6.6 and AutoDock Vina 1.1.2 generated the docking results used as DeepVS inputs.Dock 6.6 provides physics-based and force-field-related scores, whereas AutoDock Vina uses a hybrid scoring function.
- Cross-validation: 40-fold leave-one-out cross-validation trained on all but one receptor and tested on the held-out receptor.Receptors sharing a biological class or reported positive cross-enrichment with the test receptor were removed from training.
- Cross-validation: The same DeepVS hyperparameters were used across all leave-one-out iterations and for both docking programs.The authors describe this as stricter than tuning hyperparameters separately with a hold-out set in each iteration.
Evaluation Metrics
The study evaluates virtual-screening rankings with AUC ROC and enrichment factor. AUC summarizes ranking discrimination across thresholds, while EF measures enrichment among the top-ranked compounds relative to random selection.
- Metrics: AUC ROC and enrichment factor are the two metrics used to compare DeepVS with existing virtual-screening methods.Both metrics assess the quality of compound rankings.
- AUC ROC: AUC quantifies the ROC curve relating true-positive and false-positive behavior across continuous score thresholds.An AUC ≤ 0.50 indicates random selection, while AUC 1.0 indicates perfect active-compound identification.
- AUC ROC: The AUC calculation uses the numbers of actives and decoys together with the number of decoys ranked above each active.These quantities determine the ranking-based area under the ROC curve.
- Enrichment factor: EF at x% compares the active-compound fraction among the top x% ranked compounds with an equally sized random selection.It indicates how effectively the ranking concentrates actives near the top of the list.
Results and Discussion
DeepVS improved docking-based virtual screening across DUD receptors and docking inputs, with performance varying according to docking quality and selected hyperparameters. It also compared favorably with feature-engineered and other state-of-the-art systems.
- Overall performance: DeepVS-ADV achieved the best average AUC, ef2%, and ef20% among the four evaluated systems, with the best AUC for 20 of 40 receptors.Table 2 compares Dock6.6, AutodockVina1.1.2, DeepVS-Dock, and DeepVS-ADV using AUC ROC, ef at 2% and 20%, and efmax.
- Robustness to docking quality: DeepVS-Dock raised AUC above 0.70 for receptors where Dock6.6 produced AUC below 0.40, including AR, COX1, HSP90, InhA, PDE5, PDGFrb, and PR.This result shows improvement in several cases with poor docking-program AUC, although the reported relationship is an observed performance comparison.
- DeepVS-ADV versus ADV: 31% better AUC: DeepVS-ADV averaged 0.81 versus 0.62 for ADV, while its ef at 20% was 55% larger, 3.1 versus 2.0.DeepVS-ADV had higher AUC than ADV for 31 receptors; it achieved AUC > 0.70 for 33 receptors versus 13 for ADV.
- DeepVS-Dock versus Dock6.6: 54% better AUC: DeepVS-Dock averaged 0.74 versus 0.48 for Dock6.6, and its ef at 20% was more than two times larger, 3.0 versus 1.3.DeepVS-Dock achieved higher AUC than Dock6.6 for 36 receptors and exceeded AUC 0.70 for 27 receptors versus 4 for Dock6.6.
- Hyperparameter sensitivity: A learning rate of 0.1 produced the best AUC and ef outcomes on DUD, while larger embedding sizes mainly improved ef and sizes above 200 provided no further improvement.AUC improved with convolutional filters up to 400, whereas 200 filters produced the best ef2%.
- Comparison with other systems: DeepVS-ADV produced higher AUC than DDFA-ADV for 27 receptors and than DDFA-ALL for 25 receptors, despite DDFA-ALL using three docking programs.DDFA uses human-defined docking-output features, whereas DeepVS uses data from the best pose only; the paper reports DeepVS-ADV as the best AUC among compared systems and the best reported for the 40 DUD receptors.
Conclusions
DeepVS improves docking-based virtual screening without human-defined features and achieves the best reported AUC on DUD using AutodockVina output. Its representation strategy may benefit from additional training data, although DUD contains only 40 different proteins.
- DeepVS improves docking-based virtual screening and does not require human-defined features.
- Using AutodockVina output, DeepVS produced the best AUC reported so far for virtual screening on DUD.
- DeepVS achieves good results using the output of a single docking program.
- DeepVS has greater potential for performance improvement if more data are added to the training set.
- DUD contains more than 100k protein-compound complexes but only 40 different proteins.
- Atom and amino acid embeddings, together with convolutionally processed atom contexts, provide a way to model protein-compound complex raw data for deep learning.The atom-context representation was reported as effective for learning protein-compound complex representations.