Protein-Ligand Scoring with Convolutional Neural Networks

Matthew Ragoza; Joshua Hochuli; Elisa Idrobo; Jocelyn Sunseri; David Ryan Koes

Source-linked AI summary

Protein-Ligand Scoring with Convolutional Neural Networks

Matthew Ragoza, Joshua Hochuli, Elisa Idrobo, Jocelyn Sunseri, David Ryan Koes

arXiv:1612.02751v1Dec 8, 2016stat.MLcs.LGq-bio.BM

TL;DR

Protein-ligand scoring needs methods that can rank binding poses and affinities from growing structural data. This paper develops CNN scoring functions using comprehensive 3D interaction representations and trains them for pose and binder discrimination. The CNN scoring method outperforms AutoDock Vina in pose prediction and virtual screening, although generalization beyond the training task is limited.

Problem
Protein-ligand scoring functions are needed to rank binding affinities and poses for structure-based drug design.
Method
CNN scoring functions learn from comprehensive 3D protein-ligand representations to classify correct versus incorrect poses and binders versus non-binders.
Results
CNN scoring outperforms AutoDock Vina in pose prediction and virtual screening.
Takeaways & Limitations
The method provides a direct, comprehensive 3D approach for scoring protein-ligand interactions and can support informative atomic-contribution visualizations.
Takeaways & Limitations
Generalization beyond the task represented in the training data is limited.

Abstract

from arXiv · show

Computational approaches to drug discovery can reduce the time and cost associated with experimental assays and enable the screening of novel chemotypes. Structure-based drug design methods rely on scoring functions to rank and predict binding affinities and poses. The ever-expanding amount of protein-ligand binding and structural data enables the use of deep machine learning techniques for protein-ligand scoring. We describe convolutional neural network (CNN) scoring functions that take as input a comprehensive 3D representation of a protein-ligand interaction. A CNN scoring function automatically learns the key features of protein-ligand interactions that correlate with binding. We train and optimize our CNN scoring functions to discriminate between correct and incorrect binding poses and known binders and non-binders. We find that our CNN scoring function outperforms the AutoDock Vina scoring function when ranking poses both for pose prediction and virtual screening.

Introduction

Protein-ligand scoring methods must balance flexibility with preservation of structural information. This paper applies CNNs to comprehensive 3D protein-ligand representations so the model can learn informative interaction features directly.

Machine-learning scoring functions provide greater flexibility and expressiveness than more constrained approaches.
Handcrafted descriptors such as interaction counts, pairwise distances, and fingerprints can eliminate or approximate precise spatial relationships.
CNNs hierarchically learn higher-level features while maintaining spatial relationships, motivating their use for protein-ligand structures.
The proposed CNN scoring method automatically identifies informative features from protein-ligand structures rather than requiring prior feature extraction.
The model classifies docked compound poses as binders or non-binders using a 3D grid representation of protein-ligand structures.
The CNN scoring method outperforms AutoDock Vina when selecting poses for pose prediction and virtual screening.

Methods

The study trains separate CNN models for pose prediction and virtual screening from docked protein-ligand structures encoded as multichannel 3D atom-density grids. Networks are optimized by clustered cross-validation and evaluated on independent test sets.

Two training sets target pose prediction and virtual screening, with structural information translated into a CNN-compatible custom input format.
Network topology and parameters are systematically optimized using clustered cross-validation before full-training-set evaluation on independent test sets.
Docked poses generated with smina and AutoDock Vina are used for both active and decoy compounds.
Pose prediction: The pose-prediction set labels poses below 2Å heavy-atom RMSD as positive and poses above 4Å RMSD as negative, omitting intermediate poses.
Virtual screening: The DUD-E virtual-screening training set contains 22,645 positive and 1,407,145 negative examples.
Input format: Protein-ligand structures are encoded in a 24Å3 grid at default 0.5Å resolution, with separate channels for distinct protein and ligand atom types.
Input format: Atom types are represented as continuous density distributions, with Gaussian and quadratic components, and a hard discrete representation is also evaluated.
Input format: Random rotations and translations reduce overfitting during training.

Training

The models were trained with stochastic gradient descent and backpropagation on balanced batches, using augmentation and clustered cross-validation to evaluate generalization across protein targets.

Training minimized multinomial logistic loss using stochastic gradient descent and backpropagation.
Positive and negative examples were balanced by sampling equal numbers of each per batch.
Random rotations and translations were applied on the fly as data augmentation, which significantly improved training.
Models were evaluated with 3-fold cross-validation using ROC curves and AUC, where AUC = 1 is perfect classification and AUC = 0.5 is chance.
Training and test folds were clustered by protein-target similarity to reduce evaluation on targets resembling those in training.
Independent test sets were restricted to targets with available bound crystal structures and excluded high sequence or binding-site structural similarity to training targets.

Optimization

The CNN architecture and input representation were systematically varied, then selected through iterative comparisons of accuracy and training time. The final design combines convolutional and pooling layers with a two-output softmax classifier and supports interaction visualization.

The initial architecture used five 3x3x3 convolutional layers with rectified linear units alternating with max pooling.
The convolutional and pooling stack was followed by a fully connected layer with two outputs and a softmax layer for binary classification.
A reference CNN was optimized by varying one parameter at a time and combining changes that improved accuracy without increasing training time.
Input representations varied atom typing, occupancy, atomic-radius scaling, grid resolution, layer width and depth, pooling, and fully connected-layer structure.
The visualization algorithm rescored complexes after removing ligand atoms or fragments and protein residues to estimate their relative score contributions.
Whole-residue protein removals preserved enough information to assess spatial relationships between protein and ligand.

Results

Systematic network and training-parameter optimization improved CNN performance in clustered cross-validation, and the optimized architecture was evaluated across pose prediction, virtual screening, and affinity prediction.

Systematic optimization improved CNN performance in clustered cross-validation while revealing the relative importance of parameter choices.
The optimized architecture was evaluated for pose prediction, virtual screening, and affinity prediction.
The evaluation also examined how the training set affected model performance.

Optimization

Iterative optimization increased cross-validation AUC while reducing training time, with performance shaped by input resolution, network width and depth, pooling, and atom typing.

AUC increased from 0.78 to 0.82 while training time decreased from 580ms to 120ms per iteration between the initial and final models.
The final architecture reduced network depth from five to three convolutional layers.
Smina atom types achieved the best AUC, while binary protein/ligand atom typing reduced AUC by at most 0.05 and simpler types could improve early enrichment.
0.25Å resolution improved AUC by 0.02 over 0.5Å but increased training time more than 4X, so the higher resolution was not selected.
Reducing layer width improved both AUC and training time up to a limit, with the final first convolutional layer generating 32 feature maps.
Average pooling prevented the model from learning, whereas changing pooling-kernel size did not improve AUC.

Pose Prediction

Pose prediction evaluates both cross-target classification of low- versus high-RMSD poses and within-target selection of the best pose. CNNs classify across targets well, but Vina selects the top pose more reliably, while their scores capture different ranking criteria.

Inter-target ranking: The CNN achieves an AUC of 0.815 versus 0.?? for Vina in cross-target CSAR pose classification.
Intra-target ranking: Both Vina and CNN perform substantially better than random selection when ranking poses within targets.
Score–RMSD relationships: CNN scores weakly correlate with RMSD, whereas Vina scores do not correlate overall but show a funnel in which the best scores have very low RMSDs.
Score–RMSD relationships: CNN and Vina scores show no correlation, indicating that they use different criteria to rank poses.

Virtual Screening

Virtual screening tests whether scoring functions distinguish active from inactive compounds using docked structures. CNN scoring outperforms Vina on the DUD-E benchmark even when both methods rank identical poses.

The evaluation compares single-pose scoring of Vina’s top-ranked pose with multi-pose scoring that selects among all docked ligand poses.
Using identical poses, CNN single-pose scoring achieves an AUC of 0.85 versus 0.68 for Vina.
Multi-pose CNN scoring achieves an AUC of 0.86 on the DUD-E benchmark.
CNN scoring outperforms Vina on 90% of DUD-E targets.

Combined Training

Training data strongly shapes CNN behavior: DUD-E data favors virtual screening, whereas CSAR pose data improves pose sensitivity. Combining datasets yields models that balance these objectives but can reduce per-target screening performance.

CNN models trained on one data type do not generalize particularly well to the other task.
A 2:1 DUD-E/CSAR model achieves AUCs of 0.79 for pose prediction and 0.83 for virtual screening.
Adding pose-prediction data increases the difference between single-pose and multi-pose evaluation, suggesting improved pose selection.
DUD-E-only training outperforms Vina on 90% of targets, versus 81% for the combined training set.
All fpps actives contain bisphosphonate, while fewer than 1% of decoys contain phosphorus, favoring models that recognize this chemical feature.
Pose-quality training penalizes erroneous poses and reduces the dominance of non-structural polarity features.

Independent Test Sets

Independent tests assess models trained on CSAR, DUD-E, or both across pose prediction and virtual screening. Results preserve the training-task trade-off: DUD-E models screen better, while pose-trained models improve pose selection, and MUV remains difficult.

The independent-test models comprise CSAR-only pose prediction, DUD-E-only virtual screening, and a combined 2:1 DUD-E/CSAR model.
Pose prediction: CSAR-trained CNNs classify low- versus high-RMSD poses better across targets, while DUD-E training performs poorly at pose prediction.
Pose prediction: The CSAR-trained model identifies a low-RMSD pose as top-ranked in 46% of PDBbind complexes, compared with 57% for Vina.
Pose prediction: The combined CNN models outperform Vina at identifying a low-RMSD pose within the first three ranked poses.
Virtual screening: DUD-E-trained CNNs outperform Vina on 86% of ChEMBL targets and 56% of MUV targets.
Virtual screening: Average ChEMBL AUCs are 0.64 for Vina, 0.78 for the combined CNN, and 0.78 for the DUD-E CNN.
Virtual screening: Average MUV AUCs are 0.55 for Vina, 0.50 for the combined CNN, and 0.52 for the DUD-E CNN, with overall performance essentially random.
Virtual screening: No method achieves an AUC greater than 0.6 on more than two MUV targets.

Visualization

The visualization links CNN score contributions to atomic and residue-level features in protein–ligand interactions. It identifies critical residues and distinguishes aligned from divergent regions in partially correct poses.

Visualization: The visualization qualitatively indicates which atomic features drive the CNN model’s output.It is designed as an interpretable indication of the features contributing to the score.
Mutation Analysis: Single-residue mutation analysis tested alanine or glycine substitutions with the largest binding-affinity changes in proteins excluded from training.The analysis filtered the Platinum database for single mutations to alanine or glycine.
Mutation Analysis: The CNN identified critical residues in many tested examples, including all three protein–ligand pairs with the largest affinity changes.In each case, many residues received heavy green coloring, including the experimentally mutated residue.
Mutation Analysis: The residue analysis was performed without training the CNN on protein mutational data.This supports evaluating whether the model’s highlighted residues reflect interaction features learned from other data.
Pose Analysis: For five partially correct high-RMSD poses, the CNN ranked the crystal pose above the docked pose.The visualization showed positive contributions in aligned regions and negative contributions in divergent ligand regions.

Discussion

The study presents CNNs that use comprehensive 3D protein–ligand representations and perform strongly for inter-target pose prediction and virtual screening. Performance is weaker for intra-target ranking and binding-affinity regression, revealing task-specific generalization limits.

Contribution: The work describes CNN models that score protein–ligand interactions from a direct, comprehensive 3D representation.The authors frame this as a detailed evaluation of deep-learning convolutional models for protein–ligand scoring.
Pose Prediction: CNN models can perform substantially better than AutoDock Vina for inter-target pose prediction in cross-validation and independent-test evaluations.The reported advantage is supported across both evaluation settings.
Pose Ranking: CNN models perform worse than AutoDock Vina for intra-target pose ranking, which is more relevant to molecular docking.The authors suggest that changing the training protocol could improve this task.
Limitations: The current training protocol treats ligands identically below a 10µM affinity threshold, preventing distinctions between affinity and pose quality.The authors propose incorporating binding affinity or relation classification to improve intra-target performance.
Virtual Screening: CNN models perform well in virtual-screening evaluations, including clustered cross-validation and independent test sets.Independent virtual-screening results are reported for most targets.
Limitations: DUD-E virtual-screening performance may be overly optimistic because of artificial enrichment, although clustered cross-validation is intended to mitigate this issue.The independent test sets used a different dataset-construction method.
Affinity Prediction: CNN scores do not correlate with CSAR crystal-pose binding affinities, with |R| < 0.1, whereas Vina shows R = 0.37.Classification and pose-scoring training does not generalize to binding-affinity regression.
Future Directions: The authors report that multi-task training has not yet produced a synergistic improvement across all tasks.They identify larger datasets spanning related objectives as a possible improvement avenue.