Source-linked AI summary
DeepQA: Improving the estimation of single protein model quality with deep belief networks
Renzhi Cao, Debswapna Bhattacharya, Jie Hou, Jianlin Cheng
TL;DR
Selecting good protein models, particularly from pools dominated by low-quality candidates, remains a challenge in protein structure prediction. DeepQA addresses this with a deep-belief-network single-model QA method using protein-model features, and it achieves state-of-the-art CASP11 performance while outperforming comparison methods. The authors note that it could improve with more relevant features and larger training datasets.
Problem
Selecting a few good protein models from large pools, especially pools consisting mostly of low-quality models, remains a largely unsolved quality-assessment problem.
Method
DeepQA uses a deep belief network with selected features relevant to protein model quality, including energy, physio-chemical, and structural information.
Results
DeepQA outperforms Support Vector Machines, Neural Networks, and two established QA methods, achieving state-of-the-art performance on CASP11 and selecting good outlier models from mostly low-quality ab initio sets.
Takeaways & Limitations
DeepQA is presented as a useful tool for protein single-model quality assessment and protein structure prediction.
Takeaways & Limitations
The method could be further improved by incorporating more relevant features and training on larger datasets.
Abstract
from arXiv · showhide
Protein quality assessment (QA) by ranking and selecting protein models has long been viewed as one of the major challenges for protein tertiary structure prediction. Especially, estimating the quality of a single protein model, which is important for selecting a few good models out of a large model pool consisting of mostly low-quality models, is still a largely unsolved problem. We introduce a novel single-model quality assessment method DeepQA based on deep belief network that utilizes a number of selected features describing the quality of a model from different perspectives, such as energy, physio-chemical characteristics, and structural information. The deep belief network is trained on several large datasets consisting of models from the Critical Assessment of Protein Structure Prediction (CASP) experiments, several publicly available datasets, and models generated by our in-house ab initio method. Our experiment demonstrate that deep belief network has better performance compared to Support Vector Machines and Neural Networks on the protein model quality assessment problem, and our method DeepQA achieves the state-of-the-art performance on CASP11 dataset. It also outperformed two well-established methods in selecting good outlier models from a large set of models of mostly low quality generated by ab initio modeling methods. DeepQA is a useful tool for protein single model quality assessment and protein structure prediction. The source code, executable, document and training/test datasets of DeepQA for Linux is freely available to non-commercial users at http://cactus.rnet.missouri.edu/DeepQA/.
Background
DeepQA is a deep-belief-network method for single-model protein quality assessment that combines diverse model features and outperforms several QA baselines.
- DeepQA uses a deep belief network with features capturing energy, physio-chemical characteristics, and structural information.
- The network is trained on models from CASP experiments, public datasets, and an in-house ab initio modeling method.
- Deep belief networks perform better than Support Vector Machines and Neural Networks for protein model quality assessment.
- DeepQA achieves state-of-the-art performance on the CASP11 dataset.
- DeepQA outperforms two established methods when selecting good outlier models from mostly low-quality ab initio model sets.
Conclusion
DeepQA is presented as a useful tool for single-model quality assessment and protein structure prediction, with Linux resources available to non-commercial users.
- DeepQA is useful for protein single-model quality assessment and protein structure prediction.
- The Linux source code, executable, documentation, and training/test datasets are freely available to non-commercial users.
Introduction
Protein structure prediction faces challenges in sampling models and selecting good models from generated pools, especially when many candidates are low quality. The paper proposes DeepQA, a deep-learning single-model QA method evaluated across multiple datasets.
- Protein structure prediction involves sampling structural models from sequences and selecting good models from the generated pool.
- Consensus methods can be inaccurate when many models are low quality and commonly require O(n^2) time to assess a model pool.
- Single-model QA predicts quality from one model, so its performance does not depend on the model-pool distribution.
- DeepQA combines deep learning with useful protein-model features for single-model quality assessment.
- The method is benchmarked on CASP datasets, four 3DRobot decoy datasets, and an in-house UniCon3D dataset.
- DeepQA’s performance across these datasets demonstrates the potential of deep learning for protein model quality assessment.
Methods
DeepQA is trained and evaluated as a deep belief network for single-model protein quality assessment using diverse model datasets, sequence-derived features, structural descriptors, and energy-based scores.
- Model architecture: Its architecture stacks two Restricted Boltzmann Machine layers and logistic regression to output a predicted quality score between 0 and 1.RBM weights are pretrained with contrastive divergence, followed by error backpropagation and BFGS fine-tuning.
- Training: Five-fold cross-validation estimates accuracy using mean absolute error while tuning hidden-layer sizes, learning rate, weight cost, and momentum.MAE is computed as the absolute difference between predicted and real values.
Results and Discussion
DeepQA outperforms or matches comparison methods across CASP11 evaluations and performs well when selecting high-quality models from ab initio model pools.
- Learning-method comparison: Deep belief networks obtain the best average per-target correlation on both CASP11 stages, while DeepQA loss is lower than or equal to competing methods.The comparison is against support vector machines and neural networks trained with five-fold cross-validation.
- CASP11 comparison: 0.64 average per-target correlation on CASP11 stage 1 matches ProQ2 and exceeds Qprob, while DeepQA’s 0.09 loss matches ProQ2 and ProQ2-refine.These comparisons use correlation and loss based on GDT-TS scores.
- CASP11 comparison: DeepQA has the highest per-target average correlation on CASP11 stage 2, with loss equal to ProQ2 and better than the other QA methods.The comparison covers stage 2 models evaluated with average per-target correlation and loss.
- CASP11 comparison: DeepQA achieves state-of-the-art performance against established single-model QA methods on CASP11 stage 1 and stage 2 models.On stage 1, its correlation is 0.64 and loss is 0.09; on stage 2, it has the highest correlation and matches ProQ2 in loss.
- Model selection: DeepQA improves selected-model quality for most evaluated top-performing prediction methods, increasing overall Z-scores by 6.39, 16.34, and 6.66 for three systems.The reported improvements apply to Zhang-Server, BAKER-ROSETTASERVER, and RaptorX, respectively.
Conclusions
The paper presents DeepQA as a deep-belief-network method for single-model quality assessment that performs well for ranking ab initio protein models.
- Conclusions: DeepQA performs better than support vector machines and neural networks and achieves state-of-the-art performance against established QA methods.The conclusion identifies DeepQA as a single-model QA method based on a deep belief network.
- Conclusions: DeepQA is useful for ranking ab initio protein models and for protein single-model quality assessment and structure prediction.The conclusion specifically connects the method to ab initio model ranking.
- Conclusions: The method could be further improved by incorporating more relevant features and training on larger datasets.This is the paper’s stated scope for future improvement.
Availability and requirements
DeepQA is distributed for Linux with source code, an executable, documentation, and training and test datasets for non-commercial users.
- Availability: The DeepQA source code, executable, documentation, and training/test datasets are freely available to non-commercial Linux users.The project homepage is cactus.rnet.missouri.edu/DeepQA/ and the implementation uses Perl.
Authors’ contributions
The authors divided project responsibilities across conception, implementation, testing, and manuscript preparation.
- JC and RC conceived and designed the project, while RC, DB, and JH implemented and tested the tool.
- RC, DB, JH, and JC wrote the manuscript, and all authors read and approved it.
Tables
The tables summarize DeepQA’s feature set, evaluation metrics, comparisons with alternative quality-assessment methods, and model-selection results.
- The feature benchmark includes 16 features spanning energy, physicochemical, structural, secondary-structure, and solvent-accessibility information.
- DeepQA was evaluated against Support Vector Machines and Neural Networks using MAE, per-target correlation, and CASP11 stage-specific loss.
- DeepQA is compared with other top-performing single-model quality-assessment methods on CASP11 using average per-target correlation and loss.
- On ab initio datasets, Table 4 compares DeepQA with ProQ2 and Dope2 for model-selection ability.
Figures
Figure 1 presents the deep belief network architecture used for DeepQA.
- Figure 1 depicts the deep belief network architecture underlying DeepQA.