Source-linked AI summary
Protein secondary structure prediction using deep convolutional neural fields
Sheng Wang, Jian Peng, Jianzhu Ma, Jinbo Xu
TL;DR
Predicting protein structure from sequence alone is extremely challenging, and existing shallow architectures had not broken the long-standing performance record. DeepCNF combines deep hierarchical modeling of sequence–structure relationships with interdependency modeling between adjacent secondary-structure labels, achieving substantially better performance than state-of-the-art methods.
Problem
Predicting protein structure from sequence alone is extremely challenging, while existing shallow architectures had not broken the long-standing performance record.
Method
DeepCNF is a deep-learning sequence-labeling method that models complex sequence–structure relationships hierarchically and exploits interdependency between adjacent secondary-structure labels.
Results
DeepCNF significantly outperformed state-of-the-art methods, breaking the long-standing ~80% Q3 accuracy record and achieving better SOV scores.
Takeaways & Limitations
DeepCNF provides a general framework for predicting other protein-structure properties, including contact number and disorder regions.
Takeaways & Limitations
DeepCNF has no significant advantage when the protein under prediction has a very sparse sequence profile, with Neff≤2.
Abstract
from arXiv · showhide
Protein secondary structure (SS) prediction is important for studying protein structure and function. When only the sequence (profile) information is used as input feature, currently the best predictors can obtain ~80% Q3 accuracy, which has not been improved in the past decade. Here we present DeepCNF (Deep Convolutional Neural Fields) for protein SS prediction. DeepCNF is a Deep Learning extension of Conditional Neural Fields (CNF), which is an integration of Conditional Random Fields (CRF) and shallow neural networks. DeepCNF can model not only complex sequence-structure relationship by a deep hierarchical architecture, but also interdependency between adjacent SS labels, so it is much more powerful than CNF. Experimental results show that DeepCNF can obtain ~84% Q3 accuracy, ~85% SOV score, and ~72% Q8 accuracy, respectively, on the CASP and CAMEO test proteins, greatly outperforming currently popular predictors. As a general framework, DeepCNF can be used to predict other protein structure properties such as contact number, disorder regions, and solvent accessibility.
2 Department of Human Genetics, University of Chicago, Chicago, IL
Protein secondary structure prediction connects sequence information to structural and functional analysis, but sequence-only prediction has remained difficult. DeepCNF addresses this challenge by combining deep convolutional modeling with explicit dependencies among adjacent secondary-structure labels.
- Protein secondary structure links primary sequence to tertiary structure and supports structure and functional analysis.
- ~80% Q3 accuracy has remained the best reported level for sequence-profile-based, ab initio 3-state prediction.PSIPRED and other state-of-the-art approaches achieved this level, creating a long-standing benchmark.
- Existing shallow architectures may struggle to model the complex sequence–structure relationship required for further improvement.
- DeepCNF combines conditional neural fields with deep convolutional neural networks for both 3-state and 8-state prediction.
- DeepCNF models both complex sequence–structure relationships and correlations among adjacent secondary-structure labels.Its convolutional network can also include longer-range sequence information.
- DeepCNF differs from prior deep-learning methods by using convolutional architectures and explicitly modeling secondary-structure label interdependency.Compared with Cheng’s method, it captures longer-range sequence information; compared with Zhou’s method, it explicitly models label interdependency.
- DeepCNF greatly outperforms state-of-the-art methods, particularly for challenging high-curvature, beta-loop, and irregular-loop regions.
RESULTS
DeepCNF is evaluated across five datasets and multiple secondary-structure metrics, consistently outperforming competing predictors, especially on challenging targets and non-ordinary states.
- CASP10 and CASP11: 84.4% Q3 accuracy on CASP10 lets DeepCNF slightly exceed SSpro with template at 84.2%.
- CAMEO: 84.4% Q3 and 72.1% Q8 accuracy on 179 CAMEO targets exceed SSpro with template at 78.9% and 65.7%.
- SOV: 86.2%, 84.8%, 85.6%, 85.8%, and 84.5% SOV scores across five datasets significantly outperform all other predictors, including SSpro with template.
CONCLUSION AND FUTURE WORK
DeepCNF combines deep hierarchical modeling with adjacent-label interdependency to improve protein secondary-structure prediction. Its performance exceeds state-of-the-art methods, while the framework also extends to other sequence-labelling tasks.
- DeepCNF models complex sequence–structure relationships through a deep hierarchical architecture and exploits interdependency between adjacent SS labels.
- DeepCNF significantly outperforms state-of-the-art methods, breaking the long-standing ~80% Q3 accuracy ceiling.
- DeepCNF performs much better on challenging SS types, including high-curvature regions, beta loops, and irregular loops.
- DeepCNF performs reasonably well on proteins without good PDB homologs, but has no significant advantage in another homolog-related setting.
- Predicting secondary structure from primary sequence rather than sequence profile remains challenging.
- DeepCNF can be directly applied to predicting solvent accessibility, contact number, structural alphabet, and order/disorder regions.
METHOD
DeepCNF combines a conditional random-field label layer with a deep convolutional neural network, using sequence-derived features to model local representations and adjacent secondary-structure dependencies.
- Architecture: DeepCNF consists of a conditional random-field module and a deep convolutional neural-network module.The CRF module includes the top and label layers, while the DCNN covers the input-to-top-layer mapping.
- Conditional neural fields: The model assigns sequence-level probabilities by combining adjacent-label potentials with feature-label potentials and normalizing with a partition function.Adjacent-label potentials quantify correlations among neighboring secondary-structure types, while feature-label potentials connect labels to neural-network outputs.
- Deep convolutional neural network: The DCNN uses position-independent convolutional weights over windows of size 2N_k + 1, with sigmoid or tanh activations across layers.At the first layer, the representation is the input feature matrix; subsequent layers produce learned feature matrices.
- Training: Model parameters are trained by maximum likelihood, with gradients calculated for the CRF and DCNN using forward-backward expectations and back-propagation.The implementation trains parameters simultaneously with L-BFGS, while also supporting supervised layer-by-layer training followed by fine-tuning.
- Regularization and optimization: L2 regularization penalizes the log-likelihood objective to reduce overfitting, while the regularization factor is selected to avoid under-fitting.An excessively large factor can constrain parameters too strongly and prevent sufficient learning from the training data.
- Input features: The input feature for each residue is built from a PSI-BLAST-derived PSSM and a 21-element amino-acid identity vector, totaling 42 features.PSI-BLAST uses an E-value threshold of 0.001, three iterations, and UniRef90; PSSM entries are transformed with a sigmoid function.
COMPETING FINANCIAL INTERESTS
The paper describes DeepCNF's convolutional and conditional-random-field components, alongside experiments evaluating Q3, Q8, SOV, and per-state recall and precision across protein datasets.
- METHOD: Convolutional deep networks capture longer-range sequence information than typical deep networks when both use the same window size.
- EXPERIMENTS: The study examines Q3 accuracy across Neff intervals on CB513 and CASP10-11 test sets, averaging proteins within each interval.
- EXPERIMENTS: Q8 accuracy is evaluated across five datasets: CullPDB, CB513, CASP10, CASP11, and CAMEO.The experiments also report Q3 accuracy and SOV scores on these datasets.
- METHOD: DeepCNF combines a deep convolutional neural network with a conditional random field to model sequence features and adjacent-residue label correlations.The DCNN uses hidden layers, while the CRF connects the top layer to labels and models correlations among adjacent residues.