Source-linked AI summary
Protein Secondary Structure Prediction Using Cascaded Convolutional and Recurrent Neural Networks
Zhen Li, Yizhou Yu
TL;DR
Protein secondary-structure prediction is an important intermediate problem for protein structure prediction. The paper proposes DCRNN, integrating local and global sequence context with joint secondary-structure and solvent-accessibility prediction; it reports 69.7% Q8 accuracy on CB513.
Problem
Protein secondary-structure prediction is an important intermediate step toward protein structure prediction.
Source: §1 · p. 1Method
DCRNN combines multiscale CNNs for local context and stacked bidirectional RNNs for global context, with joint secondary-structure and solvent-accessibility classification.
Source: §1 · p. 2Results
69.7% Q8 accuracy on CB513 is reported for the model-averaged system.
Source: Training on Filtered CB6133 and… · pp. 5–6Takeaways & Limitations
The results support combining local and global contexts for protein secondary-structure prediction, with potential application to other structure-prediction tasks.
Source: §4 · p. 7Existing BGRUs cannot handle extremely long, especially low-frequency, dependencies, motivating architectures with an implicit attention mechanism.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Protein secondary structure prediction is an important problem in bioinformatics. Inspired by the recent successes of deep neural networks, in this paper, we propose an end-to-end deep network that predicts protein secondary structures from integrated local and global contextual features. Our deep architecture leverages convolutional neural networks with different kernel sizes to extract multiscale local contextual features. In addition, considering long-range dependencies existing in amino acid sequences, we set up a bidirectional neural network consisting of gated recurrent unit to capture global contextual features. Furthermore, multi-task learning is utilized to predict secondary structure labels and amino-acid solvent accessibility simultaneously. Our proposed deep network demonstrates its effectiveness by achieving state-of-the-art performance, i.e., 69.7% Q8 accuracy on the public benchmark CB513, 76.9% Q8 accuracy on CASP10 and 73.1% Q8 accuracy on CASP11. Our model and results are publicly available.
1 Introduction
The paper introduces DCRNN for protein secondary-structure prediction, combining local and global sequence context; it reports that the model outperforms existing methods across several datasets. It motivates this work by the importance of secondary structure for protein analysis and structure prediction.
1 Introduction
- Secondary structure is a vital intermediate for protein structure prediction because it reflects local structural types and can reduce the degrees of freedom in tertiary-structure prediction.Source: §1 · p. 1
- Earlier statistical models achieved under 60% Q3 accuracy, while later methods using evolutionary information and position-specific scoring matrices exceeded 70% before progress stalled on Q8 classification.Source: §1 · p. 1
- CNNs extract multiscale local sequence context, while bidirectional GRUs address long-range dependencies among amino acids.Source: §1 · pp. 1–2
- Experiments on CB6133, CB513, CASP10, and CASP11 show that DCRNN outperforms existing methods and achieves state-of-the-art performance.Source: §1 · p. 2
- DCRNN combines multiscale CNNs for local context, stacked bidirectional RNNs for global context, and joint secondary-structure and solvent-accessibility classification.Source: §1 · p. 2
- The authors report that this is the first application of bidirectional GRU layers to protein secondary-structure prediction, and their ablation study identifies them as the network’s most important component.Source: §1 · p. 2
2 Network Architecture
The network combines embedded sequence and profile features, multiscale CNN local contexts, and stacked bidirectional GRU global contexts for joint classification.
2 Network Architecture
- The architecture cascades feature embedding, multiscale CNNs, three stacked BGRUs, and fully connected layers for joint 8-state structure and 4-category accessibility classification.Source: §2 · p. 2
2.1 Feature Embedding
- 21-dimensional one-hot sequence vectors are embedded to 50 dimensions, then concatenated with 21-dimensional dense PSI-BLAST profile features.Source: §2.1 · pp. 2–3
2.2 Multiscale CNNs
- Three CNNs with kernel extents 3, 7, and 11 extract and concatenate multiscale local contexts to model adjacent-residue dependencies.Source: §2.2 · p. 3
2.3 BGRUs
- Three stacked bidirectional GRUs capture global contexts beyond the CNN kernel range, combining forward and backward states at each residue.Source: §2.3 · pp. 3–4
2.4 Multi-Task Joint Feature Learning
- Shared fully connected layers jointly predict eight-state secondary structure and four-category solvent accessibility, using a regularized joint loss.Source: §2 · p. 2, §2.4 · p. 4
3 Experimental Results
Experiments evaluate DCRNN on CB6133, CB513, CASP10, and CASP11, reporting benchmark results and ablations of network components. The strongest reported Q8 results are on CASP10 and CASP11.
3.1 Datasets and Features
- Evaluation uses four public datasets; filtered CB6133 has 5,534 proteins after sequences with over 25% similarity to CB513 are removed.CB6133 contains 6,128 proteins; CASP10 and CASP11 contain 123 and 105 domain sequences, respectively. Inputs have 55 channels per residue, including sequence and profile features.Source: §3.1 · p. 4
3.2 Implementation Details
- Implementation uses CNN kernels of 3, 7, and 11, three stacked BGRU layers, dropout of 0.5, and two fully connected layers.Bagging combines 10 weak models; early stopping selects models based on validation F1 score.Source: §3.2 · pp. 4–5
3.3 Performance
- 73.2±0.6% Q8 accuracy on CB6133 is 1.1% above the previous best GSN result, and solvent accessibility accuracy is 76.1%.The CB6133 testing set differs from the one used by Wang et al. (2016), whose result was not compared.Source: Training and Testing on CB6133 · p. 5
- DCRNN improves precision and recall on almost all CB6133 labels versus GSN, while CB513 results are lower on frequent labels and higher on rare labels.The CB513 comparison reports 3% to 4% lower accuracies on H, E, and L, and significantly higher accuracies on T, S, and B.Source: Training and Testing on CB6133 · p. 5, Training on Filtered CB6133 and… · p. 5, Table 1 · p. 5
- 69.7% Q8 accuracy on CB513 is 4.8% higher than the previous best ensemble result; DCRNN also exceeds CNF overall despite CNF using extra features.The CB513 ensemble averages 10 independently trained models.Source: Training on Filtered CB6133 and… · pp. 5–6
- 76.9% Q8 accuracy on CASP10 and 73.1% on CASP11 exceed prior best results by 5.1% and 0.8%, respectively, using a single model.Both datasets were tested using a model trained on filtered CB6133.Source: Training on Filtered CB6133 and… · p. 6, Table 3 · p. 6
3.4 Ablation Study
- Ablations show lower CB513 performance without backward RNN passes (66.9%) or multiscale CNNs (68.1%); stacked BGRUs and direct local-context input also help.The authors report that GRU gates are necessary for handling long-range dependencies relative to bidirectional simpleRNN layers.Source: §3.4 · p. 6
4 Conclusions
The proposed end-to-end model combines multiscale CNNs and bidirectional GRUs to extract local and global contexts, improving prior state-of-the-art performance in secondary-structure prediction. The conclusion also notes that existing BGRUs cannot handle extremely long, especially low-frequency, dependencies.
4 Conclusions
- The integrated local and global contexts improved the previous state of the art in protein secondary-structure prediction.The model uses multiscale CNNs and stacked bidirectional GRUs; multi-task joint feature learning is used to further refine performance.Source: §4 · p. 7
- Models combining local and global contexts may also apply to other challenging structure-prediction tasks in protein and computational biology.Source: §4 · p. 7
- Existing BGRUs cannot handle extremely long dependencies, particularly low-frequency long dependencies.The conclusion suggests architectures with an implicit attention mechanism, such as neural Turing machines, may address this limitation.Source: §4 · p. 7