Source-linked AI summary
Protein Secondary Structure Prediction with Long Short Term Memory Networks
Søren Kaae Sønderby, Ole Winther
TL;DR
Common feed-forward and SVM approaches use sliding windows, limiting learned dependencies to the window, while earlier RNNs on this task learned only ±15-amino-acid dependencies. The paper applies bidirectional LSTMs to protein secondary-structure prediction, using feed-forward networks to combine directional predictions and between recurrent hidden states. The LSTM network achieved a correct classification rate of 0.674, better than state-of-the-art GSN and CNF performance and significantly better than SSpro8’s BRNN (0.511).
Problem
Common feed-forward and SVM approaches use sliding windows, limiting learned dependencies to the window, while earlier RNNs on this task learned only ±15-amino-acid dependencies.
Source: §1 · p. 1Method
The paper applies bidirectional LSTMs to protein secondary-structure prediction, using feed-forward networks to combine directional predictions and between recurrent hidden states.
Source: §1 · p. 1, §2.1 · p. 2Results
The LSTM network achieved a correct classification rate of 0.674, better than state-of-the-art GSN and CNF performance and significantly better than SSpro8’s BRNN (0.511).
Source: §3 · p. 4Takeaways & Limitations
The authors conclude that their LSTM architecture performs significantly better than existing secondary-structure methods, with different feed-forward architectures left for future investigation.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Prediction of protein secondary structure from the amino acid sequence is a classical bioinformatics problem. Common methods use feed forward neural networks or SVMs combined with a sliding window, as these models does not naturally handle sequential data. Recurrent neural networks are an generalization of the feed forward neural network that naturally handle sequential data. We use a bidirectional recurrent neural network with long short term memory cells for prediction of secondary structure and evaluate using the CB513 dataset. On the secondary structure 8-class problem we report better performance (0.674) than state of the art (0.664). Our model includes feed forward networks between the long short term memory cells, a path that can be further explored.
1. INTRODUCTION
The introduction contrasts fixed-window approaches with recurrent models for protein secondary-structure prediction and presents bidirectional LSTMs as the paper’s approach. It motivates LSTMs by their ability to learn longer dependencies than earlier RNNs reported for this task.
1. INTRODUCTION
- Earlier work applied neural networks and RNNs to the task, while more recent approaches include conditional-random-field hybrids and generative stochastic networks.The introduction also notes that position-specific scoring matrices improved performance in earlier neural-network work.Source: §1 · p. 1
- Feed-forward networks and SVMs commonly use sliding windows, which limit learned dependencies to the input window.The paper notes earlier RNNs on this task learned dependencies of only ±15 amino acids, while LSTMs are designed to learn across hundreds of time steps.Source: §1 · p. 1
- The paper applies bidirectional LSTMs to secondary-structure prediction, combining forward and backward predictions with feed-forward networks.The architecture also places feed-forward networks between recurrent hidden states; the paper identifies this as an extension explored in earlier RNN work [Pascanu et al., 2013].Source: §1 · p. 1
2. MATERIALS AND METHODS
The methods use a bidirectional LSTM with feed-forward connections, train on filtered protein sequences labeled with eight-class DSSP structures, and evaluate on CB513. The setup also describes the recurrent architecture and data-label mappings.
2.1. Model
- The model runs forward and backward RNNs over the known sequence, combines their outputs for a softmax prediction, and adds feed-forward networks between recurrent hidden states.It also uses shortcut connections between recurrent hidden layers; Figure 1 depicts the bidirectional arrangement and the feed-forward connections between time slices.Source: §2.1 · p. 2, Figure 1 · p. 3
2.2. Data
- The 6,128-sequence dataset uses 8-class DSSP labels and excludes sequences with more than 25% identity to CB513; 5,278 sequences train, 256 validate, and CB513 tests.Each amino acid is represented by 42 dimensions: 21 for orthogonal encoding and 21 for sequence profiles; the usual Q8-to-Q3 mapping is provided for reference.Source: §2.2 · p. 2, Table 1 · p. 3
2.3. Experimental setup
- The model has three LSTM layers with either 300 or 500 units per layer, plus two-layer ReLU networks for recurrent-state updates and bidirectional-output concatenation.The recurrent feed-forward network uses skip connections; the concatenation network uses 50% dropout. Gradients are scaled when their norm exceeds 0.5, and the batch size is 128.Source: §2.3 · pp. 2–3
3. RESULTS
The results report that the LSTM achieves higher CB513 classification accuracy than the cited state-of-the-art methods and significantly outperforms the bidirectional RNN baseline.
3. RESULTS
- The LSTM network achieved a correct classification rate of 0.674, outperforming state-of-the-art GSN and CNF results and significantly exceeding the BRNN’s rate of 0.511.Source: §3 · p. 4
4. DISCUSSION AND CONCLUSION
The authors report state-of-the-art CB513 performance for their LSTM architecture, which combines forward and backward networks with feed-forward connections. They identify exploration of feed-forward network architectures as future work.
4. DISCUSSION AND CONCLUSION
- The authors report CB513 performance of 0.674 as state of the art and say the architecture significantly outperforms existing secondary-structure methods.The architecture uses feed-forward networks between recurrent connections and to concatenate forward and backward predictions; future work will investigate different feed-forward architectures.Source: §4 · p. 4
5. AUTHORS CONTRIBUTIONS
SS, a PhD student supervised by OW, developed the model and conducted the experiments; both authors read and approved the final article.