Source-linked AI summary
Convolutional LSTM Networks for Subcellular Localization of Proteins
Søren Kaae Sønderby, Casper Kaae Sønderby, Henrik Nielsen, Ole Winther
TL;DR
The paper addresses the difficulty of predicting protein subcellular localization from variable-length sequences without heavily engineered features. It evaluates LSTM-based models with convolutions, attention, and visualization methods, finding that sequence-only LSTM models achieve strong performance while yielding biologically interpretable filters, attention patterns, and representations.
Problem
Existing sequence-analysis methods such as SVMs and feed-forward networks do not naturally handle variable-length sequences and rely on heavily engineered features.
Method
The paper uses LSTM networks, including convolutional and attention-enhanced variants, to classify protein subcellular localization and visualize learned filters, sequence importance, and fixed-length representations.
Results
LSTM ensemble accuracy is 0.902 versus 0.767 for MultiLoc, while remaining close to SherLoc2 at 0.930 despite using sequence alone.
Takeaways & Limitations
The models can extract biologically interpretable sequence features, including plausible important regions, motif-like convolutional filters, and meaningful protein-sequence clusters.
Abstract
from arXiv · showhide
Machine learning is widely used to analyze biological sequence data. Non-sequential models such as SVMs or feed-forward neural networks are often used although they have no natural way of handling sequences of varying length. Recurrent neural networks such as the long short term memory (LSTM) model on the other hand are designed to handle sequences. In this study we demonstrate that LSTM networks predict the subcellular location of proteins given only the protein sequence with high accuracy (0.902) outperforming current state of the art algorithms. We further improve the performance by introducing convolutional filters and experiment with an attention mechanism which lets the LSTM focus on specific parts of the protein. Lastly we introduce new visualizations of both the convolutional filters and the attention mechanisms and show how they can be used to extract biological relevant knowledge from the LSTM networks.
1. INTRODUCTION
The paper applies LSTM networks to protein sequences to address limitations of conventional sequence-analysis methods, then extends them with convolutions, attention, and interpretable representations.
- Conventional neural networks and SVMs lack a natural way to handle biological sequences of varying length and require heavily engineered input features.
- LSTM networks are used to predict the subcellular compartment of a protein directly from its sequence.
- Convolutional filters are introduced for amino acid sequence analysis and visualized as interpretable motif detectors.
- An attention mechanism assigns importance to specific protein-sequence regions and enables visualization of where the network focuses.
- The LSTM extracts fixed-length representations from variable-length proteins, supporting visualization of biologically meaningful sequence clusters.
2. MATERIALS AND METHODS
The study develops LSTM-based models for protein subcellular localization from sequence, including regular, bidirectional, convolutional, and attention-based components. It also describes sequence preparation, model training, and visualizations of learned filters, attention weights, and hidden representations.
- LSTM network design: The LSTM methodology includes regular and attention-based networks for producing protein-localization predictions from sequence data.The attention mechanism weights hidden states across sequence positions before classification.
- LSTM network design: A regular LSTM predicts a single target from the final sequence position, but this requires information from earlier positions to reach and be stored in the last hidden state.The authors note that a regular bidirectional LSTM is not useful for this single-target setting because its backward network sees only x_T at the prediction position.
- Attention mechanism: The attention model assigns importance to each sequence position, combines hidden states into a context vector, and feeds that vector to a softmax classification network.The context vector is a convex combination of the sequence’s hidden states.
- Data preparation: The dataset contains 5959 proteins annotated to 11 subcellular locations, with sequences truncated to length 1000 while retaining both terminal regions containing sorting signals.Each amino acid was represented using 80 features derived from one-of-K encoding, substitution matrices, and sequence profiles.
- Interpretability: The study visualizes amino-acid convolutional filters as position-specific scoring matrices, attention weights by α, and sequence representations with t-SNE.The attention representation uses the context vector, whereas the regular LSTM representation uses the last hidden state.
- Model training: The architecture uses a 1D convolutional layer, an LSTM layer, a fully connected layer, and a final softmax layer, with 50% dropout throughout.Convolution sizes were 1, 3, 5, 9, 15, and 21, using 10 filters of each size; models were trained for 100 epochs with an 80/20 train-test split.
3. RESULTS
The sequence-only R-LSTM outperformed the A-LSTM and MultiLoc models, and a 10-model R-LSTM ensemble achieved the highest reported accuracy in the comparison.
- Accuracy results: 0.902 Acc. was achieved by the 10-ensemble R-LSTM model, exceeding the single R-LSTM model’s 0.879 Acc.The comparison used models trained on the same dataset and sequence information.
- Accuracy results: 0.879 Acc. was achieved by R-LSTM, compared with 0.854 Acc. for A-LSTM and 0.767 Acc. for MultiLoc using only sequence information.The passage reports that R-LSTM performed better than A-LSTM, while MultiLoc performed significantly worse.
4. DISCUSSION AND CONCLUSION
The convolutional LSTM approach predicts subcellular localization from sequence alone while producing biologically interpretable filters, attention patterns, and fixed-length representations. Its sequence-only performance exceeds MultiLoc and approaches methods using external information, while visualizations reveal plausible localization signals and sequence motifs.
- Performance: 0.902 accuracy for the LSTM ensemble exceeded MultiLoc’s 0.767 using sequence information alone.The ensemble nearly matched SherLoc2’s 0.930, despite SherLoc2 using information beyond the sequence.
- Performance: Sequence-only LSTM models can use both global and local sequence features, unlike the compared SVM-based models.
- Interpretability: Attention weights highlighted biologically plausible regions, including N-terminal signal peptides, retention signals, sorting peptides, and transmembrane-related signals.The model distinguished signal-peptide and transit-peptide patterns and showed additional C-terminal emphasis for some plasma-membrane proteins.
- Interpretability: Learned convolutional filters detected amino-acid characteristics and longer motifs, including charged regions, an arginine-rich motif, and a possible transmembrane-to-cytoplasmic transition.The authors interpret these filters as potential motif detectors for discovering sequence features.
- Representation: Fixed-length hidden representations generally clustered proteins by compartment, with biologically related categories grouped together and cytoplasmic and nuclear proteins overlapping.Lysosomal and vacuolar proteins were close in representation space, consistent with their considered homology.
- Conclusion: The paper concludes that convolutional LSTMs support sequence-based localization while providing biologically interpretable fixed-length vectors and visualizable motif detectors.