Source-linked AI summary
Deep Supervised and Convolutional Generative Stochastic Network for Protein Secondary Structure Prediction
Jian Zhou, Olga G. Troyanskaya
TL;DR
Protein secondary-structure prediction must assign local structural states despite difficult long-range dependencies and limited homologous information. The paper develops a supervised convolutional GSN that samples labels conditioned on protein inputs using hierarchical representations, achieving superior 8-state prediction accuracy while retaining limitations for rare states and hard-coded spatial connectivity.
Problem
The paper addresses 8-state secondary-structure prediction for proteins without close homologs, where local labels depend on complex long-range interactions.
Method
The method extends GSNs to supervised conditional sampling and adds convolutional hierarchical representations that combine local and global information.
Results
The model demonstrates superior accuracy on 8-state protein secondary-structure prediction over previous methods.
Takeaways & Limitations
The experiments suggest supervised GSNs can effectively model structured prediction and may apply to other protein structure-prediction tasks.
Takeaways & Limitations
The convolutional structure is hard-coded and may not optimally capture spatial organization, especially for long-range interactions.
Abstract
from arXiv · showhide
Predicting protein secondary structure is a fundamental problem in protein structure prediction. Here we present a new supervised generative stochastic network (GSN) based method to predict local secondary structure with deep hierarchical representations. GSN is a recently proposed deep learning technique (Bengio & Thibodeau-Laufer, 2013) to globally train deep generative model. We present the supervised extension of GSN, which learns a Markov chain to sample from a conditional distribution, and applied it to protein structure prediction. To scale the model to full-sized, high-dimensional data, like protein sequences with hundreds of amino acids, we introduce a convolutional architecture, which allows efficient learning across multiple layers of hierarchical representations. Our architecture uniquely focuses on predicting structured low-level labels informed with both low and high-level representations learned by the model. In our application this corresponds to labeling the secondary structure state of each amino-acid residue. We trained and tested the model on separate sets of non-homologous proteins sharing less than 30% sequence identity. Our model achieves 66.4% Q8 accuracy on the CB513 dataset, better than the previously reported best performance 64.9% (Wang et al., 2011) for this challenging secondary structure prediction problem.
1. Introduction
The paper targets 8-state secondary-structure prediction for proteins without close homologs, where local labels depend on complex long-range interactions. It introduces supervised and convolutional GSN techniques to learn hierarchical representations while combining local and global information.
- Protein secondary-structure prediction is difficult for sequences lacking close homologs with known three-dimensional structures.
- 8-state prediction is presented as a challenging, less-addressed problem for proteins without close homologs.
- Long-range interactions, such as hydrogen bonds between distant β-strands, make global sequence information important for local structure labels.
- The supervised GSN learns a Markov chain for sampling outputs conditioned on inputs without explicitly marginalizing hidden variables.
- The convolutional architecture learns hierarchical representations for full-sized protein sequences and communicates high- and low-level features across spatial locations.
2. Preliminaries
GSNs learn data distributions through noisy reconstruction and Markov-chain sampling, while supervised GSNs adapt this framework to conditional structured prediction. The supervised model samples labels conditioned on inputs and trains reconstruction distributions without hidden-variable marginalization.
- A GSN learns a deep generative model by reconstructing corrupted inputs and sampling from a learned Markov chain.
- GSNs avoid intractable inference and explicit marginalization over latent variables by learning to sample directly from the data distribution.
- The supervised GSN captures the conditional dependency structure of outputs given inputs by sampling from P(Y |X).
- Training estimates Pθ(Y | ˜Y , X) from corrupted labels and inputs using a reconstruction cost interpretable as regularized conditional log likelihood.
- Prediction initializes labels arbitrarily and repeatedly samples through the learned Markov chain conditioned on X.
3. Algorithms
Convolutional GSNs use shared one-dimensional filters, pooling, noise, and recurrent bidirectional sampling to build hierarchical representations for sequential data. In supervised use, label channels are reconstructed while feature channels remain available as inputs.
- Convolutional GSN replaces fully connected spatial connectivity with one-dimensional convolutions suited to sequential protein data.
- A convolutional layer uses shared filters over local windows, producing feature maps from visible input channels.
- Noise enters through a stochastic activation function, and gradients through stochastic neurons use a straight-through estimator.
- Pooling reduces upper-layer feature-map size and supports higher-level representations by averaging adjacent features.
- Unlike feed-forward convolutional networks, recurrent convolutional GSN connections let different layers and spatial positions communicate through alternating upward and downward passes.
- For supervised prediction, the architecture corrupts and reconstructs label channels while leaving feature channels uncorrupted as inputs to hidden activations.
4. Experimental Results
The experiments evaluate the model on non-homologous protein datasets, compare architectural and training choices, and report strong 8-state secondary-structure prediction performance. Results also identify challenges for rare structure states and show benefits from deeper models and kick-start training.
- Features and dataset: The study targets 8-state secondary-structure and solvent-accessibility prediction using multi-task learning on protein sequences.The dataset contains 57 feature and label channels, with sequences padded to a maximum length of 700 amino acids.
- Performance: 72.1 ± 0.6% Q8 accuracy was achieved on the Cull PDB test sequences with the best tested 3-layer convolutional architecture.The architecture uses convolutional and mean-pooling layers with 80 channels in each convolutional layer.
- Performance: 0.664 Q8 accuracy was achieved on CB513, exceeding the previous CNF/Raptor-SS8 result of 0.649.The CB513 evaluation trained on Cull PDB sequences filtered to remove sequences sharing more than 25% identity with CB513.
- Analysis of architecture: The 3-layer model slightly outperforms the 2-layer model, while omitting kick-start training reduces convergence speed and prediction accuracy.The authors attribute kick-start’s benefit to learning a better reconstruction distribution from arbitrary initial states; rare states G, S, and especially I remain difficult.
5. Conclusions
The paper combines supervised GSNs with convolutional architectures for structured prediction, capturing local information while incorporating high-level and distant features. The authors report success on protein secondary structure prediction but identify hard-coded connectivity as a limitation for modeling long-range interactions.
- The method combines supervised generative stochastic networks with convolutional architectures to learn hierarchical representations for full-sized protein sequences.This combination is presented as an effective approach for structured prediction.
- The architecture targets low-level structured predictions using local information informed by high-level and distant features.The authors suggest this design may extend beyond bioinformatics to tasks such as scene parsing and image segmentation.
- The current convolutional structure is hard-coded and may not optimally capture protein sequence organization, especially for structures formed by long-range interactions.The authors propose adaptive, dynamic connectivity as a possible direction for improvement.