Source-linked AI summary
ProLanGO: Protein Function Prediction Using Neural~Machine Translation Based on a Recurrent Neural Network
Renzhi Cao, Colton Freitas, Leong Chan, Miao Sun, Haiqing Jiang, Zhangxin Chen
TL;DR
Protein function prediction must bridge rapidly growing sequence data and slower experimental characterization. ProLanGO reframes prediction as translation from a protein-sequence language to a Gene Ontology-function language using recurrent neural networks, and its combined model performs better than its component models on recall and precision while outperforming the sequence-only SMISS score on average. The authors identify remaining evaluation and modeling limitations, including a gap relative to the top homologous-based method.
Problem
Protein sequences can be generated quickly and cheaply, whereas extracting their functions experimentally remains slow and resource-demanding, leaving a gap between sequence volume and known function.
Method
ProLanGO represents protein sequences as k-mer-based language and Gene Ontology terms as alphabet-based language, then translates between them with a recurrent-neural-network neural machine translation model.
Results
ProLanGO’s combined NMT and extended-NMT model performs better than either component on recall and precision and better than SMISS’s sequence-only score on average across most thresholds.
Takeaways & Limitations
The reported results demonstrate the potential of ProLanGO as a new direction for protein function prediction from sequences.
Takeaways & Limitations
The evaluation used 332 proteins pending further releases, and the method remained behind the top homologous-based method; the authors also note limited k-mer choices and the need for cross-validation.
Abstract
from arXiv · showhide
With the development of next generation sequencing techniques, it is fast and cheap to determine protein sequences but relatively slow and expensive to extract useful information from protein sequences because of limitations of traditional biological experimental techniques. Protein function prediction has been a long standing challenge to fill the gap between the huge amount of protein sequences and the known function. In this paper, we propose a novel method to convert the protein function problem into a language translation problem by the new proposed protein sequence language "ProLan" to the protein function language "GOLan", and build a neural machine translation model based on recurrent neural networks to translate "ProLan" language to "GOLan" language. We blindly tested our method by attending the latest third Critical Assessment of Function Annotation (CAFA 3) in 2016, and also evaluate the performance of our methods on selected proteins whose function was released after CAFA competition. The good performance on the training and testing datasets demonstrates that our new proposed method is a promising direction for protein function prediction. In summary, we first time propose a method which converts the protein function prediction problem to a language translation problem and applies a neural machine translation model for protein function prediction.
1 Introduction
Protein sequencing has become fast and inexpensive, but determining protein function remains slower and resource-demanding. ProLanGO addresses this gap by translating protein sequences into predicted functions with a recurrent-neural-network language model.
- Next-generation sequencing can generate millions of protein sequences quickly and at low cost, increasing the need for computational function prediction.
- Traditional experiments for determining protein properties are slow, resource-demanding, and may not faithfully reflect activity in vivo.
- Protein function prediction is commonly framed as multi-label classification using sequence databases, interaction networks, structural or expression data, and combinations of these resources.
- Direct sequence-only prediction is more challenging because machine-learning methods must learn function categories from sequence-derived features without database or other-resource searches.
- ProLanGO converts protein sequences and functions into ProLan and GOLan, then uses recurrent-neural-network neural machine translation to predict functions.
- CAFA provides large-scale, unbiased benchmarking by evaluating predictions on proteins whose functions become known after the prediction period.
2 Method
ProLanGO converts protein sequences and functions into the ProLan and GOLan languages, then uses recurrent neural-network translation to predict protein functions. The method combines standard and extended translation models with a probability-based benchmark and evaluates predictions using top-n precision and recall.
- Language representations: ProLanGO represents protein sequences as ProLan and protein functions as GOLan, converting protein-function prediction into a language-translation problem.Protein “words” are generated from sequence fragments, while GO terms are encoded as Alphabet IDs.
- Language representations: Protein “words” are k-mers selected from UniProtKB, with longer matching fragments preferred and unmatched residues grouped into new words.The word database includes k-mers with frequency above 1,000 for k ∈[3,5].
- Language representations: GO terms are encoded as maximum four-letter Alphabet IDs assigned through depth-first traversal of the three Gene Ontology trees.The encoding preserves relationships in the Gene Ontology tree and orders multiple function IDs according to UniProtKB.
- Neural machine translation: A recurrent neural-network encoder maps ProLan sentences to fixed-length vectors, and a decoder generates GOLan sentences by maximizing conditional probability.Sampled softmax, bucketing, and padding reduce training or decoding complexity for the large vocabularies and variable sentence lengths.
- Extended prediction: The extended NMT model adds GO-term descendants, allows up to 60 predicted functions, and decreases scores for descendants farther from the original term.ProLanGO combines the NMT and extended NMT models using weights based on training performance.
3 Results
ProLanGO was trained on large UniProtKB datasets and evaluated on held-out proteins, selected proteins, and CAFA3 targets. It combined NMT variants and generally outperformed sequence-only SMISS predictions, while showing weaker performance for top-ranked GO terms and some comparisons.
- Training and evaluation: 419,192 protein sequences trained ProLanGO and 104,798 sequences tested its performance using perplexity and bucket-based evaluation.The model was also blindly tested on approximately 100,000 CAFA3 protein sequences.
- Selected-protein evaluation: The ProLanGO model combined NMT and extended NMT predictions, achieving better precision and recall than either component on selected proteins.The extended NMT model addressed proteins with more than 10 functions, while ProLanGO weighted both models according to training performance.
- CAFA3 comparison: On CAFA3, ProLanGO generally outperformed the sequence-only SMISS SEQ score but underperformed methods using additional information in several comparisons.PANNZER and the SMISS MIS score performed better, while ProLanGO had similar threshold-metric performance to DeepGO in some evaluations.
- Selected-protein evaluation: The selected-protein precision–recall AUC was 0.286 for NMT, 0.305 for extended NMT, 0.333 for ProLanGO, and 0.390 for the probability-based model.The comparison used the top n metric with n set to 20.
- CAFA3 comparison: ProLanGO performed poorly for top-one GO-term prediction, likely because its recurrent-neural-network output did not rank GO-term predictions.The authors identify GO-term ranking as an area for future improvement.
4 Conclusions
ProLanGO reframes protein function prediction as translation between sequence and Gene Ontology languages using recurrent-neural-network neural machine translation. Its performance is competitive with several methods, but the authors identify ranking, validation, representation, and evaluation improvements.
- 4 Conclusions: ProLanGO converts protein sequences and Gene Ontology terms into language spaces and translates between them with recurrent-neural-network neural machine translation.The method represents GO terms with at most four letters while considering their Gene Ontology directed-acyclic-graph relationships.
- 4 Conclusions: Figures 4 and 5 compare average GO similarity for ProLanGO and other methods using threshold and top n metrics.The supplied figure descriptions identify the two evaluation metrics but do not report their numerical values.
- 4 Conclusions: ProLanGO combines NMT and extended NMT models, outperforming either component on recall and precision while matching the probability-based model.The comparison also included four state-of-the-art protein function prediction methods.
- 4 Conclusions: ProLanGO outperforms the SEQ score from SMISS but remains behind the top homologous-based method.This comparison was based on an evaluation of 332 selected proteins.
- 4 Conclusions: A more rigorous evaluation was deferred until additional CAFA proteins and official results became available.The authors state that later CAFA releases would enable broader comparison with other protein function prediction methods.
- 4 Conclusions: The authors propose cross-validation, larger k-mer sizes, biologically meaningful fragments, alternative bucket sizes, GO-term ranking, and k-mer ordering as future improvements.These changes are presented as ways to improve reliability or performance.