Source-linked AI summary
Deep Learning in Protein Structural Modeling and Design
Wenhao Gao, Sai Pooja Mahajan, Jeremias Sulam, Jeffrey J. Gray
TL;DR
Protein structural modeling remains challenging because obtaining protein structures is important while conventional approaches face energy-accuracy and sampling problems. This review examines deep learning methods for protein structural modeling and design, highlighting improved prediction accuracy and the future importance of structural insights.
Problem
Protein structure modeling remains challenging because conventional approaches rely heavily on energy accuracy and face a sampling problem known as Levinthal’s paradox.
Method
The review examines deep learning approaches for protein structural modeling, including architectures that combine sequence, predicted structural, solvent-accessibility, and pairwise features to predict inter-residue distances.
Results
Deep learning methods achieved a striking improvement in protein structure prediction accuracy, especially for difficult targets.
Takeaways & Limitations
Models incorporating structural insights may play a more critical future role as protein sequences become abundant.
Takeaways & Limitations
An important gap in current deep learning work in protein modeling, especially protein design, is the lack of experimental validation.
Abstract
from arXiv · showhide
Deep learning is catalyzing a scientific revolution fueled by big data, accessible toolkits, and powerful computational resources, impacting many fields including protein structural modeling. Protein structural modeling, such as predicting structure from amino acid sequence and evolutionary information, designing proteins toward desirable functionality, or predicting properties or behavior of a protein, is critical to understand and engineer biological systems at the molecular level. In this review, we summarize the recent advances in applying deep learning techniques to tackle problems in protein structural modeling and design. We dissect the emerging approaches using deep learning techniques for protein structural modeling, and discuss advances and challenges that must be addressed. We argue for the central importance of structure, following the "sequence -> structure -> function" paradigm. This review is directed to help both computational biologists to gain familiarity with the deep learning methods applied in protein modeling, and computer scientists to gain perspective on the biologically meaningful problems that may benefit from deep learning techniques.
1 Introduction
Protein structural modeling is important for understanding and engineering biology, but experimental structures are limited and computational prediction remains difficult. This review surveys deep learning approaches across protein modeling and design, emphasizing structure in the sequence → structure → function paradigm.
- Protein structures support understanding health and disease and developing therapeutic molecules, motivating computational alternatives when experiments are limited.
- Deep learning adaptively learns task-specific features from data and has opened new avenues for protein modeling.
- CASP13 showed striking accuracy improvements for deep-learning structure prediction, especially on difficult targets where comparative modeling is ineffective.
- The review covers protein function and structure prediction, protein design, representations, and widely used deep-learning frameworks.
- The authors emphasize protein structure as central to the sequence → structure → function paradigm and argue that structure-based approaches may be most fruitful.
2 Protein structure prediction and design
Protein structure prediction and design seek to connect amino acid sequences, three-dimensional structures, and desired functions, but both remain challenging. Conventional methods face immense sampling spaces and depend on accurate energy functions, while design explores sequences compatible with desired topologies.
- Structure prediction maps amino acid sequence to three-dimensional structure, whereas design seeks sequences that fold into desired structures or perform specific functions.
- Atomic-level protein structures often provide the starting point for understanding, modulating, or engineering protein function.
- More than 180 million protein sequences contrast with only 158,000 experimentally determined structures, creating a major data imbalance for structural modeling.
- De novo design explores sequence space beyond naturally occurring proteins to seek robustness or enhanced properties, but current approaches require substantial domain expertise.
- Protein design samples sequences that fold into a desired topology and includes modifying existing proteins or generating novel sequences.
- Conventional prediction relies on sampling immense conformational spaces and accurately scoring candidate states with free-energy functions.
3 Deep learning architectures
The review introduces deep learning architectures for protein modeling, framing them as ways to learn predictive functions, sequence representations, latent spaces, or generative mappings. CNNs support structure prediction, RNNs support autoregressive sequence processing, VAEs learn latent representations, and GANs generate protein-like folds.
- General framework: Deep learning models learn relationships between inputs such as amino acid sequences and outputs such as residue pairwise distances by minimizing empirical loss.Neural networks parameterize the function class with weights optimized over training samples.
- 3.1 Convolutional neural networks (CNN): CNNs use convolutional kernels and are widely applied to protein structure prediction, including AlphaFold’s residue-pair features to distance-map pipeline.Residual networks add skip-connections between layers.
- 3.2 Recurrent neural networks (RNN): RNNs apply shared transformations recurrently along sequences, retaining prior information and accepting variable-length inputs for autoregressive amino-acid prediction.LSTM and GRU variants are more widely used because standard RNNs suffer from vanishing or exploding gradients.
- 3.3 Variational auto-encoder (VAE): VAEs learn stochastic mappings between complex input spaces and simpler latent spaces, training encoders and decoders by maximizing the ELBO.The KL divergence term measures the distance between the approximate and model posterior distributions.
- Generative models: In protein modeling, VAE representations can generate antimicrobial sequences, while GANs adversarially map simple prior samples into design spaces and generate novel protein-like folds.The GAN setup uses a generator and discriminator trained through an adversarial game.
4 Protein representation and function prediction
Protein function prediction is commonly formulated as supervised learning, making representation choice central because sequences, structures, and coarse-grained models expose different information and computational challenges. Learned sequence representations show promising task performance, but comparisons indicate that conventional alignment-based inputs can still outperform self-supervised models across multiple tasks.
- Function prediction: Function prediction uses supervised learning to predict protein-level or residue-level properties, with representation serving as the input to prediction or output of generation.Examples include enzyme classification and phosphorylation or polyadenylation site prediction.
- Representation schemes: Protein representations include sequence-based, structure-based, and coarse-grained forms, with sequence inputs often augmented by evolutionary and biophysical features.Additional features include MSAs, PSSMs, residue co-evolution, predicted secondary structure, hydrophobicity, charge, and solvent accessibility.
- Amino acid sequence: One-hot sequence encodings are directly usable by many models but are sparse and sample-inefficient, motivating learned or augmented representations.Deep models can learn representations from abundant unlabeled sequence data, potentially reducing labeled-sample requirements.
- Learned sequence representations: ProtVec predictions achieved 93% accuracy for approximately 7,000 proteins in Pfam classification, compared with 69.1–99.6% and 75% for previous methods.The approach represents proteins using summed embeddings of overlapping k-mers.
- Learned sequence representations: UniRep achieved lower mean squared errors on 15 property-prediction tasks than former models, and UniRep Fusion outperformed Rosetta for stability ranking predictions.The comparison with Rosetta concerns a model using sequence, structure, and a biophysically trained scoring function.
- Benchmarking representations: TAPE results showed that conventional alignment-based inputs still outperform current self-supervised models on multiple tasks, while single-task evaluation cannot assess model capacity.The review therefore calls for comprehensive comparisons of protein representations.
- Coarse-grained and physical modeling: Neural potentials reproduced energies, forces, and time-averaged properties at linear cost scaling versus cubic scaling typical for AIMD with DFT, but full-protein applications remain rare.Reported applications include docking with small ligands and proteins up to 82 residues.
5 Structure determination
Deep learning reframed protein structure prediction as supervised mapping from amino acid sequences and evolutionary information to 3D structures. Residual networks and end-to-end differentiable models produced major gains in CASP evaluations.
- Problem: Protein structure prediction maps amino acid sequences to 3D structures using experimental structures as ground truth.The formulation fits supervised learning, with the main challenge being the complex input-output relationship.
- Deep residual networks: RaptorX-Contact used a deep dilated ResNet to convert sequence-derived and pairwise features into discretized inter-residue distance predictions.Its inputs included sequence profiles, predicted secondary structure, solvent accessibility, co-evolution information, contacts, and distance potentials.
- CASP12: RaptorX-Contact ranked first on free-modeling targets at CASP12 and significantly improved accuracy on blind targets with novel folds.Its accuracy was attributed primarily to co-evolution information from CCMpred and the depth of the 2D ResNet.
- CASP13: AlphaFold ranked first at CASP13 and significantly improved prediction accuracy using learned distance distributions and deep neural-network-based folding protocols.Gradient-based optimization performed slightly better than simulated annealing, suggesting that optimizing a network-predicted potential could reach state-of-the-art performance.
- Geometric features: trRosetta improved a relatively fixed deep-learning framework by predicting inter-residue orientations alongside β-carbon distances.The added geometric outputs directly described coordinates relevant to side-chain interactions and produced significant improvement.
6 Protein design
Deep learning protein design methods generate sequences from learned sequence distributions or design sequences compatible with specified folds and structures. The review highlights experimental activity, property enrichment, and the continuing challenge of representation and validation.
- Design paradigms: Protein design divides into sequence-first approaches that use other sequences and fold-before-function approaches that stabilize specified 3D structures.These correspond broadly to function→sequence design and structure-guided inverse design.
- Sequence generation: Autoregressive models generate new sequences residue by residue from categorical amino-acid distributions learned from training data.The sampled residue becomes input for predicting the following position, with the goal of matching training-set properties.
- Sequence-first design: Six of twelve generated anticancer peptides killed MCF7 cells with at least three-fold selectivity against human erythrocytes.The peptides were produced by an LSTM-based framework trained on antimicrobial and anticancer peptide sequence patterns.
- Latent-space models: More than 82% of peptides generated by a conditionally refined VAE were predicted to have antimicrobial properties by a state-of-the-art AMP classifier.The model used 1.7 × 10^6 unlabeled sequences, a 15,000-sequence labeled AMP dataset, and a conditional antimicrobial code.
- Representation and validation: Protein design outcomes depend strongly on structural representation: Cα distance maps produced the most meaningful structures, while torsion representations recovered φ and ψ asymmetry.Other approaches reported stable designed folds in molecular simulation and higher TM-score distributions with more diverse sequence profiles than CVAE.
- Structure-conditioned design: SPIN2 reached a 34% sequence-recovery rate for designing amino-acid sequences compatible with target structures.Because sequence-to-structure mapping is not unique, the review notes that higher recovery may not necessarily indicate more meaningful design.
7 Outlook and conclusion
The review surveys deep-learning approaches for protein modeling and design while identifying experimental validation, benchmarking, physics-based priors, data quality, reinforcement learning, and interpretability as central challenges and opportunities. It emphasizes structure-aware modeling because sequence alone does not always determine native structure, and argues that structural insights will become increasingly important.
- Review scope: The review summarizes current deep-learning techniques for protein modeling and discusses their potential advantages and disadvantages.Its scope includes protein structure and function prediction and protein design.
- Experimental validation: Experimental validation remains a major gap, especially in protein design, because computational models are still prone to error.The review calls for collaboration between machine-learning experts and experimental protein engineers to test emerging approaches.
- Benchmarking: Standardized benchmarks and well-defined questions with proper, especially experimental, evaluation could broaden competition and support more diverse, powerful algorithms.CASP is cited as a standardized platform for benchmarking diverse algorithms, including deep-learning approaches.
- Physics-based priors: Physics-based priors and empirical structural knowledge can restrict plausible solutions and reduce training effort in protein modeling.Examples include backbone torsion-angle distributions and side-chain conformation libraries.
- Data and experimentation: Closed-loop strategies could generate experimental data for informative queries and update training datasets, addressing incomplete and noisy public datasets.The approach depends on feasible automated synthesis and characterization, while high-throughput protein, DNA, and RNA testing can be performed in parallel.
- Structure-aware modeling: Structure-aware or mechanism-aware frameworks can accelerate learning and improve predictive power, whereas sequence alone may fail when context, chaperones, or folding kinetics affect structure.Many successful frameworks, including convolutional and graph convolutional neural networks, incorporate structural information.