Deep Learning in Protein Structural Modeling and Design

• Protein structures are important for understanding health and disease and developing therapeutics, while computational prediction offers an alternative when experiments are limited.Source: §1 · p. 3
• Because nature has explored only a small part of protein sequence space, de novo design [7,8] can seek molecules with enhanced or novel properties.The review notes their potential to address challenges in biomedicine and biotechnology.Source: §1 · p. 3
• Deep learning’s adaptively learned, task-specific features have opened new avenues in protein modeling and can surpass traditional methods.Source: §1 · pp. 3–4
• The review covers deep-learning approaches for protein structure and function prediction, design, protein representation, and popular application frameworks.Source: §1 · p. 4
• It argues that structure is central to the sequence → structure → function paradigm and that structure-based approaches may be most fruitful.Source: §1 · p. 4
• CASP13 [4] showed deep-learning methods markedly improved model accuracy, especially on difficult targets where comparative modeling is ineffective.The review says these results demonstrate that neural networks can learn sequence-to-structure mappings and generalize to unseen cases.Source: §1 · p. 4, Figure 1 · p. 5

2.1 Problem definition

• Over 180 million UniProt sequences contrast with only 158,000 experimentally determined PDB structures, underscoring the need for computational structure prediction.Atomic-level protein and complex structures can help explain and engineer function.Source: §2.1 · p. 6
• Protein design seeks novel sequences that fold into a desired structure or perform a specific function, potentially exploring sequence space beyond naturally occurring proteins [7].Desired properties include greater thermal stability, resistance to degradation, faster catalysis, or tighter binding.Source: §2.1 · p. 6

2.2 Conventional computational approaches

• Conventional structure prediction follows the lowest-free-energy principle, but faces the sampling problem of exploring immense conformational space and the scoring problem of ranking conformations accurately [41,42].Current methods use multi-scale approaches, from coarse-grained large-scale sampling to higher-resolution modeling of side-chain packing and molecular interactions.Source: §2.2 · pp. 6–7
• Conventional design samples sequences that fold into a desired topology, either by modifying an existing protein or by generating unrelated sequences de novo.Both accurate prediction and reliable design remain challenging because conventional approaches depend on energy-function accuracy and efficient sampling of immense sequence and structure spaces [7,33].Source: §2.2 · p. 7

3 Deep learning architectures

• Supervised learning infers input-output relationships from examples, while empirical risk minimization seeks to minimize loss over the training samples.The examples can pair amino acid sequences with outputs such as residue pairwise distances.Source: §3 · p. 8

3.1 Convolutional neural networks (CNN)

• CNNs use convolutional kernels to capture translational invariance; ResNet skip connections aid optimization, and AlphaFold [22] maps residue-pair features to distance maps.CNNs are widely used in protein structure prediction; ResNets incorporate skip connections between layers [45].Source: §3.1 · pp. 9–10

3.2 Recurrent neural networks (RNN)

• RNNs apply the same function iteratively along sequences; protein applications predict the next amino acid, while vanishing or exploding gradients motivate LSTM [48] and GRU [49] variants.An example predicts the next amino acid from an N-terminal protein subsequence [47].Source: §3.2 · pp. 10–11

3.3 Variational auto-encoder (VAE)

• VAEs [51,52] stochastically map inputs to a latent space; protein sequence representations can support generation of new sequences likely to have antimicrobial activity [54].The example learns representations of antimicrobial protein sequences [54].Source: §3.3 · pp. 11–12

3.4 Generative adversarial network (GAN)

• GANs [55] train a generator to map samples from a simple distribution toward data and a discriminator to distinguish real from generated samples; WGAN [56] is a popular extension.The original GAN objective can suffer from mode collapse and training instability.Source: §3.4 · p. 12
• Wasserstein-1 distance between distributions can lead to easier and more robust GAN training in practice [57].Source: §3.4 · p. 12
• Anand et al. learned protein backbone-distance distributions to generate novel protein-like folds, with a generator producing folds and a discriminator distinguishing generated folds from fake folds [58].The approach is illustrated in Figure 3d.Source: §3.4 · p. 13

4.1 Amino acid sequence as representation

• One-hot sequence encodings are directly usable but sparse and sample-inefficient; sequences can be augmented with structural, evolutionary, and biophysical features, including MSAs and PSSMs.The amino acid sequence is widely used in function and structure prediction because it contains information essential to folding for most proteins.Source: §4.1 · pp. 13–14

4.2 Learned representation from amino acid sequence

• ProtVec achieved 93% accuracy for ~7,000 proteins in Pfam family classification, outperforming prior methods reporting 69.1–99.6% and 75% accuracy [71–74].ProtVec represents a protein by summing embeddings for overlapping sequence fragments of length k.Source: §4.2 · p. 15
• UniRep compared squared errors on 15 property-prediction tasks, including absorbance, activity, and stability, with former models such as Yang et al.’s Doc2Vec [76]; UniRep Fusion outperformed Rosetta in stability-ranking predictions.The passage notes that earlier models were trained on much smaller datasets (0.5 million).Source: §4.2 · pp. 15–16
• A VAE model captured evolutionary relationships between sequences and protein stability [79], while VAE latent vectors predicted mutation effects on fitness and activity across a range of proteins.Source: §4.2 · p. 16
• Conventional alignment-based inputs still outperform current self-supervised models on multiple tasks in TAPE, and a single task cannot assess model capacity [82].TAPE benchmarks protein representations across multiple biologically relevant semi-supervised learning tasks.Source: §4.2 · p. 16

4.3 Structure as representation

• Graph-based representations and GCNs support protein function prediction, with Class Activation Maps used to interpret structural determinants of function [99].Graph-based transformers have also been applied to protein sequence design [23].Source: §4.3 · p. 18
• MaSIF learns surface-patch fingerprints with geodesic convolution for binding-site prediction and rapid protein–protein interaction search, approaching existing docking and function-prediction baselines [101].Source: §4.3 · p. 18

4.4 Score function and force field

• Neural potentials reproduced energies, forces, and time-averaged properties comparable to AIMD while scaling linearly rather than cubically, though direct full-protein applications remain rare [113].Combining QM/MM with neural potentials enabled docking with small ligands and proteins up to 82 residues [116,117].Source: §4.4 · p. 19

4.5 Coarse-grained models

• CGNet learned a quantitatively correct free-energy surface and reproduced folded, unfolded, and misfolded metastable states; coarse-graining level is critical to model performance [122].The example maps each chignolin residue’s atoms to its Cα atom.Source: §4.5 · p. 20

5.1 Protein structure prediction

• An approach using information from many MSAs is ineffective when a target has few sequence homologs; RaptorX-Contact significantly improved accuracy on blind, challenging novel-fold targets and ranked first in CASP12 free-modeling (FM) targets.132Co-evolution information was a primary contributor to prediction accuracy.Source: §5.1 · pp. 21, 23
• At CASP13, AlphaFold (A7D/DeepMind) ranked first and significantly improved accuracy on free-modeling targets without available templates (22,26,137).Its gradient-based optimization performed slightly better than simulated annealing, and the authors emphasized the importance of learned distance distributions.Source: §5.1 · pp. 23–24
• On CASP13 targets, trRosetta improved prediction accuracy by training a shallower network to learn inter-residue orientations alongside β-carbon distances (87), using 61 rather than AlphaFold’s 220 ResNet blocks.Source: §5.1 · p. 24
• The end-to-end Recurrent Geometric Network predicts backbone torsions directly and produced structures within 1–2 Å of top-ranked CASP13 groups while offering faster prediction (85).RGN uses sequence, PSSM, and positional information without MSA-based information.Source: §5.1 · pp. 24–25

5.2 Related applications

• Protein–protein interface prediction identifies contacting residues; SASNet uses a 3D CNN over voxelized residue neighborhoods, while Fout et al. model partners as graphs with a GCN (83,96).Predicted interface residues can guide local search and scoring to determine complex structure.Source: §5.2 · p. 25
• RaptorX-Complex uses sequence information alone to predict protein–protein interactions and outperforms ECA-based methods at contact prediction [140].Source: §5.2 · p. 25
• MaSIF compares protein-patch geometry and predicts binding partners by nearest-neighbor scanning, achieving accuracy comparable to traditional docking but low accuracy on targets involving conformational changes [101].It trains by bringing complementary surface patches closer and separating non-interacting patches.Source: §5.2 · p. 25
• One-shot transfer learning from soluble proteins to membrane proteins works well because their most frequent contact patterns are similar; related efforts classify transmembrane topology [141, 142].Source: §5.2 · p. 26
• Loop modeling applies deep-learning methods based on inter-residue distance prediction and image inpainting to complete missing, often flexible segments [146, 147].Loops are difficult to capture experimentally or computationally.Source: §5.2 · p. 26
• A RaptorX-like network with trRosetta geometric representation predicts antibody hypervariable CDR H3 loop structure, which is critical for antigen binding [145].Source: §5.2 · p. 26

6 Protein design

• Deep-learning protein design uses sequence-informed generation for existing-like functionality or first stabilizes a chosen 3D structure, then seeks a compatible sequence.The first route relies on sequence information alone; the second follows the fold-before-function scheme and may or may not target a desired function.Source: §6 · p. 26

6.1 Direct design of sequence

• Six of 12 generated peptides killed MCF7 breast cancer cells with at least three-fold selectivity over human erythrocytes.The peptide models were trained on antimicrobial peptides and then applied to design membranolytic anticancer peptides.Source: §6.1 · p. 27
• Learned continuous latent spaces can interpolate between training sequences or extrapolate beyond training data to generate novel sequences.Autoencoders map discrete sequences to a continuous latent space that can represent a protein class and support mutation-effect prediction.Source: §6.1 · pp. 27–28
• More than 82% of PepCVAE-generated peptides were predicted to be antimicrobial by a state-of-the-art classifier.The semi-supervised VAE was trained on 1.7 × 10^6 unlabeled sequences, refined with 15,000 labeled AMP sequences, and conditioned on antimicrobial status for sampling.Source: §6.1 · p. 28
• ProteinGAN generated active, soluble malate dehydrogenase variants with a 24% success rate, while FBGAN biased sequences toward antimicrobial activity and desired secondary structure.ProteinGAN variants included some with over 100 mutations; FBGAN used an oracle-feedback loop to favor high-scoring sequences.Source: §6.1 · p. 28

6.2 Design with structure as intermediate

• Cα distance maps yielded the most meaningful generated structures, torsion representations recovered ψ and φ asymmetry, and designed folds were stable in simulation.A CVAE also generated sequences conditioned on protein topology represented as a string, with stability verified by molecular simulation.Source: §6.2 · pp. 29–30
• Wang et al. and O’Connell et al.’s SPIN2 further improved recovery rate to 34% (158,159), while another report found 70% recovery with experimentally validated mutation sites (161), though higher rates may not be meaningful.Source: §6.2 · pp. 29–30
• For five proteins, including a de novo TIM-barrel, designed sequences recovered 33%–87% native identities and predicted structures were 2–5 Å RMSD from native conformations.The protocol iteratively samples from predicted conditional amino-acid distributions for a given backbone.Source: §6.2 · p. 30
• gcWGAN had higher TM-score distributions under Rosetta-based structure prediction and more diverse sequence profiles than CVAE [163].Source: §6.2 · p. 30
• Ingraham et al.’s graph transformer [23] encodes structure as a graph and generates sequence profiles as a structure-to-sequence translation using an encoder-decoder with self-attention [167].The model dynamically learns relationships between information in neighboring layers.Source: §6.2 · p. 30
• Single-chain per-residue perplexity was 9.15, lower than SPIN2; lower perplexity is better [168].Source: §6.2 · p. 30

Experimental validation

• Experimental validation remains a major gap, especially in protein design, because computational models are prone to error; the authors call for collaboration between machine-learning experts and protein engineers.The review notes a few exceptions to the lack of experimental validation [149,153,161].Source: §7 · p. 31

Importance of benchmarking

• Standardized benchmarks such as CASP can support open competition, while well-defined questions and proper, especially experimental, evaluation may foster more diverse and powerful algorithms [170–172].Source: §7 · p. 31

Imposing a physics-based prior

• Protein modeling can use biophysical constraints to narrow plausible solutions, including physics-based priors, physically meaningful regularization, symmetry, backbone torsion distributions, and side-chain libraries [173,174,175,176,177,155,178,179].Source: §7 · pp. 31–32

Closed-loop design

• Public datasets may miss important sample space and contain harmful noise from non-uniform experimental protocols and conditions; a proposed solution combines model training with on-the-fly experimental-data generation for queries likely to improve the model, updating the training dataset.Source: §7 · p. 32

Reinforcement learning

• Reinforcement learning could address data limitations by generating biologically meaningful data in simulation, including training an agent to fold proteins through a sequence of moves in a Foldit-like task [187,188].Source: §7 · p. 32

Model interpretability

• Deep-learning models are often black boxes, and interpretability methods have been unsatisfactory or largely absent in protein modeling, leaving limited understanding of the patterns these models learn [189–194].Source: §7 · pp. 32–33

Sequence, structure, and context

• Sequence alone does not always determine native structure: some proteins need chaperones to fold, and folding kinetics can favor non-native conformations without them [41].Allosteric proteins can adopt different structures under physiological conditions or environmental stimuli, making context important [195].Source: §7 · p. 33

Structure-aware modeling

• Structure- or mechanism-aware deep learning can accelerate learning and improve predictive power, and the authors expect models with structural insights to play a more critical future role.Successful approaches such as convolutional and graph convolutional neural networks incorporate structural information.Source: §7 · pp. 33–34