PaperlayerView exact revision on arXiv

Source-linked AI summary

Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds

Yeqing Lin, Mohammed AlQuraishi

arXiv:2301.12485v3q-bio.BMcs.LG

TL;DR

Search-based protein design is challenging across the large space of sequences and structures. Genie uses diffusion with residue reference frames to generate protein backbones, and the authors report strong design metrics, including higher confident designability than comparison models.

  • Problem

    Search-based protein design is limited by the combinatorially large space of protein sequences and structures, while existing approaches have explored only a small region of fold space.

    Source: §1 · p. 1

  • Method

    Genie is a diffusion model that noises Cα coordinates and uses an SE(3)-equivariant denoiser reasoning over residue reference frames to generate protein backbones.

    Source: §1 · p. 2, §4 · p. 9

  • Results

    58.3% of Genie domains were confidently designable, compared with 3.2% for ProtDiff and 17.7% for FoldingDiff.

    Source: §3.1.1 · p. 5

  • Takeaways & Limitations

    Genie generates diverse, designable, and novel structures and achieves state-of-the-art performance on key design metrics.

    Source: §1 · p. 2

    The current versions appear capacity-limited, making model scaling a stated future direction.

    Source: §4 · p. 9

Abstract

from arXiv · show

Proteins power a vast array of functional processes in living cells. The capability to create new proteins with designed structures and functions would thus enable the engineering of cellular behavior and development of protein-based therapeutics and materials. Structure-based protein design aims to find structures that are designable (can be realized by a protein sequence), novel (have dissimilar geometry from natural proteins), and diverse (span a wide range of geometries). While advances in protein structure prediction have made it possible to predict structures of novel protein sequences, the combinatorially large space of sequences and structures limits the practicality of search-based methods. Generative models provide a compelling alternative, by implicitly learning the low-dimensional structure of complex data distributions. Here, we leverage recent advances in denoising diffusion probabilistic models and equivariant neural networks to develop Genie, a generative model of protein structures that performs discrete-time diffusion using a cloud of oriented reference frames in 3D space. Through in silico evaluations, we demonstrate that Genie generates protein backbones that are more designable, novel, and diverse than existing models. This indicates that Genie is capturing key aspects of the distribution of protein structure space and facilitates protein design with high success rates. Code for generating new proteins and training new versions of Genie is available at https://github.com/aqlaboratory/genie.

1. Introduction

Earlier protein-generation approaches faced limits in how they represented geometry or explored structure space. Genie combines diffusion with residue reference frames to generate structures while retaining Gaussian noising and orientation-aware generation.

1. Introduction

  • Distance-matrix generation lacks a guarantee of 3D embeddability, while internal-coordinate diffusion relies heavily on local geometry and often yields structures predicted not to be designable.Distance-matrix errors can degrade structural quality and prevent end-to-end optimization for final 3D geometry.Source: §1 · p. 2
  • ProtDiff generates Cα coordinates with an EGNN, but reflection invariance can produce left-handed α-helices, which are exceedingly rare in real proteins.The paper identifies this as one potential reason ProtDiff struggles to produce geometries with realizable protein sequences.Source: §1 · p. 2
  • Anand and Achim (2022) combined Invariant Point Attention with diffusion over residue frames, but their rotation-matrix noise does not satisfy the original DDPM Gaussian assumption.Their approach generated structures and sequences conditioned on secondary-structure topology.Source: §1 · p. 2
  • Genie uses point clouds in the forward process and reference frames in the reverse process, retaining Gaussian noising while enabling diverse, designable, and novel structure generation.The authors report state-of-the-art performance on key design metrics.Source: §1 · p. 2

2. Methods

Genie diffuses Cα-coordinate sequences and predicts noise using an SE(3)-equivariant denoiser that reasons over residue frames. Training minimizes noise-prediction error, and sampling reverses the diffusion process from random coordinates.

2. Methods

  • Genie generates protein backbones as Cα-coordinate sequences and uses an SE(3)-equivariant denoiser to predict noise displacements at each diffusion step.Its denoiser reasons over a cloud of reference frames.Source: §2 · pp. 2–3

2.1. Denoising Diffusion Probabilistic Model

  • The forward process adds isotropic Gaussian noise to protein coordinates under a cosine variance schedule over 1,000 steps; the reverse process iteratively removes noise.Generation starts from white noise and repeatedly predicts the noise added at each step.Source: §2.1 · p. 3

2.2. Noise Prediction

  • At each step, Genie converts noisy Cα coordinates into discrete Frenet–Serret frames, applies invariant encoding and equivariant frame updates, then predicts coordinate displacements.The frames encode residue position and orientation; their rotations and translations are updated by the decoder.Source: §2.2 · p. 3

2.3. Training

  • Training minimizes per-residue L2 error between sampled noise and predicted noise, using SCOPe and filtered AlphaFold-SwissProt structures.The training sets contain 8,766 SCOPe domains and 195,214 AlphaFold-SwissProt structures with pLDDT > 80.Source: §2.3 · pp. 4–5

2.4. Sampling

  • To sample a backbone, Genie draws random Cα coordinates from a standard Gaussian and recursively applies reverse-diffusion updates until step 0.The update rule also uses Gaussian noise vectors.Source: §2.4 · p. 5

3. Results

Genie outperforms ProtDiff and FoldingDiff on designability, diversity, and novelty in short-protein comparisons, and shows competitive performance against long-protein models. The long-model results reveal trade-offs across designability, structural diversity, novelty, and generation speed.

3.1. Comparisons with short models

  • Genie outperforms ProtDiff and FoldingDiff on all three assessed criteria: designability, diversity, and novelty.Source: §3.1 · p. 5

3.1.1. DESIGNABILITY

  • 58.3% of Genie domains are confidently designable, versus 3.2% for ProtDiff and 17.7% for FoldingDiff.Using scTM > 0.5 alone, Genie reaches 81.5%, versus ProtDiff’s 5.1% (retrained) and 11.8% (reported), and FoldingDiff’s 19.6% (resampled) and 22.7% (reported).Source: §3.1.1 · p. 5, Figure 2 · p. 6

3.1.3. NOVELTY

  • 98 out of 455 Genie structures (21.5%) are novel, compared with ProtDiff’s 1 out of 25 (4%) and FoldingDiff’s 28 out of 138 (20.3%).Source: Figure 2 · p. 6, §3.1.3 · pp. 6–7

3.1.2. DIVERSITY

  • Genie’s designed domains show more secondary-structure variety, while its mean maximum TM score to other designs is 0.561 ± 0.086, below ProtDiff’s 0.583 ± 0.115 and FoldingDiff’s 0.668 ± 0.178.Source: §3.1.2 · p. 6, Figure 2 · p. 6

3.2.1. DESIGNABILITY

  • Genie outperforms FrameDiff in sampling quality, even with the smaller SCOPe training set, but underperforms RFDiffusion; more training data improves Genie’s quality.Source: §3.2.1 · p. 7, Table 1 · p. 8

3.2.2. DIVERSITY

  • RFDiffusion leads tertiary diversity, followed by both Genie variants and then FrameDiff; more training data raises Genie’s designability but reduces diversity.For SSE content, FrameDiff and RFDiffusion show greater diversity than Genie, particularly for beta sheets.Source: §3.2.2 · pp. 7–8, Figure 3 · p. 8, Table 1 · p. 8

3.2.3. NOVELTY; 3.2.4. SPEED

3.3. Visualization of Design Space

  • The 818 confidently designable Genie-SwissProt structures span the MDS design space, with eight novel examples selected from diverse embedding locations.The short Genie model shows similar trends and generates more diverse structures than Genie-SwissProt.Source: §3.3 · pp. 8–9, Figure 4 · p. 9

4. Conclusion

Genie combines Cartesian-space diffusion with frame-based denoising to balance simple noising with expressive structural reasoning. The appendices describe its diffusion, encodings, encoder-decoder architecture, and hyperparameter tables.

4. Conclusion

  • Genie substantially outperforms short structure-based models and performs competitively with much larger long models.Its dual residue representations use Cartesian Cα coordinates for Gaussian noising and Frenet–Serret frames for noise prediction; the denoiser combines Invariant Point Attention with backbone updates.Source: §4 · p. 9
  • The authors identify model scaling, concurrent sequence generation, and conditional structure generation as future directions.They describe current Genie versions as apparently capacity-limited and identify classifier-guided generation toward desired properties as promising.Source: §4 · p. 9

A.1. Illustration of Protein Backbone Diffusion

  • The forward process adds isotropic Gaussian noise to Cα coordinates, and the reverse process denoises them with an SE(3)-equivariant model.Figure 5 illustrates this Cartesian-space diffusion process.Source: §A.1 · p. 12, Figure 5 · p. 12

A.2. Encodings

  • The encodings cover diffusion steps and residue indices, while relative positions are clipped at k, one-hot encoded, and projected to pair-representation dimensions.Residue pairs more than k positions apart receive the same learned encoding.Source: §A.2 · p. 12

A.3. SE(3)-Invariant Encoder

  • The invariant encoder forms single representations from step and residue-index encodings, then builds pair representations from relative position, single representations, and pairwise distances.Its pair-transform network uses five layers of triangular multiplicative updates with pair transitions.Source: §A.3 · p. 13

A.4. SE(3)-Equivariant Decoder

  • The equivariant decoder uses Invariant Point Attention to refine single representations and a backbone-update module to update residue frames.The backbone-update network projects updated single representations into translation and rotation updates applied to the input frames.Source: §A.4 · pp. 13–14

A.5. Model Hyperparameters

A.6. Training

Genie training uses Adam and data-parallel training, with different GPU, batch-size, and training-duration configurations for the short model and two dataset-specific versions.

A.6. Training

  • The short Genie model was trained for 50,000 epochs, about nine days, on two A100 GPUs with effective batch size 48.Training used Adam with learning rate 10^-4.Source: §A.6 · p. 14
  • Genie-SCOPe was trained for 30,000 epochs, about two weeks, on 12 A100 GPUs with effective batch size 48.Source: §A.6 · p. 14
  • Genie-SwissProt was trained for 100 epochs, about eight days, on six A6000 GPUs with effective batch size 24.Source: §A.6 · p. 14

A.7. Evaluation

The evaluation appendix describes retraining choices for ProtDiff and FoldingDiff, and illustrates the self-consistency pipeline used to assess generated structures.

A.7. Evaluation

  • ProtDiff was retrained on the filtered SCOPe dataset because its original training dataset and pretrained weights were unavailable.The retrained model performed slightly worse than the reported model, possibly because of differences in training and dataset size.Source: §A.7 · p. 14
  • FoldingDiff was retrained on filtered SCOPe, but its retrained performance degraded, mainly due to dataset-size differences; comparisons with Genie used the better-performing original weights.The retrained model was compared with the original using scRMSD versus pLDDT and scTM versus pLDDT distributions.Source: §A.7 · pp. 14–15, Figure 8 · p. 15
  • FrameDiff and RFDiffusion were evaluated using their provided weights without retraining.Source: §A.7 · p. 15

B.1. Illustration of the Self-consistency Pipeline

  • The self-consistency pipeline passes a generated structure through inverse folding and structure prediction, then compares the generated and designed structures with scTM and scRMSD.scTM is their TM score; scRMSD is their root mean square deviation.Source: Figure 9 · p. 16, §B.1 · p. 16

B.2. Visualization of the Distribution of scTM versus scRMSD

For Genie-SCOPe and Genie-SwissProt, some structures pass the scTM threshold but fail the scRMSD threshold; every structure passing the scRMSD threshold also passes scTM.

B.2. Visualization of the Distribution of scTM versus scRMSD

  • 33.8% of Genie-SCOPe structures and 18.3% of Genie-SwissProt structures have scTM > 0.5 but scRMSD > 2.These cases suggest a mismatch between generated and designed structures.Source: §B.2 · p. 17, Figure 10 · p. 17
  • Every generated structure with scRMSD < 2 also satisfies scTM > 0.5.Source: §B.2 · p. 17
  • The asymmetric thresholds make scRMSD a more stringent designability measure than scTM in these evaluations.Source: §B.2 · p. 17

C.1. Extra Evaluations using ProteinMPNN-ESMFold scTM Pipeline

Additional evaluation with ESMFold preserves the overall trends seen with OmegaFold, while visualizations summarize designability and the range of Genie’s generated structures.

C.1. Extra Evaluations using ProteinMPNN-ESMFold scTM Pipeline

  • With ESMFold replacing OmegaFold for scTM calculations, overall trends remain unchanged except for fewer confidently designable domains containing β-strands.Source: §C.1 · p. 18
  • Figure 11 summarizes ESMFold evaluations through heatmaps of scTM/pLDDT combinations, SSE content, and SCOPe reference frequencies, plus sequence-length and designable-domain counts.Each method’s designable-domain count is shown against a fixed budget of 780 attempted designs.Source: Figure 11 · p. 18

C.2. Visualization of Genie’s Design Space

  • Figure 12 embeds 455 confidently designable Genie structures in 2D using pairwise TM scores, with panels coloring domains by training-set similarity, secondary-structure fractions, and sequence length.Eight novel designed domains are included as representatives.Source: Figure 12 · p. 19

C.3. Visualizations of Genie-Generated Protein Backbones

The appendix provides additional visual examples of Genie-generated domains described as novel and confidently designable.

C.3. Visualizations of Genie-Generated Protein Backbones

D.1. Discussion on the Effect of Sampling Noise Scale

Lower sampling noise generally improves structure quality at the cost of diversity, while the appendix visualizes quality distributions, designability, novelty, and additional generated examples.

D.1. Discussion on the Effect of Sampling Noise Scale

  • At sampling noise scale 0.4, Genie achieves an optimal balance between sample quality and sample diversity.Generally, lower sampling noise leads to higher-quality structures but lower diversity.Source: §D.1 · p. 21
  • Figure 14 presents additional examples of novel and confidently designable domains generated by Genie.Source: Figure 14 · p. 21

D.2. Visualizations of the Distribution of scRMSD/scTM versus pLDDT for Long Models

  • Figure 15 heatmaps show relative frequencies for lowest scRMSD and pLDDT, and highest scTM and pLDDT; denser top-left or top-right regions indicate higher sampling quality.Source: Figure 15 · p. 22

D.3. Visualization of Tertiary Diversity versus Designability

  • Figure 16 compares lowest scRMSD with maximum TM scores to generated structures, with lower-left density indicating higher designability and tertiary diversity.Source: Figure 16 · p. 23

D.4. Visualization of Novelty versus Designability

  • Figure 17 compares lowest scRMSD with maximum TM scores to PDB structures, with lower-left density indicating higher designability and novelty.Source: Figure 17 · p. 24

D.5. Visualizations of Protein Backbones Generated by Genie-SwissProt

  • Figure 18 presents additional examples of novel and confidently designable structures generated by Genie-SwissProt.Source: Figure 18 · p. 25

Related papers

Loading 2301.12485v3…