Source-linked AI summary
Protein structure generation via folding diffusion
Kevin E. Wu, Kevin K. Yang, Rianne van den Berg, James Y. Zou, Alex X. Lu, Ava P. Amini
TL;DR
Generating diverse, physically foldable protein structures directly remains difficult, despite its potential relevance to biological discovery and therapeutic development. FoldingDiff addresses this by diffusing over shift- and rotation-invariant backbone angles with a simple transformer, producing realistic structures with natural structural patterns and measurable designability.
Problem
Direct neural generation of diverse, realistic protein structures remains difficult, while existing methods often generate constraints that require post-processing.
Method
FoldingDiff applies a denoising diffusion probabilistic model with a vanilla transformer to consecutive internal backbone angles rather than Cartesian coordinates.
Results
177 of 780 generated structures, or 22.7%, are designable with scTM ≥0.5 without refinement or relaxation.
Takeaways & Limitations
The model generates high-quality, biologically plausible, diverse protein structures that respect chirality and exhibit high designability.
Takeaways & Limitations
Generated structures are relatively short compared with natural proteins, which typically contain several hundred residues.
Abstract
from arXiv · showhide
The ability to computationally generate novel yet physically foldable protein structures could lead to new biological discoveries and new treatments targeting yet incurable diseases. Despite recent advances in protein structure prediction, directly generating diverse, novel protein structures from neural networks remains difficult. In this work, we present a new diffusion-based generative model that designs protein backbone structures via a procedure that mirrors the native folding process. We describe protein backbone structure as a series of consecutive angles capturing the relative orientation of the constituent amino acid residues, and generate new structures by denoising from a random, unfolded state towards a stable folded structure. Not only does this mirror how proteins biologically twist into energetically favorable conformations, the inherent shift and rotational invariance of this representation crucially alleviates the need for complex equivariant networks. We train a denoising diffusion probabilistic model with a simple transformer backbone and demonstrate that our resulting model unconditionally generates highly realistic protein structures with complexity and structural patterns akin to those of naturally-occurring proteins. As a useful resource, we release the first open-source codebase and trained models for protein structure diffusion.
1 Introduction
Computational protein generation aims to produce diverse, realistic folds, but existing approaches rely on expert-designed fragments or intermediate constraints requiring post-processing. FoldingDiff instead generates protein backbones directly with diffusion over internal angles.
- Computationally generating new protein structures could support biological discoveries and treatments for currently incurable diseases.
- Existing methods struggle to create protein folds that are both diverse and realistic.
- Traditional methods assemble experimentally profiled fragments, limiting generation by expert knowledge and available data.
- Many deep generative models produce constraints such as residue pairwise distances rather than structures, requiring substantial post-processing.
- FoldingDiff generates protein backbones by diffusing inter-residue angles with a denoising model and a vanilla transformer.
2 Related work
Earlier protein-generation methods depend on known structures, post-processing, or equivariant models. Diffusion-based approaches broaden generative modeling, while FoldingDiff operates directly on relative angles instead of Cartesian coordinates.
- Some generative models sample pairwise distance matrices that require correction, refinement, and conversion into realizable backbones.
- Fragment assembly and heuristic perturbation methods rely on known proteins and limit structures to relatively small deviations from natural proteins.
- Recent protein diffusion methods predominantly model Cartesian coordinates and therefore use equivariance to translation, rotation, and/or reflection.
- ProteinSGM generates image-like matrices of pairwise angles and distances rather than directly generating backbone structures.
- Small-molecule diffusion has modeled coordinates or torsional angles, but coordinate-based approaches still require equivariant processing.
3 Method
FoldingDiff represents protein backbones as sequences of internal angles and applies periodic diffusion from disordered noise toward folded structures. This representation enables a vanilla transformer while the model is trained and evaluated on protein backbones.
- Simplified framing: The six modeled angles comprise three dihedral torsion angles and three bond angles.
- Simplified framing: FoldingDiff models a length-N backbone as a sequence of six consecutive internal angles in [−π, π)^(N−1)×6.
- Simplified framing: Each residue acts as an independent reference frame, making the angle representation shift- and rotation-invariant and removing the need for an equivariant neural network.
- Denoising diffusion: The diffusion process corrupts folded angles with noise until they resemble random angles, then trains a reverse model to denoise them toward protein-like samples.
- Denoising diffusion: The forward process uses wrapped-normal noise for periodic angular values, with a cosine variance schedule over T = 1000 timesteps.
- Denoising diffusion: Training predicts injected noise with a wrapped smooth L1 loss, while angles are normalized using an element-wise angular mean shift.
- Sampling: Sampling refines internal angles from an unfolded state toward a folded state, motivating the name FoldingDiff.
4 Experiments
FoldingDiff generates protein backbones whose angular and secondary-structure distributions resemble natural proteins, with 177 of 780 structures designable by ProteinMPNN and OmegaFold.
- 4.1 Generating protein internal angles: The model’s generated angular distributions are compared with held-out experimental structures across six internal angles.The analysis covers dihedral and bond angles aggregated across 780 generated backbones.
- 4.1 Generating protein internal angles: All three major Ramachandran regions corresponding to right-handed α helices, left-handed α helices, and β sheets recur in generated backbones.The model also captures the greater frequency of right-handed α helices relative to left-handed α helices.
- 4.2 Analyzing generated structures: Generated structures frequently contain multiple α helices, multiple β sheets, and mixtures of both, matching the secondary-structure co-occurrences of natural proteins.This behavior is consistent across multiple generation replicates, unlike an autoregressive transformer that collapses into endless α-helix generation.
- 4.2 Analyzing generated structures: 177 of 780 generated backbones, or 22.7%, are designable with scTM ≥0.5 without refinement or relaxation.The result is consistent across runs and yields 163/780 designable structures when AlphaFold2 without MSAs replaces OmegaFold.
- 4.2 Analyzing generated structures: FoldingDiff exceeds the prior-work designability counts for both short and long sequences under the reported scTM pipeline.For sequences up to 70 residues, 76/210 are designable versus 36/210; beyond 70 residues, 87/570 versus 56/570.
- 4.2 Analyzing generated structures: Training-set similarity correlates with designability, with Spearman’s r = 0.78, while generated backbones are reported as structurally diverse and not simply memorized.Only 16 of 177 designable backbones contain β sheets, suggesting lower designability for generated β-sheet structures.
5 Conclusion
The paper presents an internal-angle diffusion formulation that enables a simple transformer to generate diverse, biologically plausible protein backbones, while identifying important limitations and releasing code and trained models.
- A relative internal-angle parameterization lets a vanilla transformer diffusion model generate high-quality, diverse, biologically plausible protein structures.The generated backbones respect protein chirality and exhibit high designability.
- Early angle errors can substantially alter overall structures through a lever arm effect, and some generations contain self-collisions.
- The generated structures remain relatively short, exclude multi-chain complexes and ligand interactions, and represent static rather than dynamic proteins.
- Training code, downstream analysis code, and trained model weights are publicly available.
A Internal angle formulation of protein backbones
The model represents backbones with inter-residue angles, fixes bond distances to averages, and evaluates how representation lossiness and denoising reconstruction vary with structure length.
- A.1 Choice of angles for representation: Nine values per residue fully specify a backbone: three bond distances, three bond angles, and three dihedral torsional angles.The model treats bond distances as least important and generally replaces them with constant average values.
- A.1 Choice of angles for representation: Using all angles and distances perfectly reconstructs Cartesian coordinates, while the six-angle formulation with average bond distances remains the main usable reduced representation.
- A.1 Choice of angles for representation: The N−1 angle representation omits the first residue’s orientation, so generation is seeded from a fixed coordinate set.
- A.2 Effect of length on structure reconstruction: Longer structures produce greater reconstruction divergence when converted between Cartesian and internal-angle representations.
- A.2 Effect of length on structure reconstruction: The denoising evaluation adds t = 750 noise steps to held-out test angles before running the model for 750 iterations to reconstruct them.Training uses T = 1000 diffusion timesteps.
- A.2 Effect of length on structure reconstruction: 98% of test structures achieve reconstruction TM scores above 0.95, with an average test reconstruction TM score of 0.988.
- A.2 Effect of length on structure reconstruction: Reconstruction TM score shows no significant length correlation for the denoising model: Spearman’s r = −0.0024 and p = 0.89.
- A.2 Effect of length on structure reconstruction: The representation becomes somewhat lossier with length, but the model learns robust angle relationships without significant reconstruction impact across lengths.
B Additional notes on self-consistency TM score
The self-consistency TM evaluation adapts prior pipelines by using OmegaFold and ProteinMPNN under controlled sampling choices for comparison.
- The evaluation primarily differs from earlier pipelines by using OmegaFold instead of AlphaFold for sequence folding.OmegaFold does not rely on multiple sequence alignments and is described as generalizing better to orphan proteins.
- OmegaFold uses default parameters and release1 weights, while AlphaFold is benchmarked without MSA input using one model and 15 recycling iterations.
- ProteinMPNN generates 8 candidate sequences per structure in the adopted primary comparison, matching the sampling count used for direct comparability.
- The analysis generates 10 structures for each length l ∈[50, 128), keeping the generation strategy aligned with Trippe et al. except for the generated structures.
C.1 Substituting internal angle formulation for Cartesian coordinates
Replacing the internal-angle representation with Cartesian Cα coordinates severely degrades generation quality, producing point clouds rather than meaningful protein-like structures.
- The Cartesian ablation keeps the diffusion schedule, transformer architecture, and loss function while replacing internal angles with Cα coordinates.Unlike the angle representation, Cartesian coordinates are not rotation- or shift-invariant.
- Cartesian generations show no significant pairwise-distance patterns, whereas internal-angle generations resemble real proteins’ compact structural patterns.
- The Cartesian model generates a nondescript point cloud, indicating that changing only the input representation drastically degrades performance.
C.2 Autoregressive baseline for angle generation
The autoregressive angle-generation baseline predicts each next residue-angle set from prior angles, but collapses to biologically unrepresentative, endlessly repeated α helices despite realistic seeds and added noise.
- Baseline design: The baseline uses the same six-angle formulation as FoldingDiff and predicts each next angle set from all preceding angle sets.It uses a transformer with positional and sequence-length embeddings and is trained on the same data splits and preprocessing.
- Generation behavior: All generated structures consist of one singular α helix, confirming modal collapse and a lack of structural diversity.P-SEA detects exclusively α helices and no β sheets; these long coils are uncommon in natural proteins.
- Generation behavior: Adding small amounts of angle noise during generation does not meaningfully prevent collapse into endlessly repeated α helices.The authors compare this intervention to temperature-based randomness used to disrupt repetitive patterns in language models.
- Evaluation: 693 of 780 generated structures are designable, yielding an overall designability ratio of 0.89, but this does not imply useful or novel proteins.The structures’ singular endless coils make the designability increase biologically uninformative.
C.3 Baselines contextualizing scTM scores
The shuffled-angle baseline preserves marginal angle distributions and pairwise correlations but destroys their ordering, producing structures with much poorer foldability and unnatural secondary-structure patterns.
- Random angle baseline: The baseline samples angle sets from the test data, preserving the natural angle distribution and pairwise dihedral correlations while losing their correct ordering.It generates 780 structures across lengths from 50 through 127 residues for direct comparison with FoldingDiff.
- Designability: No randomly sampled structure is designable, and its scTM scores are significantly poorer than FoldingDiff’s (p = 1.6 × 10−121, Mann-Whitney test).Designability is evaluated with ProteinMPNN in CA-only mode followed by OmegaFold.
- Secondary structure: The shuffled-angle structures have a secondary-structure distribution unlike both natural structures and FoldingDiff generations.The random baseline produces only a few β-sheet hits by chance and lacks the α helices found in natural and FoldingDiff structures.
D Additional Supplementary Figures and Tables
Supplementary analyses compare generated and natural angle distributions, assess replicate consistency and inverse-folding sensitivity, and examine structural complexity, similarity, and diversity.
- Angle distributions: The supplementary CDFs compare generated and test-set distributions for dihedral angles and bond angles.The top row shows dihedrals involving four atoms, while the bottom row shows bond angles involving three atoms.
- Angle distributions: Compared with Anand and Achim’s broader angular spread, the proposed method matches the natural angle distribution more tightly.The comparison specifically highlights the Cαi−Ci−Ni+1 bond angle corresponding to θ2.
- Replicates: Table S1 reports self-consistency TM scores across replicate generations evaluated with ProteinMPNN and OmegaFold.Each replicate contains 780 structures spanning lengths from 50 through 127 residues.
- Evaluation sensitivity: Using ESM-IF1 instead of ProteinMPNN for inverse folding consistently produces much lower self-consistency TM scores.The analysis otherwise evaluates the same FoldingDiff generations and folds them with OmegaFold.
- Replicates: Across five replicates, generated structures contain mixtures of α helices and β sheets comparable to natural structures.P-SEA annotations indicate that FoldingDiff consistently produces reasonable, complex structures.
- Similarity and diversity: Generated structures span combinations of designability and training similarity, including examples that differ visually from their closest training match.The supplementary comparisons indicate that TM scores alone may understate structural distinctness.
- Similarity and diversity: Among 177 designable generated backbones, hierarchical clustering shows loosely related groups but no clearly delineated clusters.Natural structures cluster more similarly to these generations than to prior-work generations, which contain blocks of near-duplicate structures.