PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Denoising Diffusion for Inverse Protein Folding

Kai Yi, Bingxin Zhou, Yiqing Shen, Pietro Liò, Yu Guang Wang

arXiv:2306.16819v2q-bio.QMcs.AI

TL;DR

Inverse protein folding must account for many amino-acid sequences that can correspond to one protein backbone, a diversity that discriminative models struggle to capture. GRADE-IF uses graph-conditioned diffusion with amino-acid substitution priors to generate sequences for a given backbone. It reports state-of-the-art sequence recovery and potential to generate diverse candidates.

  • Problem

    Inverse folding has a one-to-many mapping from a protein structure to possible amino-acid sequences, which transformer-based discriminative models have struggled to capture.

    Source: §1 · pp. 1–2

  • Method

    GRADE-IF diffuses amino-acid types on graph nodes conditioned on protein structure and uses substitution matrices to guide transitions.

    Source: §1 · p. 3, §3 · p. 4

  • Results

    GRADE-IF reports state-of-the-art recovery, improving recovery by 4.2% for single-chain proteins and 5.4% for short sequences.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Inverse-folding outputs can facilitate the design of novel proteins with desired structural and functional characteristics.

    Source: §1 · p. 1

    The zero-shot model is trained on a general protein database, and particular protein or family design may require auxiliary modules or modified training procedures.

    Source: A Broader Impact and Limitations · p. 15

Abstract

from arXiv · show

Inverse protein folding is challenging due to its inherent one-to-many mapping characteristic, where numerous possible amino acid sequences can fold into a single, identical protein backbone. This task involves not only identifying viable sequences but also representing the sheer diversity of potential solutions. However, existing discriminative models, such as transformer-based auto-regressive models, struggle to encapsulate the diverse range of plausible solutions. In contrast, diffusion probabilistic models, as an emerging genre of generative approaches, offer the potential to generate a diverse set of sequence candidates for determined protein backbones. We propose a novel graph denoising diffusion model for inverse protein folding, where a given protein backbone guides the diffusion process on the corresponding amino acid residue types. The model infers the joint distribution of amino acids conditioned on the nodes' physiochemical properties and local environment. Moreover, we utilize amino acid replacement matrices for the diffusion forward process, encoding the biologically-meaningful prior knowledge of amino acids from their spatial and sequential neighbors as well as themselves, which reduces the sampling space of the generative process. Our model achieves state-of-the-art performance over a set of popular baseline methods in sequence recovery and exhibits great potential in generating diverse protein sequences for a determined protein backbone structure.

1 Introduction

GRADE-IF treats inverse protein folding as a generative task, targeting the diversity of amino-acid sequences compatible with one backbone. It combines graph-conditioned denoising with biologically informed substitution transitions and reports state-of-the-art sequence recovery.

1 Introduction

  • Inverse folding seeks feasible amino-acid sequences for a specified 3D structure, but transformer-based discriminative models struggle with the structure-to-sequence one-to-many mapping.The sequence space is immense, and protein folding is complex.Source: §1 · pp. 1–2
  • GRADE-IF reports state-of-the-art recovery, with improvements of 4.2% for single-chain proteins and 5.4% for short sequences.The authors also report that generated sequences have predicted structures identical to the native sequence structure.Source: §1 · p. 2
  • GRADE-IF models inverse folding as graph-node denoising, recovering amino-acid types from random assignments while conditioning on the protein graph.The graph represents spatial and biochemical information about amino acids.Source: §1 · p. 2
  • GRADE-IF conditions on secondary and tertiary structure and is designed to produce a wide array of sequence candidates.The paper presents this diversity as a distinction from models that focus on a most-likely sequence.Source: §1 · p. 2
  • The model uses a Blocks Substitution Matrix as its diffusion transition kernel and adapts DDIM to discrete sampling to accelerate generation.The authors describe BLOSUM as encoding prior knowledge about amino-acid substitutions under evolutionary pressures.Source: §1 · p. 3

2 Problem Formulation

The paper formulates inverse folding as learning the native amino-acid sequence distribution conditioned on a protein backbone. It represents that backbone as a residue graph and describes discrete diffusion transitions and iterative denoising for generating sequences.

2.1 Residue Graph by Protein Backbone

  • A residue graph represents each amino acid as a node, with node attributes for physiochemical and topological properties and edges describing relationships between connected nodes.Spatial neighbors are selected by k-nearest neighbors within a 30Å contact region; edge attributes include distances, local N-C positions, and sequential position encoding.Source: §2.1 · p. 3

2.2 Inverse Folding as a Denoising Problem

  • Inverse folding is formulated as learning p(Xaa|Xpos), predicting a native amino-acid sequence from backbone Cα coordinates.The model considers the density jointly with other amino acids along the chain and is trained by minimizing negative log-likelihood against the native sequence.Source: §2.2 · p. 3
  • Sequences can be designed by sampling or by finding sequences with maximum conditional probability given the desired secondary and tertiary structure.Source: §2.2 · p. 3

2.3 Discrete Denoising Diffusion Probabilistic Models

  • Discrete diffusion progressively corrupts amino-acid data, while iterative denoising reduces noise to generate an output sequence.The generative probability is determined using the transition kernel, model output, and current state.Source: §2.3 · pp. 3–4
  • A diffusion transition matrix is applied to one-hot categorical features, and its cumulative product gives the transition probabilities through step t.Source: §2.3 · pp. 3–4
  • The uniform transition combines identity retention with uniform noise, and its α_t value decays toward zero so the terminal distribution approaches uniformity.The prior distribution is intended to become independent of the observation.Source: §2.3 · p. 4

3 Graph Denoising Diffusion for Inverse Protein Folding

GRADE-IF diffuses amino-acid types on protein graph nodes and denoises them conditioned on graph structure and protein-specific information. Its design combines substitution-based transitions, secondary-structure conditions, an equivariant graph network, and an accelerated discrete sampling process.

3 Graph Denoising Diffusion for Inverse Protein Folding

  • GRADE-IF defines a discrete graph denoising diffusion model conditioned on a protein graph, with diffusion on amino-acid features and graph structure encoded by equivariant neural networks.It incorporates amino-acid substitution scores and secondary structure as protein-specific prior information.Source: §3 · p. 4

3.1 Diffusion Process and Generative Denoising Process

  • The forward process independently adds noise to each amino-acid node using transition matrix Q_t, with the cumulative matrix Q̄_t giving transition probabilities through step t.The denoising network predicts clean amino-acid types from noisy graph inputs, including amino-acid type and geometry features.Source: §3.1 · p. 4

3.2.1 Markov Transition Matrices

  • BLOSUM supplies nonuniform amino-acid substitution probabilities for diffusion and generation, favoring substitutions estimated to be more likely from empirical protein evolution.The figure contrasts BLOSUM’s preference for amino acids similar to leucine with uniform diffusion’s even dispersion.Source: §3.2.1 · p. 5, Figure 2 · p. 5

3.2.2 Secondary Structure

  • The model conditions amino-acid denoising on one-hot encodings of eight secondary-structure types predicted with DSSP.The paper states that these conditions narrow the sequence-sampling space.Source: §3.2.2 · p. 5

3.3 Equivariant Graph Denoising Network

  • The modified equivariant graph convolution updates node and edge representations while preserving rotation and translation equivariance and permutation invariance over nodes.The network is intended to support consistent representations when 3D input molecules are repositioned or reoriented.Source: §3.3 · p. 6

3.4 DDIM Sampling Process

  • The discrete DDIM process controls whether sampling is deterministic or stochastic through the temperature of the predicted amino-acid distribution.The paper introduces this as an adaptation of DDIM’s acceleration strategy to discrete diffusion sampling.Source: §3.4 · p. 6

4 Experiments

On CATH, GRADE-IF experiments report improved recovery for single-chain proteins and short sequences, then assess generated sequences for foldability and diversity. The results also examine recovery across residue environments and secondary structures.

4 Experiments

  • GRADE-IF improves recovery by 4.2% for single-chain proteins and 5.4% for short sequences.Table 1 reports recovery-rate performance on CATH zero-shot models.Source: Table 1 · p. 7, §4.2 · p. 8

4.1 Experimental Protocol

  • The protocol partitions CATH into 18,024 training, 608 validation, and 1,120 test proteins, and evaluates short, single-chain, and all-protein categories.Short proteins have sequences shorter than 100 residues.Source: §4.1 · p. 7

4.2 Inverse Folding

  • Internal residues recover more often than external residues across all three protein classes, recovery rises with contact number, and most secondary structures show high recovery.The reported exception is the infrequent 5-turn helix; internal residues are identified using SASA < 0.25.Source: Figure 3 · p. 7, §4.2 · p. 8

4.3 Foldability

  • GRADE-IF-generated sequences fold near the native structure, with RMSD about 1 Å over 139 residues; on 100 test structures, the study reports high pLDDT, low RMSD, and high foldability.The 1 Å RMSD is below the crystal structure's 2.2 Å resolution; the authors note that several NMR-based targets challenge both GRADE-IF and baselines.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 Diversity

  • GRADE-IF produces 60% sequence difference per sample versus diversity below 50% for PIFOLD and PROTEINMPNN; at step size 1, diversity exceeds 0.6 while pLDDT and TM scores approach 0.8.Decreasing step size increases diversity, with a slight reduction in pLDDT and TM scores.Source: Table 3 · p. 9, §4.4 · p. 9, Figure 5 · p. 9

5 Related Work

Protein sequence-design methods include structure-conditioned graph models and sequence-based language models, while discrete diffusion offers another framework for generative tasks. The section situates these approaches across protein design and broader diffusion research.

Deep Learning models for protein sequence design

  • Structure-based generative models predict amino-acid node labels with graph neural networks, while sequence-based models use autoregressive or masked-language-modeling frameworks [2] [20] [43].Sequence-based approaches infer residue relationships with attention-based methods [25, 29, 37, 23, 26, 34, 47].Source: §5 · p. 10
  • Protein design methods are grouped into structure-based and sequence-based generative models, with the latter drawing parallels between protein sequences and natural language processing.Sequence-based models may generate autoregressively or predict masked sequence parts [25, 29, 37, 23, 26, 34, 47].Source: §5 · p. 10

Denoising Diffusion models

  • Discrete diffusion operates either by applying Gaussian diffusion after mapping categorical data to continuous space or by defining diffusion directly on categorical data [4] [16] [3].D3PM was extended to jointly generate graph node features and structure [48].Source: §5 · p. 10

6 Conclusion

GRADE-IF uses diffusion denoising and substitution matrices to generate diverse sequences conditioned on a protein backbone. The conclusion reports sequence recovery and biological plausibility, and connects inverse folding to protein design and the study of folding and evolution.

6 Conclusion

  • GRADE-IF combines substitution matrices for diffusion and sampling with an equivariant graph neural network that analyzes backbone geometry without intensive data augmentation.The model generates sequences conditioned on a predetermined backbone using an iterative denoising process.Source: §6 · p. 10
  • The generated sequences are diverse, show significant recovery, preserve more natural designs in conserved regions, and are likely to fold into structures highly similar to the native protein.Source: §6 · p. 10
  • Identifying different sequences that fold into the same structure can help investigate protein-folding principles and evolutionary history while preserving protein functions.The conclusion also points to applications such as targeted drug delivery and enzyme design.Source: §6 · p. 10

A Broader Impact and Limitations

The section discusses potential misuse and limits of the model’s zero-shot scope, then presents appendix derivations for non-Markovian forward-process posteriors. The proposed mitigation of excluding viral proteins may trade off performance and generalizability.

A Broader Impact and Limitations

  • Restricting training to nonviral proteins could limit misuse involving viral protein construction, but may compromise model performance and generalizability.Source: A Broader Impact and Limitations · p. 15
  • The zero-shot model is trained on a general protein database, and protein- or family-specific design may require auxiliary modules or modified training procedures.The text notes that stricter sampling conditions may be needed for functional proteins, such as those with transmembrane α-helices.Source: A Broader Impact and Limitations · p. 15

B Non-Markovian Forward Process

  • The appendix derives one-step and k-step posteriors for the non-Markovian forward process; its discrete DDIM derivation is intended to accelerate generation.The one-step posterior derivation relies on Bayes’ rule, the Markov property, and the predefined amino-acid transition matrix.Source: B Non-Markovian Forward Process · pp. 15–16

C Graph Representation of Folded Proteins

The graph represents a folded protein with residues as nodes and spatially selected edges, using node and edge attributes to encode geometry, biochemical properties, and residue relationships. Its neighborhood is determined by a k-nearest-neighbor graph with a distance cutoff.

C Graph Representation of Folded Proteins

  • Edges connect nearby residues in a k-nearest-neighbor graph subject to a distance cutoff, representing local spatial relationships.Each node connects to up to k nearest nodes; the example cutoff is 30 Å.Source: C Graph Representation of Folded… · p. 16
  • Each residue node encodes amino-acid type as 20 binary values and includes normalized B-factor, SASA, surface-aware features, backbone dihedral angles, and 3D position represented by its α-carbon.Backbone dihedral angles are represented by sine and cosine values for ϕ, ψ, and ω.Source: C Graph Representation of Folded… · p. 16
  • Edge features encode kernel-based distances, relative spatial positions, and sequence separation, with local frames representing fine-grained residue relations.The representation includes 15 distance-based features, 12 relative positions, 66 binary sequence-distance features, and a binary contact signal for Cα distances below 8 Å.Source: C Graph Representation of Folded… · p. 16

D Training and Inference

Training corrupts amino-acid types and learns to denoise them with an equivariant graph network. Inference iteratively samples sequences, with DDIM step skipping and fixed-residue partial generation available.

D Training and Inference

  • Training samples a timestep, corrupts amino-acid types, and trains an equivariant graph network to denoise them using cross-entropy.The network uses noisy amino acids, other node properties, and graph geometry to predict the original types.Source: D Training and Inference · p. 17
  • Inference starts from a uniform amino-acid prior and iteratively samples reverse posteriors using the network's denoising predictions.The prior samples amino acids uniformly from 20 classes.Source: D Training and Inference · p. 17
  • DDIM accelerates sampling by skipping reverse steps while retaining close performance, and known amino acids can be fixed during partial generation.At known positions, predictions are adjusted to the specified amino-acid types during the sampling process.Source: D Training and Inference · p. 17

E Inverse Folding Performance on TS50 and T500

GRADE-IF is evaluated on TS50 and T500, with reported recovery accuracy on both zero-shot test sets.

E Inverse Folding Performance on TS50 and T500

F Ablation Study

The ablations report effects from transition-matrix choice, equivariance, secondary-structure input, sampling-step skipping, and sequence diversity comparisons.

F Ablation Study

  • 2% higher recovery was reported with BLOSUM as the transition matrix, while equivariance enhanced performance and secondary-structure input reduced uncertainty.The comparison of transition-matrix selection, equivariance, and secondary-structure use is shown in Figure 6.Source: F Ablation Study · p. 18, Figure 6 · p. 19
  • 52.21% recovery with one skipped step contrasts with 47.66% at 100 skipped steps, which is 100 times faster.Recovery declined as the number of skipped steps increased.Source: F Ablation Study · pp. 18–19
  • At equal diversity, GRADE-IF included the wild-type sequence and generated many samples, while PIFOLD and PROTEINMPNN reverted to deterministic results.Figure 8 projects 100 generated sequences with self-similarity below 50% into two dimensions using t-SNE.Source: Figure 8 · p. 19, F Ablation Study · p. 19

G Additional Folding Results

Additional folding comparisons indicate that sequences generated by GRADE-IF retain the core structures of three selected native proteins.

G Additional Folding Results

  • Three sequences were generated for each of three randomly selected CATH proteins: 1ud9 chain A, 2rem chain B, and 3drn chain B.The generated sequences were folded with AlphaFold 2 and compared with native crystal structures.Source: G Additional Folding Results · p. 20
  • The authors conclude that the generated sequences retain the core structure of the native proteins.Source: G Additional Folding Results · p. 20
  • Across all cases, the generated-sequence RMSD was lower than the preparation accuracy of the wet experiment.Figure 9 compares predicted structures from generated sequences with structures from the native sequences.Source: Figure 9 · p. 20, G Additional Folding Results · p. 20

Related papers

Loading 2306.16819v2…