Source-linked AI summary
Protein Sequence and Structure Co-Design with Equivariant Translation
Chence Shi, Chuanrui Wang, Jiarui Lu, Bozitao Zhong, Jian Tang
TL;DR
Protein co-design faces a large sequence-and-structure search space, and existing co-design approaches can be costly or fail to cross-condition sequence and structure. PROTSEED iteratively translates both from random initialization using context features and an equivariant decoder. Across multiple design tasks, it outperforms prior baselines, producing high-fidelity sequence and structure with orders-of-magnitude faster runtime than sampling-based methods.
Problem
Protein design must search a large space of sequences and structures, while existing co-design approaches can be computationally costly and may generate sequence and structure without cross-conditioning.
Source: §1 · pp. 1–2Method
PROTSEED encodes geometric constraints from context features and iteratively translates sequence and structure together, updating all residues in one shot.
Source: §1 · p. 2Results
Across multiple protein-design tasks, PROTSEED outperforms previous state-of-the-art baselines with high fidelity in sequence and structure and orders-of-magnitude faster runtime than sampling-based methods.
Source: §1 · p. 2, §4.2 · p. 8, §5 · p. 9Takeaways & Limitations
Case studies show PROTSEED can design novel proteins with altered loop lengths, idealized topologies, and custom numbers of secondary structures.
Source: §4.4 · p. 9
Abstract
from arXiv · showhide
Proteins are macromolecules that perform essential functions in all living organisms. Designing novel proteins with specific structures and desired functions has been a long-standing challenge in the field of bioengineering. Existing approaches generate both protein sequence and structure using either autoregressive models or diffusion models, both of which suffer from high inference costs. In this paper, we propose a new approach capable of protein sequence and structure co-design, which iteratively translates both protein sequence and structure into the desired state from random initialization, based on context features given a priori. Our model consists of a trigonometry-aware encoder that reasons geometrical constraints and interactions from context features, and a roto-translation equivariant decoder that translates protein sequence and structure interdependently. Notably, all protein amino acids are updated in one shot in each translation step, which significantly accelerates the inference process. Experimental results across multiple tasks show that our model outperforms previous state-of-the-art baselines by a large margin, and is able to design proteins of high fidelity as regards both sequence and structure, with running time orders of magnitude less than sampling-based methods.
1 INTRODUCTION
Protein sequence-and-structure design remains difficult because of its large search space, while existing co-design approaches can be costly or fail to cross-condition sequence and structure. PROTSEED jointly translates both from random initialization using context features, and experiments report high-fidelity designs and substantially faster inference than sampling-based methods.
1 INTRODUCTION
- Existing de novo protein design faces a tremendous joint sequence-and-structure search space, and handcrafted energy functions with heuristic sampling can be computationally intensive and yield suboptimal solutions.The paper motivates design by proteins’ essential roles in living organisms.Source: §1 · p. 1
- Earlier co-design methods may be limited to antibodies or generate structures, sequences, and rotamers sequentially, risking sequence–structure inconsistency while incurring high inference costs.The sequential three-diffusion approach is described as expensive and unable to cross-condition on sequence and structure.Source: §1 · pp. 1–2
- PROTSEED uses a trigonometry-aware context encoder and a roto-translation equivariant decoder to translate sequence and structure together, updating every residue in one shot.The method allows sequence and structure to cross-condition on one another and on context features.Source: §1 · p. 2
- PROTSEED outperforms previous state-of-the-art methods across antibody CDR, context-conditioned protein, and fixed-backbone design, with high fidelity in sequence and structure and orders-of-magnitude faster runtime than sampling-based methods.The experiments also present proof-of-concept de novo designs with new folds.Source: §1 · p. 2
2 RELATED WORK
Prior protein-design work includes sequence-only or structure-only generation, fixed-backbone sequence design, and co-design methods based on hallucination, autoregression, or diffusion. The paper positions PROTSEED as a co-design approach that cross-conditions sequence and structure while targeting broader topology coverage and faster inference.
2 RELATED WORK
- Fixed-backbone design predicts sequences for a given structure, but requires the desired protein structure to be known beforehand.The cited work reports promising results for this setting.Source: §2 · p. 2
- Network-hallucination co-design uses gradient descent in sequence space, while antibody co-design uses autoregressive or diffusion models; the cited approaches may depend on structure-prediction accuracy, specific topologies, or costly sampling.Anand & Achim (2022) instead generates sequence, structure, and rotamers with separate diffusion models, which the paper describes as inefficient and lacking sequence–structure cross-conditioning.Source: §2 · p. 3
- Existing equivariant 3D structure-translation methods are described as inapplicable to protein structures, while folding models require complete sequences and MSAs and cannot directly co-design sequence and structure.Source: §2 · p. 3
3 METHOD
The method treats sequence-and-structure co-design as conditional generation from context features, then iteratively translates residue types and structures together. Its encoder models geometric relations among context features, while its decoder exchanges information across context, sequence, and structure using equivariant updates.
3.1 PRELIMINARIES
- The task is conditional generation of residue types and 3D conformations for N residues, given prior single-residue and pairwise context features.Examples include secondary-structure annotations, contact features, and antibody framework and antigen information.Source: §3.1 · p. 3
3.2 TRIGONOMETRY-AWARE CONTEXT ENCODER
- The context encoder propagates information between single and pair features, using triangle-based interactions to encourage geometric consistency and satisfy the triangle inequality.Its trigonometry-aware attention updates pair features from neighboring features that form residue triangles, tying the attention score within each triangle to reduce computation.Source: §3.2 · p. 4
3.3 JOINT SEQUENCE-STRUCTURE DECODER
- The decoder jointly updates context, residue structures, and residue types across translation layers, updating all amino acids in one shot rather than autoregressively.This design is intended to preserve sequence–structure consistency while accelerating decoding.Source: §3.3 · p. 5
- SeqIPA integrates residue types, residue structures, and context features into updated context representations, allowing sequence and structure to inform each other during decoding.The attention is orientation-aware with respect to residue structures and roto-translation invariant to other representations.Source: §3.3 · p. 5
- For equivariant structure translation, the decoder predicts coordinate changes in local residue frames and converts them into global-frame updates; it updates residue orientations using predicted rotations.The predicted quaternion is converted to a rotation matrix for the frame update.Source: §3.3 · p. 5
- The decoder predicts residue-type distributions over 20 amino acids and can generate sidechains by predicting four torsion angles alongside backbone positions.The training objective combines cross-entropy over residue types with an invariant frame-alignment loss over backbone atom positions.Source: §3.3 · pp. 5–6
4 EXPERIMENTS
Experiments evaluate PROTSEED across three benchmark tasks and a de novo case study, spanning antibody CDR design, context-conditioned co-design, fixed-backbone sequence design, and novel folds.
4.1 ANTIBODY CDR CO-DESIGN
- For antibody CDR co-design, antigen-complex residue types and distances condition randomly initialized CDRs; evaluation fixes CDR length to native and samples 100 lowest-PPL candidates.Source: §4.1 · pp. 6–7
4 EXPERIMENTS
- Across tasks, PROTSEED leads baselines on all three antibody CDR metrics and all three co-design metrics, while fixed-backbone results are competitive on PPL and best on AAR.Source: Table 1 · p. 7, §4.1 · p. 7, Table 2 · p. 8
4.2 PROTEIN SEQUENCE AND STRUCTURE CO-DESIGN
- Secondary-structure annotations and binary contacts specify known topology without a specified structure; across four sizes, PROTSEED is orders of magnitude faster than AFDESIGN and DIFFUSION.Source: Figure 3 · p. 7, §4.2 · pp. 7–8
4.3 FIXED BACKBONE SEQUENCE DESIGN
- Fixed-backbone sequence design conditions on backbone-derived dihedral angles and inter-residue distances, with structures held fixed in the decoder; evaluation uses CATH 4.2.Source: §4.3 · pp. 8–9
4.4 CASE STUDY
- Hand-built secondary-structure and contact contexts produced loop extensions, β-barrels, and 12-helix transmembrane complexes; predicted folds agreed with AlphaFold and database searches found sequence and structure dissimilarity.Source: Figure 4 · p. 9, §4.4 · p. 9
5 CONCLUSION AND FUTURE WORK
PROTSEED translates sequence and structure jointly and iteratively, with equivariant position and orientation updates and invariant sequence-type updates. The paper reports broad performance gains and identifies scaffolding and context-free design as future directions.
5 CONCLUSION AND FUTURE WORK
- PROTSEED translates proteins in joint sequence-structure space iteratively and end-to-end, with one-shot updates supporting efficient inference.The conclusion also reports broad benchmark gains and case studies demonstrating potential for biological research; future work includes scaffolding and context-free design.Source: §5 · p. 9
REPRODUCIBILITY STATEMENT
- The paper states that code, datasets, and experimental environments will be released upon acceptance.The appendix also provides PROTSEED pseudocode, SeqIPA parameterization, hyperparameters, and implementation details.Source: REPRODUCIBILITY STATEMENT · p. 10
A EQUIVARIANCE PROPERTY OF SEQUENCE AND STRUCTURE TRANSLATION
- Under rigid transformations, each translation step updates Cα positions and frame orientations equivariantly, while type distributions update invariantly.The proof attributes invariance of predicted Cα deviations to SeqIPA's orientation awareness and invariant updated context features.Source: A EQUIVARIANCE PROPERTY OF SEQUENCE… · p. 14
B.1 PSEUDO CODE
PROTSEED uses encoded context features to iteratively translate protein sequence and structure from random initialization. Its encoder propagates single and pair features, while its equivariant decoder updates all residues in one shot without MCMC sampling.
B.1 PSEUDO CODE
- The decoder iteratively translates sequence and structure from random initialization using context features, without MCMC sampling.The paper states this runs much faster than autoregressive and diffusion-based models.Source: §B.1 · p. 15
- The trigonometry-aware encoder propagates updated single and pair features through L layers before passing them to the decoder.Its attention mechanisms update the features at each layer.Source: §B.1 · p. 15
- The roto-translation equivariant decoder uses orientation-aware SeqIPA to capture interactions among residue types, structures, and context, then updates every residue's sequence and structure in one shot.It maps invariant predictions in local frames to global frames to guarantee equivariance of structure translation.Source: §B.1 · p. 15
B.2 PARAMETERIZATION OF SEQIPA
SeqIPA adapts Invariant Point Attention to incorporate residue types while preserving invariance to global transformations. The paper describes addition and attention parameterizations, selecting the lighter addition strategy for experiments.
B.2 PARAMETERIZATION OF SEQIPA
- SeqIPA adapts Invariant Point Attention by adding residue types as input to model interactions among decoded sequences, structures, and context features while retaining global-transform invariance.The paper requires a SeqIPA parameterization to model these interactions without changing IPA's invariance property.Source: §B.2 · p. 16
- SeqIPA-Addition adds residue-type embeddings to single representations and leaves pair features unchanged before vanilla IPA.SeqIPA-Attention instead constructs sequence-conditioned single and pair representations, updates them with a lightweight encoder, and feeds them to vanilla IPA.Source: §B.2 · p. 16
- Both SeqIPA strategies work well with comparable performance, and the experiments use SeqIPA-Addition to keep the model lightweight.The paper describes the parameterization as flexible provided it captures sequence, structure, and context interactions while remaining invariant to global transformations.Source: §B.2 · p. 16
B.3 HYPER-PARAMETERS AND IMPLEMENTATION DETAILS
The implementation uses an eight-layer encoder and decoder with fixed single- and pair-feature dimensions. Training uses Adam with warmup on four Tesla V100 GPUs, and inference fixes the sequence-distribution temperature at 1.
B.3 HYPER-PARAMETERS AND IMPLEMENTATION DETAILS
- The trigonometry-aware encoder and sequence-structure decoder each use 8 layers, with hidden dimensions of 256 for single features and 128 for pair features.The model is implemented in PyTorch.Source: §B.3 · p. 16
- Training uses Adam at learning rate 0.001 with 2,000 linear warmup iterations on four Tesla V100 GPUs, with convergence estimated at 24 hours.The authors report that proper warmup can yield faster convergence and higher performance.Source: §B.3 · p. 16
- Inference fixes the sequence-distribution temperature λ at 1 because larger λ raises AAR but worsens PPL, while smaller λ has the opposite trade-off.Source: §B.3 · p. 16
- The authors state that code, datasets, and experimental environments will be released upon acceptance.Source: §B.3 · p. 16
B.4 FULL ATOM POSITIONS RECONSTRUCTION
The reconstruction describes backbone atom positions from Cα coordinates and frame orientation, while sidechain positions are specified through amino-acid-specific torsion angles.
B.4 FULL ATOM POSITIONS RECONSTRUCTION
- Backbone bond lengths and bond angles are described as relatively conserved.The Cα is connected to N, C, and Cβ, except that glycine has a single hydrogen atom as its side chain.Source: §B.4 · p. 16
- Backbone atom positions can be derived from Cα positions and frame orientation using averaged relative positions recorded in the literature.The Cα frame is defined with respect to N, C, and Cβ atoms.Source: §B.4 · p. 17
- Sidechain atom positions for the 20 amino acids are compactly specified by four torsion angles, χ1 through χ4.The angles follow amino-acid-specific distributions recorded in the literature.Source: §B.4 · p. 17
C.1 CASE STUDY
The case studies use secondary-structure annotations and contact features to guide three de novo design settings; the authors report that the generated proteins are novel.
C.1 CASE STUDY
- For loop extension, the context inserts loop annotations and zero-filled contact rows and columns, then assigns each new residue contacts to nearby positions.Each inserted residue is set in contact with i−2 through i+2.Source: §C.1 · p. 18
- For β-barrel design, the context repeats a simple β-barrel pattern in both contact features and secondary-structure annotations.Source: §C.1 · p. 18
- For helical-complex design, contact features repeat a simple pattern and all secondary-structure annotations are set to “H”.Source: §C.1 · p. 18
- The authors report that one generated protein from each setting confirms novelty when superimposed with its most similar PDB protein.Figure 5 depicts loop extension, a β-barrel, and a helical complex.Source: §C.1 · p. 18
C.2 ABLATION STUDY
The ablations report degraded performance when sequence–structure cross-conditioning, iterative decoding, or context-feature updates are removed; the paper also describes context features and marks context-free design as future work.
C.2 ABLATION STUDY
- Removing sequence–structure cross-conditioning causes a significant drop in all three metrics, especially PPL and AAR.The ablation replaces SeqIPA with vanilla IPA and predicts amino-acid distributions at the final decoding iteration.Source: §C.2 · p. 19
- A single-layer decoder performs much worse than the T=8 tied-weight decoder across all metrics, despite equal trainable-parameter counts.The authors present the equal parameter count as making this a fair comparison.Source: §C.2 · p. 19
- Freezing context-feature updates and removing SeqIPA dramatically degrades performance, indicating that context-feature updates play a key role.Source: §C.2 · p. 19
D.1 CONTEXT FEATURES
- Context features comprise single features {mi} and pair features {zij}, which vary by task and can encode constraints such as antibody frameworks, antigen structures, annotations, contact maps, or starting motifs.The paper states that more specified context features provide researchers more control over designed proteins.Source: §D.1 · p. 20
D.2 CONTEXT-FREE PROTEIN DESIGN
- Context-free protein design is outside this work’s scope; the authors leave a VAE-based adaptation for future work.They describe PROTSEED as a general framework that could handle this setting with minor modifications.Source: §D.2 · p. 20