PaperlayerView exact revision on arXiv

Source-linked AI summary

SymFold: Synergizing Evolutionary and Structural Priors for Accurate Protein Inverse Folding

Handong Wang, Jiaxin Qi, Baisheng Lai, Jianqiang Huang

arXiv:2609.01353v1cs.AI

TL;DR

Protein inverse folding seeks sequences compatible with specified 3D structures, but post-hoc PLM refinement is limited by upstream predictions and standalone MPLMs are ineffective. SymFold combines PLMs and MPLMs through symmetric dual-path refinement with adaptive fusion and self-correction, achieving state-of-the-art performance across standard benchmarks.

  • Problem

    Post-hoc PLM refinement cannot overcome poor structure-encoder predictions, while standalone MPLMs are ineffective for inverse folding despite directly mapping structures to sequences.

  • Method

    SymFold uses a symmetric dual-path framework that jointly fuses PLM sequence-evolution knowledge and MPLM structural knowledge, with adaptive synergistic fusion and self-correction during iterative generation.

  • Results

    SymFold achieves state-of-the-art performance across standard inverse folding benchmarks, including CATH, TS50, and TS500, while ablations support its architecture and modules.

  • Takeaways & Limitations

    The results support continuous interaction between structural constraints and sequence-evolution knowledge as a promising direction for computational protein design.

  • Takeaways & Limitations

    Evaluations rely primarily on in silico metrics, and limited wet-lab validation leaves the functional viability of generated sequences unverified.

Abstract

from arXiv · show

Protein inverse folding aims to recover amino acid sequences for a given 3D protein structure, underpinning broad applications such as enzyme engineering and drug discovery.Current methods often follow a serial pipeline, in which a structure encoder predicts a coarse sequence, which is then refined by protein language models (PLMs). However, because PLMs only perform post-hoc sequence edits, the refinement is bounded by the quality of upstream predictions.Thanks to recent multimodal protein language models (MPLMs), we could directly encode structure to generate sequences with pretrained structural knowledge, but we observe that they are not effective for inverse folding. Therefore, we introduce a symmetric dual-path architecture that both leverages PLMs for pretrained sequence evolution knowledge and MPLMs for pretrained structural knowledge to iteratively guide protein sequence generation.Through extensive experiments across standard protein inverse folding benchmarks, our method achieves state-of-the-art performance, surpassing prior approaches, and ablation studies validate the rationale of our symmetric design, revealing a promising direction for the community.

1 Introduction

Protein inverse folding designs amino acid sequences that stably fold into specified 3D structures, supporting applications such as enzyme engineering and drug discovery. SymFold addresses limits of serial refinement by combining pretrained sequence and structural knowledge in a symmetric framework.

  • Motivation: Protein inverse folding designs sequences for specified 3D structures, enabling applications including enzyme engineering and drug discovery.The paper illustrates designing a binding-site structure and deriving candidate sequences for synthesis and validation.
  • Existing limitations: Traditional structure encoders perform position-wise residue classification, but limited data, model capacity, and missing sequence context constrain performance.The resulting sequences are described as usually biologically unreasonable.
  • Existing limitations: PLM-based methods add post-hoc sequence refinement, yet the refinement remains bounded by the structure encoder's output because PLMs lack structural information during editing.The paper identifies this as a structure-ignorant or decoupled refinement problem.
  • Motivation: MPLMs can directly map structures to sequences, but the paper finds that standalone MPLMs are ineffective, potentially because of structural distribution shifts or limited sequence modeling capacity.These observations motivate combining MPLMs with PLMs rather than using MPLMs alone.
  • SymFold: SymFold uses a structure encoder for a coarse sequence, then jointly applies PLMs and MPLMs to refine it with evolutionary sequence context and structural knowledge.Its symmetric dual-path refinement accounts for contextual dependencies and residue-specific positional information.
  • SymFold: Adaptive Synergistic Fusion dynamically integrates structure cues, context dependencies, and structural priors, while self-correction training mirrors iterative inference to reduce train–test mismatch.The method stochastically mixes intermediate outputs during iterative training.
  • Results: SymFold achieves new state-of-the-art performance on CATH, TS50, and TS500, with ablations supporting its dual-path, adaptive-fusion, and self-correction design.The reported contribution also diagnoses shortcomings of post-hoc refinement and insufficiency of MPLMs alone.

2 Related Work

Protein inverse folding has progressed from physics-based search to deep-learning methods that combine geometric models with pretrained sequence knowledge. Related PLM-based approaches use serial refinement, whereas SymFold combines evolutionary and structural priors through a symmetric dual-path design.

  • Physics-based methods: Early inverse-folding methods used physics-based low-energy sequence search, but approximate energy functions and high computational costs limited them.Rosetta Design relied on sampling strategies such as Monte Carlo simulation.
  • Deep-learning methods: Recent deep-learning methods couple PLMs with geometric models, typically refining a structure encoder's initial sequence in a serial pipeline.This progression introduced pretrained sequence knowledge into inverse folding.
  • Deep-learning methods: Serial PLM refinement is decoupled because PLMs lack access to structural constraints during refinement.SymFold addresses this limitation by combining PLM evolutionary knowledge with MPLM structural priors.
  • MPLMs: MPLMs seek to unify sequence and structure modeling; examples include ESM-3's joint modeling and DPLM-2's handling of discrete and structural information.The passage notes that ESM-3's coordinate discretization constrains direct inverse-folding use.

3 Method

SymFold refines a coarse structure-derived sequence by jointly incorporating evolutionary knowledge from PLMs and structural knowledge from MPLMs. Its adaptive fusion and self-correction strategy support residue-aware iterative refinement while aligning training with inference.

  • Preliminaries: Baseline inverse-folding methods encode the full 3D structure into per-position amino-acid predictions using sequence-averaged cross-entropy training.The structure encoder produces logits for residue classes at each position.
  • Preliminaries: PLM-based methods train a structure encoder first, then use a trainable PLM for post-hoc sequence refinement without direct structure access.The refined input is the frozen structure encoder’s predicted sequence, which limits editing by the upstream prediction.
  • SymFold Overview: SymFold combines PLM sequence priors and MPLM structure priors through a symmetric dual-path refinement architecture initialized by a structure encoder’s coarse sequence.The two paths adjust sequence predictions jointly rather than treating PLM refinement as a separate post-hoc stage.
  • Adaptive Synergistic Fusion: Adaptive Synergistic Fusion dynamically weights sequence-context and structure signals because residue dependence on local geometry and long-range dependencies varies.The fusion uses residue-wise structure-aware coefficients and combines the corresponding path outputs.
  • Self-Correction: Self-Correction unrolls symmetric refinement during training and computes loss from the final iterative prediction to mirror multi-step inference.This addresses the train–test discrepancy created when iterative inference repeatedly feeds predictions through multiple fused knowledge sources.

4 Experiments

SymFold is evaluated across standard and challenging inverse-folding benchmarks, ablations, encoder variants, and AlphaFold3-based plausibility tests. Results consistently support its dual-path design and show stronger recovery and predicted structural compatibility than baselines.

  • Benchmark performance: SymFold reaches 63.11% recovery on CATH4.2 and 62.23% on CATH4.3, with the lowest perplexity across protein categories.On CATH4.2, it exceeds Knowledge-Design by 2.34% and PiFold by 11.45 percentage points.
  • Benchmark performance: SymFold achieves 66.02% recovery on TS50 and 70.48% on TS500, while recording perplexities of 2.84 and 2.74, respectively.The reported perplexity reductions are 8.4% on TS50 and 4.2% on TS500.
  • Generalization: SymFold obtains 56.57% and 52.18% recovery on CASP15 and CASP16, outperforming ProteinMPNN, PiFold, and LM-Design on both datasets.Knowledge-Design and Bridge-IF were excluded because of reproducibility issues with their released code.
  • Ablation studies: 63.11% recovery and perplexity 3.23 are achieved by full SymFold, while removing either the PLM or MPLM branch significantly degrades performance.The ablation results identify structural and sequence information as complementary inputs.
  • Encoder compatibility: 61.45% recovery with ProteinMPNN and 63.11% with PiFold show that SymFold improves both encoders, with stronger results from the more advanced PiFold encoder.The corresponding baseline recoveries are 49.87% and 51.66%.
  • Biological plausibility: 61.65% recovery for T1234 and 50.85% for T1266 exceed the reported LM-Design, ProteinMPNN, and PiFold results in the AlphaFold3 validation.SymFold also obtains pLDDT values of 78.24 and 93.15 and TM-scores of 0.85 and 0.97 for the two targets.

5 Conclusion

The conclusion presents SymFold as a symmetric dual-path inverse-folding framework that continuously combines structural constraints with sequence-evolution knowledge. It reports state-of-the-art benchmark performance while identifying limited wet-lab validation as an unresolved challenge.

  • Framework: SymFold combines structural constraints and Sequence Evolution knowledge through symmetric dual paths, Adaptive Synergistic Fusion, and self-correction.The framework is presented as addressing limitations of serial architectures.
  • Conclusion and limitations: SymFold achieves state-of-the-art performance across benchmarks, but limited wet-lab validation leaves functional viability unverified.The evaluations primarily rely on in silico metrics.

6 Supplementary Material

Supplementary experiments examine multimodal adapter choices, training and input modalities, iterative refinement, and self-correction. They show that fusion design and correction strategy materially affect inverse-folding performance.

  • Adapter designs: The supplementary adapter study compares structure-aware attention fusion with parameter-tuning adapters under matched experimental setups.It also varies encoder freezing versus training from scratch and multimodal versus sequence-only inputs.
  • Adapter designs: Richer multimodal inputs can degrade performance, whereas parameter-tuning adapters more stably leverage the structure-aware capacity of MPLMs.The paper terms this effect the modality paradox and attributes it to issues such as feature redundancy and optimization conflicts.
  • Iterative refinement: Self-Correction improves performance across iteration counts and enables peak recovery of 63.11% early in refinement.Without Self-Correction, convergence is slower and the reported overall performance remains lower.

6.3 CASP15 and CASP16 Protein Details

The appendix lists the protein targets used in CASP15 and CASP16 experiments. Official release names and dates support reproducible identification and cross-referencing of these targets.

  • Target documentation: Tables 9 and 10 provide official release names and dates for the CASP15 and CASP16 protein targets.The collection is intended to support future identification and cross-referencing.

6.4 Ablation on ASF Module Components

The ASF module compares fusion strategies for PLM and MPLM outputs, finding that residue-wise token-level weighting best preserves local structural and evolutionary differences. Its lightweight weighting network produces position-specific normalized coefficients from structure-encoder features.

  • Adaptive Synergistic Fusion: The ResidueWeightingNetwork derives normalized per-position model weights from structure-encoder features using SiLU and local 1D convolution layers.The module also includes Layer Normalization and a Linear transformation before producing adaptive coefficients.
  • Adaptive Synergistic Fusion: ASF dynamically balances PLM sequence priors and MPLM structural priors using a residue-wise gating vector α(x).The gating vector is conditioned on frozen structure-encoder embeddings and has one value per sequence position.
  • ASF fusion variants: Uniform average fusion fixes α(x) at 0.5, whereas model-level fusion uses two global learnable scalars shared across all residues.Sequence-level mixing instead derives one weight vector from global structural features for every token in a sequence.
  • ASF fusion variants: Residue-specific weighting is especially beneficial when structural constraints differ across local regions such as protein cores and surfaces.The authors attribute the weaker performance of coarser fusions to their inability to capture these local variations.

6.5 Additional Folding Results with Alphafold3

Additional AlphaFold3 analyses evaluated DualFold-designed sequences on four CASP16 targets of varying lengths. All predicted structures aligned closely with their native target folds, with consistently high confidence and strong backbone agreement.

  • Evaluation setup: Four CASP16 targets—T1214, T1235, T1299, and T1259—were evaluated by comparing AlphaFold3 predictions with native target backbones.The targets have lengths 652, 114, 68, and 204, respectively.
  • Structural results: All four designed sequences produced AlphaFold3 structures that closely aligned with their respective native folds.The overlays showed strong backbone agreement across targets.
  • Structural results: The predictions maintained consistently high pLDDT confidence across targets with different lengths and topologies.These results support the in silico structural plausibility of the generated sequences.

6.6 LoRA Configuration for MPLM and PLM Fine-tuning

The fine-tuning setup applies identical LoRA configurations to the MPLM and PLM while freezing their base parameters. This parameter-efficient design adapts both models symmetrically with approximately 0.1% trainable parameters.

  • LoRA hyperparameters: Both models use identical LoRA hyperparameters, including rank 8, scaling factor 32, dropout 0.1, and disabled bias adaptation.The configuration balances parameter efficiency, learning capacity, and regularization.
  • Parameter-efficient adaptation: Approximately 0.1% of the full model parameters are trainable under the LoRA configuration.The base model parameters remain frozen while only LoRA weights are updated.
  • Parameter-efficient adaptation: Freezing pretrained weights while updating LoRA adapters is intended to preserve pretrained knowledge during task-specific adaptation.The implementation uses the PEFT framework with pretrained ESM-3 and ESM-C models.
  • Symmetric adaptation: The symmetric configuration adapts sequence and structural knowledge in parallel by targeting attention and feed-forward components.This preserves the fundamental representations encoded in both pretrained models while allowing representation learning to adapt.

6.7 Mathematical Formulations for Evaluation Metrics

The evaluation metrics quantify language-model prediction quality and amino-acid recovery while masking invalid sequence positions. Global perplexity aggregates valid-position negative log-likelihoods, whereas dataset-level recovery uses the median across sequences.

  • Perplexity: Perplexity is the exponential of cross-entropy loss and measures the model’s accuracy in predicting authentic protein sequences.Lower perplexity corresponds to better language-model prediction quality under this evaluation.
  • Perplexity: The model represents each sequence position with logits over a vocabulary of 20 standard amino acids before computing log probabilities.The target amino-acid log-likelihood is extracted at each position.
  • Additional folding results: Figure 5 overlays AlphaFold3-predicted structures with target structures to assess structural agreement for designed sequences.The figure covers four additional CASP16 targets and reports strong alignment across the cases.
  • Perplexity: The valid-position mask excludes padding, class, and end-of-sequence tokens while retaining structurally valid coordinates.The mask combines token exclusions with the coordinate mask c_i.
  • Perplexity: Global perplexity exponentiates the average negative log-likelihood over valid positions.The formulation aggregates masked target log probabilities across the sequence.
  • Sequence Recovery Rate: Sequence recovery rate measures the proportion of correctly predicted amino acids at valid positions.It compares a predicted sequence with its target sequence using position-wise correctness indicators.
  • Dataset-level Statistics: For a test set, the final recovery metric is the median of sequence-level recovery rates rather than their mean.Using the median reduces the influence of outliers and reflects overall performance more robustly.
Loading 2609.01353v1…