PaperlayerView exact revision on arXiv

Source-linked AI summary

SimpleFold: Folding Proteins is Simpler than You Think

Yuyang Wang, Jiarui Lu, Navdeep Jaitly, Josh Susskind, Miguel Angel Bautista

arXiv:2509.18480v4cs.LGq-bio.QM

TL;DR

Protein folding has relied on specialized architectures, motivating the question of whether those designs are necessary for strong performance. SimpleFold instead uses a general-purpose transformer with flow matching and achieves competitive folding results, strong ensemble prediction, and efficient deployment.

  • Problem

    Protein folding models commonly rely on computationally heavy domain-specific components, raising whether such designs are necessary for performant folding models.

  • Method

    SimpleFold maps protein sequences to full 3D structures using general-purpose transformer blocks and a flow-matching objective without MSA, pairwise representations, or triangle updates.

  • Results

    SimpleFold-3B achieves competitive standard folding performance and strong protein ensemble generation, while the model family supports efficient inference on consumer-level hardware.

  • Takeaways & Limitations

    SimpleFold shows that protein folding can be approached with a substantially simplified general-purpose architecture, providing an alternative design space for future models.

Abstract

from arXiv · show

Protein folding models have achieved groundbreaking results typically via a combination of integrating domain knowledge into the architectural blocks and training pipelines. Nonetheless, given the success of generative models across different but related problems, it is natural to question whether these architectural designs are a necessary condition to build performant models. In this paper, we introduce SimpleFold, the first flow-matching based protein folding model that solely uses general purpose transformer blocks. Protein folding models typically employ computationally expensive modules involving triangular updates, explicit pair representations or multiple training objectives curated for this specific domain. Instead, SimpleFold employs standard transformer blocks with adaptive layers and is trained via a generative flow-matching objective with an additional structural term. We scale SimpleFold to 3B parameters and train it on approximately 9M distilled protein structures together with experimental PDB data. On standard folding benchmarks, SimpleFold-3B achieves competitive performance compared to state-of-the-art baselines, in addition SimpleFold demonstrates strong performance in ensemble prediction which is typically difficult for models trained via deterministic reconstruction objectives. Due to its general-purpose architecture, SimpleFold shows efficiency in deployment and inference on consumer-level hardware. SimpleFold challenges the reliance on complex domain-specific architectures designs in protein folding, opening up an alternative design space for future progress.

1 Introduction

SimpleFold reframes protein folding as conditional generative modeling, replacing domain-specific architectural components with a general-purpose transformer and flow-matching objective. The resulting model family achieves competitive folding performance while supporting ensemble generation and efficient inference.

  • 1 Introduction: Protein folding models traditionally rely on computationally heavy MSAs, pair representations, and triangle updates that encode domain knowledge into their architectures.These designs underpin established systems such as AlphaFold2 and RoseTTAFold.
  • 1 Introduction: SimpleFold proposes a general-purpose transformer backbone trained end-to-end with flow matching, eliminating MSA, pairwise representations, and triangle modules.The model directly maps an amino-acid sequence to full 3D atomic coordinates without specialized equivariant geometric modules.
  • 1 Introduction: SimpleFold scales from 100M to 3B parameters and is trained on approximately 9M distilled structures alongside experimental PDB data.The paper presents scaling model size and training data as part of its general-purpose approach.
  • 1 Introduction: SimpleFold-3B achieves competitive folding performance against models using hard-coded heuristic designs and performs strongly on protein ensemble generation.The generative formulation is intended to represent non-deterministic structural outcomes rather than only deterministic reconstruction.
  • 1 Introduction: SimpleFold-100M recovers ∼90% performance of the best model on major folding benchmarks while remaining efficient on consumer-level devices.The released family spans an efficient small model and a larger model optimized for performance.

2 SimpleFold

SimpleFold treats protein folding as conditional flow-matching generation using general-purpose transformer blocks rather than domain-specific pair representations or triangular updates. Its pipeline combines noisy atomic coordinates, sequence conditioning, hierarchical atom–residue processing, structural supervision, and timestep resampling.

  • 2.2 Folding with Flow-Matching: The training objective combines flow-matching velocity regression with an LDDT loss measuring pairwise-distance errors between predicted and ground-truth structures.The LDDT contribution is weighted by a timestep-dependent term α(t).
  • 2.2 Folding with Flow-Matching: SimpleFold models folding as a flow-matching process that transforms noise into all-atom coordinates conditioned on an amino acid sequence.The model predicts a time-dependent velocity field from noisy structures, sequence information, and timestep.
  • 2.2 Folding with Flow-Matching: Timestep resampling places most samples near clean data, improving refined side-chain modeling within proteins’ coarse-to-fine structural hierarchy.The stated sampling distribution mixes a logistic-normal component with a small uniform component and shifts weight toward t = 1.
  • 2.3 Architecture: SimpleFold uses a general-purpose transformer architecture with adaptive layers across atom encoder, residue trunk, and atom decoder, avoiding explicit pair representations and triangular updates.The architecture contains lightweight symmetric atom modules and a larger residue trunk, all implemented with standard transformer blocks.
  • 2.3 Architecture: The architecture encodes sequence and structural position through pretrained protein-language-model embeddings, RoPE, local atom attention, and residue grouping and ungrouping.ESM2-3B supplies per-residue conditioning embeddings, while 4D axial RoPE uses coordinate and residue-index axes in atom modules.
  • 2.6 Training on Distilled Data: SimpleFold addresses the underused scale of distilled structural data by drawing on the large publicly available pool of computationally predicted protein structures.The cited motivation identifies AFDB and the ESM Metagenomic Atlas as sources containing hundreds of millions of predicted structures.

3 Related Work

Related protein-folding models established strong performance through domain-specific architectures, while later systems reduced MSA dependence with pretrained protein-language models. Flow-matching methods extend this direction toward generative ensemble prediction, although earlier examples adapted deterministic folding models rather than being built generatively from the ground up.

  • Protein Folding: AlphaFold2 and RoseTTAFold introduced domain-specific modules such as triangle attention and explicit interactions between single and pair representations.These models achieved groundbreaking performance while using architectures tailored to protein folding.
  • Protein Folding: OmegaFold and ESMFold replace multiple sequence alignments with learned pretrained protein-language-model embeddings, improving inference efficiency for orphan proteins.FastFold and MiniFold instead focus on efficient implementations of AlphaFold2 modules.
  • Flow-Matching for Proteins: AlphaFlow and ESMFlow apply flow-matching objectives to pretrained AlphaFold2 and ESMFold models, demonstrating advantages for ensemble generation without building generative folding models from scratch.The cited distinction is between adapting deterministic regression models and designing generative models natively.

4 Experiments

SimpleFold is evaluated as a scalable, general-purpose flow-matching folding model across standard structure-prediction, confidence-estimation, ensemble-generation, and scaling experiments. It achieves competitive benchmark performance, strong ensemble results, and improved performance with larger models and more training data.

  • 4.5 Effects of Scaling in Protein Folding: Scaling model size improves folding performance across benchmarks, with larger gains on the more challenging CASP14 benchmark.The study evaluates models ranging from 100M to 3B parameters.
  • 4.3 Confidence Measure with pLDDT: SimpleFold’s pLDDT confidence module correlates with actual LDDT-Cα at Pearson correlation 0.77 across CAMEO22 and sampled PDB chains.The model is generally confident on secondary structures and uncertain on flexible loops.
  • 4.4 Ensemble and Multi-state Prediction: SimpleFold demonstrates strong ensemble-generation performance, outperforming baselines on MD-simulation distributions and achieving state-of-the-art Apo/holo results.Performance also improves with model size in multi-state structure prediction.

5 Conclusions and Future Work

SimpleFold simplifies protein folding with general-purpose transformer blocks and flow-matching training. Its largest model achieves competitive folding performance and strong ensemble-generation results.

  • 5 Conclusions and Future Work: SimpleFold-3B demonstrates competitive performance on standard folding tasks and very strong or state-of-the-art results on multiple ensemble-generation tasks.Its generative training objective supports ensemble prediction, while the simplified architecture enables scaling in model size and training data.
  • 5 Conclusions and Future Work: SimpleFold uses adaptive general-purpose transformer blocks instead of pair representations and triangular updates, with flow matching plus an additional LDDT loss.The framework avoids multiple protein-specific loss terms used in earlier approaches.
  • 5 Conclusions and Future Work: The simplified architecture reduces reliance on computationally complex network blocks while preserving a scalable protein-folding framework.The authors describe SimpleFold as a strong departure from previous architectural designs.
  • 5 Conclusions and Future Work: Public code and checkpoints allow SimpleFold to be extended with adapter or LoRA fine-tuning and distillation for faster inference.The paper identifies applications beyond folding through protein-specific fine-tuning and faster deployment.

A Data Pipeline

SimpleFold uses a simplified input and preprocessing pipeline without MSA or template search. Training standardizes centered, rotated atomic coordinates and uses combined spatial and contiguous cropping for larger proteins.

  • A Data Pipeline: SimpleFold omits MSA and template search, so its input features are simpler than AlphaFold’s.The data pipeline largely follows Boltz-1 while using reduced input features.
  • A Data Pipeline: Larger proteins are cropped with spatial and contiguous strategies, using a neighborhood size sampled uniformly from zero to 40 tokens.This balances the two cropping strategies during training.
  • A Data Pipeline: Atomic positions are mean-centered, randomly rotated, and scaled by 1/16 into the [−1, 1] interval, while reference conformers are scaled by 1/5.These transformations standardize protein and reference-conformer coordinates.

B Model Architecture

SimpleFold replaces AlphaFold2-style pair and triangle modules with sequence-level representations and general-purpose transformer blocks. Its architecture supports substantially lower forward compute while grouping and ungrouping atom tokens through residue tokens.

  • B Model Architecture: SimpleFold uses sequence-level representations and pretrained PLM embeddings without explicit pair representations or MSA, unlike AlphaFold2’s Evoformer.The comparison highlights the removal of expensive triangle attention and pairwise modeling.
  • B Model Architecture: SimpleFold’s general-purpose transformer blocks require about 1.4Tflops for SimpleFold-3B, versus about 30Tflops for AlphaFold2 despite similar parameter counts at the smallest scale.The lower compute is attributed to avoiding triangle updates and explicit pair representations.
  • B Model Architecture: Figure 5 contrasts AlphaFold2’s Evoformer blocks with SimpleFold’s transformer block and adaptive layer design.The figure summarizes the major computational blocks used by the two models.
  • B Model Architecture: Grouping averages atom tokens within each residue into a residue token, while ungrouping replicates updated residue tokens across that residue’s atoms.These operations connect atom-level and residue-level processing.

C.1 Training Comparison to AlphaFlow and ESMFlow

SimpleFold differs from AlphaFlow and ESMFlow by using standard transformer blocks and training generatively from scratch. This design improves performance on multi-state benchmarks over models that only fine-tune deterministic pretrained models.

  • C.1 Training Comparison to AlphaFlow and ESMFlow: SimpleFold, AlphaFlow, and ESMFlow all use flow-matching objectives, but their architectural designs and training paradigms differ substantially.The comparison separates the shared objective from the models’ differing architectures and training procedures.
  • C.1 Training Comparison to AlphaFlow and ESMFlow: Unlike AlphaFlow and ESMFlow, SimpleFold uses no domain-specific pair representations or triangle attention and relies purely on standard transformer blocks.AlphaFlow and ESMFlow inherit these designs from AlphaFold and ESMFold architectures.
  • C.1 Training Comparison to AlphaFlow and ESMFlow: Training SimpleFold from scratch with flow matching improves multi-state benchmark performance over fine-tuned deterministic AlphaFold and ESMFold models.The comparison attributes the difference to building the generative model from the ground up rather than using generative fine-tuning alone.

C.2 Additional Training Details

The training and inference pipeline uses timestep resampling, rigid alignment, structural losses, and specified numerical settings, with efficiency evaluated against baseline models. These details include both optimization choices and reproducibility settings.

  • Training details: Training resamples timesteps with a mixture of uniform and logit-normal distributions, concentrating samples near t = 1 to capture refined details.The stated distribution is p(t) = 0.02 U(0, 1) + 0.98 LN(0.8, 1.7).
  • Training details: Rigidly aligning one-step denoised coordinates with true coordinates before flow-matching loss computation reduces loss variance.The alignment uses a one-step Euler estimate and the Kabsch algorithm.
  • Inference settings: Inference uses Euler–Maruyama integration for 500 steps, logarithmic timestep discretization, numerical stabilization, and coordinate rescaling to ångström units.Sampling starts at tϵ = 0.0001, uses η = 0.01, sets w(t) = 0 for t ≥ 0.99, and multiplies final coordinates by 16.
  • Inference efficiency: SimpleFold shows an inference-efficiency advantage over baseline models, especially for longer sequences such as length 1024.The comparison uses inference time measured on a single H100 with batch size 1.

D.4 Targets in Folding Tasks

The folding evaluation covers targets from CAMEO22 and CASP14, with the supplied lists containing 183 and 70 targets, respectively.

  • CAMEO22: CAMEO22 contributes 183 targets to the folding evaluation.The passage supplies the target identifiers used for this benchmark.
  • Target identifiers: The target lists include chain-specific identifiers and CASP14 variants such as s1, s2, and s3.These identifiers distinguish the evaluated target entries within each benchmark.
  • CASP14: CASP14 contributes 70 targets to the folding evaluation.The passage supplies the target identifiers used for this benchmark.

D.5 Evaluation Pipeline

The evaluation pipeline applies OpenStructure metrics to all-atom folding outputs and TM-score to models that cannot produce all-atom structures. It also evaluates ensemble and challenging low-homology settings.

  • Folding evaluation: OpenStructure computes LDDT, backbone LDDT, rigid scores, and TM-score for all-atom folding outputs.The evaluation uses OpenStructure 2.9.13 and its compare-structures command.
  • Folding evaluation: TM-score is used instead when folding or generation models cannot output all-atom structures because the OpenStructure pipeline fails in that setting.The structures are compared with the TMscore.cpp implementation.
  • Ensemble evaluation: MD ensemble evaluation samples 250 conformations for each of 82 ATLAS test targets and reports the median across targets.The pipeline covers flexibility and ensemble observables.
  • Two-state evaluation: Two-state prediction evaluation measures residue-flexibility correlation after sequence alignment and structural superposition, alongside a TM-ensemble score at ensemble size 5.The flexibility metric is RMSD Pearson correlation r.
  • Challenging targets: SimpleFold is evaluated against AlphaFold2 and ESMFold on de novo and orphan proteins, where orphan targets have no known sequence homologs.The orphan set contains 77 targets with maximal MSA depth 1.

E.2 Training with Self-Distilled Data

Self-distillation experiments test whether SimpleFold can achieve reasonable performance without data distilled from domain-specific teacher models. The results indicate that self-distillation and scaling improve some evaluation settings, while LDDT loss weighting is important for local structural quality.

  • Training procedure: The self-distillation study trains SimpleFold-700M-PDB, then generates two successive self-distilled datasets for SimpleFold-700M-R1 and SimpleFold-700M-R2.Each later model is trained on data generated by the preceding model using the same filtered sequence set for comparison.
  • Results: SimpleFold does not fundamentally require training data distilled from other models to obtain reasonable CASP14 and CAMEO22 performance.AFDB/AFESM data provides an edge, potentially because AlphaFold2 uses MSA, but is not presented as necessary.
  • MD ensembles: Larger SimpleFold and SimpleFold-MD models achieve better performance on ATLAS MD ensemble generation.SimpleFold is not tuned on ATLAS training data, whereas SimpleFold-MD is tuned on it.
  • LDDT loss: LDDT loss is required for refined local atomic positions, which substantially affects LDDT performance in folding tasks.The authors also vary LDDT loss weighting during finetuning on PDB and filtered SwissProt data.
  • LDDT loss: The weighting schedule α = 1 + 8ReLU(t − 0.5) achieves better overall performance than α = 1 during finetuning.The schedule is reported as the best overall strategy in the CAMEO22 ablation.

F.3 Failure Cases

SimpleFold’s failure cases show that it often recovers secondary structures but struggles to model the relative positioning of distinct secondary-structure domains. The same failure mode appears in ESMFold, suggesting a shared representation-related limitation.

  • F.3 Failure Cases: SimpleFold’s shown failure cases have TM-score < 0.6, with secondary structures mostly correct but their relative domain positions poorly modeled.These examples come from CAMEO22 and CASP14 targets.
  • F.3 Failure Cases: The same domain-positioning failure appears in ESMFold for examples including 7SZ2-B and 7WF9-A.The paper attributes this shared failure mode to the ESM2 embedding used by both models.
  • F.3 Failure Cases: The paper identifies stronger protein language models for representation learning as a future direction for improving protein folding models.
Loading 2509.18480v4…