Source-linked AI summary
La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching
Tomas Geffner, Kieran Didi, Zhonglin Cao, Danny Reidenbach, Zuobai Zhang, Christian Dallago, Emine Kucukbenli, Karsten Kreis, Arash Vahdat
TL;DR
Generating protein sequences jointly with fully atomistic structures remains challenging because side chains vary in length and many approaches separate sequence from structure. La-Proteina explicitly models the backbone and uses fixed-dimensional per-residue latents for sequence and atomistic detail, with flow matching to generate both. It reports state-of-the-art performance across atomistic generation and motif scaffolding tasks, including co-designable proteins up to 800 residues.
Problem
Few methods directly generate amino acid sequence jointly with fully atomistic structure, which requires handling discrete sequences, continuous coordinates, and residue-dependent side-chain dimensionality.
Source: §1 · p. 1, §3.1 · p. 4Method
La-Proteina models α-carbon coordinates explicitly and encodes sequence and non-α-carbon details in fixed-dimensional per-residue latents, then uses flow matching to jointly generate coordinates and latents.
Source: §1 · p. 1, §3.1 · p. 4, §3.2 · p. 5Results
La-Proteina reports state-of-the-art unconditional and conditional atomistic design performance, including co-designable proteins up to 800 residues and strong indexed and unindexed motif-scaffolding results.
Source: §1 · p. 2, §5 · p. 10Takeaways & Limitations
The results support using La-Proteina for de novo monomer generation and atomistic motif scaffolding, including the unindexed setting.
Source: §1 · p. 2, §5 · p. 10This work evaluates de novo monomer design; the presented model was not trained to handle protein complexes.
Source: A LIMITATIONS AND FUTURE WORK · p. 17
Abstract
from arXiv · showhide
Recently, many generative models for de novo protein structure design have emerged. Yet, only few tackle the difficult task of directly generating fully atomistic structures jointly with the underlying amino acid sequence. This is challenging, for instance, because the model must reason over side chains that change in length during generation. We introduce La-Proteina for atomistic protein design based on a novel partially latent protein representation: coarse backbone structure is modeled explicitly, while sequence and atomistic details are captured via per-residue latent variables of fixed dimensionality, thereby effectively side-stepping challenges of explicit side-chain representations. Flow matching in this partially latent space then models the joint distribution over sequences and full-atom structures. La-Proteina achieves state-of-the-art performance on multiple generation benchmarks, including all-atom co-designability, diversity, and structural validity, as confirmed through detailed structural analyses and evaluations. Notably, La-Proteina also surpasses previous models in atomistic motif scaffolding performance, unlocking critical atomistic structure-conditioned protein design tasks. Moreover, La-Proteina is able to generate co-designable proteins of up to 800 residues, a regime where most baselines collapse and fail to produce valid samples, demonstrating La-Proteina's scalability and robustness.
1 INTRODUCTION
La-Proteina addresses the challenge of jointly generating protein sequence and fully atomistic structure by modeling backbone coordinates explicitly and encoding sequence and atomistic details in fixed-size per-residue latents. It reports strong unconditional-generation and motif-scaffolding results, including co-designable proteins up to 800 residues.
1 INTRODUCTION
- Existing approaches often separate sequence and structure generation, while direct atomistic modeling must handle discrete sequences, continuous coordinates, and side chains whose dimensionality depends on residue type.Prior data-space methods can struggle with accuracy and scalability, while several latent-representation approaches have not delivered competitive performance.Source: §1 · p. 1
- La-Proteina models α-carbon coordinates explicitly while representing sequence and non-α-carbon atom details in a continuous, fixed-size latent variable for each residue.A VAE encodes sequence and side-chain details, then a flow-matching model jointly generates α-carbon coordinates and latents for decoding into sequences and full-atom structures.Source: §1 · p. 1, Figure 1 · p. 2
- The partially latent representation shifts the main learning problem from mixed discrete–continuous data with variable dimensionality to a continuous space of fixed dimensionality per residue.This is intended to make the generation problem amenable to methods such as flow matching.Source: §1 · p. 1
- The model uses efficient transformer architectures and was trained on up to 46 million protein structure–sequence pairs.The paper presents this implementation as supporting scalability to long proteins, large models, and large training data.Source: §1 · p. 2
- La-Proteina achieves state-of-the-art atomistic performance on co-designability and diversity, generates co-designable proteins up to 800 residues, and outperforms baselines across indexed and unindexed motif-scaffolding setups.The paper also reports strong geometric-quality evaluations and structural validity; other methods collapse or run out of memory in the long-protein regime.Source: §1 · p. 2
2 PRELIMINARIES
The preliminaries introduce protein sequence-and-structure representations, variational autoencoders, and flow matching, then situate atomistic protein generation among prior design approaches.
2 PRELIMINARIES
- A VAE encodes a sample into a latent-variable distribution and decodes latents into data, with ELBO training balancing reconstruction quality against KL regularization toward a prior.The cited description gives the objective as expected log-likelihood minus KL divergence.Source: §2 · pp. 2–3
- Flow matching trains a velocity network to transport samples from a base distribution to the data distribution along a probability path.The described linear path interpolates between base and data samples, and training minimizes conditional flow-matching error.Source: §2 · p. 3
- A protein structure can be stored in Atom37 form as an [L, 37, 3] tensor, with the residue type determining which subset of coordinates is relevant.Residues share a backbone including the α-carbon but differ in their side-chain atoms.Source: §2 · p. 3
- Prior protein generators have addressed backbone design, sequence–structure co-design, and fully atomistic generation using data-space or latent-variable approaches.The cited landscape also includes sequence-only language models and models that tokenize structure and sequence jointly.Source: §2 · p. 3
3 LA-PROTEINA
La-Proteina combines explicit α-carbon coordinates with fixed-dimensional per-residue latents to model atomistic protein structure and sequence. A VAE handles sequence and non-α-carbon details, while flow matching generates coordinates and latents with separate schedules.
3.1 MOTIVATION: PARTIALLY LATENT REPRESENTATION FOR ATOMISTIC PROTEIN DESIGN
- Fixed-dimensional per-residue latents encode sequence and non-α-carbon details while explicit α-carbon coordinates preserve the backbone representation, avoiding mixed continuous-categorical modeling in the main generator.The approach also permits established backbone architectures to be used without increasing internal sequence length.Source: §3.1 · p. 4
3.2 PROBABILISTIC FORMULATION
- The joint protein model factorizes into a flow-matched distribution over α-carbon coordinates and latents, and a decoder distribution over sequence and non-α-carbon atoms conditioned on both.The latent variables capture dependencies among backbone, sequence, and side chains; conditioning supports simple factorized decoder distributions despite mixed data types and variable side-chain dimensions.Source: §3.2 · pp. 4–5
3.2.1 VARIATIONAL AUTOENCODER
- The VAE encoder maps complete proteins to per-residue latents, and its decoder reconstructs sequence and non-α-carbon coordinates from latents and α-carbon coordinates.The decoder uses factorized categorical sequence distributions and factorized Gaussian atom-coordinate distributions with unit variance.Source: §3.2.1 · p. 5
3.2.2 PARTIALLY LATENT FLOW MATCHING
- Separate interpolation times for α-carbon coordinates and latents allow different inference schedules; coupling them to one time leads to worse results.The schedules are chosen so that α-carbon coordinates are generated faster than latents, a strategy reported to improve results.Source: §3.2.2 · p. 6, §3.4 · p. 6
3.3 NEURAL NETWORK ARCHITECTURES
- The encoder, decoder, and denoiser share transformer architectures with pair-biased attention, while the main models omit costly triangular update layers to maintain scalability.The passage reports approximately 130M parameters each for the encoder and decoder, and 160M for the denoiser; triangular layers can optionally improve pair representations and co-designability.Source: §3.3 · p. 6
3.4 MODEL SAMPLING
- Sampling generates α-carbon coordinates and latents with partially latent flow matching, then decodes them into atom coordinates and sequence.The decoder takes the Gaussian mean for non-α-carbon coordinates and the categorical arg max for sequence; Table 1 covers unconditional all-atom design at lengths 100–500 and reports diversity, novelty, and secondary structure on co-designable samples.Source: §3.4 · pp. 6–7, Table 1 · p. 7
4 EXPERIMENTS
The experiments evaluate unconditional atomistic generation and atomistic motif scaffolding, alongside analyses of La-Proteina’s learned latent representation. They compare generated structures on designability, diversity, structural quality, and motif-scaffolding success.
4.1 ALL-ATOMISTIC UNCONDITIONAL PROTEIN STRUCTURE GENERATION BENCHMARK
- Both La-Proteina variants outperform all baselines in all-atom co-designability, designability, and diversity, while remaining highly competitive in novelty.The variant without triangular multiplicative layers establishes state-of-the-art performance while remaining highly scalable.Source: §4.1 · p. 7
- Above 500 residues, La-Proteina produces co-designable, diverse proteins while all-atom baselines fail to produce realistic samples.La-Proteina also performs best on backbone and all-atom (co-)designability and diversity in the reported length comparisons.Source: Figure 4 · p. 8, §4.1 · p. 8
- MolProbity analyses show La-Proteina produces higher-quality, more physically realistic structures than all baselines.The evaluated properties include bond angles, clashes, and other physical quantities.Source: §4.1 · p. 8, Figure 5 · p. 8
- La-Proteina recovers the major tryptophan χ1 rotamer states and their reference frequencies, while baselines often miss modes or populate unrealistic angles.The comparison uses PDB and AFDB references.Source: §4.1 · p. 8, Figure 6 · p. 8
4.2 ATOMISTIC MOTIF SCAFFOLDING
- La-Proteina solves 21–25 of 26 motif-scaffolding tasks across the four setups, outperforming both Protpardelle variants; for motifs with at least three segments, its unindexed version consistently beats its indexed version.The authors hypothesize that fixing multiple segment positions limits flexibility to explore structural solutions.Source: Figure 7 · p. 9, §4.2 · p. 9
4.3 AUTOENCODER EVALUATION AND LATENT SPACE ANALYSIS
- Perturbing one residue’s latent vector predominantly affects reconstruction of that residue, leaving other residues almost unaffected.The analysis measures sequence and structure reconstruction losses as perturbation magnitude varies.Source: Figure 8 · p. 9, §4.3 · p. 10
5 CONCLUSIONS
La-Proteina achieves state-of-the-art performance in unconditional and conditional atomistic protein design, generating realistic structures up to 800 residues. Its partially latent flow-matching design combines explicit backbone coordinates with fixed-size per-residue latents for sequence and side chains.
5 CONCLUSIONS
- La-Proteina achieves state-of-the-art performance in unconditional and conditional atomistic protein design and generates realistic structures up to 800 residues.The conclusion identifies unindexed atomistic motif scaffolding as a task with potential to support applications such as binder and enzyme design.Source: §5 · p. 10
- The partially latent design uses explicit backbone structure and fixed-size per-residue latents for sequence and side chains, avoiding variable-dimensional side-chain representations.The paper says this choice helps sidestep scalability and accuracy issues faced by other methods.Source: §5 · p. 10
- The authors suggest strong atomistic-design performance, including unindexed motif scaffolding, could enable binder and enzyme design applications.Source: §5 · p. 10
6 ETHICS STATEMENT
The ethics statement notes potential scientific, societal, and industrial benefits of de novo protein design, alongside possible misuse risks that call for ethical oversight.
6 ETHICS STATEMENT
- De novo protein-design technologies could accelerate discovery and development of protein-based therapeutics.Source: §6 · p. 10
- Designed enzymes could support greener chemical processes and sustainable materials in biotechnology and industry.Source: §6 · p. 10
- The statement cautions that novel functional protein design could be misused, including hypothetically for biosecurity threats, without ethical oversight.Source: §6 · p. 10
7 REPRODUCIBILITY STATEMENT
The reproducibility statement reports a public code release and supplementary materials covering data, models, training, and evaluation procedures.
7 REPRODUCIBILITY STATEMENT
- The authors release La-Proteina code at the stated GitHub repository.Source: §7 · p. 10
- Supplementary materials specify dataset-construction filters, model architectures, and training parameters, including GPU count and training steps.Source: §7 · p. 10
- The supplement gives metric descriptions and exact commands for evaluating unconditional models and atomistic motif scaffolding.Source: §7 · p. 10
A LIMITATIONS AND FUTURE WORK
La-Proteina’s current application focuses on monomeric proteins, while its unconditional samples are diverse, co-designable, and span a broad range of sizes.
A LIMITATIONS AND FUTURE WORK
- The current work applies La-Proteina to monomeric proteins and does not train this instantiation to handle protein complexes.The authors frame complex design as outside the current application scope, not a constraint of the underlying framework.Source: A LIMITATIONS AND FUTURE WORK · p. 17
B.1 UNCONDITIONAL La-Proteina SAMPLES
- La-Proteina generates diverse, co-designable fully atomistic proteins across a broad range of residue counts.Source: §B.1 · p. 17
B.2 ATOMISTIC MOTIF SCAFFOLDING La-Proteina SAMPLES
La-Proteina reproduces partial atomistic motifs in valid scaffolds, including diverse scaffolds of the same motif. The appendix also describes its training datasets and staged VAE training.
B.2 ATOMISTIC MOTIF SCAFFOLDING La-Proteina SAMPLES
- The motif-scaffolding examples reproduce conditioning atoms in almost all cases and yield valid proteins, with diverse solutions for the same motif.The examples scaffold enzyme active sites when only the tips of conditioning side chains are given.Source: §B.2 · p. 17, Figure 10 · p. 19, Figure 11 · p. 20
- Figure 9 shows co-designable fully atomistic unconditional samples, with residue counts indicated for each protein.Source: Figure 9 · p. 18
C.1 DATASETS
- The Foldseek Clustered AFDB dataset is filtered from cluster representatives to about 350k samples using confidence, length, and secondary-structure criteria.The initial clustering yields about three million samples; an optional beta-sheet filter raises generated-sample beta-sheet content from about 3–4% to about 10%.Source: §C.1 · p. 22
- Long-protein training uses a custom AFDB subset of 46,942,694 structures, clustered into 4,035,594 sequence clusters for sampling during training.Source: §C.1 · p. 22
C.2.1 VAE TRAINING
- The VAE is trained in three length stages, and the stage-(ii) parameters serve flow models up to 512 residues while stage (iii) supports models up to 800.The stages cover 32–256, 32–512, and 32–896 residues, respectively.Source: §C.2.1 · p. 22
C.2.2 FLOW MATCHING TRAINING
The appendix details flow-model training, baseline sampling, evaluation metrics, and structural-quality analyses. MolProbity analyses find realistic La-Proteina structures across lengths, while most baselines degrade at longer lengths.
C.2.2 FLOW MATCHING TRAINING
- The denoiser is trained with conditional flow matching, and the α-carbon and latent interpolation times are independently sampled using distributions shown in Figure 13.The appendix reports separate training runs for models with and without triangular multiplicative update layers, including a long-protein model trained up to 896 residues.Source: §C.2.2 · pp. 22–23, Figure 13 · p. 23
C.3 BASELINE SAMPLING
- Baseline comparisons sample 100 proteins per model at each length, using the official implementations where available.P(all-atom) could not generate a sample beyond 500 residues on a 140GB GPU because of memory use; PLAID supports only lengths divisible by eight.Source: §C.3 · p. 23
D.1 CO-DESIGNABILITY, DESIGNABILITY, DIVERSITY, NOVELTY
- Co-designability counts samples whose generated sequences refold to within 2 Å all-atom RMSD, while designability tests whether any ProteinMPNN-designed sequence folds within 2 Å Cα RMSD.The appendix also reports pLDDT, three sequence/structure diversity variants, and novelty against PDB and AFDB reference sets; novelty values vary with Foldseek version.Source: §D.1 · pp. 24–25
D.2 MOLPROBITY FOR STRUCTURAL QUALITY ASSESMENT
- MolProbity evaluates structural geometry, stereochemistry, and contacts; the appendix reports composite and component metrics, including rotamer and Ramachandran outliers.Rotamer outliers assess side-chain torsions against a reference library, and side-chain torsions cluster into distinct rotamers rather than varying randomly.Source: §D.2 · pp. 25–26, §D.3.1 · p. 27
- La-Proteina produces highly realistic structures across lengths, while other baselines generally become less plausible at longer lengths.P(all-atom) is an exception for rotamer and Ramachandran outliers, but has poor clash, angle-outlier, and bond-outlier scores; its Figure 14 results stop at 500 residues due to memory limits.Source: §D.2 · p. 26, Figure 14 · p. 26
D.3.1 BACKGROUND ON AMINO ACID ROTAMERS
- Rotamer preferences vary by residue, and some terminal side-chain torsions, including ASP χ2 and ASN χ2, are non-rotameric and span a more continuous range.The appendix also notes additional backbone-dependent and backbone-independent factors affecting rotamer populations.Source: §D.3.1 · p. 27
D.3.2 ANALYSIS OF GENERATED AMINO ACID ROTAMERS
La-Proteina often recovers side-chain rotamer modes at approximately the reference frequencies, while some baselines miss modes or reproduce them at different frequencies. Sampling uses an SDE with separately scheduled backbone and latent-variable paths, and motif scaffolding tests both indexed and unindexed conditioning.
D.3.2 ANALYSIS OF GENERATED AMINO ACID ROTAMERS
- La-Proteina often captures the correct rotamer modes at approximately the PDB and AFDB reference frequencies, while P(all-atom) and Protpardelle often miss modes.PLAID and ProteinGenerator often recover modes but at different frequencies from the reference datasets.Source: §D.3.2 · p. 27
- For ARG χ4 and LYS χ3/χ4, La-Proteina tracks AFDB more closely, while other methods miss PDB modes despite training on datasets that include PDB.La-Proteina was trained on AFDB structures.Source: §D.3.2 · p. 28
E SAMPLING
- Sampling simulates an SDE whose noise parameters ηx and ηz control sampling temperature; ηx = ηz = 1 gives unbiased sampling, while smaller values produce low-temperature samples.The paper reports ablations for these choices in Sec. G.Source: E SAMPLING · p. 31
E.1 NUMERICAL DISCRETIZATION SCHEME
- Empirical analyses favor evolving α-carbon coordinates faster than latent variables, using an exponential schedule for the former and a quadratic schedule for the latter, with 400 integration steps.The time paths are independently parameterized and discretized over a shared time variable.Source: §E.1 · p. 31
F ATOMISTIC MOTIF SCAFFOLDING
- Motif scaffolding evaluates all-atom tasks, which provide backbone and side-chain positions plus residue identity, and tip-atom tasks, which provide residue identity and atoms beyond the final rotatable bond.Indexed tasks provide motif sequence positions; unindexed tasks withhold them so the model must find a placement while generating the scaffold.Source: F ATOMISTIC MOTIF SCAFFOLDING · pp. 31–32
F.1 TRAINING
Motif-scaffolding models receive task-specific motif features and are evaluated on sequence recovery, motif geometry, and co-designability. La-Proteina outperforms Protpardelle and Protpardelle-1c under direct evaluation and remains state of the art when generated sequences are refolded.
F.1 TRAINING
- All-atom motif conditioning includes absolute and α-carbon-relative coordinates, residue type, side-chain angles, and backbone torsions; tip-atom conditioning uses the available atom coordinates and residue type.Indexed inputs attach these features to residue indices; unindexed inputs omit motif-index information.Source: §F.1 · pp. 32–33
- The indexed and unindexed scaffolding setups differ in whether motif residue positions are supplied, with the unindexed model required to find a viable placement during scaffold generation.The dataset is Foldseek-clustered AFDB, capped at 356 residues and filtered for average pLDDT of at least 80.Source: §F.1 · p. 33
F.3 EVALUATION
- Each method generates 200 samples per task, and unique successes are counted by clustering successful samples with Foldseek.Unindexed evaluation first identifies corresponding motif residues through greedy structural matching, independently for each generated protein.Source: §3 · p. 34
- La-Proteina outperforms Protpardelle and Protpardelle-1c in direct motif-scaffolding evaluation, and achieves state-of-the-art results when evaluated on ESMFold-refolded sequences.Protpardelle solves 4/26 tasks in both all-atom and tip-atom setups under the stricter evaluation; the direct-evaluation criteria align closely with its reported strict definition.Source: §3 · p. 34, Figure 33 · p. 34, §1 · p. 35
F.4 EVALUATION VIA REFOLDED STRUCTURES
- Refolded evaluation predicts structures from generated sequences with ESMFold and compares those structures with the motif; La-Proteina remains state of the art under this evaluation.Protpardelle is omitted from this comparison because its performance was not competitive with La-Proteina and Protpardelle-1c.Source: §F.4 · pp. 34–35, §1 · p. 35
G.1 VAE ABLATIONS
- The VAE ablation tests KL-term weights, decoder architecture, and encoding α-carbons into the latent space rather than modeling them explicitly.The α-carbon-encoding variant uses an eight-dimensional latent space and a transformer decoder.Source: §G.1 · p. 36, §G.1.1 · p. 36, §G.1.2 · p. 36
G.1.4 RESULTS
VAE and sampling ablations support explicitly modeling α-carbon coordinates and generating them faster than latent variables. The architecture uses transformer networks with sequence and pair representations, and a denoiser conditioned on interpolation times.
G.1.4 RESULTS
- Lower KL weights of 10^-4 and 10^-5 outperform 10^-3, a 7M-parameter feed-forward decoder performs worse than the transformer, and explicit α-carbon modeling substantially beats encoding the whole structure in latents.The comparisons report all-atom co-designability and diversity; the CA-encoding variant received a hyperparameter search for fair evaluation.Source: §G.1.4 · p. 37, Table 7 · p. 38
G.2 FLOW MATCHING SAMPLING HYPERPARAMETERS
- Only tested sampling configurations that evolve α-carbon coordinates faster than latent variables exceed 0.5 all-atom co-designability; successful configurations use exponential discretization for α-carbons and quadratic or uniform discretization for latents.Langevin scaling also affects performance, but less than the discretization scheme.Source: §G.2 · p. 39, Table 8 · p. 39
G.3 MAIN CONCLUSIONS FROM ABLATION STUDIES
- The ablations identify explicit α-carbon modeling and faster α-carbon evolution than latent-variable evolution as the two key factors for strong performance.The latent variables encode remaining atomic and sequence details.Source: §G.3 · p. 39
H ARCHITECTURES
- The encoder, decoder, and denoiser use transformer architectures with sequence and pair representations; the denoiser also conditions on interpolation times tx and tz.Pair features encode relationships such as residue distances and sequence separation, while pair-biased attention incorporates them into sequence processing.Source: H ARCHITECTURES · pp. 39–40
I MODEL PARAMETERS, SAMPLING SPEED AND MEMORY CONSUMPTION
La-Proteina combines fast single-sample generation with high batched throughput, and its results remain strong across shorter-protein evaluations and structural-confidence analyses. Amino-acid distributions move closer to UniProt at higher temperatures, but with lower co-designability.
I MODEL PARAMETERS, SAMPLING SPEED AND MEMORY CONSUMPTION
- La-Proteina without triangle multiplication layers is among the fastest methods despite its parameter count; the triangle-layer variant is slower but beats several baselines and PLAID at short lengths.At maximum batch size, La-Proteina generates hundreds of proteins per batch, with sub-second inference times for short proteins.Source: I MODEL PARAMETERS, SAMPLING SPEED AND… · p. 41, Table 9 · p. 41
J.1 NOVELTY RESULTS WITH NEWER FOLDSEEK VERSION
- Novelty values differ across Foldseek versions, so the newer-version results should be contrasted with the main-paper values from the earlier version.The other metrics are unchanged between the reported versions.Source: §J.1 · p. 42, Table 11 · p. 42
J.2 UNCONDITIONAL EVALUATION ON SHORTER PROTEINS
- La-Proteina achieves state-of-the-art all-atom co-designability and diversity across three shorter-protein evaluation scenarios.The scenarios contain proteins of length 100; lengths 100 and 200; or lengths 100, 200, and 300.Source: §J.2 · p. 42, Table 12 · p. 43
J.3 PLDDT VALUES
- All La-Proteina variants consistently have the highest median and interquartile-range pLDDT among methods for the evaluated co-designable samples.The analyzed samples are filtered to all-atom co-designable structures.Source: §J.3 · p. 42, Figure 34 · p. 44
J.4 RESIDUE TYPE DISTRIBUTION
- All methods over- and under-represent some residues relative to UniProt, while La-Proteina's amino-acid distribution is similar to ProteinMPNN's.The comparison uses co-designable samples.Source: §J.4 · p. 42, Figure 35 · p. 44
- Increasing both temperature parameters moves amino-acid frequencies closer to UniProt, while all-atom co-designability falls from 90% at (0.1, 0.1) to 54% at (0.5, 0.5) and 11% at (0.9, 0.9).This analysis generated 1,000 samples of length 100.Source: §J.4 · p. 43