PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Latent Protein Languages for Autoregressive Generation

Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi, Mohammadreza Pourmirzaei, Dong Xu

arXiv:2610.03978v1cs.LGq-bio.BM

TL;DR

Autoregressive protein models remain comparatively weak for sequence and structure generation. This paper introduces decodable latent sequence and structure languages and separately trains next-token transformers on them. The reported results support learned target representations as a promising direction, while identifying limitations in quality-centered generation and joint sequence-structure modeling.

  • Problem

    Autoregressive protein modeling remains less effective than non-autoregressive alternatives across sequence design and structure generation, motivating study of whether target representation matters.

    Source: §1 · p. 1

  • Method

    The paper introduces decodable sequence and structure latent languages and separately trains decoder-only transformers to generate their tokens.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    Matched sequence comparisons show PLLM has steeper fitted compute scaling and less low-complexity drift, while SLL improves target predictability and supports structure generation and sequence-to-structure prediction.

    Source: §5 · p. 13, §6 · p. 14

  • Takeaways & Limitations

    The results are early evidence that learned latent protein languages are a promising direction for autoregressive sequence and structure generation.

    Source: §6 · p. 14

    The models are not strongest on quality-centered metrics, and unified multimodal pretraining and joint PLL–SLL co-generation remain unevaluated.

    Source: §5 · pp. 13–14

Abstract

from arXiv · show

Autoregressive transformers remain comparatively weak for protein sequence and structure generation. We study the role of target representation: amino acid tokens encode residue identities without explicit contextual semantics, while backbone coordinates require a discrete representation in our framework. We introduce two learned latent protein languages. Protein Latent Language (PLL) maps sequences to a 4,096-state contextual alphabet built on a frozen ESM-2 encoder, with one token per residue. Structure Latent Language (SLL) adapts GCP-VQVAE Lite with auxiliary sequence and confidence supervision while retaining decoding to backbone coordinates. We separately pretrain autoregressive transformer models on PLL and SLL tokens using next-token prediction, yielding PLLM and SLLM. Under matched downstream sequence training, PLLM has a fitted compute-scaling exponent of 0.038 versus 0.020 for the amino acid autoregressive model. In unconditional sequence generation, PLLM reduces the fraction of samples below a heuristic 1.5-bit residue-composition entropy threshold by 54% relative to the amino acid model across sampling temperatures. For sequence-to-structure prediction, replacing the original GCP-VQVAE Lite tokenizer with SLL reduces best validation perplexity by 34% under matched training. For long proteins, latent-token sampling is approximately 1,000 times faster than MSA-based AlphaFold2 in our measurements. In backbone generation, SLLM compares favorably with other generative models on diversity and novelty. We also observe early signs that using SLLM's internal token confidence for inference-time sampling can improve sequence-to-structure prediction quality beyond a single decoded sample. These results position learned latent protein languages as a promising substrate for autoregressive transformer scaling and inference-time sampling in protein generation.

1. Introduction

Autoregressive protein models remain weaker than non-autoregressive alternatives for both sequence and structure generation. The paper investigates whether learned, decodable latent target languages can improve autoregressive modeling while retaining the standard recipe.

1. Introduction

  • Autoregressive protein models lag non-autoregressive alternatives in sequence design and backbone generation, while sequence sampling can drift into repetitive, low-complexity sequences.The paper identifies representation as a potential part of the problem and studies learned target languages without changing the autoregressive recipe.Source: §1 · p. 2
  • The paper introduces PLL, an ESM-2-derived contextual sequence language, and SLL, a structure language decodable to backbone coordinates.PLLM and SLLM are trained separately; the study examines scaling, de novo generation, and sequence-to-structure prediction.Source: §1 · p. 2
  • Under matched downstream sequence training, PLLM shows steeper fitted compute scaling and substantially less low-complexity drift than amino acid modeling.The contributions also report selected diversity and novelty comparisons for autoregressive models using learned latent languages.Source: §1 · p. 2
  • SLL adapts GCP-VQVAE Lite with auxiliary supervision, while confidence filtering and token-space consensus are used to select generated structure candidates.Both latent languages remain decodable to amino acids or backbone coordinates.Source: §1 · p. 2

2. Related Work

Prior work applies autoregressive modeling to protein sequences and, increasingly, structures, while diffusion, flow-matching, and related non-autoregressive approaches set many current generation baselines.

2. Related Work

  • Decoder-only models have been trained on amino acid strings, while structure-side autoregression has explored discrete backbone codes and conditional structure language models [17] [34] [14] [31].Other autoregressive approaches predict structure targets or use multimodal interfaces for sequence, structure, and description generation [37] [4].Source: §2 · pp. 2–3
  • Discrete diffusion has been applied to protein sequence generation and joint sequence-structure modeling [48] [49].These approaches use discrete structure tokens in the joint setting.Source: §2 · p. 3
  • Diffusion, flow matching, and programmable geometric generation provide strong controllability and designability baselines for backbone or sequence-structure generation [2] [15].These methods avoid left-to-right sampling and motivate testing whether latent tokenization makes autoregressive transformers competitive.Source: §2 · p. 3

3. Method

The method changes the protein target representation while retaining decoder-only next-token modeling. It defines sequence and structure latent languages, trains separate autoregressive models on their token streams, and adapts SLLM for sequence-conditioned structure generation and confidence-based candidate selection.

3. Method

  • The method targets vocabulary size, token semantics, and autoregressive friendliness by learning discrete, ordered representations for sequences and backbones.The amino acid–PLL comparison evaluates the complete PLL representation and does not isolate the effects of its individual components.Source: §3 · p. 3

3.1. PLL

  • PLL uses frozen ESM-2 contextual representations to produce one discrete code per residue, with a 4,096-state alphabet that remains decodable to amino acids.Its two-stage construction first learns a quantized latent representation, then trains amino acid recovery from fixed PLL codes.Source: §3.1 · pp. 3–4

3.2. SLL

  • SLL adapts GCP-VQVAE Lite, retaining backbone reconstruction while adding sequence, confidence, and ESM-2 representation supervision to the tokenizer objective.The final recipe is selected with PST/StructTokenBench for embedding utility beyond coordinate reconstruction, with the Lite architecture and training corpus fixed.Source: §3.2 · pp. 4–5

3.3. Autoregressive Protein Modeling

  • Raw amino acid, PLL, and SLL streams are pretrained separately with decoder-only transformers using cross-entropy next-token prediction.Unconditional generation uses beginning, content, and end tokens; controlled generation can prepend tokens such as a length condition.Source: §3.3 · p. 5

3.3.1. Sequence-to-Structure Generation

  • For sequence-to-structure generation, E1 sequence embeddings condition the decoder, which autoregressively generates SLL tokens that the fixed structure decoder maps to backbone coordinates.The amino acid sequence is encoded outside the causal token stream.Source: §3.3.1 · p. 5

3.3.2. Confidence-Based Candidate Selection

  • For candidate selection, SLLM full-vocabulary entropy serves as an internal confidence signal, and the practical selector combines confidence summaries with token-space consensus.Oracle best-of-k is an evaluation ceiling; the practical selector does not use ground truth.Source: §3.3.2 · pp. 5–6

4. Experiments

The experiments first assess whether PLL and SLL preserve useful sequence or structure information, then test whether autoregressive models trained on those latent tokens scale and generate effectively. The evaluations cover tokenizer quality, unconditional sequence and backbone generation, and sequence-to-structure prediction.

4.1.2. SLL Results

  • The combined SLL tokenizer recipe has the strongest average supervised PST performance among comparable ablations, improving over the base tokenizer by 24.2% in functional-site AUROC and 35.0% in physicochemical Spearman.The ablation keeps the GCP-VQVAE Lite architecture and training corpus fixed.Source: §4.1.2 · pp. 6–7, Table 1 · p. 7
  • SLL's average TM-score is 2.4% below the original GCP-VQVAE Lite tokenizer and close to AIDO (0.9456 versus 0.9470), while its average RMSD is lower (1.4722 versus 1.8189 Å).SLL also exceeds AIDO on functional-site AUROC and physicochemical Spearman in the reported supervised PST comparison.Source: §4.1.2 · p. 7, Table 3 · p. 7
  • 34% lower best validation perplexity and a comparable training-loss regime reached 49% faster by epoch when the target tokenizer changes to GCP-VQVAE Lite 2.The comparison uses a matched Prot2Token-style sequence-to-structure setup.Source: §4.1.2 · p. 7, Figure 4 · p. 10
  • In the matched sequence-to-structure tokenizer comparison, Lite 2 lowers the best validation-perplexity value by 34%, with learning-speed annotations comparing training progress by epoch.Source: Figure 4 · p. 10

4.2.1. Do Latent Languages Scale Better?

  • PLLM's fitted compute-scaling exponent is 0.038 versus 0.020 for the amino acid model under matched downstream sequence training.SLLM's exponent is 0.049 on structure-token data with different training exposure; the paper compares fitted slopes, not absolute cross-entropy across vocabularies.Source: Figure 5 · p. 10, §4.2.1 · p. 11

4.2.2. PLL De Novo Generation

  • 43.0% of amino acid samples versus 19.9% of PLLM samples fall below the 1.5-bit residue-composition entropy threshold across the unconditional temperature sweep.PLLM reduces this diagnostic failure mode, but low-complexity samples remain; these are unfiltered unconditional generations.Source: §4.2.2 · p. 11, Figure 6 · p. 11

4.2.3. SLL De Novo Generation

  • SLLM has the highest cluster ratio and lowest Max TM among backbone generators with mean scTM above 0.5, while La-Proteina has the highest mean scTM and lowest mean scRMSD.SLLM's self-consistency remains below specialized backbone generators, and the systems differ in training data, model size, and modality.Source: §4.2.3 · p. 12, Table 5 · p. 12, §4.2.2 · p. 12

4.2.4. Sequence-to-Structure Prediction

  • On CAMEO-2024, lDDT-CA and RMSD improve overall as sampled SLL trajectories increase, and DeepConf consensus recovers a meaningful fraction of the oracle gain before coordinate decoding.Oracle best@k selects after coordinate decoding; DeepConf consensus@k uses internal token confidence and token-space consensus to select one candidate beforehand.Source: §4.2.4 · pp. 12–13, Figure 7 · p. 13

5. Discussion

Learned target representations support stronger autoregressive protein modeling on selected measures, with PLLM improving sequence-generation behavior and SLL offering a complementary structure-side study. The discussion also identifies quality and evaluation boundaries, including untested joint sequence–structure modeling.

5. Discussion

  • PLLM shows steeper fitted compute scaling and less low-complexity drift than amino acid modeling, while SLL improves target predictability and is evaluated for backbone generation.Source: §5 · p. 13
  • SLLM’s backbone-generation strengths are diversity and novelty, while self-consistency quality remains a limitation in a tradeoff among quality, cluster diversity, and novelty.Source: §5 · p. 13
  • PLLM reduces low-complexity drift and better matches UniRef50 composition entropy and length statistics at moderate temperatures, while amino acid modeling has higher pLDDT and PLLM a higher cluster ratio.Source: §5 · p. 13
  • SLLM generates long-protein structure-token candidates about 1,000 times faster than MSA-based AlphaFold2, and internal confidence with token-space consensus recovers part of the oracle best@k gain.Source: §5 · p. 14
  • The models are not strongest on quality-centered metrics, especially de-novo backbone generation; joint sequence–structure pretraining and compatibility between PLL and SLL remain unevaluated.Source: §5 · p. 14
  • The PLL comparison omits continuous-target generation and similarly resourced ESM-2 fine-tuning or distillation alternatives.Source: §5 · p. 14
  • Extending SLL to multichain complexes would require modeling relative chain poses, which this work does not evaluate.Source: §5 · p. 14

6. Conclusion

The paper presents PLL and SLL as learned target vocabularies for decoder-only transformers, reporting improved sequence-model scaling and low-complexity behavior, plus more predictable SLL targets. The authors describe these findings as early evidence for learned latent protein languages as a promising direction.

6. Conclusion

  • PLL and SLL provide learned sequence- and structure-side target vocabularies for standard decoder-only transformers.Both languages remain decodable to amino acids or backbone coordinates.Source: §6 · p. 14
  • PLLM shows steeper fitted compute-scaling trends and substantially less low-complexity drift than amino acid modeling, while SLL improves autoregressive target predictability.The conclusion describes the evidence as early and the direction as promising for autoregressive protein sequence and structure generation.Source: §6 · p. 14

A.1. PLL Tokenizer Details

  • PLL training first learns a quantized latent autoencoder, then freezes its encoder and codebook while training a decoder for residue recovery.The Stage 1 codebook has 4096 codes; Stage 2 uses fixed quantized vectors and a 21-class residue head.Source: §A.1 · pp. 21–22

A.2. PLL Metrics

The appendix specifies how PLL and SLL tokenizer quality is evaluated, how autoregressive streams and candidate selection are handled, and how compute-scaling exponents are estimated. It also states that the compute comparison excludes representation-building costs.

A.2. PLL Metrics

  • PLL evaluation separates Stage 1 reconstruction of ESM-2 representations from Stage 2 amino-acid recovery, with recovery reported over valid residues.The Stage 2 residue vocabulary has 21 classes, including X for unknown or non-standard residues.Source: §A.2 · pp. 22–23

A.3. SLL Tokenizer Details

  • SLL adds sequence-identity, pLDDT-confidence, and ESM-2 representation supervision during tokenizer training, while its fixed decoder reconstructs backbone coordinates.The reconstruction objective includes aligned-coordinate and pairwise-distance terms; auxiliary heads are removed from the autoregressive generation interface.Source: §A.3 · pp. 23–24

A.4. SLL Metrics

  • SLL reconstruction is evaluated with aligned TM-score and RMSD: higher TM-score indicates greater fold similarity, while lower RMSD indicates less coordinate error.The metrics follow the PST protocol from StructTokenBench [54].Source: §A.4 · pp. 24–25

A.5. Autoregressive Tokenization Protocol

  • Autoregressive streams contribute one content token per valid residue, while padding targets and condition-prefix targets are excluded from training loss.For sequence-to-structure, the sequence is encoded outside the causal stream and the decoder predicts SLL tokens autoregressively.Source: §A.5 · p. 25

A.6. Confidence-Based Candidate Selection

  • Offline sequence-to-structure selection scores candidate streams by internal token confidence and can choose a token-space medoid; oracle best-of-k uses native structure and is unavailable at inference.The selected stream is decoded to coordinates by the fixed SLL decoder.Source: §A.6 · pp. 26–27

A.7. Scaling-Law Estimation

  • Scaling fits use lower-loss frontiers under a zero-offset power law; compute excludes representation-building costs, so total resources are not matched.The estimates cover downstream autoregressive training and exclude encoder pretraining, tokenizer learning, and tokenized-corpus preprocessing.Source: §A.7 · p. 27

A.8. Sequence-Side De Novo Evaluation

Sequence-side generation is assessed after decoding both models into amino acid sequences, using composition entropy before structural evaluation. ProteinBench then evaluates quality, diversity, and novelty, with fixed-seed subsets and no uncertainty estimates across independent repeats.

A.8. Sequence-Side De Novo Evaluation

  • Both models are compared in decoded residue space, with the same length-token protocol for length-controlled generation.PLLM code IDs are decoded with the fixed PLL decoder before evaluation.Source: §A.8 · p. 28
  • The low-entropy filter retains sequences with Hseq(x) > 1.5 bits, but is a fairness control rather than a biological-correctness label.The 1.5-bit drift threshold is a heuristic, and composition entropy does not measure residue order, structural quality, or function.Source: §A.8 · p. 28
  • Sequence quality is mean ESMFold-predicted pLDDT, which measures predicted local confidence rather than experimental validation.Generated sequences have no original structure for comparison, so sequence-level evaluation omits scTM and scRMSD.Source: §A.8 · pp. 28–29
  • Diversity uses pairwise TM and Max Clust., with lower pairwise TM and higher Max Clust. indicating greater diversity.Pairwise TM is calculated over unordered non-self pairs; Max Clust. is the number of Foldseek clusters divided by sample count.Source: §A.8 · p. 29
  • Novelty is measured by mean best TM-score to PDB references, where lower Max TM indicates greater novelty.Main-table means give equal weight to four length rows and report fixed-seed evaluation sets without independent-repeat uncertainty estimates.Source: §A.8 · pp. 29–30

A.9. Structure-Side De Novo Evaluation

Structure-side generation evaluates SLLM after its fixed decoder converts sampled SLL codes into backbone coordinates. Decoded backbones are assessed for refolding quality, diversity, and novelty under a length-grouped ProteinBench protocol.

A.9. Structure-Side De Novo Evaluation

  • SLLM samples SLL token sequences that its fixed decoder converts into backbone coordinates for structure-design ProteinBench evaluation.The largest SLLM receives two further epochs with length prefixes on half of samples for length-conditioned generation.Source: §A.9 · p. 30
  • Quality evaluation tests 100 backbones per target length using ProteinMPNN-designed sequences refolded by ESMFold, retaining maximum scTM and independently minimum scRMSD.Those two extrema may come from different designs; higher scTM and lower scRMSD indicate refolding back to the generated structure.Source: §A.9 · pp. 30–31
  • Diversity uses nearest-neighbor TM and Max Clust. on maximum-scTM refolds; lower nearest-neighbor TM and higher Max Clust. indicate greater diversity.The measure uses the N = 100 selected refolds.Source: §A.9 · p. 31
  • Novelty uses mean best TM similarity to PDB references, and the main comparison equally weights lengths 50, 100, 300, and 500.PLAID outputs are clipped to target lengths before evaluation; DPLM-2 uses its 650M unconditional backbone-generation mode.Source: §A.9 · p. 31

B. Data Sources

The paper draws sequence and structure training data from UniRef50 and AFDB-derived corpora, with validation splits specified for each. It also reports UniRef50 reference statistics and qualitative per-residue PLL recovery.

B. Data Sources

  • UniRef50 supplies PLL learning and sequence-generation data, while AFDB representative structures train SLL and AFDB∩UniRef50 structures train SLLM generation.Starting from UniRef50 clusters limits sequence redundancy at 50% identity; the generation corpus suits SLLM because inverse-folding supervision retains sequence-identity information.Source: B. Data Sources · p. 33, Table 15 · p. 33
  • Held-out validation contains 10,000 UniRef50 sequences, 10,000 AFDB representative structures, and 5,000 AFDB∩UniRef50 structure-token pairs.SLL reconstruction uses GCP-VQVAE benchmark splits and reporting, while representation selection follows the StructTokenBench PST protocol.Source: B. Data Sources · p. 33
  • Figure 8 provides UniRef50 reference length and amino acid composition distributions after sequences are truncated to 1,022 residues.These are the reference statistics used to analyze de novo amino acid and PLLM generation.Source: Figure 8 · p. 33, B. Data Sources · p. 33

C.1. PLL Per-Amino-Acid Recovery

  • The Stage 2 PLL residue head achieves high recovery across standard amino acids, while the X class is lower because it groups unknown or non-standard residues.Table 16 reports validation accuracy for each residue class.Source: §C.1 · p. 33, Table 16 · p. 34

C.2.1. SLL Supervised PST Ablation

The SLL experiments test tokenizer supervision, reconstruction, representation, and sequence-to-structure predictability, while the generation analysis compares PLLM entropy and length patterns. Under matched sequence-to-structure training, the adapted tokenizer has lower validation perplexity than the original Lite tokenizer.

C.2.1. SLL Supervised PST Ablation

  • The selected SLL recipe combines MSE plus distance reconstruction, inverse folding at weight 0.1, pLDDT pre-decoder, ESM post-decoder, and the rotation trick.The ablation compares auxiliary-head placement and weights, reconstruction losses, and vector-quantization choices against the Base GCP-VQVAE Lite recipe.Source: §C.2.1 · p. 34

C.2.2. SLL Supervised PST Full-Run

  • The supervised PST full-run comparison uses the PST-standard two-layer MLP for non-GCP baselines and four-layer MLP and transformer probes for GCP-VQVAE variants.GCP-VQVAE Lite 2 denotes SLL in the comparison.Source: §C.2.2 · p. 34, Table 18 · p. 36

C.2.3. SLL Reconstruction Comparison

  • The full SLL reconstruction comparison is reported in Table 19, with baseline values following the original work’s reconstruction table.Source: §C.2.3 · p. 34

C.2.4. SLL Unsupervised PST Full-Run

  • Table 20 reports unsupervised PST scores for GCP-VQVAE Lite 2 and original GCP-VQVAE tokenizers as secondary SLL representation diagnostics.Source: §C.2.4 · p. 34

C.2.5. SLL Sequence-to-Structure Compatibility

  • 22.16 final validation perplexity for GCP-VQVAE Lite 2 remains below the original Lite tokenizer’s best 22.68, with Lite 2 lower at every epoch.Both runs reach their best validation perplexity at epoch four and then overfit.Source: §C.2.5 · p. 35

C.3.1. PLLM Unconditional Generation

  • Across 12 sampling temperatures, 19.9% of PLLM samples versus 43.0% of amino acid samples fall below the heuristic 1.5-bit entropy threshold.At T = 1.0, mean sequence entropy is 3.927 for PLLM and 3.331 for the amino acid model, compared with a UniRef50 mean of 3.952.Source: §C.3.1 · p. 35
  • At T = 1.0, PLLM’s mean length is 277.1 residues, near UniRef50’s 273.6, while the amino acid model averages 609.7 with 41.7% near the context cap.At T = 2.0, amino acid entropy improves, but median length returns to the cap and 58.5% of samples are capped.Source: §C.3.1 · p. 36

C.3.2. PLLM Conditional Generation

Length conditioning largely controls requested output size for both amino acid generation and PLLM, but amino acid generations retain lower entropy. A partial length-conditioning curriculum also lowers unconditional SLLM validation loss.

C.3.2. PLLM Conditional Generation

  • The fixed-length comparison uses conditional generation at target lengths 100, 200, 300, and 500, avoiding post hoc cutting that could create incomplete samples or hide stopping-policy failures.Each model contributes 8,000 samples across 10 temperatures; the length token sets intended content length, but the model must still emit <EOS>.Source: §C.3.2 · p. 36
  • PLLM reaches the exact requested length in 95.71% of samples and is within five residues in 96.98%, while amino acid generation reaches the requested length in 99.99%.Despite this length control, amino acid generation retains substantially lower entropy; PLLM approaches the UniRef50 entropy reference at moderate to high temperatures across target lengths.Source: §C.3.2 · p. 37
  • Without removing drifted sequences, amino acid generation keeps a stronger positive entropy-pLDDT correlation across temperatures, while PLL weakens toward zero and becomes negative at higher temperatures.The controlled comparison retains non-drifted sequences with Hseq(x) > 1.5 bits, folds them with ESMFold, and matches retained sample counts across modalities.Source: §C.3.2 · pp. 37–38
  • After matching entropy-filtered folded sets, amino acid generation retains a stronger positive entropy-pLDDT correlation, while PLL is near zero or negative around T = 0.8 to T = 1.0.The authors interpret this as pLDDT being more entangled with escape from low-complexity residue patterns in the amino acid model.Source: §C.3.2 · p. 38

C.3.3. SLLM Length-Conditioned Continuation

  • After two continuation epochs with length tokens prepended to 50% of training samples, unconditional validation loss falls from 3.891 to 3.670, a 5.70% reduction.The continued checkpoint also supports length-conditioned generation with nearly exact target-length control.Source: §C.3.3 · p. 38

C.3.4. SLLM Sequence-to-Structure Model Selection

A DeepConf-style selector using SLL-token confidence improves average TM-score over single-sample generation on both its tuning set and a held-out verification set. Latent-token generation is fast, with long-protein candidate generation approximately 1,000 times faster than MSA-based AlphaFold2 in these measurements.

C.3.4. SLLM Sequence-to-Structure Model Selection

  • The selector is tuned on 118 CASP14/15/16 targets with 64 candidates each, then verified on 574 CAMEO-2024 targets with 256 candidates each.Both runs use temperature 1.0 and top-p threshold 0.90.Source: §C.3.4 · p. 38
  • The selector scores candidates by maximum mean raw full-vocabulary entropy over sliding windows of 256 SLL tokens, retains the most confident 50%, then selects their token-similarity medoid.Lower scores mean higher internal confidence; at least two candidates must be retained.Source: §C.3.4 · p. 38
  • With KV caching, FA2, and model compilation, the sequence-to-structure model generates about 1,400 tokens per second on two NVIDIA RTX 6000 Ada GPUs, or about three completed samples per second on this workload.Longer targets mainly add more generated tokens.Source: §C.3.4 · p. 38
  • Candidate-token generation for long proteins is approximately 1,000 times faster than MSA-based AlphaFold2, and cached autoregressive decoding scales close to linearly with sequence length.The fixed SLL decoder adds negligible overhead after token generation.Source: §C.3.4 · p. 39
  • At k = 64 on CASP14/15/16, DeepConf raises average TM-score from 0.564 for a single sample to 0.589; oracle best@64 reaches 0.667.Oracle best@k uses the native structure and is a ceiling on the sampled candidate pool, not an inference method.Source: §C.3.4 · p. 39
  • At k = 256 on CAMEO-2024, the fixed selector raises average TM-score from 0.725 for a single sample to 0.754; oracle best@256 reaches 0.825.The selector was tuned on CASP14/15/16 and then fixed for CAMEO-2024 verification.Source: §C.3.4 · pp. 38–39

Related papers

Loading 2610.03978v1…