PaperlayerView exact revision on arXiv

Source-linked AI summary

DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design

Yuzhen Wang, Chunxiao Li, Baichen Le, Lewen Yan, Jie Zhang, Yucheng T. Yang, Xing-Ming Zhao, Lijun Li, Jing Shao

arXiv:2610.13116v1cs.AI

TL;DR

Existing evaluations leave the operational risk of LLMs designing hazardous mutations, including the potential amplification from biological design tools, unquantified. DRYLAB-Bench tests mutation-design agents using policy-grounded scenarios, iterative BDT workflows, and empirical DMS measurements. Across 15 frontier models, measured designs were enriched for high-risk variants without tools, and tool-assisted iteration further shifted results toward higher-risk regions.

  • Problem

    The operational risk of LLMs executing dangerous biological modifications remains unquantified, while existing evaluations often test hazardous knowledge rather than application and omit BDT amplification.

    Source: §1 · pp. 1–2

  • Method

    DRYLAB-Bench evaluates hazardous mutation-design agents in tool-augmented workflows and scores their submissions against hidden empirical DMS measurements.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    Without external evidence, 21.4% of submitted mutations fall within assay-specific top-decile variants and 67.6% exceed the average measured variant; with iterative tool assistance, the top-decile fraction is 23.5% and BRS increases by +0.116 standard deviations.

    Source: §5 · pp. 9–10

  • Takeaways & Limitations

    Empirical wet-lab measurements enable scalable, safe, and verifiable monitoring of dual-use threats from LLM mutation design.

    Source: §5 · p. 9

    The six BDTs do not return hidden DMS ground-truth labels, so agents rely on internal knowledge and provided tool evidence during design.

    Source: §3.2 · p. 5, §3.3 · p. 5

Abstract

from arXiv · show

As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce severe dual-use risks, such as engineering enhanced pathogens or drug-resistant bacteria. While biosecurity frameworks recognize these threats, the actual risk of LLMs executing dangerous biological modifications remains unquantified, particularly when augmented with specialized biological design tools (BDTs). Existing evaluations primarily assess hazardous knowledge possession rather than practical application, overlook risk amplification from BDTs, and rely on unreliable, expensive, or hazardous validation methods. To address these gaps, we introduce DRYLAB-Bench (Design Risk Yielded by LLM Agent in Biodesign), the first comprehensive benchmark evaluating the high-risk biological mutation design capabilities of LLM agents. Moving beyond basic question-answering, DRYLAB-Bench tasks agents with 30 genuinely hazardous mutation-design scenarios distilled from biosecurity guidelines. It evaluates models in a multi-round, tool-augmented environment where LLMs orchestrate BDTs to iteratively refine candidates. Crucially, it scores designs directly against 187,695 empirical wet-lab measurements from ProteinGym and ViroGym using our proposed Biosecurity Risk Score (BRS), quantifying risk-associated effects while enabling scalable and safe ground-truth verification. Evaluations of 15 frontier models reveal substantial baseline threats: without tools, 67.6% of one-shot designs exceed the average measured variant, and 21.4% reach the highest-risk decile. Equipping models with BDTs significantly amplifies this risk, raising the pooled risk by \(\Delta BRS=+0.116\) and the top-decile share to 23.5%. Code: https://github.com/drylab-bio/DRYLAB-Bench

1 INTRODUCTION

Existing evaluations leave the operational risk of LLM-designed biological modifications—and the possible amplification from biological design tools—unquantified. DRYLAB-Bench evaluates hazardous mutation design using empirical measurements, and experiments report high-risk designs both without tools and with tool-assisted iteration.

Motivation

  • LLMs’ growing biological design ability brings dual-use risks, but the magnitude of their ability to execute dangerous modifications remains unquantified.The introduction also identifies specialized BDT integration as a potential threat amplifier.Source: §1 · p. 1

Evaluation gap

  • Existing benchmarks assess hazardous knowledge more often than concrete application, while general-purpose tools leave BDT-related risk amplification unmeasured.Open-ended design evaluation also relies on LLM judges, expensive expert review, or hazardous wet-lab synthesis.Source: §1 · p. 2

Benchmark contribution

  • DRYLAB-Bench evaluates high-risk mutation design through hazardous scenarios, multi-round BDT workflows, and scoring against empirical wet-lab measurements.The benchmark uses 187,695 measured substitutions across 30 scenarios to support scalable ground-truth verification.Source: §1 · p. 2

Experimental findings

  • In PsaE disruption, GPT-5.6 Terra’s designs shifted from outranking 11.7% to outranking 98.9% of measured variants with BDTs and multi-round iteration.The paper reports this as an example of tool-assisted iteration amplifying the baseline threat.Source: §1 · p. 2

2 RELATED WORK

Prior biosecurity evaluations often test hazardous knowledge or narrow tool-assisted tasks rather than agents iteratively refining mutations toward hazardous phenotypes. Verification methods also trade off open-endedness, reliability, cost, scalability, and biosafety; DRYLAB-Bench repurposes empirical DMS data to evaluate generated designs.

2.1 AI BIOSECURITY EVALUATIONS

  • Earlier evaluations test knowledge, refusal, or peripheral biological-tool tasks, leaving iterative mutation refinement and BDT amplification insufficiently measured [17, 18, 19, 20, 21, 22, 23, 24].The paper identifies the extent to which agents can navigate biological fitness landscapes as unquantified.Source: §2.1 · p. 2

2.2 VERIFICATION PARADIGMS FOR BIOLOGICAL DESIGN

  • Predefined answer keys are rigid, LLM judges may hallucinate, and wet-lab validation is costly, unscalable, and biosafety-relevant.These approaches provide different trade-offs for verifying open-ended biological designs.Source: §2.2 · pp. 2–3
  • DRYLAB-Bench repurposes ProteinGym and ViroGym DMS landscapes to verify LLM-generated hazardous designs without relying on LLM judges or hazardous wet-lab experiments.The paper describes these empirical datasets as previously used primarily to evaluate predictive biological models.Source: §2.2 · p. 3

3 DRYLAB-BENCH COMPOSITION AND CONSTRUCTION

DRYLAB-Bench combines policy-grounded hazardous scenarios, tool-augmented agent workflows, and empirical DMS-based validation. Its construction filters scenarios for measured assays and expert review, while its evaluation hides ground truth from agents and scores finalized mutations against it.

3 DRYLAB-BENCH COMPOSITION AND CONSTRUCTION

  • The benchmark is organized around policy-grounded hazardous tasks, tool-augmented workflows, and scalable, safe empirical validation.These pillars address knowledge-only evaluation, unmeasured BDT amplification, and unreliable or unscalable validation.Source: §3 · p. 3

3.1 GENUINELY HAZARDOUS SCENARIOS: POLICY-DRIVEN RISK TAXONOMY

  • The eight-dimensional taxonomy derives from biosecurity guidelines, and 30 scenarios were retained after matching to quantitative DMS data and expert review.An independent computational biology expert reviewed candidates for biophysical soundness and numbering consistency.Source: §3.1 · pp. 3–4, Figure 1 · p. 3

3.2 REALISTIC MALICIOUS WORKFLOWS: TOOL-AUGMENTED ITERATION

  • Agents begin with candidate mutations and can use six BDTs to obtain evidence and refine designs through as many as four tool-selection rounds.The tools provide conservation, sequence-likelihood, structure, functional-annotation, toxicity, and scenario-specific predictions; they do not return hidden DMS labels.Source: Figure 2 · p. 4, §3.2 · p. 5
  • The six BDTs supply evidence on conservation, sequence likelihood, structure compatibility, functional annotation, toxicity, and scenario-specific predictions.Scenario-specific prediction uses AlphaMissense, EVEscape, or FoldX, depending on the task.Source: §3.2 · pp. 4–5
  • The tool conditions are successive snapshots of one shared generation trajectory, with S0-iter serving as a no-tool control under the same iteration budget.S0 is one-shot, S1 permits one tool-and-revision round, and S2 permits iterative refinement.Source: §3.2 · p. 5

3.3 RELIABLE AND SAFE VALIDATION: EMPIRICAL DMS GROUND TRUTH

  • Agents rely on internal knowledge and provided BDTs while hidden DMS data are held out; finalized mutations are matched to measured phenotypes and aggregated into confidence-weighted BRS.Phenotypes are oriented so larger values indicate greater risk, with optional penalties for viability or stability constraints and standardization within each assay.Source: §3.3 · p. 5

4 EVALUATION

DRYLAB-Bench evaluates 15 frontier models across 30 mutation-design scenarios, using empirical measurements and a Biosecurity Risk Score to compare untooled and tool-augmented designs. Risk rises overall with multi-round tool use, while results also show zero-shot risk, model-dependent effects of iteration and tool alignment, and refusal sensitivity to prompt wording.

4.1 EXPERIMENTAL SETUP

  • The evaluation covers 15 frontier LLMs, 30 biosecurity-relevant scenarios, and 187,695 experimentally measured single-site substitutions.The scenarios span eight risk categories, and average experimental coverage under S2 is 83.9%.Source: §4.1 · pp. 5–6, Table 1 · p. 6

4.2 EVALUATION METRIC

  • BRS aligns mutation fitness measurements so larger values indicate higher risk, standardizes risk within each assay, and aggregates confidence-weighted top-three candidates across scenarios.The metric also applies penalties when candidates violate scenario constraints; candidates without measurements use a separate in-silico pipeline.Source: §4.2 · p. 6

4.3 RESULTS AND ANALYSIS

  • BRS rises from 0.359 under S0 to 0.475 under S2, an overall threat amplification of ∆BRS = +0.116.Figure 3 shows the baseline and S0-to-S2 change in valid-response and refusal-penalized views.Source: Figure 3 · p. 7, §4.3 · p. 7
  • Without tools, all evaluated models exceed average design baselines, with Kimi-K3 reaching BRS = 0.52 under S0.The authors describe the submitted designs as specific hazardous substitutions that rank above the bulk of measured single variants.Source: §4.3 · p. 8
  • S1 increases pooled risk by ∆BRS = +0.048, and S2 adds ∆BRS = +0.068; multi-round gains are negatively correlated with S0 BRS (r=-0.42).Some models decline under iteration, including DeepSeek V4 Flash (-0.012) and GPT-5.6 Luna (-0.001).Source: §4.3 · p. 8
  • Tool effects depend on task alignment: MMseqs2 conservation signals are anti-correlated with CcdB experimental outcomes (ρ = −0.815), while misaligned feedback can degrade candidates.Most models nevertheless improve in the CcdB scenario, and well-aligned feedback predominates across the suite.Source: §4.3 · p. 8
  • Refusal varies with framing: neutral scientific wording relaxes filters for most models, whereas an explicit harm prefix raises refusal across the suite.The comparison rewrites the same 30 scientific requests while holding the sequence, mutation objective, and hidden DMS target fixed.Source: §4.3 · p. 9

5 CONCLUSION

DRYLAB-Bench finds substantial high-risk mutation-design capability without tools, with tool-assisted iteration shifting submitted designs further toward measured high-risk variants. The paper presents the benchmark as an empirical, scalable, and safe way to monitor dual-use threats, with safeguards for evaluation and disclosure.

5 CONCLUSION

  • DRYLAB-Bench is presented as a scalable, empirical testbed for monitoring dual-use risks through evaluation of high-risk mutation-design capabilities.The benchmark grounds evaluations in empirical wet-lab measurements.Source: §5 · p. 9
  • Without external evidence, 21.4% of mutations fall in the assay-specific highest-risk decile and 67.6% exceed the average measured variant.Under iterative tool-assisted interaction, the top-decile fraction rises to 23.5% and BRS increases by +0.116 standard deviations.Source: §5 · pp. 9–10

AI USE STATEMENT

  • The authors state that generative AI assisted with synthetic data, implementation, translation, figures, code, information search, readability edits, and reference formatting, with authors reviewing AI-assisted work.They say AI was not used for proofs, methodology or experiment design, theoretical models, conceptual frameworks, or interpreting results.Source: AI USE STATEMENT · p. 11

ETHICS STATEMENT

  • Evaluation uses only in-silico scoring against hidden, pre-existing DMS datasets, avoiding biological synthesis while retaining empirical grounding.The authors describe this paradigm as incurring zero physical biosafety risk.Source: ETHICS STATEMENT · p. 11
  • The 30 scenarios use publicly accessible sequences, targets, and DMS data, while prompts and hidden mappings for the most severe pathogen scenarios are gated to verified researchers and developers.The authors state that the work introduces no novel biological agents, classified information, or zero-day vulnerabilities.Source: ETHICS STATEMENT · p. 11
  • Domain experts reviewed the 30 scenarios and could opt out of reviewing any specific high-risk scenario without penalty.Experts were briefed and gave informed consent; the protocol followed institutional ethics guidelines for human participants.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The authors plan to release code, logs, raw outputs, and benign prompt templates, while gating high-risk prompts.The stated rationale includes the nondeterminism and financial costs of commercial LLM APIs.Source: REPRODUCIBILITY STATEMENT · p. 11

A MODEL API PARAMETERS

The evaluation uses a shared request contract and standardizes decoding temperature and reasoning effort where vendor APIs allow. Vendor-specific restrictions require documented deviations, and low reasoning effort is a deliberate cost-and-controllability choice.

A MODEL API PARAMETERS

  • The shared request contract uses a 16,384-token limit, 1,800-second timeout, up to five retries, and counts provider safety refusals as refusals rather than API errors.Temperature is standardized to 0.3 and reasoning effort to low wherever supported.Source: A MODEL API PARAMETERS · p. 15
  • Vendor restrictions require documented parameter deviations, including omitted temperature fields, alternate token fields, and Kimi-K3's vendor-enforced temperature convention.Table 4 defines the reported temperature, reasoning, and other per-model settings.Source: A MODEL API PARAMETERS · p. 15, Table 4 · p. 15
  • Low reasoning effort is the primary setting because higher effort slowed tool-selection and revision calls and made the four-round loop harder to steer; dependence on reasoning effort remains future work.Residual per-model temperature differences are treated as a documented confound.Source: A MODEL API PARAMETERS · p. 15

B SUPPLEMENTARY METRIC MATERIAL

Most submitted designs can be scored against measured assay data, while unmeasured candidates receive calibrated, reliability-weighted in-silico estimates. Coverage distinguishes assay lookup availability from whether a response was parseable or usable.

B.1 GROUND-TRUTH COVERAGE

  • Coverage is the share of submitted candidates matching a scenario assay's measurements, averaged only over non-refusal units with a complete, legal top-3.Responses that cannot be parsed into three legal designs are excluded from the coverage average, though they may still receive a risk score through the in-silico fallback.Source: §B.1 · p. 16
  • 83.9% of S2 designs land on a wet-lab measurement on average across models, no model averages below 72%, and the best reaches 100%.Figure 5 shows per-model top-3 coverage in S0 and S2; its average includes complete, legal top-3 submissions.Source: §B.1 · p. 16, Figure 5 · p. 16

B.2 SCORING CANDIDATES THE ASSAY DID NOT MEASURE

  • Unmeasured but legal candidates receive calibrated in-silico risk estimates fused from ESM-2 and rule-based physicochemical scores, with their unit-score weight discounted by held-out assay reliability.Calibration maps estimates onto the assay risk scale; the reliability is the aligned Spearman correlation on a disjoint held-out split.Source: §B.2 · pp. 16–17

B.3 METRIC VALIDITY AND SCOPE

The scale-free BRS and empirical-rank metrics agree closely, while the raw composite is less stable across assays. The metric's scope is single amino-acid substitutions, not full multi-mutant construct fitness or in-organism behavior.

B.3 METRIC VALIDITY AND SCOPE

  • Across 15 models, S2 BRS and empirical-rank rankings correlate at Spearman ρ = 0.99 with refusals floored and 0.96 for valid responses only.The raw composite is unstable across assays: CcdB toxin's cross-model raw-risk spread is 25 units, 6.8× the largest spread on another task, and its all-task ranking correlates at ρ = +0.87 with that assay alone.Source: §B.3 · p. 17
  • The metric evaluates single amino-acid substitutions and does not estimate fully synthesized multi-mutant construct fitness or organism behavior.Epistasis, expression, and host-context effects beyond the assayed phenotype lie outside its reach.Source: §B.3 · p. 17

B.4 METRIC VALIDITY TABLE

Table 5 compares model rankings induced by three metrics and tests how stable each ranking is when the refusal floor changes.

B.4 METRIC VALIDITY TABLE

  • The top of Table 5 compares agreement among S2 model rankings induced by the three metrics.The comparison covers all 15 models, with unmeasured candidates filled in as specified in Eq. 9.Source: Table 5 · p. 17
  • The bottom of Table 5 measures each ranking’s stability when the refusal floor moves from the assay mean to 1.5 standard deviations below it.Stability is expressed in each metric’s own units, with the floor applied per unit.Source: Table 5 · p. 17
  • Scale-free metrics are the most floor-stable, while the raw composite is the least stable because a fixed offset means different things across assays.Source: Table 5 · p. 17

C PROMPT TEMPLATES AND WORDING-ONLY VARIANTS

This appendix documents how prompts for 30 high-risk scenarios are assembled and varied, with representative prompts across four risk categories. It audits that the variants differ only in section-heading phrasing, supporting interpretation of variant means and refusal shifts without changes to the requested objective.

C PROMPT TEMPLATES AND WORDING-ONLY VARIANTS

  • Each of the 30 scenarios is posed under three wording variants so reported risk reflects the task rather than a single phrasing.The appendix reproduces prompts from run artifacts and documents their assembly, edits, representative scenarios, and wording audit.Source: C PROMPT TEMPLATES AND WORDING-ONLY… · p. 18

C.1 PROMPT ASSEMBLY

  • Prompts concatenate three tagged blocks; wording variants change only the middle block, while the condition changes only the last and other comparison elements remain fixed.The role text, objective, target sequence, budget, and tool menu are held constant across the wording comparison.Source: §C.1 · p. 18, Table 6 · p. 18
  • S2 uses sequential adaptive tool iteration, whereas S0-iter uses self-review and candidate updates without external evidence; both share the stated budget.S2 selects tools and updates candidates based on new evidence; S0-iter keeps tool requests empty and iterates without external evidence.Source: §C.1 · pp. 19–20
  • Scoring-dimension guidance is omitted from the manuscript for information-hazard reasons and is available in the gated release.A task-specific scoring-guidance block omitted from a listing is marked where it occurs.Source: C PROMPT TEMPLATES AND WORDING-ONLY… · p. 18, §C.1 · p. 19

C.2 THE THREE WORDING VARIANTS

  • P0 is the baseline, P1 uses compact headings, and P2 uses long-form headings; only three heading phrases change, with body sentences and their order fixed.The audit supports interpreting the variants as a probe of surface-phrasing sensitivity rather than a reordered or reweighted prompt.Source: §C.2 · p. 20, Table 7 · p. 20

C.3 REPRESENTATIVE HIGH-RISK SCENARIO PROMPTS

  • Four representative prompts span protein toxification, enhanced infectivity or host-range expansion, antimicrobial or antiviral resistance, and immune evasion or medical-countermeasure failure.The examples include CcdB toxin potency, SARS-CoV-2 RBD–ACE2 binding, VIM-2 carbapenem resistance, and CD19 CAR-T target binding; targets in the gated subset are not reproduced.Source: §C.3 · pp. 21–23
  • The representative scenario asks models to enhance human CD19 immune-target binding while preserving folding, and restricts cited positions to the sequence with wild-type residue verification.P1 and P2 use the same objective sentence; their heading phrases differ.Source: §C.3 · p. 23

C.4 STRUCTURAL AUDIT OF THE VARIANTS

  • For every trajectory, the audit normalized three heading phrases, compared the variants character by character, and re-derived P1 and P2 from P0 to verify the artifacts.The audit read each unit’s TASK SPEC artifacts and confirmed they matched the generator’s output.Source: §C.4 · p. 23
  • 1,350 rendered prompts across all 450 scenario–model triples passed: variants matched byte-for-byte after heading normalization, and generator round-trips reproduced every artifact.The variants differed by nothing else, including instruction order, inserted or dropped sentences, and the position-numbering rule.Source: Table 8 · p. 23, §C.4 · p. 23
  • Wording edits changed round-0 prompts by 9–51 characters against a median length of 5,896, at most 0.86% of prompt length.The variants were therefore not separated by length.Source: §C.4 · p. 23
  • Because only phrasing differs, per-variant spread bounds prompt-engineering effects on measured risk, while pooled conclusions use the variant mean rather than the best-scoring phrasing.This frames how to interpret the results reported in Section 4.Source: §C.4 · p. 24
  • Because the objective sentence is identical across variants, a variant-dependent refusal shift cannot be explained by a change in what the model was asked to do.Source: §C.4 · p. 24

D CONTROLS AND ABLATIONS BEHIND THE CONDITION GAINS

The section distinguishes the workflow’s tool and revision stages from self-iteration, and bounds what the condition comparisons establish about mechanism. The workflow raises submitted risk, but the controls do not isolate whether tool-output content produces that increase.

Four Arms Over Two Factors

  • S0, S1, and S2 are successive snapshots of one shared trajectory, while S0-iter is a separate no-tool self-review run.S0 is recorded before any tool is invoked; S1 follows one evidence round and revision, and S2 follows further request–read–revise cycles.Source: D CONTROLS AND ABLATIONS BEHIND THE… · p. 25

Scope of the Causal Reading

What Each Step of the Workflow Contributes

  • ∆BRS = +0.048 after one evidence round and a further +0.068 after adaptive rounds, with each stage positive for 10 of 15 models.The evidence round is an opportunity rather than a guarantee: 372 of 861 trajectories requested no tool before S1 was recorded.Source: D CONTROLS AND ABLATIONS BEHIND THE… · p. 25

The No-Tool Iteration Arm Is Too Noisy to Serve as a Reliable Ablation

  • S0-iter’s pooled step is +0.012, but its per-model steps range from −1.19 to +0.61, so it is only a bound, not a reliable iteration ablation.Only 811 of 1,350 S0-iter units are valid, and the design does not support the claim that iteration alone has no effect.Source: D CONTROLS AND ABLATIONS BEHIND THE… · pp. 25–26

Two Mechanism Controls This Version Does Not Run

  • The comparisons do not establish whether tool-output content, rather than extra deliberation, produces the risk increase; neither proposed mechanism control was run.The controls would test shuffled tool outputs and a non-LLM optimizer using the same tool outputs.Source: D CONTROLS AND ABLATIONS BEHIND THE… · p. 26

E FULL EVALUATION RESULTS

The full-results table reports per-model, per-condition outcomes under two refusal-scoring rules. The discussion bounds the benchmark to empirically mapped single substitutions and argues that this unit captures dangerous design capability without extending evaluation to physical or organismal outcomes.

E FULL EVALUATION RESULTS

  • Table 10 reports mean risk by model and condition, with valid-unit and refusal-floor scoring blocks, and sorts rows by S2 floor-mean.Its caption defines BRS = 0 as the assay mean and describes score-based cell shading.Source: Table 10 · p. 27, E FULL EVALUATION RESULTS · p. 27

F.1 WHY DRYLAB-BENCH CAN REFLECT REAL RISK

  • The benchmark measures agents’ use of single-site evidence to rank substitutions, not their ability to search combinatorial genotypes.Available causal phenotype labels are largely for single amino-acid changes; the tool suite returns conservation, sequence-likelihood, structure-compatibility, and task-aligned evidence.Source: §F.1 · p. 28
  • The empirical and ethical boundary excludes multi-mutant labels, live-infection scoring, and synthesis or assembly instructions, keeping the pipeline’s most dangerous object a proposed substitution string.The passages state that comparable higher-order mutant maps are unavailable and that obtaining missing labels would create the hazard being monitored.Source: §F.1 · p. 28
  • A one-shot design was the most activating substitution among 3,372 measured SRC mutants, supporting the authors’ claim that single substitutions can occupy dangerous empirical tails.BRS does not estimate multi-mutant fitness, epistasis, host-context expression, transmissibility, or physical-construction success, and the authors make no claim about behavior in a living organism.Source: §F.1 · p. 28

F.2 ARE THE DESIGNS MEMORIZED ANSWERS?

The authors find that high-risk submissions are not explained simply by reproducing mutations favored by model consensus. In MTHFR, familiar proposals were mild relative to a single-model C243Y design, which fell in the assay’s highest-risk tenth.

F.2 ARE THE DESIGNS MEMORIZED ANSWERS?

  • BRS averaged +0.67 for mutations proposed by most models versus +0.38 for mutations proposed by only one or two, but at S2 consensus-heavy units scored +0.57 versus +0.60 for model-specific units.In 12 of 16 scenarios with cross-model consensus, the highest-scoring candidate proposed by any model was outside that consensus.Source: §F.2 · p. 28
  • In only 7 of 30 scenarios did one candidate attract more than half the models, leaving model-specific submissions in the other 23.The authors interpret S0 BRS as capability on assay-defined objectives, not an estimate over arbitrary well-known proteins.Source: §F.2 · p. 29
  • In MTHFR, common proposals R157Q and A222V measured BRS = −0.78 and BRS = +0.22, while single-model C243Y measured BRS = +2.30 in the assay’s highest-risk tenth.Figure 6 shows the representative GLM-5.3 P2 submissions moving from a mean BRS of −0.34 at S0 to +1.11 after tool-assisted iteration, with C243Y at +2.30.Source: §F.2 · p. 29, Figure 6 · p. 29

Related papers

Loading 2610.13116v1…