PaperlayerView exact revision on arXiv

Source-linked AI summary

Artificial intelligence and biosecurity: capabilities, threat pathways, and defense-in-depth governance

Candace S. Y. Chan, Aris Karatzikos, Ilias Georgakopoulos-Soares

arXiv:2609.16213v1cs.AI

TL;DR

Biological AI can benefit public health while also creating potential misuse risks, and the field lacks clear boundaries for prioritizing hazardous capabilities. This review examines biological AI threats and safeguards across the research workflow. It highlights that fine-tuning can bypass data-filtering safeguards, and argues for proportionate governance across the model life cycle.

  • Problem

    Biological AI has dual-use potential, while clear and consistently applied boundaries around risky capabilities remain unsettled.

    Source: §1 · p. 2

  • Method

    The review examines biological AI capabilities and threat pathways alongside safeguards spanning training, evaluation, access, deployment, and monitoring.

    Source: §1 · p. 2, §5 · p. 18, §6 · p. 20

  • Results

    Fine-tuning Evo 2 on data from 110 harmful human-infecting viruses bypassed data-filtering safeguards and produced modest ability to identify SARS-CoV-2 variants with immune evasion [16].

    Source: §1 · p. 2

  • Takeaways & Limitations

    Governance should link predetermined risk thresholds to proportionate measures such as tiered access, restricted deployment, enhanced monitoring, or limits on model-weight release.

    Source: §5 · p. 18

    Refusal training and related alignment techniques transfer poorly to biological foundation models that operate on sequence, structure, or function representations rather than natural-language requests.

    Source: §4.4 · p. 16

Abstract

from arXiv · show

Artificial intelligence is reshaping biological research across an increasingly connected digital-to-physical workflow. General-purpose large language models can retrieve and integrate scientific information, support experimental planning, and computational analysis; biological foundation models can predict, optimize, and generate proteins, genes, and genome-scale sequences; agentic systems can coordinate multistep research tasks; automated laboratories can partially close the design-build-test-learn cycle. These technologies could greatly benefit medicine, public health, and biotechnology. However, their biosecurity risk depends not only on what the AI can do, but also on who uses it, their expertise and intent, their access to laboratory tools and materials, and the safeguards in place. Current evidence shows that AI uplift exists but primarily affects digital rather than physical tasks. Frontier systems have exceeded expert baselines on in-silico, and screening-evasion benchmarks, whereas controlled wet-laboratory studies find that tacit knowledge and physical execution remain substantial barriers. This review describes the different biological threats from AI tool use, from information gathering and biological design to procurement, synthesis, testing, scale-up, and potential release. We further examine why alignment techniques for general-purpose models transfer poorly to biological ones, and the emerging role of interpretability in auditing whether hazardous capabilities are genuinely removed. We argue for defense-in-depth governance that links capability thresholds to proportionate responsibilities across the biological AI ecosystem, reducing high-consequence risk while preserving beneficial use.

1. Introduction

Biological AI connects information retrieval, biological design, and experimental workflows, creating both public-health benefits and potential misuse risks. The review highlights unclear risk boundaries and the limits of data filtering alone as reasons to examine safeguards across the field.

1. Introduction

  • General-purpose and biological models, agents, and automated laboratories support distinct but increasingly connected research tasks, while also enabling potential misuse.The review describes information retrieval and planning, sequence-based prediction and generation, multistep tool use, and robotic experimentation.Source: §1 · p. 2, Figure 1 · p. 3
  • Biological AI has supported public-health research, including pandemic-era prediction of viral fitness and immune escape, but the same capabilities could be misused.The review cites applications to vaccine and treatment development alongside the possibility of biosecurity risks.Source: §1 · p. 2
  • Clear, consistently applied boundaries for risky model capabilities remain unsettled, complicating risk prioritization as biological AI tools proliferate [11, 12].An analysis identified 1,107 tools across 76 countries and found no correlation between misuse-relevant capabilities and tool accessibility [12].Source: §1 · p. 2
  • Fine-tuning bypassed data-filtering safeguards, and the resulting model showed modest ability to identify SARS-CoV-2 variants with immune evasion [16].The study fine-tuned Evo 2 on data from 110 harmful human-infecting viruses; the authors conclude data filtering alone is insufficient.Source: §1 · p. 2
  • The review distinguishes biosecurity risks of deliberate or unauthorized biological harm from biosafety risks of accidental harm in legitimate research.It examines biological AI capabilities and challenges in developing effective safety guidelines.Source: §1 · pp. 2–3

2. Biological AI capabilities

Biological AI can expand access to research tasks and design capabilities, but uplift differs by user, task, and the path from digital design to physical implementation. The section surveys protein and genome design, iterative optimization, DNA synthesis as a control point, and unresolved mirror-biology risks.

2.1 User expertise and AI capability uplift

  • AI may broaden access to pathogen prediction, replication, and automated-lab capabilities, but uplift varies by actor and evaluation setting.Lone actors mainly gain information and troubleshooting support while remaining constrained by expertise and physical execution; organized groups can pool skills and infrastructure, while insiders can bypass safeguards. Written benchmarks, in-silico analyses, and controlled wet-lab workflows should not be directly compared.Source: §2.1 · pp. 4–5
  • 4.16 times more accurate than internet-only controls on eight in-silico task sets, LLM-assisted novices showed greater digital than physical-task uplift.A randomized study of 153 participants found no significant improvement in novices’ ability to complete laboratory tasks modeling parts of a viral reverse-genetics workflow, although modest intermediate gains were observed. Another analysis reports that frontier models can guide recovery of live poliovirus from synthesized DNA.Source: §2.1 · p. 4
  • 43.8% accuracy on specialized laboratory-troubleshooting questions, versus 22.1% among virologists answering within their expertise, was achieved by the Virology Capabilities Test’s leading model.Benchmark performance does not establish end-to-end experimental capability. Agentic scaffolding can raise capability above that of a bare model; tested agents outperformed median expert-human baselines on DNA-fragment design, synthesis-screening evasion, and laboratory-automation tasks, and generated scripts directed a robot to assemble DNA with expected sequences.Source: §2.1 · pp. 4–5

2.2 De novo biological design and the prospect of novel pathogens

  • 3- to 300-fold stronger experimentally measured affinities than earlier design methods across seven target proteins were reported for AlphaProteo-generated binders.Other examples include functional ProGen lysozymes with sequence identities as low as 31% and an ESM3-generated functional green fluorescent protein with 58% identity to its closest known counterpart.Source: §2.2 · pp. 5–6
  • 16 viable phages were experimentally produced from Evo- and Evo 2-generated ΦX174-based genomes, marking a move toward viable biological systems.Evo 2-generated sequences have reported limitations in preserving genomic organization, regulatory architecture, and other features; viable phage genomes were also reported within a narrow evolutionary neighborhood of known sequences.Source: §2.2 · p. 6
  • AI-guided iterative optimization uses experimental feedback to improve sequences, while repurposed optimization can enumerate candidate toxic molecules.No published study has demonstrated end-to-end AI design and experimental validation of a novel human pathogen, although design-to-realization barriers are narrowing. Sequence novelty alone does not establish pathogenicity; evaluations should prioritize capabilities associated with large-scale harm while preserving beneficial research.Source: §2.2 · pp. 5–7

2.3 DNA synthesis: the digital-to-physical bottleneck

  • DNA synthesis is a control point where a digital sequence becomes a physical molecule, but similarity-based screening may miss divergent sequences that retain hazardous functions.The passage identifies sequence-divergent reformulations of known hazardous functions as a near-term concern for existing similarity-based safeguards.Source: §2.3 · p. 7

2.4 Mirror-image biology

  • Mirror-image molecules have potential therapeutic uses, but mirror life remains technically infeasible and current AI predictions are limited by scarce training data.Researchers have synthesized mirror-image nucleic acids, proteins, polymerases, and parts of mirror cellular machinery, but no self-replicating or fully functional mirror organism has been produced. Experts and workshop participants also identify possible future applications and risks as the field advances.Source: §2.4 · pp. 7–8

3. Risk assessment: taxonomy and capability evaluation

This section proposes a context-sensitive heuristic for classifying biological AI risk and discusses capability evaluation, provenance, and downstream verification as complementary safeguards.

3.1 A risk taxonomy for biological AI

  • Risk tiers coordinate governance by combining misuse-relevant capability, accessibility, and potential consequences rather than treating capability alone as decisive.The tiers are a heuristic, not a validated quantitative scoring system.Source: §3.1 · p. 8
  • High-consequence capabilities can bypass safeguards or reduce oversight, while context such as synthesis access, insider use, or autonomy can raise a capability’s risk tier.Protein design may be Moderate in isolation but rise in risk when paired with these factors.Source: Table 1 · p. 8, §3.1 · p. 9

3.2 Capability evaluation, provenance, and downstream verification

  • Pre-release evaluation can test hazardous sequence generation, virulence prediction, screening evasion, and performance after adversarial fine-tuning, but lower benchmark scores do not prove robust capability removal.The section names BioRiskEval, GeneBreaker, WMDP-Bio, Virology Capabilities Test, and ABC-Bench as existing frameworks.Source: §3.2 · p. 9
  • Benchmark results require caution because training contamination, weak construct validity, metric gaming, and shortcut features can distort what measured performance indicates about biological capability.Sequence benchmarks may rely on nucleotide composition, codon usage, or sequence length; language-level refusal also poorly predicted the functional-risk scores of generated protein sequences.Source: §3.2 · pp. 9–10
  • Provenance and downstream verification complement capability evaluation, but watermark robustness, experimental validation, proof costs, and proof scope remain limitations.Zero-knowledge proofs cannot establish that a model entirely lacks hazardous capabilities.Source: §3.2 · p. 10

4. Safeguards against AI-enabled biological threats

This section describes layered safeguards across biological AI models, sequence synthesis, agent workflows, and defensive surveillance. It emphasizes that external controls remain central, while alignment and interpretability provide supporting evidence rather than safety guarantees.

4.1 Access, deployment, and bioagent controls

  • Access controls determine whether downstream safeguards can be enforced, while data filtering and weight locking are supplementary because fine-tuning or informed attacks can recover restricted capabilities.Open weight release makes monitoring, revocation, secure fine-tuning, and output screening unenforceable by the developer.Source: §4.1 · p. 12
  • Bioagents need least-privilege permissions, sandboxed tools, human authorization for safety-critical actions, execution limits, logging, and evaluation of the complete agent-tool configuration.Agent scaffolding can enable multistep tasks involving DNA-fragment design, synthesis-screening evasion, and liquid-handling automation; safeguards should be coordinated across the operating loop.Source: §4.1 · pp. 12–13

4.2 Sequence screening

  • Sequence screening should combine customer verification, sequence and functional screening, aggregation of related fragments and orders, and controls integrated into benchtop synthesizers.AI-designed variants can evade similarity-based screening, while decentralized synthesis can bypass provider-level identity checks and cross-order monitoring.Source: §4.2 · pp. 13–14

4.3 Securing models against compromise

  • Model security requires authenticated data provenance and integrity checks because poisoned training examples can implant backdoors or alter outputs without consistently affecting standard benchmark performance.Risks also arise after training through modified sequences, malicious fine-tuning, compromised software or weights, indirect prompt injection, and data-extraction attacks.Source: §4.3 · p. 16

4.4 Alignment, interpretability, and the limits of internal safeguards

  • Conversational refusal safeguards transfer poorly to biological foundation models, so safety relies heavily on external controls; interpretability can audit suppression versus removal but cannot certify capability absence.The review describes interpretability findings as preliminary and largely correlational, and agentic alignment failure modes as not yet demonstrated in wet-lab incidents.Source: §4.4 · pp. 16–17

4.5 AI-enabled detection and surveillance

  • AI-enabled surveillance can support earlier pathogen warning and faster public-health responses, but alerts should complement laboratory testing, epidemiological investigation, secure data sharing, and expert review.Surveillance remains vulnerable to sampling bias, incomplete reference databases, sequencing artifacts, and uneven coverage.Source: §4.5 · p. 17

5. Governance of high-risk life-sciences research and AI-enabled biological systems

The section proposes risk-proportionate safeguards across the life cycle of AI-enabled biological research, coordinated among responsible actors. It also describes policy progress alongside unresolved verification and implementation constraints.

5. Governance of high-risk life-sciences research and AI-enabled biological systems

  • Risk-proportionate review should assess capabilities, intended applications, safer alternatives, and safeguards before research begins and throughout its life cycle.The section recommends independent and institutional review, risk-proportionate containment and operational controls, accountability, responsible communication, and international harmonization, while noting uneven resources and enforcement capacity.Source: §5 · p. 18
  • In interviews with 130 participants, 76% identified AI misuse in biology as an urgent concern and 74% called for clearer governance standards.Participants also emphasized functional screening and more consistent oversight.Source: §5 · p. 18
  • Governance should span model development, deployment, and the digital-to-physical pathway, with risk thresholds linked to proportionate controls and responsibilities distributed across ecosystem actors.Measures include tiered access, monitoring, sequence screening, customer verification, and human authorization at safety-critical stages.Source: §5 · p. 18, Table 2 · p. 19
  • Recent U.S. restrictions and NIH guidance address some gain-of-function research, while policy and arms-control precedents leave verification and enforcement challenges unresolved.The Biological Weapons Convention lacks a verification protocol, and digital and biological artifacts are harder to account for than fissile material (U.S. AI Action Plan 2025; WHO 2022).Source: §5 · pp. 17, 19

6. Discussion

The discussion prioritizes empirically tractable biosecurity threats while emphasizing the need to preserve beneficial uses. It identifies open questions about marginal risk and verification, and cautions that emerging agentic laboratories may change current capability boundaries.

6. Discussion

  • Near-term priorities include sequence optimization, screening evasion, assembly of fragmented dangerous sequences, and autonomous execution of narrow protocols, while governance must avoid conflating capability with intent or feasibility.Indiscriminate suppression could forfeit benefits in vaccine and antimicrobial design, diagnostics, antitoxins, and pathogen surveillance.Source: §6 · p. 20
  • Whether open release adds capability beyond published sequences and search tools, and whether biological designs and model capabilities can be verified, remain open questions.The passage notes that digital and biological artifacts are harder to account for than fissile material.Source: §6 · p. 20
  • As agentic, closed-loop laboratories mature, the physical-execution barrier could fall, and reviewed autonomous-experimentation results suggest that shift is beginning.The demonstrated-versus-speculative distinction is presented as a way to track a changing frontier, not as a reason to expect slow change.Source: §6 · p. 20

Related papers

Loading 2609.16213v1…