Source-linked AI summary
Bioinfoysis Technical Report
Qingyang Shao, Xin Zhang, Zhouyang Yuan, Xianying Chen, Yujia Xiang, Zihao Yang, Tong Ye, Yangqi Zhang, Jiakang Xu, Xiaoqing Yan, Xuan Luo, Keyi Li, Enci Fan, Kai Kang, Zhuohan Liu, Xingyu Jin, Chunran Teng, Tao Li, Xinyu Lv, Minghui Wang, Wenfeng Li, Yidan Gao, Siyu Liu, Mingrui Luo, Zhu Liang, Guanren Qiao, Zhiping Xu
TL;DR
Long-horizon bioinformatics requires more than final answers: conclusions must remain connected to data, computations, and intermediate evidence. Bioinfoysis addresses this gap with persistent, adaptive, artifact-grounded multi-agent runs, achieving strong benchmark gains while remaining limited by benchmark coverage and available skill modules.
Problem
Existing bioinformatics agents often treat planning, tool use, and code execution as transient interactions, weakening the connection between final conclusions and their supporting evidence.
Method
Bioinfoysis combines persistent runs, adaptive checklist-based planning, structured handoffs, controlled artifact validation, role-specific context, memory, and governed bioinformatics skills.
Results
82.4% accuracy was achieved on BixBench, while Bioinfoysis improved average accuracy from 41.65% to 69.69% on SeqQA2 and from 3.13% to 27.60% on DbQA2 across four model families.
Takeaways & Limitations
Reliable bioinformatics agents depend on an execution harness that preserves evidence, coordinates specialized agents, and produces verifiable scientific outputs alongside capable language models.
Takeaways & Limitations
Evaluation focuses primarily on BixBench and LAB-Bench2, which cannot fully characterize reliability across modalities, environments, and open-ended research settings.
Abstract
from arXiv · showhide
Large language model agents have shown promise in bioinformatics, but most existing systems focus primarily on producing final answers, treating planning, tool use, and code execution as transient interactions. This design is poorly suited to long-horizon bioinformatics tasks, where conclusions must remain connected to the data, computations, and intermediate evidence that support them. We introduce \textbf{Bioinfoysis}, a multi-agent harness that represents each request as a persistent, artifact-grounded analysis run. Bioinfoysis combines global planning with step-wise, evidence-driven replanning: the planner maintains an executable checklist and revises pending steps using structured handoffs returned after each worker execution. These handoffs bind intermediate results to their responsible agent, checklist step, and plan generation, preventing stale evidence from being silently reused after replanning. A controlled runtime validates generated scripts, tables, and figures before they are used in downstream analysis or reporting, while role-specific context, persistent memory, and governed bioinformatics skills support reliable execution over long analysis trajectories. We evaluate Bioinfoysis on BixBench and two question-answering tracks of LAB-Bench 2. On BixBench, Bioinfoysis achieves state-of-the-art accuracy of 82.4\%. Across four underlying language models, Bioinfoysis increases average accuracy from 27.81\% to 64.13\% on SeqQA2 and from 3.13\% to 31.25\% on DbQA2. These results demonstrate that reliable bioinformatics automation depends not only on model capability, but also on the harness that governs planning, execution, memory, and evidence flow. We hope that the emergence of Bioinfoysis will play a driving and leading role in the development of the bioinformatics community. Our demo website can be seen in https://report.bioinfoysis.com/.
1 Introduction
Bioinfoysis addresses the difficulty of making long-horizon bioinformatics analyses executable, inspectable, and connected to their underlying evidence. It combines persistent multi-agent workflows, structured handoffs, artifact validation, and trace-based auditing, with strong benchmark performance across tasks and models.
- Bioinformatics analyses require chained computational decisions that transform growing datasets into evidence, making complete workflows difficult to manage.
- Bioinformatics agents must support traceable data, tool, and execution histories rather than only producing final answers.
- Bioinfoysis represents each request as a persistent analysis run with coordinated planning, agent routing, runtime state, tools, workspaces, artifacts, and reporting.
- Structured handoffs bind conclusions, evidence, artifacts, agents, checklist steps, and plan versions, preventing earlier-plan results from being reused after replanning.
- Trace-based audits connect final answers to inputs, commands, skills, tools, outputs, and runtime information for model comparison and failure localization.
- 82.4% accuracy was achieved with Kimi-K2.6 on BixBench, while average accuracy rose from 41.65% to 69.69% on SeqQA2 and from 3.13% to 27.60% on DbQA2 across four model families.
2 Architecture
Bioinfoysis uses layered orchestration, scientific execution, and observability to turn open-ended bioinformatics requests into adaptive, artifact-grounded workflows. Specialized agents execute checklist steps, while structured evidence and validated artifacts guide replanning and reporting.
- System Overview: Bioinfoysis represents analysis as a persistent run with explicit inputs, workflow state, intermediate results, deliverables, and interacting orchestration, execution, and observability layers.
- System Overview: The default workflow uses coordinator, planner, researcher, coder, and reporter roles, with an optional human-feedback gate.
- Adaptive Planning: The planner creates a global checklist, then refines pending steps from worker evidence and execution feedback without rewriting completed or in-progress steps.
- Scientific Execution: Researchers gather biological and methodological evidence, while coders inspect data, execute scripts, and produce tables and figures in an isolated environment.
- Scientific Execution: Bioinfoysis validates reported artifacts and computation-dependent conclusions before accepting a step, using bounded correction for recoverable problems.
- Observability: Run evidence links inputs, scripts, references, intermediate results, artifacts, failures, agents, and checklist steps to support reporting, replanning, diagnosis, and evaluation.
3 Agent Infrastructure
Bioinfoysis infrastructure makes analyses persistent, recoverable, and separated from transient interaction state. Run-specific workspaces, artifact indexes, role-specific context, controlled publication, and traces preserve execution continuity and inspectability across deployments.
- Persistent Runs: Each persistent run has a stable identifier, selected inputs, workflow state, event history, and generated-artifact index for monitoring, failure inspection, and revisiting outputs.
- Controlled Execution: Run-specific working directories separate registered inputs from generated outputs, associate commands and artifacts with the active run, and serialize state-changing operations.
- Deployment: Local and distributed deployments use the same run interface and workflow representation while differing in where computation and storage occur.
- Context and Memory: Role-specific contexts combine recent summaries, artifacts, structured results, memory, and recovery information instead of sending every agent the complete conversation and tool history.
- Context and Memory: Structured handoffs record conclusions, evidence, references, artifacts, uncertainty, and execution status, with reuse checked against the expected agent, checklist step, and plan generation.
- Context and Memory: Only completed runs contribute evidence-backed facts to persistent memory, excluding failed, cancelled, or waiting runs from settled knowledge.
- Artifact Management: Artifact validation checks workspace paths, file existence, execution evidence, and quantitative consistency before attaching outputs to successful steps.
- Artifact Management: Public artifacts are separated from internal memory and raw execution payloads, then frozen and indexed before successful completion is reported.
4 Evaluation
Bioinfoysis is evaluated on BixBench, SeqQA2, and DbQA2 using benchmark protocols that compare final answers alongside execution and artifact correctness. It achieves leading BixBench performance and consistently improves model accuracy on both LAB-Bench2 tracks, while ablations show contributions from replanning, memory, and governed skills.
- BixBench: 82.4% accuracy (169/205) makes Bioinfoysis the top BixBench system, 7.3 percentage points above the second-ranked system.The comparison includes systems with different models and execution configurations at the system level.
- LAB-Bench2: Bioinfoysis improves over corresponding base models on SeqQA2, raising all four model accuracies to 66.00%–75.00% from 31.61%–64.75%.The largest reported gains are 36.89, 34.50, and 30.50 percentage points for DeepSeek, GLM, and Kimi, respectively.
- LAB-Bench2 limitations: SeqQA2 agent accuracies remain below 80% because plausible workflows can violate validator criteria or misinterpret calculation scope, units, precision, formatting, or biological definitions.These limitations show that executable analysis alone does not fully solve the benchmark.
- LAB-Bench2: 27.60% average accuracy on DbQA2 exceeds the base-model average of 3.13% by 24.48 percentage points.GPT-5.5 achieves the highest Bioinfoysis accuracy at 37.20%, followed by Kimi-K2.6 at 30.20%.
- Ablations: 79.02% to 72.20% accuracy after replacing ReAct-style planning with Plan-Execute shows a 6.83-point loss associated with removing evidence-driven revision.The reported rationale concerns revising pending steps after file inspection, tool execution, or partial validation.
- Ablations: Removing persistent memory reduces accuracy from 79.02% to 56.59%, while removing bioinformatics-specific skills reduces it from 79.02% to 60.00%.Persistent memory preserves exact inputs, operations, quantitative results, and artifact paths; governed skills provide task-specific analytical procedures and validation.
5 Conclusion, Limitations, and Future Work
Bioinfoysis combines adaptive planning, persistent memory, governed skills, controlled execution, and artifact-grounded reporting for long-horizon bioinformatics analysis. Its evaluation shows strong benchmark performance, while the authors identify limits in tool coverage, evaluation scope, and autonomous discovery.
- 5.1 Limitations: Bioinfoysis may not adequately support custom statistical models, specialized platforms, rare species, non-standard genomes, emerging algorithms, or highly customized analyses.
- 5.1 Limitations: Evaluation focuses primarily on BixBench and LAB-Bench2 and does not capture all dimensions of scientific quality, including model suitability, biological interpretation, and reproducibility.
- 5.1 Limitations: Bioinfoysis remains user-directed rather than autonomously formulating hypotheses, designing validation experiments, or pursuing open-ended research directions.
- 5.2 Future Work: Future work will expand skill coverage and support the full research lifecycle toward autonomous bioinformatics research.
- 5.3 Conclusion: Bioinfoysis integrates adaptive planning, persistent memory, governed skills, controlled execution, and artifact-grounded reporting.
- 5.3 Conclusion: 82.44% accuracy was achieved on BixBench, with strong performance across different underlying models and improvements on SeqQA and DbQA of LAB-Bench2.
- 5.3 Conclusion: Persistent memory, bioinformatics skills, and ReAct-style planning were important contributors in ablation results.
A Author List and Acknowledgments
The author list identifies core contributors, additional contributors, and contributors who have since left the team.
- Qingyang Shao, Xin Zhang, Zhouyang Yuan, and Xianying Chen are listed as core contributors.
- Yujia Xiang, Zihao Yang, Tong Ye, and additional named collaborators are listed as contributors.
- An asterisk identifies individuals who have since left the team.
B.1 Extended Evaluation on Bixbench
The BixBench audit shows that disagreements arise from underspecified tasks, reference mismatches, changing software or databases, data defects, and agent errors. Bioinfoysis also exposes domain-specific analytical choices that can change results or undermine reproducibility.
- Official notebooks can reproduce stored ideal answers without answering the literal question, making benchmark auditing necessary alongside task solving.
- Across 15 analysis domains and 205 BixBench tasks, disagreements arose from five sources: underspecified requirements, reference mismatches, software or database changes, data or provenance defects, and agent errors.
- Clinical regression estimates can change when case definitions, repeated records, analysis populations, category coding, or reference groups differ.
- CHIP and m6A DEG results depend on counting units and aggregation rules, because repeated annotations or records can represent the same variant or gene.
- Power calculations may overstate independent sample size when left- and right-hemisphere measurements are paired within animals.
- Expression correlations, miRNA analyses, and CpG-density calculations are sensitive to score direction, normalization, genome assembly, species labels, and processing rules.
- ENO1 proteomics is comparatively stable, but a large effect estimate must remain distinct from statistical significance after multiple-testing correction.
- Trimmomatic, BWA, GATK, clustering, and pathway analyses can vary with command settings, software versions, stochastic seeds, duplicate handling, and counting conventions.
B.2 Extended Evaluation on LAB-Bench 2
LAB-Bench 2 evaluation combines manual review for open-ended sequence tasks with revised grading comparisons between Bioinfoysis and base models. Manual review shows that exact-reference scoring can underestimate biologically valid solutions, while the agent condition consistently outperforms direct model use.
- Manual review accepts alternative primers, oligonucleotides, amplicons, and coding sequences when they satisfy the prompt’s biological and task-specific constraints.
- 63 of 91 manually reviewed SeqQA cases were correct, yielding 69.23% accuracy after assessing biological and task-specific requirements.
- Exact reference matching underestimates performance on sequence-design tasks that admit multiple biologically valid solutions.
- The 28 incorrect cases included benchmark-reference disagreements, invalid primer or oligonucleotide designs, incomplete amplicons, placeholder responses, and incorrect sets.
- Agent accuracies ranged from 62.50% to 71.25%, compared with 25.25% to 63.50% for the corresponding Direct base-model conditions.
- DeepSeek-V4 showed the largest improvement at +40.00 percentage points, followed by Kimi-2.6 at +37.00 and GLM-5.2 at +32.50.