Source-linked AI summary
BioDyad: Synchronize Biomedical Discovery and Machine Learning Engineering
Xingbo Du, Fadli Aulawi Al Ghiffari, Leonard Song, Loka Li, Duzhen Zhang, Zixiao Wang, Xiuying Chen, Le Song
TL;DR
Biomedical ML must coordinate evidence acquisition with executable program search, but sustaining that coordination while managing execution feedback and validation demands remains challenging. BioDyad couples discovery and engineering through two hierarchies within Monte Carlo graph search. On 76 BioXArena tasks, it achieves the highest penalized all-task score and success rate under each of three matched LLM backends.
Problem
Method
BioDyad combines prior guidance, candidate-level biomedical discovery and plan-outcome memory with staged program execution in a shared Monte Carlo graph search.
Source: §1 · p. 2Results
Takeaways & Limitations
The results support using execution outcomes to guide how biomedical evidence is integrated and reused throughout program search.
Source: §6 · p. 9On one cancer-drug-sensitivity task, added biomedical features executed successfully but did not improve prediction, and the unproductive extension consumed search budget.
Source: §5.4 · p. 8
Abstract
from arXiv · showhide
Agentic biomedical machine learning (ML) draws on complementary advances in biomedical evidence acquisition and executable program search. Existing systems connect aspects of these capabilities, but coordinating them throughout program search remains challenging. New evidence must guide candidate construction, execution outcomes must inform subsequent discovery and reuse, and validation demands must fit the search budget. We introduce BioDyad, which couples biomedical discovery and ML engineering through two hierarchies within Monte Carlo graph search. Its scientific hierarchy combines prior biomedical guidance with iterative discovery, then links biomedical plans to execution outcomes in memory for reuse across candidates. Its engineering hierarchy moves candidate programs from smoke execution, through train/validation evaluation, to full-data retraining. We evaluate BioDyad on the 76-task BioXArena benchmark under a two-hour per-task budget with three matched LLM backends. It achieves the highest penalized all-task score and task success rate among four agent methods and a one-shot baseline under each backend. These results support coordinating biomedical discovery and ML engineering to integrate external knowledge into executable programs across heterogeneous biomedical tasks.
1 INTRODUCTION
BioDyad coordinates biomedical discovery and machine learning engineering throughout program search, where evidence integration and execution feedback must be managed within a fixed budget. It combines two hierarchies and achieves the strongest reported aggregate benchmark results across three matched LLM backends.
1 INTRODUCTION
- Autonomous biomedical ML must coordinate evidence acquisition with executable program construction because new evidence and execution outcomes can change modeling and validation choices.Evidence integration can also introduce errors and additional validation demands under a fixed budget.Source: §1 · pp. 1–2
- General-purpose MLE agents search executable programs, while biomedical agents acquire and interpret domain evidence; VCHarness combines aspects of both streams.VCHarness centers its node lifecycle on full fine-tuning and does not explicitly adapt exploration to the remaining budget.Source: §1 · p. 1
- BioDyad couples task-level biomedical guidance and candidate-specific discovery with memory linking plans to execution outcomes for later reuse.The shared MCGS backbone also uses execution feedback to update the search graph.Source: §1 · p. 2
- Across three matched LLM backends, BioDyad achieves the highest penalized all-task score and success rate among four baselines on BioXArena.The evaluation uses a two-hour per-task budget.Source: §1 · p. 2
- Its engineering hierarchy advances knowledge-augmented programs through smoke execution, train/validation evaluation, and full-data retraining.These stages separate feasibility screening, predictive assessment, and final model fitting.Source: §1 · p. 2
- Component ablations examine the coupled hierarchies, while extended-search trajectories characterize progress as more time becomes available.Ablations are reported on two domains.Source: §1 · p. 2
2 RELATED WORK
Prior work develops biomedical evidence acquisition and executable ML program search, with some systems already combining elements of both. BioDyad builds on these streams by integrating candidate-level discovery and staged, budget-aware validation into program search.
Agents for Biomedical Discovery
- Biomedical agents combine LLM reasoning with domain tools, data, and experimental context, while targeted systems address perturbation proposals, nanobody design, and biomedical ML program development.VCHarness couples a coding agent with biological foundation models for perturbation-response search; BioDyad revises candidate-level findings and retains them with execution outcomes.Source: §2 · p. 2
- Figure 1 depicts BioDyad's scientific and engineering hierarchies, with biomedical evidence guiding candidate expansion and execution feedback updating the graph and memory.Its example is the hERG Binding Affinity task from BioXArena.Source: Figure 1 · p. 3
Agentic Search for Executable ML Programs
- Executable-program agents use iterative experimentation, code tree search, memory, and empirical feedback, while BioML-bench and BioXArena evaluate heterogeneous biomedical tasks.BioDyad builds on MLEvolve's search to integrate external biomedical resources through candidate-level discovery and staged, budget-aware validation.Source: §2 · p. 3
3 PROBLEM FORMULATION
The formulation treats biomedical ML as a search over complete executable programs under a wall-clock budget. It frames the challenge as coupling evolving biomedical evidence with staged execution, since evidence integration can affect validation cost and fragility.
3 PROBLEM FORMULATION
- A biomedical ML task specifies public data, a natural-language description, an evaluation metric, a submission schema, and a total wall-clock budget.Candidate programs load data, train and validate a predictive model, and produce a submission conforming to the required schema.Source: §3 · p. 3
- The program and its supporting biomedical evidence are coupled: execution outcomes inform later resource use, while new evidence can increase validation cost and fragility.BioDyad addresses this by evolving evidence across search and advancing programs through staged execution.Source: §3 · p. 3
- Figure 2 depicts a plan-only biomedical agent alternating reasoning with read-only probes to compile verified data findings and biomedical insights.The prior guidance is established before search.Source: Figure 2 · p. 4
4 BIODYAD
BioDyad coordinates biomedical discovery and program engineering within an MCGS search loop, linking candidate plans to execution outcomes. Its hierarchies iteratively refine evidence-informed programs, retain outcomes for reuse, and evaluate candidates in stages.
4 BIODYAD
- BioDyad maintains an MCGS program graph and biomedical memory, using retrieved records and a candidate-specific plan to generate and engineer each child program.The child’s plan and observed outcome update memory for later expansions.Source: §4 · p. 4
4.1 SCIENTIFIC HIERARCHY
- The scientific hierarchy combines shared prior guidance, node-level discovery, and memory linking candidate plans to their computational consequences.This coordinates how biomedical evidence is initialized, revised, and retained.Source: §4.1 · p. 4
4.1.1 BIOMEDICAL DISCOVERY
- A plan-only agent probes task files and relevant resources, then compiles accumulated evidence into a BIO PLAN that conditions later discovery without prescribing a model family.The BIO PLAN contains verified data findings and biomedical implications supported by those findings.Source: §4.1.1 · p. 4
- During each node expansion, biomedical discovery uses the task, prior guidance, selected parent, and retrieved memory to produce a candidate-specific plan for modeling and validation.Later evaluations let discovery refine supported directions, abandon contradicted ones, or branch to another hypothesis.Source: §4.1.1 · p. 4
4.1.2 BIOMEDICAL MEMORY
- Biomedical memory stores each node’s plan, search stage, execution outcome, validation score change, and success or failure status, updating the record after execution.Keeping successful and failed attempts available helps later discovery distinguish integration errors from executable changes with little predictive benefit.Source: §4.1.2 · p. 5
4.2 ENGINEERING HIERARCHY
- Candidates progress from reduced-cost smoke execution to train/validation evaluation, while the best valid program is selected by its pre-retraining score for full-data retraining.Smoke failures return to MCGS; smoke-passing candidates receive metric, artifact, and schema checks.Source: §4.2 · p. 5
4.3 COUPLING THE HIERARCHIES
- MCGS uses execution history and remaining wall-clock budget to select expansions; validity and validation performance guide search, while memory preserves the rationale associated with outcomes.Later discovery can revise resource integration using feedback from earlier candidates.Source: §4.3 · p. 5
5 EXPERIMENTS
Across heterogeneous BioXArena tasks, experiments compare BioDyad under matched budgets and backends, then examine component contributions, case studies, and longer searches.
5.1 BENCHMARK AND EVALUATION PROTOCOL
- BioXArena spans 76 held-out tasks across nine domains, including 46 that combine multiple modality families; the primary comparison fixes the task interface, hardware, and two-hour limit.Agents train in a no-internet sandbox, and all phases are included in the time limit.Source: §5.1 · pp. 5–6
5.2 MAIN RESULTS
- 0.023 with GPT-5.6-sol, 0.042 with GLM-5.1, and 0.086 with DeepSeek-V4: BioDyad leads penalized all-task score and execution success under each backend.Each value is the improvement in penalized score over the strongest matched alternative.Source: Figure 3 · p. 6, §5.2 · p. 6
- Across three runs on a fixed 18-task subset with GPT-5.6-sol, BioDyad combines the highest mean penalized score with a small sample SD.The subset includes the first two tasks in each of BioXArena’s nine domains.Source: §5.2 · p. 7, Table 3 · p. 7
5.3 COMPONENT ABLATIONS
- The full framework averages 0.763, versus 0.718 for MLEvolve and 0.697 for MLEvolve w/ discovery; removing memory or staged evaluation lowers the average to 0.736 or 0.733.The ablations use GPT-5.4 on 16 chemical-biology and phenotype-disease tasks.Source: §5.3 · pp. 7–8, Table 4 · p. 7
- Discovery alone improves phenotype-disease prediction but reduces chemical-biology performance relative to MLEvolve; both domains score highest when discovery, memory, and staged evaluation are combined.Staged evaluation without the scientific hierarchy scores 0.665, below MLEvolve.Source: §5.3 · p. 8
5.4 CASE STUDIES
- Drug annotations raise validation Pearson r from 0.1016 to 0.1146; smoke execution detects a dimension mismatch in 1.09 s and guides a feature-retaining repair.The annotations are mechanism-of-action and target information; the smoke-detected error skips full execution.Source: §5.4 · p. 8
- On the cancer-drug-sensitivity task, added features execute successfully but do not improve prediction, so the original representation is retained and the extension consumes search budget.Source: §5.4 · p. 8
5.5 BUDGET EXTENSION WITHIN BIODYAD
- From 2 to 12 hours, mean best observed test scores rise by 0.010 in chemical biology and 0.050 in phenotype-disease; task trajectories vary, with Alzheimer’s staging plateauing at about five hours.BACE1 and CYP inhibition reach high scores early, while breast-cancer subtype also shows an early plateau.Source: §5.5 · p. 9, Table 5 · p. 9, Figure 5 · p. 9
6 CONCLUSION
BioDyad coordinates biomedical discovery and ML engineering under a fixed budget, and its matched-backend evaluation shows strong BioXArena performance. The conclusion also identifies resource, cost, and evaluation-scope limitations.
6 CONCLUSION
- BioDyad achieves the highest penalized all-task score and success rate on 76 BioXArena tasks under each of three matched LLM backends.A separate leaderboard comparison finds every BioDyad configuration exceeds the highest listed baseline overall score, though backend configurations differ.Source: §6 · p. 9, A COMPARISON WITH THE BIOXARENA… · p. 12
- The study supports using execution outcomes to guide how biomedical evidence is integrated and reused throughout program search.The scientific hierarchy combines prior guidance, candidate-level discovery, and biomedical memory; the engineering hierarchy provides feasibility and predictive feedback through staged evaluation.Source: §6 · p. 9
- The study is limited by external-resource quality and access, additional LLM calls, and evaluation on supervised BioXArena tasks with component and extended-budget analyses in two domains.Wet-lab and other long-horizon scientific workflows remain unevaluated.Source: §6 · p. 9
AI USE STATEMENT
- The manuscript used generative AI tools to polish language and improve readability and to draft figures; the authors take responsibility for the final manuscript.The responsibility statement includes all AI-assisted text and figures.Source: AI USE STATEMENT · p. 10
A COMPARISON WITH THE BIOXARENA LEADERBOARD
- At least one BioDyad configuration achieves the highest score in eight of nine BioXArena domains in the leaderboard comparison.The comparison retains each baseline's reported backend configuration, so the matched-backend results isolate the framework's contribution.Source: A COMPARISON WITH THE BIOXARENA… · p. 12
B IMPLEMENTATION DETAILS
Implementation uses Biomni resources selectively to inform biomedical plans and candidate changes, then records execution outcomes for reuse. Search combines staged candidate settings with MCGS selection strategies that adapt to the remaining budget.
B.1 BIOMEDICAL RESOURCES FROM BIOMNI
- Biomni resources support prior guidance and candidate-level discovery, which checks resource relevance and input compatibility before proposing representation or analysis changes.If no suitable resource matches the task, the candidate can retain its representation; proposed changes pass through staged execution and validation, with outcomes recorded for later search.Source: §B.1 · p. 12
B.2 DISCOVERY, MEMORY, AND EXECUTION SETTINGS
- Prior guidance runs once before search using a Biomni A1-based planner with at most six read-only probe blocks and no candidate training.Memory retrieval combines keyword and embedding rankings through reciprocal-rank fusion, supplying related successful or pending plans, failures, and a dissimilar successful plan.Source: §B.2 · p. 13
- Smoke execution follows static checks and preserves the full program's feature representation, model, and output format in a reduced-cost run.Passing requires successful execution, a validation score, and a submission artifact; finalization retains the selected program's pre-retraining score and keeps the original valid program if full-data execution fails.Source: §B.2 · p. 13
B.3 MCGS BACKBONE AND SEARCH CONFIGURATION
- MCGS organizes candidate programs as nodes, selects and expands them, and propagates reward updates through primary ancestors while cross-branch references add context.BioDyad's scientific hierarchy supplies plans and retrieved memory at expansion; the engineering hierarchy evaluates programs, and each plan and outcome updates biomedical memory.Source: §B.3 · p. 13
- UCT balances observed reward against exploration of less-visited nodes, while elite-guided sampling prioritizes high-scoring valid candidates.The elite strategy samples from top-K valid candidates in proportion to inverse score rank.Source: §B.3 · p. 13
- As the wall-clock budget advances, selection increasingly favors elite candidates, with UCT used according to a budget-dependent probability.The elite pool shrinks from up to five candidates before ρe to three afterward, with branch-specific limits; these rules follow the MLEvolve backbone.Source: §B.3 · p. 13
C ABLATION STUDIES AND SEARCH DIAGNOSTICS
The appendix analyzes component-ablation outcomes at the candidate level, distinguishing execution reliability from task-level success. It defines how candidate runs are counted and labeled.
C ABLATION STUDIES AND SEARCH DIAGNOSTICS
- The diagnostics examine how ablated components affect reliability of executed candidates, using GPT-5.4 on 16 chemical-biology and phenotype-disease tasks with a two-hour task budget.Counts are pooled across tasks within each configuration.Source: C ABLATION STUDIES AND SEARCH… · p. 13
C.1 DIAGNOSTIC SETUP AND COUNTING RULES
- Full-execution runs begin train/validation evaluation; with staged evaluation, only candidates passing static checks and smoke execution advance to that stage.Runs are valid when they finish with a finite validation score, no buggy flag, and the required artifact; defects, unparseable outputs, missing terminal records, timeouts, or interrupted bookkeeping count as invalid. These outcomes are distinct from task-level success.Source: §C.1 · p. 14
C.2 CANDIDATE EXECUTION OUTCOMES
Across candidate-level ablations, the complete framework combines the strongest execution reliability with the highest average score. Staged evaluation reduces invalid full executions, while biomedical memory improves the reliability of integrating discovery.
C.2 CANDIDATE EXECUTION OUTCOMES
- 80.9% of full executions are valid for the complete framework, which has 458 valid runs and 108 invalid runs, the fewest invalid runs among six configurations.Table 7 reports counts and percentages of each configuration's full-execution runs.Source: Table 7 · p. 14, §C.2 · p. 14
- Staged evaluation raises valid shares in all three matched pairs, reduces full executions, cuts invalid runs by 40–58%, and increases valid-run counts.The valid-share changes are 55.1% to 71.5%, 46.7% to 66.5%, and 61.5% to 80.9%; the corresponding full-execution counts fall from 552 to 446, 672 to 639, and 668 to 566.Source: §C.2 · p. 14
C.3 SCIENTIFIC HIERARCHY AND EXECUTION RELIABILITY
- Adding discovery without memory lowers valid share in both matched pairs, and without staged evaluation increases invalid runs from 248 to 358.These outcomes are consistent with the stated coordination problem that evidence integration introduces execution errors and additional validation demands.Source: §C.3 · p. 14
- Adding memory to discovery raises valid share from 46.7% to 61.5% without staged evaluation and from 66.5% to 80.9% with it.Memory links biomedical plans to execution outcomes so later discovery can revise failed integrations rather than propose them again.Source: §C.3 · p. 14
- The highest valid share alone does not determine predictive performance: BioDyad without the scientific hierarchy has 71.5% valid share but the lowest average score, 0.665.Only the complete framework is best on both measures, with the highest valid share and highest average score of 0.763.Source: §C.3 · p. 15
D BUDGET-DEPENDENT SCORE ANALYSIS
This analysis reconstructs 12-hour normalized test-score trajectories for 16 tasks from one GPT-5.6-sol run per task, using candidate completion times to track each task’s cumulative best.
D BUDGET-DEPENDENT SCORE ANALYSIS
- The trajectories cover all 16 chemical-biology and phenotype-disease tasks using one 12-hour run per task with GPT-5.6-sol.The analyses use the same fixed set of runs.Source: D BUDGET-DEPENDENT SCORE ANALYSIS · p. 15
D.1 TRAJECTORY RECONSTRUCTION
- The cumulative best at each checkpoint is the highest normalized test score among candidates completed by that time, carried forward until a better candidate completes.The last observed best is retained through 12 hours.Source: §D.1 · p. 15
- These retrospective test-score maxima characterize the evaluated candidate pool, with every task’s checkpoints drawn from the same run.Figure 6 shows all 16 trajectories.Source: §D.1 · p. 15
D.2 TASK-LEVEL GAINS AND THEIR TIMING
The task-level analysis compares checkpoint gains across 16 tasks and shows that improvement timing varies by task, a pattern that domain averages or a few examples may obscure.
D.2 TASK-LEVEL GAINS AND THEIR TIMING
- Table 8 reports normalized test-score checkpoints and absolute gains for the 12-hour traces, with each checkpoint a retrospective maximum over completed candidates.Its task-level view exposes timing differences behind the domain summaries in Table 5.Source: Table 8 · p. 15, §D.2 · p. 15
- Figure 6 plots cumulative best normalized test scores for all 16 tasks, with separate vertical scales and dashed guides at 2 and 4 hours.Steps mark logged candidate completion; the last observed best is carried forward to 12 hours.Source: Figure 6 · p. 16
- Timing and magnitude of gains vary across tasks, with Cell painting and Alzheimer’s staging gaining mainly at 2–4 hours, but COVID-19 severity and GPCR mainly after 4 hours.Genotype-to-phenotype gains mainly at 2–4 hours with a smaller later increase; Autism’s gain after 2 hours is below 0.001.Source: Table 8 · p. 15, §D.2 · p. 16, Figure 6 · p. 16
E.1 REPEATED-RUN TASK SELECTION AND AGGREGATION
This block specifies the repeated-run task subset and reference-system scope, then details three case studies illustrating predictive gain, execution repair, and a feature addition without predictive improvement.
E.1 REPEATED-RUN TASK SELECTION AND AGGREGATION
- The repeated-run study fixes the first two tasks in each of BioXArena’s nine domains; all four methods use the same tasks, backend, inputs, evaluator, hardware allocation, and two-hour limit across three runs.Mutable search state, generated artifacts, and method-specific memory reset between task-runs, while configured static resources remain fixed.Source: §E.1 · p. 17
E.2 BACKEND CONFIGURATIONS OF REPORTED SYSTEMS
- VCHarness is excluded from the two-hour comparison because its fine-tuning-centered node lifecycle targets multi-day execution, and compressing it would change the method.Table 11 records original backends for BioXArena’s reported-system reference, distinct from the comparison in Section 5.2.Source: §E.2 · p. 17, Table 11 · p. 18
E.3 CASE STUDY DETAILS
- The three cases use candidate programs, execution outputs, biomedical plans, and memory records; their scores are internal validation metrics, not normalized hidden-test benchmark scores.They illustrate predictive gain, successful execution repair, and feature integration that consumes budget without improving prediction.Source: §E.3 · p. 18
E.3.1 CASE (A): SUCCESSFUL CASE-PREDICTIVE GAIN
- The drug-transcriptional-response case increased mean per-gene Pearson correlation from 0.1016 to 0.1146 (+0.0130, or 12.8% relative) after adding Drug Repurposing Hub annotations.The 104 training drugs and 84 test drugs do not overlap; the same program and drug-grouped validation were compared with only the MoA/target block removed.Source: §E.3.1 · p. 18
E.3.2 CASE (B): SUCCESSFUL CASE-SMOKE-GUIDED REPAIR
- In the kinase-selectivity case, smoke execution rejected a faulty feature-dimension allocation in 1.09 s; after repair, validation achieved macro ROC-AUC of 0.9210.The repair changed the allocation and two dimension assertions while retaining the feature computations, classifiers, and hyperparameters.Source: §E.3.2 · pp. 18–19
E.3.3 CASE (C): FAILED CASE-NO PREDICTIVE GAIN
- The cancer-drug-sensitivity feature expansion scored 0.9555 versus 0.9556 for the original representation, which was retained.The 18-to-24-feature trial used the same model, hyperparameters, seed, and cell-line-grouped splits; the authors note the added features consumed budget without improving prediction.Source: §E.3.3 · p. 19