DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration

1 Introduction and Related Work

• Drug-discovery ML pipelines span several specialized fields, and general-purpose ML agents lack expert workflow knowledge while chemistry-focused tools offer limited support for large-scale ML.Small mistakes such as choosing the wrong library or misinterpreting biological data types can be difficult to debug.Source: §1 · p. 1
• DrugAgent distinguishes itself through environment interaction, ML-programming specialization, drug-discovery domain knowledge, and idea-space planning.Its stated contributions include systematic domain-knowledge integration, planning guided by experimental observations, and specialized tools and documentation.Source: Table 1 · p. 2, §1 · p. 2
• DrugAgent’s iterative search starts from a natural-language drug-discovery task and combines Planner and Instructor work toward actionable, high-performing solutions.The overview illustrates the input with an Absorption prediction task using the PAMPA dataset.Source: Figure 1 · p. 2

2 Methodology

• An ML programming task comprises a natural-language Task Description, Starter Files such as datasets or code templates, and an Evaluator that assesses output quality with a performance metric.The Evaluator is the metric function used to assess the output.Source: §2 · p. 2
• The Planner generates multiple ideas, explores them through Instructor evaluation, revises the set using success or failure reports, and submits the highest-performing idea after a maximum number of iterations.Underperformance or infeasibility can lead to an idea being discarded.Source: §2 · p. 2
• The Instructor incorporates domain knowledge throughout coding, executes standard ML actions, and uses targeted documentation to build or refine specialized tools.Its performance report can instead be a failure report when critical functionalities are absent.Source: §2 · p. 3
• The curated resources cover biological data acquisition, molecule and protein featurization, and domain-specific models including ChemBERTa and ESM.The framework aims to reduce errors from incomplete or incorrect handling of drug-discovery subtleties.Source: §2 · p. 3

3.1 Experimental Setup

• The experiments cover ADMET prediction, high-throughput screening, and drug-target interaction prediction, using PAMPA, HIV, and DAVIS datasets, respectively.Each task is a binary classification problem; the study conducts eight independent runs for each AI-based method and reports average ROC-AUC over valid submissions.Source: §3.1 · p. 3

3.2 Quantitative Results

• DrugAgent achieves the highest ROC-AUC and Valid Rate among AI-based methods across PAMPA, HIV, and DAVIS, performing comparably to human-selected baselines.The reported relative ROC-AUC improvement over ReAct on DTI is 4.92%.Source: §3.2 · pp. 3–4, Table 2 · p. 4
• The DAVIS ablation table examines how removing the Planner or Instructor affects ROC-AUC and Valid Rate, with results averaged across runs.The supplied passage identifies the comparison but does not provide its values.Source: Table 3 · p. 4
• Both the Planner and Instructor contribute significantly to overall performance, and DrugAgent’s case study highlights idea diversification, domain-knowledge integration, and learning from failures.The framework also reports that considering multiple submissions can mitigate suboptimal selection from validation results.Source: §3.2 · p. 4, §3.3 · p. 4

3.3 Case Study

• The case study compares DrugAgent with ReAct and reports that DrugAgent has no errors from incorrect operations requiring domain knowledge and the lowest overall error rate.The analysis examines DTI run traces and categorizes the four most common error types.Source: §3.3 · p. 4, Figure 2 · p. 4

4 Conclusion

• DrugAgent is a multi-agent framework for automating critical aspects of drug discovery, with case studies spanning three tasks.The conclusion describes the framework as addressing pharmaceutical research’s nuanced demands.Source: §4 · p. 4
• The Planner searches for ideas, while the Instructor supports reliable implementation by integrating a specialized toolset.The authors attribute the framework’s improvements largely to these two agent roles.Source: §4 · p. 4
• The paper reports improvements over general-purpose frameworks including ReAct and ResearchAgent across its three drug-discovery case studies.Source: §4 · p. 4

• Evaluation covers only three case-study tasks, which the authors say are insufficient for a comprehensive assessment of drug-discovery ML programming.They call for more extensive benchmarks in drug-discovery settings.Source: Limitations · p. 5
• DrugAgent is compared with classic state-of-the-art baselines rather than the latest cutting-edge methods.The authors say advancing agent capabilities will require significant research efforts.Source: Limitations · p. 5
• The current DrugAgent documentation is relatively basic and could be expanded to cover more of the drug-discovery process.Source: Limitations · p. 5
• The framework could incorporate a human-in-the-loop approach to improve usability for scientists working on real-world drug discovery.Source: Limitations · p. 5

Ethics Statement

• Hallucinated or fabricated results could waste wet-lab verification resources or misguide drug-discovery directions, so the current framework is not ready for direct pipeline deployment.The authors say further safety checks and human oversight are essential.Source: Ethics Statement · p. 5

A Related Work

• Existing ML-programming agents cannot handle domain-specific drug-discovery tasks, while biomedical agents generally lack ML-specific expertise for end-to-end programming.Source: A Related Work · p. 8

B Action

• The instructor can list, read, write, append, copy, inspect, undo, execute, understand, and edit files or scripts, including script segments.The paper recommends MLAgentBench for detailed explanations of these commonly used ML-agent actions.Source: B Action · p. 8

C Documentation

• Documentation covers 66 AI/ML-ready datasets, drug and protein preprocessing methods, and ChemBERTa and ESM models.It draws on TDC, DGL-LifeSci, RDKit, PyBioMed, and Transformers resources, and the authors say it can be extended for specific needs and available resources.Source: C Documentation · p. 8

• DAVIS contains 68 drugs and 379 proteins, with 2,086 training, 3,006 validation, and 6,011 test samples.The passage points to MolTrans for dataset and preprocessing details.Source: D Dataset Description · p. 8
• PAMPA has 1,424 training, 203 validation, and 407 test samples, split using the TDC random split strategy.Source: D Dataset Description · p. 8
• HIV has 28,789 training, 4,113 validation, and 8,225 test samples, split using the TDC random split strategy.Source: D Dataset Description · p. 8

E Human Baseline

• The task overview focuses on small-molecule drugs, which the paper says constitute over 90% of approved drugs, represented as SMILES strings.Source: Table 4 · p. 9
• For ADMET and HTS, the human baseline is a random forest with Morgan fingerprinting, chosen because tree-based models consistently outperform several alternatives on those tasks.Source: E Human Baseline · pp. 8–9
• DeepDTA, which uses two CNN encoders for drug and protein representations, serves as the human baseline for DTI.The paper describes it as a well-established, widely adopted state-of-the-art baseline for DTI studies.Source: E Human Baseline · p. 9

F Settings

• All agent frameworks are limited to 100 actions; ResearchAgent processes only the first three blocks when understanding a file and receives error messages to aid debugging.Source: F Settings · p. 9

G.1 Without Instructor

• Without the Instructor, exploring multiple ideas improved overall performance over ReAct, but results remained unsatisfactory because molecules were sometimes encoded ineffectively.Source: §G.1 · p. 9
• In the example, ReAct encodes a protein as a fixed 1024-feature vector containing each sequence character's ordinal value.Source: §G.1 · p. 9

G.2 Without Planner

• Without the Planner, iterative model experiments often stayed within similar ideas, limiting exploration of more optimal approaches.Source: §G.2 · p. 9

H Comparing DrugAgent with ReAct

• DrugAgent diversifies solution ideas, integrates domain-specific methods, and uses observations to guide future choices, whereas ReAct focuses on refining a single approach and struggles to recover from poor planning.In the case study, DrugAgent avoids further exploration of graph-based protein encoding after identifying its limitations.Source: H Comparing DrugAgent with ReAct · pp. 10–11
• DrugAgent identifies protein preprocessing needs and uses fingerprint-based biological-data encoding, producing a bug-free model on its first attempt.ReAct misses early protein preprocessing, leading to debugging time and suboptimal molecular-data encoding choices.Source: H Comparing DrugAgent with ReAct · p. 10

H.1 Sample Code by DrugAgent

• The sample code loads train, validation, and test CSVs, extracts drug and protein features, computes ROC-AUC scores, and predicts test-set probabilities.The code uses ECFP4 for drug features and calls a protein featurizer for training and validation data.Source: §H.1 · p. 11

• Hallucination means fabricating results or falsely claiming progress, such as reporting a submission without editing the training script.Source: I Error Type · p. 11
• Debugging errors occur when the agent cannot resolve code-modification issues, such as mismatched tensor shapes.Source: I Error Type · p. 11
• Domain errors are poor performance caused by incorrect operations requiring domain knowledge, including improper drug or protein fingerprinting.Source: I Error Type · pp. 11–12
• Format errors occur when the agent changes the submission format so the evaluator cannot recognize it.Source: I Error Type · p. 12