MADD: Multi-Agent Drug Discovery Orchestra

1 Introduction

• AI-based virtual screening aims to reduce wet-lab resources and improve success rates, but growing tool complexity limits accessibility for wet-lab researchers.Source: §1 · p. 1
• MADD outperformed existing LLM-based solutions and demonstrated effectiveness across seven cases, including unseen thrombocytopenia.The evaluation also compared MADD-produced molecules with experimentally validated GSK-3β inhibitors.Source: §1 · p. 2
• MADD uses four specialized agents to coordinate hit-identification subtasks, addressing the proposed need for end-to-end query analysis, molecule generation, and property calculation.Source: §1 · p. 2
• The benchmark pairs drug-discovery queries with generated molecules and provides docking scores for over 3M molecules across six conditions.It comprises two synchronized datasets: query–molecule pairs and docking scores.Source: §1 · p. 2
• MADD pioneered AI-first drug design for STAT3, ABL, COMT, ACL, and PCSK9, identifying molecules with favorable bioactivity, binding affinity, and other physicochemical properties.Source: §1 · p. 2

2 Related work

• LLMs and agent frameworks have been applied to chemistry tasks including data analysis, molecule generation, optimization, property prediction, synthesis planning, and molecular analysis.The passage cites ChemDFM (Zhao et al., 2024), LlasMol (Yu et al., 2024a), X-Lora-Gemma, CancerGPT, DrugAgent (Liu et al., 2024a), ChemCrow (M. Bran et al., 2024), CACTUS (McNaughton et al., 2024), and ChemAgent (Yu et al., 2024b).Source: §2 · p. 2
• Prior studies had not demonstrated successful automation of the whole drug-discovery pipeline; the authors identify missing complete-pipeline datasets and weaknesses in separated models and agent architectures as possible reasons.Source: §2 · p. 2
• The authors propose a stronger approach and a newly assembled dataset of pharmaceutical research tasks and solutions.Source: §2 · pp. 2–3

3 MADD orchestra

• MADD integrates generative and predictive tools into an end-to-end pipeline, using specialized agents to isolate workflow functions and reduce each agent’s possible actions.The authors describe specialization as supporting step accuracy and flexibility in constructing multi-step pipelines.Source: §3 · p. 3

3.1 Agents

• The Decomposer breaks complex queries into subtasks, the Orchestrator plans actions and uses generation and prediction tools, and the Summarizer presents results; the Chat Agent clarifies queries and requests missing data.Source: §3.1 · p. 3

3.2 Integrated tools

• The integrated toolkit groups molecular generation, property prediction, and data processing; it includes pretrained or fine-tunable generators, custom-data prediction pipelines, and docking-based binding-energy estimates.Supported generators include an LSTM-based GAN and transformer-based CVAE; FEDOT can train prediction pipelines on custom datasets.Source: §3.2 · pp. 3–4

4.1 Benchmark creation

• The benchmark links user queries, target molecular structures, and properties across six drug-discovery cases, addressing the lack of complex screening scenarios in existing datasets.Queries were divided by scenario complexity into single-task Dataset S, 1–3-task Dataset M, and 4–5-task Dataset L.Source: §4.1 · pp. 4–5

4.2 Comparison of underlying LLMs

• Llama-3.1-70b with an optimized prompt achieved 92.3% orchestrator accuracy and was selected as MADD’s default agent LLM.It exceeded o1-mini (67.5%), DeepSeek-R1 (88.11%), and Llama-3.2-90b (90.7%), while also being cheaper.Source: §4.2 · p. 5

4.3 Ablation study

• MADD achieved almost 80% final accuracy on the most complex Dataset L scenarios; systems without a dedicated Summarizer had inconsistent responses, and broader Orchestrator roles performed worse.The evaluation considered tool-selection, summarization, and final accuracy.Source: §4.3 · p. 6

4.4 Analysis of generative models

• The MADD Transformer ranked first in three of six cases under the strictest filter, while GAN never ranked first but produced hits in five cases.Other generative approaches sometimes performed better in individual cases, but MADD’s overall performance remained superior.Source: §4.4 · p. 6

4.5 Comparison with existing solutions

• Standalone LLMs largely failed to produce structured outputs with valid compounds; ChemAgent had no more than 2.5% GR1 hits and 16.4% final accuracy, while MADD led tool selection by up to 29.1%.The tool-selection advantage was reported for Datasets S and M; accuracy was similar on Dataset L.Source: §4.5 · pp. 6–7
• In a brief comparison, Phoenix produced one molecule passing GR2 and TxGemma produced none passing any filter.Source: §4.5 · p. 7

4.6 End-to-end hit identification with MADD

• MADD outperforms LLM-based approaches in all filter groups and mean docking score during end-to-end hit identification.Other models can generate valid SMILES but do not consistently connect molecular properties with molecules.Source: §4.6 · p. 7

• Predictive models guide generative-model learning by stopping training early and selecting the best epoch using loss and errors in matching desired molecular properties.The comparison of automatic and manual generative-model training used Alzheimer’s disease; predictive performance was assessed across six diseases.Source: §5 · p. 7
• Automatically trained MADD pipelines outperformed manually pre-trained models in most IC50 prediction cases.Source: §5 · p. 7, Figure 4 · p. 7
• Tool-selection accuracy decreased from 83.7% to 80.5% on average when a tool for generative- or predictive-model training was added.Source: §5 · p. 8

Alzheimer’s disease case study

• MADD-generated Alzheimer’s disease molecules had higher mean SA and QED scores, improved binding affinities, and an average Tanimoto similarity of 0.43 versus ChEMBL inhibitors.The comparison used experimentally validated GSK-3β inhibitors from ChEMBL.Source: §6 · p. 8, Figure 5 · p. 8

Thrombocytopenia case study

• MADD generated 10k thrombocytopenia molecules, with 132 passing all five property filters, versus 76k SYK-FBRL molecules, of which 139 met the required pIC50 and DS properties.The reported counts use different filter descriptions for the two approaches.Source: §6 · p. 8
• MADD-generated thrombocytopenia molecules had improved docking scores, QED, and SA properties over SYK-FBRL molecules, without configuration changes or additional expert input.Figure 5b compares the molecules generated by the two approaches.Source: §6 · p. 8, Figure 5 · p. 8

Discussion

• MADD’s current implementation requires users to supply molecules and target properties for training, and assumes prior knowledge of the target or relevant properties, limiting discovery for fundamentally novel targets.Users may lack well-curated datasets, which can affect model performance and reproducibility.Source: §7 · p. 8
• Table 4 compares automatically trained MADD models with pretrained models for Alzheimer’s disease and thrombocytopenia; thrombocytopenia novelty is measured against the SYK-FBRL compound dataset.Source: Table 4 · p. 9
• Future work should integrate automated data curation and hypothesis-generation modules to address the stated data and open-ended discovery constraints.Source: §7 · pp. 8–9
• The system’s generalization is predominantly demonstrated in silico, without broader experimental validation; confirming predictions would require biochemical, cell-based, and animal studies.The authors describe those studies as time-consuming, costly, and resource-intensive.Source: §7 · p. 9

Conclusion

• 79.8% overall pipeline accuracy and up to 6.63% novel hits meeting eight key drug criteria are reported for MADD, which the authors say consistently outperforms baselines in hit-identification efficiency.Source: §8 · p. 9
• Validation on two case studies, including an unseen thrombocytopenia case, is reported as further evidence of MADD’s autonomous discovery capabilities.Source: §8 · p. 9
• The paper presents a benchmark and discusses current challenges in generalization and wet-lab experimentation.Source: §8 · p. 9

Limitations

• Adapting MADD to a new disease requires data preparation, generative-model training, and predictive-model fine-tuning; the authors estimate this should not exceed one day but call it a practical constraint.Source: Limitations · p. 10
• In chat mode, MADD hides its internal workflows; RDKit-based tools allow partial evaluation, and decision logging is proposed to improve interpretability.Source: Limitations · p. 10
• Adding tools requires direct code modification, which may challenge users with limited programming expertise despite the system being open source.Source: Limitations · p. 10
• User-specified thresholds for disease-relevant molecules require domain expertise, which may limit accessibility for non-specialists.Source: Limitations · p. 10

A.1 Impact Statement and Potential Risks

• MADD is presented as simplifying candidate search and widening chat-based access, while the authors discuss job-displacement and harmful-design risks alongside task-assignment and filtering mitigations.The authors undertake to minimize misuse risks when releasing the framework.Source: §A.1 · p. 15

A.2 Declaration of AI assistance

• ChatGPT was used for grammar checking and LaTeX support, and the authors reviewed and edited the study’s content.Source: §A.2 · pp. 15–16

B.1 LLM-based drug design

• The related-work review presents LLM applications spanning chemical-data analysis, molecular optimization, property prediction, synergy prediction, and de novo molecule generation.Examples include ChemDFM, X-Lora-Gemma, CancerGPT, and Ye’s generation method.Source: §B.1 · p. 16

B.2 Chemical agent-based LLM pipelines

• Existing agent systems differ in scope: DrugAgent optimizes proposed molecules, ChemCrow focuses on synthesis, CACTUS handles one-step tasks, and Phoenix supports de novo design and multi-target optimization.ChemAgent has 29 tools, but the passage reports low validity for its generated molecules.Source: §B.2 · p. 16
• Prior work had not demonstrated successful automation of the whole drug-discovery pipeline; the authors cite limited datasets and the weakness of separate models and agent architectures as possible reasons.Existing benchmarks also lack the complexity of real-world screening scenarios involving target identification, molecule generation, and property evaluation.Source: §B.2 · p. 16

C.1 Description of related experiments conducted outside the benchmark.

• Adding automated sequential training of predictive and generative models on user-prepared data did not compromise MADD’s accuracy, supporting extension beyond its original seven use cases.Automatic expansion requires training monitoring, dataset search and preparation, and coordination of predictive-model readiness; the authors report implementing these functions in AutoML-DL and DatasetBuilder.Source: §C.1 · p. 17

C.2 Multi-agent system ablation study; C.3 Comparisons with existing LLM solutions

• The ablation variants differed in agent roles and workflow structure, while ChemAgent’s GPT-4o evaluation was costly because it required multiple queries per example.MADD separates summarization and user interaction into modular steps; the ablation experiments used Llama-3.1-70b across agents.Source: §C.2 · pp. 17–18, §C.3 · p. 18

C.3.1 Phoenix

• Of Phoenix’s 40 generated structures, 6 passed GR1 and one Alzheimer-focused molecule met GR2; interruptions left final answers for only slightly more than half of questions.Many proposed molecules matched known ChEMBL compounds, although some appeared to have no PubChem or ChEMBL matches.Source: §C.3.1 · p. 19

C.3.2 TxGemma

• Neither TxGemma-9B-chat nor TxGemma-27B-chat generated structures that met the GR1 or GR2 thresholds, reflecting limited robustness for open-ended molecule design.The authors attribute this to training focused on narrower predictive and supervised tasks, while noting potential for reasoning and explanation in therapeutic R&D.Source: §C.3.2 · p. 19

C.4.1 Benchmark Preparation Pipeline; C.4.2 Dataset diversity

• The query benchmark was expanded to 400 examples and expert-reviewed to 245; Dataset M combines 1–3 tasks per query, whereas Dataset L contains 4–5.Dataset S contains single-task queries, and query diversity was assessed using pairwise cosine similarity of embeddings.Source: §C.4.1 · p. 20, §C.4.2 · p. 20

C.4.4 Dataset Preparation for Generative Model Training

• Six disease-specific training datasets totaling over 3 million molecules were assembled, with docking energies and predicted binary IC50 activity among the calculated data.The molecules came from ChEMBL and were selected for low molecular weight; the collection totaled approximately 500,000 molecules per disease.Source: §C.4.4 · pp. 20–21

C.6 Properties prediction by manual pre-trained ML models experimental studies

• Five candidate models—CatBoost, XGBoost, Random Forest, Extra Trees, and LightGBM—were cross-validated, with the best selected for each lgIC50 prediction task.The study used cross-validation to check predictive-model accuracy before generating molecules with the full system.Source: §C.6 · p. 21
• CatBoost achieved F1 scores of 0.82 for Dyslipidemia, 0.85 for Drug resistance, and 0.92 for Parkinson’s disease; Extra Trees achieved 0.83 for Alzheimer’s disease and 0.84 for lung cancer.These were the most effective models for the respective conditions.Source: §C.6 · p. 21
• Random Forest showed the best result for multiple sclerosis, with an F1 score of 0.92.The selection process resulted in selecting the top 3 best models.Source: §C.6 · p. 21

C.7 Additional AutoML tool results

• Alzheimer’s stacking achieved 0.963 accuracy and 0.978 F1, while multiple-sclerosis weighted averaging reached 0.889 accuracy and 0.921 F1, near Random Forest’s 0.887 accuracy.For Parkinson’s disease, CatBoost’s 0.910 accuracy exceeded the stacking pipeline’s 0.872 accuracy.Source: §C.7 · p. 22
• CatBoost led dyslipidemia standalone models at 0.778 accuracy and 0.818 F1; tuned Random Forest reached 0.845 versus vanilla Random Forest’s 0.828 accuracy for drug resistance.Extra Trees’ 0.843 accuracy exceeded lung-cancer stacking’s 0.780; on SYK, MADD achieved MSE 0.32 and R2 0.75 versus SYK-FBRL’s MSE 0.27 and R2 0.78.Source: §C.7 · p. 22

C.8 Training tool selection

• Tool selection metrics averaged 0.79 for Alzheimer’s disease and 0.82 for thrombocytopenia across experiments testing whether to train a model or use an available trained model.Each disease had 28 generation queries, and the orchestrator used the trained-model dictionary to make its selection.Source: §C.8 · p. 22

C.9 Generative model comparisons study

• LLM-based generators produced 27.7%–99.84% chemically valid molecules, but no more than 2.56% passed filters for two or more properties; ChemDFM reached 10.71% at Alzheimer’s fifth filter.ChemDFM was pretrained, so the authors could not verify how many generated molecules were absent from its training data.Source: §C.9 · p. 24
• MADD’s non-LLM transformer was the most stable method, outperforming alternatives on three diseases; MTDD-EF reached 28% at the fifth filter for multiple sclerosis, with similar molecules by Tanimoto similarity.MTDD-EF’s high multiple-sclerosis result was not reproduced in other diseases.Source: §C.9 · p. 24

C.9.1 Our developed generative models

• Generating 10,000 molecules took the Transformer about 45 minutes versus the GAN’s 1.88 seconds on an RTX A6000, and 3.73 hours versus 3.37 seconds on an RTX 2070.The authors note that combining generative models is needed for more comprehensive chemical-space exploration.Source: §C.9.1 · p. 24

C.10 Analysis of an evolutionary approach for hit molecule generation

• Evolutionary generation looked promising for Alzheimer’s and Parkinson’s but yielded no satisfactory multiple-sclerosis molecules, indicating sensitivity to initialization; molecular-mass differences were 16% for Alzheimer’s and 35% for multiple sclerosis.The authors conclude that effective evolutionary optimization requires a different initial-population method than MTDD-EF uses.Source: §C.10 · p. 24

C.11 Overall efficiency analysis

• MADD assessed novelty against its GAN and Transformer training data, but novelty could not be tested for other LLM-based approaches because their training data were unavailable.The authors note that larger training datasets make novel-molecule discovery more challenging, while Transformers may benefit from larger training data.Source: §C.11 · pp. 24–25

D.4.1 IC50

• IC50 prediction for GSK-3β, BTK, and ABL2 was framed as binary classification using a median lgIC50 split and molecular fingerprints or descriptors.The models used ChEMBL and BindingDB data; representations included Morgan fingerprints, Avalon, and RDKit descriptors.Source: §D.4.1 · pp. 28–29

D.4.2 Docking score

• 0.14 seconds with QuickVina GPU 2.1 versus 5 seconds with AutoDock Vina reduced dataset docking time from 1667 hours to 19 hours.Docking scores were calculated for disease-specific target proteins.Source: §D.4.2 · p. 29

D.5 Justification of the choice of filter groups

• The five filter groups progress from biological activity (GR1), through synthesizability (GR2) and Brenk screening (GR3), to exclusion filters (GR4) and QED > 0.6 (GR5).GR4 requires SureChEMBL, Glaxo, and PAINS scores of 0; GR5 is the most stringent group and focuses on drug-likeness.Source: §D.5 · p. 29

D.6.1 Alzheimer’s disease

• Alzheimer’s disease motivates new GSK-3β inhibitors because tau hyperphosphorylation contributes to pathology, while tideglusib showed insufficient efficacy and gastrointestinal toxicity.Tideglusib modestly improved cognitive function versus placebo in phase I and II trials, motivating inhibitors with enhanced properties.Source: §D.6.1 · p. 29

D.6.2 Multiple sclerosis

• For multiple sclerosis, the case targets noncovalent BTK inhibitors with improved IC50 values and blood-brain barrier permeability.The passage describes noncovalent inhibitors as less potent and requiring higher doses, but more selective and less prone to resistance.Source: §D.6.2 · pp. 29–30

D.6.3 Parkinson’s disease

• For Parkinson’s disease, the case focuses on ABL inhibitors because ABL hyperactivation is associated with oxidative stress and accumulation of damaged proteins linked to neuronal death.The study aims to generate new ABL inhibitors with improved properties.Source: §D.6.3 · p. 30

D.6.4 Drug resistance; D.6.5 Dyslipidemia

• The drug-resistance case seeks selective STAT3 inhibitors amid toxicity and limited selectivity challenges, while dyslipidemia motivates new ACL and PCSK9 ligands with distinct benefits.STAT3 strategies include predictive biomarkers and combinations with RTK-targeting therapies; new ACL ligands may avoid overlapping statin mechanisms, while PCSK9 antibodies face cost-effectiveness and long-term safety concerns (Berkhout et al., 1990; Ballantyne et al., 2016; Cohen et al., 2006; Bilen and Ballantyne, 2016; Iqbal et al., 2019).Source: §D.6.4 · p. 30, §D.6.5 · p. 30

D.6.6 Lung cancer

• For lung cancer, the case targets improved KRAS inhibitors because KRAS mutations drive uncontrolled growth and current G12C inhibitors have limited reach, side effects, and resistance.The passage states that non-small cell lung cancer accounts for over 80% of lung cancer cases.Source: §D.6.6 · pp. 30–31

D.7.1 Detailed evaluation: Alzheimer’s disease

• 16,082 novel GSK-3β inhibitors were generated by the transformer for the Alzheimer’s case.Their average QED increased 11.8%, average SA was lower than for experimentally validated compounds, all passed the Brenk filter, and Tanimoto similarity to ChEMBL molecules was 0.43; detailed validation is presented only for Alzheimer’s disease.Source: §D.7.1 · p. 31

D.8.2 Formulas for calculating the accuracy per query as a whole

• Final accuracy multiplies separately measured tool-selection and summarization probabilities; a correct response preserves molecules and property values, while a correct query uses the right tool.Source: §D.8.2 · p. 31

D.9 Schemes of the considered agent architectures

• The appendix points readers to figures visualizing the two-agent and three-agent architectures.Source: §D.9 · p. 31

D.10 MADD failure cases examples

• Baseline experiments excluded iterative resampling, although that validation feature can resample generation tools until user-specified criteria are met.Generated compounds may still have low synthetic feasibility despite validation for chemical validity, drug-likeness, and target activity.Source: §D.10 · p. 32
• Reported failure examples include an extra tool call after invalid task decomposition and selection of a drug-resistance model for dyslipidemia.Other examples include an invalid SMILES and a structure with trivalent oxygen; errors are reported at both decomposition and tool-selection stages.Source: §D.10 · pp. 32–33
• 86.9% accuracy for Dataset S drops to 79.8% for Dataset L as query complexity increases.The authors describe Orchestrator errors as more critical than Decomposer errors and say correct selection is high, making such errors rare.Source: §D.10 · pp. 32–33

E.1 Results of system workflow.

• MADD’s example dialogue response is divided into paragraphs according to the number of requests in the user’s message.The example is shown in Figure 17 and uses a question from Dataset L, described in its caption as the hardest dataset.Source: §E.1 · p. 33, Figure 17 · p. 43

E.2 Examples of prompts extracts for the main agents in the system.

• The Orchestrator receives function and available-model dictionaries, with instructions to use a trained generator when available and train a model for an unrepresented case.The prompt examples specify generation model options and training parameters, including a default of 100 epochs.Source: §E.2 · pp. 33–34

E.3 Examples of Agents decision. System Logs

• The logged GSK-3beta workflow decomposes a generation request, selects the Alzheimer’s molecule-generation tool, and returns two molecules with docking scores of -6.36 and -6.79.The log also lists property and screening fields, including QED, synthetic accessibility, PAINS, and docking score.Source: §E.3 · p. 35

E.4.1 MADD answers

• MADD’s dialogue examples show it splitting combined molecule-generation requests into separate steps and using tools to retrieve 653 ChEMBL entries for GSK with IC50 values.Source: §E.4.1 · p. 35

E.4.2 ChemAgent answers

• For an ABL inhibitor query, ChemAgent suggested dasatinib with an approximately 4-hour half-life but said the requested LD50 value was not precisely documented and provided no molecules.Source: §E.4.2 · p. 35

E.4.3 LlasMol answers

• For a request to design bile acid sequestrants, the LlasMol example instead returns a reaction-product prompt and product string.Source: §E.4.3 · p. 36

E.4.4 X-Lora-Gemma answers

• For a KRAS G12C selectivity request, X-Lora-Gemma claimed its proposed molecule inhibits the target without interacting with HRAS or NRAS and blocks the G-box to prevent GTP interaction.For a separate BTK request, it claimed high binding affinity, specificity, solubility, and stability for a proposed agent.Source: §E.4.4 · p. 36

E.4.5 Phoenix answers

• For Alzheimer’s disease, Phoenix proposed donepezil and curcumin derivatives with multifunctional pharmacophore rationales; for P-gp, it proposed five structures predicted to block efflux channels.The Alzheimer’s rationales cite AChE inhibition, oxidative-stress reduction, and interference with amyloid-beta aggregation; the P-gp response links channel blocking to intracellular drug retention and resistance reversal.Source: §E.4.5 · pp. 36–37

E.4.6 ChemDFM answers

• ChemDFM’s example prompt directs an expert-chemist model to return only a valid, chemically reasonable SMILES string for six compounds requested to selectively bind KRAS G12C without affecting HRAS or NRAS.Source: §E.4.6 · p. 37
• An example ChemDFM query requests selective tyrosine kinase inhibitors with strong binding affinity for BTK.Source: §E.4.6 · p. 37

E.5 Analyzing the effectiveness of the agent’s generative model selection for training.

• The generative-model training agent receives case-specific architecture performance information based on Table 8 to help select an architecture for a new case.The passage introduces an example prompt containing performance information for the CVAE model.Source: §E.5 · p. 37

F Licenses of used artifacts

• ChemAgent and ChemCrow use MIT licenses; LlaSMol uses CC BY 4.0, and X-LoRA-Gemma uses Apache 2.0.The passage specifies attribution for ChemAgent, credit and change notices for LlaSMol, and retention of copyright notices and license text for X-LoRA-Gemma.Source: F Licenses of used artifacts · p. 38
• DeepSeek, including DeepSeek-Coder and DeepSeek-VL, is licensed under MIT; RDKit uses BSD, and ChEMBL uses CC BY-SA 3.0.The ChEMBL license requires source credit and sharing derivative work under the same license.Source: F Licenses of used artifacts · p. 38
• The listed model and docking-tool licenses are AGPL v3.0 for ChemDFM-13B, Apache 2.0 for ChemLLM-7B-Chat, Mistral-7B, and AutoDock Vina GPU 2.1, and Llama 3.1 Community License for LLaMA-3.1-8B.The SYK article data is identified as CC BY-NC-ND 4.0.Source: F Licenses of used artifacts · p. 38