Source-linked AI summary
TxGemma: Efficient and Agentic LLMs for Therapeutics
Eric Wang, Samuel Schmidgall, Paul F. Jaeger, Fan Zhang, Rory Pilgrim, Yossi Matias, Joelle Barral, David Fleet, Shekoofeh Azizi
TL;DR
Therapeutic development needs more efficient ways to evaluate diverse properties amid costly experiments and high failure rates. This paper introduces TxGemma, generalist therapeutic LLMs with predictive and conversational capabilities, plus Agentic-Tx for tool-using workflows. Across 66 tasks, TxGemma matches or surpasses leading generalist and specialist models, while its open release supports adaptation and validation by researchers.
Problem
Therapeutic development is costly and high-risk, requiring evaluation of diverse properties through complex experimental procedures.
Method
TxGemma fine-tunes 2B, 9B, and 27B Gemma-2 models on therapeutic instruction-tuning data to provide generalist prediction and conversational reasoning.
Results
TxGemma matches or surpasses leading generalist and specialist models across a broad set of 66 therapeutic development tasks.
Takeaways & Limitations
TxGemma’s open release enables researchers to adapt and refine the models on private data, while Agentic-Tx supports complex multi-step reasoning workflows.
Takeaways & Limitations
The models have not yet been validated in real-world wet-lab experiments, making prospective validation a crucial next step.
Abstract
from arXiv · showhide
Therapeutic development is a costly and high-risk endeavor that is often plagued by high failure rates. To address this, we introduce TxGemma, a suite of efficient, generalist large language models (LLMs) capable of therapeutic property prediction as well as interactive reasoning and explainability. Unlike task-specific models, TxGemma synthesizes information from diverse sources, enabling broad application across the therapeutic development pipeline. The suite includes 2B, 9B, and 27B parameter models, fine-tuned from Gemma-2 on a comprehensive dataset of small molecules, proteins, nucleic acids, diseases, and cell lines. Across 66 therapeutic development tasks, TxGemma achieved superior or comparable performance to the state-of-the-art generalist model on 64 (superior on 45), and against state-of-the-art specialist models on 50 (superior on 26). Fine-tuning TxGemma models on therapeutic downstream tasks, such as clinical trial adverse event prediction, requires less training data than fine-tuning base LLMs, making TxGemma suitable for data-limited applications. Beyond these predictive capabilities, TxGemma features conversational models that bridge the gap between general LLMs and specialized property predictors. These allow scientists to interact in natural language, provide mechanistic reasoning for predictions based on molecular structure, and engage in scientific discussions. Building on this, we further introduce Agentic-Tx, a generalist therapeutic agentic system powered by Gemini 2.5 that reasons, acts, manages diverse workflows, and acquires external domain knowledge. Agentic-Tx surpasses prior leading models on the Humanity's Last Exam benchmark (Chemistry & Biology) with 52.3% relative improvement over o3-mini (high) and 26.7% over o3-mini (high) on GPQA (Chemistry) and excels with improvements of 6.3% (ChemBench-Preference) and 2.4% (ChemBench-Mini) over o3-mini (high).
1 Introduction
Therapeutic development is costly and failure-prone because evaluating efficacy, safety, stability, pharmacokinetics, and developability requires extensive experiments. TxGemma addresses this need with generalist predictive, conversational, and agentic models spanning therapeutic workflows.
- Therapeutic development faces high attrition, lengthy timelines, and costly experiments across efficacy, safety, stability, pharmacokinetics, and developability.
- TxGemma comprises 2B, 9B, and 27B parameter generalist models fine-tuned from Gemma-2 on therapeutic data covering diverse biomedical entities.
- TxGemma-Predict outperforms or nearly matches the state-of-the-art generalist model on 64 of 66 tasks, including 45 where it outperforms it.
- TxGemma-Chat enables natural-language interaction, explanations for predictions based on molecular structure, and scientific discussion.
- Agentic-Tx integrates TxGemma models as tools within a Gemini 2.5 system for complex, multi-step therapeutic problems.
- The models are released openly using commercially licensed datasets so researchers can adapt and refine them on their own data.
2 Methods
The authors construct TxGemma from Therapeutic Data Commons instruction-tuning data, fine-tune Gemma-2 predictive and conversational variants, and evaluate them with task-specific metrics and cross-task statistical tests. Agentic-Tx extends this setup through ReAct-based tool orchestration and external knowledge access.
- Data and instruction tuning: The Therapeutic Data Commons supplies 66 datasets spanning drug discovery and development, with more than 15 million biomedical datapoints.
- Data and instruction tuning: Each training example combines an instruction, biochemical context, a therapeutic-property question, and a task-specific answer format.
- Data and instruction tuning: The resulting splits contain 7,080,338 training, 956,575 validation, and 1,917,297 test examples using TDC-recommended split strategies.
- Modeling: Predictive models fine-tune Gemma-2 at 2B, 9B, and 27B parameters across all TDC tasks, while conversational models mix 30% therapeutic and 70% general instruction-tuning data.
- Evaluation: Evaluation uses TDC-preferred metrics, 10-shot nearest-neighbor prompting, bootstrapping, and Wilcoxon signed-rank tests for overall cross-task comparisons.
- Agentic system: Agentic-Tx uses ReAct to interleave reasoning and tool use across 18 tools for external knowledge, molecular transformations, and TxGemma-based predictions.
3 Results
TxGemma-27B-Predict is competitive with therapeutic generalist and specialist models across TDC tasks, while conversational variants add reasoning and explanation capabilities. Agentic-Tx extends these capabilities to tool-using workflows and achieves strong results on chemistry and biology benchmarks.
- Comparison with therapeutic generalist models: 45 of 66 tasks outperform Tx-LLM M, and 62 outperform Tx-LLM S, with aggregate improvement significant at p=0.003.The comparison uses a Wilcoxon signed-rank test over task-level performance.
- Comparison with specialist models: 26 tasks outperform best-in-class specialist models, while 50 perform near SOTA.Near SOTA is defined in the cited results as performance close to the specialist benchmark.
- Small-molecule comparisons: TxGemma-27B-Predict and TxGemma-9B-Predict outperform LlaSMolMistral on 2 of 5 shared tasks and fall within 95% CIs on 2 additional tasks.LlaSMolMistral is the top-performing 7B model in the LlaSMol suite.
- Conversational capabilities: TxGemma-27B-Chat provides mechanistic BBB reasoning involving lipophilicity, molecular weight, hydrogen bonding, and charged groups.The model connects these molecular properties to the likelihood of crossing the blood-brain barrier.
- Agentic-Tx: Agentic-Tx improves over o3-mini (high) by 52.3% on HLE Chemistry & Biology, 6.3% on ChemBench-Preference, 2.4% on ChemBench-Mini, and 26.7% on GPQA.Agentic-Tx uses tools and workflow orchestration for reasoning-intensive chemistry and biology tasks.
4 Related work
Related work spans task-specific chemistry and therapeutic models, biology-focused language and structure models, and agentic systems that combine reasoning with external tools for scientific workflows.
- Graph neural networks and molecular fingerprints are established representations for small-molecule prediction and generation tasks.
- Specialized biological models have advanced drug repurposing, protein structure prediction, protein design, and structure-based drug development.
- Large language models have been applied to chemistry, neuroscience, molecular, protein, and other scientific data through fine-tuning or multimodal objectives.
- Agentic systems differ from passive models by planning, selecting actions, interacting with external tools, and supporting scientific or biomedical workflows.
5 Discussion
The discussion positions TxGemma as a lightweight generalist therapeutic model with predictive, explanatory, and workflow-orchestration capabilities. It also emphasizes data efficiency and the need for wet-lab validation before real-world therapeutic use.
- TxGemma achieves competitive performance across diverse therapeutic tasks despite specialist models retaining value for complex domain-specific challenges.
- TxGemma-Chat provides reasoning for predictions and supports molecularly grounded explanations, but its explanations are correlations rather than necessarily causal accounts.
- Agentic-Tx combines TxGemma with external tools to support multi-step reasoning, flexible workflows, and access to up-to-date domain information.
- Adding textual clinical-trial context improves overall results but dilutes the relative influence of molecular representations.
- TxGemma’s in-silico performance has not yet been validated in real-world wet-lab experiments, making prospective validation a stated next step.
6 Conclusion
TxGemma combines broad therapeutic prediction with conversational reasoning and explanations, while Agentic-Tx extends these capabilities to complex, multi-step scientific problems. The work emphasizes more efficient, transparent, and collaborative AI-driven therapeutic research.
- TxGemma achieves strong performance across predictive and generative therapeutic tasks, surpassing or matching generalist and specialist state-of-the-art models.
- Conversational TxGemma models provide reasoning and explanations that move beyond black-box predictions toward mechanistic understanding and scientific discourse.
- Agentic-Tx demonstrates the capacity to solve complex, multi-step problems and achieves state-of-the-art results on challenging reasoning-intensive tasks.
- The open release of TxGemma allows researchers and scientists to adapt and refine the models using private data.
Inclusion and ethics
The paper highlights ethical and transparency concerns surrounding therapeutic AI, especially potential bias in training data and the difficulty of explaining complex models. It also notes that the study remains at the research stage and relies on publicly available datasets and components.
- Biases in training data can lead to inequities, motivating the use of diverse datasets and explainable AI systems.
- The model remains in the research stage and requires continued development and refinement.
- The authors acknowledge that explaining the inner workings of complex models remains difficult.
- The TDC and benchmarking datasets used in the study are publicly available through their respective websites.
- All components and data needed to reproduce the work are publicly accessible, with technical methods and data-curation details documented.
Version control
The revised work upgrades Agentic-Tx orchestration and adds results for models trained solely on commercially licensed datasets. These changes expand the reported evaluation of workflow performance and licensing-constrained training settings.
- Agentic-Tx’s orchestrator was upgraded from Gemini 2.0 to Gemini 2.5, improving complex workflow orchestration performance.
- The revision adds binary classification, regression, and generation results for TxGemma-Predict and TxGemma-Chat trained only on commercially licensed datasets.
A Summary
The supplementary materials document excluded tasks, dataset splits and descriptions, feature types, modeling details, statistical testing, additional prediction results, and extensive comparisons and examples. They also cover conversational behavior, agentic tool use, contamination analyses, inference speed, and a therapeutic proof-of-concept.
- Data details: The supplement lists excluded TDC tasks and provides sample counts for training, validation, and test sets across classification, regression, and generation tasks.
- Data details: It describes binary classification, regression, and generation tasks, including the processed TDC feature types and task-size distributions.
- Method and modeling details: Method materials include task prompts, few-shot examples, clinical adverse-event prompts, an Agentic-Tx response example, available tools, and Tanimoto-similarity analysis for the AMES task.
- Method and modeling details: The supplement documents the Wilcoxon signed-rank test used to assess model performance.
- Additional results: Additional results cover specialist comparisons, LlaSMol and MolE comparisons, inference speed, contamination analyses, feature-type and model-size effects, and toxicity and clinical-trial correlations.
- Additional results: Further materials examine conversations with TxGemma models, MMLU comparisons, Agentic-Tx tool-use frequencies, and a therapeutic end-to-end proof-of-concept.
B Data details
The study assembles diverse therapeutic-development tasks from TDC, with substantial variation in task types, inputs, and dataset sizes. Outputs use standardized formats for classification, regression, and molecular generation.
- Task selection: The study excludes tasks because of relevance, LLM limitations, unclear metrics, redundancy, or other data characteristics.Quantum-property tasks such as QM7b, QM8, and QM9 were excluded as not directly relevant.
- Dataset sizes: Binary classification datasets range from 196 to 1,406,988 samples, while regression and generation datasets range from 345 to 775,767.The ranges reflect substantial variability in data availability across tasks.
- Task coverage: TDC tasks span gene–disease associations, protein interactions, antibody affinity, drug–target interactions, screening, and therapeutic-specific problems.Examples include CRISPR repair, peptide–MHC binding, miRNA–target interaction, and TCR–epitope binding.
- Input representations: Inputs include small molecules, proteins, nucleic-acid-related entities, diseases, and cell lines, with specialized representations for MHC molecules and TCRs.MHCs use contact-residue pseudo-sequences, while TCRs use CDR3 hypervariable loops.
- Output formats: Binary classification outputs are “(A)” for negative and “(B)” for positive answers.This convention is used across the processed classification prompts.
- Output formats: Regression outputs are integers from 0 to 1000 that map back to task-specific labels, whereas USPTO generation outputs predicted molecular SMILES strings.The integer representation is a binned transformation of the original numeric label space.
C Method details
TxGemma represents therapeutic tasks as structured text prompts containing instructions, contextual background, questions, and formatted answers. Few-shot examples are sampled randomly during training but use nearest neighbors for evaluation.
- Prompt construction: Each therapeutic prompt contains an instruction, biochemical context, a property question, and a task-specific answer format.Answers are formatted as binary labels, binned continuous values, or SMILES strings for generation.
- Few-shot prompting: Training uses random examples for few-shot learning, while evaluation uses nearest-neighbor shots.The training choice reflects higher within-training-set similarity than similarity between training and test sets.
C.1 Aggregated method comparison
The evaluation compares TxGemma with specialist and generalist baselines across therapeutic prediction and generation tasks using task-specific metrics and statistical normalization. The prompts cover diverse therapeutic inputs and outputs, while conversational behavior is assessed alongside predictive performance.
- Task prompts: TxGemma prompt examples cover drug responses, drug–target interactions, retrosynthesis, adverse-event prediction, clinical trials, and chemical-preference questions.Inputs include molecular strings, cell-line descriptions, trial information, and reaction products.
- Agentic evaluation: Agentic-Tx uses diverse tools and adjusts tool usage across benchmarks, with molecule-based tools used consistently for chemical-preference questions.The system’s tool descriptions and example prompts are provided in supplementary materials.
- Baselines: Performance comparisons use specialist state-of-the-art values from TDC leaderboards or literature, alongside generalist and prior therapeutic LLM baselines.Detailed comparisons include TxGemma-Predict, TxGemma-Chat, Gemma-2, and Tx-LLM.
D.4 Proof-of-concept use of TxGemma for end-to-end therapeutic development
A proof-of-concept applies TxGemma to sequential target identification and therapeutic selection for ovarian cancer, while supplementary analyses characterize predictive, conversational, and general-purpose behavior. The demonstration is explicitly limited in scope and requires experimental validation.
- Target identification: TxGemma identifies PIK3CA as an ovarian-cancer-associated target with a 0.7 association score, nearly 2.5 standard deviations above the mean.PIK3CA was absent from the training set; JAK2 and RET each received 0.3.
- Therapeutic selection: The model selects candidate therapeutics by prioritizing predicted IC50 against the oncogenic E545K PIK3CA mutant, toxicity, and clinical-trial success.The shortlist includes alpelisib, afatinib, and a randomly generated novel molecule, all unseen during training.
- Limitations: The proof-of-concept limits clinical-trial predictions to Phase 1 success and mutagenicity predictions to only part of small-molecule toxicity.The authors call for experimental validation and additional toxicity factors, including hematologic toxicity.
- Model comparison: TxGemma-27B-Predict is compared with generalist and specialist state-of-the-art models across binary classification, regression, and generation tasks.Supplementary tables report comparisons against specialist models, Gemma-2, TxGemma-Chat, and Tx-LLM.
- Small-molecule comparison: TxGemma performs comparably to LlaSMol and MolE on small-molecule tasks, outperforming MolE on 10 of 22 reported tasks.Figure S.5 reports accuracy for binary classification and RMSE for regression, while Figure S.6 separates MAE, AUROC, Spearman, and AUPRC.
- Conversational behavior: TxGemma-27B-Chat provides conversational reasoning for therapeutic predictions, whereas TxGemma-27B-Predict can struggle with follow-up dialogue outside its processed prompt format.The conversational model explains a blood-brain-barrier prediction after a follow-up rationale request.