PaperlayerView exact revision on arXiv

Source-linked AI summary

TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Tools

Shanghua Gao, Richard Zhu, Zhenglun Kong, Ayush Noori, Xiaorui Su, Curtis Ginder, Theodoros Tsiligkaridis, Marinka Zitnik

arXiv:2503.10970v1cs.AIcs.LG

TL;DR

Precision therapy needs models that can reason over patient-specific factors and current biomedical evidence. TXAGENT addresses this with an AI agent that uses multi-step reasoning and biomedical tools, and the paper reports a concrete personalized-treatment interaction analysis while noting that gaps in its toolbox constrain available data.

  • Problem

    Precision treatment decisions must account for patient-specific factors such as age, comorbidities, pregnancy status, disease severity, and immune function.

  • Method

    TXAGENT uses multi-step reasoning and an extensive toolbox to retrieve treatments and evaluate patient-specific drug interactions and contraindications.

  • Results

    TXAGENT determines that Xolremdi should not be used with Prozac because Prozac inhibits CYP2D6 and alters the treatment interaction.

  • Takeaways & Limitations

    TXAGENT generates clinically relevant and personalized treatment recommendations.

  • Takeaways & Limitations

    Gaps in TOOLUNIVERSE restrict access to specific data types and limit TXAGENT’s capabilities.

Abstract

from arXiv · show

Precision therapeutics require multimodal adaptive models that generate personalized treatment recommendations. We introduce TxAgent, an AI agent that leverages multi-step reasoning and real-time biomedical knowledge retrieval across a toolbox of 211 tools to analyze drug interactions, contraindications, and patient-specific treatment strategies. TxAgent evaluates how drugs interact at molecular, pharmacokinetic, and clinical levels, identifies contraindications based on patient comorbidities and concurrent medications, and tailors treatment strategies to individual patient characteristics. It retrieves and synthesizes evidence from multiple biomedical sources, assesses interactions between drugs and patient conditions, and refines treatment recommendations through iterative reasoning. It selects tools based on task objectives and executes structured function calls to solve therapeutic tasks that require clinical reasoning and cross-source validation. The ToolUniverse consolidates 211 tools from trusted sources, including all US FDA-approved drugs since 1939 and validated clinical insights from Open Targets. TxAgent outperforms leading LLMs, tool-use models, and reasoning agents across five new benchmarks: DrugPC, BrandPC, GenericPC, TreatmentPC, and DescriptionPC, covering 3,168 drug reasoning tasks and 456 personalized treatment scenarios. It achieves 92.1% accuracy in open-ended drug reasoning tasks, surpassing GPT-4o and outperforming DeepSeek-R1 (671B) in structured multi-step reasoning. TxAgent generalizes across drug name variants and descriptions. By integrating multi-step inference, real-time knowledge grounding, and tool-assisted decision-making, TxAgent ensures that treatment recommendations align with established clinical guidelines and real-world evidence, reducing the risk of adverse events and improving therapeutic decision-making.

Main

TXAGENT combines multi-step reasoning with real-time biomedical tool integration to produce evidence-grounded therapeutic recommendations. Across five benchmarks, it outperforms larger language models and tool-use systems while supporting drug and personalized treatment reasoning.

  • Approach: TXAGENT combines multi-step reasoning with real-time biomedical tool integration for evidence-grounded treatment recommendations.It performs goal-driven tool selection and uses external databases and specialized models.
  • ToolUniverse: 211 biomedical tools support retrieval across drug mechanisms, interactions, clinical guidelines, disease annotations, and trusted biomedical sources.TOOLUNIVERSE integrates sources including openFDA, Open Targets, and the Human Phenotype Ontology.
  • Benchmark results: 92.1% accuracy was achieved on open-ended DrugPC reasoning, exceeding GPT-4o by 25.8% and Llama-3.1-70B-Instruct by 39.3%.TXAGENT is based on the 8-billion-parameter Llama-3.1-8B-Instruct model.
  • Representation robustness: TXAGENT maintains accuracy variance below 0.01 across brand, generic, and descriptive drug references, compared with 9.96 for GPT-4o.On DescriptionPC, it achieves 56.5% accuracy and surpasses GPT-4o by 8.3%.
  • Personalized treatment: TXAGENT outperforms GPT-4o by 13.6% and Llama-3.1-70B-Instruct by 25.4% on open-ended TreatmentPC scenarios.TreatmentPC evaluates 456 real-world personalized treatment scenarios.
  • Ablation findings: Ablations show that larger toolboxes, tool-assisted decisions, structured reasoning before calls, and more reasoning steps improve performance.Tool-assisted decision-making outperforms LLM-only reasoning, while structured reasoning contributes more than multi-round calls alone.

Results

TXAGENT performs therapeutic reasoning by generating plans, selecting tools dynamically, iterating over tool feedback, and grounding answers in continuously updated biomedical sources. This process produces transparent reasoning traces and supports current, context-aware decisions.

  • Agent workflow: TXAGENT uses multi-step white-box reasoning and tool use to solve precision treatment problems.It generates reasoning traces, function-call arguments, and tool calls during inference.
  • Goal-oriented tool selection: TOOLRAG selects suitable tools from TOOLUNIVERSE based on the query and the agent’s stated objectives.The process supports adaptive tool integration rather than reliance on a static tool set.
  • Knowledge grounding: Function-call outputs support transparent, verifiable answers grounded in trusted biomedical information.Users can inspect reasoning traces and tool feedback to assess response reliability.
  • Multi-step therapeutic reasoning: Iterative reasoning combines multiple tool calls and feedback when direct retrieval is insufficient.For protein-target identification, TXAGENT first retrieves a disease identifier, then selects a target-association tool and ranks proteins.
  • Real-time retrieval: Direct queries to FDA and Open Targets data provide continuously updated approvals, guidelines, and treatment indications without model retraining.The system retrieved current indications for Bizengri, approved after the base model’s knowledge cutoff.

TXAGENT-INSTRUCT dataset of therapeutic tasks and reasoning traces

TXAGENT-INSTRUCT combines tool, therapeutic-question, and reasoning-trace data generated from verified biomedical sources. The resulting benchmarks evaluate drug reasoning across formats and show strong performance in both general and personalized therapeutic tasks.

  • Dataset construction: TXAGENT-INSTRUCT contains 378,027 instruction-tuning samples generated from three datasets.The datasets cover tooling, therapeutic questions, and reasoning traces.
  • Trace generation: TRACEGEN uses helper, tool-provider, solver, and evaluation components to construct and check complex reasoning traces.The evaluation step detects hallucinations, arbitrary outputs, and repetitive reasoning patterns.
  • Reasoning traces: The reasoning-trace dataset includes 85,340 traces, 177,626 reasoning steps, and 281,695 function calls.TRACEGEN produces stepwise traces integrating external tool feedback and final answers.
  • Therapeutic questions: QUESTIONGEN creates therapeutic questions from verified biomedical knowledge and validates them for grounding, solvability, and reasonableness.Its questions address treatments, diseases, drugs, patient populations, side effects, and interactions.
  • Benchmark design: DrugPC evaluates 3,168 questions across 11 drug-reasoning tasks in multiple-choice and open-ended settings.The open-ended format removes answer choices before response generation.

TXAGENT outperforms tool-use LLMs in multi-step reasoning

TXAGENT outperforms tool-use LLMs across multi-step therapeutic reasoning tasks. Its goal-oriented tool selection, iterative calls, and explicit reasoning support stronger performance and drug-name generalization.

  • Benchmark performance: 62.5% and 59.1% are TXAGENT’s multiple-choice gains over ToolACE and WattTool, respectively.In open-ended tasks, the gains are 59.4% and 55.0%.
  • Benchmark performance: 58.9% and 63.1% of multiple-choice questions are invalid for WattTool-8B and ToolACE-8B, respectively.Their open-ended invalid-answer rates are 56.6% and 60.7%.
  • Agent design: TXAGENT combines goal-oriented tool selection, iterative function calls, and multi-step reasoning to use TOOLUNIVERSE for therapeutic reasoning.Existing tool-use models struggle with large toolboxes, single-round calls, and adaptive refinement.
  • Drug-name generalization: 93.6% and 93.7% accuracy are achieved on BrandPC and GenericPC, surpassing pure and tool-use LLMs.The gains over the strongest pure-LLM references are 20.6% and 16.4%.
  • Drug-name generalization: TXAGENT has 0.00667 variance across original, BrandPC, and GenericPC, lower than every listed baseline.The reported variance values are 9.96 for GPT-4o, 2.42 for Llama3.1-70B-Instruct, 13.07 for WattTool-8B, and 1.05 for ToolACE-8B.

Examples of TXAGENT reasoning traces for specialized treatments

Four treatment cases show TXAGENT combining retrieved biomedical evidence with patient-specific constraints. The traces cover pediatric safety, drug interactions, geriatric dosing, and comorbidity-based filtering.

  • Mechanism and pediatric use: TXAGENT identifies Duvyzat as the only candidate meeting a pediatric DMD patient’s criteria and confirms safety for children over six.It then recommends Duvyzat after integrating mechanism and pediatric-use evidence.
  • Drug-drug interactions: Prozac and Xolremdi are judged unsuitable together because both affect CYP2D6-mediated metabolism.Xolremdi is contraindicated with CYP2D6-dependent drugs, while Prozac is both a CYP2D6 substrate and inhibitor.
  • Geriatric use: 100 mg/20 mg twice daily is the maximum geriatric dose of Cobenfy, below the 125 mg/30 mg twice-daily younger-patient dose.The adjustment is attributed to increased urinary-retention risk in elderly patients.
  • Cross-case capability: Parallel tool calls let TXAGENT identify and justify patient-specific treatment recommendations using verified external information.The case studies emphasize simultaneous reasoning threads and evidence-based outputs.
  • Comorbidities: Two hypertension candidates are contraindicated for second-degree AV block and are removed from consideration.TXAGENT retrieves ten candidates by indication, filters them using FDA contraindication labels, and summarizes the remaining mechanisms.

Impact of tools in TOOLUNIVERSE on TXAGENT’s performance

Performance improves as TOOLUNIVERSE expands and as TXAGENT uses real tools, explicit thoughts, and longer reasoning traces. These effects are especially pronounced for complex TreatmentPC tasks.

  • Tool reliability: 93.8% to 68.7% is the DrugPC accuracy drop when real tools are replaced with Llama3.1-8B-Instruct-based tools.Using GPT-4o-based tools yields 72.7%, also below TOOLUNIVERSE.
  • Tool reliability: 86.84% TreatmentPC accuracy with real TOOLUNIVERSE tools exceeds 74.78% with Llama3.1-8B-Instruct and 67.11% with GPT-4o-based tools.Replacing real tools with LLM-based substitutes reduces accuracy and limits precise biomedical retrieval.
  • Tool scaling: DrugPC accuracy rises from 78.4% to 93.8%, while TreatmentPC accuracy rises from 71.7% to 86.8%, as the tool subset expands.The evaluation compares nested TOOLUNIVERSE subsets containing 10%, 20%, 50%, 75%, and all tools.
  • Explicit reasoning: Removing explicit thought generation lowers DrugPC accuracy from 93.8% to 71.5% and TreatmentPC accuracy from 86.4% to 64.9%.TXAGENT differs from function-call-only tool-use models by producing both thoughts and function calls.
  • Training-trace depth: A one-step training trace lowers TreatmentPC accuracy from 86.8% to 66.9% and DrugPC accuracy from 93.8% to 71.6%.The decline is larger on TreatmentPC, whose decisions require stronger multi-step reasoning.
  • Inference-trace depth: 73.5% TreatmentPC accuracy under a one-step inference limit is 13.3% below unrestricted multi-step reasoning.Gains continue up to five steps, after which improvements plateau, although full reasoning capacity remains optimal.

Discussion

TXAGENT combines multi-step reasoning, iterative biomedical tool use, and patient-specific constraints to produce transparent, evidence-based therapeutic recommendations. Its scope remains bounded by tool coverage, uncertainty in internal knowledge, and support for natural-language inputs only.

  • Approach: TXAGENT applies multi-step reasoning and biomedical tool usage to therapeutic problems while producing transparent reasoning traces.It uses external tools to retrieve real-time biomedical knowledge and lets users trace decision steps.
  • Personalized therapy: Patient-specific factors, including age and comorbidities, are incorporated into treatment selection rather than relying solely on disease-drug links.The agent evaluates drug suitability against patient characteristics through adaptive, iterative function calls.
  • Personalized therapy: Xolremdi should not be used with Prozac because CYP2D6 inhibition alters Xolremdi’s metabolism.This example illustrates how TXAGENT integrates drug interactions and patient-specific constraints into therapeutic reasoning.
  • Limitations: TOOLUNIVERSE gaps restrict access to specific data types, limiting TXAGENT’s ability to address a broader range of questions.The system also lacks uncertainty quantification for its internal knowledge.
  • Limitations: TXAGENT processes only natural-language inputs and does not yet support pathology images, EHR data, or web-based laboratory results.The paper identifies multimodal support and extended patient-history memory as future directions.
  • Evidence grounding: Verified sources including FDA drug labels and Open Targets support evidence-based recommendations and access to updated medical knowledge.Tool grounding also enables recommendations involving newly approved drugs and traceable prescriptions.

1 TXAGENT

TXAGENT is an agentic model for complex drug reasoning that combines iterative thought generation, biomedical tool use, and transparent reasoning traces. It selects and executes tools across multiple steps, synthesizes retrieved information, and produces answers with rationales that users can inspect.

  • Transparency and verification: Users can inspect grounded reasoning traces containing thought processes, function calls, and tool-derived information to verify the basis of conclusions.The paper links this knowledge-grounding approach to factual validation, transparency, and reliability.
  • 211 biomedical tools support TXAGENT’s multi-step reasoning and access to high-quality biomedical knowledge.The tools include sources such as FDA drug information.
  • Adaptability and deployment: TXAGENT can incorporate additional tools without retraining and can support local deployment using open-source models for privacy-sensitive applications.The framework is described as adapting to newly added tools and supporting patient-information privacy.
  • Inference process: TXAGENT generates thoughts, function calls, and tool responses that are accumulated into a reasoning trace for subsequent steps.Tool outputs are returned to the agent and incorporated into the updated trace.
  • TXAGENT is an agentic AI model designed to address complex drug reasoning problems expressed in natural language.
  • Logical multi-step reasoning and decision-making: At each reasoning step, TXAGENT analyzes prior traces, decides whether new information is needed, and either generates function calls or produces a final answer.The decision depends on whether the special token [FinalAnswer] is generated.

2 TOOLUNIVERSE

TOOLUNIVERSE provides 211 biomedical tools covering drug, disease, target, safety, and clinical information. TOOLGEN automates tool construction across heterogeneous APIs, while checker and human-verification stages remove or refine invalid tools.

  • TOOLUNIVERSE contains 211 biomedical tools providing real-time information on diseases, drugs, targets, and related biomedical data.
  • Tool coverage: The tools cover categories including adverse events, safety, pharmacology, drug mechanisms, clinical annotations, diseases, targets, publications, and search.
  • Data sources and integration: TOOLUNIVERSE tools are built on APIs from OpenFDA, OpenTargets, and the Monarch Initiative, with descriptions and backend code translating calls into API requests.
  • TOOLGEN: TOOLGEN converts diverse API documentation into tools by summarizing capabilities, generating specifications and mappings, and checking their functionality.Its agents include SUMMARIZER, TOOL GENERATOR, and TOOL CHECKER.
  • Tool validation: Generated tools are tested through mappings, sampled API-linked data, questions, and function calls; tools that fail are marked invalid and removed.
  • Tool validation: Human experts then verify tool usefulness, described functionality, and stability under unexpected inputs before inclusion in TOOLUNIVERSE.
  • Tool graph: The tool graph connects tools when one tool’s output can serve as another’s input and is used to construct multi-round training questions, not during TXAGENT inference.The authors cite difficulty constructing an exceptionally precise graph as the reason for excluding it from inference.
  • Tool graph: Excluding the tool graph from inference allows TXAGENT to integrate newly added tools directly.

3 Constructing TXAGENT-INSTRUCT dataset

The TXAGENT-INSTRUCT dataset is built with multi-agent systems that generate, ground, and evaluate diverse drug and treatment questions with verified reasoning traces. It combines information from FDA and biomedical knowledge resources to cover patient-specific and multi-tool reasoning scenarios.

  • Open-source LLMs are instruction-tuned on the TXAGENT-INSTRUCT dataset to provide comprehensive specialized treatment and drug information.
  • Dataset construction: QUESTIONGEN generates diverse questions from verified information, while TRACEGEN uses a helper agent to produce complex step-wise reasoning traces.
  • Information sources: The dataset draws on FDA drug labeling, Open Targets, the Human Phenotype Ontology, and PrimeKG for drug, disease, phenotype, target, and interaction information.
  • Question types: Questions span drug-centered, disease-centered, and tool-chain-centered formats, including indications, dosage, safety, contraindications, patient profiles, and sequential tool use.
  • Information extraction: Information extraction retrieves drug and disease attributes from source documents before question construction creates questions, answers, explanations, and options.
  • Question evaluation: Generated questions are evaluated for knowledge-based grounding, answerability, and reasonableness using GPT-4o.The checks compare questions with reference information and assess whether their answers and explanations are supported.
  • Reasoning trace generation: Reasoning traces are retained only after checks for answer correctness, function-call correctness, and undesired behaviors.Traces that fail correctness or behavior checks are discarded.

4 Training TXAGENT model

TXAGENT-INSTRUCT trains TXAGENT for multi-step reasoning and function calling through decomposed traces, augmented tools, and supervised fine-tuning. The resulting training process supports tool selection and generalization beyond tools seen during training.

  • Step-wise supervision: Each reasoning trace is decomposed into step-wise supervision pairing the prior context and available tools with the current thought and function calls.The final step additionally outputs the FINISH call and final answer.
  • Step-wise supervision: Training samples expose the system prompt, question, preceding reasoning steps, and available tools, while the final sample also includes the answer.Intermediate outputs are [Ti, Ci], whereas the final output is [TM, CM, A].
  • Tool augmentation: Randomly rephrasing tool names, descriptions, and arguments helps the model call functions from context rather than memorize specific training tools.Each tool receives 20 rewritten versions of its fields, sampled into augmented training examples.
  • Model and optimization: TXAGENT-INSTRUCT trains models to generalize to new and unseen tools, enabling flexible scaling of TOOLUNIVERSE.The training objective uses autoregressive next-token prediction over formatted reasoning samples.
  • Tool augmentation: Tool augmentation teaches TXAGENT to select tools from broader candidate sets by adding retrieved and randomly sampled tools during training.This addresses the challenge of selecting suitable tools when TOOLRAG returns additional candidates.
  • Tool augmentation: Shuffling tool-list positions encourages context-based selection instead of reliance on ordering.Long tool results are also summarized when samples exceed the context limit.
  • Model and optimization: LoRA fine-tuning adapts instruction-tuned Llama-based models with low-rank updates that reduce computational and parameter costs.The approach preserves knowledge from the pretrained model while adapting it to TXAGENT-INSTRUCT.

5 Benchmarking TXAGENT

The evaluation suite tests TXAGENT across comprehensive drug reasoning, name-variant robustness, description-based identification, and personalized treatment selection. It combines multiple-choice and open-ended evaluation, with accuracy as the reported metric.

  • Benchmark suite: Five benchmarks evaluate drug reasoning across DrugPC, BrandPC, GenericPC, DescriptionPC, and TreatmentPC.DrugPC covers 11 therapeutic task categories, including safety, dosage, pharmacology, and patient-focused information.
  • Benchmark suite: DrugPC contains 3,168 questions covering 11 common therapy-related tasks.Questions are constructed from FDA document sections and reviewed through human evaluation.
  • Name-variant robustness: BrandPC and GenericPC replace drug names with brand or generic forms to test robustness to naming variation.Questions requiring name conversion retain the corresponding brand-generic relationship.
  • Description-based reasoning: DescriptionPC replaces explicit drug names with descriptions containing indications, mechanisms, contraindications, and interactions.The resulting benchmark contains 626 questions and evaluates whether reasoning is based on the intended drug description.
  • Description-based reasoning: DescriptionPC uses two-step evaluation: identify the drug from its description, then answer the associated multiple-choice question.An incorrect drug identification automatically makes the second step incorrect.
  • Personalized treatment: TreatmentPC tests personalized treatment selection under conditions such as pregnancy, comorbidities, contraindications, and concurrent medications.Each question presents treatments for a disease, with only one suitable for the patient’s specific condition.
  • Personalized treatment: TreatmentPC questions are generated from FDA indications, population restrictions, warnings, precautions, and contraindications to distinguish similar treatments.The benchmark requires analysis of the patient’s condition before selecting an appropriate solution.
  • Evaluation protocol: Open-ended evaluation omits answer options and uses an additional evaluation step because generated answers are inherently challenging to assess.Both multiple-choice and open-ended evaluations report benchmark accuracy.

6 Settings for analysis of TXAGENT

The analysis settings compare TOOLUNIVERSE with simulated LLM-based tools and isolate the contribution of intermediate reasoning thoughts. A no-thought variant directly generates function calls while retaining iterative tool execution and termination.

  • Tool comparison: In TOOLUNIVERSE versus LLM-as-tools comparisons, an LLM receives TXAGENT’s function-call arguments and the retrieved tool description to simulate tool outputs.The simulated tool is instructed to answer only from its description and input.
  • Tool comparison: The LLM uses TOOLUNIVERSE descriptions as references while producing outputs for function calls generated by TXAGENT.This setup evaluates tool use with language-model-simulated responses.
  • Reasoning ablation: A modified TXAGENT version removes explicit reasoning thoughts to test their role in the agent’s performance.The variant preserves the multi-step inference framework but directly produces function calls.
  • Reasoning ablation: The no-thought process initializes an empty reasoning trace, an initial tool set, and a step counter before iterative inference.At each iteration, the model generates function calls or a final answer.
  • Reasoning ablation: Textual model output becomes the final answer and triggers FINISH, whereas function-call arguments are executed and can update available tools through TOOLRAG.The process returns the reasoning trace and final answer when termination occurs.

7 Prompt sketches

The prompt sketches specify how TXAGENT, tool generators, question generators, and trace generators produce tool-using biomedical training data. They emphasize structured functions, patient-specific treatment comparisons, and one-step reasoning guidance.

  • 7.1 System prompt for TXAGENT: TXAGENT is prompted to solve problems through detailed step-by-step reasoning and actions using provided functions.The system prompt exposes the available functions to the agent.
  • 7.2 Prompts for TOOLGEN: TOOLGEN prompts require functions to follow API schemas, define searchable and return fields, and provide unique capabilities aligned with the source schema.The prompts cover OpenTarget and openFDA tool generation.
  • 7.2 Prompts for TOOLGEN: OpenFDA tool generation produces paired functions for retrieving drug names from field information and retrieving field information from drug names.Generated functions must align with expected fields and differ from existing examples.
  • 7.2 Prompts for TOOLGEN: Tool-chain question prompts ask for specific, diverse biomedical questions that require using as many relevant functions as possible.Questions are based on generated functions and related keywords while requiring named arguments in calls.
  • 7.3 Prompt for QUESTIONGEN: Personalized-treatment prompts construct patient profiles using diseases, phenotypes, genetic variation, interactions, warnings, restrictions, pregnancy, and contraindications.They compare similar drugs and make one the most suitable answer for the patient’s profile.
  • 7.3 Prompt for QUESTIONGEN: Treatment-question prompts distinguish drugs by side effects, interactions, contraindications, age, comorbidities, dosage, and other patient-specific factors.Incorrect options may treat the disease but remain unsuitable for the particular patient.
  • 7.3 Prompt for QUESTIONGEN: Drug-centered prompts generate multiple non-overlapping questions from drug names and field information without requiring information beyond the supplied fields.They support both generic and brand drug names and fields such as contraindications.
  • 7.4 Prompts for TRACEGEN: TRACEGEN provides one-step solution hints without revealing the final answer or information that directly leads to it.The prompts also require the eventual final answer to contain the correct answer.
Loading 2503.10970v1…