PaperlayerView exact revision on arXiv

Source-linked AI summary

ClinAgent: A ReAct-Based Agent for Conversational Access to Clinical Trial Information

Antonino Vaccarella, Riccardo Cantini, Domenico Talia, Paolo Trunfio, Marianna Talia, Rosamaria Lappano, Marcello Maggiolini

arXiv:2609.13860v1cs.AIcs.CLcs.IR

TL;DR

Clinical trial registry queries remain largely manual and lack natural-language interaction and cross-source synthesis, while clinical use requires grounded responses. ClinAgent combines agentic RAG with registry, literature, and trial-data analysis tools, and evaluation finds Gemini strongest overall while thinking-mode DeepSeek excels in planning quality.

  • Problem

    Clinical trial registries are queried manually, without natural-language interaction or a mechanism to synthesize information across records and scientific literature.

    Source: §1 · p. 1

  • Method

    ClinAgent uses an LLM agent following a ReAct loop to coordinate registry search, PubMed literature retrieval, and Python analysis of locally cached trial data.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Gemini 3.0 Flash achieves the highest overall score and strongest expert ratings, while reasoning-enabled DeepSeek V3.2 offers superior multi-step planning at lower factual reliability.

    Source: §5 · p. 6

  • Takeaways & Limitations

    The authors recommend Gemini 3.0 Flash for deployment in this setting, while backend selection must weigh its overall performance against DeepSeek’s planning advantage and factual-reliability trade-off.

    Source: §5 · p. 6

    The stated future extension is to add domain-specific sources such as regulatory databases and institutional electronic health records.

    Source: §5 · p. 6

Abstract

from arXiv · show

Querying clinical trial registries remains a manual and error-prone process, requiring researchers to navigate large volumes of semi-structured data without support for natural language interaction or cross-source synthesis. To address this, we introduce ClinAgent, a conversational system based on agentic Retrieval-Augmented Generation (RAG) that enables clinicians and researchers to query clinical trial information in plain language and receive grounded, up-to-date responses across multi-turn interactions. The system centers on a Large Language Model (LLM) agent following the ReAct paradigm, which iteratively reasons over queries, selects among a set of integrated tools, and refines its actions based on intermediate outputs. These tools include a ClinicalTrials.gov search interface, a PubMed module, and a Python-based analyzer operating on a locally cached structured dataset of clinical trials. We evaluate the system using a three-phase framework assessing operational effectiveness, planning quality, tool-use efficiency, and expert qualitative judgments, comparing three LLM backends: Gemini 3.0 Flash and two variants of DeepSeek V3.2 (thinking and non-thinking). Results reveal complementary strengths, with DeepSeek (thinking mode) excelling in planning quality, while Gemini achieves the highest overall performance and strongest expert ratings. Overall, our findings highlight the potential of agentic AI systems to improve the accessibility and synthesis of clinical trial information, supporting more efficient and user-centered biomedical research workflows.

1 Introduction

Clinical trial registry queries remain manual and lack natural-language interaction or cross-record synthesis, while LLM hallucinations are unacceptable in clinical contexts. ClinAgent combines agentic RAG and three tools; evaluation finds complementary backend strengths, with Gemini strongest overall and thinking-mode DeepSeek best in planning quality.

1 Introduction

  • Registry users rely on keyword searches or CSV inspection, without natural-language interaction or synthesis across records and scientific literature.ClinicalTrials.gov indexes over a million studies across therapeutic areas and phases.Source: §1 · p. 1
  • LLM hallucinations make ungrounded outputs unacceptable in clinical contexts, while single-pass RAG limits multi-source retrieval and iterative refinement.The passage describes ReAct as alternating reasoning, tool use, and observation until sufficient evidence is accumulated.Source: §1 · pp. 1–2
  • Gemini 3.0 Flash achieves the highest overall score and strongest expert ratings, while thinking-mode DeepSeek leads on plan quality.The paper describes agentic RAG as practically viable for clinical trial information retrieval and identifies a trade-off between reasoning depth and factual reliability in backend selection.Source: §1 · p. 2
  • ClinAgent integrates ClinicalTrials.gov search, PubMed literature access, and Python analysis of a locally cached structured trial dataset under one conversational agent.The paper also proposes a three-phase evaluation framework covering operational effectiveness, plan quality and tool-use efficiency, and expert assessment.Source: §1 · p. 2

2 Background and Related Work

Prior work applies retrieval grounding to medical question answering, but the reviewed systems do not combine conversational trial-registry querying with agentic multi-source synthesis. ClinAgent is positioned as addressing that gap while differing from patient-matching systems such as TrialGPT.

2 Background and Related Work

  • OpenEvidence grounds general clinical answers in PubMed and guidelines, but does not support structured registry queries, trial-record analysis, or agentic multi-step reasoning across sources.The passage characterizes RAG as grounding outputs in dynamically retrieved documents to improve factual accuracy and access to post-training knowledge.Source: §2 · p. 2
  • TrialGPT matches patients to eligible trials, whereas ClinAgent addresses open-ended natural-language queries through grounded synthesis across heterogeneous sources.TrialGPT evaluates eligibility criteria and aggregates judgments into a trial-level score.Source: §2 · p. 2
  • The paper identifies a gap in agentic retrieval across both trial registries and biomedical literature for conversational clinical information access.The authors connect this capability to clinicians’ and researchers’ need for timely, grounded trial evidence without manual registry navigation and inspection.Source: §2 · p. 3

3 Proposed System

ClinAgent uses a ReAct reasoning–action–observation loop to plan and iteratively carry out clinical-trial queries. Its tools support registry retrieval, local dataset analysis, and literature search, with replanning and persisted conversation state supporting multi-turn use.

3 Proposed System

  • The agent interprets a query, decomposes it into sub-goals, invokes tools, and checks whether accumulated evidence is sufficient before responding.Figure 1 depicts the closed loop selecting among three specialized tools during multi-turn interaction.Source: §3 · p. 3, Figure 1 · p. 3
  • The three tools retrieve ClinicalTrials.gov records, analyze the locally cached trial dataset with Python, and search PubMed for biomedical literature.Registry queries are translated into API requests, and retrieved records are cached for reuse during the session.Source: §3 · p. 3
  • When a primary tool fails or returns insufficient information, the agent replans and tries an alternative strategy without external intervention.The paper characterizes this as an iterative, self-correcting approach for complex and evolving queries.Source: §3 · p. 3
  • Conversation state persists across requests, allowing the agent to resolve follow-up queries by referring to earlier exchanges.The workflow uses LangChain and LangGraph, with MemorySaver persisting state across successive requests.Source: §3 · p. 4

4 Experimental Evaluation

Across the three evaluation phases, Gemini 3.0 Flash achieved the highest aggregate score, while the DeepSeek variants showed different strengths in planning and expert-rated responses. The evaluation covered operational effectiveness, plan quality and tool-use efficiency, and expert assessment.

4 Experimental Evaluation

  • The evaluation compared Gemini 3.0 Flash with DeepSeek V3.2 thinking and non-thinking across factual, comparative, literature-based, and mixed queries.Its three phases assessed operational effectiveness, plan quality and tool-use efficiency, and expert judgments.Source: §4 · p. 4

4.4 Overall Results

  • Gemini 3.0 Flash scored 0.80 overall, versus 0.72 for DeepSeek V3.2 non-thinking and 0.71 for thinking mode, with the thinking variant stronger in planning but lower in factual reliability.Source: Table 1 · p. 4, §4.4 · p. 6

4.1 Phase 1 — Operational Effectiveness

  • Gemini achieved a Goal Achievement Rate (GAR) of 1.00; DeepSeek V3.2 thinking and non-thinking scored 0.94 and 0.88, respectively.Failures were primarily attributed to incomplete tool invocations or premature termination before sufficient evidence was gathered.Source: §4.1 · p. 5

4.2 Phase 2 — Plan Quality and Tool-Use Efficiency

  • DeepSeek V3.2 thinking achieved the highest Plan Quality (0.97), while Gemini led Tool Use Efficiency (0.56) and composite ReAct Quality (0.75), narrowly ahead of thinking mode (0.73).Plan Quality credits successful, non-redundant actions while accounting for recovery; Tool Use Efficiency measures first-attempt tool successes.Source: §4.2 · p. 5

4.3 Phase 3 — Expert Qualitative Assessment

  • Gemini showed the most balanced expert-rated profile, leading in logical consistency and clarity but scoring lower in completeness; DeepSeek variants had contrasting strengths and weaknesses across criteria.Source: §4.3 · p. 5

5 Conclusion

The conclusion presents ClinAgent as a ReAct-based conversational system for clinical-trial information retrieval and reports trade-offs among three LLM backends. Gemini had the highest overall score and strongest expert ratings, while DeepSeek thinking showed a planning advantage alongside lower factual reliability.

  • Gemini 3.0 Flash had the highest overall score and strongest expert ratings, while DeepSeek thinking offered better multi-step planning at the cost of lower factual reliability.The authors identify additional domain-specific sources, including regulatory databases and institutional electronic health records, as future work.Source: §5 · p. 6

Conflict of interests

The authors report no conflicts of interest. The passage also acknowledges project funding and provides the ClinAgent source-code location.

Loading 2609.13860v1…