Source-linked AI summary
Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis
Ayhan Can Erdur, Daniel Scholz, Jiazhen Pan, Benedikt Wiestler, Daniel Rueckert, Jan C. Peeken
TL;DR
Current LLMs lack native 3D reasoning for volumetric medical imaging, and agentic neuro-radiology workflows remain underexplored. This paper evaluates a training-free brain MRI tool-orchestration framework; results show high query inclusion and a handoff design that matches or exceeds single-agent accuracy at lower token cost.
Problem
Current LLMs lack native 3D volumetric reasoning, and agentic workflows for neuro-radiology remain largely unexplored.
Source: §1 · p. 2Method
The paper benchmarks training-free LLM agents that orchestrate brain MRI tools across single- and multi-timepoint workflows and RANO assessment.
Source: §1 · p. 2, §3.3 · p. 5Results
Query inclusion approaches 1.0 across tasks, while handoffs matches or exceeds single-agent accuracy with lower input and output token costs.
Source: §4 · pp. 6–7Takeaways & Limitations
The findings demonstrate that multiple general-purpose LLMs can perform complex neuro-radiological analysis through tool use without training or fine-tuning.
Source: §1 · p. 2Performance, latency, and supported workflows are constrained by the underlying tools, and the study focuses only on brain MRI.
Source: §5 · p. 8
Abstract
from arXiv · showhide
State-of-the-art large language models (LLMs) show high performance in general visual question answering. However, a fundamental limitation remains: current architectures lack the native 3D spatial reasoning required to directly analyze volumetric medical imaging, such as CT or MRI. Emerging agentic AI offers a new solution, eliminating the need for intrinsic 3D processing by enabling LLMs to orchestrate and leverage specialized external tools. Yet, the feasibility of such agentic frameworks in complex, multi-step radiological workflows remains underexplored. In this work, we present a training-free agentic pipeline for automated brain MRI analysis. Validating our methodology on several LLMs (GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6) with off-the-shelf domain-specific tools, our system autonomously executes complex end-to-end workflows, including preprocessing (skull stripping, registration), pathology segmentation (glioma, meningioma, metastases), and volumetric reasoning. We evaluate our framework across increasingly complex radiological tasks, from single-scan tumor and anatomy analysis to longitudinal response assessment requiring multi-timepoint comparisons. We analyze the impact of architectural design by comparing single-agent models against multi-agent "domain-expert" collaborations. To support rigorous evaluation of future agentic systems, we release a benchmark dataset of image-prompt-answer tuples derived from public data. Our results demonstrate that agentic AI can solve highly neuro-radiological image analysis tasks through tool use without the need for training or fine-tuning.
1 Introduction
Clinical MRI workflows require labor-intensive, multi-step analysis, while current LLMs lack native 3D reasoning. The paper introduces training-free agentic orchestration and benchmarks architectures for brain MRI analysis.
1 Introduction
- Quantitative MRI workflows require hours of expert annotation and user-managed orchestration, limiting broader clinical adoption.Rule-based scripts become difficult to extend as task domains, inputs, and user needs expand.Source: §1 · p. 2
- Recent LLMs and medical variants support natural-language interaction, 2D visual question answering, and diagnostic summarization.These capabilities have lowered barriers to computational workflows and extended language models to vision.Source: §1 · p. 2
- Current LLMs lack native reasoning for 3D volumes, and agentic neuro-radiology workflows remain largely unexplored.Agentic systems can orchestrate external algorithms, but their use in neuro-radiology has received limited study.Source: §1 · p. 2
- The paper introduces a training-free brain MRI pipeline, compares single- and multi-agent architectures, and releases a benchmark dataset.The evaluation spans single- and multi-timepoint volumetric analysis and RANO assessment across GPT-5.4, Gemini 3.1 Pro, and Claude Sonnet 4.6.Source: §1 · p. 2
2 Related Work
Prior agentic imaging systems cover 2D imaging, broader quantification, or diagnostic dialogue, while this work targets operational 3D brain MRI workflows. Existing neuro-oncology benchmarks measure static 2D VQA rather than tool planning and iterative use.
Agentic AI in Medical Imaging
- Prior agentic systems address 2D chest X-ray VQA, pathology and radiology quantification, or multi-agent diagnostic dialogue; this work targets multi-step 3D brain MRI.The paper positions its contribution as training-free LLM orchestration for operational neuro-radiology.Source: §2 · p. 3
Benchmarking in Neuro-Oncology Workflows
- OmniBrainBench and BraTS-derived QA assess static, single-step VQA on 2D representations, not tool planning or iterative tool invocation.The cited benchmarks also do not focus on 3D imaging.Source: §2 · p. 3
3 Materials and Methods
The methods describe a text-mediated LLM framework that orchestrates local brain MRI tools, compares single- and multi-agent designs, and benchmarks increasingly complex tasks. Evaluation measures workflow fidelity, computational cost, and response quality.
3 Materials and Methods
- The framework plans and executes tools for 3D MRI queries, while images stay local and the LLM interacts through text and object pointers.Intermediate segmentation and measurement outputs can be returned for continued interaction.Source: §3 · p. 3, Figure 1 · p. 4
3.1 Available Tools
- The 24-tool set spans preprocessing, tumor and anatomical segmentation, lesion analysis, registration utilities, retrieval, and visualization.Tools include skull stripping, registration, segmentation models, longitudinal lesion matching, and volumetric measurements.Source: §3.1 · pp. 3–4
- Atomic tools are exposed to the agent to avoid redundant actions and intermediate products, and to stress-test planning and ease future tool integration.Multi-agent variants specialize preprocessing, segmentation, and analysis; coordination uses agents-as-tools or handoffs.Source: §3.1 · p. 4, §3.2 · p. 4
3.2 Agent Architectures
- In agents-as-tools, agents issue structured requests and responses; handoffs transfer the full execution context and history to another agent.Registration verification is assigned to segmentation so prerequisite checks remain local to that agent; remaining utility tools go to analysis.Source: §3.2 · pp. 4–5
3.3 Dataset
- The benchmark records questions, expected tool-call plans, and keyword-value answers, with architecture-specific ground-truth sequences and tasks from single-timepoint analysis to RANO assessment.It includes single-timepoint measurements, longitudinal response analysis, and RANO assessment; the imaging cohort includes several tumor types and longitudinal cases with expert RANO ratings.Source: §3.3 · pp. 5–6
3.4 Evaluation
- Evaluation measures tool-call precision and recall, errors, token cost, query inclusion, and answer accuracy, combining LLM judging, deterministic checks, and human review.Tool-call precision and recall are order-invariant; human experts manually review a fixed 30% of cases per run.Source: §3.4 · p. 6
4 Results and Discussion
Handoffs match or exceed the single-agent baseline’s accuracy at lower token cost, while performance varies across tasks and models. Results also show that tool-call fidelity does not guarantee final-answer accuracy.
4 Results and Discussion
- Query inclusion approaches 1.0 across tasks, but longitudinal dependencies make Task 2 outputs vulnerable to error propagation.Table 2 covers longitudinal response analysis without RANO across 75 cases and 809 queries.Source: §4 · p. 6, Table 2 · p. 8
- Handoffs match or exceed the single baseline’s accuracy while maintaining lower input and output token costs.The single framework is robust but most expensive because it ingests the full tool description for every API call.Source: §4 · pp. 6–7
- Agents-as-tools underperforms across nearly all models; redundant preprocessing can alter images and degrade segmentation, with Gemini 3.1 Pro making 21 Task 2 segmentation errors.The passage attributes the large drop in human-judged accuracy to the prominence of faulty cases in the evaluated subset.Source: §4 · p. 7
- Perfect tool-call fidelity did not ensure perfect final accuracy: 17 of 64 GPT-5.4 handoff workflows fell short of 1.0 response accuracy.The cited setup had 1.0 precision and recall; the authors state that correctness is bounded by the underlying tools.Source: §4 · p. 7
- GPT-5.4 and Claude Sonnet 4.6 with handoffs or a single-agent architecture reach peak RANO accuracy of 0.867, misclassifying only 2 of 18 Progressive Disease cases.A recurring pitfall is interpreting an empty follow-up after two successive zero-volume timepoints as CR, although LUMIERE labels it SD.Source: §4 · p. 7, Table 3 · p. 8
- Claude Sonnet 4.6 is costlier across tasks, excels at single-timepoint volumetry, trails other LLMs longitudinally, and adheres more strictly to RANO guidelines than Gemini.No LLM is the definitive best across all architectures and tasks.Source: §4 · p. 7
5 Conclusion
The handoff-based multi-agent configuration achieves the highest overall performance and efficiency, with architectural trends consistent across evaluated models. The framework offers workflow assistance and a practical deployment blueprint, while its performance, latency, and scope remain constrained by underlying tools.
5 Conclusion
- The training-free framework offers radiologists workflow assistance without exposing sensitive patient data to commercial servers, at 5–20 cents per run depending on API pricing.Source: §5 · p. 8
- The handoff-based multi-agent configuration achieves the highest overall performance and efficiency, while agents-as-tools is subpar and these trends hold across evaluated models.The single-agent baseline closely trails the handoff-based configuration.Source: §5 · p. 8
- Performance, latency, and supported workflow scope are constrained by underlying tools; the study focuses on brain MRI and does not yet support multi-tiered cascaded decisions.The modular design enables future extension into other domains and modalities.Source: §5 · p. 8
- The work demonstrates agentic AI capabilities in clinical workflows, offers a practical deployment blueprint, and introduces a VQA dataset for future research.Source: §5 · p. 9