Source-linked AI summary
TIAgent: From Natural Language to Grounded Agentic Workflows for Computational Pathology
Aakash Madhav Rao, Mark Eastwood, Danielle Belgrave, Fayyaz Minhas
TL;DR
Computational pathology workflows require programming expertise, while hallucinations and incorrect specifications limit the reliability of LLM-generated analyses. TIAgent grounds natural-language requests in existing CPath operations to construct verifiable computational graphs; benchmark performance varied substantially by backbone model, with Kimi-K2.7-Code achieving the highest UWS (0.967).
Problem
CPath workflows require substantial programming expertise, and hallucinated operations or incorrect specifications limit the reliability of LLM-generated analyses.
Source: §1 · pp. 1–2Method
Results
UWS varied substantially across backbone LLMs, with Kimi-K2.7-Code achieving the highest UWS (0.967).
Source: §3 · p. 4Takeaways & Limitations
TIAgent provides a foundation for more accessible construction of CPath workflows and systematic evaluation of LLM-driven workflow generation.
Source: §4 · p. 4The current benchmark evaluates individual operations; extending it to multi-step tasks remains future work to assess how reliably those capabilities compose.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Computational pathology (CPath) analyses increasingly depend on multi-step computational workflows that require substantial programming expertise, limiting their accessibility to pathologists and other domain experts without extensive coding experience. Although large language models (LLMs) provide a promising natural-language interface for constructing such analyses, hallucinated operations and incorrect workflow specifications limit their reliability in scientific settings. We introduce Tissue Image Analysis Agent (TIAgent), an automated CPath workflow generation framework that, given a natural-language prompt specifying a complex analytical task together with an input dataset, automatically constructs a verifiable computational graph employing existing CPath operations. The resulting graph can be inspected and executed, enabling natural-language construction of complex CPath workflows for applications including automated biomarker discovery and tissue analysis. We demonstrate end-to-end workflow generation and execution and systematically benchmark 14 backbone LLMs across 90 unit tasks spanning 18 CPath operation types. The ability of each LLM to generate a complete and correct unit workflow is quantified using Unit Workflow Success (UWS). Performance varied substantially across backbone models, with Kimi-K2.7-Code achieving the highest UWS of 0.967 (95% CI: 0.907-0.989). Analysis of node and parameter correctness further revealed that selecting the appropriate computational operation and correctly configuring it represent distinct challenges for LLM-based workflow generation. TIAgent provides a foundation for making complex CPath workflow construction more accessible while enabling systematic evaluation of LLM-driven scientific workflow generation.
1 Introduction
CPath workflows can yield quantitative insights from whole-slide images, but constructing them requires programming expertise that limits accessibility for domain experts. TIAgent addresses the reliability challenge by grounding natural-language workflow generation in existing operations and evaluating backbone models.
1 Introduction
- CPath workflows require substantial computational expertise, creating a barrier for pathologists and other domain experts with limited programming experience.These workflows combine image loading, processing, model inference, and post-hoc analysis.Source: §1 · pp. 1–2
- LLM-generated workflows can hallucinate or omit operations, misconfigure parameters, and make unsupported assumptions, limiting reliability and verifiability.Executable code is not necessarily functionally correct, particularly for underspecified and compositionally complex tasks.Source: §1 · p. 2
- TIAgent grounds natural-language workflow generation in existing CPath operations and models, enabling graph inspection and validation while evaluating 14 LLMs across 90 tasks and 18 operation types.The evaluation assesses whether models select and configure the computational components correctly.Source: §1 · p. 2
2 Methods
TIAgent separates LLM-based workflow planning and node selection from deterministic graph construction, validation, and execution. Its catalogue defines supported operations and models, while its benchmark and metrics test the ability to generate individual workflows.
2 Methods
- TIAgent converts a user request into a reviewed plan, grounds its operations in available CPath nodes, then deterministically builds, validates, and compiles an executable graph.Validation failures are returned for repair before recompilation.Source: §2 · p. 2
- The CPath catalogue defines executable nodes and compatible models, allowing the framework to reason separately about the required operation and, where applicable, the model.Nodes specify their inputs, parameters, and outputs; models supply task-specific algorithms to compatible nodes.Source: §2 · p. 2
- The benchmark contains 90 unit-level tasks across 18 operation types, evaluated with 14 backbone LLMs to produce 1,260 workflows against manually constructed references.Each operation type has five tasks, and each model generates one workflow per task at temperature 0.1.Source: §2 · pp. 2–3
- Format Validity, Node Correctness, Parameter Correctness, and Unit Workflow Success distinguish executability from correct operation selection and configuration.UWS captures successful generation of a fully correct unit workflow.Source: §2 · p. 3
3 Results
TIAgent generated and executed a multi-step workflow for quantifying stromal tumor-infiltrating lymphocytes in a breast cancer image. Benchmark results varied substantially across backbone LLMs, and node selection and parameter configuration emerged as distinct challenges.
3 Results
- Kimi-K2.7-Code achieved the highest UWS (0.967), while UWS varied substantially across the 14 evaluated models.Figure 2 compares UWS, Node Correctness, and Parameter Correctness across backbone LLMs.Source: Figure 2 · p. 4, §3 · p. 4
- For a breast cancer whole-slide image, TIAgent executed a six-step sTIL quantification workflow that produced intermediate segmentation and detection outputs and an sTIL density map.The workflow included tissue and nuclei segmentation, tumor-associated stroma extraction, lymphocyte filtering, and density computation.Source: §3 · pp. 3–4
- Gemma3-12B [12] achieved NC (0.922) but PC (0.578), corresponding to UWS (0.467), illustrating that node selection and parameter configuration can diverge.The results identify operation selection and configuration as distinct challenges for LLM-based workflow generation.Source: §3 · p. 4
4 Conclusions and Future Work
TIAgent constructs verifiable, executable computational graphs from natural-language requests and input data, and its evaluation found substantial differences in unit-workflow reliability across backbone models. The current benchmark evaluates individual operations; assessment of their composition into multi-step workflows remains ongoing.
4 Conclusions and Future Work
- Across 14 backbone LLMs and 90 unit-level tasks, workflow-generation reliability differed substantially, revealing challenges in translating scientific instructions into correctly specified operations.TIAgent was also demonstrated on an end-to-end multi-step sTIL quantification workflow in breast cancer.Source: §4 · p. 4
- The benchmark is limited to individual operations; extending it to multi-step CPath tasks remains future work to assess how reliably those capabilities compose.Source: §4 · pp. 4–5
A Complete Computational Graph for sTIL Quantification
Figure 3 presents an executable computational graph for quantifying spatial sTIL density from a breast cancer whole-slide image. It shows how tissue and nucleus analysis feed into lymphocyte filtering and density mapping.
A Complete Computational Graph for sTIL Quantification
- The workflow combines tissue segmentation and nucleus detection to identify lymphocytes in tumor-associated stroma for spatial sTIL density mapping.It loads a breast cancer whole-slide image and tissue- and nucleus-analysis models.Source: Figure 3 · p. 7
- Tissue segmentation extracts tumor-associated stroma, while nucleus detection identifies nuclei across the whole-slide image.Source: Figure 3 · p. 7
- Detected nuclei are filtered to retain lymphocytes within the tumor-associated stroma before computing the density map.Source: Figure 3 · p. 7
B Unit Task Evaluation: Confidence Intervals
Table 1 reports TIAgent performance across underlying LLMs on the Unit Task benchmark. It gives 95% Wilson confidence intervals for proportions calculated across 90 unit tasks.
B Unit Task Evaluation: Confidence Intervals
- Table 1 compares underlying LLMs using performance proportions on the Unit Task benchmark.Source: Table 1 · p. 7
- Each reported proportion is calculated across 90 unit tasks.Source: Table 1 · p. 7
- The table reports 95% Wilson confidence intervals for its proportions.Source: Table 1 · p. 7