PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain-Grounded Tool Orchestration for LLM-Guided Scientific Analysis

Jeff Lee, Sebastien Jourdain, Cory Quammen, Patrick O'Leary, Berk Geveci

arXiv:2608.30696v1cs.CEcs.HC

TL;DR

Scientific visualization workflows require domain expertise that script-generating LLM systems often encode incorrectly, even when their code executes. The paper separates intent interpretation, deterministic tool execution, and explanation through MCP and ontology-constrained planning, then demonstrates the architecture across CFD and TTK. Across both domains, ontology grounding raises interpretation accuracy from 0.41 to 0.91 when the relevant fact is retrieved in scoped form, while tool selection remains unchanged.

  • Problem

    Scientific toolkits are powerful, but choosing, ordering, parameterizing, and interpreting analyses from an engineering question requires domain expertise that current LLM script-generation approaches handle unreliably.

  • Method

    The paper introduces a Plan–Execute–Interpret architecture in which an ontology-constrained LLM plans and explains, while deterministic domain tools execute through MCP.

  • Results

    Across CFD and TTK, ontology grounding leaves tool selection unchanged but raises interpretation accuracy from 0.41 to 0.91 when the relevant fact is retrieved in scoped rather than bulk form.

  • Takeaways & Limitations

    The architecture generalizes across two domains on shared ParaView infrastructure, with the second domain requiring only an ontology and wrappers around existing filters.

  • Takeaways & Limitations

    Queries outside encoded phenomena revert to unconstrained LLM planning, and critical applications still require expert review of potentially overconfident explanations.

Abstract

from arXiv · show

Scientific analysis workflows encode deep domain knowledge through sequences of tightly coupled operations where correctness depends on tool selection, execution order, and parameterization. A CFD engineer investigating flow separation must extract wall shear stress, identify zero-crossings in skin friction, and confirm with boundary-layer profiles: a chain that requires both domain expertise and proficiency with visualization tools. Current approaches to LLM-assisted scientific visualization generate scripts that encode this knowledge implicitly, and often incorrectly, producing code that executes but yields wrong results. We present an architecture that separates intent interpretation (LLM) from execution (deterministic domain tools) from explanation (LLM), connected by the Model Context Protocol (MCP) and grounded by domain ontologies that constrain planning to valid analysis chains. We instantiate the architecture in two domains on the same ParaView server infrastructure: computational fluid dynamics post-processing and topological data analysis via the Topology ToolKit (TTK). Adding the second domain required only an ontology and tool wrappers around existing filters, with no change to the architecture, protocol, or deployment. By construction the design removes whole classes of failure that affect script generation (such as API hallucination and missing pipeline stages) and narrows the strategic errors that remain. An ablation across both domains locates the ontology's empirical effect: it does not change which tools the planner selects, which is already reliable, but corrects how the model interprets results, raising interpretation accuracy from 0.41 to 0.91, and only when the relevant fact is retrieved in scoped rather than bulk form. ParaView's client-server model carries analysis to production-scale datasets through a thin browser client.

1 Introduction

The paper addresses the accessibility gap between powerful scientific toolkits and the domain expertise needed to select, sequence, parameterize, and interpret analyses. It proposes an ontology-grounded Plan–Execute–Interpret architecture that separates LLM reasoning from deterministic execution and demonstrates it across CFD and TTK.

  • Scientific toolkits provide extensive analysis capabilities, but applying them correctly requires domain knowledge about tool choice, ordering, parameters, and result meaning.
  • The architecture assigns intent interpretation and explanation to the LLM while deterministic domain tools execute validated procedures through MCP.A domain ontology constrains planning to physically or topologically valid analysis chains.
  • The pattern generalizes across CFD post-processing and TTK-based topological analysis on the same ParaView server infrastructure.The CFD instantiation covers four use cases, while TTK covers feature identification, structural decomposition, and comparative topology across timesteps.
  • Adding TTK required only an ontology file and MCP wrappers around existing filters, with no changes to the architecture, protocol, deployment, or user interface.
  • A domain-independent in-situ control loop supports connecting, pausing, stepping, monitoring, and extracting from live Catalyst-instrumented HPC solvers.
  • The design removes API hallucination and missing pipeline stages by construction and narrows the strategic errors remaining in script-generation approaches.

2 Related Work

Prior systems commonly have LLMs generate executable visualization code, but this leaves strategic analysis errors unresolved. The paper instead emphasizes domain-grounded orchestration, structured tool interaction, and ontology-based planning for scientific workflows.

  • LLM-assisted visualization systems such as ChatVis and VizGenie generate executable scripts that directly invoke visualization APIs.
  • Generated-script workflows are one-shot, rely on human-driven correction, and can execute while performing the wrong analysis.
  • ParaView-MCP demonstrates viable MCP-based tool invocation, whereas this work adds ontology-constrained planning, structured quantitative returns, and cross-domain validation.
  • Scientific visualization is harder than chart-oriented interfaces because engineering questions map to multi-step procedures that vary across disciplines, solvers, and data formats.
  • The paper’s ontology maps observable phenomena to physical quantities and analysis tools, grounding planning in domain knowledge.

3 Architecture

The architecture uses an ontology-constrained Plan–Execute–Interpret loop: the LLM selects named operations, MCP dispatches deterministic tools, and structured results support grounded interpretation and follow-up analysis.

  • Plan–Execute–Interpret Loop: Plan converts a domain-language question into an ordered sequence of named tool invocations selected from ontology-declared valid operations.
  • Plan–Execute–Interpret Loop: Execute dispatches deterministic domain workflows through MCP; tools modify the server-side pipeline and return structured quantitative results rather than renders.The pipeline persists across invocations, enabling multi-step analysis within one session.
  • Plan–Execute–Interpret Loop: Interpret uses structured results and the relevant ontology entry to produce quantitative explanations and propose diagnostic follow-ups.For example, detected separation can trigger boundary-layer confirmation and an upstream transition check.
  • Domain Ontology: Each ontology entry contains a phenomenon, indicators, tools, follow-ups, and significance, with schemas instantiated separately for CFD and TTK.
  • System architecture: The browser is a thin client, while the LLM, MCP layer, deterministic tools, and ParaView server perform planning, execution, and interpretation.Both CFD and TTK tools run on the same server instance.
  • Domain Ontology: Representative TTK chains compute persistence, simplify it, and extract critical points to identify significant features and separate them from noise.
  • Tool layer: Domain workflow tools encapsulate expert multi-step procedures, including CFD boundary-layer profiles and TTK persistence diagrams with quantitative outputs.
  • Tool layer: Structured returns, dynamic semantic-field resolution, and consumer-aware pipeline management guide tool implementation.

4 Domain 1: Computational Fluid Dynamics

The CFD instantiation uses ontology-guided, deterministic tool chains for iterative diagnostics, cross-quantity reasoning, and live simulation monitoring. These workflows return structured results that drive interpretation and follow-up analysis.

  • Domain grounding: The CFD ontology maps aerodynamic phenomena to indicator quantities and deterministic tools, including Q-criterion or λ2 extraction for vortex structures.The tool set wraps ParaView filters with domain knowledge about phenomena such as flow separation, vortex breakdown, and wake structure.
  • Iterative diagnostic reasoning: For flow separation, the planner extracts skin friction and locates Cf zero-crossings before interpreting sign reversals and recommending boundary-layer profiles.The execution tools return zero-crossing locations and Cf distributions as structured data; the ontology supplies the follow-up recommendation.
  • Iterative diagnostic reasoning: The iterative pressure-diagnostic chain uses Cp plateaus to trigger boundary-layer profiles, whose velocity and shape-factor results characterize separation and reattachment.The walkthrough reports reverse flow at x/c = 0.60 and 0.70, H = 4.2 at x/c = 0.65, and an attached profile by x/c = 0.75.
  • Cross-quantity synthesis: Cross-quantity synthesis decomposes drag into pressure, viscous, and wake contributions across angles of attack within a persistent pipeline session.The interpreter reports separation moving from x/c = 0.78 to 0.62, pressure drag increasing 40%, and wake momentum deficit increasing 55%.
  • In-situ monitoring and control: The same tool layer connects to live Catalyst simulations, enabling deterministic control, residual monitoring, field extraction, and re-inspection during execution.The system forms a closed observe-reason-control loop that can pause, advance, monitor, and inspect a running solver under guidance from intermediate findings.

5 Domain 2: Topological Data Analysis (TTK)

The TTK workflow identifies, denoises, and structures topological features in CFD pressure fields, then uses diagram distances to assess convergence and support feature-aware analysis.

  • Implementation: TTK maps dominant-feature queries to persistence diagrams, gap-based simplification, and critical-point extraction in that order.The ordering prevents unsimplified critical-point counts from being dominated by discretization noise.
  • Feature Identification and Denoising: 138 birth-death pairs were returned, with a persistence threshold near 0.0038 retaining roughly the 25 most persistent pairs.The persistence diagram exposes the signal-versus-noise gap used for simplification.
  • Feature Identification and Denoising: A 38% reduction cut critical-point counts from 342 to 213, mainly removing low-persistence saddles and minima while preserving dominant extrema.The retained extrema and connecting saddles represent dominant pressure structures; multi-saddles reflect discretization rather than aerodynamic meaning.
  • Structural Decomposition: Morse-Smale decomposition returns surviving critical points, their separatrix skeleton, and ascending and descending manifold segmentation.The workflow presents basin structure as an illustrative segmentation rather than a measured breakdown of basin volumes.
  • Comparative Topology: The Wasserstein distance is 0.67 between the first and converged diagrams but 0.001 between timestep 90 and the final state, indicating stabilized topology.On steady runs this supports topology-based convergence monitoring; on unsteady runs the same machinery tracks feature births and deaths over physical time.
  • Why Quantitative Topology: Persistence-based feature inventories, Morse-Smale structure, and diagram distances provide reproducible counts, structural comparisons, and feature-aware signals for analysis and surrogate scoring.The paper does not claim a trained surrogate; Wasserstein-2 is preferred over bottleneck distance for training because it is smoother and gradient-usable.

6 Cross-Domain Analysis

CFD and TTK tools compose on one ParaView server because shared infrastructure and a common pipeline let topology locate features that CFD workflows then characterize.

  • Cross-Domain Architecture: The shared architecture, protocol, deployment, and interface support both domains; adding TTK required only an ontology and wrappers around existing filters.No modifications to the shared layers were needed.
  • Domain Independence of In-Situ Operation: In-situ control is domain-independent: Catalyst sessions can be connected, paused, resumed, stepped, breakpointed, and queried across HPC solvers.Only monitored field names and diagnostic quantities are domain-specific.
  • Combined Workflows: A single session composed topology critical-point extraction with CFD streamline visualization, after resolving the solver-specific pressure array name and a ParaView seeding API change.The planner produced a cross-domain plan from 38 available tools.
  • Topology-Guided Deep Dive: Topology-guided analysis supplies locations of interest, allowing CFD tools to apply expensive localized characterization without switching systems or transferring data.The shared server-side pipeline lets topology outputs feed directly into CFD operations.
  • Topology-Guided Deep Dive: The lowest-pressure minimum was located at x/c=0.37 and 70% span, where CFD returned boundary-layer thickness 0.0086 chord and shape factor about 2.The same-session workflow also located the wake, linking topological feature detection to boundary-layer and wake analysis.
  • Topology-Guided Deep Dive: Shape factor near 2, combined with eddy-viscosity evidence and three to four cells across the layer, indicates under-resolved turbulent flow rather than a resolved profile.The chordwise scan supports this interpretation: edge velocity falls from 0.86 to 0.52 while shape factor remains near 2 and rises aft.

7 Why Not LLM-Generated Scripts

LLM-generated scripts make scientific analysis brittle and difficult to verify because executable code can still choose the wrong strategy or silently mis-handle data. The proposed separation assigns planning and interpretation to the LLM while deterministic tools execute named, validated operations.

  • Generated scripts can execute while performing the wrong analysis, because wrong intention or logic is a strategic domain-knowledge failure rather than a code-quality failure.The alternative script-based architecture is described as one-shot, with human-driven correction and no protection against incorrect analysis strategies.
  • Solver-specific assumptions about array names, mesh types, and file formats can silently produce wrong results across data sources.Dynamic field resolution instead maps semantic names such as skin friction to solver-specific arrays at runtime.
  • Fire-and-forget scripts return images rather than structured results, preventing the LLM from performing quantitative reasoning about outputs.The Plan–Execute–Interpret loop returns field statistics, extracted profiles, and zero-crossing locations.
  • Named tool operations express analysis plans as declared intent, making provenance legible to domain users without programming expertise.An example plan is to extract skin friction on the upper surface and find zero-crossings.
  • The architecture eliminates module errors and API hallucinations, absorbs parameterization through tools, and mitigates wrong-strategy errors with ontology constraints.The LLM selects from a fixed MCP tool set, while tools provide defaults and dynamic resolution; the ontology constrains valid analysis chains.

8 Validation

Validation examines the architecture’s error classes and an ontology ablation across CFD and topology queries. Ontology grounding leaves reliable tool selection unchanged but substantially improves result interpretation when the relevant fact is retrieved in scoped form.

  • 8.1 Elimination of Failure Classes: Two failure categories are removed by construction, one is handled by the tool layer, and wrong intention or logic remains mitigated by ontology constraints.The design removes API hallucination and module errors, while defaults and dynamic field resolution address missing parameterization.
  • 8.2 Where Grounding Helps: Selection versus Interpretation: Tool-selection precision and recall were unchanged with ontology grounding, indicating selection was reliable and ontology-independent.The tool schemas already describe each tool’s function sufficiently for a capable planner.
  • 8.2 Where Grounding Helps: Selection versus Interpretation: 0.41 to 0.91: interpretation accuracy rose on a 34-case benchmark when the relevant ontology fact was placed in context.Grounding corrected 18 errors; topology questions were already answered correctly and were neither helped nor harmed.
  • 8.2 Where Grounding Helps: Selection versus Interpretation: 0.56, 0.74, and 0.91: interpretation improved monotonically from bulk ontology retrieval to tool-mediated retrieval to scoped contextual retrieval.The get ontology tool returns the scoped entry for a hinted key rather than the full ontology.
  • 8.2 Where Grounding Helps: Selection versus Interpretation: The ontology’s empirical contribution is interpretive rather than selective: it corrects domain errors only when the relevant fact is retrieved in scoped form.The measurements used a single planning model at demonstration scale, so generalization beyond GPT-4o-mini remains untested.

9 Discussion

The discussion frames scientific analysis as procedural, domain-constrained, and interpretive, matching the proposed division between language models and deterministic tools. It also identifies ontology coverage and evaluation scope as important boundaries.

  • Scientific analysis combines ordered operations, domain-validity constraints, and interpretation, which motivates separating planning, execution, and explanation.The constraints encode domain expertise, while results require contextualization within a domain framework.
  • A small ontology of approximately 25–30 entries per domain can encode prerequisite knowledge for analysis chains that LLMs otherwise frequently order suboptimally.Examples include simplifying before Morse-Smale segmentation and confirming separation with boundary-layer profiles rather than streamlines.
  • Ontology coverage limits constrained planning: phenomena absent from the ontology revert to unconstrained LLM reasoning.Semi-automatic ontology expansion from literature or tool documentation remains an open research question.
  • The paper evaluates domain correctness rather than correction cost, targeting whether the system reaches the right physical or topological conclusion.A script may require no executable-code corrections while still performing the wrong analysis.
  • The approach is complementary to code-generation RAG: retrieved code examples could augment tool discovery, while the ontology could constrain pipeline planning.The proposed roles are presented as complementary rather than competing.
  • Interpretation can remain overconfident or subtly incorrect, and the CFD evaluation is scoped to external aerodynamics, requiring expert review for critical applications.These are identified as limitations alongside ontology coverage and evaluation scope.

10 Future Work

Future work extends the architecture toward live simulation steering, additional scientific domains, cross-domain composition, and evolving ontologies. These directions reuse the existing architecture and infrastructure where the passages explicitly support that claim.

  • Live Catalyst integration already closes an observe–reason–control loop over simulation state, motivating future steering of solver inputs.The demonstrated controls include run, pause, step, breakpoints, residual monitoring, and extraction of live fields.
  • Adding domains such as medical imaging, materials science, and climate science would require ontology authoring and tool wrappers while reusing the architecture and deployment infrastructure.Examples include 3D Slicer modules, Tomviz filters, and CESM/E3SM outputs.
  • A shared ontology linking layer could support multi-domain sessions in which tools from different disciplines consume one another’s outputs.Suggested compositions include CFD-predicted surface loads feeding structural analysis or topology characterizing materials scalar fields.
  • Ontology evolution could combine semi-automatic expansion from documentation, literature, and usage logs with expert review.The current ontologies are static and manually authored.
  • Uncertainty-aware planning could annotate ontology entries with confidence levels so the system can signal queries near the boundary of encoded knowledge.This is proposed as a direction for improving planning beyond static ontology coverage.

11 Conclusion

The architecture separates intent interpretation, deterministic execution, and explanation through MCP and domain ontologies, and its design generalizes across CFD and topological analysis. Across both domains, ontology grounding improves result interpretation accuracy without changing already reliable tool selection, but only with scoped retrieval.

  • The architecture assigns intent interpretation and explanation to LLMs, execution to deterministic tools, and workflow constraints to domain ontologies connected by MCP.This division targets procedural, domain-constrained, and interpretive properties of scientific analysis workflows.
  • The same architecture supports CFD post-processing and TTK-based topological analysis on shared ParaView infrastructure.Adding the TTK domain required an ontology and wrappers around existing filters rather than new infrastructure.
  • The design produces physically and topologically correct analyses where script generation can produce silent failures.The CFD instantiation also supports iterative investigation and cross-quantity synthesis beyond one-shot scripts.
  • 0.41 to 0.91: ontology grounding raises interpretation accuracy across both domains without changing tool selection.Tool selection was already reliable, while interpretation improved only when the relevant fact was retrieved in scoped rather than bulk form.
  • The resulting division of labor makes the system more correct, robust, and interpretable than assigning the entire workflow to an LLM.The LLM serves as the interface, deterministic workflows as the engine, MCP as the protocol, and the ontology as the knowledge layer.
Loading 2608.30696v1…