PaperlayerView exact revision on arXiv

Source-linked AI summary

Deploying Frontier Agentic Technology in MOOSEnger, a Multiphysics-Capable AI Assistant

Zaid Abulawi, Mengnan Li, Guillaume Giudicelli, Yang Liu, Cody Permann

arXiv:2608.15881v1cs.LGcs.CE

TL;DR

Specialized MOOSE multiphysics workflows require expertise that limits accessibility for many scientists and engineers. This paper extends MOOSEnger with a locally hosted, retrieval-and-validation harness, finding that harness-enabled GPT-5.2 reached 90% success versus 76.5% for Gemma4, while unaided baselines performed far worse.

  • Problem

    Using MOOSE effectively requires specialized expertise, limiting access to multiphysics simulation for many domain scientists and engineers.

  • Method

    The paper extends MOOSEnger with a locally hosted-model harness that retrieves repository knowledge, validates and repairs inputs, and stores reusable lessons.

  • Results

    Across 200 prompts, harness-enabled Gemma4 and GPT-5.2 achieved 76.5% and 90% success, versus 0% and 5% for unaided baselines.

  • Takeaways & Limitations

    The results suggest a well-designed harness can substantially narrow open-weight and proprietary model performance differences on specialized MOOSE simulation tasks.

  • Takeaways & Limitations

    Gemma4 remained 16–24 points behind GPT-5.2 in plasticity, porous flow, Navier–Stokes, and phase field categories.

Abstract

from arXiv · show

The Multiphysics Object-Oriented Simulation Environment (MOOSE) is an open-source finite-element framework for building multiphysics simulation applications. Using a multiphysics environment effectively demands specialized expertise, creating a barrier for many domain scientists and engineers. MOOSEnger, developed at Idaho National Laboratory (INL), is a domain-specific, tool-enabled AI agent built for the MOOSE Framework. This work extends MOOSEnger with a harness focused on locally-hosted models. The harness gives the agent a full pipeline: it retrieves contextual knowledge from the MOOSE repository, validates and diagnoses the resulting input through interaction with the simulation executable environment, and extracts and stores lessons in a persistent memory. The resulting framework is demonstrated on an engineering problem from the National Reactor Innovation Center Virtual Test Bed (VTB), illustrating its potential to support realistic multiphysics simulation workflows. Additionally, the agent performance is evaluated on different categories including diffusion, Navier--Stokes, phase field, plasticity, porous media flow, solid mechanics, transient heat transfer, and reactor mesh generation. Each category consists of 25 prompts/cases. We compare MOOSEnger-Gemma4 against MOOSEnger-GPT-5.2, alongside baseline Gemma4 and GPT-5.2 without agentic capabilities. MOOSEnger-GPT-5.2 shows a slight edge, achieving a 90\% success rate versus 76.5\% for MOOSEnger-Gemma4. The baseline models perform far worse, at just 5\% (GPT-5.2) and 0\% (Gemma4), underscoring the impact of the agentic harness.

INTRODUCTION

MOOSEnger addresses the expertise barrier of MOOSE multiphysics simulation by combining domain-specific tools with a locally hosted, tool-enabled agent harness. The framework retrieves repository knowledge, validates and diagnoses simulation inputs, stores lessons persistently, and substantially outperforms non-agentic baselines.

  • Motivation: MOOSE is an open-source finite-element framework for building multiphysics applications, but effective use requires specialized expertise that can exclude many domain scientists and engineers.Domain-specific knowledge and tailored tools improve LLM performance on demanding modeling and simulation tasks.
  • Simulation interface: MOOSE simulations use Hierarchical Input Text files specifying meshes, variables, materials, boundary conditions, execution settings, and outputs under strict formatting and parameter relations.These structured-input requirements create challenges distinct from ordinary text or code generation.
  • System contribution: The locally hosted harness retrieves MOOSE repository context, validates and diagnoses inputs through the simulation executable, and stores extracted lessons in persistent memory.Local hosting also mitigates data-leakage risks for export-controlled and sensitive MOOSE applications.
  • Evaluation: The framework is demonstrated on an engineering problem from the National Reactor Innovation Center Virtual Test Bed and evaluated across eight multiphysics categories with 25 prompts/cases each.Categories include diffusion, Navier–Stokes, phase field, plasticity, porous media flow, solid mechanics, transient heat transfer, and reactor mesh generation.
  • Results: 90% success rate was achieved by MOOSEnger-GPT-5.2 versus 76.5% for MOOSEnger-Gemma4, while baseline GPT-5.2 reached 5% and Gemma4 reached 0%.The comparison included agentic MOOSEnger variants and baseline models without agentic capabilities.

METHODOLOGY

MOOSEnger is extended with a LangGraph-based, stateful agentic methodology that grounds tasks in relevant files, executes and diagnoses inputs, repairs them using feedback, and stores reusable knowledge in persistent memory.

  • The methodology retrieves relevant input files and examples to ground MOOSEnger’s responses.
  • An interactive execution environment enables structured diagnostics, feedback, and input repair.
  • Implemented with LangGraph, the agent follows a stateful reasoning-and-action loop that selects tools and evaluates their outputs.
  • The agent extracts reusable knowledge into persistent memory.

MOOSE Repository Grounding

MOOSEnger grounds input-file drafting in the local MOOSE source tree by identifying relevant physics modules and examining working examples, tests, and tutorials. Repository browsing and HIT-file search provide reusable syntax, configurations, naming conventions, and solver settings for translating modeling requests into relevant examples.

  • MOOSE Repository Grounding: The local MOOSE source tree supplies task-specific context through physics modules containing tests, examples, and tutorials with valid HIT syntax and established modeling practices.Covered modules include heat transfer, phase field, Navier–Stokes, solid mechanics, porous flow, and reactor meshing.
  • MOOSE Repository Grounding: Before drafting inputs, MOOSEnger identifies relevant modules, explores example and test directories, and reads representative HIT files for reusable structures, configurations, naming conventions, and solver settings.These repository files provide concrete examples grounded in working MOOSE applications.
  • MOOSE Repository Grounding: MOOSEnger can list modules, browse input-file directories, enumerate candidates, read files fully, and search HIT files for object names, parameters, or syntax.Case-insensitive substring search returns bounded file paths, line numbers, and matching snippets to connect modeling requests with repository examples.

MOOSE Validation and Diagnostic

MOOSEnger validates MOOSE input files through layered structural, semantic, LSP, and executable checks. It consolidates diagnostics to repair inputs and uses staged execution tests to distinguish input errors from convergence or physics-modeling problems.

  • Static validation: MOOSEnger first checks HIT structure, then verifies block names, action paths, object types, and parameter names against the local MOOSE environment.Malformed files are rejected before more expensive semantic checks.
  • Language-server and executable checks: LSP diagnostics provide line- and column-level feedback, formatting, symbol context, and parameter templates for precise configuration checks.The candidate is subsequently checked against the target MOOSE executable using –check-input.
  • Diagnostic repair: Diagnostic reports include each issue’s source object, message, location, type, and suggested next step, enabling the agent to repair the input and rerun validation.The report evidence supports correction of the actual input file rather than reliance on free-form model judgment.
  • Execution-based diagnosis: Execution checks range from mesh-only validation and short test runs to full simulations, helping separate input errors from convergence or physics-modeling problems.These checks use the same simulation environment after validation.

Persistent Memory

Persistent memory extends MOOSEnger beyond single-session history by building a reusable knowledge database from interactions across sessions. The system extracts, validates, stores, and retrieves memories so the agent can use knowledge from earlier sessions during current tasks.

  • Persistent Memory: Persistent memory builds a reusable knowledge database from user–agent interactions across sessions, addressing MOOSEnger’s single-session limitation.Conversation history can restore an earlier session, but it does not organize knowledge learned across multiple tasks.
  • Persistent Memory: 4 memory types are managed with Mem0: user preferences, project facts, lessons learned, and verified error resolutions.A locally hosted Gemma 4 model extracts concise memory records from conversation and tool evidence.
  • Persistent Memory: 2 workflow points use Gemma 4 for extraction: after each turn for preferences and project facts, and at session end for lessons and verified error resolutions.Validation evidence prevents unsuccessful repairs from being stored.
  • Persistent Memory: Retrieved memories are appended to the agent’s context, allowing current conversations to incorporate useful knowledge from earlier sessions.Startup loads bounded preferences and project facts into a temporary cache, while each turn searches for relevant lessons and error resolutions.

CASE-STUDY DEMONSTRATION

The case-study demonstration applies MOOSEnger to an IAEA sodium fast reactor hexagonal duct bowing verification problem from the VTB. The agent combines contextual retrieval, input drafting, validation, and diagnostic repair in a multiphysics thermo-mechanical workflow.

  • Case-study setup: The demonstration case uses a sodium fast reactor hexagonal duct bowing model from the VTB and is identified as an IAEA benchmark.
  • Agent workflow: The agent follows five phases: task interpretation, contextual grounding, input drafting, diagnosis and repair, and lesson extraction and storage.Grounding retrieves relevant MOOSE modules, directories, reference inputs, and object documentation before drafting the input file.
  • Case-study setup: The demonstration models free thermal bowing of a mechanically fixed, three-dimensional hexagonal duct under radial and axial thermal gradients.The published benchmark uses Reactor-module mesh generation and the Solid Mechanics module to calculate thermo-mechanical deformation.
  • Case-study setup: The case exercises coupled thermal and mechanical physics, function interplay, and auxiliary variables that challenge an AI agent.
  • Agent workflow: 25 tool calls support input validation and repair after diagnostics identify invalid Executioner parameters and an invalid Solid Mechanics subblock name.The workflow validates the drafted input and enters a repair loop to resolve the reported issues and errors.

AGENT EVALUATION

The evaluation measures end-to-end, tool-enabled MOOSE input authoring across eight physics-oriented benchmark categories. MOOSEnger substantially outperforms standalone models, with GPT 5.2 slightly ahead of Gemma 4 31b within the harness.

  • Evaluation design: Eight categories covered diffusion, Navier–Stokes, phase field, plasticity, porous media flow, solid mechanics, transient heat transfer, and reactor mesh generation.A prompt counted as successful when the generated input ran successfully and matched the requested physics, geometry, boundary conditions, and outputs.
  • Benchmark results: 179 of 200 prompts (90%) were resolved by harness-enabled GPT 5.2, versus 153 of 200 (76.5%) for harness-enabled Gemma 4 31b.This corresponds to a 13.5 percentage-point gap.
  • Benchmark results: 90% versus 76.5% shows GPT 5.2 slightly outperforming Gemma 4 31b within the MOOSEnger harness.The gap is smaller on diffusion, transient heat conduction, and solid mechanics, while Gemma 4 31b leads on reactor mesh generation, 88% versus 84%.
  • Standalone baselines: 5% and 0% were achieved by standalone GPT 5.2 and Gemma 4 31b, respectively, compared with their harness-enabled configurations.Standalone GPT 5.2 resolved 10 of 200 prompts, while standalone Gemma 4 31b resolved none.
  • Limitations: The remaining Gemma 4 31b–GPT 5.2 gap is concentrated in plasticity, porous flow, Navier–Stokes, and phase field.The passage identifies family-specific retrieval augmentation, additional skills, and instructions as potential ways to narrow or close this difference.

CONCLUSIONS

The extended MOOSEnger harness combines repository grounding, validation and diagnostics, and persistent memory to substantially improve locally hosted model performance on specialized MOOSE simulation tasks. Harness-enabled Gemma 4 reached 76.5% success versus 90% for GPT 5.2, compared with 0% and 5% for their unaided baselines, while remaining gaps identify directions for improvement.

  • Harness impact: 76.5% success rate for Gemma 4 and 90% for GPT 5.2 with the harness exceeded their unaided baselines of 0% and 5% across 200 prompts.The harness added grounding in the MOOSE repository, validation and diagnostics, and persistent memory.
  • Category performance: Diffusion, transient heat conduction, and solid mechanics showed near-parity between the two harness-enabled models.These results indicate that the harness, rather than the underlying model, drives most of the capability gain in these categories.
  • Category performance: 16–24 points separated Gemma 4 from GPT 5.2 in plasticity, porous flow, Navier–Stokes, and phase field.The remaining gap suggests that these categories depended more strongly on the base model’s reasoning.
  • Future direction: Category-specific retrieval, expanded skills, and refined instructions are proposed to close the remaining gap toward fully capable locally hosted MOOSE agents.The harness could make up much of the gap between open-weight and proprietary models where external API-based models are ruled out by export controls.
Loading 2608.15881v1…