PaperlayerView exact revision on arXiv

Source-linked AI summary

ChemGraph: An Agentic Framework for Computational Chemistry Workflows

Thang D. Pham, Aditya Tanikanti, Murat Keçeli

arXiv:2506.06363v1physics.chem-phcond-mat.mtrl-scics.AIcs.LGphysics.comp-ph

TL;DR

Molecular simulation workflows require expertise and manual setup across diverse methods and software. ChemGraph uses an LLM-powered agent system with simulation tools to automate these workflows; across its benchmarks, smaller models handle simpler tasks well, while task decomposition improves their performance on complex ones.

  • Problem

    Running molecular simulations requires expertise and manual effort to configure parameters, select methods, and manage software-specific inputs and interfaces.

    Source: Introduction · p. 2

  • Method

    ChemGraph is an LLM-powered agent system that uses simulation tools to carry out molecular workflows from structure generation through thermochemistry.

    Source: Introduction · p. 2

  • Results

    Smaller LLMs perform well on simpler workflows but decline on complex tasks; decomposition improved complex-task accuracy by up to 73% in small LLMs.

    Source: Introduction · p. 3, Conclusions · p. 16

  • Takeaways & Limitations

    Intuitive prompts abstract low-level coding and tool configuration, with the stated potential to lower research barriers and enhance high-throughput workflows.

    Source: Introduction · p. 3

    None of the four evaluated LLMs achieved perfect accuracy, and occasional errors remain.

    Source: Results · p. 15

Abstract

from arXiv · show

Atomistic simulations are essential tools in chemistry and materials science, accelerating the discovery of novel catalysts, energy storage materials, and pharmaceuticals. However, running these simulations remains challenging due to the wide range of computational methods, diverse software ecosystems, and the need for expert knowledge and manual effort for the setup, execution, and validation stages. In this work, we present ChemGraph, an agentic framework powered by artificial intelligence and state-of-the-art simulation tools to streamline and automate computational chemistry and materials science workflows. ChemGraph leverages graph neural network-based foundation models for accurate yet computationally efficient calculations and large language models (LLMs) for natural language understanding, task planning, and scientific reasoning to provide an intuitive and interactive interface. Users can perform tasks such as molecular structure generation, single-point energy, geometry optimization, vibrational analysis, and thermochemistry calculations with methods ranging from tight-binding and machine learning interatomic potentials to density functional theory or wave function theory-based methods. We evaluate ChemGraph across 13 benchmark tasks and demonstrate that smaller LLMs (GPT-4o-mini, Claude-3.5-haiku, Qwen2.5-14B) perform well on simple workflows, while more complex tasks benefit from using larger models like GPT-4o. Importantly, we show that decomposing complex tasks into smaller subtasks through a multi-agent framework enables smaller LLM models to match or exceed GPT-4o's performance in specific scenarios.

Introduction

ChemGraph targets the expertise and manual effort required to configure simulation workflows across diverse software. It combines an LLM-powered agent with simulation tools, and its evaluation finds that task decomposition helps smaller models handle complex workflows.

Introduction

  • Simulation setup remains time-consuming and expertise-intensive because researchers must configure parameters, choose methods, and handle software-specific inputs and interfaces.Even minor input mistakes can produce incorrect results, unnecessary computational expense, or failures.Source: Introduction · p. 2
  • ChemGraph is an LLM-powered agent system for molecular simulation workflows, from structure generation to geometry optimization, vibrational analysis, and thermochemistry.The described tasks include generating SMILES strings and molecular structures.Source: Introduction · p. 2
  • Smaller LLMs perform well on simpler tasks but decline on complex ones, while decomposing complex tasks lets them match or sometimes surpass GPT-4o.The framework spans semi-empirical methods, ML potentials, and DFT.Source: Introduction · p. 3
  • By abstracting low-level coding and tool-specific configuration behind intuitive prompts, ChemGraph aims to lower research barriers and support high-throughput workflows.The passage presents high-throughput enhancement as a potential benefit.Source: Introduction · p. 3

Methodology

ChemGraph combines graph-based workflow coordination, simulation tools, and configurable prompts to run molecular workflows. Its methodology also defines standardized benchmark tasks and a multi-agent approach for complex reaction calculations.

Methodology

  • ChemGraph uses LangGraph's shared state, agent or operation nodes, and directed or conditional edges to coordinate workflow execution.Directed edges pass messages along a fixed path; conditional edges route them according to a condition.Source: Methodology · p. 3
  • Its LLM selects tools from a prompt, receives each tool result, and decides whether another call is needed before returning a natural-language or structured response.The structured-output path is designed to address a request using a predefined JSON schema.Source: Methodology · p. 3, Figure 1 · p. 4
  • Python tools wrap RDKit, ASE, and PubChemPy for cheminformatics, structure generation, geometry optimization, vibrational analysis, and thermochemistry.AtomsData serializes atomic structure information for LangGraph message passing.Source: Methodology · pp. 4–5
  • ASE calculators connect ChemGraph to methods ranging from xTB and MACE to DFT and post-Hartree–Fock calculations through NWChem and Orca.The framework is intended to let users balance computational cost and accuracy.Source: Methodology · p. 5
  • ChemGraph evaluates 13 benchmark experiments across six integrated tools, with up to four calls in the first 11 tasks and 9–12 calls in the final two reaction tasks.The final reaction-level properties are computed from thermochemical calculations for individual species.Source: Methodology · p. 5, Table 2
  • Benchmark instances vary molecule names, SMILES, or reactions while holding other conditions fixed, and expert-style JSON answers specify expected tool calls and results.Molecules were randomly sampled from PubChem subject to atom-count limits; the two DFT optimization tasks used a fixed list of small molecules.Source: Methodology · pp. 6–7
  • Evaluation uses single-turn structured outputs, scoring final-answer accuracy and tool-call counts across four LLMs, with GPT-4o tested only on the two most complex tasks.Single-turn evaluation allows self-correction within one interaction; the study cites divergent correction paths as a challenge for reproducible multi-turn benchmarks.Source: Methodology · pp. 7–8
  • For the two most complex reaction tasks, a Planner decomposes requests, Executors handle subtasks sequentially, and an Aggregator combines their results into a final answer.Each Executor has the single-agent system's tools.Source: Methodology · p. 8, Figure 2 · p. 9

Results

ChemGraph performed strongly on simpler molecule-name and SMILES workflows, while reaction-thermochemistry accuracy was lower for several models; multi-agent decomposition substantially improved smaller-model performance on those complex tasks.

Single-agent evaluation

  • Claude-3.5-haiku and GPT-4o-mini exceeded 87% accuracy in every molecule-name task and both exceeded 87% on SMILES tasks; Qwen2.5-14B reached lows of 60% and 73%, respectively.Figure 3 groups accuracy across 260 instances in 13 experiments; GPT-4o was evaluated only on the two reaction tasks.Source: Figure 3 · p. 10, Results · pp. 11, 13
  • Claude-3.5-haiku’s name2smi errors included declining the conversion tool or constructing SMILES manually, and single-turn scoring marked the responses incorrect despite possible recovery with prompting.The model's average tool calls per molecule were the lowest for this task.Source: Results · p. 11
  • 93% accuracy on name2gibbs followed GPT-4o-mini’s recovery from omitted calculator parameters using error messages, at an average of four tool calls.The initial omission caused tool-call failures.Source: Results · pp. 12–13
  • SMILES-input tasks showed the same common errors as name-input tasks, including incorrect tool arguments and occasional summarization mistakes on complex outputs.The affected outputs included nested dictionaries and long lists mixing floating-point and complex numbers.Source: Results · p. 13
  • Above 83% average accuracy on both reaction tasks made GPT-4o the strongest single agent; GPT-4o-mini and Qwen2.5-14B did not exceed 20%.Claude-3.5-haiku achieved 57% and 80% on the two tasks, respectively; results were averaged over three runs.Source: Results · p. 13

Multi-agent evaluations

  • 17% to 77% for GPT-4o-mini and 57% to 90% for Claude-3.5-haiku on react2enthalpy; on react2gibbs, they rose from 20% to 93% and 80% to 97%, respectively.Qwen2.5-14B showed only modest gains, with persistent tool-call errors limiting the aggregator's final answers.Source: Results · p. 14, Figure 6 · p. 14

Flexibility and Robustness of ChemGraph

  • ChemGraph’s ASE calculator layer supports workflows across DFT, tight-binding methods, and ML potentials, allowing method substitution for different accuracy-cost trade-offs.The authors also integrated UMA and describe the framework as able to incorporate emerging ASE-compatible methods with minimal overhead.Source: Results · pp. 14–15

Limitations, safety and future directions

  • None of the four evaluated LLMs was perfectly accurate, proprietary models carry computational cost, and large-scale or long-running HPC workflows remain under development.Automated job-submission scripts and HPC deployment are ongoing work.Source: Results · pp. 15–16

Conclusions

ChemGraph automates molecular simulation workflows and was evaluated across 13 experiments spanning molecule-name-to-SMILES conversion through thermodynamic calculations. Multi-agent decomposition improved accuracy on complex workflows, while modular calculator compatibility covered multiple simulation backends.

  • ChemGraph was evaluated across 13 experiments, from molecule-name-to-SMILES conversion to thermodynamic property calculations, with state-of-the-art LLMs achieving high task completion accuracy.Source: Conclusions · p. 16
  • Up to 73% accuracy gains in small LLMs were reported for reaction enthalpy and Gibbs free energy calculations with multi-agent decomposition.Source: Conclusions · p. 16
  • ChemGraph workflows ran with DFT, tight binding, and machine learning potentials through modular, ASE-style calculator compatibility.The framework supports rapid prototyping and benchmarking across computational methodologies.Source: Conclusions · p. 16
Loading 2506.06363v1…