Toward Greater Autonomy in Materials Discovery Agents: Unifying Planning, Physics, and Scientists

1 Introduction

• Existing LLM materials agents mostly perform predefined tasks in fixed, expert-designed workflows, primarily coordinating AI tools.The computational cost of DFT also makes it infeasible to explore the vast, largely uncharted space of stable materials.Source: §1 · pp. 1–2
• MAPPS uses LLM planning to build and adapt structured sequences of domain-specific actions for scientific problems.The paper reports that broader LLM planning remains limited, while scientific workflow planning appears promising, especially with scientist interaction.Source: §1 · p. 2
• Five-fold improvement in stability, uniqueness, and novelty rates over prior generative models was reported on MP-20.MAPPS combines workflow planning, code generation, and expert guidance.Source: §1 · p. 2
• MAPPS combines a Workflow Planner, Tool Code Generator, and Scientific Mediator with human scientists to plan tasks, generate code, and integrate expert guidance.Its workflow-planning approach starts from high-level goals and scientific intuition rather than prescribed step-by-step procedures.Source: §1 · p. 2

2 Materials Agents Unifying Planning, Physics, and Scientists

• Crystal discovery covers unconditional stable-structure generation, composition-conditioned structure prediction, and property-guided design, represented as sequential actions.A crystal structure is represented as atom types, Cartesian coordinates, and a lattice matrix.Source: §2 · p. 2

2.1 Different Levels of Autonomy in Science Agent Design

• The three autonomy levels range from executing fixed workflows, through human-guided planning, to designing workflows freely, with greater validity and reliability challenges at Level 3.Most existing materials agents are described as Level 1, executing isolated tasks in expert-designed workflows.Source: §2.1 · pp. 2–3, §2.2 · p. 3

2.2 Overview of MAPPS

• MAPPS targets Level 2 autonomy: its Workflow Planner creates multi-step plans, its Tool Code Generator synthesizes code with physics tools, and its Scientific Mediator coordinates people and agents.High-level goals, constraints, or domain heuristics guide the agents' workflow construction.Source: §2.2 · p. 3

2.3 Workflow Planner

• The Workflow Planner generates actionable workflow steps from a task description and human-provided intuition, which guides plans toward validity while slightly limiting freedom to explore arbitrary strategies.The planner uses a large reasoning model; the human intuition can include domain-specific heuristics.Source: §2.3 · p. 3

2.4 Scientific Mediator

• The Scientific Mediator sends plans to scientists for review, then coordinates execution and seeks human approval or feedback on each intermediate result.It passes the current action, previous result, and step-specific human intuition to the Tool Code Generator.Source: §2.4 · p. 4

2.5 Tool Code Generator

• The Tool Code Generator translates workflow steps into Python functions, integrates physics tools such as symmetry analyzers and force fields, and revises code after runtime errors.ML force fields relax candidate structures toward energetically favorable configurations with less computational overhead than DFT calculations.Source: §2.5 · p. 5

Crystal Structure Generation

• Existing crystal-generation methods include diffusion-based models such as CDVAE and DiffCSP, and language-model approaches such as Mat2Seq and CrystaLLM.DiffCSP conditions generation on chemical composition and uses surrogate energy models; Mat2Seq generates token sequences, while CrystaLLM uses CIF-like representations.Source: §3 · pp. 5–6
• These generative approaches are designed for specific tasks and assist discovery but do not plan, reason, guide, or control the discovery process.Source: §3 · p. 6

LLM Agents for Science

• Materials-science agents include systems for alloy discovery, zeolite synthesis, material modification, and crystal generation, but typically execute predefined, expert-designed workflow steps.The passage also describes tool-integrating agents in other scientific fields, including ChemCrow for autonomous chemical synthesis.Source: §3 · p. 6

Differences with Prior Work

• MAPPS introduces a Level 2 framework for human-guided planning and reports outperforming Level 1 agent systems, including systems following expert-crafted workflows.It autonomously designs workflows, implements code, and incorporates human-expert intuition and feedback.Source: §3 · p. 6

4 Experiments

• Experiments cover real-world crystal generation, prediction, and desired-property discovery, with workflow generation analyzed separately.The experiments use the OpenAI API and a single NVIDIA A100 GPU.Source: §4 · p. 6

4.1 Crystal Structure Generation

• The generation evaluation uses MP-20 and Matbench retrieval databases, generates 1,000 candidates per dataset, and assesses validity, metastability, stability, and S.U.N. rate.S.U.N. denotes stable, unique, and novel structures.Source: §4.1 · pp. 6–7
• MAPPS outperforms CDVAE, DiffCSP, FlowMM, and CrystalTextLLM on MP-20, and surpasses MatLLMSearch on Matbench without training a new model.The authors attribute this to extracting scientific knowledge embedded in pretrained LLMs.Source: Table 1 · p. 7, §4.1 · p. 7, Table 2 · p. 7

4.2 Crystal Structure Prediction

• Prediction is evaluated on MP-20, MPTS-52, and a recent-discovery challenge set, with match rate and RMSE; baselines receive training data and MAPPS receives corresponding retrieval data.Each model generates a one-shot candidate per input composition.Source: §4.2 · pp. 7–8

4.3 Discovering crystal structures with desired properties

• Bandgap-guided generation tests high-bandgap targets above 3 eV and low-bandgap targets below 0.5 eV, generating 500 structures per condition and evaluating bandgaps with DFT.Source: §4.3 · p. 8

4.2 Crystal Structure Prediction

• 63.9% match rate and 0.022 RMSE on MP-20; on MPTS-52, MAPPS reaches 27.6% match rate and 0.097 RMSE, beating Mat2Seq by 4.5% in match rate.The reported challenge-set result is a 31.0% match rate and 0.055 RMSE, outperforming CrystaLLM by 8.6% in match rate.Source: §4.2 · p. 8, Table 3 · p. 9

4.3 Discovering crystal structures with desired properties

• 74.6% of high-bandgap candidates exceed 3 eV, while 92.2% of low-bandgap candidates fall below 0.5 eV; validity, uniqueness, and novelty each exceed 90%.Source: §4.3 · pp. 8–9, Table 4 · p. 9

4.4 Can current LLMs achieve level 3?

• With human intuition, O3-mini reaches 60% workflow validity on CSP and CSD and 100% on CSG; without guidance, all tested models fail on all three tasks.O3-mini workflows average 4.37 steps, while other models often produce five-step workflows with useless or redundant steps.Source: Table 5 · p. 9, §4.4 · p. 9

5 Summary and Outlook

• MAPPS combines a Workflow Planner, a Tool Code Generator that synthesizes executable physics-grounded code, and a Scientific Mediator that incorporates human intuition and feedback.Source: §5 · p. 9
• MAPPS performs better than Level 1 agent systems, but it does not yet reach full Level 3 autonomy.The authors describe full Level 3 autonomy as agents independently designing and adapting workflows without human input.Source: §5 · p. 10
• MAPPS currently relies on human experts and focuses on materials discovery; extensions to molecules, polymers, and other systems remain underexplored.Source: §5 · p. 10

A.1 Workflow Planning for Crystal Structure Prediction

• The Workflow Planner maps the task description and human intuition to a multi-step action sequence.Source: §A.1 · p. 13
• For Ba2Fe2F9, the example supplies a structure-prediction task and intuition about force fields, similar compositions, and available structural prototypes.Source: §A.1 · p. 13
• The workflow retrieves similar prototypes, generates candidate structures, optimizes them with machine-learning force fields, compares energies, and validates the lowest-energy candidate when reference data are available.Source: §A.1 · p. 13
• The example workflow ends by asking the scientist to review it and suggest revisions.Source: §A.1 · p. 13

A.2 Tool Code Generation

• For step t, the Tool Code Generator receives the current action, previous result, expert intuition, and available physics tools, then synthesizes code c_t.Source: §A.2 · p. 13
• The prompt template requests one executable Python function named stepX, absolute file paths, saved result files with a returned folder path, and a descriptive purpose comment.Source: §A.2 · p. 14
• For the optimization step, the example combines candidate CIF files, expert guidance about CHGNetCalculator, and tools including CHGNetCalculator, ASE, and pymatgen.Source: §A.2 · p. 14
• The generated function reads candidate CIFs, assigns CHGNetCalculator, optimizes lattice and atomic positions with ExpCellFilter and BFGS for up to 100 steps, and saves optimized structures.Source: §A.2 · p. 14
• The example is described as a successful Tool Code Generator output for structure relaxation with physics-informed machine-learning force fields.Source: §A.2 · p. 15

B Bandgap Distribution under Targeted Generation

• Many of 500 generated structures meet the target thresholds: below 0.5 eV for low-bandgap generation and above 3 eV for high-bandgap generation.The low-bandgap setting yields many narrow-gap candidates, while the high-bandgap distribution shifts toward larger values.Source: B Bandgap Distribution under Targeted… · p. 15

• Metrics cover six aspects of generated crystals, and are generally computed after DFT or surrogate post-processing and relaxation.Source: C Evaluation Metric Details · p. 15
• Structural validity requires pairwise distances of at least 0.5 Å and unit-cell volume of at least 0.1 Å3, while compositional validity requires SMACT charge-neutrality and electronegativity checks.Source: C Evaluation Metric Details · p. 15
• Stable crystals have DFT energy above the convex hull below 0.0 eV/atom and at least two unique elements.Source: C Evaluation Metric Details · p. 15
• Uniqueness is the fraction of stable crystals that are mutually distinct, while novelty requires no StructureMatcher match in the original dataset.Source: C Evaluation Metric Details · p. 15
• For CSP, match rate is the percentage of generated structures matching the ground truth for their composition under StructureMatcher.Source: C Evaluation Metric Details · p. 16
• For matching CSP structures, RMSE measures fractional-coordinate error after alignment by symmetry operations and cell transformation.Source: C Evaluation Metric Details · p. 16

• MP-20 stability and S.U.N evaluations use PBE within GGA in VASP, with inputs generated by MPRelaxSet and energies measured against the Matbench Discovery convex hull [33, 34, 35, 36, 37, 38].Source: D DFT calculations · p. 16
• JARVIS-DFT evaluation uses vdW-DF-OptB88 with a 0.001 eV Å−1 force-convergence threshold and a 10−7 eV electronic self-consistency threshold [1, 39].Source: D DFT calculations · p. 16