PaperlayerView exact revision on arXiv

Source-linked AI summary

ALL-FEM: Agentic Large Language models Fine-tuned for Finite Element Methods

Rushikesh Deotale, Adithya Srinivasan, Yuan Tian, Tianyi Zhang, Pavlos Vlachos, Hector Gomez

arXiv:2603.21011v2cs.CEcs.AIcs.LGcs.MSmath.NA

TL;DR

Finite-element analysis requires substantial numerical, mechanics, and programming expertise, while conventional LLM-generated code can be unreliable. ALL-FEM combines verified FEniCS data, domain-specific fine-tuning, and multi-agent runtime feedback to automate the workflow. On 39 benchmarks, fine-tuned GPT-OSS 120B in the multi-agent framework achieved 71.79% code-level success and surpassed non-agentic GPT-5 Thinking within the reported setup.

  • Problem

    Finite-element code development and validation are expertise-intensive, while conventional LLMs can generate hallucinated or incorrect code and cannot reliably close the loop to verified solutions.

  • Method

    ALL-FEM builds over 1000 FEniCS codes from expert seeds and synthetic augmentation, fine-tunes 3B–120B LLMs, and embeds them in agentic workflows with execution feedback.

  • Results

    71.79% code-level success was achieved by fine-tuned GPT-OSS 120B in a multi-agent framework, surpassing non-agentic GPT-5 Thinking on 39 benchmark problems.

  • Takeaways & Limitations

    Fine-tuned models, including relatively small ones, can produce correct finite-element codes when embedded in an appropriate agentic framework.

  • Takeaways & Limitations

    The framework is not fully end-to-end autonomous because a human must terminate runs after the Evaluator indicates acceptance.

Abstract

from arXiv · show

Finite element (FE) analysis guides the design and verification of nearly all manufactured objects. It is at the core of computational engineering, enabling simulation of complex physical systems, from fluids and solids to multiphysics systems. However, implementing FE codes and analyzing simulation results demands expertise across numerical analysis, continuum mechanics, and programming. Conventional Large Language Models (LLMs) can generate FE code, but they hallucinate, lack awareness of variational structures, and cannot close the loop from problem statement to a verified solution. Here, we propose ALL-FEM, an autonomous simulation system that integrates agentic AI with domain-specific, fine-tuned LLMs for FEniCS code generation across solid, fluid, and multiphysics applications. We construct a corpus of 1000+ verified FEniCS scripts by combining 500+ curated expert codes with a retrieval-augmented, multi-LLM pipeline that generates and filters codes for diverse PDEs, geometries, and boundary conditions. We used the corpus to fine-tune LLMs with 3B to 120B parameters. Our agentic framework orchestrates specialized agents, powered by fine-tuned LLMs, to formulate problems as PDEs, generate and debug code and visualize the results. We evaluated the system on 39 benchmarks that include problems of linear/nonlinear elasticity, plasticity, Newtonian/non-Newtonian flow, thermofluids, fluid-structure interaction, phase separation, and transport on moving domains. Embedded in a multi-agent workflow with runtime feedback, the best fine-tuned model (GPT OSS 120B) achieves code-level success of 71.79%, outperforming a non-agentic deployment of GPT 5 Thinking. By showing that relatively small, fine-tuned LLMs, orchestrated through agentic frameworks, can automate FE workflows, ALL-FEM offers a blueprint for autonomous simulation systems in computational science and engineering.

1 Introduction

ALL-FEM addresses the expertise, reliability, and duplication costs of finite-element workflows by combining verified FEniCS data, domain-specific fine-tuning, and coordinated agents. Across diverse computational-mechanics problems, the system automates formulation, coding, debugging, validation, and visualization, achieving 71.79% correct code on 39 benchmarks.

  • Motivation: Finite element workflows support simulation across fluids, solids, and multiphysics, but implementing, testing, and validating codes remains time-consuming and expertise-intensive.Even high-level libraries require knowledge of variational formulations, time integration, and linear algebra.
  • Motivation: Conventional LLMs can generate computational-mechanics code but often hallucinate syntactic or semantic errors, limiting use where correctness is critical.Existing software also involves duplicated development effort, while commercial tools impose high licensing and operational costs and constrain non-standard problems.
  • Approach: ALL-FEM fine-tunes 3B–120B open-weight LLMs on over 1000 FEniCS codes generated from expert seeds through a multi-LLM data-augmentation pipeline.The pipeline expands a small curated dataset into diverse, verified codes for PDE boundary-value problems.
  • Approach: Its multi-agent workflow formulates weak problems and boundary conditions, generates and debugs FEniCS code, checks formulation–implementation consistency, and plots numerical results.Agent selection and coordination are orchestrated by an LLM to adapt across computational-mechanics challenges.
  • Evaluation: 71.79% of benchmark problems produced correct code, outperforming a non-agentic deployment of GPT-5 Thinking across 39 problems spanning solid, fluid, thermofluid, multiphysics, and transport tasks.The benchmark includes elasticity, plasticity, Newtonian and non-Newtonian flow, fluid-structure interaction, multiphase models, and diffusion-reaction systems on moving domains.

2 Methods

ALL-FEM develops domain-specialized fine-tuned LLMs and a multi-agent workflow for automating FEniCS-based computational mechanics. Its methods combine an augmented code dataset, parameter-efficient adaptation, and validation across diverse simulation tasks.

  • LLM selection: The study selects open-source models from 3B to 120B parameters for fine-tuning on FEniCS code generation.The selected models include Llama 3.2 3B, Qwen3 32B, Llama 3.3 70B, and GPT-OSS 120B; GPT-5 is not fine-tuned.
  • Dataset construction: The dataset represents computational-mechanics problems through instruction, input, and output triples describing the problem and its FEniCS solution.Entries include governing equations, geometry, boundary conditions, and corresponding code, using the Alpaca format.
  • Dataset construction: The training dataset combines curated FEniCS resources and in-house scripts with automated augmentation, yielding 1004 codes solving computational-mechanics PDEs.The augmentation pipeline uses a multi-model workflow with retrieval-augmented generation to generate additional synthetic samples from the seed dataset.
  • Parameter-efficient fine-tuning: LoRA adapts pretrained models by adding a scaled low-rank update to the original weights rather than updating all parameters.The update uses matrices A and B with rank r much smaller than the weight dimensions; α controls update influence, while r controls adaptation capacity.

2.3 Agentic AI

ALL-FEM uses agentic frameworks to divide finite-element problem solving across specialized agents and close the loop with execution feedback. The multi-agent design coordinates planning, formulation, coding, execution, correction, evaluation, and termination, while retaining automated code revision.

  • Two-agent Framework: The two-agent framework uses an Assistant Agent to generate FEniCS code and a User Proxy Agent to execute it and return runtime feedback.The coder revises the implementation from returned errors until execution succeeds or the token limit is reached.
  • Two-agent Framework: A single LLM in the two-agent setup handles problem formulation, coding, debugging, and execution through a fixed interaction loop.The framework does not require a Group Chat Manager because the interaction sequence is predetermined.
  • Two-agent Framework: The two-agent setup can lose critical constraints as error logs and code revisions exhaust the context window, while executable code may still contain formulation or implementation errors.The framework revises code after compilation failures but receives no numerical feedback to detect executable yet incorrect solutions.
  • Multi-Agent Framework: The multi-agent framework assigns planning, formulation, coding, execution, critique, correction, and coordination to five LLM-powered Assistant Agents, two User Proxy Agents, and a Group Chat Manager.The Coordinator selects agents based on the conversation context and information needed to make progress.
  • Multi-Agent Framework: During code generation, the FEniCS Coder, Executor, and Corrector iterate until execution succeeds, after which control returns to the Coordinator.The Executor reports exit status and errors, and the Corrector proposes code modifications when execution fails.
  • Multi-Agent Framework: The multi-agent workflow remains automated, but an Admin agent is used for manual termination because the Coordinator otherwise keeps selecting agents indefinitely.The human decides whether to exit or continue after the Evaluator responds.

3.1 Evaluation Methodology

The evaluation benchmarks agentic FEniCS systems against GPT-5 Thinking on expert-designed computational mechanics problems spanning increasing difficulty and diverse physical settings. Correctness is assessed against reference solutions, with accuracy defined as the fraction of correctly solved questions.

  • Benchmark Design: 39 expert-designed problems, separate from training data, evaluate the agentic frameworks across computational mechanics tasks.The test set emphasizes coupled and higher-complexity systems, including five problems with PDE families or couplings absent from training.
  • Benchmark Design: The benchmark includes easy solid mechanics, medium fluid mechanics, and hard multiphysics problems with specified geometries, governing models, boundary conditions, parameters, meshes, and outputs.Examples include plane-stress elasticity, buoyancy-driven flow with heat transport, and fluid-structure interaction involving a moving elastic flag.
  • Correctness Assessment: Reference-solution evaluation first labels non-executable codes incorrect, then compares executable results through a multi-step process against expert-prepared solutions.The evaluation uses reference solutions for every question rather than relying solely on code execution.
  • Correctness Assessment: Relative L2 error was not selected as the primary metric because frameworks can use different meshes, making common-space transfer difficult and potentially introducing projection errors.The chosen evaluation avoids requiring robust automated projection between finite-element spaces.
  • Correctness Assessment: Accuracy is defined as the number of correctly solved questions divided by the total number of questions, and Figure 5 compares frameworks across difficulty levels.The figure distinguishes two-agent and multi-agent results and includes GPT-5 Thinking as a reference.

3.2 Results

Across the 39-problem test set, larger models and FEniCS fine-tuning improved two-agent performance across difficulty levels and physics categories. The multi-agent framework added limited gains over GPT-OSS 120B FT, while the evaluation remains bounded by mesh and geometry choices and human termination.

  • Model Size Improves Performance in a Two-agent Framework: 71.79% accuracy was achieved by GPT-OSS 120B FT across all 39 test problems, exceeding Llama 3.3 70B FT at 20.51% and Qwen 3 32B FT at 12.82%.The same benchmark reports 58.97% for non-fine-tuned GPT-OSS 120B.
  • Model Size Improves Performance in a Two-agent Framework: GPT-OSS 120B FT solved 15 solid, 11 fluid, and 2 multiphysics questions, compared with 6, 2, and 0 for Llama 3.3 70B.The comparison spans all three physics categories shown in Figure 6.
  • Fine-tuning Improves Performance in a Two-agent Framework: Fine-tuning produced consistent gains across two-agent frameworks, including 8 correctly solved problems for Llama 3.3 70B FT versus none for its non-fine-tuned version.Fine-tuning also enabled Qwen 3 32B FT to outperform the larger non-fine-tuned Llama 3.3 70B.
  • Fine-tuning Improves Performance in a Two-agent Framework: On a plane-stress elasticity problem, the fine-tuned model correctly applied plane stress, whereas the base model incorrectly assumed plane strain.The comparison attributes the improved result to interpreting and applying the plane-stress assumption correctly.
  • Comparison with GPT-5 Thinking: GPT-OSS FT outperformed GPT-5 Thinking across easy, medium, and hard sets despite fewer parameters and lower inference cost.The authors present this combination as a lower-cost open-source alternative for FEniCS computational mechanics problems.
  • Multi-agent Framework Performs Better than Two-agent Framework: The multi-agent framework solved two more medium-difficulty questions than two-agent GPT-OSS-FT, while both had the same easy- and hard-set accuracy.For the hard-set example, the multi-agent framework produced better code but still failed to impose boundary conditions accurately.
  • Limitations of the framework: The benchmark does not specify mesh details, so different frameworks may use different meshes and cannot be compared uniformly with a single L2-error metric.The authors instead use a standardized multi-step comparison performed by domain experts.
  • Limitations of the framework: The multi-agent system still requires a human Admin step to terminate execution after the Evaluator confirms acceptance.In the studies, the human supplied no problem-solving help or corrections.

4 Conclusions

ALL-FEM combines curated domain data, fine-tuned LLMs, and agentic orchestration to automate substantial portions of FEniCS-based finite-element workflows. Across diverse benchmarks, the framework improved code correctness, though reliable fully automatic simulation remains unresolved.

  • System and dataset: ALL-FEM combines over 500 expert-generated FEniCS codes, synthetic code generation, 3B–120B-parameter fine-tuning, and agentic workflows.The system includes both a minimal Coder-Executor loop and a richer multi-agent system.
  • Evaluation and findings: Runtime execution feedback improved performance, while fine-tuning consistently improved code correctness across 39 problems spanning solid, fluid, and multiphysics applications.The benchmark includes elasticity, plasticity, flow, thermofluids, fluid-structure interaction, phase separation, and transport on moving domains.
  • Evaluation and findings: GPT-OSS 120B achieved the highest performance in the multi-agent framework and surpassed a non-agentic deployment of GPT-5 Thinking within the study setup.The result supports the reported comparison between fine-tuned models used agentically and larger proprietary models used non-agentically.
  • Limitations and future work: A reliable and automatic simulation system is not yet possible, leaving gaps for further research in dataset scale, agent coverage, and result analysis.The authors specifically identify larger code resources and result-analyzer agents as future priorities.
  • Conclusion: Combining curated domain data, parameter-efficient fine-tuning, and agentic orchestration is presented as a feasible approach for autonomous simulation systems in computational mechanics.The authors suggest that similar approaches may extend to other software-dependent scientific and engineering workflows.

Appendix A Code correctness criterion based on the L2 norm

The appendix validates the multi-step code-correctness procedure against L2-error criteria using interpolated reference solutions and same-mesh comparisons. Agreement with the same-mesh test supports using the multi-step procedure for evaluation.

  • Validation approach: The validation compares the multi-step solution-comparison process with quantitative L2-error metrics on five representative problems.The tests compare pass/fail outcomes from the two evaluation approaches.
  • Test 1: Test 1 interpolates framework-generated fields onto a finer reference mesh before computing relative L2 errors for primary fields.Outputs are labeled correct when the relative L2 error falls below a threshold swept from 1% to 4%.
  • Test 2: Test 2 removes interpolation by constructing matched reference solutions on each framework-generated mesh with identical discretization settings.The governing equations, conditions, and physical parameters remain identical to the original reference formulation.
  • Test 1: Table A1 reports Test 1 sensitivity of pass counts and average L2 error norms relative to the multi-step evaluation process.The table is used to examine threshold sensitivity in the interpolated-reference comparison.
  • Test 2: Table A2 compares the multi-step evaluation process against a 0.001% L2-error threshold applied directly on framework-generated meshes.This same-mesh comparison avoids interpolation-related threshold sensitivity.

Appendix B Mesh statistics for the Multi-agent framework

Appendix B reports mesh statistics for all benchmark problems in the multi-agent framework, organized by solid, fluid, and multiphysics categories. These statistics make the spatial discretizations and their relationship to performance variations verifiable.

  • Overview: Mesh statistics are reported for every benchmark problem executed in the multi-agent framework.The appendix provides discretization information to support verification of the benchmark setup.
  • Reported statistics: Each case reports the number of elements, maximum and minimum element sizes, and number of degrees of freedom.The listed quantities are hmax, hmin, and DOFs.
  • Solid mechanics: Table B1 covers solid mechanics Problems 1–16.The table groups mesh statistics for the solid mechanics benchmark set.
  • Fluid and multiphysics: Table B2 covers fluid mechanics Problems 1–15, while Table B3 covers multiphysics Problems 1–8.The tables are grouped by benchmark category to reflect differing governing equations and discretization choices.

Appendix C Computational cost

The computational-cost appendix compares the two-agent and multi-agent workflows using API calls, token usage, and wall-clock time. Its representative examples provide practical cost differences rather than an exhaustive benchmark-wide accounting.

  • Cost metrics: Computational cost is measured using API calls, prompt, completion, and total tokens, plus wall-clock time in minutes.These metrics are used to compare agentic framework runs.
  • Comparison scope: Table C1 compares computational costs for fine-tuned GPT-OSS runs on three representative example problems.The reported subset is intended to show practical cost differences between the two-agent and multi-agent workflows.
  • Comparison scope: The cost comparison is not an exhaustive accounting across the full benchmark.The appendix explicitly limits the interpretation of Table C1 to a practical sense of workflow cost differences.

Appendix D Sample Solution Plots

The appendix presents representative solution-field plots comparing two-agent, GPT-5 Thinking, and multi-agent frameworks across selected solid- and fluid-mechanics problems.

  • Visual comparison: For Solid Mechanics Q1, all frameworks produce displacement fields visually identical to the reference solution.This comparison is stated in the accompanying discussion of the representative plots.
  • Solid Mechanics: Representative plots cover Solid Mechanics Q1, Q4, and Q15 alongside corresponding framework comparisons.The cited figures identify displacement or solution fields for these solid-mechanics test problems.
  • Fluid Mechanics: Representative plots cover Fluid Mechanics Q7 and Q10 for the two-agent, GPT-5 Thinking, and multi-agent frameworks.The captions specify the compared frameworks and fluid-mechanics problems.

Appendix E Benchmark Problems

The appendix lists the 39 computational-mechanics problems used to evaluate the agentic systems and notes that they are displayed in LaTeX fonts but prompted as plain text with Unicode symbols.

  • Benchmark scope: The evaluation set contains 39 computational-mechanics problems.The appendix introduces the complete benchmark-problem list used for evaluation.
  • Prompt format: The listed problems are shown in LaTeX fonts, while the agent prompts use plain text with Unicode symbols.This describes a presentation and prompting-format distinction.

Fluid Mechanics Problems

The fluid-mechanics benchmark problems specify geometries, meshes, governing flow models, material parameters, boundary conditions, and requested outputs for progressively varied simulations.

  • Governing models: The fluid problems include Stokes flow, Navier–Stokes flow, transport, and flow through channels, cavities, steps, and perforated domains.Examples include body-force Stokes flow, lid-driven cavities, backward-facing steps, flow around a circular hole, and advection-diffusion transport.
  • Boundary conditions: Boundary conditions span no-slip walls, prescribed inlet or lid velocities, traction-free outlets, periodicity, pressure gauges, and concentration conditions.The listed problems combine these conditions in different flow and transport configurations.
  • Discretization: The benchmark specifies structured meshes ranging from 96 × 96 and 128 × 128 elements to channel meshes such as 160 × 16 and 200 × 20.Mesh resolutions are provided for several individual fluid problems.
  • Outputs: Fluid outputs include speed, velocity, pressure, concentration, drag coefficient, and wall-shear-stress fields or derived quantities.The requested files and diagnostics vary by problem, including drag around a circular obstacle and re-attachment based on wall shear stress.

Solid Mechanics Problems

The solid-mechanics benchmark problems cover linear and nonlinear elasticity with varied geometries, materials, loading conditions, constitutive assumptions, and field outputs.

  • Discretization: The solid benchmarks use structured meshes including 20 × 4, 40 × 8, 50 × 25, and 100 × 20 subdivisions.Mesh specifications vary across the listed solid-mechanics problems.
  • Models and geometries: The problems include plane-stress and plane-strain elasticity for plates, strips, holes, notches, layered materials, and heterogeneous domains.Geometries range from rectangular plates and strips to plates with circular or semicircular cutouts and bonded material subdomains.
  • Boundary conditions and loading: Loads and constraints include clamping, prescribed displacements, uniform tractions, traction-free boundaries, and follower pressure on hole boundaries.Different problems impose these conditions on plate, strip, notch, and hole geometries.
  • Outputs: Requested outputs include displacement fields, displacement-magnitude maps, von Mises stress, and stress-concentration factors.Several problems require stress extrema at hole boundaries or visualization and XDMF export of computed fields.
  • Materials: Material models include isotropic linear elasticity, orthotropic rotated stiffness, spatially varying Young’s modulus, and nearly incompressible formulations.The benchmark specifies both conventional material constants and specialized constitutive settings.

Multiphysics Problems

The section specifies hard multiphysics finite-element problems spanning phase-field evolution, transport on moving domains, coupled Stokes–Darcy flow, and fluid–structure interaction. Each problem defines governing equations, geometry, boundary or interface conditions, numerical parameters, and requested outputs.

  • Transport on moving domains: The moving-domain transport problem solves a diffusion-reaction equation with first-order decay on an expanding circular disk.The disk radius grows as R(t) = R0 + s·t, with D = 1.0 × 10−5 m2 s−1, κ = 1.0 × 10−4 s−1, and Δt = 0.01 s.
  • Phase-field evolution: The Allen–Cahn problem evolves a phase field with implicit time stepping, homogeneous Neumann boundaries, and a square-interface initialization.It uses ε = 0.01, M = 1.0, Δt = 1.0e−3, and runs to T = 0.20.
  • Phase-field evolution: The Cahn–Hilliard problem models spinodal decomposition with concentration-dependent mobility, periodic boundaries, and backward Euler time integration.The mixed formulation evolves concentration and chemical potential using θ = 1.5 and α = 3000.
  • Coupled flow: The Stokes–Darcy problem couples incompressible Stokes flow above an interface to incompressible Darcy flow below it.The interface imposes mass continuity, normal traction balance, and a Beavers–Joseph–Saffman tangential condition.
  • Fluid–structure interaction: The fluid–structure interaction problems model an elastic tube and a flexible flag in moving incompressible-fluid domains.They enforce no-slip and traction balance, while using plane-strain linear elasticity for the tube walls and a St. Venant–Kirchhoff model for the flag.
Loading 2603.21011v2…