PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM-Assisted Optimization for Simulation-Based Redesign

Chan-Eui Song, Jeremy J. Michalek, Christopher McComb

arXiv:2610.39275v1cs.AIcs.CEmath.OC

TL;DR

Simulator-driven redesign must adapt an existing design to changed requirements, yet numerical optimization may not use the engineering context of those changes. This paper compares three ways to integrate LLM guidance with numerical search in battery-pack redesign. LLM-assisted reoptimization finds feasible designs in fewer evaluations and at lower cost than numerical baselines, while the best integration depends on the requirement change.

  • Problem

    In simulator-driven redesign, numerical search uses evaluation history while engineering knowledge about changed context and reusable parts of a prior design may not be captured there.

    Source: §1 · p. 1

  • Method

    The study compares warm starting, variable fixing, and ReAct for battery-pack redesign, placing LLM guidance at different stages of simulator-driven reoptimization.

    Source: §1 · p. 2

  • Results

    LLM-assisted reoptimization reaches feasible redesigns in fewer simulator evaluations and at lower cost than cold-start and reference-warm numerical optimization under a common budget.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The most effective LLM-assisted method depends on the redesign scenario, and larger-model benefits grow with the extent of LLM control over search.

    Source: §5 · p. 11

    The comparison uses one simulator and one product family, and different numerical-optimizer constraint-handling choices could change the comparison.

    Source: §5 · p. 11

Abstract

from arXiv · show

Engineering design frequently involves redesigning a prior solution to adapt to changes in context, such as material prices or system requirements. Redesign often leverages simulators too complex and/or computationally intensive for traditional numerical solvers, which lack engineering knowledge and context, to ensure global solution quality. Large language models (LLMs) can interpret semantic information, including engineering knowledge about the system and changes in context, to help guide search, and recent studies have shown that incorporating this knowledge into optimization can improve search performance. However, how best to combine LLMs with numerical solvers in redesign remains unclear. We develop and test an integrated LLM-optimization approach for redesign and compare three methods for leveraging LLMs (warm start, variable fixing and ReAct) and four black-box solvers (Bayesian optimization, genetic algorithms, mesh adaptive direct search and random search) on an electric vehicle battery pack cost minimization case study across four types of redesign scenarios. We find that the integrated approach can match feasible design discovery of the numerical-solver baselines with 80% fewer simulator evaluations while tripling the fraction of runs that find near-optimal solutions. We find that the best LLM-solver integration approach and the way the prior solution is leveraged depend on the structure of the design requirement change, and that the benefit of increasing LLM model size grows with the extent to which the LLM controls the search. These results show clear potential for LLMs in redesign tasks for guiding or replacing numerical solvers.

1 Introduction

Redesign reuses an existing design under changed requirements, but simulator-driven optimization can require many evaluations and numerical histories may not capture the engineering meaning of the change. This study tests where LLM guidance belongs in that process and finds its most effective role depends on the redesign scenario.

1 Introduction

  • Numerical search relies mainly on evaluation history, while engineers can use knowledge of what changed and which parts of the prior design remain useful.Prior evaluations may become obsolete or misleading when the design context changes.Source: §1 · p. 1
  • The study compares warm starting, variable fixing, and ReAct because semantic guidance can shape initial designs, preserve reference-design decisions, or interpret feedback during search.These roles are evaluated on battery-pack redesign scenarios involving material cost, pack space, fast charging, and vehicle energy requirements.Source: §1 · p. 2
  • LLM-assisted reoptimization reaches feasible redesigns in fewer simulator evaluations and at lower cost than cold-start and reference-warm numerical optimization under a common budget.The contribution frames these comparisons as evidence for reducing simulator-driven redesign effort.Source: §1 · p. 2
  • The benefit of larger models grows with their control over search, peaking for ReAct and remaining inconsistent for warm starting and variable fixing.The latter methods use a numerical optimizer for the remaining search, and the study reports inference time and token use alongside performance.Source: §1 · p. 2

2 Battery-Pack Redesign Problem

The study formulates battery-pack redesign as simulator-based reoptimization of a reference design under changed conditions. Four scenarios alter material prices, available space, charging time, or energy requirements.

2 Battery-Pack Redesign Problem

  • The testbed uses BatPaC to evaluate coupled cell-, module-, and pack-level design decisions through a shared external input-output interface.The five design variables include cathode chemistry, positive-electrode thickness, cells per module, modules per row, and module rows.Source: §2.1 · p. 2

2.1 Battery-Pack Design Problem.

  • The cost-minimization formulation constrains energy, power, voltage, cell capacity, and pack dimensions, with practical chemistry-specific upper bounds on electrode thickness.The reference design costs 96.5 USD/kWh and serves as the best-known solution for the reference condition.Source: §2.1 · p. 3

2.2 Redesign Scenarios.

  • The four scenarios change distinct requirements: cobalt-related material cost, pack width, recharge time, or target energy and packaging space.S1 leaves feasibility requirements unchanged; S2 invalidates the 1400 mm-wide reference layout with a 1100 mm width limit; S3 halves recharge time from 60 to 30 minutes; S4 raises energy from 84 to 130 kWh and requires topology redistribution.Source: §2.2 · pp. 3–4

3 Methods

The methods frame redesign as reoptimization of a known reference design after conditions change, then compare LLM roles and numerical optimizers under a shared simulator budget. Configurations vary semantic information, model size, and search strategy, with performance and computational effort measured across repeated runs.

3.1 Engineering Redesign.

  • Redesign reconsiders the best-known reference design when the design condition changes from the reference condition to a target condition.The redesign procedure has access to both the formulation and the reference design.Source: §3.1 · p. 5

3.2 Experimental Design.

  • Runs vary scenario, semantic-information level, and model size, with cold-start and reference-warm baselines compared against three WAFER methods.Every method receives 20 accepted BatPaC evaluations; ReAct proposes candidates itself, while warm starting and variable fixing also select a downstream optimizer.Source: §3.2 · p. 5

3.3 Semantic Information and Language Models.

  • L1 supplies the problem definition, while L2 adds compact engineering relationships relevant to battery redesign; the study uses Qwen3.5 at 4B, 9B, and 27B sizes.The comparison tests whether explicit engineering relationships add guidance beyond the problem definition alone.Source: §3.3 · p. 5

3.4 LLM Integration Strategies.

  • Warm starting proposes initial designs, variable fixing selects reference variables to retain, and ReAct uses simulator feedback throughout iterative search.ReAct carries prior actions and observations into each subsequent prompt.Source: §3.4 · p. 5

3.5 Numerical Search and Reference Conditions.

  • Warm starting and variable fixing are paired with Bayesian optimization, a mixed-variable genetic algorithm, MADS, or random search.The four optimizers receive the same penalized scalar objective, and non-LLM baselines use the same optimization methods.Source: §3.5 · p. 6

3.6 Evaluation and Statistical Analysis.

  • Evaluation measures feasible-design discovery, the first feasible evaluation, lowest feasible cost, LLM time and token use, and unusable decisions across 50-seed runs.The study uses Fisher exact tests with Holm adjustment for proportions, Wilson intervals, and Spearman rank correlation for the specified association.Source: §3.6 · p. 6

4 Results

The results assess how LLM-assisted redesign strategies and prompt engineering information relate to optimization outcomes and redesign decisions across different problem structures. They also present a strategy-selection rule and note that a larger iteration budget could change the observed trends.

4.1 LLMs can improve feasible discovery and design qual-

  • After four evaluations, LLM-assisted runs reach the 39% feasible-discovery rate of numerical baselines after 20; 23% finish within 2% of best-known cost versus 7% for baselines.Across scenarios, WS and ReAct generally outperform numerical methods, while VF is dominant only in S3. Figures 3 and 4 pool configurations without matching methods to scenarios.Source: Figure 3 · p. 6, Figure 4 · p. 7, §4.1 · p. 7
  • With optimizers matched, WS and ReAct are significantly better than RW in many comparisons and worse in few, while VF is significantly worse in most S1 and S4 comparisons and better in S3.VF is significantly better in every S3 comparison for 2% gap attainment and in most S3 discovery comparisons.Source: Table 4 · p. 7, §4.1 · p. 8

4.2 The best use of LLMs depends on the redesign scenario.

  • The best LLM strategy varies by scenario: WS and ReAct perform best in S1, S2, and S4, while VF and ReAct perform best in S3.The reference design is already feasible in S1, but lowering its cost remains difficult; two 27B ReAct configurations reach 100% discovery and the lowest median final cost in S3 and S4, despite using no numerical solver.Source: §4.2 · p. 8

4.2.1 Variable fixing works best when the necessary redesign involves only a few variables.

  • Variable fixing works best when required redesign variables are easy to identify, because releasing fewer variables can shrink the search space and improve search efficiency.The required redesign variables are xM and xR in S2, xT in S3, and xM in S4; VF and WS judgments correlate across scenario-variable pairs (Spearman ρs = 0.79). In S3, 499 of 509 runs on RRFFF reached feasibility, compared with 39 of 204 on RRRRR; incorrect fixing can eliminate feasibility, while a suitable optimizer such as MADS can help in narrow subspaces.Source: §4.2 · p. 8, §4.2.1 · p. 9

4.2.2 Warm starting works best when initial points focus on promising redesign directions.

  • For warm starts, focusing initial points on promising redesign directions is more effective than exploring diverse directions.Near-best attainment rises when at least one of the three proposed points is feasible in S2–S4, and increases with each additional feasible point in S4; including the reference design among the points reduces near-best attainment in S3 and S4.Source: §4.2.2 · p. 9

4.2.3 ReAct is especially useful when objective improvement remains important after feasibility.

  • Among runs that find feasibility, 27B ReAct converts 27% in S1, 94% in S2, 94% in S3, and 96% in S4 into designs within 2% of best-known cost, outperforming each scenario’s best alternative.The advantage persists with at least ten evaluations remaining: within ten evaluations after first feasibility, 27B ReAct reaches the 2% gap in 94%, 93%, and 94% of runs in S2–S4. ReAct’s reasoning shifts from constraints toward objective and cost after feasibility; it is recommended when objective improvement matters and inference cost is affordable.Source: §4.2.3 · pp. 9–10

4.3 Larger models generally perform better, with the largest gains for ReAct.

  • Feasibility and optimality generally improve with model size, most clearly for ReAct, whose per-step LLM control makes performance more dependent on model quality.VF near-best attainment in S3 is nonmonotone: its configuration median is 56% at 4B, 29% at 9B, and 62% at 27B, reflecting differences in selected masks. ReAct token medians are 775,090 at 4B and 254,678 at 27B; smaller models often repeat candidates, increasing tokens spent.Source: Figure 10 · p. 10, §4.3 · p. 10
  • Additional engineering information in the prompt (L2) produced no clear difference in optimization outcomes or redesign decisions, motivating further study of how that information is supplied.Appendix F provides further details.Source: §4.3 · p. 11

4.4 Implications.

  • Variable fixing reduces search dimension, warm starting directs search toward promising regions and performs consistently across four scenarios, while ReAct can change strategy each step and keep improving after feasibility.Which approach is most effective depends on the redesign problem’s structure.Source: §4.4 · p. 11
  • Use variable fixing when a requirement change maps confidently to one or two design variables, warm starting when promising regions are known but responsible variables are not, and ReAct when the objective can improve after feasibility.A larger ReAct model is preferable when inference cost is not the main concern; a larger iteration budget could change the observed trends in solver feasibility and progress toward global optimality.Source: §4.4 · p. 11

5 Conclusions

Across the battery-pack redesign study, LLM guidance found feasible redesigns in fewer evaluations and ended at lower cost than numerical baselines; the strongest strategy and model-size effects varied with how the search was guided.

5 Conclusions

  • LLM guidance found feasible redesigns in fewer simulator evaluations and ended at lower cost than numerical baselines under the same budget.The gain was largest when the reference design became infeasible and function values alone made the feasible region difficult to locate.Source: §5 · p. 11
  • The most effective LLM-assisted method differed across redesign scenarios, with the choice depending on how each method uses the reference design and on the requirement change.Source: §5 · p. 11
  • The benefit of a larger language model grew with its control over search, was largest for ReAct, and was noisy for warm starting and variable fixing.Warm starting and variable fixing use a numerical optimizer for the remaining budget.Source: §5 · p. 11
  • The study tested one simulator and product family, used only Qwen3.5 at three sizes, and notes that other model families or different variable coupling could change the observed patterns.Source: §5 · p. 11
  • Simulator evaluations were the search-effort measure, although median BatPaC evaluation took 6.5 seconds and a ReAct run required several hundred seconds of inference.The authors note that testing on a simulator taking hours per evaluation would clarify comparisons when simulation dominates cost.Source: §5 · p. 11
  • Further comparisons across engineering domains, models, integration approaches, and numerical solvers could help establish best practices for LLM use in redesign.Source: §5 · p. 11

Appendix A: Preliminary Reference-Design Search

The common reference design was selected by deterministic discrete grid search under the reference requirements, using the preliminary grid reported in Table 6.

Appendix A: Preliminary Reference-Design Search

  • A deterministic discrete grid search selected the common reference design under the reference requirements defined in Eq. (2).Source: Appendix A · p. 12
  • The grid search and Table 6 together describe the preliminary procedure and grid used to establish the common reference design.Source: Appendix A · p. 12, Table 6 · p. 12
  • Table 6 reports the preliminary search grid used to select the common reference design.Source: Table 6 · p. 12

Appendix B: Prompts and Model Configuration

The appendix specifies the prompt contents and shared model-generation settings, along with validation, retry, and ReAct step-handling rules; the full prompts and schemas are reproduced in the code repository.

Appendix B: Prompts and Model Configuration

  • Each user prompt includes the redesign definition, reference design and result, context change, L1 or L2 information, strategy instructions, and required output format.The system message is identical under L1 and L2.Source: Appendix B · p. 12
  • All strategies use temperature 0.7, top-p 0.8, a 4096-token generation limit, a 32,768-token context, and a run-index generation seed.Invalid or repeated responses are handled differently across warm starting, variable fixing, and ReAct; ReAct allows at most 60 agent steps per run.Source: Appendix B · p. 12
  • The complete system prompts, strategy instructions, schemas, and Level 1 and Level 2 semantic texts are reproduced in the code repository.Source: Appendix B · p. 12

Appendix C: L2 Engineering-Information Comprehension Check

The comprehension check tested whether Qwen3.5-4B could interpret the L2 engineering information across four redesign scenarios; responses generally recovered the supplied relationships and recognized simulator-dependent quantities.

Appendix C: L2 Engineering-Information Comprehension Check

  • The comprehension test covered four redesign scenarios with three independent Qwen3.5-4B responses per scenario, for 12 responses total.Source: Appendix C · p. 12
  • The 16 questions tested interpretation of supplied relationships, mapping them to design variables, distinguishing prompt-inferable from simulator-dependent quantities, and identifying uncertainty.Source: Appendix C · p. 12
  • The tested L2 text matched the main-experiment prompt, and Table 7 lists feasible preliminary-grid designs while omitting S1 because its reference design remains feasible.The scenario contexts, questions, and responses are reproduced in the code repository.Source: Table 7 · p. 12, Appendix C · p. 12
  • Questions 1–11 were answered correctly in nearly all responses, all responses identified simulator-dependent quantities, and Question 12 responses sometimes added design inferences beyond L2.One response had a minor equation-transcription error, while retaining the correct relationship direction.Source: Appendix C · p. 12

Appendix D: Numerical Optimizer and Simulator Configuration

The appendix specifies the shared objective, numerical-optimizer settings, simulator interface, and treatment of unusable LLM decisions. It also identifies the engineering-information comparison summarized in Table 8.

Objective and constraints

  • Every optimizer uses a shared penalized objective: feasible designs receive reported cost, while infeasible designs and simulator failures receive penalized losses.Constraint violations are normalized by bound magnitude, with a special 115 μm normalization width for the S3 fast-charge thickness limit.Source: Appendix D · p. 12

Variable representation

  • Ax and pymoo treat chemistry as categorical, NOMAD uses an integer code, and all methods encode topology variables as integer ranges.A shared repair step clips cathode thickness to chemistry-specific bounds and rounds then clips the three topology variables.Source: Appendix D · p. 12

Optimizer configuration

  • Bayesian optimization uses Ax automatic selections, the genetic algorithm uses pymoo defaults with a population of five and a 20-evaluation limit, and MADS uses NOMAD defaults.Supplied warm-start candidates are evaluated first, and the lowest-penalized-objective candidate becomes the single starting point.Source: Appendix D · pp. 12–13

Engineering-information comparison

  • Table 8 compares optimization outcomes and redesign decisions under engineering-information levels L1 and L2, with decisions pooled over scenarios.The table caption identifies the comparison but does not provide outcome values.Source: Table 8 · p. 13

Run accounting

  • 119 of 1,600 variable-fixing runs at 4B and 62 at 9B ended without a schema-valid decision; these runs count as failures in every reported rate.The corresponding counts were zero at 27B, 23 of 4,800 warm-start runs, and zero of 1,200 ReAct runs.Source: Appendix D · p. 13

Appendix E: Coding of the Reasoning Categories

The appendix defines how rationale categories are coded and reports supplementary comparisons, including the L1–L2 information test and variable-fixing analyses. It also qualifies the overall configuration comparison.

Reasoning-category coding

  • Rationale categories are coded as present or absent, and each reported value is the fraction of rationales containing that category at least once.Table 10 defines seven categories through logical rules over term sets, with sentence-level matching criteria.Source: Appendix E · p. 13

Preliminary grid search

  • The preliminary grid search found feasible designs in every redesign problem.The passage points to Table 7 for the results.Source: Appendix F · p. 13

Engineering-information levels

  • L1 and L2 show no noticeable performance difference, no consistently favored metric across twelve strategy–scenario combinations, and nearly unchanged redesign decisions.The authors limit this finding to the L2 text and delivery format used; the smallest model also reads the supplied relationships correctly in the comprehension test.Source: Appendix F · p. 13

Overall configuration comparison

  • The best reference-warm configuration has higher feasible discovery than every LLM-assisted method’s worst configuration in all four problems, so the aggregate advantage does not hold for every configuration.The worst variable-fixing configuration is below the best cold start in S1, S2, and S4.Source: Appendix F · p. 13

Mechanism analysis

  • The mechanism analysis compares variable-fixing masks and downstream optimizers in S2 and S4, warm starts with or without the reference design, and post-feasibility conversion rates.Figure 12 shows feasible fractions in the most often selected S2 and S4 variable-fixing subspaces, with the reference design outlined in gold.Source: Appendix F · p. 13, Figure 12 · p. 13

Appendix G: Computational Setup

The appendix documents the workstation and execution environment and points to supplementary variable-fixing results reported by downstream optimizer.

Computational environment

  • Experiments ran on a workstation with a 12-core Xeon w5-3423, 93 GiB of memory, and four RTX 4000 Ada GPUs; BatPaC evaluations ran sequentially on the CPU.The BatPaC workbook was driven through a Python wrapper and headless LibreOffice process.Source: Appendix G · p. 13

Variable-fixing results

  • Table 9 reports supplementary variable-fixing discovery by downstream optimizer for runs selecting the most frequent masks in S2 and S4.The relevant masks were FFFRR in S2 and FRRRR in S4, with 46 to 62 runs per optimizer.Source: Table 9 · p. 14

(c) Warm starting: with and without the reference design

The supplied appendix passages define supplementary comparison measures and coding rules, but provide no warm-start comparison values.

Reasoning-category coding

Comparison measures

  • Table 11 reports feasible discovery and near-best attainment as percentages of 50 runs per configuration, counting unusable decisions as failures.Near-best attainment means an optimality gap of 2% or less relative to the best-known cost in Table 3.Source: Table 11 · p. 15

Related papers

Loading 2610.39275v1…