Source-linked AI summary

LLMs Can Design Near-Optimal OR Algorithms

Jackie Baek

arXiv:2608.27296v1cs.AIcs.LG

TL;DR

This paper asks whether general-purpose LLMs can design algorithms for hard, well-specified operations research problems. Across inventory control, queueing network control, and assortment optimization, the strongest model matches or outperforms existing methods on nearly all evaluated instances, including when producing transferable level-2 algorithms.

  • Problem

    The paper asks how effectively LLMs can design algorithms for operations research problems, isolating algorithm design from the broader problem-solving workflow.

    Source: §1 · p. 1
  • Method

    The study evaluates LLM-generated solutions and transferable algorithms at two levels across inventory control, queueing network control, and assortment optimization benchmarks.

    Source: §1 · p. 2
  • Results

    Across all three OR domains, the strongest model matches or outperforms the best existing method on nearly all evaluated instances; on one lead-time sweep, it improves 2.5% to 4.1%.

    Source: §3.3 · p. 15, §7 · p. 28
  • Takeaways & Limitations

    Frontier LLMs can serve as serious empirical baselines for algorithm design in well-specified OR problems with rich algorithmic traditions.

    Source: §1.2 · p. 4, §7 · p. 28

    The empirical evidence covers only three OR settings, does not establish approximation guarantees, and may be affected by benchmark-specific information in model training.

    Source: §1.2 · p. 5

Abstract

from arXiv · show

We ask whether large language models (LLMs) can design effective algorithms for well-specified operations research (OR) problems. We study inventory control, queueing network control, and assortment optimization. We evaluate two levels of LLM use: at level 1, the model receives one problem instance and returns a solution for that instance; at level 2, it receives only the problem class description and broad parameter ranges, and returns an algorithm that maps instance parameters to solutions. Human input is minimal: we give one untuned prompt that describes the problem, and the model has access to a Python sandbox tool with a fixed compute budget. The strongest model we test, gpt-5.6-sol, matches or outperforms the best existing method on almost all evaluated instances. This holds even at level 2, where the returned algorithm is fixed before seeing the evaluation instances. Performance also improves sharply across models released less than eight months apart, suggesting that this capability is moving quickly. Thus, for the well-specified operations problems we study, a single untuned LLM query can already produce algorithms competitive with specialized methods. These results suggest that frontier LLMs can be a serious empirical baseline for algorithm design in well-specified OR problems.

1 Introduction

The paper tests whether frontier LLMs can design high-performing algorithms for mathematically specified OR problems under a minimal single-query protocol. Across inventory, queueing, and assortment benchmarks, the strongest model is competitive with specialized methods, while the evidence remains empirical and limited in scope.

1 Introduction

  • The study isolates algorithm design by giving LLMs one untuned prompt for three well-specified OR domains, testing instance-specific solutions and reusable class-level algorithms.The benchmarks contain 34 inventory, 13 queueing, and 3,393 assortment instances; level 2 returns one algorithm per class, whereas level 1 solves individual instances.Source: §1 · pp. 1–2

1.1 Main findings

  • gpt-5.6-sol matches or beats the best existing method on nearly every instance, with mean performance no worse in eight of ten classes at both levels.The comparison selects the strongest reported method instance by instance, including exact dynamic programs or solvers where available.Source: §1.1 · p. 2, Table 1 · p. 3

1.2 Implications and limitations

  • In the tested settings, level-2 algorithms often nearly match level-1 solutions and produce inspectable combinations of dynamic programming, greedy, relaxation, local-improvement, and pressure-based ideas.The results suggest LLMs can reduce the cost of part of the algorithm-design step, but generated methods often resemble combinations of known structures and do not replace algorithmic research.Source: §1.1 · p. 4, §1.2 · p. 4

1.3 Related Work

  • The paper differs from formulation, input-generation, iterative-search, solver-component, and direct-decision work by supplying exact mathematical specifications and evaluating a frontier LLM under a fixed single-query protocol.Its level-2 artifacts are executable algorithms whose logic can be inspected, diagnosed, and modified, unlike typical black-box ML/RL policies or value functions.Source: §1.3 · pp. 6–7

2 Setup

The paper formalizes OR problem classes, instances, feasible solutions, and deterministic rewards, then defines three possible LLM invocation levels and a controlled experimental protocol. The experiments use levels 1 and 2, while level 0 is excluded.

2.1 Problem Classes and Instances

  • A problem class C = (Θ, X, R) specifies parameters, feasible solutions, and a deterministic reward for each instance, including randomness aggregated through expectations or long-run averages.Solutions may be single choices or sequential policies π : S → A.Source: §2.1 · p. 8

Prompt schema example

  • The framework’s breadth is itself a modeling choice: classes can be narrow, such as MMNL assortment optimization, or broader, as in the queueing experiments’ arbitrary multiclass networks.The study compares each LLM result with the best reported or implemented method on the same instance, including exact solvers when available.Source: §2.1 · p. 9, §2.3 · p. 10

2.2 Levels of LLM invocation

  • Level 1 returns an instance-specific solution, whereas level 2 returns reusable executable code mapping every class instance to a feasible solution.For sequential problems, level 0 would query the LLM at every state for an action, but the paper does not test it.Source: §2.2 · pp. 9–10
  • Resource constraints prevent level-2 artifacts from collapsing into per-instance LLM calls: returned code cannot call an LLM, and both queries and executable objects are budgeted.Level 1 has more instance information than level 2, although the experiments compare their empirical performance.Source: §2.2 · p. 10

2.3 Experimental protocol

  • The protocol uses one tool-using query per model-instance pair at level 1 and per model-class pair at level 2, with an untuned prompt, a Python sandbox, and a stated compute budget.The sandbox includes the Python standard library, numpy, and scipy, but no external optimization solvers; returned code is evaluated outside the session.Source: §2.3 · p. 10

3 Inventory Experiments

The experiments evaluate LLMs on three inventory settings, focusing in the main text on 26 lost-sales instances. gpt-5.6-sol outperforms tuned benchmarks on most instances, including through a reusable level-2 algorithm that uses projected, risk-adjusted inventory.

3.1 Instances

  • The study uses lost-sales, dual-sourcing, and multi-echelon settings, with the main text emphasizing 26 lost-sales instances and deferring the other settings to Appendix B.Lost-sales instances include deterministic and stochastic lead-time classes; short deterministic cases are exactly solvable, while larger cases test heuristic construction.Source: §3.1 · pp. 11–12

3.2 LLM experiments and benchmark policies

  • LLMs are queried per instance at level 1 or per problem class at level 2, while tuned benchmarks include base-stock, constant-order, capped base-stock, mixed, and myopic policies.Each query has up to 3600 seconds of Python sandbox compute across 50 calls, and benchmark parameters are tuned separately for each instance.Source: §3.2 · pp. 12–13

3.3 Results

  • gpt-5.6-sol beats the best tuned benchmark on 23 of 26 lost-sales instances at level 1 and 21 at level 2, staying within 0.5% on all instances.Level 2 uses only two algorithms, one for deterministic and one for stochastic lead times.Source: §3.3 · p. 13, Figure 4 · p. 14
  • Across models released within eight months, performance shows a capability gradient: claude-fable-5 is less reliable than gpt-5.6-sol, gpt-5.4 is competitive only at level 1, and gpt-5.1 is worse at both levels.The qualitative distinction is that the frontier model uses a state representation combining expected usable inventory with uncertainty rather than only tuning a base-stock threshold.Source: §3.3 · p. 13, §3.4 · p. 16
  • On the deterministic lead-time sweep, gpt-5.6-sol improves on the best tuned benchmark by 2.5% to 4.1% at every lead time, while gpt-5.4 remains near capped base-stock at level 2.The strongest model also recovers the exact optimum on short-lead-time instances at level 1 and remains essentially optimal at level 2.Source: §3.3 · pp. 14–15

3.4 Structure of LLM algorithms for deterministic lead times

  • The strongest deterministic level-2 algorithm generalizes capped base-stock by using projected, risk-adjusted inventory summarized by propagated mean and variance.It tunes projection and correction parameters by simulation, using exact Poisson moments initially and a normal approximation later, while handling penalty and capacity edge cases explicitly.Source: §3.4 · p. 16

4 Queueing Experiments

In queueing experiments, gpt-5.6-sol is near-optimal on tractable instances and outperforms PPO on most larger networks, with level-2 performance depending strongly on class specification. Its returned policies combine dynamic programming, threshold rules, and simulation-tuned pressure policies.

4.1 Instances

  • The benchmark comprises seven DP-solvable criss-cross and N-model instances plus extended six-class networks compared primarily with Dai and Gluzman’s PPO policy.The extended networks are not tractable by dynamic programming, whereas the small-instance optima reproduce published values to three or four digits.Source: §4.1 · p. 17, §4.2 · p. 17, Table 4 · p. 17

4.3 Results

  • gpt-5.6-sol is essentially optimal on all seven DP-solvable instances, beats PPO on four of six larger networks, ties one, and is 2.2% worse on the largest.At level 2, it remains essentially optimal for criss-cross and N-model classes and beats PPO on five of six reentrant-line instances.Source: Table 5 · p. 18, §4.3 · p. 18
  • Broad L2 is competitive on extended six-class networks but performs poorly on criss-cross and N-model instances, showing that overly broad classes elicit generic rules with topology-specific failures.The broad query covers networks up to 7 servers and 21 queues, whereas the main L2 setting uses structural subclasses.Source: §4.2 · p. 17, §4.3 · pp. 18–19
  • Across models, all are within 1% of optimal on small classes at level 1 except gpt-5.1 on N-model, while only gpt-5.6-sol improves over existing methods on reentrant lines at level 2.gpt-5.1’s N-model policies fail to stabilize the system, and its reentrant-line algorithm reduces to a static priority rule.Source: Table 6 · p. 19, §4.3 · p. 19

4.4 Structure of the gpt-5.6-sol policies

  • gpt-5.6-sol’s artifacts use dynamic programming-derived threshold or switching policies on small networks and simulation-tuned pressure rules accounting for backlog, holding costs, service rates, and downstream congestion on reentrant lines.The level-2 reentrant-line routine simulates a small rule menu, selects parameters per instance, and returns a lightweight event-level policy.Source: §4.4 · p. 19

5 Assortment Experiments

Across three hard assortment benchmarks, level-2 LLM solvers nearly match the best existing methods, with constrained-MMNL matching them exactly and nested logit lagging only on a difficult tail. The MMNL result is enabled by a highly structured two-type benchmark, while the strongest solvers combine portfolios of search, relaxation, repair, and local-improvement methods.

5.1 Instances

  • The experiments cover 628 cardinality-constrained MMNL, 971 nested-logit, and 1,794 feasible constrained-MMNL instances, with prompts specifying broad size and constraint ranges.The constrained-MMNL set regenerates 1,800 instances and drops six infeasible degenerate cases; the MMNL and nested-logit sets include all 628 and 971 released instances, respectively.Source: §5.1 · p. 20

5.2 LLM experiments and benchmark algorithms

  • Level 2 queries each problem family once and runs the returned solver across its instances, comparing revenues against Guo et al. (2025) benchmark algorithms; level 1 uses smaller stratified sets.Level 1 allows one query per instance with a 900-second sandbox budget, while benchmark comparisons use the authors’ per-instance results.Source: §5.2 · pp. 20–21, §5.3 · p. 21

5.3 Results

  • gpt-5.6-sol matches the best existing method on every constrained-MMNL instance, while strong level-2 solvers are exact on the released MMNL benchmark; nested-logit solvers are within 0.1% on 87.4% of instances but lose on a hard tail.Figure 6 reports revenue relative to the best existing method; the nested-logit comparator is already within 0.1% of an upper bound on 911 of 971 instances and within 0.5% on all instances.Source: §5.3 · p. 21, Figure 6 · p. 22
  • For stronger models, level 1 and level 2 are essentially identical, although instance-specific querying raises gpt-5.1’s nested-logit within-0.1% share from 50.0% to 77.1%.The comparison uses the smaller stratified level-1 subset; the visible benefit of level 1 is concentrated in gpt-5.1 on nested logit.Source: §5.3 · p. 21

5.4 The two-type MMNL structure

  • All 628 released MMNL instances contain only two distinct product-utility vectors, allowing exact optimization by sorting within utility classes and enumerating class counts.An independent solver confirms the benchmark optima on every instance, so the LLM’s MMNL exactness does not establish general exact MMNL optimization.Source: §5.4 · pp. 21, 23

5.5 Structure of the LLM algorithms

  • The selected MMNL solver uses heuristic portfolios and local search, the nested-logit solver exploits separable nest structure with fractional-programming updates, and the constrained-MMNL solver combines relaxation, repair, local improvement, and occasional restricted MILPs.The constrained-MMNL solver filters infeasible products and maintains feasible candidates; the MMNL solver generates candidates from prefixes, greedy paths, randomized starts, and a continuous relaxation.Source: §5.5 · p. 23

6 Robustness and Sensitivity Checks

The robustness checks show that level-2 algorithms often remain strong without query-time computation, generalize to new instances, and are usually stable across repeated draws. Remaining weaknesses arise mainly in harder queueing settings, occasional defective artifacts, and soft runtime constraints.

6.1 The role of query-time tools

  • With zero query-time compute, level-2 algorithms still beat the tuned inventory benchmark on all six instances and preserve assortment performance, while queueing results vary by class.The zero-compute inventory algorithm stays within 0.4 percentage points of the full-budget algorithm; queueing remains strong for criss-cross but worsens for N-model and extended reentrant-line classes.Source: §6.1 · pp. 24–25, Table 7 · p. 24, Table 8 · p. 24
  • Queueing zero-compute performance depends on whether returned code can optimize during evaluation: it is near-optimal for criss-cross but trails optimal or PPO on harder classes.The N-model is about 7% above optimum and extended reentrant-line is about 2.4% above PPO without compute, versus about 5.0% better than PPO with full budget.Source: §6.1 · p. 25
  • Adding unlimited web search leaves four tested level-2 artifacts unchanged, including the nested-logit hard tail, because the model used search only briefly or not at all.The tested classes were deterministic lost sales, MMNL, nested logit, and constrained MMNL.Source: Adding a web-search tool · p. 25

6.2 Generalization to new instances

  • Frozen level-2 algorithms generalize to new inventory and assortment instances: gpt-5.6-sol beats every inventory benchmark and matches assortment comparators throughout the holdouts.The holdouts alter inventory parameters and remove published assortment structures, but generated MMNL instances remain easy and nested-logit comparisons are heuristic rather than certified.Source: §6.2 · pp. 25–26, Figure 7 · p. 26

6.3 Sensitivity to repeated draws

  • Repeated draws are usually stable: inventory and assortment spreads are narrow, every criss-cross draw is within 0.02% of optimal, and five of 97 draws fail evaluation.Multi-echelon inventory varies by 6.8 percentage points, while seven of ten dual-sourcing draws recover the exact dynamic program and the remaining three stay within 0.3% of optimal.Source: §6.3 · pp. 27–28, Figure 8 · p. 27
  • The repeated-draw N-model failures enforce soft runtime targets too strictly, returning unconverged or closed-form policies that misplace a critical switching threshold.Two weak draws are 14.0% and 17.6% above optimum, whereas near-optimal draws overrun the target by factors of 2 to 18 and ship converged policies.Source: §6.3 · pp. 27–28

7 Conclusion · A Protocol details · B Additional inventory details

Across three well-specified OR problem classes, the strongest LLM matches or outperforms existing methods on nearly all evaluated instances and can produce transferable level-2 algorithms. The study details its evaluation protocol while noting that real-world deployment still requires formulation, validation, and judgment.

7 Conclusion

  • Across inventory, queueing, and assortment benchmarks, the strongest model matches or outperforms the best existing method on nearly all evaluated instances, including transferable level-2 algorithms.The level-2 model produces procedures that transfer across many instances rather than searching for only one solution.Source: §7 · p. 28
  • Generated algorithms often recombine familiar OR ideas, including capped base-stock policies, small-state dynamic programs, pressure-based scheduling, and local search.The paper cautions that performance may depend on exposure to existing OR literature, while viewing recombination as a potentially low-cost route to strong solvers.Source: §7 · p. 28
  • The demonstrated automation is limited to algorithm design for well-specified benchmark problems; real operations still require formulation, measurement, validation, deployment, and objective trade-offs.A strong LLM can turn a mathematical description into high-performing, interpretable code, but when this succeeds or fails remains open.Source: §7 · p. 28

A Protocol details

  • The protocol tests four models—gpt-5.6-sol, gpt-5.4, gpt-5.1, and claude-fable-5—using level-specific contracts for instance-level policies and cross-instance solver generation.Level-2 prompts provide problem descriptions and parameter ranges without evaluation instances, and each returned solver is evaluated across its family.Source: A Protocol details · p. 35
  • Queries receive a 3600-second sandbox budget, except level-1 assortment queries with 900 seconds, while returned solvers target 30-second per-instance setup on one CPU core.The 30-second target is measured rather than enforced, and a 600-second guard catches solvers that never return.Source: A Protocol details · p. 35
  • All local computation, tuning, simulation, and returned-code evaluation ran on one Apple M2 Pro laptop using Python 3.10, NumPy 1.24, and SciPy 1.10.The appendix additionally provides inventory validation checks, settings, full per-instance lost-sales results, and timing results.Source: A Protocol details · p. 35, B Additional inventory details · p. 35
  • Comparisons use the best existing method as comparator, with common-random-number inventory simulation, exact or simulated queueing evaluation, and author-reported or evaluator-computed assortment revenues.Logs, prompts, code, execution details, token usage, and model versions are released with the code and prompts.Source: A Protocol details · p. 35

B.1 Validation · B.2 Dual sourcing · B.3 Multi-echelon distribution

The paper validates its simulators against exact solutions, published figures, internal consistency checks, and limiting cases. It then evaluates LLM policies against exact dual-sourcing optima and tuned multi-echelon benchmarks under specified dynamics and assumptions.

B.1 Validation

  • Benchmark policies are tuned by simulation using streams separate from final scoring, with the multi-echelon benchmark sharing warehouse and store order-up-to levels across identical stores.Dual-sourcing benchmarks include tailored base-surge, dual-index, and capped dual-index policies.Source: §B.1 · p. 36, §B.2 · p. 36, §B.3 · p. 37
  • In lost-sales validation, simulated long-run costs match the exact dynamic program within simulation error, differing by at most 0.009 on costs around five.The stochastic simulator also passes deterministic-case, Little’s-law, lead-time-distribution, and qualitative source-pattern checks.Source: §B.1 · p. 36
  • Dual-sourcing validation reproduces the source paper’s Figure 9 within a hundredth of a percentage point for sixteen of eighteen policy-instance pairs.The remaining two pairs are the only exceptions reported.Source: §B.1 · p. 36
  • Multi-echelon validation finds exact agreement between scalar and vectorized simulators on shared random streams and matches a reduced single-location model within simulation error.The reduced regime has one retailer, no special delivery, and a warehouse that always fills orders.Source: §B.1 · p. 36

B.2 Dual sourcing

  • The dual-sourcing test uses six instances with a slower, cheaper regular supplier, immediate expedited supply, uniform demand, backlog costs, and exact dynamic-program optima.Regular lead times are ℓr ∈{2, 3, 4}, expedited costs are ce ∈{105, 110}, and gaps are relative to the exact optimum.Source: §B.2 · p. 36
  • Dual-sourcing results report exact optima for six instances and every LLM policy’s gap above the optimum at both evaluation levels.The exact dynamic program is feasible for all six instances and serves as the best existing method.Source: §B.2 · pp. 36–37

B.3 Multi-echelon distribution

  • The multi-echelon system has one warehouse supplying ten stores, stochastic special delivery, capped inventories, and two parameter settings from Gijsbrechts et al. (2022).Store demand is independently rounded normal and truncated below zero; in the second setting, printed µ = 0 is retained, but effective demand remains positive.Source: §B.3 · p. 37
  • Multi-echelon results report tuned benchmark costs for both settings and every LLM policy’s gap versus the benchmark at both levels.Exact optimization is intractable in this system.Source: §B.3 · p. 37

B.4 Per-instance inventory results

This section reports per-instance lost-sales results, with level-1 and level-2 policies compared on identical simulation streams against tuned benchmarks. Results are expressed as percentage cost gaps relative to the strongest available method for each instance.

Results coverage

  • Table 12 covers the remaining 20 lost-sales instances, including four level-1 models and two level-2 algorithms per model evaluated across each class.The six exactly solvable short-lead-time instances appear in Table 3; dual-sourcing and multi-echelon results are reported separately.Source: §B.4 · p. 37

Evaluation protocol

  • Level-1 and level-2 entries within each row are directly comparable because policies use identical demand and lead-time streams against the same tuned benchmarks.Source: §B.4 · p. 37

Metric interpretation

  • Percentage gaps are measured against the best existing method, with negative values indicating lower LLM-policy cost; exact dynamic programs define the reference in Tables 3 and 10, while tuned benchmarks define it in Tables 11 and 12.Bold entries indicate matching the exact method within simulation noise in Tables 3 and 10, or beating the tuned benchmark in Tables 11 and 12.Source: §B.4 · p. 37

B.5 Compute and wall time · C Additional queueing details · C.1 Instance definitions

The appendix details gpt-5.6-sol’s computational resources and defines the queueing-network benchmark instances, including which cases admit exact dynamic programming and which require PPO comparison. Level 1 spends substantially more sandbox time on simulation-tuned problems, whereas Level 2 shifts computation to per-instance policy construction.

B.5 Compute and wall time

  • The resource accounting distinguishes wall time, metered sandbox time, and evaluation setup through module loading at level 1 or design(θ) at level 2.Wall time includes prompt transmission, reasoning, and sandbox execution; Table 15 reports the corresponding breakdown behind Table 7.Source: §B.5 · p. 37, Table 15 · p. 39
  • Level 2 uses only 18–237 s of the 3,600 s sandbox budget, then spends 3–13 s per evaluation instance in design(θ).This computation is deferred from the class-level query to evaluation-time policy construction.Source: §B.5 · p. 38, Table 15 · p. 39
  • Level 1 uses mean sandbox times of 1,959–2,841 s on lost-sales and multi-echelon problems, with several queries reaching the 3,600 s cap.These policies require simulation-based tuning, unlike dual sourcing, where the model identifies a small exact dynamic program.Source: §B.5 · p. 38

C Additional queueing details

  • The appendix supplies queueing-instance definitions, validation checks, timing, and robustness results supporting the queueing experiments in Section 4.This identifies the scope of the additional queueing material rather than reporting a separate quantitative finding.Source: C Additional queueing details · p. 39

C.1 Instance definitions

  • The queueing benchmark comprises thirteen multiclass networks with compatible server–queue assignments, Poisson arrivals, exponential downstream service, and linear holding costs.Each network is specified by queues, servers, compatibility matrix A, service rates µ, arrival rates λ, routing, and holding costs.Source: §C.1 · p. 39
  • Seven small instances—six criss-cross networks and one N-model—have exact dynamic programs, while six extended reentrant-line instances are too large for exact dynamic programming.The extended cases use the per-instance PPO policy of Dai and Gluzman (2022) as the primary comparator.Source: §C.1 · p. 39
  • The six criss-cross instances vary λ ∈ {0.3, 0.6, 0.9} and µ2 ∈ {1, 1.5}, while the N-model has load 0.95, arrivals (1.235, 0.38), and holding costs (3, 1).Criss-cross networks use two servers and three queues; the N-model uses two servers and two queues with the stated service rates.Source: §C.1 · p. 39

C.2 Simulation and comparator validation · C.3 Protocol cross-check · C.4 Timing

The study validates exact evaluation on DP-solvable networks and uses controlled steady-state simulation for larger instances, with protocol cross-checks confirming the level-1 comparison. It also reports substantial sandbox compute for generation but sub-millisecond policy execution.

C.2 Simulation and comparator validation

  • Exact evaluation reproduces published optima to three or four digits, while evaluating the optimal policy itself matches each optimum within 10^-10.Relative value iteration on truncated uniformized chains computes exact optima and deterministic stationary-policy gaps for seven DP-solvable criss-cross and N-model instances.Source: §C.2 · pp. 39–40
  • The six extended six-class instances use 20 independent 2 × 10^6-event runs after 2 × 10^5-event warm-ups, with common random numbers across policies.Stability checks instead use 10 runs of 10^6 post-warm-up events; invalid server assignments are converted to idling, and no reported artifact makes invalid assignments.Source: §C.2 · p. 40
  • Dai and Gluzman’s PPO policy (2022) is the primary comparator on the six extended networks, with reported costs from 14.13 to 55.12 and confidence intervals of about 0.2–1.8.LBFS and cµ are re-simulated as secondary checks on the same Markov chain.Source: §C.2 · p. 40

C.3 Protocol cross-check

  • The QGym finite-horizon cross-check agrees with the steady-state level-1 comparison on the extended six-class instances.It starts empty and averages 100 trajectories under QGym’s protocol.Source: §C.3 · p. 40, Table 16 · p. 40

C.4 Timing

  • Level-1 generation uses 45–1558 seconds on criss-cross, 2024 seconds on the N-model, and usually most of the 3600-second budget on extended six-class instances.The criss-cross and N-model calls can slightly exceed the 30-second target because they solve small dynamic programs.Source: §C.4 · p. 40
  • Level-2 artifacts use 841, 58, and 208 seconds of sandbox compute on criss-cross, the N-model, and extended six-class instances, respectively.Returned policy calls are well under one millisecond.Source: §C.4 · p. 40

D Additional assortment results

The appendix documents assortment-result evaluation choices, including complete instance coverage for most comparisons, exclusions for anomalous solver entries, and a stratified set for level-1 evaluation. It also presents the level-1 counterpart to the main assortment-results figure.

Instance and entry selection

  • All 628 MMNL and 971 nested-logit instances are evaluated for every method, while constrained MMNL uses the authors’ 1,800-instance grid after removing six infeasible instances.Twenty-seven MMNL conic-solver entries exceeding the verified exact optimum by more than 10−5 relative tolerance are excluded; LLM revenues are independently evaluated, feasibility-checked, and never capped.Source: D Additional assortment results · p. 41

Level-1 assortment

  • Level-1 assortment uses one query per instance on 72 MMNL, 48 nested-logit, and 36 constrained-MMNL stratified instances.The set includes the lowest-seed instance from each released MMNL and nested-logit configuration and the two lowest seeds from each constrained-MMNL configuration.Source: D Additional assortment results · p. 41
  • Figure 9 reports level-1 assortment results on the stratified instance set using the same comparator, axes, and conventions as Figure 6.Existing methods are scored on the same instances, making Figure 9 the level-1 counterpart of Figure 6.Source: D Additional assortment results · p. 40, Figure 9 · p. 42

D.1 Compute and wall time

The section characterizes query, sandbox, and returned-solver runtimes for gpt-5.6-sol across level-1 and level-2 assortment queries. Level-2 solvers are typically fast at evaluation time, while query and sandbox costs vary by query type.

Level-2 compute

  • Level-2 queries use about 100 to 120 s of the 3,600 s sandbox budget, and returned solvers usually run in well under a second per instance.The reported solver runtimes are evaluated against a 30 s target stated in the prompt; constrained MMNL uses 1,794 evaluation instances.Source: §D.1 · p. 42, Table 17 · p. 43

Level-1 and zero-compute queries

  • Zero-compute queries take about four minutes of reasoning and return similarly fast solvers, whereas level-1 queries usually take about a minute, with some MMNL and nested-logit cases using most of 900 s.Level-1 queries are one per instance, and their reported distributions include wall time, sandbox seconds, code cells, and output tokens.Source: §D.1 · p. 42, Table 18 · p. 43

Runtime measures

  • Tables 17 and 18 report query wall time, sandbox Python time, and returned level-2 solver runtime for gpt-5.6-sol.Table 17 measures solver runtime per evaluation instance, including mean and maximum over each family, while Table 18 summarizes level-1 query-time distributions.Source: §D.1 · p. 42, Table 17 · p. 43, Table 18 · p. 43

E Holdout generalization results

The holdout evaluation tests frozen level-2 artifacts on new inventory and assortment instances without additional model queries, using expanded stochastic, utility, and choice-model regimes.

Holdout protocol

  • Frozen level-2 artifacts are evaluated on new instances without querying the model again; assortment performance is summarized as mean revenue ratios against exact optima or stated heuristics.Figure 7 reports inventory results, while Table 19 reports assortment ratios with across-instance minima and maxima.Source: E Holdout generalization results · p. 43

Inventory holdout

  • Lost-sales inventory holdouts include 30 deterministic and 12 stochastic instances spanning short and long lead times, demand rates, prices, and uniformly distributed stochastic lead times.All instances use h = 1, c = 0, Poisson demand, and maximum order quantity ⌈2.5λ⌉+ 2.Source: E Holdout generalization results · p. 43

MMNL holdout

  • MMNL holdouts replace the benchmark’s two-utility structure with continuous utilities in iid-lognormal and correlated-quality regimes, covering exact and large assortment subsets.The exact subset varies n, m, and k with two seeds per configuration; the large subset uses n ∈{100, 200}, m ∈{10, 25}, and k ∈{10, 20}.Source: E Holdout generalization results · pp. 43–44

Nested-logit holdout

  • Nested-logit holdouts vary products per nest, nest counts, and per-nest caps across anti-correlated and cross-nest regimes with heterogeneous attraction and pricing.Instances draw nest dissimilarities, outside utilities, and product utilities from specified distributions.Source: E Holdout generalization results · p. 44

F Adding a web-search tool

Adding unrestricted web search to four gpt-5.6-sol level-2 runs left the returned algorithms and results essentially unchanged. The model searched sparingly at the start, suggesting retrieval was not binding in this limited check, though it substantially increased generation time.

  • All four level-2 queries used the same 3600-second Python budget and unchanged prompt apart from unrestricted web-search access; searches occurred only in the first turn before coding.Each run produced one algorithm evaluated as in the main experiments, and Table 20 records the search activity.Source: F Adding a web-search tool · p. 44, Table 20 · p. 44
  • −1.26% mean gap versus the best existing method matched the main artifact’s −1.27%; MMNL again attained the exact optimum on all 628 instances.Source: F Adding a web-search tool · p. 44
  • Constrained MMNL again matched the best existing method on all 1,794 instances, while nested logit matched the main artifact.Source: F Adding a web-search tool · p. 44
  • The check used one artifact per class and one model, and web search substantially increased wall-clock generation time despite no changes in algorithms or results.When used, retrieval targeted canonical references.Source: F Adding a web-search tool · p. 44

G How the model reasons · H Prompts

This section examines how the model reasons through algorithm-design tasks and how prompts specify them. It covers recurring reasoning workflows alongside frozen prompts for inventory, queueing control, and assortment optimization at both specific-instance and class-wide levels.

G How the model reasons

  • Across problem classes, the model identifies structure, compares algorithm families, budgets computation, builds testbeds, adds guardrails, and formats returned code.It typically recalls relevant ideas first, weighs exact methods against simpler or hybrid alternatives, benchmarks against small-case ground truth, checks edge cases, and then finalizes the required interface.Source: G How the model reasons · pp. 47–48

H Prompts

  • The prompts are generated from shared code, frozen in the repository, and accompanied by complete prompt files and run records for replication.The records include every query, reasoning summary, executed code, output, and token usage.Source: H Prompts · p. 48

H.1 Inventory Prompt Examples

  • Inventory prompts distinguish instance-specific policies from class-wide solvers, specifying known dynamics, long-run average-cost objectives, parameter ranges, compute budgets, and exact function contracts.The level-2 example provides no evaluation instances, requires a design(...) function returning order(...), and limits per-instance design time while keeping period calls lightweight.Source: §H.1 · pp. 48–49

H.2 Queueing Prompt Examples

  • Queueing prompts define multiclass continuous-time networks, holding-cost minimization, feasible event-based server assignments, and sandbox-based code generation under a fixed computation budget.The level-1 example fixes S = 2 servers and Q = 6 queues, while the policy must assign compatible queues or -1 to idle and cannot exceed available jobs.Source: §H.2 · p. 50
  • Queueing prompts define level 1 criss-cross and broad level 2 multiclass-network tasks that minimize steady-state holding cost across stable instances, including networks up to 7 servers and 21 queues.The controller selects feasible server assignments at event times under preemptive-resume service, with known parameters and loads ranging from light to heavy.Source: §H.2 · pp. 51–52
  • Queueing level 2 solvers receive only class parameters and must return design(network, mu, lam, next_queue, h), which produces a feasible lightweight policy within 30 seconds per instance.The returned policy is called at every event, must typically run well under a millisecond, and cannot perform simulations or expensive optimization per call.Source: §H.2 · p. 52

H.3 Assortment Prompt Examples

  • Assortment level 1 asks for one MMNL instance solution, choosing up to cap products from preloaded data with n = 200 and m = 25 to maximize expected revenue.The final response must be a valid Python list of distinct, 0-based product indices.Source: §H.3 · p. 53
  • Assortment level 2 asks for a general-purpose MMNL solver over many instances with n up to 200 and m up to 25, returning solve(u, price, v0, omega, cap) within 30 seconds per instance.The solver must output a distinct 0-based product-index list satisfying the capacity constraint and may use only the Python standard library, numpy, and scipy.Source: §H.3 · pp. 53–54
Loading 2608.27296v1…