LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization

• Industrial optimization often involves thousands to millions of variables and constraints, with problem descriptions and data distributed across files, making it an end-to-end problem-solving task.Agents must interpret context, access and analyze external data, identify structure, choose a computational strategy, and construct an executable solution.Source: §1 · p. 1
• Different problem classes and scales favor different strategies, and partially overlapping success sets on MIPLIB-NL show the complementarity of solver-based, algorithmic, and heuristic approaches.Exact formulations suit compact, tractable problems; specialized algorithms exploit problem structure, while heuristics can help when exact optimization is computationally prohibitive.Source: §1 · pp. 2–3, Figure 1 · p. 2
• SDRL trains LLMs as optimization meta-solvers, using a correctness-gated hierarchical diversity reward to preserve correct strategies across and within solver-integrated, algorithmic, and heuristic approaches.The reward is designed to discourage premature strategy collapse and promote robust exploration.Source: §1 · pp. 2–3
• Mixed-format training combines self-contained textual problems with file-grounded problems containing external structured data, supporting training on realistic industrial-scale instances while improving data efficiency.Source: §1 · pp. 2–3

Automated optimization modeling

• Prior automated optimization-modeling methods span agent reasoning, search and verification, supervised or preference tuning, verifiable-reward reinforcement learning, and process supervision, but remain mostly textbook-scale.Examples include agent-based reasoning (AhmadiTeshnizi et al., 2024; Zhang et al., 2024), search, knowledge augmentation, and verification (Astorga et al., 2024; Liu et al., 2026b; Kong et al., 2026b; Liu et al., 2026c), supervised fine-tuning (Huang et al., 2025a; Lu et al., 2025; Wu et al., 2025; Zhang et al., 2025), preference alignment, verifiable-reward reinforcement learning (Chen et al., 2026b; Liu et al., 2026d; Zhao et al., 2026; Tang et al., 2025), and process supervision (Zhou et al., 2026; Wang et al., 2026c).Source: §2 · p. 3

Automated algorithm design

• Automated exact and heuristic algorithm design offers alternatives to solver-based modeling, motivating a unified policy that selects among three paradigms for each instance.Exact methods include dynamic programming (Zhou et al., 2025), graph algorithms (Nia et al., 2025), and divide-and-conquer (Huang et al., 2024), without general-purpose mathematical programming solvers (Wang et al., 2026a). Heuristic work uses evolutionary search over executable programs (Liu et al., 2024), improved by reflective and context-aware prompting (Ye et al., 2024; Bömer et al., 2025), diversity preservation, complementary heuristic sets, and multi-objective optimization (Dat et al., 2025; Liu et al., 2026a; Yao et al., 2025); reinforcement learning can co-evolve the model with heuristic search (Huang et al., 2026). The policy selects solver-integrated reasoning, exact combinatorial algorithm, or heuristic search based on instance characteristics.Source: §2 · p. 3

Exploration in complex reasoning

• Unlike general reasoning-diversity methods, this formulation promotes algorithmic strategy diversity and distinguishes successful programs within a strategy by runtime modes.Prior work identifies entropy collapse and reduced rollout diversity (Yue et al., 2025; Cui et al., 2025; Wang et al., 2026b); remedies include token-level entropy regularization or clipping (Yu et al., 2026) and trajectory-level diversity methods (Hu et al., 2026; Liang et al., 2026; Cao et al., 2026).Source: §2 · p. 3

3 METHODOLOGY

• The meta-solver analyzes problem class, mathematical structure, and scale, then selects solver-integrated reasoning, an exact combinatorial algorithm, or heuristic search before implementing a solution.This separation makes strategy selection a learnable decision, with external solvers, specialized exact procedures, or approximate search used according to the selected paradigm.Source: §3 · p. 4

3.1 TRAINING DATA CONSTRUCTION

• The data pipeline converts some self-contained examples into file-grounded instances by externalizing scalar parameters and tabular data while preserving the reference optimal objective value.Scalar parameters go in instance.json and tabular data in CSV files; a TSP example keeps the task and optimization structure in text while placing its distance matrix in a CSV file.Source: §3.1 · pp. 4–5

3.2 STRATEGY-DIVERSE REINFORCEMENT LEARNING

• GRPO trains the policy on grouped rollouts containing reasoning, a strategy tag, and executable code, rewarding successful execution and objective values within tolerance without ground-truth strategy labels.Trajectory-level feedback supervises both routing and implementation because the generated reasoning and code are conditioned on the routing decision.Source: §3.2 · p. 5

3.3 REWARD DESIGN AND TRAINING SCHEME

• SDRL combines executable-correctness feedback with correctness-gated rewards for uncommon strategies and runtime modes, encouraging breadth while preserving implementation proficiency.Macro-level diversity spans the three strategies; micro-level diversity uses DBSCAN clusters of log-transformed runtimes within each strategy, with clipping to limit outlier bonuses.Source: §3.3 · pp. 6–7
• Training mixes self-contained textual and file-grounded instances, requiring file-grounded rollouts to read structured inputs in an isolated workspace without access to the reference optimum.Both instance types use the same response format and executable-correctness criteria; file-grounded prompts expose file paths and column names rather than table contents.Source: §3.3 · p. 7

4.1 EXPERIMENTAL SETUPS

• Across seven benchmarks, evaluation includes six self-contained text datasets and file-grounded industrial MIPLIB-NL; main models use mixed-format training.The main models are initialized from Qwen3-4B-Instruct-2507 and Qwen3-32B; ablations use Qwen3-4B-Instruct-2507, and correctness requires relative objective error below 10^-6.Source: §4.1 · p. 7

4.2 MAIN RESULTS

• SDRL-Qwen3-4B outperforms all existing fine-tuned baselines, including larger 32B models, while SDRL-Qwen3-32B is competitive with frontier LLMs, especially on MIPLIB-NL.The paper reports a new state of the art for open-source optimization modeling and says both models demonstrate frontier-comparable agentic optimization modeling capabilities on MIPLIB-NL.Source: §4.2 · pp. 7–8, Table 1 · p. 8

4.3 ABLATION STUDY OF HIERARCHICAL DIVERSITY REWARD

• Macro-level diversity broadens strategic exploration and mainly improves Pass@8, while micro-level diversity improves within-strategy implementations and mainly benefits Pass@1; Full SDRL balances both and gains most on MIPLIB-NL.Standard RL alone substantially outperforms the base model, and the two hierarchical diversity components exhibit clear complementarity.Source: §4.3 · p. 8, Table 2 · p. 9

4.4 ABLATION STUDY OF MIXED-FORMAT TRAINING

• Text-only training achieves the highest Pass@8 on textual benchmarks, file-grounded training on MIPLIB-NL, while mixed-format training gives the best Pass@1 in both settings and best overall balance.The text-only and file-grounded conditions use the same underlying instances and differ in input representation, not content.Source: §4.4 · pp. 8–9, Table 3 · p. 9

4.5 TRAINING DYNAMICS

• During training, solver-integrated reasoning initially dominates, heuristic search steadily increases, exact algorithms remain relatively stable, and Full SDRL achieves the highest hierarchical diversity among four reward ablations.The authors suggest heuristic strategies may become increasingly important for industrial-scale optimization, given the evaluation protocol’s penalty for near-optimal solutions.Source: §4.5 · p. 9, Figure 4 · p. 9

5 CONCLUSION AND LIMITATIONS

• SDRL rewards diversity only for correct, executable solutions, favoring underrepresented strategies and runtime modes to reduce strategy collapse.The mixed-format training scheme supports both self-contained textual problems and file-grounded instances.Source: §5 · pp. 9–10
• SDRL-Qwen3-4B outperforms fine-tuned NL-to-Opt models on average Pass@1, while SDRL-Qwen3-32B exceeds DeepSeek-V4-Pro and GPT-5.5 and shows clearer Pass@k advantages.SDRL-Qwen3-32B remains competitive with other frontier LLMs.Source: §5 · p. 10
• The strict objective-value tolerance may undervalue heuristics that produce high-quality solutions at lower computational cost.The authors propose future evaluation metrics that consider both solution quality and computational cost.Source: §5 · p. 10

AI USE STATEMENT

• Generative AI tools generated and reformatted training data, served as baselines, and assisted with code; authors reviewed the work and take responsibility for its content and artifacts.The authors state that the core ideas, methodology, experimental design, and conclusions are their original work.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

• The work uses public optimization benchmarks, involves no human subjects or sensitive data, and reports no anticipated direct negative societal impacts.File-grounded instances are machine-generated conversions of existing textual instances and are verified as described in Appendix B.1.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• Reproducibility materials specify prompts, data construction and verification, training configuration, reward function, evaluation settings, and runtime hardware; code, data, and scripts will be released upon publication.The statement also identifies where benchmark instance counts and the final training-set mixing ratio are reported.Source: REPRODUCIBILITY STATEMENT · p. 10

A PROMPT TEMPLATES

• The prompt template directs an Operations Research engineer to assess problem type, mathematical structure, and scale before routing the problem to a solution paradigm.Source: A PROMPT TEMPLATES · p. 16

1. SIR

• The SIR prompt routes each problem to one strategy: solver-based modeling for manageable LP, MILP, or convex quadratic problems, exact algorithms for suitable structures, or heuristics for large or complex problems.Source: §1 · p. 16

B.1.1 CONVERSION

• DeepSeek-V4-Pro converts each OptMATH question into a parameterized problem, scalar parameters, and structured tables serialized as CSV, while retaining the original optimal value as supervision.Values already represented in tables are omitted from the parameter dictionary, so conversion changes information access but not the optimization task.Source: §B.1.1 · p. 17
• The conversion format requires parameter placeholders to map to scalar values, while table descriptions define column meanings and explain how missing, inapplicable, or empty values are represented.Table-stored values are excluded from the parameter dictionary to avoid duplication.Source: §B.1.1 · p. 17

B.1 CONSTRUCTION OF FILE-GROUNDED TRAINING DATA

• Only instances that pass all three increasingly costly verification stages are retained, addressing the risk that conversion errors could mismatch an unchanged label and specification.The stages are deterministic integrity checks, model-based source–conversion review, and solve-back; Table 4 describes conditional pass rates for each stage.Source: §B.1.2 · p. 19, Table 4 · p. 19

B.1.2 VERIFICATION

• Stage 1 deterministically rejects missing or malformed tables, unresolved placeholders, numerical mismatches, unrealized declared counts, and instances that expose the optimal value in the problem text.The numerical-mismatch check rejects instances when more than 10% of source-question values are absent from parameters or tables, or table values are absent from the source.Source: §B.1.2 · p. 19
• Stage 2 uses Claude-Sonnet-4.6 to compare the source and conversion across objectives, variables, constraints, indices, and column meanings; instances pass at confidence of at least 0.7 when judged consistent.The optimal value is withheld from the judge.Source: §B.1.2 · p. 19
• Stage 3 uses Claude-Opus-4.8 to solve from the converted files, retaining an instance if any of three attempts reproduces its original optimal value within relative tolerance 10−6.The solver reads CSV files at runtime and runs in an isolated workspace with a 120-second limit; the prompt, executor, and tolerance match training so failures indicate instances that cannot earn correctness reward.Source: §B.1.2 · p. 20

B.2 BENCHMARKS

• Seven benchmarks span problem structures, difficulty levels, and domains, from classical linear programming to industrial-scale combinatorial optimization.Table 5 summarizes the number of validated instances used in each benchmark.Source: §B.2 · p. 21, Table 5 · p. 21
• NL4Opt (Ramamonjison et al., 2023) tests translation of natural-language LP problems, while MAMO-EasyLP and MAMO-ComplexLP (Huang et al., 2025b) assess formulation correctness and structural complexity.ComplexLP adds nested conditions and hierarchical constraints beyond standard LP formulations.Source: §B.2 · p. 21
• IndustryOR (Huang et al., 2025a) covers manufacturing, logistics, finance, and energy problems with difficulty labels and often implicit or non-standard constraints.OptMATH-Bench (Lu et al., 2025) extends coverage to mixed-integer, nonlinear, and combinatorial formulations and diverse problem topologies.Source: §B.2 · p. 21
• OptiBench (Yang et al., 2025b) uses reverse Socratic synthesis to provide natural-language problems with intermediate reasoning structure; MIPLIB-NL (Li et al., 2026) contains industrial-scale mixed-integer instances.Three MIPLIB-NL instances with empty data files are excluded, leaving 220 instances for the experiments.Source: §B.2 · p. 21

C.1 TRAINING PARAMETERS

• Training uses Qwen3-4B-Instruct-2507 and Qwen3-32B with GRPO, veRL, asymmetric clipping (ϵlow = 0.20, ϵhigh = 0.28), and eight NVIDIA H200 GPUs.The same configuration is used for both backbone models unless otherwise specified.Source: §C.1 · pp. 21–22

C.2 MIXED-FORMAT TRAINING DETAILS

• Main SDRL training uniformly shuffles about 17K self-contained textual instances and 3K file-grounded instances, where programs can access instance information in external structured files at runtime.Self-contained prompts provide all information needed to construct the optimization problem.Source: §C.2 · p. 22
• Both input formats use the same response format and reward, with no extra reward for file access; the diversity-reward ablation omits file-grounded instances to isolate diversity effects.File-grounded responses are assessed by successful program execution and correct objective value.Source: §C.2 · p. 22

C.3 REWARD FUNCTION DESIGN

• The reward combines format validity (a fenced Python block and one valid strategy tag), execution success, objective-value correctness, and group-level diversity credit only for correct rollouts.Format credit is partial if only one of the two format requirements is met.Source: §C.3 · pp. 22–23
• Diversity bonuses favor underrepresented strategies and rare runtime modes, while rule-based checks compare declared tags with code and explanation evidence to prevent mislabeled rollouts from earning or distorting diversity credit.Invalid strategy declarations receive no diversity reward and are excluded from the strategy distribution; instance-level rewards remain unaffected.Source: §C.3 · pp. 23–24

C.4 EVALUATION SAMPLING SETTINGS

• Pass@1 uses one generation at temperature 0.5, while Pass@K samples K = 8 responses at temperature 1.0 and counts an instance solved if any response is correct.Both settings use top-p 0.95, repetition penalty 1.05, and top-k = −1; correctness requires successful execution and an objective value meeting the relative-tolerance criterion with ϵ = 10−6.Source: §C.4 · p. 24

C.5 DETAILED PASS@1 AND PASS@8 RESULTS

• Table 7 compares the Base Model and SDRL with Qwen3-4B-Instruct-2507 and Qwen3-32B across seven benchmarks under the same decoding configuration.Pass@1 measures single-sample performance; Pass@8 measures whether at least one of eight sampled responses is correct.Source: §C.5 · p. 24, Table 7 · p. 24
• SDRL consistently improves both Pass@1 and Pass@8 across the two model scales.The supplied text characterizes these gains as improvements in single-sample reliability and repeated-sampling coverage.Source: Table 7 · p. 24, §C.5 · p. 24

D.1 EFFECT OF TRAINING DATA SIZE

• The 5K, 10K, and 20K runs use the same configuration and 400 training steps, isolating distinct-instance count while holding optimization budget constant.Smaller datasets are revisited more often; the 20K model is the one used in the main results.Source: §D.1 · pp. 24, 26
• 64.6 average Pass@1 at 20K exceeds 62.9 at 5K and 62.7 at 10K, making 20K the best-overall-accuracy training size.The 20K setting is the default used for the paper’s main results.Source: Figure 5 · p. 25, §D.1 · p. 26
• Average Pass@8 rises from 72.3 at 5K to 73.1 at 10K, then plateaus at 20K.Figure 5 reports average accuracy across seven benchmarks and per-benchmark Pass@1 and Pass@8.Source: Figure 5 · p. 25, §D.1 · p. 26
• Data-size gains concentrate on harder benchmarks such as OptMATH-Bench and MIPLIB-NL, and even 5K beats all Table 1 fine-tuned baselines on average Pass@1.The authors characterize this as data-efficient diversity-driven exploration.Source: §D.1 · p. 26

D.2 SENSITIVITY TO THE MACRO/MICRO WEIGHT λ

• The λ sweep changes only the balance between cross-strategy Rmacro and within-strategy runtime-diversity Rmicro, with λ = 0 and 1 representing micro-only and macro-only variants.The experiments use Qwen3-4B-Instruct-2507, text-only training data, and the Table 6 configuration.Source: §D.2 · p. 26
• With λ = 0.7 as the default, the model gives the best average Pass@1 and Pass@8; average Pass@8 is 60.9–62.3 for λ ≥ 0.1, versus 59.0 at λ = 0.The Pass@8 loss at λ = 0 is concentrated on MIPLIB-NL.Source: Table 8 · p. 26, §D.2 · p. 26
• Average Pass@8 stays at 60.9–62.3 for λ ≥ 0.1 but falls to 59.0 at λ = 0, with the loss concentrated on MIPLIB-NL.The results indicate that cross-strategy exploration drives repeated-sampling coverage.Source: Table 8 · p. 26, §D.2 · p. 26
• Average Pass@1 varies by about 2.6 points across the λ sweep, while per-benchmark fluctuations show no consistent trend.The authors retain λ = 0.7 as the default despite this variability.Source: §D.2 · p. 26

D.3 DISENTANGLING THE EFFECTS OF STRATEGY ROUTING AND REINFORCEMENT LEARNING

• The comparison uses the same Qwen3-4B-Instruct-2507 backbone and text-only data; SDRL is the Meta Prompt text-only variant, not the stronger mixed-format model.Base (SIR) and Base (Meta) are untrained; SIR-RL uses the SIR Prompt without the diversity reward, while SDRL uses the Meta Prompt with that reward.Source: §D.3 · pp. 26–27
• SDRL averages 62.0 vs. 60.4 Pass@1 against SIR-RL and leads by 3.6 points on average Pass@8, winning four of seven benchmarks at Pass@1 and six at Pass@8.Its largest Pass@8 margins are on OptMath (+6.6), IndustryOR (+6.0), MIPLIB-NL (+5.0), and MAMO-ComplexLP (+4.5).Source: Table 9 · p. 27, §D.3 · p. 27
• SIR-RL gains 9.7 Pass@1 points over Base (SIR), while SDRL gains 18.7 over Base (Meta), nearly twice as much; Base (Meta) trails Base (SIR) by 7.4 points.The base model ties at Pass@8 under the two prompts, and SDRL learns routing without strategy labels.Source: §D.3 · p. 27
• SDRL’s Pass@k curves keep rising as sampling increases, while SIR-RL flattens early; SDRL closes and overtakes small-k gaps except on NL4Opt, where both saturate above 95%.The plotted k = 1 value uses a temperature-1.0 sample and is not directly comparable to Table 9 Pass@1, which uses temperature 0.5.Source: §D.3 · p. 27
• The results attribute SDRL’s advantage to learning both when to switch among paradigms and how to keep rollouts diverse, beyond improving trajectory quality within one paradigm.The mixed-format model in the main results extends this advantage further.Source: §D.3 · p. 27

D.4 RUNTIME ANALYSIS

• Runtime was measured by re-executing Pass@8 rollouts in isolated subprocesses, with correct programs assessed using a 10^-6 objective-value tolerance.The setup used 32 concurrent workers, one Gurobi thread and 8 GB memory each, with a 1,800-second time limit; wall time included interpreter startup and imports.Source: §D.4 · pp. 27–28
• On MIPLIB-NL, SDRL’s P90 was 84.5 s versus 150.4 s for the base model, while solving 3.4 times as many rollouts correctly; timeouts were 0.5% versus 0.8%.Across six textual benchmarks, both models solved almost every instance in under a second; the authors conclude that the micro-level reward diversifies runtime modes without inducing slow programs.Source: Figure 7 · p. 28, §D.4 · p. 28, Table 10 · p. 29

E CASE STUDIES

• The case studies illustrate file-grounded problem solving and strategy diversity: SDRL uses SIR, an exact combinatorial algorithm, and heuristic search on a self-contained textual instance.The supplied passages describe the file-grounded example but do not provide details of the second instance.Source: E CASE STUDIES · p. 29

E.1 FILE-GROUNDED PROBLEM SOLVING

• The MIPLIB-NL beasleyC1 case requires loading external instance data at runtime to construct an optimization problem rather than relying on data serialized in the prompt.The example has 500 sites and 1,250 candidate directed links, with instance-specific numerical data in four CSV files.Source: §E.1 · p. 29
• SDRL selects SIR and models beasleyC1 as a fixed-charge network-flow MILP with continuous arc flows, binary activation variables, activation-cost minimization, and node-balance and capacity constraints.The model enforces that flow can use only activated arcs.Source: §E.1 · p. 29
• The generated program loaded and integrated all four external files, constructed the MILP, and obtained the reference optimum of 85 without requiring instance-specific data in the prompt.The example demonstrates file-grounded optimization using data accessed at runtime.Source: §E.1 · p. 30

E.2 STRATEGY DIVERSITY ON A TEXTUAL INSTANCE

• 310.6 is the optimal tour length obtained by all three strategies on the five-exhibit instance.The strategies are solver-integrated reasoning (SIR), an exact combinatorial algorithm, and heuristic search.Source: §E.2 · p. 30, Table 11 · p. 30
• SIR formulates an MILP with Miller–Tucker–Zemlin subtour constraints and solves it with Gurobi.The formulation uses binary edge variables and continuous visit-order variables to enforce a single tour.Source: §E.2 · pp. 30–31
• The exact combinatorial strategy fixes the starting exhibit and exhaustively enumerates 4! = 24 directed tours.For this five-node instance, exhaustive search is described as computationally trivial and guarantees global optimality.Source: §E.2 · pp. 30–31
• Heuristic search uses simulated annealing with 2-opt moves, accepting uphill moves probabilistically as temperature decreases.Its schedule uses T0 = 100, α = 0.99, 20 moves per temperature, and stops below 10^-4; the random seed is 42.Source: §E.2 · pp. 30, 32