Source-linked AI summary
Towards Optimizing with Large Language Models
Pei-Fu Guo, Ying-Hsuan Chen, Yun-Da Tsai, Shou-De Lin
TL;DR
This paper assesses how LLMs optimize across different tasks and data sizes, using interactive prompting and three metrics to evaluate performance. It finds strong capabilities on small-scale problems, alongside task and input sensitivities that motivate further research.
Problem
The paper examines LLM effectiveness in optimization beyond multiple-choice or single-question tasks, where solutions are generated and evaluated iteratively.
Source: §1 · p. 1Method
The study evaluates four optimization tasks with interactive prompting and introduces three metrics for assessing performance across tasks with reduced sensitivity to sample variation.
Source: §1 · p. 1Results
LLMs show strengths across the examined tasks, with their greatest strength in Gradient Descent, greater difficulty in meta-heuristics, and stronger Black-Box performance with limited samples.
Source: §8 · p. 6Takeaways & Limitations
The findings indicate that pretrained knowledge dominates LLM optimization capability and that LLMs have a richer solution space on small-scale problems.
Source: §8 · p. 6LLM outputs are sensitive to numerical values, starting points, and supplied data, so instruction descriptions and data initialization require care when interpreting results.
Source: §7 · p. 5
Abstract
from arXiv · showhide
In this work, we conduct an assessment of the optimization capabilities of LLMs across various tasks and data sizes. Each of these tasks corresponds to unique optimization domains, and LLMs are required to execute these tasks with interactive prompting. That is, in each optimization step, the LLM generates new solutions from the past generated solutions with their values, and then the new solutions are evaluated and considered in the next optimization step. Additionally, we introduce three distinct metrics for a comprehensive assessment of task performance from various perspectives. These metrics offer the advantage of being applicable for evaluating LLM performance across a broad spectrum of optimization tasks and are less sensitive to variations in test samples. By applying these metrics, we observe that LLMs exhibit strong optimization capabilities when dealing with small-sized samples. However, their performance is significantly influenced by factors like data size and values, underscoring the importance of further research in the domain of optimization tasks for LLMs.
1 INTRODUCTION
The paper assesses LLMs as interactive optimizers across diverse tasks and introduces metrics for evaluating their performance. It reports strong capabilities on small-scale problems, while noting sensitivity to sample size and value range.
1 INTRODUCTION
- The study explores LLM optimization through iterative generation and evaluation, with each new solution informed by previous solutions and their values.The authors frame the work as assessing LLMs’ potential and limitations in these settings, not as an attempt to outperform state-of-the-art optimization algorithms.Source: §1 · p. 1
- The evaluation spans Gradient Descent, Hill Climbing, Grid Search, and Black Box Optimization, and introduces three metrics for assessing performance across tasks with reduced sensitivity to sample variation.The contributions also emphasize examining the impact of problem dimension and task type.Source: §1 · p. 1
- LLMs show strong optimization capabilities on small-scale problems, but performance is notably affected by sample size and value range.The authors call for further research on optimization tasks tailored to LLMs.Source: §1 · p. 1
2 RELATED WORKS
The related work describes LLMs applied to optimization and learning through natural-language prompting, linguistic feedback, and combinations with evolutionary algorithms.
Related work overview
- The section situates these approaches as examples of LLM use in optimization and reinforcement-learning challenges across domains.Source: §2 · p. 1
OPRO
Reflexion
3 PROBLEM SETTING
The paper defines four tasks that require LLMs to search algorithmically for optimal parameter values, spanning distinct optimization domains and discrete or continuous search spaces.
Problem setting
- The four tasks are Gradient-Descent, Hill-Climbing, Grid-Search, and Black-Box Optimization, representing gradient-based, meta-heuristic, decision-theoretic, and Bayesian domains.Grid-Search and Hill-Climbing use discrete spaces; Gradient-Descent and Black-Box Optimization use continuous spaces.Source: §3 · p. 2
Gradient-Descent
- Gradient-Descent tests whether the LLM can compute gradients and update parameters using the specified learning rate and loss function.Source: §3 · p. 2
Hill-Climbing
- Hill-Climbing tests adherence to unseen custom rules by changing one solution element by one unit and selecting the neighbor with minimum loss.Source: §3 · p. 2
Grid-Search
- Grid-Search tests exhaustive search by generating grid points and finding the point with the lowest loss in a predefined space.Source: §3 · p. 2
Black-Box Optimization
- Black-Box Optimization gives the LLM a limited set of solutions with true losses and asks it to propose lower-loss solutions without knowing the loss function.Source: §3 · p. 2
4 METHODOLOGIES
The methodology uses iterative prompting to have LLMs search for better solutions, with prior results and reasoning carried forward between steps.
4 METHODOLOGIES
- An iterative prompting framework guides LLMs to incrementally improve solutions while navigating the search space.Source: §4 · p. 2
- The prompting method combines Chain of Thoughts with iterative prompting, using reasoning thoughts as intermediate outputs.Source: §4 · p. 2
- At each task’s outset, the LLM formulates a loss function from samples; each iteration then generates a solution from algorithm instructions and prior results, evaluates it, and feeds the result into the next prompt.The cycle repeats until the stopping criteria are met.Source: §4 · p. 2, Figure 1 · p. 2
- The interactive setup uses chat history to retain prior search results and reasoning, appending instructions each iteration; when the token limit is exceeded, earlier dialogue is removed.Source: §4 · p. 2
5 EVALUATION
The paper introduces three metrics to assess optimization progress, alignment with a ground truth, and consistency across repeated trials. Their ratio-based design is intended to make evaluation less sensitive to sample variations.
5 EVALUATION
- The three metrics provide a multifaceted evaluation across diverse tasks, using ratios rather than differences to reduce sensitivity to sample variations.Source: §5 · p. 2
5.1 Goal Metric
- The goal metric measures optimization progress by comparing LLM output losses with initial solution losses; higher values indicate greater progress.It is particularly useful when ground truth is absent, as in Black-Box optimization.Source: §5.1 · pp. 2–3
5.2 Policy Metric
- The policy metric measures alignment between the final model output and ground truth, with lower values indicating closer alignment.It also indicates how well the LLM adheres to task-specific instructions.Source: §5.2 · p. 3
5.3 Uncertainty Metric
- The uncertainty metric quantifies variability in an LLM’s solutions across trials under identical conditions.The paper frames consistency across repeated trials as important for stable, reproducible results.Source: §5.3 · p. 3
6 EXPERIMENTS
The experiments evaluate LLM optimization across increasing problem dimensions and report strong performance on small-scale problems, alongside task-specific differences. Performance changes with dimension, and the evaluation omits some tasks from particular metric plots because of their nature or unavailable ground truth.
6.1 Dataset
- The dataset varies optimization dimension across five values: 3, 6, 12, 24, and 48, with instances drawn from [0, 10]^d.True solutions and their losses serve as ground truth for evaluating LLM solutions.Source: §6.1 · p. 3
6.2 Detailed Settings
- Each instance was tested in five repetitions, with ten parameter-search iterations per repetition, using GPT-turbo-3.5 ’0613’ at temperature 0.8.Excessively biased results were excluded to prevent metrics from being skewed by a minority of poorly performing outcomes.Source: §6.2 · p. 3
6.3 Main Results
- Plots use optimization dimension on the x-axis; Goal and Policy Metric plots show average metric values with confidence intervals, while the Uncertainty Metric reflects the standard deviation of final-solution loss.Source: §6.3 · p. 3
- LLMs show strong optimization capabilities on small-scale problems, while performance varies across tasks and with problem dimension.The Goal Metric plot excludes non-iterative Grid-Search, and the Policy Metric plot excludes Black-Box optimization because ground truth is unattainable.Source: §6.3 · p. 3
- GPT-4 excels at Black-Box optimization at dimensions three and six, while both models’ performance diminishes as dimension increases.GPT-turbo-3.5 performs notably at dimension three; GPT-4 edges it out slightly in optimization and stability.Source: §6.3 · pp. 3–4
- In Gradient-Descent, the policy metric remains near zero, indicating alignment with ground truth, even as the goal metric declines with increasing sample size.The paper reports GPT’s performance in this task as nearly on par with the truth model.Source: §6.3 · p. 4
7 ANALYSIS AND DISCUSSION
The analysis attributes LLM optimization performance partly to pretrained knowledge and finds stronger performance on small-scale problems, while noting sensitivity to numerical values and sample conditions. It also identifies potential for switching optimization methods and reports model-dependent effects from self-consistency prompting.
Pretrained Knowledge dominates the optimization capability of LLM.
- Gradient Descent leads the tested tasks, while Hill-Climbing is more challenging, a contrast the authors relate to mathematical principles versus user-specific rules.They suggest optimization capability draws more on pretrained knowledge than user-provided context.Source: §7 · p. 4
LLMs are potential hybrid optimizers.
- Positive goal metrics across most tasks and datasets suggest LLMs could switch optimization methods within a task, potentially helping them explore and escape local optima.The proposed hybrid role also includes using algorithms as tools and selecting among them based on the optimization path.Source: §7 · pp. 4–5
LLMs possess richer solution space in small-scale problems.
- LLMs perform more effectively on smaller-dimension instances, where higher uncertainty and metric variation suggest a richer solution space.The authors note uncertainty initially rises and then declines as dimensions grow.Source: §7 · p. 5, Figure 5 · p. 5
LLMs are sensitive to numerical values.
- Interpretation is constrained by sensitivity to numerical values, sample randomness, starting points, and prompt details.The authors note that numerical preferences may bias responses and that initialization and instructions can influence outputs.Source: §7 · p. 5
Self-consistency prompting improves stability.
- Five repetitions per iteration improved GPT-4's performance and narrowed policy- and goal-metric confidence intervals, but did not yield favorable outcomes for GPT-turbo-3.5.The authors identify variance reduction as needing further investigation.Source: §7 · p. 6, Figure 6 · p. 6
8 CONCLUSION AND FUTURE DIRECTIONS
The paper evaluates LLMs as iterative optimizers across four tasks, finding strengths in gradient descent, grid search, and small-sample black-box optimization alongside greater difficulty with unseen meta-heuristic rules. Its analysis highlights pretrained knowledge, richer small-scale solution spaces, and potential for hybrid optimization, while leaving questions for further research.
8 CONCLUSION AND FUTURE DIRECTIONS
- The study examines iterative optimization in which LLMs generate new solutions based on prior solutions and corresponding values.The evaluation asks models to understand algorithmic instructions across four optimization tasks.Source: §8 · p. 6
- LLMs performed best in Gradient-Descent, showed difficulty with unseen meta-heuristic rules, and also showed strengths in Grid-Search and limited-sample Black-Box optimization.The evaluation covers four tasks and generates new solutions using previous solutions and their values.Source: §8 · p. 6
- The authors identify pretrained knowledge, richer solution spaces in small-scale problems, and potential hybrid optimization as key insights requiring further study.They describe these findings as raising unresolved questions.Source: §8 · p. 6
A PROMPT TEMPLATES
The section presents example prompts for objective-function setup and four optimization methods, alongside iterative prompts for those methods.
A PROMPT TEMPLATES
- A separate example prompt is provided for obtaining the objective function.Source: Figure 7 · p. 7
- The prompt examples cover Gradient-Descent, Grid-Search, Hill-Climbing, and Black-Box Optimization.Source: Figure 8 · p. 7, Figure 9 · p. 7, Figure 11 · p. 8
- The iterative prompt examples cover Gradient-Descent, Grid-Search, Hill-Climbing, and Black-Box Optimization.Source: Figure 14 · p. 8, Figure 15 · p. 9, Figure 16 · p. 9