PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Supervised Prompt Optimization

Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, Yuyu Luo

arXiv:2502.06855v3cs.CLcs.AIcs.LG

TL;DR

Existing prompt-optimization methods often depend on external references or human evaluation, limiting use where reference data is unavailable or costly. SPO uses pairwise comparisons of LLM outputs to evaluate and optimize prompts without external references. It reports superior performance across diverse tasks at 1.1%–5.6% of existing methods’ costs, while requiring three samples.

  • Problem

    Existing prompt-optimization methods often depend on ground truth or human feedback, which can be unavailable, costly, or time-consuming, and may require many samples.

    Source: §1 · p. 2

  • Method

    SPO uses pairwise comparisons of LLM outputs for evaluation and uses outputs to guide prompt optimization without external references.

    Source: §1 · p. 2

  • Results

    1.1%–5.6% of existing methods’ costs accompanies superior performance across diverse tasks.

    Source: §1 · p. 3, §6 · p. 8

  • Takeaways & Limitations

    SPO’s reference-free optimization achieves state-of-the-art performance across diverse tasks while requiring only 1.1%–5.6% of existing methods’ costs.

    Source: §6 · p. 8

    SPO’s effectiveness depends on the evaluation model’s ability to assess output quality, and specialized-domain misunderstanding may introduce bias; the approach currently targets a single model.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Well-designed prompts are crucial for enhancing Large language models' (LLMs) reasoning capabilities while aligning their outputs with task requirements across diverse domains. However, manually designed prompts require expertise and iterative experimentation. While existing prompt optimization methods aim to automate this process, they rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. To address this, we propose Self-Supervised Prompt Optimization (SPO), a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without requiring external reference. Motivated by the observations that prompt quality manifests directly in LLM outputs and LLMs can effectively assess adherence to task requirements, we derive evaluation and optimization signals purely from output comparisons. Specifically, SPO selects superior prompts through pairwise output comparisons evaluated by an LLM evaluator, followed by an LLM optimizer that aligns outputs with task requirements. Extensive experiments demonstrate that SPO outperforms state-of-the-art prompt optimization methods, achieving comparable or superior results with significantly lower costs (e.g., 1.1% to 5.6% of existing methods) and fewer samples (e.g., three samples). The code is available at https://github.com/FoundationAgents/SPO.

1 Introduction

Prompt optimization can reduce manual prompt-design effort, but existing approaches often depend on costly or unavailable external references and many samples. SPO instead derives evaluation and optimization signals from model outputs, reporting strong performance at substantially lower cost.

1 Introduction

  • Existing methods often require ground truth or human feedback, which can be unavailable for open-ended tasks, time-consuming, and costly to obtain.Many methods also evaluate prompts on numerous samples, increasing computational overhead.Source: §1 · p. 2
  • SPO derives evaluation and optimization signals from LLM outputs, using pairwise comparisons without external references.The framework builds on the Optimize-Execute-Evaluate loop.Source: §1 · p. 2
  • The LLM evaluator compares outputs for relative quality, while the optimizer uses the current best output to guide prompt changes toward task requirements.SPO’s proposed signals draw on the premise that output quality reflects prompt quality and LLMs can judge task adherence.Source: §1 · p. 2
  • SPO requires $0.15 per dataset and three samples, reducing computational and sample requirements.The introduction describes these as minimal resource demands.Source: §1 · p. 3
  • 1.1%–5.6% of state-of-the-art methods’ costs accompanies superior performance across closed and open-ended tasks.Figure 2 reports SPO’s costs at 1.1% to 5.6% of other methods’ costs across datasets.Source: Figure 2 · p. 2, §1 · p. 3

2 Preliminary

Prompt optimization seeks a prompt that improves task outputs through iterative optimization, execution, and evaluation. Its evaluation framework distinguishes reference-based assessment from output comparison, with output-versus-output feedback forming SPO’s core approach.

2.1 Problem Definition

  • Prompt optimization seeks an effective prompt for a task by revising prompts, executing them with an LLM, and evaluating the resulting outputs.The evaluation function’s feedback guides subsequent prompt assessment and improvement.Source: §2.1 · p. 3

2.2 Evaluation Framework in Prompt Optimization

  • Evaluation frameworks combine evaluation sources, methods, and feedback types; Figure 3 highlights SPO’s selected approach across these components.The framework distinguishes model-generated outputs and task-specific ground truth, benchmark, LLM-as-a-judge, and human evaluation methods, as well as numerical, textual, and ranking or selection feedback.Source: §2.2 · p. 3, Figure 3 · p. 4
  • Output-versus-ground-truth feedback compares an output with a known answer, but requires defined ground truth and is unsuitable when open-ended tasks lack practical reference answers.The approach provides direct quality assessment when a suitable external reference exists.Source: §2.2 · p. 3
  • Output-versus-output comparison estimates relative quality without external references, making it useful for open-ended tasks and the core feedback method in SPO.The comparisons support iterative optimization in both closed and open-ended tasks.Source: §2.2 · p. 4

3 Self-Supervised Prompt Optimization

SPO uses model outputs both to compare prompt quality and to guide prompt revision, iterating through optimization, execution, and evaluation. The paper reports superior performance across tasks with three samples and eight LLM calls per iteration, while noting that evaluator bias cannot be fully eliminated.

3.1 An Overview of SPO

  • SPO retains the Optimize-Execute-Evaluate loop while using model outputs as both evaluation sources and optimization guidance.Its core challenge is constructing effective reference-free evaluation and optimization signals.Source: §3.1 · p. 4
  • The optimizer analyzes the current best prompt and its outputs to generate candidates, and execution applies those prompts to obtain outputs.The process starts from a basic prompt template and a small question set sampled from the dataset.Source: §3.1 · p. 4
  • An LLM evaluator compares outputs pairwise to select the superior prompt, which becomes the best candidate for the next iteration until the iteration limit.The accompanying running example shows the best output and prompt pair being updated after evaluation.Source: §3.1 · p. 4, Figure 4 · p. 5
  • With three samples, SPO requires eight LLM calls per iteration and demonstrates superior performance across a range of tasks.The passage contrasts its call count with existing methods and points to the later section for task results.Source: §3.1 · p. 4

3.2 Understanding the Effectiveness of SPO

  • SPO’s reference-free premise combines outputs that reflect prompt effectiveness with LLMs’ ability to assess answer quality against task requirements.The paper cites step-by-step reasoning paths as an example of outputs reflecting prompting effects.Source: §3.2 · pp. 4–5
  • Pairwise selection uses the evaluator’s task preferences as an internal signal, avoiding the large sample sizes score-based feedback may need for stable scoring.The selected output then guides optimization toward the LLM’s understanding of superior task solutions.Source: §3.2 · p. 5
  • Four rounds of randomized evaluation mitigate potential biases, but cannot eliminate them completely.The authors state that this does not affect the overall optimization trend because evaluator feedback is a reference for the next round, while optimization follows the optimization model’s task understanding.Source: §3.2 · p. 5

4 Experiment

SPO was evaluated on five closed-task benchmarks and open-ended MT-Bench tasks, alongside conventional prompts and automated optimization methods. It outperformed conventional prompts on average, used a small fraction of the costs of ground-truth-dependent methods, and showed gains across model configurations and open-ended tasks.

4.1 Experimental Setup

  • The evaluation covered five closed-task benchmarks and writing, roleplay, and humanities tasks from MT-Bench.Closed-task baselines included conventional prompting and automated prompt optimization methods; MT-Bench outputs were compared using GPT-4o.Source: §4.1 · p. 6

4.2 Experimental Results and Analysis

  • SPO exceeded the best conventional-prompt baseline by 1.9 on average across the five closed-task benchmarks.It also performed comparably to ground-truth-dependent optimization methods across most datasets and reached optimal results on GPQA and BBH-navigate.Source: §4.2 · p. 6, Table 1 · p. 6
  • SPO required 1.1% to 5.6% of the optimization costs of ground-truth-dependent methods, averaging $0.15 per dataset.The comparison includes SPO using Claude-3.5-Sonnet and GPT-4o as optimization models.Source: Table 1 · p. 6, §4.2 · p. 7
  • On BBH-Navigate, SPO showed robust performance across model configurations, with GPT-4o-mini used for optimization, execution, and evaluation achieving 97.8.SPO also raised Claude-3-Haiku execution performance from 62.2 to 89.7.Source: §4.2 · p. 7, Table 2 · p. 7, Table 3 · p. 7
  • With more samples or iterations, performance initially improved but eventually converged or declined, which the authors attribute to overfitting to sample questions.The study used BBH-Navigate to examine sample number and iteration rounds.Source: §4.2 · p. 7
  • Across writing, roleplay, and humanities tasks, SPO significantly improved performance across model configurations, and optimized GPT-4o-mini often outperformed larger models.The comparison used pairwise win rates across three tasks; the heatmap encodes each row model’s win rate against each column model, before and after SPO.Source: §4.2 · p. 7, Figure 5 · p. 7

4.3 Case Study

  • The case study reports optimization on four tasks without conventional benchmarks and presents both successful and unsuccessful BBH-Navigate optimization trajectories.The four tasks were Advertising Design, Social Media Content, Modern Poetry Writing, and Concept Interpretation.Source: §4.3 · p. 8

5 Related Work

Prompting research spans task-agnostic techniques and domain-specific prompting, while manual domain-specific prompt design becomes impractical as applications grow more complex. SPO differs from prior prompt-optimization approaches by using model-output pairwise comparisons without external references.

5.1 Prompt Engineering

  • Prompt engineering covers task-agnostic methods such as chain-of-thought and output-format specification, as well as domain-specific prompting for applications including code, data analysis, and question answering.Expanding application complexity makes manually crafting effective domain-specific prompts impractical.Source: §5.1 · p. 8

5.2 Prompt Optimization

  • Prior prompt-optimization methods use ground-truth evaluation, LLM judges that still depend on ground truth, human preferences, or alternative criteria, whereas SPO uses pairwise model-output comparisons without external references.The paper describes SPO as requiring neither ground truth nor human feedback or extensive sampling.Source: §5.2 · p. 8

6 Conclusion

SPO addresses prompt optimization’s dependence on external references by deriving evaluation and optimization signals from pairwise model-output comparisons. The paper reports state-of-the-art performance across diverse tasks at a fraction of existing methods’ costs.

6 Conclusion

  • SPO targets prompt optimization’s dependence on external references.The paper identifies this dependency as a key challenge.Source: §6 · p. 8
  • SPO constructs evaluation and optimization signals through pairwise comparisons of model outputs, enabling reference-free optimization.The framework draws inspiration from self-supervised learning.Source: §6 · p. 8
  • SPO achieves state-of-the-art performance across diverse tasks while requiring 1.1%-5.6% of existing methods’ costs.The conclusion cites success on benchmarks and real-world applications as evidence of effectiveness and generalization.Source: §6 · p. 8

Limitations

SPO’s effectiveness depends on the evaluation model’s ability to assess output quality, and its current approach optimizes prompts for a single model.

Limitations

  • An evaluation model with insufficient specialized-domain understanding may introduce bias, and the current approach focuses on optimizing prompts for a single model.The authors identify cross-model prompt optimization as future work.Source: Limitations · p. 9

Ethical Consideration

SPO offers reduced costs and improved performance across tasks, while identifying potential risks from bias amplification, harmful-content misuse, and overreliance on LLMs.

Ethical Consideration

  • SPO’s stated risks include potential bias amplification, misuse for harmful-content generation, and overreliance on LLMs.The passage also lists democratized access, reduced costs, and improved performance as benefits.Source: Ethical Consideration · p. 9

A.1 Detailed Prompts of SPO

The appendix provides prompt templates for iterative prompt improvement and pairwise response evaluation. The optimizer template targets requirement compliance and incremental changes.

A.1 Detailed Prompts of SPO

  • The iteration meta-prompt is described as a simple starting template that could be further optimized for specific domains.The prompt structure uses analysis, modification, and optimized-prompt fields.Source: §A.1 · p. 12
  • The optimizer reconstructs and optimizes a prompt from the requirements, with instructions to make a single modification in XML tags.It can build on a prompt that performed well in a previous iteration, but the modified prompt must differ from that example.Source: §A.1 · p. 12
  • The optimization template asks the LLM to analyze weaknesses in reference-prompt outputs, propose modifications, and produce an optimized prompt.It emphasizes incremental improvements while maintaining requirement compliance.Source: §A.1 · p. 12
  • The evaluation template compares responses A and B against task requirements and requires a choice after XML-formatted analysis.It asks for strengths and weaknesses of the two responses before selecting one.Source: §A.1 · p. 12

A.2 Detailed Prompt Template of Iteration Start

The iteration-start example configures a BBH-navigate task with an initial prompt, requirements, and three questions. Its examples request a yes-or-no answer in XML.

A.2 Detailed Prompt Template of Iteration Start

  • The example instructions require step-by-step reasoning and place the final Yes-or-No answer in an XML answer tag.The navigation prompts specify an answer at the end and give Yes/No options.Source: §A.2 · pp. 12–13
  • The configuration illustrates navigation questions with different movement sequences, including one using turns and another specifying an initial facing direction.Both examples ask whether the instructions return the user to the starting point.Source: §A.2 · pp. 12–13
  • The BBH-navigate YAML configures an initial prompt, requirements, and three task questions to start iterative optimization.The answer section is a reference for thought process and output format, not the actual answer.Source: §A.2 · pp. 12–13

A.3.1 Tasks and Data Details

This section documents the benchmark setup and prompt variants, examines SPO’s sample and iteration choices and optimization trajectories, and presents examples across navigation and creative tasks. It also reviews prompt-optimization evaluation approaches and contrasts task-agnostic with domain-specific prompting.

A.3.1 Tasks and Data Details

  • Experiments use LIAR LIAR (Wang, 2017), BBH-Navigate, WSC, AGIEval-MATH, GPQA, and MT-bench.BBH-Navigate uses random sampling with seed 42 for 200/25/25 test/train/validation examples; AGIEval-MATH uses Level 5 for test and Level 4 for training and validation, while GPQA uses GPQA-Diamond for test and GPQA-main-exclusive questions for training and validation.Source: §A.3.1 · p. 13

A.3.2 Configuration

  • SPO runs 10 optimization iterations per task, sampling three questions without answers per iteration, with Claude-3.5-Sonnet optimizing and GPT-4o-mini evaluating and executing.The sampled questions come from the pre-split Train&Validate dataset, and the setup demonstrates optimization without ground-truth answers.Source: §A.3.2 · p. 14

A.3.3 Baseline Prompt

  • All prompt-optimization methods requiring an initial prompt receive the same chain-of-thought baseline prompt.The supplied baseline fragments include step-by-step reasoning and an instruction to conclude with the answer.Source: §A.3.3 · p. 14

A.3.4 Prompt Optimized by SPO

  • SPO’s optimized prompts use task-specific guidance, including structured analysis and solutions, coordinate tracking for navigation, and evidence-based claim ratings.The supplied prompts also prescribe concise final-answer formats.Source: §A.3.4 · p. 15

A.4.1 Stability of SPO

  • Table A3 reports standard deviations for the main experiments and compares SPO using Claude-3.5-Sonnet with SPO* using GPT-4o as optimization models.Source: Table A3 · p. 16

A.4.2 Ablation Study on Sample Number

  • Three samples achieve the reported optimal balance between cost-efficiency and performance as performance first improves, then converges or declines with more samples.The authors attribute this pattern to overfitting with too few samples and increased cost and longer evaluation context with too many.Source: Figure A1 · p. 16, §A.4.2 · p. 16

A.4.3 Ablation Study on Iteration Rounds

  • 95.8% is the Claude-3.5-Sonnet performance peak at iteration 7 on BBH-Navigate, while SPO typically finds optimal prompts within 10 iterations.Further iterations raise costs and may cause question-specific overfitting; similar patterns were observed across optimization models.Source: §A.4.3 · pp. 16–17, Figure A2 · p. 17

A.5.1 Optimization Trajectory

  • Iteration 0 is marked successful and iteration 1 failed in the GPT-4o-mini BBH-Navigate trajectory, whose optimization, evaluation, and execution models are all GPT-4o-mini.The successful iteration-0 prompt asks for step-by-step reasoning and an XML yes/no answer; the failed iteration-1 prompt requests step-by-step analysis in XML.Source: §A.5.1 · p. 17
  • The final BBH-navigation prompt shifts from textual position descriptions to coordinate-based reasoning, although the prompt did not explicitly request coordinates.Across 10 iterations, successful prompts improve the elicited thought process while failed prompts can produce oversimplified or redundant reasoning.Source: §A.5.1 · pp. 19–20

A.5.2 Real-World Applications

  • The real-world demonstrations optimize four creative tasks for 10 iterations using Claude-3.5-Sonnet, GPT-4o-mini, and Llama-3-8B as optimizer, evaluator, and execution model, respectively.The tasks are Advertising Design, Social Media Content, Modern Poetry Writing, and Concept Interpretation.Source: §A.5.2 · p. 20
  • Optimized advertising titles replace generic Black Friday copy with more compelling, specific language that combines technological innovation with discount value for tech enthusiasts.The example changes from “BLACK FRIDAY ALERT: Unbeatable Tech Deals Inside!” to “Unlock the Future: Up to 50% Off Top Tech Deals”.Source: §A.5.2 · p. 21
  • The optimized coffee tweet replaces a free-giveaway pitch with humorous, relatable content that encourages interaction and emotional connection.The comparison contrasts the pre-optimization giveaway with the line “When coffee is the only adult in the room.”Source: §A.5.2 · p. 22
  • The optimized climate poem uses scientific and interdisciplinary metaphors, including “fractal whispers” and “thermodynamic entropy,” to deepen its artistic and intellectual reflection.Source: §A.5.2 · p. 23
  • The optimized mRNA explanation uses a “recipe card” metaphor and chef roles to make the concept more focused, vivid, and intuitive.The example also states that mRNA vaccines are relatively new and that more research is needed on long-term effects and potential side effects.Source: §A.5.2 · p. 24

A.6.1 Prompt Engineering

  • Prompting research includes task-agnostic techniques for general capabilities and domain-specific techniques for areas such as code generation, data analysis, question answering, and decision-making.Source: §A.6.1 · p. 24
  • As LLM applications expand into complex real-world scenarios, manually crafting effective prompts for each domain becomes impractical.This has motivated research on systematically developing domain-specific prompts rather than discovering general prompting principles.Source: §A.6.1 · p. 24

A.6.2 Prompt Optimization

  • Prompt-optimization evaluation frameworks shape both optimization effectiveness and computational efficiency, and methods can be categorized by evaluation source and mechanism.The passage describes an evolution from collecting evaluation feedback to generating optimization signals.Source: §A.6.2 · pp. 24–25
  • Ground-truth-based numerical evaluation has succeeded on specific tasks but typically needs many iterations and samples, raising computational overhead.The cited approaches include Zhou et al. (2023), Guo et al. (2024a), Yang et al. (2023), Fernando et al. (2024), Wang et al. (2024d), and Khattab et al. (2023).Source: §A.6.2 · p. 25
  • LLM-as-a-judge methods can provide detailed feedback with fewer samples, but still require ground truth, limiting use on open-ended tasks without reference answers.The passage cites Yan et al. (2024), Yüksekgönül et al. (2024), Wu et al. (2024b), Wang et al. (2024c), Pryzant et al. (2023), and Li et al. (2025), and identifies Zheng et al. (2023) for LLM-as-a-judge.Source: §A.6.2 · p. 25
  • Human-preference approaches handle open-ended tasks but require extensive human involvement, while output consistency can be a suboptimal signal because it relates nonlinearly to effectiveness.The passage cites Chen et al. (2024) and Lin et al. (2024) for human-preference approaches and Zhang et al. (2024c) for output consistency.Source: §A.6.2 · p. 25
  • SPO evaluates prompts through pairwise comparisons of model outputs, avoiding ground truth, human feedback, and extensive sampling for efficient real-world use.The passage describes this as SPO’s evaluation paradigm, using model outputs alone.Source: §A.6.2 · p. 25

Related papers

Loading 2502.06855v3…