PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Refine: Iterative Refinement with Self-Feedback

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, Peter Clark

arXiv:2303.17651v2cs.CLcs.AIcs.LG

TL;DR

LLMs often struggle to satisfy intricate requirements in tasks with multifaceted or hard-to-define objectives. Self-Refine uses a single LLM to generate, critique, and iteratively refine outputs without additional training. Across 7 generation tasks, it outperforms direct generation from GPT-3.5 and GPT-4 by 5-40% absolute.

  • Problem

    LLMs often struggle to satisfy intricate requirements in tasks with multifaceted or hard-to-define objectives.

    Source: §1 · p. 1

  • Method

    Self-Refine uses one LLM to generate an initial output, provide feedback, and iteratively refine it without additional training.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    Across 7 generation tasks, Self-Refine outperforms direct generation from GPT-3.5 and GPT-4 by 5-40% absolute.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Single-pass generation can understate the capabilities of strong LLMs such as GPT-4.

    Source: §3.3 · p. 6

    Self-Refine requires base models with sufficient few-shot modeling or instruction-following abilities to provide feedback and refine outputs in context.

    Source: §6 · p. 9

Abstract

from arXiv · show

Like humans, large language models (LLMs) do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement. The main idea is to generate an initial output using an LLMs; then, the same LLMs provides feedback for its output and uses it to refine itself, iteratively. Self-Refine does not require any supervised training data, additional training, or reinforcement learning, and instead uses a single LLM as the generator, refiner, and feedback provider. We evaluate Self-Refine across 7 diverse tasks, ranging from dialog response generation to mathematical reasoning, using state-of-the-art (GPT-3.5, ChatGPT, and GPT-4) LLMs. Across all evaluated tasks, outputs generated with Self-Refine are preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation, improving by ~20% absolute on average in task performance. Our work demonstrates that even state-of-the-art LLMs like GPT-4 can be further improved at test time using our simple, standalone approach.

1 Introduction

Self-Refine improves LLM outputs through iterative self-generated feedback and revision, using one model without additional training. Across seven generation tasks, it outperforms direct generation from GPT-3.5 and GPT-4.

1 Introduction

  • Coherent initial outputs can miss intricate requirements, motivating refinement approaches that avoid extensive supervision.The challenge includes multifaceted objectives and hard-to-define goals; existing approaches can require domain-specific training data, large training sets, or expensive human annotations.Source: §1 · p. 1
  • Human self-refinement involves creating a draft and improving it through self-provided feedback.The paper illustrates this process through revising an impolite email and refactoring an initial code implementation.Source: §1 · pp. 1–2
  • Self-Refine uses the same model to generate an initial draft, provide feedback, and revise it iteratively, without additional training or human assistance.Few-shot prompting (Brown et al., 2020) guides feedback and revision. Iteration stops after a specified number of rounds or when the model determines that further refinement is unnecessary.Source: §1 · p. 2, Figure 1 · p. 2
  • 5-40% absolute improvement over direct generation is reported across seven tasks spanning natural language and source-code generation, using GPT-3.5 and GPT-4.Code-generation tasks show up to 13% absolute improvement over initial generation using Codex (code-davinci-002; Chen et al., 2021).Source: §1 · p. 2

2 Iterative Refinement with SELF-REFINE

SELF-REFINE uses the same LLM to generate an initial output, provide feedback, and iteratively refine it. The method requires three prompts but no training.

Initial generation

  • Initial generation combines the input with a task-specific few-shot prompt or instruction; few-shot prompts contain example input-output pairs.Source: §2 · pp. 2–3

FEEDBACK

  • The feedback prompt demonstrates actionable, specific critiques that identify concrete changes, potentially addressing multiple aspects of output quality.Feedback examples contain input-output-feedback triples; code critiques may address efficiency, readability, and overall quality.Source: §2 · pp. 3–4

REFINE

  • The same model revises its most recent output using its own feedback and a refinement prompt demonstrating feedback-based improvements.Refinement examples combine the input, original output, feedback, and refined output.Source: §2 · p. 4

Iterating SELF-REFINE

  • SELF-REFINE alternates feedback and refinement until a task-specific iteration limit or feedback-derived stopping indicator is reached, then returns the last refinement.The feedback prompt can ask the model to generate a stopping indicator, such as a scalar stop score.Source: §2 · pp. 2, 4
  • Previous feedback and outputs are appended to the prompt to inform subsequent iterations and help the model avoid repeating past mistakes.Source: §2 · p. 4

2 Iterative Refinement with SELF-REFINE

  • SELF-REFINE uses one LLM for generation, feedback, and refinement, relying on three prompts and supervision from few-shot examples rather than training.The prompts guide initial generation, feedback, and refinement.Source: §2 · pp. 2, 4

3 Evaluation

SELF-REFINE improves base models and surpasses previous state-of-the-art across all evaluated tasks, with especially large gains in constrained and preference-based generation. Math Reasoning gains are limited by unreliable error detection.

3 Evaluation

  • 7 evaluation tasks cover dialogue responses, code optimization, code readability, math reasoning, sentiment reversal, and newly introduced acronym generation and constrained generation.Constrained Generation increases the keyword constraints from 3-5 in Lin et al. (2020) to 20-30.Source: §3 · p. 4

3.1 Instantiating SELF-REFINE

  • SELF-REFINE is compared with the same GPT-3.5, ChatGPT, and GPT-4 models without refinement; CODEX is additionally tested on code tasks.FEEDBACK and REFINE use few-shot prompts across models. Iterations stop at the desired quality or a task-specific criterion, with a maximum of 4 iterations.Source: §3.1 · p. 4

3.3 Results

  • 8.7% absolute improvement in GPT-4 Code Optimization (27.3% to 36.0%) accompanies SELF-REFINE’s consistent gains over base models and previous state-of-the-art across all tasks.GPT-4+SELF-REFINE generally outperforms GPT-3.5+SELF-REFINE and ChatGPT+SELF-REFINE, even on tasks where GPT-4’s initial results were lower.Source: Table 1 · p. 5, §3.3 · pp. 5–6

3.2 Metrics

  • Evaluation combines task-specific automated metrics, blind human A/B preferences, and GPT-4 assessments, including contextual variable naming for Code Readability Improvement.Automated metrics are Math Reasoning % solve rate, Code Optimization % programs optimized, and Constrained Generation coverage %. GPT-4 preference correlations with human preference are 82% for Sentiment Reversal, 68% for Acronym Generation, and 71% for Dialogue Response Generation.Source: §3.2 · p. 5

3.3 Results

  • Constrained Generation shows some of the largest gains, which the authors attribute to correcting omitted concepts and exploring many possible valid outputs.The task requires a sentence containing up to 30 supplied concepts.Source: §3.3 · p. 5
  • 49.2% improvement in Dialogue Response Generation’s GPT-4 preference score (25.4% to 74.6%) exemplifies especially high gains across preference-based tasks.Sentiment Reversal and Acronym Generation also show substantial improvements across models.Source: §3.3 · p. 5
  • In Math Reasoning, ChatGPT feedback indicates everything looks good for 94% of instances, with modest SELF-REFINE performance gains traced to difficulty accurately identifying whether reasoning contains an error.Gains are much larger (5%+) when an external source can identify whether the current math answer is incorrect. Errors can be limited to a single line or incorrect operation.Source: §3.3 · pp. 5–6

4 Analysis

SELF-REFINE benefits from specific, actionable feedback and repeated refinement, with humans preferring its outputs over multiple initial ChatGPT samples. The analysis identifies inaccurate feedback and weaker-model refinement failures as limitations, while exploring website generation beyond benchmarks.

The impact of the feedback quality

  • 43.2 to 31.2: Sentiment Transfer performance drops when specific feedback is replaced with generic feedback, and the task fails without feedback.Code Optimization scores are 27.5 with SELF-REFINE feedback, 26.0 with generic feedback, and 24.8 without feedback. Acronym Generation performance drops from 56.4 to 48.0 without actionable feedback, despite iterative refinement.Source: Table 2 · p. 6, §4 · p. 6

How important are the multiple iterations of FEEDBACK-REFINE?

  • 22.0 to 28.8: Code Optimization scores improve after three iterations, with further FEEDBACK-REFINE iterations generally providing diminishing gains.After three iterations, Sentiment Reversal scores increase from 33.9 to 36.8 and Constrained Generation scores from 29.0 to 49.7. Improvements need not be monotonic: for multi-aspect tasks, SELF-REFINE generates numerical aspect scores to support balanced evaluation and output selection.Source: §4 · p. 6, Figure 4 · p. 7

Can we just generate multiple outputs instead of refining?

  • k = 4 initial ChatGPT samples are all less preferred by human evaluators than SELF-REFINE outputs in a 1 vs. k comparison.The comparison tests whether feedback-driven refinement improves outputs beyond merely generating more initial candidates without feedback or refinement.Source: §4 · p. 7

Does SELF-REFINE work with weaker models?

  • Vicuna-13B struggles to generate correctly formatted feedback and follow refinement instructions, even when supplied with Oracle or hard-coded feedback.It often repeats its output or generates a hallucinated conversation instead of refining. The authors hypothesize that conversation-based training limits its generalization to test-time few-shot tasks compared with instruction-based models.Source: §4 · p. 8

Qualitative Analysis

  • 33% of failures involve feedback mislocating errors, 61% involve inappropriate suggested fixes, and only 6% involve refiners incorrectly implementing good feedback.Manual analysis covered 70 samples—35 successes and 35 failures—from Code Optimization (Madaan et al., 2023) and Math Reasoning (Cobbe et al., 2021). Feedback was predominantly actionable, usually identifying problems and suggesting corrections.Source: §4 · p. 8
  • 61% of successful cases use accurate feedback for precise fixes, while 33% succeed despite partially incorrect feedback.A code example improves the baseline from Madaan et al. (2023) by diagnosing six nested loops and replacing brute-force search with dynamic programming, reducing complexity to O(amount * coins).Source: Figure 5 · p. 7, §4 · p. 8

Going Beyond Benchmarks

  • SELF-REFINE explores website generation by iteratively revising HTML, CSS, and JS from a rudimentary design toward improved usability and aesthetics.The exploratory use case starts from a user-provided high-level goal and illustrates potential applicability beyond benchmark tasks.Source: §4 · p. 8

5 Related work

Self-Refine differs from prior refinement approaches in its feedback source, feedback representation, and choice of refiner. It uses natural-language self-feedback and the same model for refinement across domains, without training a separate refiner.

Source of feedback

Representation of feedback

  • Natural-language feedback lets Self-Refine reuse the generating LLM for self-feedback while leveraging pretrained models such as GPT-4.Non-natural-language alternatives include human-provided example pairs (Dasgupta et al., 2019) and scalar rewards (Liu et al., 2022; Le et al., 2022b).Source: §5 · pp. 8–9

Types of refiners

Non-refinement reinforcement learning (RL) approaches

  • Self-Refine uses natural-language feedback on intermediate outputs without parameter updates, unlike the contrasted reinforcement-learning approaches.Those approaches optimize scalar rewards rather than exposing the model to feedback on intermediate generations, and require parameter updates (Stiennon et al. (2020); Lu et al. (2022); Le et al. (2022a)).Source: §5 · p. 9

6 Limitations and Discussion

Self-Refine requires capable base models, and its evaluation is limited to non-open-source models and English datasets. It also lacks explicit safeguards against prompting that steers models toward harmful output.

6 Limitations and Discussion

  • Self-Refine requires sufficient few-shot modeling or instruction-following abilities for base models to provide feedback and refine outputs in context.These capabilities enable feedback and refinement without supervised training or supervised data.Source: §6 · p. 9
  • Experiments use non-open-source, paid GPT-3.5, ChatGPT, GPT-4, and CODEX models, with incompletely documented pretraining corpora, sizes, and biases (Ouyang et al., 2022).Research use requires funding; the authors release code and model outputs to support reproducibility.Source: §6 · p. 9
  • Evaluation exclusively uses English datasets, so current models may not provide the same benefits in other languages.Source: §6 · p. 9
  • Self-Refine does not explicitly guard against malicious prompting that steers models to generate more toxic or harmful text.Source: §6 · p. 9

7 Conclusion · A Evaluation Tasks · B Broader Related Work

SELF-REFINE uses one LLM to provide self-feedback and iteratively refine outputs without additional training data or reinforcement learning. On GSM8K with the same GPT-3 base model and settings, it outperforms Self-Correction (Welleck et al., 2022).

7 Conclusion

  • SELF-REFINE performs iterative self-feedback and refinement within a single LLM, requiring neither additional training data nor reinforcement learning.The conclusion emphasizes its simplicity and applicability across diverse tasks.Source: §7 · p. 9
  • The authors make their code, data, and prompts available to support further research on iterative refinement.The resources are available at selfrefine.info.Source: §7 · p. 10

A Evaluation Tasks

  • Table 4 outlines evaluation tasks, associated datasets and sizes, and examples tracing one refinement iteration from input through output, feedback, and revised output.Few-shot prompts for FEEDBACK and REFINE are provided in Appendix S.Source: Table 4 · p. 14, A Evaluation Tasks · p. 14

B Broader Related Work

  • Compared with Reflexion, SELF-REFINE provides structured, multidimensional feedback and scores rather than free-form reflection for ReAct-based planning.The authors argue that this feedback is more actionable and supports tasks without step-by-step planning, including open-ended dialogue generation.Source: B Broader Related Work · p. 15
  • 55.7% accuracy for SELF-REFINE versus 45.9% for Self-Correction (Welleck et al., 2022) represents a 9.8-percentage-point improvement on GSM8K.The comparison used the same GPT-3 base model and benchmark settings.Source: B Broader Related Work · p. 15
  • Unlike Self-Correction (Welleck et al., 2022), SELF-REFINE generates explicit feedback and uses instructions and few-shot prompting instead of training task-specific refiners.The authors report that generating explicit feedback produces significantly better refined outputs.Source: B Broader Related Work · p. 15
  • Unlike scalar-reward reinforcement learning, SELF-REFINE makes feedback available on intermediate generations without updating model parameters.Table 5 categorizes related approaches as reinforcement-learning, trained-corrector, and few-shot-corrector methods.Source: B Broader Related Work · p. 15, Table 5 · p. 15

C Human Evaluation … F Comparison of SELF-REFINE with State-of-the-art of Few-Shot Learning Models and Fine-Tuned Baselines

These sections describe blind human evaluation, rationale-based GPT-4 evaluation, and model terminology, alongside comparisons of SELF-REFINE with few-shot and fine-tuned baselines on GSM and PIE.

C Human Evaluation

  • 150 examples per dataset undergo blind A/B evaluation by author judges, who select the output better aligned with the task instruction.Judges receive the input, instruction, and baseline and SELF-REFINE outputs without method identities. Preference rate measures how often SELF-REFINE is selected; relative improvement is the percentage increase in preference rate.Source: Table 6 · p. 16, C Human Evaluation · p. 16

D GPT-4 Evaluation

  • GPT-4 evaluates sentiment reversal, acronym generation, and dialogue responses through structured prompts that request a rationale for its decision.The prompts encourage deliberation before the evaluation decision.Source: D GPT-4 Evaluation · p. 17
  • Acronym evaluation considers pronunciation, spelling, relation to the title, and positive connotation, allowing either candidate, equal quality, or neither to be selected.The requested output includes a short explanation followed by the selection.Source: D GPT-4 Evaluation · p. 17

E Model Key

F Comparison of SELF-REFINE with State-of-the-art of Few-Shot Learning Models and Fine-Tuned Baselines

G Evaluation of Vicuna-13b · H Additional Analysis

Vicuna-13b followed initialization prompts but struggled with feedback and refinement using the study’s shared prompts. Using ChatGPT for feedback and refinement improved its Math Reasoning performance from 24.18% to 40.5%.

G Evaluation of Vicuna-13b

  • Vicuna-13b consistently followed initialization prompts but struggled with feedback and refinement prompts, often producing assistant-like responses.Source: G Evaluation of Vicuna-13b · p. 20
  • Vicuna-13b is a version of LLaMA-13b (Touvron et al., 2023) fine-tuned on conversations sourced from the web.Source: G Evaluation of Vicuna-13b · p. 20
  • Vicuna-13b received the same prompts as other evaluated models; its limited performance suggests that more extensive prompt engineering may be necessary.Source: G Evaluation of Vicuna-13b · p. 20
  • 40.5% on Math Reasoning was achieved with Vicuna-13b initialization and ChatGPT feedback and refinement, versus 24.18% for Vicuna-13b alone.This mixed-refinement setup uses external model assistance, unlike SELF-REFINE’s focus on improving a model without external help.Source: G Evaluation of Vicuna-13b · p. 20
  • In sentiment transfer, Vicuna-13b generated a good initial rewrite but then produced empty feedback, causing an error.Source: G Evaluation of Vicuna-13b · p. 21
  • Vicuna-13b’s subsequent sentiment-transfer feedback was unhelpful, and its rewritten review copied from the prompt.A later output positively recommended a hotel rather than making the original food review negative.Source: G Evaluation of Vicuna-13b · p. 21

H.1 Using Oracle Feedback … L.1 Method

Self-Refine’s gains are often statistically significant, although dialogue errors and fluctuating acronym quality reveal limitations of iterative feedback. Extensions include correctness-guided refinement, website layouts, new generation tasks, and critique-driven code readability improvements.

H.1 Using Oracle Feedback

  • 4.8% for GPT-3 and 0.7% for GPT-4 are Math Reasoning improvements from oracle feedback, which refines only incorrect answers, following Welleck et al. (2022).Correctness information determines whether the current answer proceeds to the REFINE stage.Source: §H.1 · p. 22
  • Acronym quality can fluctuate as improvements on one aspect harm others; numerical aspect scores support selecting the highest-scoring output across iterations.Math Reasoning and Sentiment Reversal instead show monotonically increasing output quality, although the same output-selection strategy is possible.Source: §H.1 · p. 22, Table 10 · p. 22
  • SELF-REFINE is robust to incorrect or generic feedback 60% of the time, despite observed feedback and refinement errors, including ignoring feedback and introducing new problems during the refinement stage.Reported feedback errors include incorrect feedback (25%), generic feedback (30%), and incorrect scoring (10%). Refinement errors include not being robust (10%), ignoring feedback (25%), and introducing a new problem (20%).Source: §H.1 · p. 24

I Beyond Benchmarks

  • Self-Refine extends to website layouts, where ChatGPT critiques initial designs and implements actionable changes to styling, content, and layout.Examples include an ice-cream page with larger headings and additional content, and a photosynthesis page with 18px text and a header divider.Source: I Beyond Benchmarks · p. 25, Figure 8 · p. 26, Figure 10 · p. 27

J Statistical Confidence Intervals

  • 4 of 7 ChatGPT datasets and 3 of 7 GPT-3.5 datasets show statistically significant gains; nearly all GPT-4 gains are significant.Table 13 reports Wilson confidence intervals, but its caption specifies 95% confidence while the accompanying prose specifies 99%.Source: Table 13 · p. 28, J Statistical Confidence Intervals · p. 28

K New Tasks

  • 20–30 concepts must appear in coherent CommonGen-Hard sentences, versus 3–5 related concepts in CommonGen (Lin et al., 2020).Acronym Generation tests tradeoffs among length, pronunciation, and relevance. Its dataset was sourced from 250 acronyms and manually pruned to remove offensive or uninformative entries.Source: K New Tasks · p. 29

L Code Readability

  • Code readability is targeted separately from correctness because it affects usability, upgradability, and maintenance; natural-language LLM critiques guide another LLM’s revisions.Readability improvements concern code quality beyond execution results.Source: L Code Readability · p. 29

L.1 Method

  • Readability refinement skips initialization: an LLM produces free-text critiques of existing code, and a code-generator LLM applies them repeatedly for N iterations.FEEDBACK freely chooses readability enhancements; REFINE receives the current code, its critique, and an explicit instruction to fix the code using that feedback.Source: §L.1 · p. 29

L.2 Experiments … M.1 Modules

For code readability, SELF-REFINE improves three heuristic metrics over iterations, with sampled-critique final outputs surpassing human rewrites on all three. For dialogue, it generates and iteratively refines responses using feedback on ten qualitative aspects.

L.2 Experiments

  • 300 randomly selected CodeNet examples test SELF-REFINE, while human annotators rewrite a 60-example subset for comparison.Both critique and editor use InstructGPT (textdavinci-003) for five refinement iterations. Critique decoding uses T = 0.0 or T = 0.7; code editing always uses T = 0.0.Source: §L.2 · p. 29
  • T = 0.7 yields final SELF-REFINE outputs with more meaningful variables, more function units, and slightly more comments than human rewrites on average.At T = 0.0, final outputs have fewer meaningful variables and fewer comments per line than human rewrites, but even more function units.Source: Table 14 · p. 30, §L.2 · p. 30
  • Readability evaluation measures the ratio of meaningful variable names, comments per code line, and the number of function units.A few-shot-prompted language model extracts distinct variables and identifies those with meaningful names; function units reflect code modularization.Source: §L.2 · p. 30
  • All three readability metrics increase across refinement iterations, with higher-temperature critiques favoring variable naming and comments, while greedy critiques favor modularization.Iteration 0 measures the original CodeNet input; subsequent measurements average each metric across test examples.Source: §L.2 · p. 30, Figure 11 · p. 30

Example

  • The code example illustrates refinement progressing from indentation changes to rewriting densely composed code into multiple lines of atomic operations.Figure 12 shows these changes across SELF-REFINE iterations.Source: Figure 12 · p. 31

M Dialogue Response Generation

  • For open-domain dialogue, SELF-REFINE applies automatically generated feedback and iterative refinement to improve responses amid challenges of coherence and engagement across diverse topics.The task requires human-like responses despite its open-ended nature.Source: M Dialogue Response Generation · pp. 30–31

M.1 Modules

  • The dialogue initializer generates a response from conversation context; the iterate module refines it using context, prior responses, and feedback.Refinement aims to make the output better match the feedback.Source: §M.1 · pp. 31–32
  • Dialogue feedback evaluates ten qualitative aspects using six in-context examples and often explains why a response receives a low score.The aspects are relevance, informativeness, interestingness, consistency, helpfulness, engagement, specificity, safety, user understanding, and fluency. The discussion references Mehri and Eskenazi (2020) for a more thorough review.Source: §M.1 · p. 31

M.2 Setup and Experiments … P Sentiment Reversal

SELF-REFINE outperforms direct generation in automatic and human dialogue evaluation. The framework also refines code optimizations, Python-based math solutions, and whole-review sentiment reversals through feedback-driven iteration.

M.2 Setup and Experiments

  • Up to 3 iterations refine dialogue responses through few-shot INIT, FEEDBACK, and ITERATE prompts using text-davinci-003.The modules share three in-context examples, with feedback demonstrations providing scores and explanations. The final response maximizes the FEEDBACK model’s total score across iterations, excluding the initial response.Source: §M.2 · p. 32
  • SELF-REFINE beats INIT by a wide margin in both automatic and human dialogue evaluation.Manual analysis finds its responses more engaging, interesting, and generally more elaborate than INIT responses.Source: Table 15 · p. 33, §M.2 · p. 33
  • 342 test instances support automatic dialogue evaluation; 100 randomly selected instances support human comparisons of SELF-REFINE against INIT, with ties allowed.Evaluation uses FED (Mehri and Eskenazi, 2020) and compares responses across ten qualities; automatic win rates use zero-shot text-davinci-003 judgments. The authors prioritize human evaluation because automated metrics may inaccurately assess generation quality.Source: §M.2 · p. 33

N Code Optimization

  • SELF-REFINE improves PIE-generated code optimizations by producing natural-language feedback on possible improvements and passing that feedback to REFINE.PIE (Madaan et al., 2023) targets algorithmic modifications that improve the runtime of functionally correct programs.Source: N Code Optimization · p. 33

O Math Reasoning

P Sentiment Reversal

  • Sentiment reversal rewrites entire reviews to flip their sentiment from positive to negative or vice versa.Few-shot INIT, FEEDBACK, and ITERATE prompts generate and revise outputs using feedback. Refinement continues until the target sentiment is reached, with a maximum of k = 4 iterations.Source: P Sentiment Reversal · p. 35

P.1 Details … S Prompts

These sections report sentiment evaluation and feedback ablations, describe acronym generation and CommonGen-Hard, and document task-specific generation, feedback, and refinement prompts.

P.1 Details

  • 100% positive-target classification accuracy was achieved by both GPT-3.5 (text-davinci-003) and SELF-REFINE; negative-target scores were 92% and 93.6%, respectively.An off-the-shelf Vader classifier judged whether transferred reviews matched the target sentiment.Source: §P.1 · p. 35
  • 85% sentiment and 80.09% dramatic-response preference rates with informative feedback fell to 73% and 58.92%, respectively, when feedback merely stated something was wrong.Informative feedback identifies specific words and phrases to alter, rather than merely flagging inadequate sentiment. Automated and human evaluations assess desired-sentiment alignment and dramatic language.Source: §P.1 · pp. 35–36

Q Acronym Generation

  • 250 acronyms remained after manual filtering of potentially offensive or uninformative entries, providing a testbed for iterative acronym refinement.Acronym generation involves tradeoffs among length, pronunciation, and relevance to the original phrase.Source: Q Acronym Generation · p. 36
  • Acronym feedback evaluates pronunciation, spelling, relation to the title, positive connotation, and familiarity, with explicit reasoning preceding each criterion’s score.Few-shot demonstrations guide these partly subjective judgments, while human evaluation assesses final acronym quality.Source: Q Acronym Generation · p. 36

R Constrained Generation

  • 20–30 concepts must appear in coherent, grammatical sentences in CommonGen-Hard, compared with 3–5 related concepts in the original CommonGen task.SELF-REFINE applies multidimensional feedback and iterative revision to address output quality, coherence, and sensibility on this harder constrained-generation task.Source: R Constrained Generation · p. 37

S Prompts

  • Generation prompts use input–output pairs, feedback prompts add feedback, and refinement prompts additionally include improved outputs.Figures 16–35 document prompts across seven tasks, with author-created demonstrations covering sentiment conversions, dialogue rubrics, acronym scores, and missing-concept or incoherence errors. Code optimization uses explanations from Madaan et al. (2023); math uses PaL (Gao et al., 2022) prompts and manually corrected CODEX failures.Source: S Prompts · pp. 37–38

Related papers

Loading 2303.17651v2…