PaperlayerView exact revision on arXiv

Source-linked AI summary

GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models

Archiki Prasad, Peter Hase, Xiang Zhou, Mohit Bansal

arXiv:2203.07281v2cs.CLcs.AIcs.LG

TL;DR

Existing instruction improvement relies on labor-intensive manual rewriting or computationally demanding gradient-based tuning that may not work with API models. GRIPS instead performs gradient-free, edit-based search over instructional prompts, improving accuracy across classification tasks and outperforming manual rewriting and example search under matched budgets. Its scope is limited because the current edits cannot add substantially new information, may be less suitable for generation tasks, and the largest InstructGPT model was not tested.

  • Problem

    Manual instruction rewriting requires subjective effort, while gradient-based tuning is computationally demanding and infeasible for models accessed only through APIs.

  • Method

    GRIPS iteratively searches instructional prompts with local, phrase-level text edits and model-performance scoring, without requiring gradients.

  • Results

    Across eight NATURAL-INSTRUCTIONS classification tasks, GRIPS improves GPT-model accuracy by 2.36–9.36 percentage points and outperforms manual rewriting and exemplar-prompt search.

  • Takeaways & Limitations

    GRIPS improves Instruction-Only and Instruction + Examples prompts, remains applicable to API-based models, and can improve instructions using as few as 20 scoring examples.

  • Takeaways & Limitations

    Current edits cannot add substantially new information, may be less effective for purely generation-based tasks, and the largest InstructGPT davinci model was not tested because of cost constraints.

Abstract

from arXiv · show

Providing natural language instructions in prompts is a useful new paradigm for improving task performance of large language models in a zero-shot setting. Recent work has aimed to improve such prompts via manual rewriting or gradient-based tuning. However, manual rewriting is time-consuming and requires subjective interpretation, while gradient-based tuning can be extremely computationally demanding for large models and may not be feasible for API-based models. In this work, we introduce Gradient-free Instructional Prompt Search (GrIPS), a gradient-free, edit-based search approach for improving task instructions for large language models. GrIPS takes in instructions designed for humans and automatically returns an improved, edited prompt, while allowing for API-based tuning. With InstructGPT models, GrIPS improves the average task performance by up to 4.30 percentage points on eight classification tasks from the Natural Instructions dataset (with similar improvements for OPT, BLOOM, and FLAN-T5). We see improvements for both instruction-only prompts and instruction + k-shot examples prompts. Notably, GrIPS outperforms manual rewriting and purely example-based prompts while controlling for the available compute and data budget. Further, performance of GrIPS is comparable to select gradient-based tuning approaches. Qualitatively, we show our edits can simplify instructions and at times make them incoherent but nonetheless improve accuracy. Our code is available at: https://github.com/archiki/GrIPS

1 Introduction

Instructional prompts can improve language-model task performance without examples, but improving them manually is laborious and gradient-based tuning is costly or unavailable for API models. GRIPS addresses these constraints with automated, gradient-free instruction editing and improves performance across tasks and models.

  • Manual instruction rewriting requires substantial effort and subjective interpretation, while optimized prompts may sometimes be semantically confusing to humans.
  • Gradient-based prompt tuning can be prohibitively expensive, infeasible for API-only models, and difficult to map continuous representations back to vocabulary tokens.
  • GRIPS performs iterative, local, edit-based, gradient-free search to improve instructional prompts for arbitrary, including API-based, language models.
  • 2.36–9.36 percentage points: GRIPS improves average GPT-2 XL and InstructGPT accuracy across eight NATURAL-INSTRUCTIONS classification tasks.
  • GRIPS outperforms manual rewriting and exemplar-prompt search under matched data and computational budgets, while also supporting prompts that combine instructions and examples.

2 Related Work

Prior prompt-improvement research spans exemplar prompts, instructional prompts, continuous prompt tuning, and prompt search. GRIPS differs by editing the content of descriptive task instructions rather than primarily changing templates, triggers, or example selection.

  • Exemplar prompting primarily uses input-output examples, whereas instructional prompting provides natural-language task descriptions that can work without examples.
  • Manual rewriting has been proposed to improve instructional prompts, but GRIPS automates instruction improvement instead of relying on human rewriting guidelines.
  • Gradient-based prompt tuning optimizes continuous vector tokens, but some gradient-free alternatives still cannot modify text for API-only models.
  • Prompt-search methods optimize prompt text through example selection, ordering, templates, or trigger words, while GRIPS edits task-instruction content.
  • GRIPS uses text-based mutations of parent instruction candidates, making its search related to genetic algorithms.

3 Methodology

GRIPS searches instructional prompts by generating phrase-level edits, scoring candidates on a held-out score set, and iteratively retaining better instructions. It supports greedy or beam search and multiple prompt modes, but its edits mainly rephrase, remove, rearrange, or restore existing information.

  • Prompt modes: GRIPS evaluates Instruction-Only and Instruction + Examples prompts by concatenating instructions, examples, and test instances in the selected arrangement.
  • Scoring: The search uses a score set of examples separate from testing, with balanced accuracy and prediction entropy guiding candidate selection.
  • Search: Each iteration generates m candidates through randomly selected phrase-level edits, scores them on S, and replaces the base instruction only when a candidate scores higher.
  • Search: Beam search retains the top-B candidates across iterations, increasing model evaluations by approximately B-fold while enabling more exhaustive search.
  • Phrase splitting: A constituency parser creates phrase-level splits that preserve instruction structure while allowing flexible edits.
  • Edit operations: GRIPS uses delete, swap, paraphrase, and addition operations to simplify, rephrase, restructure, or restore instruction details.

4 Experimental Setup

The experiments evaluate GRIPS on eight binary classification tasks from NATURAL-INSTRUCTIONS using large GPT-family models and controlled search settings. The setup includes cost and model-coverage constraints, including omission of the largest InstructGPT engine.

  • Dataset: The benchmark contains eight diverse binary classification tasks sampled from NATURAL-INSTRUCTIONS because of cost and API quota constraints.
  • Test sets: Main results use 300 random test examples per task, while other analyses and ablations use 100 examples per task.
  • Models: Experiments use GPT-2 XL, InstructGPT babbage, and InstructGPT curie, all with at least 1B parameters.
  • Scope: The largest InstructGPT davinci engine was not tested because of cost constraints.
  • Cost: A GRIPS run costs O(m×n×|S|×B) model evaluations, with reported per-seed costs of $20–25 for babbage and $125–175 for curie.
  • Hyperparameters: Each search uses l = 1 edit operation per candidate, m = 5 candidates per iteration, n = 10 iterations, patience P = 2, and three seeds unless specified otherwise.

5 Results and Discussion

GRIPS improves instructional-prompt accuracy across models and settings, outperforming manual rewriting and often exemplar search while remaining competitive with gradient-based methods. Results also show sensitivity to model type, search design, data budget, and instruction semantics.

  • 5.1 Effectiveness of GRIPS: GRIPS improves average accuracy for GPT-2 XL, InstructGPT babbage, and curie by 9.36, 4.29, and 2.36 percentage points, respectively.These improvements are statistically significant at p < 0.05; curie shows greater stability despite its smaller improvement margin.
  • 5.1 Effectiveness of GRIPS: Larger InstructGPT models outperform smaller non-InstructGPT counterparts, with significant accuracy gains from GPT-2 XL to babbage and from babbage to curie.The result supports the importance of instruction-following capability and model scale in this setting.
  • 5.1 Effectiveness of GRIPS: Removing the entropy term decreases accuracy by −1.48 points, while removing del, swap, par, or add operations decreases accuracy by −2.56, −1.01, −1.14, or −1.26 points.The entropy term helps break ties toward less skewed predictions and avoid local minima, while the ablations indicate benefits from all edit operations.
  • 5.2 Comparing with Gradient-free Methods: GRIPS outperforms manual rewriting for GPT-2 XL, babbage, and curie by 5.56, 2.29, and 1.50 points, respectively.Example search remains better for GPT-2 XL, but GRIPS outperforms exemplar search for babbage and curie by 1.54 and 1.62 points.
  • 5.3 Comparing with Gradient-based Methods: Beam-search GRIPS improves performance by 2.82 points over greedy decoding and outperforms the evaluated gradient-based methods without using gradient information.Direct finetuning and adapter-tuning outperform greedy-decoding GRIPS by 2.2 and 1.4 points, respectively.
  • 5.4–5.8 Additional Analyses: GRIPS improves performance across task-specific and task-agnostic instructions, other open-source models, and Instruction + Example prompts.Reported gains reach 5.30 and 2.42 points for task-specific and task-agnostic settings, nearly 6–7 points for several models, 3.08 points for FLAN-T5, and roughly 2 points with Instruction + Example prompts.

6 Conclusion

GRIPS automatically edits task instructions through gradient-free search to improve downstream performance. It works across models and prompt formats, outperforms manual rewriting and example search, and can approach gradient-based tuning with more compute.

  • GRIPS is an automatic search algorithm that edits task instructions to improve downstream task performance.
  • GRIPS improves GPT-2 XL and InstructGPT performance for both instruction-only and instruction-plus-examples prompts.
  • GRIPS outperforms manual rewriting and exemplar search, while beam search is at least comparable to gradient-based tuning at increased compute.
  • GRIPS remains effective with task-agnostic starting instructions and score sets containing as few as 20 examples.
  • Qualitative analysis finds that GRIPS can improve 1B+-parameter InstructGPT models using semantically incoherent instructions.

Limitations

GRIPS edits and rearranges existing instruction information but cannot add substantially new pertinent content. Its effectiveness may be limited for purely generative tasks and for models that already understand instructions well.

  • GRIPS cannot add significantly new and pertinent information or sentences beyond what is initially available in the dataset.It can rephrase existing information, remove irrelevant details, and change instruction structure.
  • GRIPS may be less effective for purely generation-based tasks because accuracy is unavailable as a suitable replacement metric for its score function.
  • Prompt engineering methods may be less useful for language models with better instruction understanding.
  • The largest InstructGPT model, davinci, was not tested because of cost constraints.

Ethical Considerations

The paper frames instructions as a way to convey extrinsic information to language models and alter outputs, while evaluating instructions drawn from a quality-controlled dataset.

  • Instructions convey extrinsic information to large language models and can alter model outputs.The paper gives generating less harmful content as an example of instructed behavior.
  • GRIPS uses instructions from NATURAL-INSTRUCTIONS, whose quality control was ensured by prior dataset work.
  • For the tasks studied, the authors verify that the instructions meet their intended task-description role.

A Additional Experimental Details

The experiments evaluate GRIPS on eight NATURAL-INSTRUCTIONS classification tasks using balanced test sampling, API and open models, and controlled search settings. Classification uses label-token probabilities, while prompt templates remain fixed across searched instructions.

  • Dataset and evaluation: The evaluation uses eight diverse, relatively balanced classification tasks from NATURAL-INSTRUCTIONS.The tasks vary in label space, instruction length and nature, and label tokens.
  • Dataset and evaluation: Test sets are sampled to be as balanced as possible, and Figure 4 reports performance before and after search on 300-sample test sets.
  • Models and compute: Experiments include OpenAI API versions text-babbage-001 and text-curie-001, while GPT-2 XL is the only model requiring GPU access.
  • Models and classification: Classification predictions are obtained by taking the argmax of label-token log-probabilities from the OpenAI completion API.This differs from evaluating classification as text generation with ROUGE.
  • Models and compute: InstructGPT experiments require no GPUs, whereas all GPT-2 XL experiments use slightly more than five NVIDIA A100 GPU-hours combined.
  • Search configuration: Hyperparameter selection uses line search over edit count, candidate count, iteration count, and patience, with l = 1, m = 5, n = 10, and P = 2 selected.
  • Search configuration: The search samples all four edit operations randomly to promote candidate diversity, without separately exploring edit-operation choices.
  • Prompt construction: Prompt-template words remain unchanged so comparisons reflect searched instruction changes rather than template-level changes.

C Extensions and Variations of GRIPS

GRIPS supports greedy, beam, and simulated-annealing variants, with score-function changes and edit-operation analyses offering alternative search behaviors and trade-offs.

  • Search variants: GRIPS begins from a base instruction, samples phrase-level edits, scores candidates, and iteratively replaces the base with higher-scoring candidates.The edit set includes delete, swap, paraphrase, and add operations.
  • Search variants: Beam search retains the top-B candidates across iterations instead of keeping only the single best candidate.Each retained candidate is searched individually before the next top-B set is selected.
  • Search variants: Simulated annealing accepts some lower-scoring candidates to explore beyond local optima, with exploration probability decaying across iterations.Tmax controls exploration magnitude, while D controls its decay.
  • Search variants: Greedy search slightly outperformed simulated annealing on average, with no statistically significant performance difference.The comparison used Tmax = 10 and D = 5 for simulated annealing.
  • Score function: Replacing BalancedAccuracy with weighted cross-entropy produced 55.08% average accuracy and a +1.4-point increase, but raised search cost by approximately 1.5x.Cross-entropy differentiates candidates with small score differences and delays early stopping.
  • Edit operations: Swap, delete, and paraphrase are frequently used, while add appears in search runs for roughly 37.5% of tasks.Add is sampled only after a delete operation in the search history.

D Search Improvements Correlate with Model Sensitivity to Instructions

GRIPS improves some tasks more than others, and the paper links this variability to how sensitive models are to instruction changes.

  • Sensitivity measure: Instruction sensitivity is defined as the standard deviation of candidate-instruction scores during the first search iteration.Higher values indicate greater sensitivity to instruction changes.
  • Observed relationship: Instruction sensitivity correlates strongly with improvement for GPT-2 XL and InstructGPT babbage, with Pearson’s r > 0.7 and p < 0.05.For curie, the correlation is weaker at r = 0.51 and is not significant at p < 0.05.
  • Practical implication: The authors suggest checking task sensitivity before running the complete search as an indicator of likely effectiveness.The paper characterizes the overall relationship as moderate to strong.

E Details on Gradient-free Methods

The paper compares GRIPS with manual rewriting and exemplar-based search, examines label-space additions and instruction types, and characterizes the resulting edits.

  • Manual rewriting: Manual rewriting involved applying multiple reframing suggestions, resolving author disagreements, and spending nearly five hours of effort.The process addressed task specificity, abstract details, structure, and label information.
  • Manual rewriting: Adding a sentence that reiterates the label space can hurt InstructGPT performance but improve GPT-2 XL performance.The authors attribute this difference to how classification outputs are handled in the compared setups.
  • Example search: For InstructGPT, GRIPS on instruction-only prompts outperforms searched example-only prompts on average by margin of improvement, despite task-level variability.The variability is especially pronounced for InstructGPT curie.
  • Instruction types: Task-agnostic instructions specify the task format and possible labels without meaningful task-specific information.Tasks sharing a label space therefore use the same task-agnostic instruction even when their tasks differ.
  • Searched instructions: Searched instructions are often shorter and sometimes semantically incoherent, with 68.5% shorter than the original and 87.5% containing task-relevant label information.For some tasks, search returns the original instruction when no edited candidate improves the score.
Loading 2203.07281v2…