PaperlayerView exact revision on arXiv

Source-linked AI summary

Instruction-following Evaluation through Verbalizer Manipulation

Shiyang Li, Jun Yan, Hai Wang, Zheng Tang, Xiang Ren, Vijay Srinivasan, Hongxia Jin

arXiv:2307.10558v2cs.CL

TL;DR

Common benchmarks emphasize instructions aligned with model training, making instruction-following ability difficult to assess. The paper introduces verbalizer manipulation to vary alignment between label words and model priors, and finds that models diverge substantially on less natural instructions, with limitations persisting even under chain-of-thought prompting.

  • Problem

    Existing benchmarks mainly use common instructions aligned with training, so success on them does not necessarily demonstrate strong instruction following.

    Source: §1 · p. 1

  • Method

    Verbalizer manipulation evaluates instruction following by varying how label words align with model priors across natural, neutral, and unnatural instructions.

    Source: §1 · pp. 1–2

  • Results

    Models perform differently across verbalizers on unnatural instructions, and even strong models show fundamental limits, including large gaps under zero-shot chain-of-thought prompting.

    Source: §4.3 · p. 7, §5 · pp. 8–9

  • Takeaways & Limitations

    The findings indicate that instruction-following capabilities remain limited when instructions contradict model priors, motivating continued research on this capability.

    Source: §5 · p. 9

    The study is limited to English classification tasks and does not examine larger models such as Llama 2 70B, Bard, and Claude.

    Source: Limitations · p. 9

Abstract

from arXiv · show

While instruction-tuned models have shown remarkable success in various natural language processing tasks, accurately evaluating their ability to follow instructions remains challenging. Existing benchmarks primarily focus on common instructions that align well with what the model learned during training. However, proficiency in responding to these instructions does not necessarily imply strong ability in instruction following. In this paper, we propose a novel instruction-following evaluation protocol called verbalizer manipulation. It instructs the model to verbalize the task label with words aligning with model priors to different extents, adopting verbalizers from highly aligned (e.g., outputting ``postive'' for positive sentiment), to minimally aligned (e.g., outputting ``negative'' for positive sentiment). Verbalizer manipulation can be seamlessly integrated with any classification benchmark to examine the model's reliance on priors and its ability to override them to accurately follow the instructions. We conduct a comprehensive evaluation of four major model families across nine datasets, employing twelve sets of verbalizers for each of them. We observe that the instruction-following abilities of models, across different families and scales, are significantly distinguished by their performance on less natural verbalizers. Even the strongest GPT-4 model struggles to perform better than random guessing on the most challenging verbalizer, emphasizing the need for continued advancements to improve their instruction-following abilities.

1 Introduction

The paper proposes verbalizer manipulation to test instruction following when label words align with model priors to different degrees. Results show that performance on unnatural instructions separates models more sharply than performance on natural instructions, and scaling alone does not consistently overcome prior conflicts.

1 Introduction

  • Common benchmarks emphasize instructions aligned with training, so success on them alone does not establish strong instruction following.The paper motivates the capability as important for real-world uses such as controlling answer certainty, response length, or assigned roles.Source: §1 · p. 1
  • Verbalizer manipulation evaluates whether instruction-tuned models follow label-mapping instructions that may conflict with their priors.It creates natural, neutral, and unnatural instructions with progressively lower alignment to prior knowledge.Source: §1 · pp. 1–2
  • The study evaluates four model families across nine datasets and finds larger models often do better on natural and neutral instructions.For small models, performance gaps tend to narrow with scale and can nearly close for davinci-003, ChatGPT, and GPT-4.Source: §1 · p. 2
  • On unnatural instructions, model families diverge and verbalizer flips can leave even ChatGPT and GPT-4 near random guessing.The paper also reports that scaling patterns vary by family and verbalizer, while scaling may not be enough when instructions contradict prior knowledge.Source: §1 · p. 2
  • Zero-shot chain-of-thought helps as models scale, but three evaluated families remain below random guessing on flipped-label instructions.Only ChatGPT and GPT-4 outperform random guessing in this setting, and performance still trails results on natural instructions.Source: §1 · pp. 2–3

2 Related Work

Related work covers instruction-tuning and increasingly broad language-model evaluation, while noting that these lines do not directly test instruction following under altered label mappings. The paper positions verbalizer manipulation as a unified, low-human-effort evaluation protocol for classification tasks.

2 Related Work

  • Instruction-tuning research transforms standard NLP datasets into text-to-text tasks and studies generalization, scaling, task balance, and instruction-following data generation.The cited work includes multi-turn data efforts such as Baize and Vicuna.Source: §2 · p. 3
  • Evaluation frameworks such as MMLU, HELM, and InstructEval assess broad model capabilities, whereas this paper targets automatic instruction-following evaluation.The paper distinguishes its instruction-only setting from Min et al. (2022), which studies demonstrations with random labels.Source: §2 · p. 3
  • Prior studies examine negated prompts, label-flipping from demonstrations, or symbol tuning, while this work manipulates verbalizers to evaluate instruction following.The comparison discusses Jang et al. (2023), Wei et al. (2023b), and Wei et al. (2023a).Source: §2 · p. 4
  • Verbalizer manipulation is presented as a unified protocol applicable to classification tasks with minimal human effort, unlike manually designed counterfactual task variants.The contrast is with Wu et al. (2023).Source: §2 · p. 4

3 Experimental Setup

The setup evaluates four instruction-tuned model families on nine binary classification datasets using verbalizer mappings grouped by alignment with model priors. Experiments use multiple model sizes in an instruction-only setting and measure accuracy.

3.1 Datasets

  • The evaluation uses nine binary classification datasets spanning sentiment, emotion, inference, entailment, duplicate detection, paraphrase identification, and subjectivity.Each dataset and verbalizer uses 100 examples; the study focuses on binary classification for simplicity, though the method can also be used for multiclass tasks.Source: §3.1 · p. 4

3.2 Verbalizer Manipulation

  • Verbalizer mappings form natural, neutral, and unnatural groups, with label meanings respectively aligned, semantically unrelated, or flipped against prior knowledge.The evaluation variants comprise 2,700 examples each for natural and unnatural instructions and 5,400 for neutral instructions.Source: §3.2 · p. 4

3.3 Instruction-tuned Models

  • The study evaluates Flan-T5, GPT-Series, Vicuna, and OPT-IML across multiple model sizes, without demonstrations, using accuracy as its metric.Decoding temperature is set to 0, and predictions are parsed from decoded strings.Source: §3.3 · pp. 4–5

3.2 Verbalizer Manipulation

  • Table 1 presents the golden label names used to manipulate verbalizers across natural, neutral, and unnatural instruction groups; for SST-2, the passage lists multiple mappings for each group.Source: §3.2 · p. 4, Table 1 · p. 5

4 Experimental results

Across model families, instructions that contradict model priors remain difficult even as scale and prompting improve performance. Zero-shot chain-of-thought helps on flipped verbalizers, but results vary by task and still lag natural instructions.

4.1 Results on Instructions with Different Naturalness

  • Larger models generally improve on natural and neutral instructions, but unnatural-instruction results diverge across families without a consistent scaling trend.Flan-T5 results are U-shaped and OPT-IML results follow inverse scaling; even GPT-4’s natural-to-unnatural performance gap is 15.7%.Source: §4.1 · pp. 5–6, Figure 2 · p. 6

4.2 Results of Different Verbalizers in Natural and Unnatural Instructions

  • Models handle different verbalizers similarly in natural instructions, while flipped verbalizers produce family- and verbalizer-dependent results, with flipped golden labels hardest for the largest models.For the largest model in each family, performance ranks unnatural 0|1 above unnatural no|yes, which is above flipped golden verbalizers; ChatGPT and GPT-4 approach random guessing on the latter.Source: §4.2 · p. 6, Figure 3 · p. 7

4.4 Per Dataset Analysis

  • ChatGPT and GPT-4 perform comparably on most datasets in both natural and unnatural settings, though their relative performance differs on RTE, SUBJ, and MRPC.They are within a 10% performance gap on 6/9 datasets in each setting; GPT-4 leads ChatGPT on MRPC by 13% naturally and 53% unnaturally.Source: §4.4 · p. 7

4.3 Results Comparison between Direct and Zero-Shot Chain-of-Thought Prompting

  • Zero-shot CoT raises ChatGPT and GPT-4 above random guessing on flipped golden verbalizers, but both retain large gaps from natural-direct prompting.The gaps are 16.8% for ChatGPT and 24.3% for GPT-4; some larger Flan-T5, Vicuna, and OPT-IML models remain below random guessing.Source: §4.3 · p. 7, Figure 4 · p. 8

4.4 Per Dataset Analysis

  • On ChatGPT and GPT-4, unnatural direct prompting preserves sentiment-task performance better than natural-language-inference performance.On SST-2, ChatGPT improves 1% and GPT-4 decreases 5%; on SICK, they decrease 96% and 99%, respectively.Source: §4.4 · pp. 7–8, Figure 5 · p. 8
  • Unnatural zero-shot CoT improves ChatGPT and GPT-4 on NLI datasets but has much less effect on sentiment and can hurt ChatGPT across all three sentiment datasets.For NLI, ChatGPT can outperform GPT-4 after zero-shot CoT.Source: §4.4 · p. 8

5 Conclusion

The paper evaluates instruction-following across prior-aligned and less-aligned verbalizers, finding that model performance diverges on unnatural instructions and across verbalizers.

5 Conclusion

  • Performance diverges significantly across model families on unnatural instructions, with no clear consistent trend between families.The paper concludes that scaling may not be enough when instructions contradict prior knowledge.Source: §5 · pp. 8–9
  • The evaluation spans four model families and nine datasets, using natural, neutral, and unnatural instruction sets.The paper examines verbalizers individually within unnatural instructions.Source: §5 · p. 8
  • Models perform significantly differently across verbalizers, even with zero-shot chain-of-thought prompting.The paper identifies continuing limitations in state-of-the-art instruction-tuned models' ability to follow human instructions.Source: §5 · p. 9

Limitations

The evaluation is bounded by model availability, classification-task coverage, English-only data, and nondeterminism in the OpenAI API.

Limitations

  • Resource constraints prevented evaluation of larger models including Llama 2 70B, Bard, and Claude.The paper identifies these models as outside its assessment.Source: Limitations · p. 9
  • Evaluations cover classification tasks, while applying verbalizer manipulation to generative tasks remains future work.Source: Limitations · p. 9
  • The study is limited to English, and OpenAI API nondeterminism may produce different results for the same input.Source: Limitations · p. 9

Ethics Statement

The authors developed verbalizer mapping rules themselves rather than using human annotations, and acknowledge possible latent bias.

Ethics Statement

  • The verbalizer mapping rules were developed without human annotations.Source: Ethics Statement · p. 9
  • The authors consider latent bias in these self-devised mapping rules possible, although their initial analysis found no indicative instances.Source: Ethics Statement · p. 9

A Results Comparison between Zero-Shot and Few-Shot Prompting

The section compares zero-shot and four-shot prompting on natural and unnatural verbalizers, finding that four-shot prompting particularly benefits GPT-4 on unnatural verbalizers.

A Results Comparison between Zero-Shot and Few-Shot Prompting

B Dataset Preprocessing

The datasets use available Huggingface versions, with selected subsets and tailored train/test splits; the experiments used specified ChatGPT and GPT-4 model versions.

B Dataset Preprocessing

  • Each dataset used its available Huggingface Datasets version.Source: B Dataset Preprocessing · p. 13
  • The prompting experiments used gpt-3.5-turbo-0125 for ChatGPT and gpt-4-0613 for GPT-4.These versions were used because older OpenAI models had been deprecated.Source: B Dataset Preprocessing · p. 13
  • FP was randomly split 80/20 into in-house training and test sets because it has only a training set.FP uses its SENTENCES_ALLAGREE subset; EMOTION uses SPLIT, and only labels listed in Table 1 were retained for FP, EMOTION, SICK, and SNLI.Source: B Dataset Preprocessing · p. 13
  • SST-2, QQP, RTE, and MRPC each contributed 100 randomly sampled validation examples, while the other five datasets contributed 100 test examples each.The test examples were randomly sampled; the passage does not specify their sampling split beyond validation versus test.Source: B Dataset Preprocessing · p. 13

C Prompt Template

The prompt-template figures cover direct prompting and zero-shot chain-of-thought prompting, with verbalizers and dataset examples inserted into designated fields.

C Prompt Template

  • In the templates, red fields take verbalizers from Table 1, while blue fields take dataset input examples in text format.Source: C Prompt Template · p. 13
  • Figure 6 compares per-dataset accuracy for zero-shot and 4-shot prompting with natural golden or unnatural flipped golden verbalizers on ChatGPT and GPT-4.Source: Figure 6 · p. 14
  • Figure 7 presents verbalizer-manipulation templates for direct prompting, and Figure 8 presents templates for zero-shot chain-of-thought prompting.Source: Figure 7 · p. 14, Figure 8 · p. 15
Loading 2307.10558v2…