PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Large Language Models Be an Alternative to Human Evaluations?

Cheng-Han Chiang, Hung-yi Lee

arXiv:2305.01937v1cs.CLcs.HC

TL;DR

Human evaluation is important but unstable and difficult to reproduce, raising the question of whether LLMs can provide an alternative. The paper applies LLM evaluation using the same instructions, samples, and questions as human evaluation across two NLP tasks, finding similar results and limited variation across tested settings.

  • Problem

    Human evaluation is important for assessing NLP text quality, but its results are unstable and difficult to reproduce, complicating comparisons among systems.

  • Method

    The paper gives LLMs the same instructions, samples, and questions used in human evaluation and parses their generated responses as ratings across two NLP tasks.

  • Results

    LLM evaluation produces results similar to expert human evaluation, with only slight variation across tested task instructions and sampling hyperparameters.

  • Takeaways & Limitations

    LLM evaluation has potential as an alternative option for evaluating NLP systems and algorithms on the two tasks studied.

  • Takeaways & Limitations

    LLMs may produce plausible nonsense, and their responses can be sensitive to prompt changes not covered by the tested instruction modifications.

Abstract

from arXiv · show

Human evaluation is indispensable and inevitable for assessing the quality of texts generated by machine learning models or written by humans. However, human evaluation is very difficult to reproduce and its quality is notoriously unstable, hindering fair comparisons among different natural language processing (NLP) models and algorithms. Recently, large language models (LLMs) have demonstrated exceptional performance on unseen tasks when only the task instructions are provided. In this paper, we explore if such an ability of the LLMs can be used as an alternative to human evaluation. We present the LLMs with the exact same instructions, samples to be evaluated, and questions used to conduct human evaluation, and then ask the LLMs to generate responses to those questions; we dub this LLM evaluation. We use human evaluation and LLM evaluation to evaluate the texts in two NLP tasks: open-ended story generation and adversarial attacks. We show that the result of LLM evaluation is consistent with the results obtained by expert human evaluation: the texts rated higher by human experts are also rated higher by the LLMs. We also find that the results of LLM evaluation are stable over different formatting of the task instructions and the sampling algorithm used to generate the answer. We are the first to show the potential of using LLMs to assess the quality of texts and discuss the limitations and ethical considerations of LLM evaluation.

1 Introduction

Human evaluation is important but unstable and difficult to reproduce, motivating LLM evaluation as a possible alternative. The paper tests this approach across two NLP tasks and reports similar results to expert human evaluation with limited sensitivity to instructions and sampling settings.

  • Motivation: Human evaluation is important for assessing NLP systems, but its quality is unstable and difficult to reproduce.Low-quality workforces can affect results, and repeating an evaluation with the same workers may produce different outcomes.
  • Approach: LLM evaluation feeds an LLM the same instructions, samples, and questions used in human evaluation, then uses its generated responses as ratings.The process is designed to test whether LLMs’ instruction-following abilities can perform the evaluation task.
  • Experiments: The paper evaluates text quality with LLMs in open-ended story generation and adversarial attacks.These two tasks provide distinct settings for testing whether LLM evaluation yields meaningful results.
  • Findings: LLM evaluation produces results similar to expert human evaluation.The paper presents this finding as evidence for the effectiveness and potential of LLM evaluation.
  • Findings: LLM evaluation results vary only slightly across task-instruction formats and sampling-algorithm hyperparameters.This stability is reported for the instruction and generation choices examined in the paper.
  • Discussion: The paper discusses both the advantages and disadvantages of LLM evaluation, including ethical considerations.These considerations are treated as part of the evaluation of LLMs as an alternative to human assessment.

2 LLM Evaluation

LLM evaluation uses large language models’ instruction-following ability to rate text quality. It reproduces the structure of human evaluation while retaining task-specific instructions and questions.

  • LLM background: Large language models are pretrained on large text corpora and can perform unseen tasks when given only task instructions.This ability is called zero-shot in-context learning and motivates using LLMs for evaluation.
  • Procedure: LLM evaluation presents the model with task instructions, a sample, and a question asking it to rate quality on a 5-point Likert scale.The model generates output sentences, which are parsed to obtain the rating.
  • Procedure: The instructions and questions used for LLM evaluation are inherited from prior human evaluations rather than tailored for LLMs.Different tasks retain their own instruction and question sets.
  • Comparison: Human and LLM evaluations use similarly formatted instructions, samples, and questions, but humans select predefined scores while LLMs generate answers.The comparison differs mainly in how the response is produced: selection from scores 1–5 versus generated output.

3 Example Task 1: Open-Ended Story Generation

The paper evaluates LLM-based assessment on open-ended story generation, comparing human-written and GPT-2-generated stories across multiple quality attributes. Results vary substantially by evaluator model, while text-davinci-003 and ChatGPT align with expert preferences and remain comparatively stable under instruction and sampling changes.

  • 3.1 Task Introduction: The experiment rates human-written and GPT-2-generated stories from the same 200 prompts using LLM and expert human evaluation.GPT-2 medium generates the model-written stories with nucleus sampling, while the human-written stories come from the corresponding WritingPrompts examples.
  • 3.3 Experiment Results: Expert teachers rate human-written stories higher than GPT-2-generated stories on all four evaluated attributes.Expert agreement is lower for GPT-2-generated texts and for likability.
  • 3.3 Experiment Results: T0 and text-curie-001 do not consistently distinguish human-written from GPT-2-generated stories, and T0 produces low agreement across sampled answers.For T0, the human-versus-model differences are not statistically significant for grammaticality and relevance.
  • 3.3 Experiment Results: text-davinci-003 rates human-written stories significantly higher on all four attributes, matching the preference found by expert teachers.Its ratings also show positive, statistically significant Kendall’s τ correlations with teacher ratings across both story types and all attributes.
  • 3.3 Experiment Results: ChatGPT also prefers human-written stories and can provide detailed explanations, but its ratings reflect evaluator-specific standards and safety-related tendencies.ChatGPT may assign lower likability to violent or impolite stories and can refuse to rate likability before regeneration.
  • 3.3.2 Variance due to Different Instructions: Instruction changes alter ratings by less than 0.1 for grammaticality and cohesiveness and by up to 0.25 for the other attributes without changing relative ranking.Changing the sampling temperature can slightly lower average ratings, but the preference for human-written stories remains consistent.

4 Example Task 2: Adversarial Attack

The adversarial-attack evaluation compares LLM and English-teacher ratings of fluency and meaning preservation for benign and adversarial news titles. ChatGPT distinguishes attack quality broadly like human experts, while rating adversarial samples less harshly.

  • 4.2 Experiment Setup: Three synonym-substitution attacks—Textfooler, PWWS, and BAE—are evaluated on benign and adversarial AG-News titles using fluency and meaning-preservation ratings.The attacks target a BERT-base-uncased classifier fine-tuned on AG-News, with 100 benign–adversarial pairs sampled for each attack.
  • 4.3 Experiment Result: English teachers rate adversarial samples much lower than benign samples in both fluency and meaning preservation.
  • 4.3 Experiment Result: 5.00 is ChatGPT’s score for meaning preservation when two identical benign titles are compared, passing the task-understanding sanity check.ChatGPT explains that identical titles merit a score of 5, indicating that it understands the meaning-preservation question.
  • 4.3 Experiment Result: ChatGPT rates adversarial samples higher than English teachers but still significantly lower than benign samples.Text-davinci-003 produces similar results, suggesting that the difference is not unique to ChatGPT.
  • 4.3 Experiment Result: ChatGPT agrees with teachers that BAE samples are better than Textfooler and PWWS samples, and its PWWS-over-Textfooler preference matches the majority of teachers.Two of three teachers rate PWWS higher than Textfooler, although the aggregate expert evaluation does not show that difference.

5 Discussions

The discussion presents LLM evaluation as a potentially reproducible, cheaper, faster alternative for judging NLP outputs, while retaining human evaluation for collecting feedback before deployment. It also identifies factual, behavioral, ethical, provider-update, and content-policy boundaries.

  • 5 Discussions: LLM evaluation is proposed as an alternative to human evaluation across open-ended story generation and adversarial attacks.The paper reports that the best InstructGPT and ChatGPT models can rate text quality like human experts on these two tasks.
  • 5 Discussions: Specifying the LLM, random seed, and generation hyperparameters makes LLM evaluation more likely to be reproduced than human evaluation.Provider updates can still make evaluations unreproducible if an older model becomes inaccessible.
  • 5 Discussions: LLM evaluation treats each sample independently, whereas human ratings can be influenced by previously seen examples and require calibration.English teachers reported needing time to calibrate their ratings after evaluating 400 stories.
  • 5 Discussions: US$140 and over a week are reported for human-rating costs and collection time, versus less than US$5 and a few hours for InstructGPT.
  • 5 Discussions: LLM evaluation can reduce human exposure to objectionable material, but LLMs may have factual errors, behavioral biases, harmful outputs, and ethical risks.The discussion notes that these limitations and potential harms also apply to human evaluation through evaluator bias.
  • 5 Discussions: Human evaluation remains best suited for collecting human feedback before deploying an NLP system in real-world applications.

Limitations

LLM evaluation inherits limitations from pretrained language models, including prompt sensitivity, potentially nonsensical outputs, lack of emotion, and inability to process visual cues.

  • Model and prompt limitations: LLM responses can sound plausible while being nonsensical, and results may vary with task-instruction modifications beyond those tested.The authors found no significant overall differences across tested instructions but do not guarantee robustness to all modifications.
  • Emotion-related evaluation: LLMs may lack emotions, challenging evaluations of emotion-related tasks such as story likability.ChatGPT often states that it does not have emotions when asked to rate story likability.
  • Visual-cue limitation: Unlike human evaluators, LLMs cannot process visual cues such as special fonts or text styles in task instructions.This limits their access to formatting information that may influence how humans approach evaluation tasks.

Ethics Statement

The authors frame LLM evaluation as an alternative intended to improve reproducibility, not as a complete replacement for human evaluation. They retain human evaluation and report ethical compliance procedures.

  • Role of human evaluation: LLM evaluation is proposed as an alternative to human evaluation for enhancing reproducibility, not as a recommendation to eliminate human evaluation.The authors recommend using both methods because human feedback remains essential for systems intended for human users.
  • Role of human evaluation: Human evaluation remains essential because NLP systems ultimately serve human users, making feedback from human evaluators important.The authors explicitly recommend using human and LLM evaluation in conjunction.
  • Experimental ethics: The experiments follow the ACL Code of Ethics and document evaluator instructions, interface screenshots, recruitment, and payment.The paper also states that evaluators were informed about the task and how their responses would be used.
  • Experimental ethics: The study reports following OpenAI’s usage policy when using InstructGPT and ChatGPT models.The models and datasets were used according to their intended usage.

A Modification Based on the Reviews

The revised paper adds analyses, supplementary experiments, and discussion in response to reviewer suggestions, while also documenting training-data and model-training details.

  • Reviewer-driven additions: The authors added an analysis of agreement between LLM and human evaluators on individual story ratings.This addition appears in Section 3.3.1.
  • Reviewer-driven additions: The authors refined Section 5 and added relevant references.These changes address reviewer suggestions concerning the paper’s wording and discussion.
  • Reviewer-driven additions: The authors added Table 6 to analyze correlations between human evaluators.This supplements the paper’s treatment of evaluator agreement.

B.3 Data Post-processing

The evaluation data are post-processed to align model-generated and human-written stories, while Google Forms collect teacher ratings for stories and adversarial-attack tasks.

  • Story data processing: 200 test prompts generate 200 model-written stories using nucleus sampling with p = 0.9.Stories are manually truncated to fewer than 150 words and made to end with a full sentence.
  • Story data processing: The corresponding 200 human-written stories use the same prompts and are likewise truncated to fewer than 150 words and complete sentences.Some tokenization artifacts are manually removed from human-written stories.
  • Human evaluation procedure: Certified English teachers rate 200 GPT-2-generated and 200 human-written stories for US$140, with a reported hourly wage of at least US$28.Each teacher rates the two sets in sequence without being told who wrote the stories beforehand.
  • Story data processing: The 150-word truncation matches the mean length of the model-generated stories.This provides the stated rationale for the truncation threshold.
  • Human evaluation procedure: The study keeps GPT-2-generated and human-written stories separate during rating because prior work found different evaluator preferences under mixed or separate presentation.The paper follows the prior work’s settings rather than mixing the story types in its main evaluation.
  • Adversarial-attack evaluation: Certified teachers rate 100 news titles for adversarial-attack quality for US$35, completing the task in less than one hour.This evaluation uses a separate teacher-rating setup from the story experiment.
  • Evaluation interfaces: Google Forms collect story responses, with each form page containing one story and the interface split across Figures 2 and 3 for resolution.The two figures show different parts of the same story-evaluation page.
  • Evaluation interfaces: The adversarial fluency form mixes equal numbers of benign, TextFooler-attacked, PWWS-attacked, and BAE-attacked news titles.A separate form compares meaning preservation before and after attacks, highlighting differences in boldface.

C.3.1 How English Teachers Rate the Stories

Three English teachers evaluated stories using grammaticality, coherence, likability, and relevance, with standards based on different textual and narrative properties. They spent 6–10 hours rating 400 stories and calibrated their ratings after roughly five to ten stories.

  • Teachers evaluated four attributes: grammaticality, coherence, likability, and relevance.The evaluation questions asked teachers to explain standards for high and low ratings, including whether personal preference affected likability.
  • 6 to 10 hours were required for teachers to rate 400 stories.
  • Coherence was judged by whether sentences formed a logical narrative sequence and whether the story made sense.Teachers described this as a holistic evaluation of the whole story.
  • Likability incorporated perceived enjoyment, stylistic flair, humor, engaging plotting, and characterization, while teachers tried to remain fair and objective.
  • Relevance was assessed by checking whether the story was based on its prompt.
  • Teachers calibrated ratings after five to ten stories, generally without changing ratings on the other attributes after reading prompts.One teacher changed ratings on the other three attributes after seeing the prompt for one story.

C.3.2 Teachers’ Comments on ChatGPT’s Rating

Teachers reviewed ChatGPT’s ratings and explanations for sampled human- and GPT-2-generated stories without being told that an AI produced them. They generally found the evaluations reasonable, though disagreements concentrated on holistic attributes, and ChatGPT’s ratings were affected by content refusals.

  • Teachers reviewed ChatGPT’s ratings and explanations while being told another teacher had produced them.The sample included highly- and lowly-rated human-written and GPT-2-generated stories.
  • Teachers generally considered ChatGPT’s ratings and explanations reasonable.Their main disagreements concerned likability and cohesiveness, which they viewed as more holistic attributes.
  • ChatGPT refused to answer questions about 2 stories and 3 news titles containing discriminatory, sexual, or violent content.ChatGPT results were calculated without those samples.
  • T0 and InstructGPT were queried separately on the four story attributes using prompts asking for 1–5 ratings.

D.2.3 Adversarial Attack Quality Evaluation

The adversarial-attack evaluation asked LLMs to judge fluency and meaning preservation using separate prompts. Fluency was assessed for individual benign or attacked titles, while meaning preservation compared title pairs before and after attack.

  • LLMs evaluated fluency and meaning preservation independently, with one question per ChatGPT conversation.Generation parameters matched the defaults in Section 3.2.
  • Fluency prompts asked how natural and fluent a single benign or adversarial news title was on a 1–5 scale.
  • Meaning-preservation prompts compared benign and adversarial titles and asked whether their semantics remained preserved on a 1–5 agreement scale.
  • The benign and adversarial title placeholders were filled with the news title before and after an adversarial attack.

E Experiment Details on Adversarial Attacks

The adversarial samples came from attacks against an AG-News BERT classifier, and the experiments compared human ratings with LLM evaluations. Across supplementary results, text-davinci-003 showed the same broad pattern as ChatGPT, while teacher ratings varied by attack and evaluator.

  • The dataset used Textfooler, PWWS, and BAE attacks against a bert-base-uncased AG-News classifier.The samples came from Yoo et al. (2022).
  • text-davinci-003 rated adversarial samples higher than humans but significantly lower than benign samples.This pattern was similar to ChatGPT’s adversarial-sample evaluation.
  • Table 11 reports text-davinci-003 and human results for benign and adversarial fluency and meaning preservation.
  • Table 12 reports ratings for the three adversarial attacks from teachers T1, T2, and T3.
  • One teacher rated Textfooler fluency significantly higher than PWWS, while two other teachers did not.
Loading 2305.01937v1…