PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization

Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, Lidong Bing

arXiv:2305.13091v2cs.CL

TL;DR

Abstractive summarization creates an evaluation gap because many summaries can be valid, while human assessment is costly and automatic metrics have limitations. The paper tests LLM evaluators with multiple scoring and comparison methods and finds they outperform common automatic metrics but are not yet reliable replacements for human judges.

  • Problem

    Abstractive summaries have many valid formulations, and existing automatic metrics may not capture their evaluation dimensions, while human judgments are costly and slow.

    Source: §1 · p. 1

  • Method

    The study tests LLM evaluation using reason-then-score, multiple-choice scoring, and head-to-head comparisons for abstractive summaries.

    Source: §1 · p. 2

  • Results

    LLM evaluators outperform existing automatic metrics but vary in reliability across candidates and dimensions, struggle with close-performing candidates, and align less well with humans for higher-quality summaries.

    Source: §1 · p. 2

  • Takeaways & Limitations

    When using an LLM evaluator, compare RTS and MCQ correlations as a preliminary reliability check and seek further human evaluation if they disagree.

    Source: §1 · p. 2

    The small SummEval sample covers 100 summaries per system and 12 systems, leaving per-system correlations and meta-correlation potentially subject to sample bias.

    Source: Limitations · p. 10

Abstract

from arXiv · show

With the recent undeniable advancement in reasoning abilities in large language models (LLMs) like ChatGPT and GPT-4, there is a growing trend for using LLMs on various tasks. One area where LLMs can be employed is as an alternative evaluation metric for complex generative tasks, which generally demands expensive human judges to complement the traditional automatic metrics for various evaluation dimensions such as fluency and consistency. In this work, we conduct extensive analysis to investigate the stability and reliability of LLMs as automatic evaluators for abstractive summarization. We found that while ChatGPT and GPT-4 outperform the commonly used automatic metrics, they are not ready as human replacements due to significant limitations. That is, LLM evaluators rate each candidate system inconsistently and are dimension-dependent. They also struggle to compare candidates with close performance and become more unreliable with higher-quality summaries by obtaining a lower correlation with humans. In other words, with better abstractive summarization systems being introduced at a fast pace, LLMs may result in misleading and unreliable evaluations.

1 Introduction

Abstractive summaries allow many valid formulations, making simple overlap metrics less suitable and human evaluation costly and difficult to standardize. This study tests whether LLM evaluators can reliably replace human judges and proposes a check for when further human evaluation is needed.

1 Introduction

  • For abstractive summarization, countless summaries can be equally good, while human evaluation is costly, slow, and difficult to standardize.Simple n-gram overlap may work for extractive summaries, where systems select source sentences, but is less suited to open-ended abstractive summaries.Source: §1 · p. 1
  • The study evaluates LLMs as judges using Likert-scale scoring and head-to-head comparisons, including reason-then-score and multiple-choice scoring.RTS asks the LLM to reason before assigning a score; MCQ selects a score with a predetermined description as its reason.Source: §1 · pp. 1–2
  • LLM evaluators using RTS and MCQ outperform existing automatic metrics, but remain unreliable substitutes for human evaluation.They struggle with close-performing candidates, vary in alignment across candidates and dimensions, and correlate less reliably with humans as summary quality rises.Source: §1 · p. 2
  • The authors propose comparing RTS and MCQ correlations as a preliminary reliability indicator, with further human evaluation when the methods disagree.They continue to call for a better automatic metric.Source: §1 · p. 2

2 Related Work

Prior work addresses summarization’s evaluation challenges with human judgments, overlap metrics, neural metrics, and LLM evaluators. This paper extends LLM evaluation work by examining reliability across both evaluation dimensions and candidate systems.

2 Related Work

  • Abstractive summarization uses generated wording, fusion, or paraphrase, making evaluation more challenging and gold summaries less sufficient as ground truth.Extractive systems instead select salient phrases or sentences from the source.Source: §2 · p. 2
  • Human evaluation commonly uses Likert scales or pairwise comparisons, and this study tests LLM evaluators with both approaches.Human evaluators may rate summaries on discrete scales or choose the better summary among outputs from different systems.Source: §2 · p. 2
  • ROUGE and neural metrics such as BERTScore and BARTScore address automatic evaluation, but may miss complex dimensions or correlate insufficiently with human judgments.ROUGE uses lexical overlap; BERTScore compares token embeddings, while BARTScore uses the probability of a summary given its article.Source: §2 · p. 2
  • Earlier LLM evaluation studies explore different tasks and dimensions, while this work focuses additionally on reliability across evaluated systems and dimensions.Prior approaches include open-ended story evaluation, summary consistency, persona-based evaluation, and evaluation across language-generation dimensions.Source: §2 · pp. 2–3

3 LLM as a Zero-Shot Evaluator

The paper evaluates LLMs on four summarization dimensions using score-based and pairwise methods, then measures whether their alignment with human judgments varies across candidate systems. Its stability analysis relates candidate quality to the LLM’s evaluation performance.

3 LLM as a Zero-Shot Evaluator

  • The evaluator is tested on coherence, consistency, fluency, and relevance using RTS, MCQ, and head-to-head comparisons.RTS generates reasoning before a 1-to-5 score; MCQ chooses a score with dictated reasoning, while H2H selects a better summary or a tie.Source: §3 · p. 3, §3.1 · pp. 3–4

3.1 Summary Evaluation Methods

  • RTS asks the LLM to reason before scoring on the human 1-to-5 scale, whereas MCQ constrains scoring with predetermined reasoning for each score.MCQ is presented as a way to avoid misguidance from nonsensical generated reasoning.Source: §3.1 · pp. 3–4
  • H2H presents two systems’ summaries for the same article and asks the LLM to choose the better one or a tie, reversing their displayed order to reduce ID bias.The same comparison is evaluated twice with the summaries’ positions swapped.Source: §3.1 · p. 4

3.2 Stability of LLM Evaluators

  • Stability is assessed by comparing LLM-human correlations across systems and by relating each system’s quality to its LLM evaluation performance.The paper defines system quality as the average human score over 100 summaries per system and uses correlation as the LLM performance indicator.Source: §3.2 · p. 4
  • Meta-correlation summarizes how LLM evaluation performance varies with candidate quality; a substantial value indicates instability, while a value near zero is ideal.The same correlation metric is used to calculate candidate-level performance and meta-correlation.Source: §3.2 · p. 4

4 Experiments

Across the experiments, ChatGPT evaluators often correlate more strongly with humans than conventional automatic metrics, but their reliability varies across candidate systems, summary quality, and evaluation dimensions. GPT-4 improves some results without eliminating these reliability concerns.

4.1 Setups

  • The experiments use SummEval annotations for 12 CNN/DM summarization systems, comparing LLM evaluations with ROUGE, BERTScore, and BARTScore baselines.The benchmark includes four human-rating dimensions and uses average human scores as reference scores.Source: §4.1 · p. 5

4.2.1 Correct Preferences

  • Across 66 system pairs, RTS leads every dimension with an average of 58.5 correct preferences (88.6% accuracy).The 11-pair challenge set instead shows that close-performing systems are harder to distinguish; RTS accuracy falls to 63.6% (7 out of 11).Source: §4.2.1 · p. 6, Table 5 · p. 6

4.2.2 Correlations with Human

  • ChatGPT RTS and MCQ correlate more strongly with humans than many automatic metrics, with up to 0.2 gains in fluency.RTS correlates more strongly in consistency and relevance, while MCQ is relatively strong in coherence and fluency.Source: §4.2.2 · p. 6, Table 4 · p. 6

4.2.3 Per-candidate Correlations

  • RTS human-correlation spreads across candidate systems by up to 0.5 in consistency, and ChatGPT’s medians also vary across dimensions.Similar candidate dependence appears with MCQ; the variation cautions against assuming stable alignment for a new system or criterion.Source: §4.2.3 · pp. 6–7, Figure 1 · p. 7

4.2.4 Summary Quality vs Human Alignment

  • RTS and MCQ show negative meta-correlations for consistency and fluency, while ChatGPT is less human-aligned on higher-quality candidates.The evaluator has lower and less consistent correlations for candidates scoring above 4.7 than for those below 3.5.Source: §4.2.4 · p. 7, Figure 2 · p. 7, Figure 3 · p. 7

4.2.5 RTS and MCQ Scores

  • RTS scores are lower than human scores across dimensions, while MCQ scores are higher and closer except for relevance.Thus, RTS cannot replace human scores in absolute terms, despite its stronger human correlation in some comparisons.Source: §4.2.5 · p. 8, Figure 4 · p. 8

4.3 GPT-4 Evaluator

  • GPT-4 outperforms ChatGPT in human correlation across most dimensions, yet remains candidate- and dimension-dependent.Its dimension medians are more aligned than ChatGPT’s; consistency meta-correlation improves, while relevance meta-correlation is more negative.Source: §4.3 · pp. 8–9, Figure 5 · p. 8

5 A Temporary Efficient Framework

The section proposes correlating reason-then-score (RTS) and multiple-choice question (MCQ) scores as a candidate-level reliability indicator, with human evaluation retained when they disagree. The indicator correlates significantly with method performance on some evaluation dimensions.

5 A Temporary Efficient Framework

  • For each candidate system, the proposed framework uses the correlation between RTS and MCQ scores as a reliability indicator.The authors present it as a cost-efficient way to gauge when further human evaluation may be needed.Source: §5 · p. 9
  • R_i significantly correlates with RTS performance on consistency and fluency, and with MCQ performance on coherence and consistency.The reported correlations support R_i as a reliability indicator for those method-dimension combinations.Source: §5 · p. 9
  • When RTS and MCQ agree above a tolerance r, the evaluator may be treated as reliable enough; disagreement calls for human evaluation.The authors propose agreement as a preliminary indicator, not a conclusive replacement for human judgment.Source: §5 · p. 9

6 Conclusion

The paper analyzes prompting techniques for evaluating abstractive summarization and concludes that LLMs outperform common automatic metrics but are not ready to replace human evaluators. If LLMs are used, agreement across evaluation methods can help indicate when further human evaluation is needed.

6 Conclusion

  • The analysis explores different prompting techniques for using LLMs as abstractive summarization metrics.The conclusion distinguishes this broader evaluation analysis from its focus on reliability across systems and dimensions.Source: §6 · p. 9
  • LLMs perform better than commonly used automatic metrics across summarization systems and dimensions, but remain unsuitable as replacements for human evaluators.The conclusion cites candidate- and dimension-dependence and weaker human alignment for high-quality candidates as limitations.Source: §6 · p. 9
  • When LLM evaluators are used, agreement between multiple evaluation methods is proposed as a preliminary signal of unreliability and need for further human evaluation.The authors frame this as a practical indicator rather than a replacement for human evaluation.Source: §6 · p. 9

Limitations

The study’s conclusions are bounded by its small annotated dataset, limited prompt-tuning scope, and reliance on commercial model snapshots. Its temporary reliability framework also has dimension and threshold limits.

Limitations

  • Only 100 summaries per system and 12 systems were evaluated, so per-system correlations and meta-correlations may be affected by sample limitations.The authors note that larger annotated datasets are needed for further investigation.Source: Limitations · p. 10
  • Prompt tuning was outside the study’s focus, although better prompts may improve LLM evaluation.The authors also report negative meta-correlation with some alternative prompts.Source: Limitations · p. 10
  • Commercial LLM snapshots may not remain available, making future comparisons with evaluations using different checkpoints potentially unfair.The paper notes that the studied gpt-3.5-turbo-0301 snapshot was replaced by a newer one.Source: Limitations · p. 10
  • The temporary framework does not apply to relevance, and the reliability threshold r may depend on the dataset.The authors leave developing better ways to gauge LLM evaluation reliability for future work.Source: Limitations · p. 10

A Evaluation Dimensions

The appendix describes four summary-evaluation dimensions, the prompt design used for RTS, MCQ, and H2H evaluation, and alternative prompt variants. It identifies the dimensions and prompt variants but does not report their comparative outcomes in these passages.

A Evaluation Dimensions

  • The four dimensions are coherence, the collective quality of sentences; consistency, factual alignment with the source; fluency, sentence quality; and relevance, selection of important content.The prompts follow these evaluation-dimension definitions.Source: A Evaluation Dimensions · p. 13

B Prompt Details and Design

  • Prompts cover relevance, consistency, fluency, and coherence using RTS, MCQ, and H2H methods, with definitions partly generated in ChatGPT’s language.The authors say prompt design was not a main focus, while noting that their prompts were compared with alternatives in Appendix C.Source: B Prompt Details and Design · pp. 14–15

C Alternative prompts

  • The alternative prompts are StarEval, which uses one-to-five stars, and RTS2 and MCQ2, which use the full definitions from Appendix A.Their results are reported in Tables 17 and 18, but these passages do not state the comparison outcomes.Source: C Alternative prompts · p. 15

D Llama 2 Results

Llama 2 models show limited promise as zero-shot evaluators, while the challenging-pairs analysis narrows head-to-head comparison to a smaller set because it requires many evaluations. The challenge set consists of pairs of systems with consecutive average human scores.

D Llama 2 Results

  • Llama 2 70B fails to outperform BARTScore’s human correlation and is overwhelmed by ChatGPT and GPT-4, so the open-source models are not suitable zero-shot evaluators.Table 19 reports Llama 2 correlations with human scores; its caption notes that light-gray values are insignificant.Source: D Llama 2 Results · p. 15, Table 19 · p. 16
  • For Llama 2, 7B has very low human-score correlations, significant only for consistency and relevance; 13B and 70B show significant full-dataset human-score correlations, and all three have significant meta-correlations for at least one dimension.Source: D Llama 2 Results · p. 15, Table 19 · p. 16

E Challenging Pairs

  • The challenging pairs are consecutive-performance systems, each pair having the smallest average-human-score difference; the example pairs are M22–M23 and M23–M17.Their average human scores are 4.57, 4.55, and 4.52, respectively; Table 22 presents the ChatGPT-H2H correct-preference calculation.Source: E Challenging Pairs · p. 16, Table 21 · p. 16, Table 22 · p. 17
  • The challenge set reduces H2H evaluation from 13,200 evaluations for the full standard set to 2,200 for 11 pairs, while RTS, MCQ, and baseline metrics each evaluate 1,200 summaries.Each full-set H2H pair is evaluated twice with the summaries in different positions.Source: E Challenging Pairs · p. 16

F Average ChatGPT scores

ChatGPT’s average scores vary by evaluation method: it scores systems more conservatively with RTS and more optimistically with MCQ. Table 25 presents these averages alongside GPT-4 and human scores.

F Average ChatGPT scores

  • ChatGPT scores systems more conservatively with RTS and more optimistically with MCQ.Table 25 reports average scores for each model across dimensions, including ChatGPT-RTS, ChatGPT-MCQ, GPT-4 RTS, and human scores.Source: F Average ChatGPT scores · p. 17, Table 25 · p. 18
  • Table 25 compares average ChatGPT-RTS and ChatGPT-MCQ scores with GPT-4 RTS and human scores.The accompanying prose says these general scoring trends also hold for individual systems.Source: F Average ChatGPT scores · p. 17, Table 25 · p. 18
  • Table 26 gives examples of ChatGPT RTS reasons that do not correspond to the evaluated dimension.Source: Table 26 · p. 18

G Incorrect Reasons in RTS

ChatGPT’s RTS reasoning sometimes does not match the dimension being evaluated, and the reported mismatch rates differ between two systems. The paper also provides examples for GPT-4.

G Incorrect Reasons in RTS

  • The paper presents examples of ChatGPT RTS reasons unrelated to the evaluated dimension and examples of similar GPT-4 errors.Tables 26 and 27 show the respective examples.Source: G Incorrect Reasons in RTS · p. 18, Table 27 · p. 18
  • On consistency, ChatGPT-RTS reasoning was dimension-irrelevant in 42% of M11 evaluations and 10% of M17 evaluations.M11’s average score was 2.36 and M17’s was 4.84; the authors did not compile an exact overall mismatch count.Source: G Incorrect Reasons in RTS · p. 18

H GPT-4 Evaluator

GPT-4 can penalize summaries for reasons unrelated to the evaluated dimension, as ChatGPT does. It also gives unusually generous scores, including uniform full scores for one system’s consistency generations.

H GPT-4 Evaluator

  • GPT-4 makes the same RTS mistake as ChatGPT, penalizing summaries for reasons unrelated to the evaluated dimension.Table 27 gives examples of GPT-4 reasons that do not match the dimension.Source: H GPT-4 Evaluator · p. 18
  • GPT-4 gives much higher average scores than ChatGPT-RTS across all dimensions; in one exceptionally extreme case, it assigns full consistency scores to every generation by M12.Source: H GPT-4 Evaluator · p. 18
  • GPT-4’s uniform 5s for M12 consistency prevent correlation calculations for that dimension.Table 28 reports GPT-4 RTS correlations with human scores across 100 summaries per model.Source: Table 28 · p. 19

Related papers

Loading 2305.13091v2…