PaperlayerView exact revision on arXiv

Source-linked AI summary

ChatGPT as a Factual Inconsistency Evaluator for Text Summarization

Zheheng Luo, Qianqian Xie, Sophia Ananiadou

arXiv:2303.15621v2cs.CL

TL;DR

Factual inconsistency in generated summaries remains a concern, and existing evaluation metrics have limitations including computational cost, pipeline uncertainty, and partial agreement with human judgments. This paper tests ChatGPT in zero-shot entailment inference, summary ranking, and consistency rating, finding it outperforms state-of-the-art metrics on six of nine datasets while retaining limitations in reliability.

  • Problem

    Factual inconsistency remains a concern in generated summaries, and existing factuality metrics face computational or pipeline limitations and only partial agreement with human assessments.

    Source: §1 · pp. 1–2

  • Method

    The study evaluates ChatGPT with zero-shot prompts on entailment inference, summary ranking, and consistency rating for summarization factuality.

    Source: §1 · p. 2

  • Results

    ChatGPT outperformed state-of-the-art evaluation metrics on six of nine datasets, with chain-of-thought prompting also significantly boosting performance.

    Source: §5 · pp. 9–10

  • Takeaways & Limitations

    The experiments indicate ChatGPT has potential as a factual-inconsistency evaluator, while its reliability limitations remain to be addressed.

    Source: §5 · p. 9

    The study tests only zero-shot prompts on three tasks and does not compare different large language models such as GPT-3.5 and GPT-4.

    Source: Limitations · p. 10

Abstract

from arXiv · show

The performance of text summarization has been greatly boosted by pre-trained language models. A main concern of existing methods is that most generated summaries are not factually inconsistent with their source documents. To alleviate the problem, many efforts have focused on developing effective factuality evaluation metrics based on natural language inference, question answering, and syntactic dependency et al. However, these approaches are limited by either their high computational complexity or the uncertainty introduced by multi-component pipelines, resulting in only partial agreement with human judgement. Most recently, large language models(LLMs) have shown excellent performance in not only text generation but also language comprehension. In this paper, we particularly explore ChatGPT's ability to evaluate factual inconsistency under a zero-shot setting by examining it on both coarse-grained and fine-grained evaluation tasks including binary entailment inference, summary ranking, and consistency rating. Experimental results indicate that ChatGPT generally outperforms previous evaluation metrics across the three tasks, indicating its great potential for factual inconsistency evaluation. However, a closer inspection of ChatGPT's output reveals certain limitations including its preference for more lexically similar candidates, false reasoning, and inadequate understanding of instructions.

1 Introduction

Factual inconsistency remains a major concern in summarization, while existing evaluation metrics face computational, pipeline, and human-agreement limitations. This study tests ChatGPT in zero-shot factuality evaluation and reports strong performance alongside reliability concerns.

1 Introduction

  • Existing factuality metrics can be computationally expensive or pipeline-dependent, and their evaluations show limited agreement with human assessments.Approaches include natural language inference, question answering, and syntactic dependency methods.Source: §1 · p. 1
  • The study systematically examines ChatGPT on entailment inference, summary ranking, and consistency rating using zero-shot and zero-shot chain-of-thought prompts.The entailment task uses six standardized datasets, with existing datasets used for the other tasks.Source: §1 · p. 2
  • ChatGPT outperforms previous state-of-the-art evaluation methods on most datasets across three tested factuality tasks in the zero-shot setting.The tasks are entailment inference, summary ranking, and consistency rating.Source: §1 · p. 2
  • ChatGPT may favor lexically similar candidates over semantic entailment and can produce false reasoning.The authors also report that tested prompts do not always keep ChatGPT aligned with the given requirements.Source: §1 · p. 2

2 Related Work

Prior factuality metrics use information extraction, natural language inference, question answering, and language-model likelihoods, with costs or data requirements noted for some approaches. Earlier ChatGPT research covered NLP performance and broad text evaluation, leaving a gap in factual-consistency assessment.

2.1 Factuality Evaluation in Text Summarization

  • Information-extraction methods assess overlap in extracted relation tuples or entity-level consistency, while NLI methods test whether a summary is entailed by its source.The cited NLI work reports poor assessment ability on summarization and links it to short premises in NLI datasets.Source: §2.1 · p. 2
  • Question-answering metrics compare answers derived from summaries and sources, while semi-supervised FactCC uses synthetic data and PLM-likelihood metrics include BARTScore and T5Score.The passage notes that some methods are computationally expensive or rely on annotated data.Source: §2.1 · p. 3

2.2 ChatGPT for Natural Language Processing

  • Prior ChatGPT studies report varied NLP performance, including strong results on some inference and reasoning tasks and weaker results on some similarity or summarization tasks.Other work explored ChatGPT summarization and human evaluation of its generated summaries.Source: §2.2 · p. 3
  • A previous ChatGPT evaluation study covered one summarization dataset and overall quality, rather than a thorough comparison with metrics designed for factual consistency.The passage identifies this as a gap in evaluating ChatGPT for factual consistency in summarization.Source: §2.2 · p. 3

3 ChatGPT as a Factual Inconsistency Evaluator

The study evaluates ChatGPT with three task formulations: binary entailment inference, choosing between summaries, and rating consistency. Its setups use zero-shot prompts and compare ChatGPT with existing evaluation models or human judgments.

3.1 Entailment Inference

  • Entailment inference asks ChatGPT to judge whether a summary is consistent with its source, with direct yes/no and zero-shot chain-of-thought prompts.The six-dataset SUMMAC benchmark standardizes instances as document-summary-label triplets; balanced accuracy is used because test labels are unbalanced.Source: §3.1 · pp. 3–5
  • The entailment-inference comparison includes NER overlap, MNLI-doc, FactCC, DAE, FEQA, QuestEval, and SummaC variants.These baselines use entity overlap, entailment models, corrupted training examples, dependency arcs, question-answering, or sentence-pair scoring approaches.Source: §3.1 · p. 4

3.2 Summary Ranking

  • Summary ranking asks ChatGPT to choose the more consistent of two candidates, and accuracy measures whether it selects the consistent summary.The Falke et al. dataset contains 373 CNN/DM source documents, each paired with one consistent and one inconsistent summary; comparisons include human judgment and evaluation models.Source: §3.2 · p. 5

3.3 Consistency Rating

  • Consistency rating asks ChatGPT to score a summary from 1 to 10, with higher scores representing greater consistency with the source.The task uses SummEval and FRANK annotations and compares model scores with human judgments using correlation measures.Source: §3.3 · pp. 5–6

4 Experiment

The experiment tests ChatGPT on entailment inference, summary ranking, and consistency rating, then examines behavioral failures. Results include strong zero-shot comparisons alongside sensitivity to prompt context and lexical similarity.

4 Experiment

  • Experiments used the ChatGPT API, with each request sent individually to avoid effects from historical dialogues.Source: §4 · p. 6

4.1 Entailment Inference

  • In five of six datasets, ChatGPT retrieves more than 95% of consistent summaries but performs poorly at identifying all inconsistent ones.The authors suggest reliance on semantic similarity may leave it vulnerable to meaning-changing edits in lexically similar summaries.Source: §4.1 · pp. 6–7, Figure 1 · p. 6
  • ChatGPTZS-COT beats SummaCZS by 3.9%, 1.6%, and 1.0% on CoGenSum, SummEval, and FRANK, respectively, while remaining comparable on three other datasets.The three other comparisons are XsumFaith (63.1% vs. 66.4%), Polytope (61.4% vs. 70.3%), and FactCC (79.5% vs. 89.5%).Source: §4.1 · p. 6, Table 2 · p. 7

4.2 Summary Ranking

  • Without in-context learning, ChatGPT outperforms existing methods and a human assessment in the summary-ranking task.With highly similar candidate summaries, it sometimes identifies the consistent one even when it missed the inconsistency in entailment inference.Source: Table 3 · p. 7, §4.2 · p. 7

4.3 Consistency Rating

  • On consistency rating, ChatGPT aligns more closely with human judgments than other metrics, especially across the full FRANK dataset.Its correlations decline from CNN/DM summaries to XSum summaries, which the authors suggest may relate to XSum summaries’ greater abstractiveness and lower lexical similarity.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 Error Analysis

  • Error cases include accepting a summary’s unsupported claim and assigning a low rating despite acknowledging its fact was consistent under the supplied rubric.The authors also report that zero-shot CoT sometimes gives its conclusion before its reasoning, so the explanation is conditioned on that conclusion.Source: §4.4 · p. 9

5 Conclusion

The study finds ChatGPT promising for zero-shot factual inconsistency evaluation, while identifying reliability limitations. Chain-of-thought prompting can boost performance, but its limitations underscore the importance of alignment research.

5 Conclusion

  • ChatGPT outperformed state-of-the-art evaluation metrics on six of nine datasets in zero-shot factual inconsistency evaluation.The study covers three coarse- and fine-grained evaluation tasks.Source: §5 · p. 9
  • ChatGPT’s evaluation remains limited by bias, false reasoning, and hallucination.The authors identify these limitations as issues to address for reliable use.Source: §5 · p. 9
  • Chain-of-thought prompting significantly boosted ChatGPT’s performance, while its limitations highlight the importance of future alignment research.The authors describe this study as an initial step toward exploring ChatGPT for factual inconsistency evaluation.Source: §5 · p. 10

Limitations

The study’s evidence is limited to zero-shot prompting with ChatGPT on three tasks, and it does not compare different large language models.

Limitations

  • API costs limited the study to zero-shot prompts on three tasks; few-shot prompting remains unexplored.The authors suggest exploring more effective prompts in future work.Source: Limitations · p. 10
  • The evaluation covers ChatGPT alone, leaving comparisons among large language models such as GPT-3.5 and GPT-4 for future work.The authors propose such comparisons to examine the models’ relative strengths and limitations.Source: Limitations · p. 10

Related papers

Loading 2303.15621v2…