Source-linked AI summary
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
TL;DR
NLG quality is difficult to evaluate automatically because conventional metrics correlate weakly with human judgments and require references, while existing LLM evaluators remain less reliable. G-EVAL uses chain-of-thought and form-filling with LLMs to evaluate generated texts, and its experiments show stronger human correspondence while identifying a bias toward LLM-generated text.
Problem
Conventional NLG metrics have low human correlation, especially for creative tasks, and require costly references, while existing LLM evaluators still show lower human correspondence than medium-size neural evaluators.
Method
G-EVAL combines task prompts, LLM-generated chain-of-thought evaluation steps, form filling, and probability-based scoring to assess generated texts.
Results
G-EVAL outperforms existing NLG evaluators by a large margin in correlation with human evaluations across text summarization and dialogue generation, with G-EVAL-4 substantially surpassing prior evaluators on SummEval.
Takeaways & Limitations
LLM-based metrics generally outperform reference-based and reference-free baselines, particularly for open-ended creative generation, while chain-of-thought and probability reweighting improve evaluator behavior.
Takeaways & Limitations
LLM-based evaluators may prefer LLM-generated texts, potentially reinforcing and over-fitting LLMs when evaluation scores are used as reward signals.
Abstract
from arXiv · showhide
The quality of texts generated by natural language generation (NLG) systems is hard to measure automatically. Conventional reference-based metrics, such as BLEU and ROUGE, have been shown to have relatively low correlation with human judgments, especially for tasks that require creativity and diversity. Recent studies suggest using large language models (LLMs) as reference-free metrics for NLG evaluation, which have the benefit of being applicable to new tasks that lack human references. However, these LLM-based evaluators still have lower human correspondence than medium-size neural evaluators. In this work, we present G-Eval, a framework of using large language models with chain-of-thoughts (CoT) and a form-filling paradigm, to assess the quality of NLG outputs. We experiment with two generation tasks, text summarization and dialogue generation. We show that G-Eval with GPT-4 as the backbone model achieves a Spearman correlation of 0.514 with human on summarization task, outperforming all previous methods by a large margin. We also propose preliminary analysis on the behavior of LLM-based evaluators, and highlight the potential issue of LLM-based evaluators having a bias towards the LLM-generated texts. The code is at https://github.com/nlpyang/geval
1 Introduction
NLG evaluation remains difficult because conventional metrics correlate weakly with human judgments and require costly references, while existing LLM evaluators have reliability and bias concerns. G-EVAL addresses these gaps with chain-of-thought evaluation, form filling, and probability-weighted scoring.
- Reference-based metrics have relatively low correlation with human judgments, especially for open-ended generation, and require costly reference outputs for new tasks.
- Existing reference-free LLM evaluators score outputs without references, but their validity and reliability remain insufficiently established.
- G-EVAL combines LLM-generated chain-of-thought evaluation steps, a form-filling paradigm, and probability-weighted output scores to assess generated texts.
- LLM-based metrics generally outperform reference-based and reference-free baselines in correlation with human judgments, especially for open-ended and creative tasks.
- Chain-of-thought improves LLM evaluator performance by providing additional context and guidance, while token-probability reweighting produces finer-grained continuous scores.
- LLM-based evaluators may prefer LLM-generated text, potentially causing self-reinforcement when evaluation scores serve as rewards for further tuning.
2 Method
G-EVAL defines evaluation criteria in a prompt, generates detailed evaluation steps through chain-of-thought, and scores outputs through form filling. It then uses token probabilities to reduce discrete-score limitations and produce more continuous evaluations.
- G-EVAL uses a task-and-criteria prompt, an LLM-generated chain-of-thought, and a probability-based scoring function.
- The evaluation prompt defines the task and desired criteria, which can be customized for dimensions such as coherence, conciseness, and grammar.
- The chain-of-thought supplies detailed intermediate evaluation instructions that provide additional context and guidance beyond simple criteria definitions.
- The form-filling scorer combines the prompt, chain-of-thought, input context, and target text before requesting a discrete rating for each evaluation aspect.
- Dominant integer ratings can reduce score variance and correlation with human judgments, while integer-only outputs create ties between subtly different texts.
- Probability-weighted summation of predefined scores yields fine-grained continuous values that better reflect generated-text quality and diversity.
3 Experiments
The experiments compare G-EVAL with existing evaluators across summarization, dialogue generation, and hallucination-focused evaluation benchmarks. G-EVAL substantially improves correspondence with human judgments, with model capacity affecting hallucination consistency while GPT-3.5 and GPT-4 perform similarly on Topical-Chat.
- Summarization: Similarity-based metrics perform poorly on most summarization dimensions, whereas neural evaluators show much higher correlations with human ratings.These correlations suggest greater reliability for summarization evaluation than similarity-based metrics.
- Summarization: G-EVAL substantially surpasses previous state-of-the-art evaluators on SummEval, and GPT-4 corresponds more closely with humans than GPT-3.5.The comparison uses summary-level Spearman and Kendall-Tau correlations; the larger GPT-4 model is reported as beneficial for summarization evaluation.
- Dialogue Generation: On Topical-Chat, similarity-based metrics agree with humans on engagingness and groundedness but not on the other evaluated aspects.Before G-EVAL, UniEval was the learning-based evaluator most consistent with human judgments across all aspects.
- Dialogue Generation: G-EVAL substantially surpasses previous state-of-the-art evaluators on Topical-Chat, while G-EVAL-3.5 achieves results similar to G-EVAL-4.The authors interpret the similar model results as indicating that this benchmark is relatively easy for G-EVAL.
- Hallucination Evaluation: G-EVAL-4 outperforms all state-of-the-art evaluators on QAGS, with a large margin on QAGS-XSum, whereas G-EVAL-3.5 performs poorly.The result indicates that evaluating consistency on this benchmark is sensitive to the LLM’s capacity.
4 Analysis
The analysis examines G-EVAL’s behavior across model size, chain-of-thought prompting, probability normalization, and comparisons between human- and LLM-generated summaries. It identifies stronger performance with larger models and CoT, while highlighting uncertainty and bias toward LLM-generated text.
- Evaluator Bias: G-EVAL-4 assigns higher scores to human-written summaries when human judges prefer them, but still scores GPT-3.5 summaries higher overall.The authors attribute this pattern partly to low agreement among human judges and partly to possible evaluator bias toward LLM-generated text.
- Evaluator Bias: Krippendorff’s alpha of 0.07 indicates very low inter-annotator agreement when judging human-written and LLM-generated summaries.This low agreement complicates interpretation of evaluator preferences in the comparison.
- Evaluator Bias: The analysis treats bias toward LLM-generated text as a preliminary issue requiring further research to understand and reduce.The authors warn that evaluator scores used as rewards could reinforce models’ own evaluation criteria rather than task-specific criteria.
- Chain-of-Thought: G-EVAL-4 with CoT has higher correlation than G-EVAL-4 without CoT on all SummEval dimensions, especially fluency.The authors suggest that CoT supplies additional context and guidance for evaluating generated text.
- Probability Normalization: On SummEval, G-EVAL-4 with probabilities is inferior to G-EVAL-4 without probabilities on Kendall-Tau correlation.The authors caution that direct scoring can create ties that affect Kendall-Tau, whereas probability normalization produces a more continuous score.
- Model Size: G-EVAL-4 has higher correlation than G-EVAL-3.5 on most dimensions and datasets, except engagingness and groundedness on Topical-Chat.The improvement is especially associated with challenging evaluation dimensions such as consistency and relevance.
5 Related Work
Related work spans lexical-overlap, embedding-based, task-specific, unified, and LLM-based evaluators. These approaches differ in their reliance on references, task specialization, semantic similarity, and generative-model probabilities.
- Ngram-based Metrics: Ngram-based metrics evaluate generated text through lexical overlap with reference text.BLEU uses modified n-gram precision and a brevity penalty, while ROUGE measures n-gram recall against reference summaries.
- Ngram-based Metrics: BLEU and ROUGE may fail to measure content quality or capture syntactic errors, limiting their reliability for NLG evaluation.The passage also notes that more than 60% of recent NLG papers rely on ROUGE or BLEU.
- Embedding-based Metrics: Embedding-based metrics measure semantic similarity between generated and reference texts using word or sentence embeddings.Examples include WMD, BERTScore, and MoverScore.
- Task-specific Evaluators: Task-specific metrics assess requirements such as summary consistency or dialogue coherence, but are not generalizable across NLG tasks.They also do not measure the overall quality of generated texts.
- Unified Evaluators: Unified evaluators assess multiple text-generation dimensions by varying input and output contents, model variants, or question formats.UniEval frames different evaluation tasks as question-answering tasks.
- LLM-based Evaluators: LLM-based evaluators assign scores using generative-model probabilities under instructions and context, including GPTScore and related GPT-based approaches.These methods extend reference-free or instruction-conditioned evaluation with generative pre-training models.
6 Conclusion
G-EVAL uses LLMs with chain-of-thoughts to evaluate generated text across summarization and dialogue generation. The paper reports higher human correspondence while identifying potential bias toward LLM-generated texts.
- Conclusion: G-EVAL applies LLM-based chain-of-thought evaluation to text summarization and dialogue generation.The framework is evaluated against existing evaluators for correspondence with human judgments.
- Conclusion: G-EVAL outperforms state-of-the-art evaluators and achieves higher human correspondence in the reported experiments.The conclusion covers both summarization and dialogue-generation tasks.
- Conclusion: The paper’s preliminary behavioral analysis highlights potential bias in LLM-based evaluators toward LLM-generated texts.The authors present this as a risk and challenge for using LLMs as evaluators.
Evaluate Coherence in the Summarization Task
The coherence evaluation prompt asks the evaluator to assess how well a news summary is structured and organized. It guides comparison with the source article before assigning a 1-to-5 score.
- Evaluation Task: The evaluation task rates one news summary on coherence using a 1-to-5 scale.The form is intended to return scores only.
- Evaluation Criteria: Coherence is defined as the collective quality of all sentences and the summary’s structure and organization.A coherent summary should build from sentence to sentence rather than form a heap of related information.
- Evaluation Steps: The evaluator first identifies the news article’s main topic and key points before reading the summary.This establishes the source content against which the summary is assessed.
- Evaluation Steps: The evaluator compares the summary with the news article for coverage and clear, logical ordering of the main topic and key points.The comparison operationalizes coherence through both content coverage and presentation order.
- Evaluation Steps: After comparison, the evaluator assigns a coherence score from 1 to 5 according to the evaluation criteria.The procedure ends with a single metric rating.
Evaluate Engagingness in the Dialogue Generation Task
The dialogue-generation evaluation asks raters to judge a response’s engagingness on a 1–3 scale. Raters must read the conversation and related fact, then explain their score using specific aspects of the response and conversation.
- Raters read the conversation, corresponding fact, and response carefully before scoring.
- The task evaluates one metric: engagingness, or whether a response is dull or interesting.
- Scores range from 1 to 3, with 1 meaning generic and dull, 2 somewhat interesting, and 3 very interesting or factually interesting.
- Raters provide a brief explanation that refers to specific aspects of both the response and the conversation.
- The evaluation form records scores only.
Evaluate Hallucinations
The summarization evaluation includes a factual-consistency criterion focused on whether summaries contain unsupported, untruthful, misleading, or otherwise inconsistent facts.
- The criterion appears within human evaluation of text summarization systems.
- Factual consistency asks whether a summary contains untruthful or misleading facts unsupported by the source text.
- The evaluation also asks directly whether the summary contains factual inconsistency.