PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM-based NLG Evaluation: Current Status and Challenges

Mingqi Gao, Xinyu Hu, Jie Ruan, Xiao Pu, Xiaojun Wan

arXiv:2402.01383v3cs.CL

TL;DR

NLG evaluation is challenging because outputs have no single correct answer and must meet multiple criteria, while existing automatic metrics remain unsatisfactory. This survey proposes a four-category taxonomy of LLM-based evaluation, reviews methods and trade-offs, and discusses open problems and future directions.

  • Problem

    NLG evaluation lacks a single correct answer and must account for multiple quality criteria, while existing metrics remain unsatisfactory.

    Source: §1 · pp. 1–2

  • Method

    The survey classifies LLM-based NLG evaluation into four categories and reviews their approaches and trade-offs.

    Source: §1 · p. 3

  • Results

    The survey characterizes LLM-derived metrics, prompting, fine-tuning, and human–LLM collaboration, and identifies open problems and future directions.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The taxonomy distinguishes coarse-grained LLM-derived metrics from flexible natural-language evaluation and human–LLM collaboration.

    Source: §1 · p. 3

    LLM-derived methods that require model parameters, representations, or logits cannot be applied to closed-source models that do not provide them.

    Source: §2.3 · p. 5

Abstract

from arXiv · show

Evaluating natural language generation (NLG) is a vital but challenging problem in natural language processing. Traditional evaluation metrics mainly capturing content (e.g. n-gram) overlap between system outputs and references are far from satisfactory, and large language models (LLMs) such as ChatGPT have demonstrated great potential in NLG evaluation in recent years. Various automatic evaluation methods based on LLMs have been proposed, including metrics derived from LLMs, prompting LLMs, fine-tuning LLMs, and human-LLM collaborative evaluation. In this survey, we first give a taxonomy of LLM-based NLG evaluation methods, and discuss their pros and cons, respectively. Lastly, we discuss several open problems in this area and point out future research directions.

1. Introduction

NLG evaluation is difficult because outputs lack a single correct answer and must satisfy multiple quality criteria, while existing automatic metrics remain unsatisfactory. This survey organizes LLM-based evaluation methods into four categories and reviews their scope, trade-offs, and future directions.

1. Introduction

  • Human evaluation remains the gold standard, but its cost and time requirements make automatic metrics indispensable despite their shortcomings.Existing metrics can be insensitive, biased, or vulnerable to loopholes on challenging texts.Source: §1 · p. 2
  • The survey focuses on LLM-based approaches using models with over one billion parameters for typical natural-language input and output tasks, while excluding evaluation of LLM capabilities.It searches literature through June 2024 and also excludes NLG datasets, benchmarks, and statistical analyses of metrics.Source: §1 · p. 2
  • The survey groups LLM-based NLG evaluation into metrics derived from LLMs, prompting, fine-tuning, and human–LLM collaboration.The categories are organized by how LLMs are used for evaluation.Source: §1 · p. 3
  • LLM-derived metrics continue the tradition of automatic evaluation, prompting and fine-tuning permit flexible natural-language requirements, and collaboration draws on human and LLM strengths.The survey also documents which NLG tasks each approach has been tested on rather than organizing the literature by task.Source: §1 · p. 3

2. LLM-derived Metrics

LLM-derived metrics extend earlier model-based metrics using LLM embeddings or probabilities. The survey describes their implementations and trade-offs, including shortcomings in robustness, efficiency, and access to model internals.

2. LLM-derived Metrics

  • LLM-derived metrics extend model-based evaluation such as BERTScore and BARTScore, using embeddings or probabilities from LLMs.Probability-based methods either convert probabilities directly into scores or measure their variation under changed conditions.Source: §2 · p. 3

2.1 Embedding-based Metrics

  • Embedding-based metrics estimate semantic similarity between reference and target texts, and some LLM-based variants do not require references.ES et al. (2024) generate possible questions from an answer and compare their embeddings with the original question; Sheng et al. (2024) adapt LLM embeddings for scoring and comparison.Source: §2.1 · pp. 3–4

2.2 Probability-based Metrics

  • Probability-based metrics treat text evaluation as conditional likelihood scoring; GPTScore uses aspect-specific instructions, while other approaches use perplexity or probability changes.FFLM (Jia et al. 2023) evaluates faithfulness through probability changes, and DELTASCORE (Xie et al. 2023) compares likelihood before and after perturbations.Source: §2.2 · p. 4

2.3 Pros and Cons

  • LLM-derived metrics address surface-form mismatch and can correlate more strongly with human judgments, but robustness tests identify metric blind spots and missed errors.He et al. (2023b) examine robustness across attack scenarios and common NLG tasks.Source: §2.3 · p. 5
  • LLM-derived metrics require more time and computation than traditional metrics, and methods needing model internals cannot use closed-source LLMs that withhold them.Eddine et al. (2022) propose a lightweight metric; vLLM improves inference throughput through memory management and batching strategies.Source: §2.3 · p. 5
  • Model-based metrics carry more social bias than traditional metrics in a study assessing six sensitive attributes.Sun et al. (2022) distinguish intrinsic bias in pretrained models from extrinsic bias introduced during similarity computation.Source: §2.3 · p. 5

3. Prompting LLMs

Prompting treats LLM evaluation as a prompted annotation task, with studies varying evaluation methods, instructions, inputs, criteria, and evaluator roles and interactions.

3.1 Evaluation Methods

  • Prompted evaluations use scoring, pairwise comparison, ranking, Boolean QA, or error analysis to elicit judgments.The methods differ in whether they assign scores, compare or rank texts, answer binary questions, or identify predefined error types.Source: §3.1 · pp. 6, 8
  • Scoring prompts have achieved strong human-judgment alignment across several NLG tasks, including multilingual dialogue evaluation.Reported studies find state-of-the-art or competitive results in several settings; simple rating prompts also surpass prior encoder-based metrics for multilingual dialogue.Source: §3.1 · pp. 7–8

3.2 Task Instructions

  • Instruction studies examine templates, examples, and requested explanations; clear directions can help, while effects of examples and explanations vary.Explicitly requested analysis or explanation has been associated with higher correlation in some evaluations, but generated explanations may need manual assessment; one-shot examples did not improve performance in one study.Source: §3.2 · pp. 8–9

3.3 Input Content

  • Input content depends on the criterion: faithfulness often needs a source document, whereas fluency can be judged from the target text alone.Other task-specific inputs can include references, relevant facts, context, or outputs from other evaluation metrics.Source: §3.3 · p. 9

3.4 Evaluation Criteria

  • Evaluation criteria usually draw on human-evaluation aspects, though some work uses LLMs to draft or refine criteria and to structure fine-grained aspects.A hierarchical aspect system with 12 subcategories is reported to yield high correlation between human and LLM-based evaluations; scoring related aspects first is another proposed approach.Source: §3.4 · p. 10

3.5 Role and Interaction

  • Evaluator interactions include chains that analyze, rank, or verify content, networks that debate or aggregate contributions, and panels of different LLMs.Network studies include debate strategies and role specialization; some report improved alignment under certain conditions or more accurate and reliable evaluations.Source: §3.5 · pp. 10–11

3.6 Pros and Cons

  • Prompting offers flexible natural-language criteria and explanations, and has achieved state-of-the-art human-judgment correlations in many NLG tasks, but remains vulnerable to biases and instability.Reported issues include position, verbosity, and self-preference biases, dataset-dependent performance, and possible irreproducibility from proprietary evaluators.Source: §3.6 · p. 11

4. Fine-tuning LLMs

Fine-tuned evaluators are presented as a response to prompting’s cost, reproducibility, and bias drawbacks, using constructed data and open-source models.

4. Fine-tuning LLMs

  • Customized data and fine-tuning can alleviate prompting’s high costs, irreproducibility, and potential biases in NLG evaluation.The survey says targeted measures such as randomly changing training-pair order can address certain GPT-4 biases.Source: §4 · p. 11, §4.4 · p. 15

4.1 Data Construction

  • Training data covers task scenarios, inputs, and target texts, while newer methods also adopt instruction-response examples and vary instruction sources and annotators.PandaLM and JudgeLM sample common instruction datasets; Prometheus and INSTRUCTSCORE use GPT-4-generated instructions, while Auto-J and Shepherd use real-world data. Human annotation may offer higher quality but is difficult to scale; LLM annotation offers the opposite trade-off.Source: §4.1 · pp. 12–14

4.2 Evaluation Method

  • Fine-tuned evaluators vary in criteria: some adapt them implicitly, while Prometheus and Themis state them in instructions and CompassJudger supports customized criteria.AUTO-J uses 332 task-matched criteria; methods also differ in result modes and reference use, and detailed explanations or MQM can support error analysis.Source: §4.2 · p. 14

4.3 Fine-tuning Implementation

  • Fine-tuning designs include balanced data, pair-order and reference changes, intermediate-output annotations, data-ratio choices, and preference or self-iterative optimization.The survey reports improved training efficiency and evaluation robustness relative to vanilla fine-tuning.Source: §4.3 · p. 14

4.4 Pros and Cons

  • Most models in Table 2 have less than 14B parameters, enabling low-cost inference and good reproducibility with performance comparable to GPT4.Fine-tuning also permits targeted measures against biases and continued model iteration as deficiencies or new issues emerge.Source: §4.4 · p. 15
  • GPT-4 annotation in data construction can leave self-biases, while migrating fine-tuned models to newer foundation LLMs is challenging and requires retraining from scratch.The survey notes that newer open-source foundation models may improve evaluation performance, but introduce repeated fine-tuning and computational expenses.Source: §4.4 · p. 15
  • Excessive evaluation settings may impair training, AUTO-J struggled with criteria and CritiqueLLM with references, and horizontal comparison across methods is challenging.Prometheus, JudgeLM, and CompassJudger show different results, leaving the effects of such settings unresolved.Source: §4.4 · p. 15

5. Human-LLM Collaborative Evaluation

The section surveys collaboration that combines nuanced human judgment with LLM scalability, including human-guided LLM evaluation and LLM assistance to human evaluators.

5.1 Human-Guided LLM Evaluation

  • COEVAL uses human-designed task checklists, LLM-generated evaluations and explanations, and human review, which revises approximately 20% of LLM scores.InteractEval combines human and LLM-generated attributes using Think Aloud methods to produce prediction scores.Source: §5.1 · p. 16
  • 99% accuracy with half the human effort is reported for HMCEval’s assignment of dialogue evaluation tasks between people and machines.EvalAssist also supports practitioners refining criteria through direct and pairwise assessment.Source: §5.1 · p. 16

5.2 LLM-Assisted Human Evaluation

  • Human feedback improves AdaTest bug detection by 5-10x, while prefilled machine-translation span annotations halve annotation time and cut the annotation budget by almost 25%.These findings are reported for Ribeiro and Lundberg (2022) and Zouhar, Kocmi, and Sachan (2024), respectively.Source: §5.2 · p. 16
  • AdaTest++ combines human schematization and hypothesis testing with LLM identification of failure modes, uncovering known and under-reported issues (Rastogi et al. 2023).Source: §5.2 · pp. 16–17
  • LLM-generated critiques can improve human performance, and conversational interfaces can help educators evaluate student assignments (Saunders et al. 2022; Pozdniakov et al. 2024).Source: §5.2 · p. 17

5.3 Pros and Cons

  • Collaboration can improve accuracy and reliability over either humans or LLMs alone, while combining human judgment with automated efficiency.Challenges include prompt sensitivity, limited confidence calibration, continued need for human oversight, and explainability.Source: §5.3 · p. 17

6. Conclusions and Future Trends

The survey compares LLM-based evaluation approaches across flexibility, reproducibility, performance, and cost, then identifies benchmark, language, task, and collaboration gaps for future work.

6.1 Comparison with traditional evaluation metrics.

  • Traditional metrics have been criticized for weak human-judgment correlation, uninterpretable results, and limited adaptability, while LLM-based evaluation mitigates these issues at higher computational and reproducibility cost.Human evaluation remains the gold standard, but its cost and time demands make automatic metrics indispensable.Source: §6.1 · p. 17

6.2 Comparison between different types of LLM-based NLG evaluation.

  • Human-LLM collaboration is most flexible, followed by prompting, fine-tuning, and LLM-derived metrics, which often target specific aspects such as text similarity.Proprietary models generally follow instructions better than smaller open-source models, favoring prompting over fine-tuning in flexibility.Source: §6.2 · pp. 17–18
  • On SummEval, human-LLM collaboration performs strongest, followed by fine-tuning, prompting, and LLM-derived metrics.Among fine-tuned methods, evaluation-specialized models such as Themis outperform prompting methods, including GPT-4; collaboration incorporates expert-informed checklists.Source: Table 4 · p. 18, §6.2 · p. 19
  • LLM-derived metrics and prompting are more reproducible than fine-tuning, while human-LLM collaboration is least reproducible because it requires recruiting and training annotators.Metrics and prompting may still become non-reproducible if proprietary models are deprecated.Source: §6.2 · p. 18
  • LLM-derived metrics and prompting open-source models have similar inference costs, while fine-tuning adds training costs and human-LLM collaboration is most expensive.Proprietary-model prompting incurs high API costs, which are difficult to compare directly with fine-tuning costs.Source: §6.2 · p. 19

6.3 Future Directions

  • A unified benchmark is needed because fine-tuned evaluators use differing test settings, while existing human judgments are often narrow in task and method, outdated, or small.The proposed benchmark need is for large-scale, high-quality human-evaluation data covering varied NLG tasks and evaluation methods.Source: §6.3 · p. 19
  • Future work should examine low-resource languages and newer NLG scenarios, since existing research is largely English-focused and concentrated on traditional tasks.The authors suggest further language-specific fine-tuning as a potential strategy, citing preliminary promising results, and point to scientific-review evaluation as one newer scenario.Source: §6.3 · pp. 19–20
  • Human-LLM collaboration remains underexplored, including how people and models should divide evaluation roles and complement one another.The authors propose exploring human-computer interaction technologies as possible implementation methods.Source: §6.3 · p. 20
Loading 2402.01383v3…