PaperlayerView exact revision on arXiv

Source-linked AI summary

The Critique of Critique

Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, Pengfei Liu

arXiv:2401.04518v2cs.CLcs.AI

TL;DR

Critiques are widely used in LLM training, evaluation, and refinement, but systematic ways to evaluate their quality are lacking. MetaCritique scores critiques through precision and recall judgments over atomic information units and provides natural-language rationales. Experiments report near-human performance and better refinement outcomes from superior critiques selected by MetaCritique.

  • Problem

    Critiques support LLM training, evaluation, and refinement, but their quality lacks systematic evaluation methods with clear criteria, reliable scoring, and a way to handle intricate judgments.

    Source: §1 · p. 1

  • Method

    MetaCritique evaluates critique factuality and coverage using precision and recall over Atomic Information Units, aggregates the judgments, and generates a natural-language rationale for each judgment.

    Source: §1 · pp. 1–2

  • Results

    MetaCritique achieves near-human performance, and superior critiques it selects lead to improved results through iterative refinement.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Superior critiques selected by MetaCritique can improve refinement outcomes, suggesting potential to enhance alignment of existing LLMs.

    Source: §1 · p. 2, §7 · p. 8

    Recall is unsuitable for creative tasks with multiple high-quality answers because the diverse effective critiques are difficult to collect comprehensively.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Critique, as a natural language description for assessing the quality of model-generated content, has played a vital role in the training, evaluation, and refinement of LLMs. However, a systematic method to evaluate the quality of critique is lacking. In this paper, we pioneer the critique of critique, termed MetaCritique, which builds specific quantification criteria. To achieve a reliable evaluation outcome, we propose Atomic Information Units (AIUs), which describe the critique in a more fine-grained manner. MetaCritique aggregates each AIU's judgment for the overall score. Moreover, MetaCritique delivers a natural language rationale for the intricate reasoning within each judgment. Lastly, we construct a meta-evaluation dataset covering 4 tasks across 16 public datasets involving human-written and LLM-generated critiques. Experiments demonstrate that MetaCritique can achieve near-human performance. Our study can facilitate future research in LLM critiques based on our following observations and released resources: (1) superior critiques judged by MetaCritique can lead to better refinements, indicating that it can potentially enhance the alignment of existing LLMs; (2) the leaderboard of critique models reveals that open-source critique models commonly suffer from factuality issues; (3) relevant code and data are publicly available at https://github.com/GAIR-NLP/MetaCritique to support deeper exploration; (4) an API at PyPI with the usage documentation in Appendix C allows users to assess the critique conveniently.

1 Introduction

The paper identifies a gap in systematic critique evaluation and introduces MetaCritique, which scores critiques through atomic judgments and generates rationales. It describes a workflow for reference generation, AIU extraction, and precision- and recall-based assessment.

1 Introduction

  • Existing critique evaluation lacks specific rating criteria, transparent scoring, and a way to capture complex relations among concepts.The paper frames these as challenges of quantification, reliability, and intricacy.Source: §1 · p. 1

2 METACRITIQUE

  • MetaCritique measures factuality with precision and coverage of necessary information with recall, then aggregates AIU-level judgments into an overall score.The workflow comprises reference generation, AIU extraction, and hypothesis critiquing.Source: §1 · p. 1, §2 · p. 2

2.1 Step 1: Reference Generation

  • GPT-4-generated reference answers and critiques act as proxies because human-created references require substantial effort, and human evaluation checks their quality.The authors note that reference answers support precision scoring and reference critiques support recall scoring.Source: §2.1 · p. 3

2.2 Step 2: AIU Extraction

  • GPT-4 extracts Atomic Information Units, which break critiques into fundamental pieces for fine-grained evaluation.The paper uses human evaluations to verify extraction quality.Source: §1 · p. 1, §2.2 · p. 3

2.3 Step 3: Hypothesis Critiquing

  • Precision labels each hypothesis-critique AIU factual or not; recall tests whether each reference-critique AIU is entailed, with a rationale for each judgment.Precision receives the question, model-generated answer, and optional reference answer; recall uses the hypothesis critique as its premise.Source: §2.3 · pp. 3–4

3 Meta-Evaluation Dataset

The dataset combines questions, flawed model answers, human- and LLM-written critiques, GPT-4-generated references, and extracted AIUs for human precision and recall judgments. It covers four domains across 16 datasets.

3 Meta-Evaluation Dataset

  • The dataset is built from question-answer pairs with flawed model answers and critiques, then GPT-4 references and extracted AIUs support human precision and recall annotation.Human annotators label AIUs for both tasks.Source: §3 · p. 4

3.1 Collection of Question and Model-Generated Answer

  • The collection spans entailment, reasoning, question answering, and summarization across 16 public datasets, excluding tasks that require specific tools to find errors.The questions and answers are drawn from the Shepherd dataset.Source: §3.1 · p. 4

3.2 Collection of Hypothesis Critique

  • Each pair has one human-written critique and two LLM-generated critiques; the LLM critiques are compared pairwise, avoiding human-versus-LLM comparisons to reduce style-related confounds.The sampled LLMs include WizardLM, GPT-3.5, LLaMA-2, Vicuna, and SelFee variants.Source: §3.2 · pp. 4–5

3.3 Collection of Human Annotation

  • Two postgraduate annotators label precision and recall, while a third reviews their work and resolves discrepancies to produce gold scores.These human labels replace GPT-4 labels and omit explanations.Source: §3.3 · p. 5

3.4 Statistics

  • Human-written critiques contain fewer than half as many AIUs as LLM-generated or reference critiques in the collected dataset.The authors state that this may indicate less information in human-written critiques.Source: §3.4 · p. 5

4 Experiments

The experiments assess MetaCritique’s baselines and whether GPT-4-generated answers, critiques, and AIUs can serve as references. Human ratings support using GPT-4 outcomes as references and for AIU extraction.

4 Experiments

  • The experiments test feasibility and effectiveness of MetaCritique, with baselines spanning AIU-level precision and recall models and critique-scoring methods.The comparison includes GPT-4 and open-source models, human performance, and GPT-4-based scoring variants.Source: §4 · p. 5, §4.1 · p. 5

Q1: Can GPT-4 outcomes serve as references?

  • Human evaluations rate GPT-4-generated answers, critiques, and AIUs; results support using GPT-4-generated outcomes as references and justify GPT-4’s effective extraction of AIUs for METACRITIQUE.Human evaluation used a 1–7 Likert scale, with 1 worst and 7 best.Source: Table 2 · p. 5, Q1: Can GPT-4 outcomes serve as… · p. 6

Q2: Which LLMs are capable of powering METACRITIQUE?

GPT-4 performs near human level on AIU-level precision and recall, while MetaCritique scoring methods correlate with human critique judgments better than simple GPT-4 scoring. The results also indicate that precision and recall are complementary.

Q2: Which LLMs are capable of powering METACRITIQUE?

  • Nearly 90% AIU-level accuracy makes GPT-4 comparable to humans and supports powering MetaCritique with GPT-4.The comparison evaluates precision and recall as binary classification tasks, treating each AIU as a test case.Source: Q2: Which LLMs are capable of powering… · p. 6
  • WizardLM-70B leads open-source models on precision and WizardLM-13B on recall, while both rival GPT-3.5; removing the reference answer degrades results by around 3% and 6%.The degradation supports pre-generating a reference answer for the evaluation.Source: Q2: Which LLMs are capable of powering… · p. 6

Q3: Which evaluation methods can give rating scores that are close to human judgments?

  • MetaCritiqueGPT4-F1 correlates with human judgments better than SingleGPT4, and MetaCritiqueOpen also exceeds SingleGPT4, while precision and recall complement one another.Precision or recall alone is slightly inferior to the combined F1 score; precision correlation is reduced on human-written critiques, where fewer factuality errors cluster scores.Source: Q3: Which evaluation methods can give… · p. 6

Q4: Which evaluation methods can choose the critiques that humans prefer?

MetaCritiqueGPT4-F1 best matches human-preferred critique selection and outperforms SingleGPT4 in agreement rate. In refinement experiments, critiques selected by MetaCritiqueGPT4-F1 significantly improve refined outcomes, with precision and recall supporting one another.

Q4: Which evaluation methods can choose the critiques that humans prefer?

  • MetaCritiqueGPT4-F1 achieves the best agreement rate and exceeds SingleGPT4 by 44%; MetaCritiqueOpen also outperforms SingleGPT4 and PairwiseGPT4.The agreement rate measures consistency between a method’s selected critique and the gold-standard choice determined by gold F1 score.Source: Q4: Which evaluation methods can… · pp. 6–7, Figure 3 · p. 7

Q5: Can critique evaluations improve the alignment of existing LLMs?

5 Exp VI: METACRITIQUE Leaderboard

The leaderboard finds AUTO-J strongest among the open-source critique models, while human annotators and GPT-3.5 score higher on precision. The related-work discussion describes the costs and subjectivity of human critique evaluation, the use of correlations in meta-evaluation, and approaches to factuality detection.

Q6: How do various critique models perform?

  • Human annotators and GPT-3.5 exceed 80% precision, outperforming open-source critique models, while AUTO-J leads the open-source group.The authors interpret the finding as highlighting factuality issues in open-source critique models.Source: Q6: How do various critique models… · p. 8

6.1 Critique Evaluation

  • Critique evaluation has relied mainly on human annotators, which entails considerable cost and risks subjectivity.GPT-4 has been used in place of human annotators, but direct numeric scoring lacks a fine-grained calculation explanation.Source: §6.1 · p. 8

6.2 Meta Evaluation

  • Meta-evaluation assesses automated metrics by correlating their scores with human evaluations.The discussion distinguishes rank-based Spearman, linear Pearson, ordinal Kendall’s Tau, and significance tests for comparing correlations.Source: §6.2 · p. 8

6.3 Factuality Detection

  • Factuality detection classifies whether a textual claim is factual, with approaches that assess claims against evidence or generated summaries against source documents.The discussion also notes LLM-based factuality detection without an explicit claim and evidence pair.Source: §6.3 · p. 8

7 Conclusion

The conclusion presents MetaCritique as a quantitative, interpretable critique-evaluation approach based on AIUs, precision, and recall, and reports that selected critiques can lead to better refined outcomes. The authors note limits for recall on creative tasks and call for more exploration of open-source LLMs.

7 Conclusion

  • MetaCritique decomposes critiques into AIUs and uses precision and recall tasks to produce numeric scores with natural-language rationales.The authors describe the approach as quantitative, reliable, and interpretable, and report a meta-evaluation dataset spanning various tasks.Source: §7 · p. 8
  • Superior critiques selected by MetaCritique can result in better refined outcomes, and the system can help identify the pros and cons of critique models.The authors suggest this holds potential to enhance generative artificial intelligence and facilitate future research in LLM critiques.Source: §7 · pp. 8–9

Limitations

  • The recall principle is not suitable for creative tasks, especially when multiple high-quality answers make it difficult to collect all effective critiques.Source: Limitations · p. 9
  • The authors call for further exploration of open-source LLMs to achieve more robust and dependable evaluations.They connect open-source development with reliability and numerical stability through a transparent and verifiable process.Source: Limitations · p. 9

Ethical Considerations

  • The authors report using publicly available Shepherd data, respecting original authors’ intellectual property and privacy rights, and informing and reasonably paying human-evaluation participants.They state that the work complies with the ACL Ethics Policy.Source: Ethical Considerations · p. 9

A Case Study

The case study uses Figure 5 to illustrate how MetaCritique scores are obtained, with further detail on the example in Figure 1.

A Case Study

  • Figure 5 presents a case with more detail than Figure 1 to illustrate how MetaCritique scores are obtained.Source: A Case Study · p. 12

B Gold Score

Human-written critiques receive higher precision but lower recall scores than LLM-generated critiques. The authors interpret this as humans making fewer fictional claims while missing more key information.

B Gold Score

  • Human-written critiques score higher in precision but lower in recall than LLM-generated critiques.Source: B Gold Score · p. 12
  • The authors infer that LLMs are more prone to fictional information, while humans are hardly possible to make such mistakes.Source: B Gold Score · p. 12
  • The authors note that humans miss more key information than LLMs, possibly because human-written critiques contain fewer AIUs.Source: B Gold Score · p. 12

C API Usage at PyPI

The appendix describes a PyPI package for convenient critique assessment and points readers to its Python usage example. It also includes a METACRITIQUE example and a table of gold scores.

C API Usage at PyPI

  • METACRITIQUE is published as a Python package on PyPI so users can assess critiques conveniently.Table 7 is identified as the Python usage example.Source: C API Usage at PyPI · p. 12
  • Figure 5 presents a case of METACRITIQUE for hypothesis critique evaluation.Source: Figure 5 · p. 13
  • Table 6 presents gold scores for METACRITIQUE.Source: Table 6 · p. 13

D Prompts and Instructions

The appendix lists prompts and instructions spanning critique generation, AIU extraction, evaluation, and answer refinement. Its verification prompt instructs the model to judge claims against the question and answer evidence, with explicit reliability guidance.

D Prompts and Instructions

  • The described workflow covers critique generation, AIU extraction, precision and recall tasks, answer refinement, and evaluation of refined answers.Source: D Prompts and Instructions · p. 13
  • The appendix includes prompts for generating reference and hypothesis critiques and extracting AIUs.It uses “claims” instead of “AIUs” in prompts to reduce possible ambiguity and confusion for LLMs.Source: D Prompts and Instructions · p. 13, Table 8 · p. 15, Table 9 · p. 15
  • The verification prompt treats the question as most reliable, the model answer as least reliable, and the reference answer as more reliable than the model answer.It labels a claim true when it is explicitly supported or logically deducible from needed information, and false otherwise.Source: D Prompts and Instructions · p. 16
  • The examples demonstrate claim verification by checking claims against the input question, reference answer, or model answer.Source: D Prompts and Instructions · p. 16
  • The appendix includes prompts for precision and recall tasks, plus instructions for human evaluation of reference answers, reference critiques, and AIU extraction.Source: Table 11 · p. 17, Table 13 · p. 17, Table 14 · p. 18
  • Additional tables provide instructions for pairwise comparison and human evaluation of refined-answer pairs, alongside prompts for critique rating and answer refinement.A separate prompt is identified for GPT-4 evaluation of refined-answer pairs.Source: Table 16 · p. 18, Table 17 · p. 19, Table 18 · p. 19
Loading 2401.04518v2…