PaperlayerView exact revision on arXiv

Source-linked AI summary

TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models

Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, Idan Szpektor

arXiv:2305.11171v3cs.CL

TL;DR

NLI models have limited success at evaluating factual consistency in summaries, while synthetic data based on perturbed human summaries can have limited error coverage and differ from model-generated summaries. TrueTeacher labels diverse model-generated summaries with an LLM to create training data. The paper reports improved performance, robustness to domain shift, and multilingual generalization, while noting resource costs and possible label noise.

  • Problem

    NLI models have limited success on summary factual consistency, and perturbed human-written summaries can have limited error coverage and differ in style from model-generated summaries.

    Source: §1 · pp. 1–2

  • Method

    TrueTeacher generates diverse model-written summaries and uses an LLM to label their factual consistency for student-model training.

    Source: §1 · p. 2

  • Results

    TrueTeacher substantially enhances a state-of-the-art model’s performance, with reported effectiveness and robustness across domain shift and multilingual scenarios.

    Source: §4.5 · p. 8, §7 · p. 9

  • Takeaways & Limitations

    The study highlights the importance of out-of-domain evaluation and releases a large-scale synthetic dataset and model checkpoint.

    Source: §7 · p. 9

    Labeling 1.4M summaries with a 540B LLM requires non-negligible resources that may not be available to the whole community.

    Source: §8 · p. 10

Abstract

from arXiv · show

Factual consistency evaluation is often conducted using Natural Language Inference (NLI) models, yet these models exhibit limited success in evaluating summaries. Previous work improved such models with synthetic training data. However, the data is typically based on perturbed human-written summaries, which often differ in their characteristics from real model-generated summaries and have limited coverage of possible factual errors. Alternatively, large language models (LLMs) have recently shown promising results in directly evaluating generative tasks, but are too computationally expensive for practical use. Motivated by these limitations, we introduce TrueTeacher, a method for generating synthetic data by annotating diverse model-generated summaries using a LLM. Unlike prior work, TrueTeacher does not rely on human-written summaries, and is multilingual by nature. Experiments on the TRUE benchmark show that a student model trained using our data, substantially outperforms both the state-of-the-art model with similar capacity, and the LLM teacher. In a systematic study, we compare TrueTeacher to existing synthetic data generation methods and demonstrate its superiority and robustness to domain-shift. We also show that our method generalizes to multilingual scenarios. Lastly, we release our large scale synthetic dataset (1.4M examples), generated using TrueTeacher, and a checkpoint trained on this data.

1 Introduction

TrueTeacher trains a student on LLM-labeled, model-generated summaries to address limitations in NLI evaluation and synthetic data based on perturbed human summaries. The paper reports improved benchmark performance, robustness to domain shift, multilingual gains, and a released dataset and model.

1 Introduction

  • TrueTeacher annotates diverse model-generated summaries with an LLM, avoiding reliance on perturbed human-written summaries.The approach is motivated by perturbations’ limited error coverage and possible label errors, and by style differences between human-written and model-generated summaries.Source: §1 · p. 2
  • ROC-AUC rises from 82.7 to 87.8 on the TRUE summarization subset, and the student outperforms its ×50-larger LLM teacher.The result is reported for a student trained by augmenting existing NLI data with TrueTeacher data.Source: §1 · p. 2
  • TrueTeacher generalizes to documents from new domains, while existing synthetic-data approaches fail to generalize across document distributions.The paper describes a systematic, fair comparison of existing methods in a challenging setting.Source: §1 · p. 2
  • TrueTeacher-generated multilingual data improves results on 35 of 45 mFACE languages.The authors present this as evidence for the usefulness of multilingual synthetic data and its generation method.Source: §1 · p. 2
  • The contributions include a human-evaluated release of 1.4 million examples and a state-of-the-art consistency model trained on that data.The paper also reports evaluating FLAN-PaLM 540B and distilling its knowledge into a significantly smaller model.Source: §1 · p. 2

2 TrueTeacher

TrueTeacher creates factual-consistency training examples from summaries generated by a diverse collection of summarization models and labeled by an LLM. This design uses model outputs rather than perturbed human-written summaries and does not require gold summaries.

2 TrueTeacher

  • Model-generated summaries can be more realistic and may include rare factual errors that are harder to incorporate with perturbation logic.The authors hypothesize that a strong-performing LLM can label these summaries with sufficient quality for training.Source: §2 · p. 3
  • The method fine-tunes summarization models of different capacities and uses them to generate summaries for documents.Different pretrained models diversify expected consistency errors, while training-set choice can shape summary characteristics such as abstractiveness.Source: §2 · p. 3
  • An LLM labels each model-generated summary for factual consistency against its source document.The document, summary, and label form a synthetic example for training a factual-consistency classifier.Source: §2 · p. 3
  • TrueTeacher does not require gold summaries, so it can use any document collection as its source.The paper describes this as making the approach more scalable than prior methods.Source: §2 · p. 3
  • The method can be applied to non-English languages using a multilingual LLM.The paper contrasts this with prior approaches that often rely on language-specific components.Source: §2 · p. 3

3 Experimental Setup

The experiments instantiate TrueTeacher with XSum-trained T5 models, CNN/DailyMail documents, and FLAN-PaLM 540B as the labeling teacher. Evaluation uses ROC-AUC on the summarization subset of TRUE and compares both trained evaluators and synthetic-data generation methods.

3.1 TrueTeacher Instantiation

  • Experiments use XSum training data, five T5 sizes from small to 11B, and CNN/DailyMail documents to generate summaries.FLAN-PaLM 540B supplies consistency labels using zero-shot prompting; few-shot and chain-of-thought prompting did not improve performance in early experiments.Source: §3.1 · p. 3
  • The instantiated process yields about 1.4M synthetic training examples, which are used to train a factual-consistency student.The authors assess data quality with human evaluation and model usefulness and comparative performance in later experiments.Source: §3.1 · p. 4
  • Self-verification filtering increased labeling accuracy but did not improve student performance, so experiments use the full dataset.The authors report this as an early experiment and choose the full data for simplicity.Source: §3.1 · p. 4

3.2 Evaluation

  • Evaluation uses ROC-AUC for binary classification on the TRUE summarization subset: MNBM, FRANK, SummEval, QAGS-X, and QAGS-C.The setup follows Honovich et al.’s TRUE benchmark evaluation.Source: §3.2 · p. 4

3.3 Baselines

  • The study compares trained consistency evaluators with leading TRUE models and compares TrueTeacher’s generation mechanism with synthetic-data methods.The listed evaluator baselines include QuestEval, Q2, SUMMACZS, T5-11B fine-tuned on ANLI, WeCheck, and the TRUE benchmark ensemble; generation baselines include DocNLI, FactCC, FactEdit, and Falsesum.Source: §3.3 · p. 4

4 Experiments and Analysis

The experiments compare TrueTeacher with existing consistency models and synthetic-data methods, then examine qualitative examples, label quality, ablations, and summary abstractiveness. The results show strong benchmark performance and robustness to domain shift, with gains linked to both model-generated summaries and improved labeling.

4.1 Main Results on the TRUE Benchmark

  • 87.8 ROC-AUC for T5-11B with TrueTeacher + ANLI exceeds the 82.7 ANLI-only baseline and the 84.9 ROC-AUC of the 540B LLM teacher.The teacher scores 84.9 versus 83.0 for the best single-model baseline; both comparisons use the TRUE summarization benchmark.Source: §4.1 · p. 5, Table 2 · p. 5

4.2 Re-evaluating Synthetic Data Generation Methods – A Study

  • TrueTeacher is the only method that consistently beats the ANLI-only baseline on TRUE, and it is the only method with a higher out-of-domain than corresponding ANLI-only score.The study restricts training examples to CNN/DM documents and evaluates out of domain on XSum-based sets; TrueTeacher’s model-generated summaries may contribute to its robustness.Source: §4.2 · pp. 5–6

4.3 Qualitative Analysis

  • In the sampled case study, perturbation methods sometimes introduced grammatical errors or failed to create factual errors, whereas TrueTeacher’s model-generated summary contained a nuanced factual error and hallucinated content.The figure marks replacements with matching colors and added content in bold red.Source: §4.3 · pp. 6–7, Figure 3 · p. 7

4.4 Human Evaluation

  • 89% of the 100 sampled TrueTeacher examples received the correct human-evaluation label.Domain experts made binary judgments using the attribution definition from Rashkin et al. (2021); Table 4 also reports precision, recall, and F1 for both classes.Source: §4.4 · p. 7, Table 4 · p. 7

4.5 Ablating Summary Distribution and Label Correctness

  • LabelAblation improves over Falsesum + ANLI by 5.6%, while SummaryAblation improves by 5.8%; combining both differences yields 86.4 ROC-AUC.The ablations indicate similar effects for label correctness and summary distribution, with a complementary effect when combined.Source: §4.5 · p. 8, Table 5 · p. 8

4.6 Abstractiveness Analysis

  • TrueTeacher produces the most abstractive summaries, with the lowest combined abstractiveness score among the evaluated methods.The combined score multiplies extractive fragment coverage and density; lower scores indicate greater abstractiveness.Source: §4.6 · p. 8, Table 6 · p. 8

5 Multi-Lingual Data Generation for Factual Consistency Evaluation

The multilingual experiment generates and labels summaries in four languages, then trains multilingual consistency models. Adding multilingual TrueTeacher data improves average ROC-AUC more than an equal amount of English-only data.

5 Multi-Lingual Data Generation for Factual Consistency Evaluation

  • The synthetic training data uses mT5-XXL summaries from WikiLingua documents in English, French, Spanish, and German, labeled by an LLM.The experiment samples 25,000 balanced-label examples per language and trains multilingual consistency models.Source: §5 · pp. 8–9
  • 75.3 average ROC-AUC with multilingual synthetic data exceeds the 73.8 achieved with the same amount of English-only data and the 71.6 baseline.English-only data improves results on 32 of 45 mFACE languages; the multilingual evaluation covers 45 languages.Source: §5 · p. 9, Table 7 · p. 9

6 Related Work

Prior work explored synthetic data, LLM-based evaluation, and ways to adapt NLI models for summarization. TrueTeacher uses model-generated summaries and LLM annotations, a simpler and inherently multilingual approach to factual-consistency data generation.

6 Related Work

7 Conclusion

TrueTeacher generates synthetic factual-consistency data by having an LLM annotate realistic model-generated summaries rather than perturbing human-written summaries. The paper reports improved model performance, robustness to domain shift, and multilingual generalization.

7 Conclusion

  • TrueTeacher generates synthetic data by prompting an LLM to annotate realistic model-generated summaries instead of perturbing human-written summaries.The approach is presented as a simple and effective method for factual-consistency evaluation.Source: §7 · p. 9
  • The authors report that the dataset substantially enhances a state-of-the-art model and that their systematic comparison demonstrates effectiveness and robustness.The conclusion highlights out-of-domain evaluation and reports generalization to multilingual scenarios.Source: §7 · p. 9
  • The authors make the large-scale synthetic dataset publicly available.The conclusion also reports release of a checkpoint trained on the data.Source: §7 · p. 9

8 Limitations

The approach may produce incorrectly labeled synthetic examples, and labeling its 1.4 million summaries with a 540B LLM requires substantial resources. The multilingual experiments cover four languages, leaving the choice of language coverage for future study.

8 Limitations

  • LLM labeling may introduce incorrect synthetic labels that affect student-model quality, and the authors recommend rechecking data quality when generating examples for new domains.Filtering for higher labeling accuracy did not improve student-model performance in these experiments.Source: §8 · p. 10
  • Labeling 1.4 million model-generated summaries with a 540B LLM requires non-negligible resources that may not be available to the whole community.The authors release the dataset and checkpoint to mitigate this constraint.Source: §8 · p. 10
  • The multilingual experiments focus on English, French, Spanish, and German, and do not establish how many languages should be included in synthetic training data.The authors note a possible tradeoff between language coverage and LLM labeling quality, especially for low-resource languages.Source: §8 · p. 10

A.1 FLAN-PaLM Prompt Design

The authors tested zero-shot, few-shot, and chain-of-thought prompting strategies and formats, then selected the best-performing option on a development set. They chose the simplest option, zero-shot, after observing only minor performance differences.

A.1 FLAN-PaLM Prompt Design

  • Zero-shot, few-shot, and chain-of-thought prompts and formats were compared on development-set accuracy; the authors chose zero-shot because performance differences were minor and it was simplest.The development set uses FactCC examples, with its dev and test sets merged following Utama et al. (2022).Source: §A.1 · p. 14
  • The zero-shot prompt frames the document as a premise and the summary as a hypothesis, asking whether the hypothesis can be inferred from the premise.The prompt requests a yes-or-no answer.Source: §A.1 · p. 14
  • Few-shot prompting uses two development-set examples, one consistent and one inconsistent, sampled from examples shorter than 200 words to limit context length.The examples are randomly sampled.Source: §A.1 · p. 14

A.2 Inference with FLAN-PaLM

FLAN-PaLM’s zero-shot responses were mostly binary, with only a tiny fraction marked impossible to say. The procedure maps these responses to consistency labels and uses the probability of “Yes” as a factual consistency score.

A.2 Inference with FLAN-PaLM

  • The vast majority of FLAN-PaLM’s zero-shot responses were “Yes” or “No,” with only a tiny fraction marked “It’s impossible to say.”Source: §A.2 · p. 15
  • The procedure labels “Yes” as consistent and “No” as inconsistent, discarding “It’s impossible to say” responses.For ROC-AUC, it uses the model’s probability of generating “Yes” as the example-level factual consistency score.Source: §A.2 · p. 15

A.3 Fine tuning T5

This appendix specifies T5 training and dataset procedures, including self-verification filtering, multilingual evaluation details, human-rating practices, and a label-noise comparison setup.

A.3 Fine tuning T5

  • T5 factual-consistency models use document-summary inputs, predict 1 for consistent and 0 otherwise, and select checkpoints by development-set ROC-AUC.Training uses a 10−4 learning rate, batch size 32, and maximum input length 512; inference allows 2048 tokens, with evaluation every 1k steps for up to 20 epochs.Source: §A.3 · p. 15

A.4 Additional Details About Our Dataset

  • The dataset uses CNN/DailyMail training documents without gold summaries; full-dataset experiments balance labels by sampling 475,563 positive examples.Source: §A.4 · p. 15

A.5 Data Filtering with Self-verification

  • Self-verification targets examples labeled consistent, reflecting the authors’ finding that the LLM’s extremely high precision for inconsistent examples left most labeling errors among consistent predictions.The procedure asks the LLM to verify its earlier judgment and filters examples when it is uncertain; it filtered out 15% of the dataset, and qualitative analysis suggested labeling accuracy increased by approximately 5%.Source: §A.5 · p. 16
  • 86 ROC-AUC on TRUE was obtained with filtered TrueTeacher + ANLI data, slightly below 86.4 with unfiltered data, so the official method omits filtering.Filtering raised labeling accuracy by approximately 5%, but the authors attribute the lack of performance gain to high initial labeling accuracy and model robustness to some labeling noise.Source: §A.5 · p. 16

A.6 Abstractiveness Analysis: Additional Details

  • Extractive Fragment Coverage and Density quantify summary extractiveness, and their product is the paper’s combined score.Coverage measures the share of summary words in extractive fragments; Density measures the average fragment length associated with each summary word.Source: §A.6 · p. 16

A.7 Using the mFace dataset

  • The mFace evaluation uses XLSum test summaries rated for attribution, treating average attribution scores above 0.5 as factually consistent.The test set contains 3,150 examples across 45 languages, with 70 examples per language.Source: §A.7 · p. 17

A.8 Human Evaluation

  • Human raters were NLP researchers with at least one year of factual-consistency evaluation experience, and each reviewed no more than 10 examples.The authors report that one rater per example was sufficient, unlike their crowd-worker experiments.Source: §A.8 · p. 17, §A.9 · p. 17

A.9 Adding noise to TrueTeacher

  • SummaryAblation approximates Falsesum’s estimated 83.5% labeling accuracy by adding 5.5% errors to TrueTeacher labels.The estimate averages Falsesum’s Intrinsic and Extrinsic human-evaluation results; TrueTeacher’s reported accuracy is 89%.Source: §A.9 · p. 17

Related papers

Loading 2305.11171v3…