Source-linked AI summary
Evaluating Factuality in Generation with Dependency-level Entailment
Tanya Goyal, Greg Durrett
TL;DR
Text generation models can produce fluent but factually inconsistent outputs, and sentence-level entailment methods neither match generation filtering well nor localize errors. This paper decomposes entailment into dependency-arc judgments, automatically derives training data, and finds better factuality classification than sentence-level and question-generation methods while localizing errors.
Problem
Existing generation models produce factual errors, while sentence-level entailment methods are mismatched to generation filtering and do not localize non-factual output.
Method
The paper evaluates whether each generated dependency arc is supported by the input and automatically derives arc-level labels from existing entailment or paraphrase data.
Results
DAE outperforms sentence-level entailment and question-generation models in summarization and paraphrasing, with predicted arc scores supporting error localization.
Takeaways & Limitations
Dependency-level judgments provide a finer-grained and interpretable way to identify factual inconsistencies in generated text.
Takeaways & Limitations
The authors have not directly compared DAE with a sentence-level model trained on the same sentences, and DAE cannot detect information missing from the output.
Abstract
from arXiv · showhide
Despite significant progress in text generation models, a serious limitation is their tendency to produce text that is factually inconsistent with information in the input. Recent work has studied whether textual entailment systems can be used to identify factual errors; however, these sentence-level entailment models are trained to solve a different problem than generation filtering and they do not localize which part of a generation is non-factual. In this paper, we propose a new formulation of entailment that decomposes it at the level of dependency arcs. Rather than focusing on aggregate decisions, we instead ask whether the semantic relationship manifested by individual dependency arcs in the generated output is supported by the input. Human judgments on this task are difficult to obtain; we therefore propose a method to automatically create data based on existing entailment or paraphrase corpora. Experiments show that our dependency arc entailment model trained on this data can identify factual inconsistencies in paraphrasing and summarization better than sentence-level methods or those based on question generation, while additionally localizing the erroneous parts of the generation.
1 Introduction
Generation models can produce fluent text that contradicts their inputs, while existing factuality metrics and sentence-level entailment systems have important limitations. The paper proposes dependency arc entailment to make finer-grained factuality judgments and localize generation errors.
- Pre-trained generation models are fluent but prone to factual errors that contradict the input text.
- ROUGE and BERTScore do not correlate with factual consistency, while entailment-based filters face dataset bias and training–test mismatch.
- Dependency arc entailment evaluates whether each generated dependency relation is supported by the source rather than making only a sentence-level decision.
- Independent arc judgments help localize generation errors and provide more interpretable model decisions.
- Experiments in summarization and paraphrasing show that automatically derived supervision enables DAE to outperform sentence-level entailment and question-generation models while supporting error localization.
2 Dependency Arc Entailment (DAE)
DAE defines entailment over dependency arcs, treating each target relation as entailed when its semantic relationship is supported by the source. Because arc annotation is difficult, the paper automatically derives training data from existing generation and paraphrase resources.
- DAE treats a dependency arc as entailed when the semantic relationship between its head and child is entailed by the source sentence.
- Unlike three-way NLI, DAE is a two-class task in which both neutral and contradictory arcs are labeled non-entailed.
- Human arc annotation is difficult because entailment can be subjective, unknowable, or undefined, and annotators must interpret dependency-label semantics.
- The method automatically labels data from existing sources and generation outputs to obtain large-scale supervision across domains.
- The automatically derived data primarily captures symmetric entailment relations, which more closely match acceptable generation behavior than forward-only entailment.
3 Model
The DAE model encodes the input and hypothesis, represents each hypothesis dependency arc using its endpoint tokens and label, and predicts an entailment probability for every arc. Arc scores can then be combined into a sentence-level factuality score for reranking.
- For each hypothesis arc, the model predicts an entailment decision with respect to the input context.
- A pre-trained encoder contextualizes the concatenated input and hypothesis before producing representations for dependency arcs.
- Each arc representation uses the contextual embeddings of its head and child together with an embedding of the dependency label.
- A linear layer followed by softmax produces entailment-label probabilities for each dependency arc.
- The network trains with binary cross entropy on entailed or non-entailed arc labels and can use partial arc annotations.
- Sentence-level factuality is computed by combining arc scores and is used to rerank generated hypotheses.
4 Automatic Dataset Creation
The paper automatically constructs dependency-arc entailment data from paraphrase corpora and paraphrase-model generations, avoiding manual arc annotation. The procedure combines reliable positive arcs with noisy negative examples while leaving uncertain arcs unlabeled.
- The dataset represents each example using an input, hypothesis, dependency arc, and gold entailment label.
- Gold paraphrases provide positive labels by treating every target-side arc as entailed by the source.This propagates sentence-level paraphrase judgments to arc-level labels and supplies diverse linguistic phenomena.
- Although the DAE definition treats an output as non-factual when any arc is non-entailed, experiments use mean-pooling because min-pooling was unstable.Positive arcs from the 1-best hypothesis are also included with a slightly different training distribution.
- Paraphrase-model generations provide realistic candidate errors, including unseen arcs that may be positively or negatively entailed.The desired data distribution includes phenomena such as passivization, reordering, synonym replacement, wrong arguments, and hallucinated content.
- The labeling procedure marks arcs found in the input or gold paraphrase as entailed and otherwise labels selected bottom-beam arcs as non-entailed.The approach assumes top-beam outputs are more factual and bottom-beam outputs are more error-prone.
- Arcs introduced by the best hypothesis but absent from the input and gold paraphrase remain unlabeled and are ignored during training.These arcs may still be factual, so the method avoids assigning them a confident label.
5 Intrinsic Evaluation of DAE
The intrinsic evaluation tests whether automatically derived dependency-arc labels form a high-quality training resource and whether DAE models classify entailment beyond simple matching. Results show that transformer DAE models outperform baselines, while auto-derived data covers a broader range of entailment phenomena than rule-based alternatives.
- 5 Intrinsic Evaluation of DAE: The evaluation examines dataset quality, DAE classification, and whether alternative synthetic data sources improve coverage of generation errors.The experiments include paraphrase-based generation, synonym replacement, word swapping, and hallucination-focused data.
- 5.1 Dataset Quality: 82% agreement with human annotations indicates that the automatic labeling strategy produces a high-quality DAE dataset.Manual annotations covered 100 sentence pairs from the development set.
- 5.3 Other Data Generation Methods: Rule-based word-swapping data performs well on similarly constructed held-out data but fails to generalize to synonym and auto-derived test sets.This indicates that artificially introduced errors do not cover the full space of generation possibilities.
- 5.3 Other Data Generation Methods: The auto-derived dataset performs well across artificial and actual generation data, covering a larger range of entailment possibilities than word-swapping data.Adding synonym- and hallucination-specific examples further improves the corresponding test sets while retaining generic entailment performance.
6 Extrinsic Evaluation: Filtering Bad Generations
The paper evaluates dependency arc entailment for filtering factual errors in summaries and paraphrases. DAE performs competitively with or better than sentence-level and question-answering approaches, while its reranking behavior differs from intrinsic entailment performance.
- DAE is evaluated for rejecting erroneous generations in summarization and paraphrasing.
- Summarization: 373 summarization test samples pair one factual and one incorrect summary sentence for reranking evaluation.
- Summarization: DAE outperforms sentence-level NLI and QAGS on summary reranking and approaches reported human performance.
- Summarization: The MNLI model’s intrinsic performance can improve while summary reranking varies substantially, unlike the more stable DAE reranking performance.
- Paraphrasing: In paraphrase reranking, MNLI and DAE perform similarly, while DAE is biased toward entailment for arcs shared with the input.
7 Analysis
The analysis compares dependency-level and sentence-level supervision and examines how DAE localizes factual errors. It also identifies scope limitations involving missing information, noisy data, and incomplete comparisons.
- Dependency- vs. sentence-level modeling: The study lacks a direct apples-to-apples comparison of DAE and sentence-level models trained on the same sentences.
- Dependency- vs. sentence-level modeling: On similarly sized datasets, dependency-level modeling outperforms sentence-level modeling for both considered data sources.
- Qualitative Evaluation: DAE can localize erroneous subject-object pairs and distinguish some rewriting types, but accuracy varies across rewriting changes.
- Limitations: Because dependency arcs omit negation and quantification, these phenomena must be handled through hypothesis context.
- Limitations: DAE cannot detect information missing from the input-output arc set, so an incomplete hypothesis may still receive a high score.
- Limitations: Training relies on noisy PARANMT-50M backtranslation data, and better paraphrase pairs are expected to improve model quality.
8 Related Work
The paper situates DAE among structured-information, multitask, and post-processing approaches to factuality. Its dependency-level model can rank or filter outputs from black-box generators while also localizing errors.
- Prior factuality work uses structured representations, multitask training, or post-generation filtering and ranking.
- DAE can rank or filter outputs from any generation model without additional augmentation or retraining.
- Compared with NLI-based post-processing, dependency-level models additionally localize errors in generated text.
9 Conclusion
The paper concludes that dependency arc entailment identifies factual errors more finely than prior approaches and provides interpretable error analysis. Its analysis focuses on a selected subset of semantically connected dependency arcs.
- The proposed dependency arc entailment formulation identifies factual errors in generated text at a finer granularity.
- The approach outperforms prior approaches while providing interpretable error analysis.
- The analysis restricts evaluation to dependency arcs considered more strongly connected to semantics and excludes specified function-word relations.
B Examples from Synonym Test Set
The augmented AD + S model is evaluated on held-out synonym examples, where it assigns entailment probabilities to individual dependency arcs and recognizes some synonym and phrasal substitutions as entailed.
- Adding synonym data to the auto-derived training data improves performance on the held-out synonym-only test set.
- The model learns some bias to recognize synonym replacements and small phrasal substitutions as entailed by the input.
- The example paraphrase changes “complicated” to “complex” and “multiple” to “many.”
- The AD + S model assigns entailment probabilities to individual arcs in held-out synonym examples.Figure 6 presents predictions from the model trained with auto-derived data augmented by synonym data.
C Implementation Details
The DAE and entailment models are fine-tuned from BERT and ELECTRA encoders with manually tuned learning rates, using the best development-set models.
- DAE models are fine-tuned from 110M-parameter BERT and ELECTRA encoders using five learning-rate trials.The models with the best development-set accuracy are selected.
- Tables 7 and 8 report the final hyperparameters for the DAE and entailment models, respectively.
- Entailment models are fine-tuned from 110M-parameter BERT and ELECTRA encoders on MNLI using three learning-rate trials.The models with the best development-set accuracy are selected.
- 84.5% and 89.0% are the reported development accuracies for the BERT and ELECTRA models, respectively.