PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Factual Consistency of News Summarization by Contrastive Preference Optimization

Huawen Feng, Yan Fan, Xiong Liu, Ting-En Lin, Zekun Yao, Yuchuan Wu, Fei Huang, Yongbin Li, Qianli Ma

arXiv:2310.19347v4cs.CLcs.AI

TL;DR

LLM summaries remain vulnerable to subtle factual hallucinations that traditional methods struggle to detect. The paper proposes CPO with probing-based specific training and reports improved factual consistency and summarization reliability across common LLM backbones. It also constructs the LESSON sentence-level consistency dataset.

  • Problem

    LLMs often generate summaries that are factually inconsistent with source articles, including sophisticated unsupported details, causal claims, and speculation presented as fact.

    Source: §1 · p. 1

  • Method

    The paper combines CPO, which separates faithful and fake generation propensities, with probing-based specific training and the LESSON dataset.

    Source: §1 · p. 2

  • Results

    CPO with probing-based specific training significantly improves factual consistency under both automatic and human evaluation, without sacrificing other summary qualities.

    Source: §4.2 · p. 6, §5 · p. 9

  • Takeaways & Limitations

    The results support improving LLM summarization reliability through CPO and probing-based SFT rather than relying on complex reinforcement-learning frameworks.

    Source: §1 · p. 2, §4.2 · p. 6, §5 · p. 9

    Unnecessary training can reduce performance, especially on in-domain data, so hyperparameter selection is essential.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Despite the recent progress in news summarization made by large language models (LLMs), they often generate summaries that are factually inconsistent with original articles, known as "hallucinations" in text generation. Unlike previous small models (e.g., BART, T5), current LLMs make fewer silly mistakes but more sophisticated ones, such as imposing cause and effect, adding false details, overgeneralizing, etc. These hallucinations are challenging to detect through traditional methods, which poses great challenges for improving the factual consistency of text summarization. In this paper, we propose Contrastive Preference Optimization (CPO) to disentangle the LLMs' propensities to generate faithful and fake content. Furthermore, we adopt a probing-based specific training method to improve their capacity of distinguishing two types of propensities. In this way, LLMs can execute the instructions more accurately and have enhanced perception of hallucinations. Experimental results show that CPO significantly improves the reliability of summarization based on LLMs.

1 Introduction

LLMs improve abstractive summarization but still produce subtle, difficult-to-detect hallucinations. The paper introduces CPO with probing-based specific training to improve factual consistency without relying on reinforcement-learning frameworks.

1 Introduction

  • Traditional correction methods require external resources or costly human revisions, while pre-training approaches demand substantial computation for LLMs.Perturbation-based contrastive methods also fail to match the error distribution of real LLM hallucinations.Source: §1 · pp. 1–2
  • Current LLMs make fewer simple mistakes but more sophisticated errors, including unsupported details, imposed causality, and speculation presented as fact.These errors can combine faithful and fake content, making hallucinations harder to detect.Source: §1 · pp. 1–2
  • CPO disentangles propensities for faithful and fake content, while dynamically probing distinguishing capacity and training target layers through SFT.This design avoids reliance on RL frameworks.Source: §1 · p. 2
  • The paper constructs LESSON, a summarization dataset with sentence-level consistency annotations for training LLMs.The dataset is one of the paper’s three stated contributions.Source: §1 · p. 2
  • CPO with probing-based specific training significantly improves LLM factual consistency without complex annotation or format requirements.The method can be directly employed in supervised fine-tuning (SFT) mode.Source: §1 · p. 2

2 Related Work

Prior factual-consistency evaluation struggles with subtle errors in long LLM-generated summaries, while probing studies reveal useful internal factuality signals but limited applicability to summarization. These limitations motivate methods tailored to long-text generation and LLM behavior.

2.1 Evaluating Factual Consistency

  • Traditional NLI- and QA-based metrics can miss subtle mistakes in longer LLM summaries, limiting factual-consistency evaluation and dataset construction.ChatGPT and GPT-4 provide more accurate assessments, but suitable domain prompts require additional effort.Source: §2.1 · pp. 2–3

2.2 Probing for Truthfulness

  • Probing research finds latent structures related to factuality and shows that models may distinguish true from false outputs while still generating false content.Some methods shift model feature space during inference to improve faithfulness.Source: §2.2 · p. 3
  • Truthfulness-probing methods designed for internal-knowledge benchmarks such as TruthfulQA are unsuitable for long-text summarization.Their target setting differs from source-grounded generation over lengthy documents.Source: §2.2 · p. 3

3 Methodology

The methodology builds LESSON from LLM-generated summaries with sentence-level factuality annotations, then applies Contrastive Preference Optimization and probing-based training to improve reliable instruction following. CPO separates contextual faithfulness from internal-knowledge generation, while probing identifies weak layers for targeted training.

3.1 Sentence-Level Data Collection

  • LESSON contains summaries generated by current decoder-only LLMs and sentence-level factuality labels, with consistency defined against the provided article rather than external facts.The dataset uses GPT-, GLM-, and LLaMA-family outputs on XSum and CNN/DM; any hallucination makes its sentence inconsistent.Source: §3.1 · p. 3
  • Human evaluation found the automatic annotation approach reached 76.70% accuracy versus 64.16% for the best prior method, supporting LESSON’s annotation reliability.The evaluation used 160 LLM-generated summaries and Balanced Accuracy to assess agreement with human labels.Source: §3.1 · p. 4

3.2 Contrastive Preference Optimization

  • CPO uses contextual and internal instructions with Incentive and Penalty Losses to encourage faithful or hallucinatory behavior as instructed while penalizing instruction-disobeying content.The loss selectively rewards or penalizes sentence types, and α balances punishment strength in the total objective.Source: §3.2 · p. 4, Figure 3 · p. 5

3.3 Probing-based Specific Training

  • Probing trains classifiers on layer hidden states to measure factuality awareness, then selects the weakest layers for targeted training rather than updating the whole model.Intermediate layers generally distinguish accurate summaries better than bottom and top layers, motivating dynamic top-k weak-layer selection.Source: §3.3 · pp. 5–6

4 Experiments

Experiments evaluate CPO+PST across multiple LLM backbones, factual-consistency assessments, propensity decoupling, probing, and competing training strategies. The results support improved factual consistency, clearer discrimination between faithful and hallucinated summaries, greater training stability, and strong performance against alternatives.

4.1 Experimental Details

  • Experiments use LESSON with sentence-level annotations and evaluate six instruction-tuned or pretrained backbones using automatic and blinded human assessment.Human evaluation uses 300 test-set articles, with two workers assessing each summary after source masking.Source: §4.1 · p. 6

4.2 RQ1: Does CPO with PST improve LLM’s factual consistency?

  • CPO+PST significantly improves LLM factual consistency under both automatic and human evaluation, while preserving other summary qualities.It also outperforms full-parameter CPO, showing the benefit of probing-based specific training.Source: §4.2 · p. 6, Table 3 · p. 7, Figure 4 · p. 7

4.3 RQ2: Does CPO decouple LLMs’ different propensities successfully?

  • CPO-trained models separate faithful summaries from hallucinated outputs, including reasonable-looking errors such as false details, overgeneralization, and temporal or conceptual substitutions.The comparison uses different generations under instructions that elicit the models’ distinct propensities.Source: §4.3 · p. 7, Table 4 · p. 7

4.4 RQ3: Does Probing-based Specific Training fill the gaps?

  • CPO enhances probing-based discrimination between consistent and inconsistent summaries, especially in originally insensitive bottom and top attention layers.The probing results are reported in Figure 5.Source: §4.4 · p. 7, Figure 5 · p. 8
  • PST maintains high factual consistency across training checkpoints, whereas full-parameter fine-tuning peaks and then declines rapidly.This pattern indicates greater stability and less sensitivity to unnecessary training.Source: §4.4 · p. 8, Figure 6 · p. 8

4.5 RQ4: Is CPO better than other training strategies?

  • CPO and CPO+PST achieve the best factual consistency on most tests, outperforming SFT and Decoupling among the compared strategies.The comparison includes the methods summarized in Table 5.Source: §4.5 · pp. 8–9, Table 5 · p. 8
  • CPO+PST achieves higher consistency than PPO and DPO while offering an SFT-based alternative that avoids their reward-model or paired-data requirements.Compared with LoRA, CPO+PST is also reported as more effective overall under restricted parameter updates.Source: §4.5 · p. 9

5 Conclusion

The paper identifies limitations of existing factual-consistency methods for LLM summarization, introduces LESSON and CPO with Probing-based Specific Training, and reports improved factual consistency across common LLMs. It also calls for further dataset construction and LLM-focused factual-consistency research.

5 Conclusion

  • The paper constructs LESSON and proposes CPO with Probing-based Specific Training to improve factual consistency across common LLMs.The reported experiments demonstrate the effectiveness of the method.Source: §5 · p. 9

Limitations

CPO+PST improves factual consistency across backbones, but unnecessary training can hurt performance, especially on in-domain data; hyperparameter selection is therefore important.

Limitations

  • CPO+PST significantly improves factual consistency across all evaluated backbones, but its gains depend on avoiding unnecessary training.Source: Limitations · p. 9
  • Unnecessary training can reduce CPO+PST performance, especially on in-domain datasets, making hyperparameter selection essential.The authors specifically discuss selecting an appropriate value for hyperparameters in Section 4.2 and Appendix C.Source: Limitations · p. 9

A The details about data collection.

The authors construct LESSON from current decoder-only LLM summaries, filtering malformed outputs and annotating sentence-level consistency with model-based prompts. They combine annotations from ChatGPT and GPT-4 to favor high recall, accepting stricter judgments for training.

A The details about data collection.

B The details of prompts.

  • Sentence-level annotations ask whether each summary sentence is consistent with the corresponding article and require indexed JSON responses.Source: B The details of prompts · p. 14
  • Combining ChatGPT and GPT-4 annotations reaches 76.7% balanced accuracy and favors high recall, although the union can label some true sentences as false.The strict labeling is retained for training because it encourages more rigorous expression; final evaluation uses the annotators separately.Source: B The details of prompts · p. 14, A The details about data collection · p. 14
  • The collection prompts separately request summaries consistent or inconsistent with the article, with a 40-word limit for the seed instructions.Source: B The details of prompts · p. 14

C The number of trainable layers.

The number and choice of trainable layers strongly affect performance: insufficient training underfits, excessive training overfits, and PST's dynamically selected weak layers outperform alternative selections.

C The number of trainable layers.

D The human evaluation criterion.

Human evaluation compares masked pairs of summaries for factual consistency, while allowing annotators to prefer greater comprehensiveness or mark cases where no better summary is clear. The evaluation also reports content-coverage win rates.

D The human evaluation criterion.

  • Content-coverage win rate is reported as a human-evaluation measure for CPO+PST.Source: Figure 8 · p. 15
  • Annotators compare one backbone-generated summary with one CPO+PST summary, choosing the better summary without seeing their sources.The comparison is conducted from the perspective of factual consistency.Source: D The human evaluation criterion · pp. 14–15
  • The evaluation includes cases where a hallucinated summary loses to a factually consistent summary.Source: D The human evaluation criterion · p. 15
  • When both summaries are consistent, annotators prefer the more comprehensive one; when both contain hallucinations, they choose the one with fewer mistakes or mark the case undecidable.Source: D The human evaluation criterion · p. 15

E The human evaluation for coherence and content coverage.

CPO+PST improves content coverage noticeably, while coherence increases only slightly relative to factual consistency. Examples further show that instruction-following outputs can embellish source content through unsupported timing, details, and entity substitutions.

E The human evaluation for coherence and content coverage.

  • Human evaluation measures coherence as orderly relations among parts and content coverage as the number of critical source points included in the summary.Figures 7 and 8 report CPO+PST win rates for these two criteria, and workers select the better output.Source: E The human evaluation for coherence… · p. 15
  • CPO+PST shows a noticeable improvement in content coverage, while its coherence increase is slight compared with factual consistency.The model’s more general rather than detailed summaries may contribute to improved factual consistency and content coverage.Source: E The human evaluation for coherence… · p. 16

F More examples about decoupling.

  • The human-evaluation examples compare outputs under IC and II, revealing differences in temporal claims, added years, and confused place names or entities.II’s hallucinations remain related to the source but embellish it rather than fabricating without any basis.Source: F More examples about decoupling · p. 16

G The head-level probing scores.

The head-level probing analysis summarizes mean, maximum, and minimum scores across attention heads. After CPO with PST, backbone heads score higher, indicating stronger discrimination between consistent and inconsistent summaries.

G The head-level probing scores.

  • Table 11 reports the mean, maximum, and minimum probing scores across all attention heads.These statistics summarize head-level probing performance after training conditions are applied.Source: G The head-level probing scores · p. 16
  • After CPO with PST, backbone attention heads achieve higher probing scores, indicating stronger discrimination between consistent and inconsistent summaries.This conclusion aligns with the visualization in Section 4.2.Source: G The head-level probing scores · p. 17
  • Higher probing scores indicate that attention heads better distinguish consistent from inconsistent summaries after CPO with PST.The analysis therefore connects head-level probing performance with the model’s ability to distinguish the two summary types.Source: G The head-level probing scores · p. 17

Related papers

Loading 2310.19347v4…