PaperlayerView exact revision on arXiv

Source-linked AI summary

Enabling Large Language Models to Generate Text with Citations

Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen

arXiv:2305.14627v2cs.CLcs.IRcs.LG

TL;DR

LLM outputs can hallucinate, making them difficult to trust and verify, while existing citation-evaluation approaches are hard to reproduce and compare. ALCE introduces a reproducible end-to-end benchmark with automatic metrics for fluency, correctness, and citation quality, showing that current systems still have substantial room for improvement and motivating better retrieval, longer context, and multi-source synthesis.

  • Problem

    LLM outputs are prone to hallucination, and reliance on commercial search engines and human evaluation makes citation-generation approaches difficult to reproduce and compare.

  • Method

    ALCE is a reproducible benchmark that evaluates end-to-end systems for retrieving evidence, generating answers, and citing supporting passages using automatic fluency, correctness, and citation-quality metrics.

  • Results

    Current systems have considerable room for improvement, with post-hoc citation approaches achieving good correctness but inadequate citation quality.

  • Takeaways & Limitations

    The analyses identify better retrieval, long-context LLMs, and improved synthesis of multiple sources as promising research directions.

  • Takeaways & Limitations

    The evaluation is limited by unstable MAUVE results, incomplete coverage of open-ended ELI5 answers, and NLI-based citation-quality errors.

Abstract

from arXiv · show

Large language models (LLMs) have emerged as a widely-used tool for information seeking, but their generated outputs are prone to hallucination. In this work, our aim is to allow LLMs to generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search engines and human evaluation, making it challenging to reproduce and compare different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs' Citation Evaluation. ALCE collects a diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve supporting evidence and generate answers with citations. We develop automatic metrics along three dimensions -- fluency, correctness, and citation quality -- and demonstrate their strong correlation with human judgements. Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement -- For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time. Our analyses further highlight promising future directions, including developing better retrievers, advancing long-context LLMs, and improving the ability to synthesize information from multiple sources.

1 Introduction

ALCE introduces a reproducible benchmark for generating and evaluating LLM text with citations. It shows that current systems remain substantially limited in correctness and citation quality despite fluent outputs.

  • LLM outputs can hallucinate, making supporting evidence important for users to verify claims and assess factual correctness.
  • ALCE requires end-to-end systems to retrieve relevant passages, generate answers, and cite supporting evidence across three diverse datasets.
  • Automatic metrics evaluate fluency, correctness, and citation quality, with strong correlation to human evaluation.
  • Around 50% of ChatGPT and GPT-4 generations on ELI5 are not fully supported by cited passages.
  • Current systems face challenges from retrieval quality, limited context windows, and difficulty synthesizing multiple documents without distraction.
  • Experiments identify retrieval improvements, long-context models, and better multi-source synthesis as promising research directions.

2 Task Setup and Datasets

ALCE formalizes citation-supported generation as producing statements linked to passages from a retrieval corpus. Its datasets target factual, long-form questions requiring coverage of multiple aspects and synthesis across sources.

  • Each output contains statements, and each statement cites a list of passages from the retrieval corpus.
  • The benchmark allows at most three citations per statement because additional citations usually do not help.
  • Corpus documents are divided into 100-word passages to ease human verification and fit more retrieved evidence within limited context windows.
  • The selected QA datasets contain factual questions, require long answers covering multiple aspects, and demand synthesis from multiple sources.
  • ASQA uses ambiguous factoid questions with Wikipedia evidence, while QAMPARI requires entity lists drawn from different Wikipedia passages.
  • ELI5 contains diverse how, why, and what questions requiring long answers and multiple passages, using Sphere as its retrieval corpus.
  • ALCE samples 1,000 development examples per dataset and provides no citation-specific training data.

3 Automatic Evaluation

ALCE evaluates generated answers across fluency, correctness, and citation quality using automatic metrics designed for different response properties. Its combined evaluation is robust to shortcut strategies, and automatic citation metrics correlate strongly with human judgments.

  • ALCE measures fluency, correctness, and citation quality as three dimensions of system responses.Citation quality requires support from cited passages and excludes irrelevant citations.
  • MAUVE evaluates fluency for ASQA and ELI5, while QAMPARI is omitted because its short-answer format is consistently followed.MAUVE is mainly used as a sanity check because it is sensitive to output length and style.
  • Correctness uses dataset-specific metrics: exact-match recall for ASQA, precision and recall-5 for QAMPARI, and entailment of generated ELI5 sub-claims.For ELI5, InstructGPT generates sub-claims and TRUE checks whether the output entails them.
  • Citation recall tests whether each statement is supported by at least one cited passage using TRUE-based NLI entailment.The score averages binary statement-level recall across the response and follows the attributable-to-identified-sources framework.
  • Citation precision identifies irrelevant citations, defined as citations that neither support the statement nor contribute to support from the remaining citations.The evaluation does not require a minimal citation set because redundant sources can enhance credibility.
  • Shortcut responses can achieve almost-perfect citation scores while receiving low fluency or correctness because they are unnaturally long or insufficiently comprehensive.This demonstrates why ALCE combines multiple evaluation dimensions rather than relying on citation scores alone.

4 Modeling

ALCE models retrieve evidence, synthesize answers with citations, and optionally post-edit outputs using prompting strategies that accommodate limited context windows. The explored methods include passage compression, interactive checking, inline search, reranking, and post-hoc citation.

  • 4 Modeling: ALCE systems comprise three modeling components: retrieval, synthesis, and post-editing.The benchmark explores these components with off-the-shelf retrievers and prompted LLMs rather than fine-tuning internal parameters.
  • 4.1 Retrieval: Retrieval uses GTR and DPR for Wikipedia, BM25 for Sphere, and the top-100 passages for each question.
  • 4.2 Synthesis: VANILLA provides the model with top-k passages and instructs it to cite them, using two in-context demonstrations.
  • 4.2 Synthesis: SUMM/SNIPPET compresses retrieved passages so more evidence fits within context, but the compression is lossy.With a 4K context window, five passages cover only 56.8% of ASQA answers.
  • 4.2 Synthesis: INTERACT lets the model check selected documents, output answer statements, or end generation to mitigate lossy passage compression.
  • 4.2 Synthesis: INLINESEARCH allows retrieval during generation, displaying the best retrieved passage and removing it after one action to save context space.
  • 4.3 Post-editing: Post-editing includes reranking four sampled responses by citation recall and attaching best-matching passages to closed-book statements.

5 Experiments

Experiments show that retrieval quality, context use, and prompting strategy jointly shape correctness and citation quality. Current systems remain limited in synthesizing and citing information from multiple retrieved passages.

  • Main results: All tested models achieve good fluency, but correctness and citation quality still have substantial room for improvement.
  • Main results: VANILLA achieves close-to-the-best performance among prompting strategies despite simply placing retrieved passages in context.
  • Prompting strategies: Summaries or snippets universally improve correctness, but lossy compression can reduce citation quality on ASQA and ELI5.
  • Prompting strategies: VANILLA outperforms INLINESEARCH on citation quality across datasets, and on correctness for ASQA and ELI5.
  • Prompting strategies: RERANK consistently improves citation quality on ASQA and ELI5, with human evaluation verifying its effectiveness despite possible automatic-score bias.
  • Main results: CLOSEDBOOK has strong correctness but poor citation quality after POSTCITE; its citation recall is 47% lower than VANILLA on ASQA.
  • Models and context: GPT-4 gains limited correctness improvements overall but benefits from more passages, unlike ChatGPT-16K; instruction-tuned models improve correctness and citation quality over original LLaMA.
  • Retrieval and context: Retrieval recall rises with more passages, yet ChatGPT correctness plateaus at top-1 and citation quality at top-3, revealing limited multi-passage use.

6 Human Evaluation

Human evaluation measures utility, citation recall, and citation precision, and broadly validates ALCE’s automatic citation-quality metrics. Models produce useful, fluent answers even when factual correctness differs.

  • Evaluation design: The human evaluation scores utility, citation recall, and citation precision for selected model generations.
  • Human evaluation: Human evaluators judge utility similarly across models, with scores of 3.7-3.9 for ASQA and 3.5-3.6 for ELI5.
  • Human evaluation: Models generally produce fluent answers related to the question despite differences in factual correctness.
  • Agreement with humans: ALCE’s automatic citation-quality metrics strongly correlate with human judgements and produce consistent relative model rankings.

7 Related Work

Prior work explored citation generation and retrieval-augmented language models, but often relied on closed systems or supplied citations. ALCE instead evaluates reproducible end-to-end retrieval, synthesis, and citation.

  • Related tasks: Scientific citation text generation provides papers to cite and asks models to recover citing text, unlike ALCE’s retrieval-and-citation setting.
  • Retrieval-augmented LMs: Retrieval-augmented language models incorporate retrieved information but generally do not explicitly provide citations to retrieved sources.
  • Citation evaluation: Earlier citation-generation studies mainly used commercial search engines and closed-source models, limiting reproducibility and comparison.
  • ALCE’s distinction: ALCE is presented as the first end-to-end setting that retrieves, synthesizes, and cites documents with LLMs.

8 Conclusion

ALCE is an automatic benchmark for evaluating LLM generations with citations across fluency, correctness, and citation quality. Experiments show substantial room for improvement and identify retrieval, long context, and multi-source synthesis as research directions.

  • Contribution: ALCE is the first automatic benchmark for evaluating LLM generations with citations.
  • Evaluation: ALCE evaluates fluency, correctness, and citation quality with automatic metrics whose efficacy is verified through human evaluation.
  • Conclusion: Experiments demonstrate that current systems have considerable room for improvement on ALCE.
  • Future directions: Promising research directions include better retrieval integration, long-context LLMs, and improved synthesis of multiple sources.
  • Future directions: ALCE can serve as a testbed for developing these directions beyond its original benchmark setup.

Limitations

The evaluation has acknowledged weaknesses in metric stability, coverage of open-ended answers, and automatic citation-precision judgments.

  • MAUVE is sensitive to output length and may produce unstable results.
  • ELI5’s automatically generated correctness claims may not cover all possible answers because its questions are open-ended.
  • The citation-quality evaluation depends on NLI-model accuracy, and citation precision is biased downward when citations partially support statements.
  • The curated datasets omit challenging scenarios including multi-hop reasoning, mathematical reasoning, and code completion.
  • Experiments use prompting without weight updates, while training citation-generating models remains future work because supervised data is scarce.

A Generating Claims for ELI5

The ELI5 correctness metric uses generated sub-claims rather than ROUGE-L, then checks whether model outputs entail those claims. Manual inspection found the generated claims largely relevant and faithful.

  • ROUGE-L can be gamed by returning a top retrieved passage and does not reward coverage of different answer aspects.
  • The method generates sub-claims from original answers to estimate the different aspects that model outputs should cover.
  • InstructGPT generated three sub-claims per ELI5 answer, guided by manually annotated demonstrations.
  • 93.33% of 120 inspected sub-claims were relevant and faithful to the ground-truth facts.
  • The generated sub-claims averaged 14 words and were typically one sentence long, matching the metric’s focus on short factual claims.
  • The shortcut analysis shows that top-passage outputs can achieve near-perfect citation quality while having dramatically lower fluency and correctness.
  • Citation precision falsely penalizes citations that partially support a statement when other citations cover the same information.

F Human Evaluation

Human evaluation measures response utility, citation recall, and citation precision by presenting workers with questions, generated statements, and their citations. Additional experiments compare prompting strategies and a trained FiD baseline.

  • Human evaluation sampled 100 ASQA and ELI5 examples and assessed outputs from ChatGPT VANILLA, ChatGPT RERANK, and Vicuna-13B VANILLA.
  • F.1 Utility: Workers rate response utility on a 1–5 Likert scale after seeing the question and model response.
  • Citation recall is the average across statements of whether each statement’s joint citations fully support all its claims.
  • Citation precision averages whether individual citations fully or partially support the factual claims in their statements.
  • The full instruction improves correctness and citation recall over the short instruction, while citation precision remains similar.
  • At least one demonstration ensures high citation recall for ChatGPT, while additional demonstrations improve original LLaMA performance.
  • A trained FiD baseline significantly lags behind prompting ChatGPT in correctness and citation quality, including when evaluated out of domain on ELI5.

G.5 More Human Evaluation

Further analyses validate automatic metrics against human annotations and examine model, prompting, and experimental settings. Results show broad performance differences across model scales and approaches.

  • ALCE achieves 85.1% accuracy for citation recall and 77.6% for citation precision against human annotations.
  • For insufficient citations, ALCE reaches 82.3% recall and 84.2% precision; for irrelevant citations, it reaches 75.6% recall and 66.1% precision.
  • The NLI model’s inability to detect partial support causes a relatively high false-positive rate for irrelevant-citation detection.
  • ChatGPT RERANK and costly ChatGPT-16K and GPT-4 evaluations use one seeded run rather than the three runs used elsewhere.
  • Open-source models generally trail GPT-4 in correctness and citation quality, although LLaMA-2-70B-Chat and Stable Beluga 2 can sometimes match its correctness.
  • Across open-source models, SUMM and SNIPPET improve correctness, while RERANK improves citation quality.
Loading 2305.14627v2…