PaperlayerView exact revision on arXiv

Source-linked AI summary

CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation

I-Hung Hsu, Zifeng Wang, Long T. Le, Lesly Miculicich, Nanyun Peng, Chen-Yu Lee, Tomas Pfister

arXiv:2406.05365v2cs.CLcs.AIcs.LG

TL;DR

Grounded generation seeks credible, accountable answers with verifiable citations, but single-run and preprocessing methods remain prone to errors. CaLM verifies a large LM’s answer using a smaller LM that sees only cited documents, then iteratively refines discrepancies. Across three open-domain QA datasets, it reports average gains of 1.5% to 7% over state-of-the-art methods in answer accuracy and citation quality.

  • Problem

    Grounded generation requires LMs to identify relevant documents and accurately ground answers in them, while single-run and preprocessing methods remain prone to errors.

    Source: §1 · pp. 1–2

  • Method

    CaLM has a smaller LM answer the same query using only the larger LM’s cited documents, then verifies aligned answers and iteratively refines discrepancies without fine-tuning.

    Source: §1 · p. 2

  • Results

    1.5% to 7% average improvement: CaLM outperforms state-of-the-art methods in answer accuracy and citation quality across QAMPARI, ASQA, and ELI5.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CaLM uses smaller LMs to enhance the grounded-generation capabilities of larger LMs without model fine-tuning.

    Source: §1 · p. 2

    Verified responses are not flawless because both the candidate and verifier may hallucinate simultaneously.

    Source: Limitation · p. 10

Abstract

from arXiv · show

Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses by accurately citing verifiable sources. However, existing methods, by either feeding LMs with raw or preprocessed materials, remain prone to errors. To address this, we introduce CaLM, a novel verification framework. CaLM leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. Our framework empowers smaller LMs, which rely less on parametric memory and excel at processing relevant information given a query, to validate the output of larger LMs. Larger LM responses that closely align with the smaller LMs' output, which relies exclusively on cited documents, are verified. Responses showing discrepancies are iteratively refined through a feedback loop. Experiments on three open-domain question-answering datasets demonstrate significant performance gains of 1.5% to 7% absolute average without any required model fine-tuning.

1 Introduction

Grounded generation seeks more credible, accountable answers by citing verifiable sources, but generating accurate answers and citations from large retrieved collections is complex. CaLM uses a smaller LM to verify a larger LM’s response against cited documents and iteratively refine discrepancies; experiments report gains across three open-domain QA datasets.

Motivation

  • Grounded generation aims to make LMs’ responses more credible and accountable by accurately grounding them in verifiable sources.Doing so requires finding relevant documents, grounding responses, and generating answers.Source: §1 · p. 1
  • Single-run and preprocessing approaches can produce inaccurate answers or citations: the former strains LMs with large inputs, while the latter risks propagated errors or information loss.Both approaches also limit opportunities to iterate, refine, and verify responses.Source: §1 · p. 2

CaLM

  • CaLM cross-references a large LM’s response with a smaller LM’s answer based only on the cited documents, verifying aligned answers and refining discrepancies.The method requires no model fine-tuning.Source: §1 · p. 2

Results

  • 1.5% to 7% average improvement: CaLM outperforms state-of-the-art methods on three open-domain QA datasets in answer accuracy and citation quality.The experiments cover QAMPARI, ASQA, and ELI5.Source: §1 · p. 2

2 Problem Statement

The task is to answer a query from a corpus of trustworthy passages with statements supported by cited passages. The response should be useful, its citations precise, and its supporting evidence integral to the answer.

Task Setup

  • Grounded generation takes a query and trustworthy text corpus as input and produces an answer composed of statements grounded in that corpus.Each statement cites a list of passages.Source: §2 · p. 2
  • The system jointly aims to make the answer useful for the query, its statement-level citations precise, and its evidence sufficient to support the answer.The answer consists of statements, each citing passages from the corpus.Source: §2 · p. 2

Evaluation

  • Responses are evaluated for fluency, correctness, and citation quality, with citation quality assessed by citation recall and precision.Recall concerns whether statements have supporting citations; precision concerns whether citations are relevant.Source: §2 · p. 2

3 Automated Verification for Grounded Generation

CaLM’s verification design uses a smaller LM to check whether a larger LM’s answer can be derived from its cited evidence. The paper motivates this contrast with smaller models’ sensitivity to document relevance and studies that sensitivity on ASQA.

3 Automated Verification for Grounded Generation

  • Grounded-generation hallucinations can affect both answers and citations, motivating automated verification that leverages different strengths of verification and generation systems.The paper introduces an unsupervised method using a small LM as verifier.Source: §3 · pp. 2–3

3.1 Key Factors for Automated Verification

  • Automated verification must assess answer correctness and citation quality without ground-truth references while remaining efficient; per-citation NLI checks can be costly for lengthy answers.Correctness includes faithful use of the knowledge base and sound reasoning.Source: §3.1 · p. 3

3.2 Contrasting Large and Small LMs for Automated Verification

  • A smaller LM answers the same query using only the larger LM’s cited documents; agreement is taken as evidence of answer fidelity and citation quality.The design assumes robust cited evidence should let even a small LM deduce the correct answer; smaller LMs are more receptive to external knowledge.Source: §3.2 · p. 3

3.3 Analyzing Model Size Impact on LMs’ Sensitivity to Input Document Relevance

  • The sensitivity analysis compares relative performance improvement as document relevance rises and uses a second-order gain ratio to represent incremental improvement.It anchors performance at low, medium, and high relevance levels; the study uses five documents with approximately 27%, 56%, and 78% answer recall rates.Source: §3.3 · pp. 3–4
  • Within the same model family, smaller LMs usually show greater relative improvement and incremental performance gain as input-document relevance increases, but are more prone to errors with irrelevant documents.The study uses ASQA; Figure 2 presents relative-improvement and second-order analyses.Source: §3.3 · p. 4, Figure 2 · p. 4

4 CaLM Framework

CaLM uses a large LM to draft grounded answers and a smaller verifier to assess them against cited evidence. When the outputs disagree, it retains supported content and iterates with refreshed documents and a correction prompt.

Step 2: Main LM Generation and Steps 3–4: Verification

  • The large LM generates an answer from the top-k retrieved documents, while a smaller verifier generates an answer using only the candidate’s cited evidence.The main LM is recommended for its robustness in identifying useful information and filtering noise; the verifier contrasts its output with the candidate.Source: §4 · pp. 4–5

Framework Overview

  • CaLM’s five-step process retrieves a document pool, generates a candidate, verifies it, then prepares supported content and refreshed documents for another iteration if needed.Figure 3 depicts the workflow; its verification steps are based on the method in § 3.2.Source: §4 · pp. 4–5, Figure 3 · p. 5

Step 4: Contrasting Answer Candidate and Verifier Output

  • If the candidate and verifier output are sufficiently consistent, CaLM accepts the candidate; otherwise, it removes inconsistent answer segments and citations for the next iteration.The retained answer and its evidence are used in preparing the next iteration.Source: §4 · p. 5

Consistency Measurement

  • Consistency is measured with ROUGE-2 between the candidate and verifier output, and the acceptance threshold can be tuned on a small development set.The paper reports satisfactory qualitative results for thresholds from 0.2 to 0.5.Source: §4 · p. 5

Step 5: Input Preparation for Next Iteration

  • The next iteration starts with retained evidence and adds passages from the next pool batch up to the document budget, then prompts the large LM to correct the incomplete answer.A maximum iteration count halts the process; if reached, the last candidate is output.Source: §4 · pp. 5–6

5 Experimental Setup

The experiments evaluate grounded generation across three open-domain QA datasets, then compare CaLM with in-context, preprocessing, postprocessing, and fine-tuned approaches.

5 Experimental Setup

  • The study evaluates three factoid QA datasets using trustworthy 100-word passages prepared following Gao et al. (2023b).Each dataset has an illustrative example in Table 1.Source: §5 · p. 6

5.1 The ASQA dataset

  • ASQA contains ambiguous questions requiring multiple short answers and long-form responses; 948 samples are tested using Wikipedia retrieval with DPR and GTR-large.For the top five documents, retrieval recall is 51.5% with DPR and 56.8% with GTR-large.Source: §5.1 · p. 6

5.2 The QAMPARI dataset

  • QAMPARI tests identification of answer entities from questions, using 1,000 test samples and DPR or GTR-large retrieval.For the top five retrieved documents, recall is approximately 17.6% with DPR and 31.6% with GTR-large; correctness uses entity precision and recall with exact string match.Source: §5.2 · pp. 6–7

5.3 The ELI5 dataset

  • ELI5 evaluates clear, insightful answers to diverse “How,” “Why,” and “What” questions using 1,000 samples and Sphere-corpus retrieval with BM25.Correctness is measured by claim recall: three sub-claims are extracted per reference answer and checked with a TRUE NLI model.Source: §5.3 · p. 7

5.4 Compared Methods

  • The experiments compare CaLM with single-run citation generation, summary- and snippet-based preprocessing, self-consistency postprocessing, and Self-RAG.CaLM uses iterative verification and refinement, with a 13B Tulu-2-DPO verifier; Self-RAG is reported only on ASQA because it requires fine-tuning.Source: §5.4 · p. 7

6 Experimental Results

CaLM achieves the strongest average results across the reported datasets and remains effective across retrievers. Analyses favor a smaller verifier and verification limited to cited documents.

6.1 Main Results

  • Snippet+ICLCite is the strongest baseline, but it does not always beat ICLCite and often fails to improve with weaker retrieval, where noisy inputs may contribute to errors.The authors attribute CaLM’s retriever robustness to releasing only high-quality responses while exploring new document batches.Source: §6.1 · pp. 7–8
  • Across three datasets, CaLM leads average performance; on ASQA it improves by over 6% with text-unicorn, with correctness and citation gains robust across retrievers.The reported average-performance leads include QAMPARI across settings and ELI5 regardless of the main LM; ELI5 citation-quality gains are especially significant.Source: Table 2 · p. 8, Table 3 · p. 8, Table 4 · p. 8

What if we use larger LM as the verifier LM?

7 Related Work

Prior work addresses citation evaluation, training, retrieval-based editing, and self-reflection. CaLM is positioned as verification of answer grounding without fine-tuning.

Evaluation

  • AIS-based work assesses attribution faithfulness, while ALCE provides an automatic benchmark for cited generation that this study uses.The cited work includes Rashkin et al. (2023) on AIS and Gao et al. (2023b) on ALCE.Source: §7 · p. 9

Finetuned LMs

Retrieval-based Methods

  • Retrieval-based post-editing and summarization improve consistency or citations, but cited prior approaches lack answer validation or correct-grounding verification.The passage distinguishes document-relevance verification from using the answer to verify its grounding.Source: §7 · p. 9

Self-reflection

  • Self-reflection methods improve factuality, while Self-RAG uses trained special tokens; CaLM is reported to outperform it without training.The passage cites Ji et al. (2023) and Asai et al. (2023).Source: §7 · p. 9

8 Conclusion

CaLM uses complementary strengths of large and small language models to verify grounded generation.

8 Conclusion

  • CaLM pairs larger LMs’ strength in finding relevant materials with smaller LMs’ ability to process focused information for grounded-generation verification.The paper notes that larger LMs may rely excessively on parametric memory.Source: §8 · p. 9

Limitation

CaLM’s postprocessing adds latency and token costs, and verification does not eliminate the risk of hallucinations. The authors identify these as limitations and call for future work on lower-latency, lower-cost grounded generation.

Limitation

  • CaLM adds unavoidable latency, although limiting correction to one iteration significantly enhances performance.The authors suggest setting the maximum iteration count T smaller as a remedy.Source: Limitation · p. 10
  • Unlike preprocessing approaches that can reduce final-LLM input tokens, CaLM initially processes all documents, increasing token usage.Source: Limitation · p. 10
  • Verification is imperfect because both the answer candidate and verifier output may hallucinate simultaneously.The authors hope future work will advance grounded generation with lower latency and reduced token costs.Source: Limitation · p. 10

Broader Considerations

CaLM inherits risks associated with LLMs, and the authors advise comprehensive evaluation before practical deployment.

Broader Considerations

A Further Analysis

CaLM’s iterations improve correctness precision and recall, with gains tending to plateau after the third iteration. The section also presents correction case studies, dataset-specific prompts, implementation details, and the evaluation artifacts used.

A Further Analysis

  • 0.4 average improvement from extending the maximum to six iterations compared with three came with significantly higher API-call cost, despite correctness precision and recall rising across iterations.The authors report that two to three iterations offer substantial gains with minimal computational overhead.Source: A Further Analysis · p. 13
  • The ASQA and QAMPARI case studies illustrate overlooked information being identified, more convincing evidence found in later rounds, and citation errors caught.The ASQA example attributes overlooked-information detection to the verifier receiving cleaner input documents.Source: A Further Analysis · p. 13

B.1 Prompts for ASQA

  • ASQA grounded-generation prompts mainly follow Gao et al. (2023b) with two-shot examples; the correction prompt is the authors’ own and uses one shot.Source: §B.1 · p. 13

B.2 Prompts for QAMPARI

  • QAMPARI grounded-generation prompts mainly follow Gao et al. (2023b) with two-shot examples, while the authors’ correction prompt uses one shot.Source: §B.2 · p. 13

B.3 Prompts for ELI5

  • ELI5 grounded-generation prompts mainly follow Gao et al. (2023b) with two-shot examples, while the authors’ correction prompt uses one shot.Source: §B.3 · p. 13

C Implementation Details

  • Public-model experiments used vLLM on a machine with 16 NVIDIA-A100-40GB GPUs; GPT-3.5-Turbo-1106 and text-unicorn used their respective official APIs.Source: C Implementation Details · p. 13

D Dataset and Evaluation Tool

Related papers

Loading 2406.05365v2…