PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering

Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, Siva Reddy

arXiv:2307.16877v2cs.CLcs.AI

TL;DR

Instruction-following QA models produce fluent but verbose answers that make traditional evaluation inadequate, motivating separate assessment of correctness and faithfulness. The paper combines human and automatic evaluation across three QA tasks, proposes Recall and K-Precision, and finds strong correctness but difficulty grounding responses and deciding when to abstain. It therefore supports reporting correctness, faithfulness, and abstention separately, while noting that other QA properties remain unevaluated.

  • Problem

    Verbose instruction-following QA responses make exact match and F1 unreliable, while unsupported supplementary information complicates faithfulness evaluation.

  • Method

    The paper evaluates four instruction-following models across three QA tasks using human analysis and automatic metrics for correctness and faithfulness.

  • Results

    Recall and K-Precision correlate well with human judgments, while models show a tradeoff: high correctness but difficulty remaining faithful and refraining from answering.

  • Takeaways & Limitations

    QA evaluation should report correctness, faithfulness, and the ability to refrain from answering rather than a single overall score.

  • Takeaways & Limitations

    The evaluation does not cover all desirable QA properties, including completeness and ease of understanding for long-form QA.

Abstract

from arXiv · show

Retriever-augmented instruction-following models are attractive alternatives to fine-tuned approaches for information-seeking tasks such as question answering (QA). By simply prepending retrieved documents in its input along with an instruction, these models can be adapted to various information domains and tasks without additional fine-tuning. While the model responses tend to be natural and fluent, the additional verbosity makes traditional QA evaluation metrics such as exact match (EM) and F1 unreliable for accurately quantifying model performance. In this work, we investigate the performance of instruction-following models across three information-seeking QA tasks. We use both automatic and human evaluation to evaluate these models along two dimensions: 1) how well they satisfy the user's information need (correctness), and 2) whether they produce a response based on the provided knowledge (faithfulness). Guided by human evaluation and analysis, we highlight the shortcomings of traditional metrics for both correctness and faithfulness. We then propose simple token-overlap based and model-based metrics that reflect the true performance of these models. Our analysis reveals that instruction-following models are competitive, and sometimes even outperform fine-tuned models for correctness. However, these models struggle to stick to the provided knowledge and often hallucinate in their responses. We hope our work encourages a more holistic evaluation of instruction-following models for QA. Our code and data is available at https://github.com/McGill-NLP/instruct-qa

1 Introduction

Instruction-following models offer flexible, natural QA responses, but their verbosity makes traditional metrics unreliable. This work evaluates correctness and faithfulness separately, proposes better metrics, and finds strong correctness alongside weak grounding in provided knowledge.

  • Motivation: Verbose responses make exact match and F1 unreliable for evaluating instruction-following QA.Responses may contain correct answers plus additional information, including unsupported claims.
  • Evaluation dimensions: Correctness measures satisfaction of the user’s information need, while faithfulness measures whether factual information is grounded in provided knowledge.Faithful models should answer when knowledge is relevant and abstain when it is irrelevant.
  • Metrics: Recall for correctness and K-Precision for faithfulness correlate strongly with human judgments.The proposed lexical metrics are presented as scalable alternatives to expensive GPT-4-based evaluation.
  • Study scope: The study evaluates four instruction-following models across three diverse QA tasks without further training.The models are compared with task-specific fine-tuned systems for correctness.
  • Findings: Instruction-following models are comparable to fine-tuned models for correctness but often fail to remain faithful to provided knowledge.They also frequently fail to identify when retrieved knowledge is relevant.

2 Related Work

Prior work established lexical, semantic, and model-based QA and faithfulness metrics, but instruction-following models create additional evaluation challenges because their responses can diverge from reference-answer distributions.

  • Instruction-following QA: Instruction-following QA differs from retrieval-augmented few-shot approaches because it uses no demonstrations and does not follow reference-answer distributions.This divergence creates new evaluation challenges.
  • QA evaluation: Traditional QA evaluation primarily relies on lexical metrics such as exact match and F1, while later methods use semantic equivalence models.BERTScore and BEM are examples of model-based semantic evaluation.
  • This work: This work extends prior evaluation by measuring both correctness and faithfulness and proposing token-overlap metrics aligned with human judgments.The approach addresses shortcomings identified in traditional metrics.
  • Faithfulness evaluation: Prior faithfulness metrics include Knowledge-F1, question-generation-based factual consistency, and trained hallucination classifiers.These methods measure overlap, consistency, or hallucination rather than correctness alone.
  • Faithfulness evaluation: Retrieval-augmented GPT-3 can produce responses that appear trustworthy but are unfaithful to retrieved passages.K-F1++ discounts response tokens that also occur in the question, but does not correlate well with human judgments in these experiments.

3 Experimental Setup

The experiments use a standardized retrieval-augmented prompt across three Wikipedia-based QA tasks and four instruction-following models, with task-specific retrievers and fixed passage counts.

  • Datasets and tasks: The study covers open-domain NQ, multi-hop HotpotQA, and conversational TopiOCQA.The tasks test information seeking, joint reasoning over passages, and conversational-context answering.
  • Evaluation setting: The datasets use short-form answers even though users generally prefer verbose responses.This mismatch motivates human evaluation of naturally generated answers.
  • Prompting: Each model receives an instruction, retrieved passages, and a question or conversation history through a standardized prompt template.TopiOCQA replaces the question with conversation history.
  • Models: The evaluated models are Flan-T5, Alpaca, GPT-3.5, and Llama-2, representing different instruction-following training regimes.The study uses Flan-T5 11B, Alpaca 7B, GPT-3.5 Turbo, and Llama-2 7B Chat.
  • Retrieval: Task-specific Dense Passage Retrieval variants provide the retrieved passages for NQ, HotpotQA, and TopiOCQA.HotpotQA uses an iterative multi-hop retriever, while TopiOCQA uses a conversational-QA checkpoint.
  • Retrieval: The experiments provide 8 retrieved passages for NQ and HotpotQA and 4 for TopiOCQA.Passage counts are held constant across models within each task for fair comparison.

4 Correctness w.r.t Information Need

The paper evaluates correctness by whether responses satisfy the user’s information need and shows that traditional QA metrics often misjudge verbose instruction-following responses. Human analysis motivates Recall as a better lexical metric, under which instruction-following models can match or outperform FiD.

  • Evaluation setup: Correctness is defined by whether a response accurately satisfies the user’s information need, while evaluation focuses on the directly relevant portion of verbose answers.The study compares lexical, semantic, and LLM-based metrics against human judgments, using 1,200 annotated responses.
  • Evaluation metrics: Recall measures the proportion of reference-answer tokens present in the model response and does not penalize verbosity when those tokens are included.The authors therefore rely on Recall for evaluating correctness.
  • Human evaluation: 78% of human-judged correct answers were falsely marked incorrect by EM, compared with 39% for F1, while false-positive rates were 0.4% and 3.8%, respectively.These results show that EM and F1 are overly strict for instruction-following responses.
  • Failure analysis: 60.81% of false-negative cases involved More Elaborate Answers, while 10.81% involved Open-ended Questions with valid answers absent from references.Recall helps with elaborate answers, but lexical and semantic metrics struggle with open-ended questions.
  • Model comparison: Using Recall, GPT-3.5 outperformed FiD by 6.48% on NQ, 10.27% on HotpotQA, and 9.33% on TopiOCQA.Overall, instruction-following models were equally or sometimes more capable than task-specific fine-tuned generators for satisfying user information needs; retriever selection also affected performance.

5 Faithfulness w.r.t Provided Knowledge

The paper evaluates faithfulness as groundedness in relevant knowledge and appropriate abstention from irrelevant knowledge, introducing metrics tailored to verbose QA responses. Human and large-scale evaluations show that K-Precision is a strong faithfulness metric, while models differ substantially in groundedness and answer refraining.

  • Faithfulness definition: Faithfulness measures whether supplementary response information is inferable from or directly stated in the provided knowledge.The analysis focuses on faithfulness with relevant knowledge and on abstention when knowledge is irrelevant.
  • Automatic metrics: K-Precision measures the proportion of response tokens that appear in the knowledge snippet, matching the asymmetric nature of information-seeking QA.Unlike K-F1, it does not require the knowledge snippet and response to be equivalent.
  • Human evaluation: GPT-4-based LLMCritic correlates most with human faithfulness judgments, while K-Precision is a close second and outperforms other lexical and semantic metrics.GPT-4 reaches a Spearman correlation of 54.99 but remains only moderately correlated; K-Precision is simpler and interpretable.
  • Large-scale evaluation: Flan-T5 achieves the highest K-Precision on all three tasks, while GPT-3.5 and Llama-2 have higher correctness but substantially lower faithfulness.The results indicate a trade-off between correctness and faithfulness across instruction-following models.
  • Answer refraining: GPT-3.5 refrains most often with irrelevant passages—98% in NQ and HotpotQA and 88% in TopiOCQA—whereas Alpaca almost always answers.Flan-T5 abstains successfully on NQ and HotpotQA but fails on TopiOCQA, which is outside its training data.
  • Answer refraining: With gold passages, GPT-3.5 and Llama-2 still sometimes refrain, with Llama-2 abstaining more than 50% of the time across all three datasets.This shows that identifying when to answer and when to abstain remains difficult.

6 Conclusion

The conclusion argues that QA evaluation should separately measure correctness, faithfulness, and answer refraining rather than rely on one overall score. Recall and K-Precision align with human judgments, but both metrics have important limitations and do not cover every desirable QA property.

  • Conclusion: Recall and K-Precision correlate well with human judgments for correctness and faithfulness, respectively.The paper proposes Recall for correctness and K-Precision for faithfulness.
  • Conclusion: Instruction-following models exhibit a trade-off: GPT-3.5 and Llama-2 score highly for correctness but struggle to remain faithful and to refrain from answering.The conclusion recommends evaluating these dimensions separately.
  • Implications: The paper urges reporting correctness, faithfulness, and answer refraining instead of a single overall QA score.This recommendation is intended to support a more holistic evaluation of instruction-following QA models.
  • Limitations: The evaluated properties are not exhaustive because completeness and ease of understanding for long-form QA remain unaddressed.The paper leaves these information-seeking QA properties for future work.
  • Limitations: Low faithfulness does not imply incorrectness because models may provide accurate information from parametric knowledge, which is difficult to interpret and modify.Faithfulness and correctness therefore capture distinct evaluation dimensions.
  • Metric limitations: Recall can score affirmative and negated statements equally, while K-Precision can be gamed by copying all knowledge from the prompt.Copying is penalized when faithfulness to irrelevant knowledge is evaluated.

A Generation parameters for instruction-following models

The experiments use fixed generation settings for all instruction-following models, with specified sampling, seed, and token-length parameters.

  • Generation settings: All models use top-p p = 0.95, temperature t = 0.95, seed s = 0, and minimum new tokens mintoken = 1.These parameters are held fixed across instruction-following models.
  • Generation settings: Maximum new tokens are capped at 500 for models other than GPT-3.5 to prevent GPU memory errors.GPT-3.5 has no specified maximum-new-token limit.

B Impact of Retriever on Correctness

Retriever choice affects correctness, with task-specific multi-hop retrieval performing best for instruction-following models on HotpotQA. BM25 also outperforms standard DPR, likely because it exploits lexical overlap with gold passages.

  • Retriever comparison: DPR (multi-hop), the task-specific retriever for multi-hop QA, performs best on HotpotQA.The comparison includes BM25, standard DPR, and a HotpotQA-trained multi-hop DPR.
  • Retriever comparison: BM25 outperforms standard DPR across all instruction-following models, potentially because it exploits high query–gold-passage overlap.The finding highlights the importance of matching retriever design to the task.

C Evaluation of Instruction-Following Models in Real-world Settings

The evaluation tests retrieval-augmented instruction-following models under different instructions using Recall for correctness, K-Precision for faithfulness, and PG for answer abstinence. Adding an instruction to avoid answering with irrelevant passages improves faithfulness for some models but can substantially reduce correctness.

  • Experiment Setup: Models receive instructions, retrieved passages, and the question or conversation history; evaluation uses Recall, K-Precision, and PG.PG measures abstinence when the gold passage is present among retrieved passages.
  • Experiment Setup: PG captures answer abstinence when gold passages are retrieved, but always answering “I don’t know” optimizes K-Precision while worsening PG.This creates a metric trade-off: lower PG is better, whereas indiscriminate abstention can make faithfulness appear strong.
  • Results: Flan-T5 has the highest faithfulness under Instr. v1, and switching to Instr. v2 increases faithfulness for Flan-T5 and GPT-3.5.GPT-3.5 shows the largest gain from the instruction change.
  • Results: 12.33%: Llama-2’s faithfulness drops across all three tasks after switching to Instr. v2.Manual inspection attributes this pattern to reasoning explanations for not answering that overlap weakly with retrieved passages.
  • Results: 37.26%: GPT-3.5 has the largest correctness decline across all three datasets after the instruction switch, followed by Llama-2 at 29.37%.On NQ, GPT-3.5 abstains 39.01% of the time with Instr. v2 versus 2.18% with Instr. v1.
  • Results: Alpaca is relatively unaffected across correctness, faithfulness, and answer abstinence, consistent with difficulty following the instruction to refrain from answering.Its limited response to the instruction change contrasts with the larger shifts observed for other models.
Loading 2307.16877v2…