Evaluating Open-Domain Question Answering in the Era of Large Language Models

Ehsan Kamalloo; Nouha Dziri; Charles L. A. Clarke; Davood Rafiei

Source-linked AI summary

Evaluating Open-Domain Question Answering in the Era of Large Language Models

Ehsan Kamalloo, Nouha Dziri, Charles L. A. Clarke, Davood Rafiei

arXiv:2305.06984v3May 11, 2023cs.CL

TL;DR

Lexical matching does not reliably evaluate increasingly generative open-domain QA answers, leaving the field’s true progress uncertain. The paper manually evaluates diverse models on an NQ-OPEN subset and compares human judgments with regex and automated evaluation. Human evaluation substantially raises measured performance and changes model rankings, while automated evaluators remain unreliable for hallucinated long-form LLM answers.

Problem
Lexical matching increasingly misses plausible answers absent from incomplete gold lists, especially longer generative and LLM outputs, making accurate open-domain QA evaluation difficult.
Method
The paper manually evaluates answers from diverse open-domain QA models on a random NQ-OPEN subset and compares human judgments with regex and automated evaluation mechanisms.
Results
Human evaluation substantially increases measured performance and changes rankings, including a nearly +60% increase for InstructGPT (zero-shot) and a new state-of-the-art for InstructGPT (few-shot).
Takeaways & Limitations
Regex matching can rank models consistently with human judgments, whereas automated evaluators cannot reliably assess hallucinated long-form LLM answers.
Takeaways & Limitations
The study focuses on factoid information-seeking questions with typically short answers, leaving more complex QA tasks for future analysis.

Abstract

from arXiv · show

Lexical matching remains the de facto evaluation method for open-domain question answering (QA). Unfortunately, lexical matching fails completely when a plausible candidate answer does not appear in the list of gold answers, which is increasingly the case as we shift from extractive to generative models. The recent success of large language models (LLMs) for QA aggravates lexical matching failures since candidate answers become longer, thereby making matching with the gold answers even more challenging. Without accurate evaluation, the true progress in open-domain QA remains unknown. In this paper, we conduct a thorough analysis of various open-domain QA models, including LLMs, by manually evaluating their answers on a subset of NQ-open, a popular benchmark. Our assessments reveal that while the true performance of all models is significantly underestimated, the performance of the InstructGPT (zero-shot) LLM increases by nearly +60%, making it on par with existing top models, and the InstructGPT (few-shot) model actually achieves a new state-of-the-art on NQ-open. We also find that more than 50% of lexical matching failures are attributed to semantically equivalent answers. We further demonstrate that regex matching ranks QA models consistent with human judgments, although still suffering from unnecessary strictness. Finally, we demonstrate that automated evaluation models are a reasonable surrogate for lexical matching in some circumstances, but not for long-form answers generated by LLMs. The automated models struggle in detecting hallucinations in LLM answers and are thus unable to evaluate LLMs. At this time, there appears to be no substitute for human evaluation.

1 Introduction

Lexical matching increasingly misses valid or trustworthy answers as open-domain QA becomes more generative, making model progress difficult to measure. Manual evaluation shows substantially higher performance and changes conclusions about leading models, while automated methods remain unreliable for hallucinated long-form answers.

Motivation: Lexical matching fails when plausible answers are absent from incomplete gold-answer lists, as illustrated by “Jicheng” for a Beijing question.The answer is correct but not annotated as a gold answer, so existing automated mechanisms mark it incorrect.
Motivation: Generative systems make lexical evaluation harder because valid answers become more varied and LLMs can produce plausible but incorrect, unattributable details.InstructGPT’s detailed “Jack Nicholson” answer looks natural but is factually wrong, illustrating the additional burden on human verification.
Findings: 12.6% EM accuracy for InstructGPT (zero-shot) became 71.4% under human judgment on the NQ-OPEN subset, a nearly +60% improvement.The result shows that lexical evaluation can substantially underestimate true performance.
Findings: Human re-evaluation altered model comparisons: InstructGPT (few-shot) achieved a new state-of-the-art, while InstructGPT (zero-shot) became comparable to top existing models.The paper reports that relative performance changes after manually judging answers rather than relying on lexical matching.
Evaluation alternatives: Regex matching captures many lexical-matching failures because mismatches are mostly shallow linguistic variations, but automated evaluation still overestimates InstructGPT (zero-shot).Semantic matching is mostly consistent with human relative performance, although its absolute improvements are lower.

2 Related Work

Prior work addresses answer equivalence through alternative-answer collection, semantic similarity, and human judgment, but these approaches retain coverage, correlation, scalability, or attributability limitations.

Answer Equivalence in QA: Auxiliary knowledge sources can supply alternative plausible answers, but their effectiveness depends heavily on whether those answers appear in the source.Numerical answers and common phrases are often unlikely to be found in a knowledge base.
Answer Equivalence in QA: Token-level semantic similarity metrics such as BERTScore correlate poorly with human judgment in QA evaluation and cannot capture attributability.These metrics rely on matching contextualized representations rather than verifying whether an answer is supported.
Human Judgment in QA: Human evaluation can reveal substantially higher QA performance than token-based metrics, but it is expensive and difficult to scale.Min et al. report an average 23% performance increase under human judgment.

3 Open-domain QA Evaluation

Open-domain QA evaluates answers to information-seeking questions over large knowledge sources, traditionally using normalized lexical metrics. The paper studies 12 diverse QA models on a random NQ-OPEN subset, including retriever-reader, end-to-end, and closed-book systems.

Task and metrics: Open-domain QA finds answers to typically factoid, short, acontextual questions using a massive knowledge source such as Wikipedia.Datasets provide annotated gold answers that serve as evaluation references.
Task and metrics: Exact-Match accuracy counts a candidate as correct only when it appears in the gold-answer set.The reported EM accuracy is the ratio of correct answers across the test collection.
Task and metrics: F1 score gives partial credit from token overlap with gold answers and assigns each candidate its maximum score over the gold set.Corpus-level F1 averages these candidate scores across the test collection.
Task and metrics: Answers are normalized by case-folding and discarding punctuation and articles before EM and F1 are computed.
Models and data: The study randomly samples 301 questions from NQ-OPEN and evaluates 1,490 unique model-generated question/answer pairs using Wikipedia settings from prior work.

4 Strategies for Evaluating Open-domain QA Models

The paper compares lexical matching with BEM, LLM-based evaluation, and human assessment for open-domain QA, using human judgments as the reference. Across 12 models, alternative evaluations raise measured accuracy and can substantially change model rankings, while automated methods remain vulnerable to hallucinated long answers.

Automated Evaluation: BEM classifies whether gold and candidate answers are semantically equivalent, while InstructGPT-eval and GPT4-eval use LLMs to verify candidate correctness from the question and gold answer.The LLM-based evaluators receive the question, gold answer, and candidate answer; GPT4-eval uses the same procedure as InstructGPT-eval.
Human Evaluation: Human annotators judge question-answer pairs for correctness, using search engines to verify evidence without knowing the associated model or gold-answer status.Two annotators independently assess answers, and a third resolves disagreements by majority vote.
Results: 16%, 21%, and 24% are the average accuracy increases under BEM, InstructGPT-eval, and human evaluation, respectively, relative to lexical matching.BEM and InstructGPT-eval trail human judgment by 7.6% and 2.9% on average.
Results: InstructGPT (few-shot) surpasses EMDR2 by nearly +3% under human evaluation, while BEM selects FiD-KD and InstructGPT-eval selects InstructGPT (zero-shot) as best.InstructGPT-eval and BEM correlate with human rankings at Kendall’s τ values of 0.75 and 0.70, compared with 0.23 for EM and 0.37 for F1.
Results: Automated evaluators often accept hallucinated information in long LLM answers, causing InstructGPT (zero-shot) to be overestimated relative to human judgment.This challenge arises because long-form answers can include unattributable supplementary information.

5 Linguistic Analysis of Correct Answers

The analysis categorizes exact-match failures in answers that humans judge acceptable, showing that semantic equivalence and other answer variations account for many misses. It also identifies data-quality failures and evaluates automated methods’ ability to repair them.

Failure-mode statistics: 50.3% of exact-match failures are semantic equivalence, with bridging/abridging, explanatory answers, and multinominal entities the most frequent subcategories.These subcategories account for 11.4%, 10.1%, and 9.3%, respectively.
Failure-mode statistics: 20.6% of exact-match failures are list-style questions, while 15.0% involve granularity discrepancies.Both categories can contain valid answers that differ from the listed gold-answer forms.
Failure-mode statistics: Regular expressions can capture many syntactical answer variations that exact matching misses.Examples include bridging or abridging, synonymous and more elaborate answers, explanatory answers, and tokenization differences.
Failure-mode statistics: 14% of exact-match failures arise from data-quality issues, specifically ambiguity and incorrect gold answers.One example pairs the annotated answer “K’iche’” with the correct answer “Mestizos.”
Automated evaluation: Automated evaluation reduces the 493 exact-match failure cases to 149 for InstructGPT-eval, 217 for BEM, and 137 for GPT4-eval.These correspond to reductions of 70%, 56%, and 72%, respectively.

6 Regex Matching on CuratedTREC

The CuratedTREC 2002 experiment compares regex matching with exact matching, automated semantic evaluators, and human judgments across modern and historical QA models. Regex preserves model rankings largely consistently, but human evaluation reveals differences between the two InstructGPT settings and higher LLM performance.

Dataset and models: CuratedTREC 2002 contains 444 questions with regular-expression gold answers and preserves the original late-1990s AQUAINT news source.The dataset supports an out-of-distribution test for neural models, while LLMs answer closed-book from memory.
Dataset and models: Seven models produce 1,872 unique answers, and four TREC 2002 teams provide historical baselines ranging from 47.3% to 88.1%.The baselines are LCCmain2002, pris2002, exactanswer, and uwmtB3.
Results: Regex matching leaves model rankings unchanged under the three alternative evaluation mechanisms, except for InstructGPT zero-shot and few-shot.The comparison uses regex, BEM, InstructGPT-eval, and human evaluation.
Results: Human evaluation scores InstructGPT few-shot at 92% and shows that it outperforms LCCmain2002 by 2.9%, while zero-shot exceeds it by 1.9%.BEM and InstructGPT-eval do not reflect this comparison against LCCmain2002.
Results: Regex remains rigid, underestimating accuracy by 6.6%, 6.4%, and 9.9% on average under BEM, InstructGPT-eval, and human evaluation.The reported gaps show that regex improves on exact matching but still misses acceptable answers.

7 Conclusion

The paper argues that lexical matching is unnecessarily rigid for open-domain QA and systematically underestimates model accuracy. Manual evaluation finds strong LLM performance, while automated evaluation remains inadequate for hallucinated long-form answers.

Conclusion: Lexical matching is unnecessarily rigid because plausible candidate answers may not appear among the gold answers.The paper studies this problem by manually judging answers from prominent open-domain QA models.
Conclusion: LLMs achieve state-of-the-art performance on NQ-OPEN, but benchmark accuracy is severely underestimated.The conclusion attributes most exact-match failures to syntactical answer variations.
Conclusion: Zero-shot prompting can reasonably substitute for human evaluation in some circumstances, but it cannot detect unattributability in long-form answers.Automated models may deem hallucinated responses correct, leading to overestimation of InstructGPT zero-shot.

Limitations

The study focuses on factoid information-seeking questions that typically require short answers. Similar evaluation analysis remains warranted for QA tasks involving complex reasoning.

Scope: The paper’s main focus is limited to factoid information-seeking questions that typically prompt short answers.The authors identify multi-hop, discrete-reasoning, and causal-relation QA as settings needing similar systematic analysis.

A Zero-shot Evaluation using GPT-4

GPT4-eval broadly tracks human-evaluation trends but can produce erratic judgments and mis-rank models, especially on CuratedTREC 2002.

Kendall’s τ correlation with human judgment is 0.79 for GPT4-eval, slightly above 0.75 for InstructGPT-eval.
GPT4-eval estimates FiD-KD as most accurate on NQ-OPEN, followed by InstructGPT (zero-shot), InstructGPT (few-shot), and EMDR2.
Figure 6 compares EM and GPT4-eval accuracy for 12 models on 301 NQ-OPEN questions, while Figure 7 compares regex matching and GPT4-eval on CuratedTREC 2002.
NQ-OPEN error analysis: GPT4-eval’s errors closely resemble InstructGPT-eval’s errors, but it sometimes rejects correct answers or abstains when it believes more context is needed.It incorrectly rejected “Jermaine Jackson,” a gold answer, and abstained on a candidate involving the show “Fall.”
CuratedTREC 2002: On CuratedTREC 2002, GPT4-eval ranks InstructGPT (zero-shot) above InstructGPT (few-shot) and LCCmain2002 above both, contrary to human evaluation.