PaperlayerView exact revision on arXiv

Source-linked AI summary

ExpertQA: Expert-Curated Questions and Attributed Answers

Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, Dan Roth

arXiv:2309.07852v2cs.CLcs.AI

TL;DR

Domain-specific evidence about LLM factuality and attribution is limited, despite the risks of false claims in high-stakes settings. ExpertQA has experts create questions, evaluate model responses, and revise claims; its findings reveal attribution gaps across systems.

  • Problem

    The factuality and attribution quality of LLM responses in specialized domains are understudied, although false claims can pose risks in high-stakes settings.

    Source: §1 · p. 1

  • Method

    Experts from varied fields formulate questions, evaluate attributed system responses to their own questions, judge claims and evidence, and revise claims.

    Source: §1 · p. 2

  • Results

    Retrieve-and-read systems produce more complete attributions than prompting and post-hoc systems but still miss some cite-worthy claims; retrieval source also affects attribution quality and factuality.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experts should treat attributed responses cautiously because supporting references can be inadequate to support claims.

    Source: §8 · p. 9

    The expert-elicited questions are not necessarily drawn from the natural distribution of query logs.

    Source: §9 · p. 9

Abstract

from arXiv · show

As language models are adopted by a more sophisticated and diverse set of users, the importance of guaranteeing that they provide factually correct information supported by verifiable sources is critical across fields of study. This is especially the case for high-stakes fields, such as medicine and law, where the risk of propagating false information is high and can lead to undesirable societal consequences. Previous work studying attribution and factuality has not focused on analyzing these characteristics of language model outputs in domain-specific scenarios. In this work, we conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality, by bringing domain experts in the loop. Specifically, we collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. In addition, we ask experts to improve upon responses from language models. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.

1 Introduction

ExpertQA addresses limited domain-specific evidence about the factuality and attribution quality of LLM responses by bringing experts into question creation and answer evaluation. Its findings show attribution gaps across systems and high-stakes fields, while automatic metrics fall short of expert judgments.

1 Introduction

  • EXPERTQA is a benchmark of expert-curated questions and verified answers, with claim-level judgments across factuality and attribution dimensions.It contains 2,177 questions from 32 fields, and experts revise claims to make them factual and supported by trustworthy sources.Source: Figure 1 · p. 1, §1 · p. 2
  • Experts write field-relevant questions, evaluate responses to their own questions, judge claims and evidence, and revise claims to be faithful to reliable evidence.Source: §1 · p. 2
  • Retrieve-and-read systems produce more complete attributions than prompting and post-hoc systems, but still miss citations for some cite-worthy claims.Source: §1 · p. 2
  • The retrieval source significantly affects attribution quality and overall factuality.Source: §1 · p. 2
  • In medicine and law, 35% and 31% of attributions are incomplete, respectively, and 51% of attributions are rated unreliable by experts.Source: §1 · p. 2
  • Existing automatic attribution and factuality metrics correlate poorly with expert judgments, while finetuning them on ExpertQA improves performance across domains.The revised answers can support future long-form QA evaluation and model improvement, though the reported baselines leave room for improvement in ROUGE and QAFactEval.Source: §1 · p. 2

2 Expert-in-the-loop Evaluation

EXPERTQA is built through a two-stage expert-in-the-loop process: experts formulate field-specific questions, then evaluate and edit generated responses to their own questions. The dataset contains 2,177 questions from 524 experts across 32 fields.

2 Expert-in-the-loop Evaluation

  • The evaluation proceeds from experts writing questions to the same experts analyzing responses to those questions.The responses were sampled from multiple systems and returned to the question authors for analysis.Source: §2 · p. 2

2.1 Stage 1: Expert-Curated Questions

  • Experts collected 2,177 field-relevant questions from 524 qualified experts across 32 fields, requiring at least two scenario-based questions from each expert.Experts had formal education and at least three years of work experience in their field; questions were manually filtered for coherence and field relevance.Source: §2.1 · p. 2

2.2 Stage 2: Answer and Claim Annotation

  • Six systems generated answers that were split into sentence-level claims for expert validation.Experts labeled answer and claim properties, then edited responses to ensure claims were factually correct and references supported them.Source: §2.2 · p. 3
  • Finer-grained claim annotation was considered but rejected because it incurs considerably higher annotation cost.The authors note that they considered increasing claim atomicity further.Source: §2.2 · p. 3

3 Systems Evaluated

The study samples systems that generate answers with inline attributions, including prompted generators, post-hoc retrieval, retrieve-and-read, and a commercial system. Sampling excludes abstentions and limits each answer to ten claims.

3 Systems Evaluated

  • Systems produce answer strings with inline citations, returned as URLs or retrieved passages with their URLs.The evaluated approaches include prompted generation, post-hoc retrieval, retrieve-and-read, and BingChat.Source: §3 · p. 4
  • Post-hoc systems retrieve claim evidence after generating unattributed answers, while retrieve-and-read systems provide retrieved evidence to GPT-4 before answering.The post-hoc systems use Sphere or Google search; retrieve-and-read uses BM25 for Sphere or Google search results and instructs the model to cite context passages.Source: §3 · p. 4

3.1 Response Sampling

  • Responses were sampled uniformly across systems, excluding abstentions and limiting answers to ten claims; broken GPT-4 links led to more samples from other systems.Table 4 reports per-system sample counts and abstention rates.Source: §3.1 · p. 4, Table 4 · p. 4

4 Analysis

EXPERTQA contains 2,177 expert-validated examples, and manual checks found high annotation agreement in two sampled fields. Expert evaluations show differences across systems and domains in answer usefulness, attribution completeness, factuality, and source reliability.

4.1 Data Statistics

  • EXPERTQA contains 2,177 validated examples, with distributions of claims and tokens summarized in Figure 3.The averages are 5.79 claims and 152.12 tokens per example.Source: §4.1 · p. 4, Figure 3 · p. 5

4.2 Manual Analysis

  • Agreement with reference labels exceeded 85% for most properties in sampled questions from Engineering & Technology and Medicine.The authors sampled 60 questions from each field and uniformly sampled answers across systems.Source: §4.2 · p. 5, Figure 4 · p. 5

4.3 Analysis of Expert Evaluations

  • GPT-4 answers were marked useful about 87–89% of the time, while retrieve-and-read systems and Bing Chat were marked useful 73–80% of the time.Google search evidence produced more useful answers than the smaller Sphere corpus; systems also struggled with targeted long-tail queries, often giving hedged, generic, or vague information.Source: §4.3 · p. 5
  • Retrieve-and-read systems often produced complete attributions but omitted citations for 18% of cite-worthy claims; post-hoc systems attributed every claim but more often gave incomplete attributions.The authors note that post-hoc retrieval may lack context needed to find valid attributions.Source: §4.3 · p. 5
  • Experts marked just over half of claims definitely correct, and the smaller Sphere retrieval corpus yielded less factual claims.The authors suggest that short evaluation times may limit experts’ confidence in judging factuality.Source: §4.3 · p. 5
  • Source reliability judgments depended on the retrieval corpus; using Google search improved reliability judgments compared with Sphere.The passage notes that Sphere could retrieve unreliable sources, including a blog comment for a breast-cancer question.Source: §4.3 · pp. 5–6
  • About 17–22% of claims were judged not cite-worthy, indicating that experts considered most response claims to warrant evidence.Figure 5 presents Likert distributions for answer quality and attribution quality.Source: §4.3 · p. 6, Figure 5 · p. 6
  • In medicine and law, incomplete attributions affected about 35% and 31% of claims, respectively, and about 51% of medical claims cited unreliable sources.Across question types, systems struggled on Type VI requests for lists of resources, whose claims were less informative, factual, and supported by evidence.Source: §4.3 · p. 6

5 Automatic Estimation of Attribution and Factuality

The paper tests whether automatic attribution and factuality measures align with expert judgments in EXPERTQA. These methods show high precision but low recall, with attribution scores varying across systems and factuality estimation missing many non-factual claims.

5 Automatic Estimation of Attribution and Factuality

  • Automatic attribution and factuality methods show high precision but low recall against expert judgments in EXPERTQA.The evaluation examines how reliably prior automatic methods reflect collected human labels.Source: §5 · p. 6

5.1 Automatic Attribution Estimation

  • AutoAIS scores varied substantially across systems, with post-hoc retrieval systems scoring much lower than retrieve-and-read systems on claims with complete attributions.Table 5 reports macro-averaged AutoAIS scores for cite-worthy claims annotated as completely attributed.Source: Table 5 · p. 7, §5.1 · p. 7
  • AutoAIS predictions had high precision but low recall against human attribution judgments, partly because NLI models require every claim component to be verifiable from evidence.Human judgments can use implicit world knowledge and synthesize multiple evidence sources, which are common for Bing Chat and retrieve-and-read systems.Source: Table 6 · p. 7, §5.1 · p. 7

5.2 Automatic Factuality Estimation

  • FActScore estimates factuality by averaging judgments across atomic claims, each checked against the top three Google-search passages retrieved for it.Claims are split into atomic claims with few-shot prompting, then classified as true or false by GPT-3.5-turbo.Source: §5.2 · p. 7
  • Automatic factuality estimation had low recall for non-factual claims, especially for retrieve-and-read systems whose answers use retrieved evidence.The authors suggest ChatGPT-like evaluators may find claims easier to judge when other systems generate answers from GPT-4’s parametric knowledge.Source: Table 8 · p. 7, §5.2 · p. 8

6 Long-form QA Evaluation

The paper evaluates long-form answers using EXPERTQA’s expert-revised responses as references. Results show that fine-tuning improves performance, while model size alone does not ensure stronger results than fine-tuned systems.

6 Long-form QA Evaluation

  • The expert-revised answers form a factual long-form QA dataset evaluated using random and domain-wise 80-10-10 splits.The domain-wise split assigns 80% of each field’s data to training and 10% each to validation and test.Source: §6 · p. 8

6.1 Evaluation Metrics

  • Evaluation uses ROUGE for similarity to reference answers and QAFactEval for factual consistency through generated QA pairs..Source: §6.1 · p. 8

6.2 Baselines

  • The fine-tuned open-source baselines include FlanT5-11B, Alpaca-7B, Vicuna-7B, and Llama2-7B-Chat.The study also reports zero-shot Llama2-70B-Chat.Source: §6.2 · p. 8

6.3 Results

  • Llama2-7B and Vicuna-7B outperformed the smaller FlanT5-11B, which the authors attribute as likely due to additional instruction fine-tuning.The comparison is reported in Table 9.Source: §6.3 · p. 8
  • The authors observe that fine-tuning significantly improves performance; under zero-shot prompting, Llama2-70B performs worse than fine-tuned systems, while results without fine-tuning appear in Table 14.Source: §6.3 · p. 8
  • The authors report that fine-tuning improves performance relative to results without fine-tuning.The no-fine-tuning results are reported in Table 14.Source: §6.3 · p. 8

7 Related Work

Related work covers attribution and factuality evaluation, long-form QA, and domain-specific QA. EXPERTQA combines expert-curated practical questions across many fields with claim-level attribution and factuality judgments.

Attribution Analysis

  • Prior attribution analyses find systems still lack precise attributions with sufficient recall, a concern the authors emphasize for domains requiring both precision and recall.The paper samples vanilla prompting, retrieve-and-read, and post-hoc retrieval system classes.Source: §7 · p. 8

Factuality Analysis

  • Factuality research includes broad evaluation work and smaller expert evaluations; EXPERTQA collects domain experts’ best-effort truthfulness judgments and compares them with automatic estimates.The authors use FActScore to assess how automatic judgments correlate with human labels.Source: §7 · pp. 8–9

Long-form QA

  • Existing long-form QA datasets based on search queries or forums have been criticized for vague questions and difficult factual verification.EXPERTQA targets experts’ practical information needs and includes fine-grained factuality judgments.Source: §7 · p. 9

Domain-specific QA

  • Domain-specific QA datasets cover fields such as medicine, law, and technology, but often have limited domain coverage.EXPERTQA combines broader field coverage with attribution and factuality judgments.Source: §7 · p. 9

8 Conclusion and Future Work

EXPERTQA’s evaluation suggests that LLMs remain short of meeting experts’ information needs with factual, verifiable answers. The benchmark is intended to support improved attribution, factuality estimation, and long-form question answering.

8 Conclusion and Future Work

  • The evaluation suggests LLMs still have substantial ground to cover in providing experts with factual, verifiable answers, and experts should treat attributed responses cautiously.Supporting references can be inadequate to support claims.Source: §8 · p. 9
  • EXPERTQA is intended to benefit work on improved attribution and factuality estimation and long-form question answering.The benchmark’s examples include verified attributions and answers edited by experts.Source: §8 · p. 9

9 Limitations

The limitations concern claim granularity and extraction, field and question coverage, and the subjectivity of expert labels. The annotation details describe participant backgrounds, field selection, and annotation effort.

Atomicity of Claims

  • Non-atomic sentence-level claims require exhaustive factuality and attribution judgments; finer-grained annotation was not explored because it can cost substantially more.The authors also considered increasing claim atomicity further, but note the higher annotation cost.Source: §9 · p. 9

Claim Extraction

  • Sentence-tokenizer errors, especially with lists or tables, made a small number of extracted claims excessively long and difficult to evaluate.Source: §9 · p. 9

Field Coverage

  • EXPERTQA omits some fields, and its examples primarily reflect English-speaking annotators based in Europe, the Americas, and Africa.The authors note that finding experts in rarer fields can be especially hard.Source: §9 · p. 9

Question Distribution

  • Expert-formulated questions may not match the natural query-log distribution, which the authors could not access.They sought to approximate realistic information-seeking through questions from experts’ professional lives or genuine curiosity.Source: §9 · p. 9

Subjectivity of labels

  • Some claim properties invite subjective judgments that vary among experts, and multiple judgments do not capture this variability in the dataset.Agreement was estimated using claims from engineering and medicine.Source: §9 · p. 9

A Annotation Details

  • The 484 Prolific participants came from 26 countries and qualified through formal field education and at least three years’ work in the area.Fields were initially based on university departments and refined to represent diverse disciplines with enough experts; stage 2 averaged 13.83 minutes per question-answer pair.Source: A Annotation Details · p. 14

B Experimental Details

The experimental details specify retrieval settings, finetuning configurations, and the prompts used for response generation and evaluation.

B.1 Hyperparameter Settings

  • Retrieve-and-read systems used top-5 Sphere passages or top-10 Google results retrieved with the question; post-hoc systems retrieved evidence per GPT-4 claim using that claim as the query.Google results were split into 1,000-token passages with 200-token overlap; post-hoc retrieval used the same chunking.Source: §B.1 · p. 14
  • AutoAIS finetuning used T5-XXL with DeepSpeed ZeRO stage 3, batch size 1, learning rate 1e−4, and three epochs; QA finetuning used three epochs for each model.FlanT5-11B used batch size 2 and sequence length 512; Llama2-7B and Vicuna-7B used batch size 4 and sequence length 2048.Source: §B.1 · p. 14

B.2 Prompts

  • The appendix provides separate prompts for GPT-4 and BingChat, retrieve-and-read systems, and Llama and Vicuna long-form QA baselines.Source: §B.2 · p. 14

C Additional Plots

The appendix provides long-form QA result tables, system prompts, and plots of question-type and claim-property distributions across fields and question types.

C Additional Plots

Related papers

Loading 2309.07852v2…