PaperlayerView exact revision on arXiv

Source-linked AI summary

"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation

Nandan Thakur, Luiz Bonifacio, Xinyu Zhang, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Boxing Chen, Mehdi Rezagholizadeh, Jimmy Lin

arXiv:2312.11361v3cs.CLcs.IR

TL;DR

Prior work lacks comprehensive evaluation of LLM robustness across languages, despite retrieval errors posing a challenge for RAG. The paper introduces NoMIRACL, a human-annotated dataset spanning 18 languages, and finds that models often struggle to balance hallucination and relevant-passage recognition, with GPT-4 providing the best tradeoff.

  • Problem

    Prior work lacks comprehensive evaluation of LLM reasoning and hallucination across multiple languages, including high- and low-resource languages.

    Source: §1 · p. 2

  • Method

    The authors build NoMIRACL, a human-annotated dataset spanning 18 languages with non-relevant and relevant query subsets for evaluating retrieval-error robustness.

    Source: §1 · p. 2

  • Results

    LLMs struggle to balance hallucination and relevant-passage recognition; LLAMA-2, Aya-101, and Orca-2 show an 88% hallucination rate, while GPT-4 has the best tradeoff across both subsets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    NoMIRACL is presented as a resource for studying how much multilingual LLMs hallucinate.

    Source: §9 · p. 9

    Because the dataset is fully human-constructed, it may contain human errors; quality checks validate a subset.

    Source: §10 · p. 10

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) output by leveraging external knowledge sources to reduce factual hallucinations. However, prior work lacks a comprehensive evaluation of different language families, making it challenging to evaluate LLM robustness against errors in external retrieved knowledge. To overcome this, we establish NoMIRACL, a human-annotated dataset for evaluating LLM robustness in RAG across 18 typologically diverse languages. NoMIRACL includes both a non-relevant and a relevant subset. Queries in the non-relevant subset contain passages judged as non-relevant, whereas queries in the relevant subset include at least a single judged relevant passage. We measure relevance assessment using: (i) hallucination rate, measuring model tendency to hallucinate, when the answer is not present in passages in the non-relevant subset, and (ii) error rate, measuring model inaccuracy to recognize relevant passages in the relevant subset.In our work, we observe that most models struggle to balance the two capacities. Models such as LLAMA-2 and Orca-2 achieve over 88% hallucination rate on the non-relevant subset. Mistral and LLAMA-3 hallucinate less but can achieve up to a 74.9% error rate on the relevant subset. Overall, GPT-4 is observed to provide the best tradeoff on both subsets, highlighting future work necessary to improve LLM robustness. NoMIRACL dataset and evaluation code are available at: https://github.com/project-miracl/nomiracl.

1 Introduction

RAG uses retrieved external knowledge to ground LLM answers, but retrieval errors can mislead models and multilingual robustness remains insufficiently evaluated. NoMIRACL assesses this challenge across two answerability subsets and finds a tradeoff between hallucination and recognizing relevant passages.

1 Introduction

  • Prior work lacks comprehensive multilingual evaluation, leaving the extent of LLM hallucination across high- and low-resource languages unclear.Incorrect or non-relevant retrieved passages can mislead LLMs into hallucinating.Source: §1 · p. 2
  • NoMIRACL is a human-annotated dataset of over 56,000 samples across 18 typologically diverse languages, with non-relevant and relevant subsets.The non-relevant subset has no known answers in its top-k passages; the relevant subset has at least one judged relevant passage.Source: §1 · p. 2
  • LLAMA-2, Aya-101, and Orca-2 show an 88% hallucination rate on the non-relevant subset, while GPT-4 provides the best tradeoff across both subsets.Mistral and LLAMA-3 hallucinate less but perform worse on the relevant subset.Source: §1 · p. 2
  • Manual inspections identify model-specific generation patterns, including LLAMA-2 models repeating queries or instructions and Mistral-7B producing rationale in over 88% of outputs.The introduction also notes that supervised fine-tuning on the NoMIRACL development set can be tricky.Source: §1 · p. 2

2 Background and Problem Identification

RAG depends on a retrieval stage that can return noisy or irrelevant passages, creating a robustness challenge for answer generation. NoMIRACL evaluates this challenge as binary classification against human judgments, using separate metrics for unanswerable and answerable queries.

2 Background and Problem Identification

  • Overreliance on retrieved content can limit RAG when passages are noisy or non-relevant, and such passages can mislead LLMs into hallucinating.RAG first retrieves top-k passages from an external corpus, then provides the query and passages to an LLM to generate an answer citing relevant passages.Source: §2 · p. 2

2.1 Robustness Evaluation

  • Hallucination rate is FP/(FP+TN) on the non-relevant subset, while error rate is FN/(FN+TP) on the relevant subset.The evaluation compares LLM predictions with ground truth supplied by human annotators.Source: §2.1 · p. 3
  • The non-relevant subset contains queries whose top-k passages are all judged irrelevant; the relevant subset contains queries with at least one judged relevant passage.The confusion matrix distinguishes correct abstention (TN) and correct answer recognition (TP) from incorrect predictions (FP and FN).Source: §2.1 · p. 3

3 NoMIRACL Dataset

NoMIRACL is a multilingual dataset for studying LLM robustness across answerable and unanswerable queries. It is built from native-speaker queries and human judgments of retrieved passages, with disjoint development and test splits.

3 NoMIRACL Dataset

  • NoMIRACL covers 18 languages and includes relevant and non-relevant subsets for evaluating hallucination across answerable and unanswerable queries.The dataset construction overview is illustrated for English.Source: §3 · p. 3

3.1 Data Construction Procedure

  • Native speakers write queries from language-specific Wikipedia snippets, then annotators judge retrieved query-passage pairs as relevant or non-relevant.The non-relevant subset has no relevant top-k passages; the relevant subset has at least one relevant passage.Source: §3.1 · pp. 3–4

3.2 Fold Creation

  • Relevant queries are reused from MIRACL development and test-B splits, while non-relevant queries are divided into disjoint 44% development and 56% test sets.The MIRACL test-A split is left out because it covers only 10 of the 18 languages.Source: §3.2 · p. 4, §3.3 · p. 4

3.3 Languages Covered and Dataset Usage

  • The 18 languages span 10 families and 11 scripts, and experiments cap each language and subset at 250 queries when available.The cap addresses uneven query counts across languages and budget constraints.Source: §3.3 · p. 4

4 Experimental Setup

NoMIRACL evaluates whether LLMs recognize answerability from human-judged retrieved passages, using separate hallucination and error rates. The experiments compare multilingual-focused models under a shared zero-shot prompting and retrieval setup.

4.1 Evaluation Setup and Metrics

  • Each query uses up to 10 passages ranked by an ensemble of BM25, mDPR, and mColBERT scores.The passages are retrieved using language-specific analyzers and manually judged by annotators.Source: §4.1 · p. 4
  • Hallucination rate measures errors rejecting non-relevant passages, while error rate measures errors identifying relevant passages in a binary response task.The LLM responds either “I don’t know” or “Yes, answer is present”; other outputs are tagged Invalid.Source: §4.1 · p. 5

4.2 Evaluation Models

  • The evaluation includes 11 multilingual-focused LLMs, spanning OpenAI, Mistral, Orca-2, Aya, LLAMA-2, and LLAMA-3 models.The set includes open- and closed-source models, from 7B models to GPT-4 and GPT-4o.Source: §4.2 · p. 5

4.3 Experimental Settings

  • All models receive a zero-shot prompt with the query and passages in the same language, an English task instruction, and a 4096-token input limit.Each of the 10 passages is truncated to its first 375 tokens, and at most 250 randomly sampled queries per language and subset are evaluated.Source: §4.3 · p. 5, Figure 4 · p. 5

5 Experimental Results

Across the two NoMIRACL subsets, models show different strengths in abstaining from answers and recognizing relevant passages. GPT-4 provides a good tradeoff, while other models often perform unevenly across the two dimensions.

5.1 NoMIRACL Non-relevant Subset

  • 17.4% was Mixtral-7x8B’s average hallucination rate, compared with 35.5% for GPT-4; LLAMA-2, Orca-2, and Aya-101 each exceeded 80%.All models except Mistral hallucinated across the evaluated languages.Source: §5.1 · p. 6
  • Hallucination rates were lowest overall in Swahili and Yoruba; GPT-3.5 ranged from 25.2% in Swahili to 95.2% in Bengali.The authors hypothesize that low-resource-language queries may have easier non-relevant retrieved passages; they report poor non-relevant-subset performance overall.Source: §5.1 · p. 6

5.2 NoMIRACL Relevant Subset

  • 62.5% was Aya-101’s average relevant-subset error rate, while LLAMA-2-70B and GPT-4o each had error rates below 10%.Aya-101, Mixtral-7x8B, and LLAMA-3-8B each had average error rates above 40%.Source: §5.2 · p. 7

5.3 NoMIRACL Overall Comparison

  • GPT-4 provides a good tradeoff across both subsets, while Mistral and LLAMA-3 have over 40% relevant-subset error despite less non-relevant-subset hallucination.Aya-101 performs poorly on both subsets, and GPT-4 is expensive to run at scale.Source: §5.3 · p. 7

6 Empirical Analysis

The empirical analysis examines English-subset outputs to characterize model behavior on answerable and non-answerable queries. It identifies recurring output patterns in both subsets.

6 Empirical Analysis

  • The analysis uses the English subset and categorizes outputs as accurate, instruction-misunderstanding, or hallucination/error cases.The categories are marked green, orange, and red, respectively.Source: §6 · p. 7

Non-relevant Subset.

  • On non-relevant queries, LLAMA-2-7B and 13B often repeat the question or instruction, while Aya-101 often answers from implicit memory rather than retrieved passages.Mistral-7B consistently provides a rationale, and Orca-2-7B often uses “No, answer is not present” instead of “I don’t know”.Source: Non-relevant Subset · p. 7

Relevant Subset.

  • On relevant queries, GPT-4 and Orca-2-13B typically output only a classification token, while LLAMA-2 models may repeat instructions and Aya-101 may use implicit memory.GPT-3.5 and Mixtral-8x7B often include a rationale or explanation.Source: Relevant Subset · p. 7

7 Further Studies

The further studies test prompt variations and fine-tuning as ways to affect NoMIRACL performance. Both experiments reveal tradeoffs or model-dependent outcomes across the two evaluation subsets.

Prompt Optimization

  • 6.3% and 15.2% lower relevant-subset error rates under role and repeat prompts came with 8.7% and 15.9% higher hallucination rates, respectively.Explanation prompting lowered hallucination by 9.7% but raised error by 8.3%, leaving users to choose based on the desired subset performance.Source: §7 · p. 8

Fine-tuning on NoMIRACL

  • The fine-tuning study applies SFT LoRA adapters to Mistral-7B and LLAMA-3 (8B) using NoMIRACL development data across 18 languages.Models larger than 8B parameters were excluded due to computational budget restrictions.Source: §7 · p. 8
  • 10.6% was the LLAMA-3 (8B) error-rate improvement after fine-tuning, but hallucination worsened by up to 17.9%; Mistral-7B worsened on both subsets.The authors characterize SFT as tricky and call for careful experimentation on the binary classification task.Source: §7 · p. 8

8 Related Work

Prior work presents RAG as a way to ground generation in external knowledge, while studying robustness to irrelevant context and unanswerable queries through several evaluation approaches.

8 Related Work

  • Grounding generation in external knowledge is presented as a way to reduce outdated or factually incorrect outputs and support RAG applications across tasks.The cited examples include open-domain question answering, argument extraction, and code generation.Source: §8 · pp. 8–9
  • Studies examine how irrelevant context affects LLM outputs, including work on controllability, non-relevant passages, and methods intended to improve RAG robustness [Li et al. (2023); Adlakha et al. (2024); Yu et al. (2023); Yoran et al. (2024)].Adlakha et al. report complementary findings in QA datasets such as NQ.Source: §8 · p. 9
  • Prior datasets include adversarial unanswerable questions and conversational QA, while concurrent RGB evaluates RAG robustness in English and Chinese [Rajpurkar et al., 2018; Reddy et al., 2019; Choi et al., 2018; Chen et al., 2024].Other cited work studies passage perturbations across QA, logical reasoning, and arithmetic reasoning [Jia and Liang, 2017; Petroni et al., 2020; Creswell et al., 2023; Misra et al., 2023; Shi et al., 2023; Kumar et al., 2021].Source: §8 · p. 9

9 Conclusion

The paper introduces NoMIRACL, a human-annotated multilingual dataset for assessing LLM relevance judgments in RAG. Across its two subsets, existing models show robustness challenges, while GPT-4 achieves the best tradeoff.

9 Conclusion

  • NoMIRACL is a human-labeled binary relevance assessment dataset spanning 18 languages, built with 31 native speakers and divided into relevant and non-relevant subsets.Queries in the non-relevant subset have only judged non-relevant passages; those in the relevant subset have at least one judged relevant passage.Source: §9 · p. 9
  • Existing LLMs face challenges in either hallucination on non-relevant passages or error in recognizing relevant passages, while GPT-4 achieves the best tradeoff across both subsets.The dataset is intended to support research into LLM hallucination and more robust multilingual-focused models.Source: §9 · p. 9

10 Limitations

The paper identifies human annotation, passage-level relevance judgments, and Wikipedia-only corpora as limitations of NoMIRACL, while describing its appendix materials and dataset release.

10 Limitations

  • Human construction may introduce errors, although additional quality checks examined a subset of questions and relevance judgments.The paper points to Appendix C for the quality-check details.Source: §10 · p. 10
  • NoMIRACL evaluates whether top-k passages are relevant rather than whether they contain actual answer spans, which would require domain experts and can be difficult for non-extractive or long-form answers.The dataset therefore treats top-k passages as information contexts judged for relevance.Source: §10 · p. 10
  • The dataset currently uses language-specific Wikipedia, which may not represent real-world domains; expansion is deferred amid corpus scarcity, cross-language domain differences, and budget and labor demands.The paper cites medical-domain coverage in English BEIR as an example of domain diversity.Source: §10 · p. 10

A Appendix

B Details on NoMIRACL Dataset Release

  • NoMIRACL follows Wikipedia's CC BY-SA 4.0 license, which the paper says permits researchers and industry to access, copy, and redistribute the dataset.The dataset is based on language-specific Wikipedia.Source: B Details on NoMIRACL Dataset Release · p. 15

C Additional Data Construction Details

The appendices describe corpus preparation, annotator hiring, quality control, and supplementary evaluation materials for NoMIRACL.

C Additional Data Construction Details

  • NoMIRACL follows MIRACL corpus preparation: language-specific Wikipedia passages are segmented at natural discourse units, using dumps from early 2019 for 11 languages and March 2022 releases for the others.The passage corpora are parsed with WikiExtractor, using two consecutive newlines in wiki markup as delimiters.Source: C Additional Data Construction Details · p. 15
  • The project hired 31 native-speaker annotators across languages, with interviews and evaluation based on a constructed onboarding and training process.Hiring and dataset construction took around six months overall.Source: C Additional Data Construction Details · p. 15

D Quality Control

  • Quality control used two phases: reviewers checked query quality and prompt overlap, then independently labeled sampled query–passage pairs to assess relevance-label agreement.Average agreement on overlapping relevance pairs exceeded 80%.Source: D Quality Control · p. 15

E Checkpoints and Additional Results

  • The supplement lists multilingual-focused LLM checkpoints in Table 5, NoMIRACL supervised-fine-tuning hyperparameters in Table 6, and SFT results in Table 7; Figures 8 and 9 report evaluations of non-relevant and relevant subsets, respectively.Table 7 gives complete SFT results on the NoMIRACL development dataset for LLAMA-3 (8B) and Mistral-7B (v0.3) across all languages.Source: E Checkpoints and Additional Results · p. 15, Table 5 · p. 16, Table 6 · p. 16
  • Figure 8 defines hallucination rate as FP/(FP + TN) on queries whose top 10 passages are all judged non-relevant; lower rates are better.Most models except Mistral hallucinate on average in this subset.Source: Figure 8 · p. 16
  • Figure 9 defines error rate as FN/(FN + TP) on queries with at least one relevant top-10 passage; lower rates are better.Most models except Mistral and Aya-101 have lower error rates on average.Source: Figure 9 · p. 16
  • Figure 10 illustrates role, repeat, and explanation prompt variants used in hallucination-evaluation ablations across 18 languages and both subsets.The variants respectively add an LLM role, repeat task instructions, or request a reasoning path before the answer.Source: Figure 10 · p. 17
Loading 2312.11361v3…