Source-linked AI summary
Benchmarking Large Language Models in Retrieval-Augmented Generation
Jiawei Chen, Hongyu Lin, Xianpei Han, Le Sun
TL;DR
Existing research lacks rigorous evaluation of how retrieval augmentation affects different LLMs and where their RAG capabilities bottleneck. The paper builds RGB, a bilingual benchmark covering four required abilities, and evaluates six LLMs. Results show some noise robustness but substantial difficulty with negative rejection, information integration, and factual errors, indicating that reliable RAG application remains challenging.
Problem
Existing research lacks rigorous evaluation of retrieval augmentation across different LLMs and their required RAG capabilities.
Method
The paper builds the English–Chinese RGB benchmark around four RAG abilities and evaluates six representative LLMs using news-derived and retrieved external documents.
Results
Current LLMs show some noise robustness but struggle significantly with negative rejection, information integration, and factual errors.
Takeaways & Limitations
Effectively applying RAG to LLMs still requires substantial further work and careful RAG design.
Takeaways & Limitations
RAG is not designed to automatically correct factual errors in retrieved context, and existing LLMs often trust retrieved false information.
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) is a promising approach for mitigating the hallucination of large language models (LLMs). However, existing research lacks rigorous evaluation of the impact of retrieval-augmented generation on different large language models, which make it challenging to identify the potential bottlenecks in the capabilities of RAG for different LLMs. In this paper, we systematically investigate the impact of Retrieval-Augmented Generation on large language models. We analyze the performance of different large language models in 4 fundamental abilities required for RAG, including noise robustness, negative rejection, information integration, and counterfactual robustness. To this end, we establish Retrieval-Augmented Generation Benchmark (RGB), a new corpus for RAG evaluation in both English and Chinese. RGB divides the instances within the benchmark into 4 separate testbeds based on the aforementioned fundamental abilities required to resolve the case. Then we evaluate 6 representative LLMs on RGB to diagnose the challenges of current LLMs when applying RAG. Evaluation reveals that while LLMs exhibit a certain degree of noise robustness, they still struggle significantly in terms of negative rejection, information integration, and dealing with false information. The aforementioned assessment outcomes indicate that there is still a considerable journey ahead to effectively apply RAG to LLMs.
Introduction
RAG is presented as a response to hallucination, outdated knowledge, and missing domain expertise in LLMs, but retrieved information can also introduce noise and misinformation. The paper therefore proposes a systematic benchmark to evaluate whether LLMs can use retrieved information reliably across four core abilities.
- LLMs still face factual hallucination, outdated knowledge, and limited domain-specific expertise despite strong general abilities.
- RAG uses external knowledge to improve response accuracy and reliability, but retrieved content may contain irrelevant or false information.
- RGB is a bilingual English–Chinese benchmark designed to evaluate these abilities using query and retrieved-document combinations.
- The paper identifies noise robustness, negative rejection, information integration, and counterfactual robustness as four abilities required for RAG.
- The benchmark evaluates six representative LLMs and analyzes their limitations when applying retrieval-augmented generation.
Retrieval-Augmented Generation Benchmark
RGB evaluates how LLMs use retrieved information across four abilities needed for RAG: noise robustness, negative rejection, information integration, and counterfactual robustness. It constructs bilingual question-answer testbeds from recent news and measures model responses with task-specific metrics.
- Required abilities of RAG: RGB evaluates noise robustness, negative rejection, information integration, and counterfactual robustness as four core RAG abilities.These abilities cover extracting answers amid noise, refusing unsupported questions, combining information across documents, and handling factual errors.
- Data construction: The benchmark uses recent news articles, search-engine documents, and question-answer instances to reduce bias from LLMs’ internal knowledge.ChatGPT generates events, questions, and answers from recent articles; retrieved snippets and text chunks provide external documents.
- Testbeds construction: RGB expands its corpus into four testbeds by varying negative-document ratios, using only negative documents, combining information across documents, and injecting factual errors.Counterfactual-robustness data is constructed from the model’s internal knowledge, while information-integration questions require multiple documents.
- Corpus composition: The corpus contains 600 base questions plus 200 additional questions for information integration and 200 for counterfactual robustness, split evenly between English and Chinese.The benchmark therefore evaluates both languages across its core and additional testbeds.
- Evaluation metrics: Accuracy measures noise robustness and information integration, rejection rate measures negative rejection, and error detection and correction rates measure counterfactual robustness.Additional ChatGPT-based evaluation checks rejection and error detection when models do not follow the required output instructions exactly.
- Noise-robustness evaluation: Increasing noise rates challenge RAG performance, with Table 1 reporting noise robustness as accuracy (%) under different noise ratios.Table 2 illustrates noise-related error cases using one positive and one negative document, with ChatGLM2-6B responses and matching versus non-matching text highlighted.
Experiments
Experiments show that RAG improves responses but exposes persistent weaknesses in rejection, information integration, long-distance evidence use, and factual-error handling. Performance declines under high noise, while models can confuse concepts, merge or ignore subquestions, and follow rejection instructions unreliably.
- Noise Robustness: RAG improves LLM responses, but increasing noise eventually causes significant accuracy losses.When the noise ratio exceeds 80%, ChatGPT falls from 96.33% to 76.00%, while ChatGLM2-6B falls from 91.33% to 57.33%.
- Failure Analysis: Models struggle with long-distance evidence and confuse similar concepts, producing answers based on nearby or mismatched information.Examples include relying on another document when question and answer information are far apart and answering automotive revenue instead of revenue.
- Negative Rejection: Negative rejection remains difficult: the highest rejection rates reach only 45% in English and 43.33% in Chinese.Models are easily misled by noisy documents and often produce incorrect answers instead of rejecting unsupported questions.
- Information Integration: Information integration is weak, with maximum accuracy falling from 60% to 43% in English and from 67% to 55% in Chinese after noise is added.Complex questions become significantly harder at a noise ratio of 0.4, whereas simple questions decline significantly only at 0.8.
- Failure Analysis: Complex-question errors include merging subanswers, ignoring one subquestion, and misaligning documents, occurring in 28%, 28%, and 6% of cases, respectively.These errors cause models to assign one answer to multiple questions, omit a subproblem, or use evidence for the wrong subquestion.
- Counterfactual Robustness: LLMs have difficulty identifying and correcting factual errors in retrieved documents, and RAG lacks a safeguard against misinformation.The paper notes that this limitation is especially important in practical settings with abundant fake news.
Conclusion
The paper evaluates four RAG abilities using the RGB benchmark, finding that current LLMs remain limited across these abilities.
- The study evaluates noise robustness, negative rejection, information integration, and counterfactual robustness in LLMs.
- RGB is built from latest news articles and search-engine documents for RAG evaluation.
- Experimental results indicate that current LLMs have limitations in all four evaluated abilities.