PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Hallucinations in Chinese Large Language Models

Qinyuan Cheng, Tianxiang Sun, Wenwei Zhang, Siyin Wang, Xiangyang Liu, Mozhi Zhang, Junliang He, Mianqiu Huang, Zhangyue Yin, Kai Chen, Xipeng Qiu

arXiv:2310.03368v4cs.CL

TL;DR

Hallucination evaluation for Chinese large language models is limited when benchmarks omit either imitative falsehoods or factual errors. The paper introduces HalluQA, a 450-question Chinese benchmark with automated hallucination evaluation, and finds substantial difficulty across 24 models while analyzing model-specific priorities.

  • Problem

    Existing hallucination benchmarks incompletely cover imitative falsehoods and factual errors in Chinese large language models.

    Source: §1 · pp. 1, 3

  • Method

    The paper constructs HalluQA with 450 Chinese adversarial questions, evaluates outputs using a non-hallucination metric, and tests multiple types of Chinese models.

    Source: §1 · p. 3, §3.1 · p. 6, §3.2 · p. 7

  • Results

    18 of 24 evaluated models achieved non-hallucination rates below 50%, indicating that HalluQA is challenging for current Chinese large language models.

    Source: §3.4 · p. 8

  • Takeaways & Limitations

    Different model types exhibit different hallucination patterns, so mitigation priorities should differ across pre-trained, chat, and retrieval-augmented models.

    Source: §3.4 · p. 8, §4 · p. 9

    ChatGLM-pro's retrieval-augmented classification is based on observed performance because its retrieval-enhancement status is not explicitly stated.

    Source: §3.1 · p. 6

Abstract

from arXiv · show

In this paper, we establish a benchmark named HalluQA (Chinese Hallucination Question-Answering) to measure the hallucination phenomenon in Chinese large language models. HalluQA contains 450 meticulously designed adversarial questions, spanning multiple domains, and takes into account Chinese historical culture, customs, and social phenomena. During the construction of HalluQA, we consider two types of hallucinations: imitative falsehoods and factual errors, and we construct adversarial samples based on GLM-130B and ChatGPT. For evaluation, we design an automated evaluation method using GPT-4 to judge whether a model output is hallucinated. We conduct extensive experiments on 24 large language models, including ERNIE-Bot, Baichuan2, ChatGLM, Qwen, SparkDesk and etc. Out of the 24 models, 18 achieved non-hallucination rates lower than 50%. This indicates that HalluQA is highly challenging. We analyze the primary types of hallucinations in different types of models and their causes. Additionally, we discuss which types of hallucinations should be prioritized for different types of models.

1 INTRODUCTION

Hallucinations in Chinese large language models can produce plausible but factually erroneous content, motivating a benchmark tailored to Chinese models and contexts. HalluQA addresses gaps in existing evaluations by covering both imitative falsehoods and factual errors.

  • HalluQA evaluates hallucinations in Chinese large language models because plausible factual errors can mislead users, including in finance, medicine, and law.The benchmark is intended to support the development of more trustworthy AI.Source: §1 · p. 1
  • Llama2 experiments suggest alignment improves responses to preference-misaligned questions, while scaling improves questions requiring background knowledge.Alignment particularly helps with subjective questions, model-identity recognition, and fiction–reality distinctions.Source: §1 · p. 2, Figure 1 · p. 2
  • Existing benchmarks incompletely cover hallucination types: TruthfulQA primarily targets imitative falsehoods, whereas factual-error benchmarks may omit imitative falsehoods.The paper therefore argues that hallucination evaluation should include both categories.Source: §1 · pp. 2–3, Table 2 · p. 3
  • HalluQA contains 450 adversarial questions spanning history, literature, folklore, science, geography, and art in Chinese cultural contexts.The questions are designed for Chinese large language models and powerful aligned models.Source: §1 · p. 3
  • The benchmark distinguishes imitative falsehoods linked to alignment from factual errors linked to knowledge capability.This distinction motivates evaluating both misleading questions and knowledge-based questions.Source: §1 · p. 3
  • The paper contributes a HalluQA benchmark, extensive evaluations across model types, and analysis of which hallucination categories different models should prioritize.The evaluated models include pre-trained, chat, and retrieval-augmented systems.Source: §1 · p. 3

2 THE HALLUQA BENCHMARK

HalluQA defines hallucinations as responses inconsistent with reality and builds a Chinese benchmark covering misleading questions, knowledge-based factual errors, quality-controlled annotations, and 30 domains. Its construction combines model-generated adversarial-question screening with author and expert review, producing balanced correct and wrong answers for challenging evaluation.

2.1 THE HALLUCINATION CRITERIA IN HALLUQA

  • HalluQA treats non-refusal responses inconsistent with reality as hallucinations, while refusals are also hallucinations unless the reference answer is unanswerable.Source: §2.1 · p. 3

2.2 DATA COLLECTION

  • The benchmark separates misleading questions targeting imitative falsehoods from knowledge questions targeting factual errors, with alignment and knowledge enhancement or retrieval identified as corresponding remedies.Source: §2.2 · p. 4
  • Misleading data includes 175 GLM-130B-screened questions and 69 ChatGPT-generated misleading-hard questions, each paired with four correct and four incorrect answers plus supporting evidence.Source: §2.2 · p. 4
  • Knowledge data was generated and screened through repeated ChatGPT and Puyu responses, expert filtering, and annotation checks, yielding 206 questions across 14 domains.Source: §2.2 · pp. 4–5

2.3 QUALITY ASSURANCE

  • Quality assurance used expert verification and external knowledge links for knowledge questions, while uninvolved authors reviewed misleading questions and rewrote or discarded low-quality items.Source: §2.3 · p. 5

2.4 DATA STATISTICS

  • The final test data covers 30 domains and uses adversarial samples designed for powerful pre-trained and conversational models, with correct and wrong answer counts matched.Source: §2.4 · p. 5

3 EXPERIMENTS

The experiments evaluate three model types with a GPT-4-based hallucination judge and find that HalluQA remains challenging, while retrieval and alignment affect question categories differently.

3.1 MODELS / 3.2 METRIC

  • The benchmark evaluates pre-trained, chat, and retrieval-augmented chat models using the non-hallucination rate, defined as the percentage of generated answers without hallucinations.Answers are judged for fluency, directness, consistency with correct examples, and appropriate refusal when questions are unanswerable.Source: §3.1 · p. 6, §3.2 · p. 7

3.3 EVALUATION METHOD

  • GPT-4 evaluates outputs against instructed criteria and reference answers, with five judgments voted to produce the final decision.A 600-sample comparison found relatively high consistency between GPT-4 and human evaluations, with randomness having little effect on consistency.Source: §3.3 · p. 7, Table 4 · p. 7

3.4 MAIN RESULTS AND ANALYSIS

  • 18 of 24 tested models achieved non-hallucination rates below 50%, showing that HalluQA is highly challenging for current Chinese large language models.ERNIE-Bot exhibited the fewest hallucinations among the tested models.Source: §3.4 · p. 8, Figure 5 · p. 8
  • Retrieval-augmented models generally achieve higher non-hallucination rates than pre-trained models, while closed-source models tend to outperform open-source models.Non-hallucination rates vary substantially among chat models, and all model stages retain room for improvement.Source: §3.4 · p. 8
  • Alignment improves chat-model handling of misleading questions but may reduce knowledge-question performance, an effect attributed to an alignment tax.Pre-trained models often hallucinate on misleading questions, whereas chat models improve there but decline on knowledge-based questions.Source: §3.4 · pp. 8–9, Figure 6 · p. 9
  • Retrieval enhancement significantly reduces hallucinations on knowledge-based questions but provides little improvement on misleading questions, especially misleading-hard cases.The study reports a slow increase in non-hallucination rates for misleading-hard questions across all three model types.Source: §3.4 · p. 9

4 DISCUSSION

The discussion argues that hallucination priorities should differ by model type because models show different weaknesses across misleading and knowledge-based questions.

4 DISCUSSION

  • Pre-trained models should prioritize misleading questions, while scaling model size and improving training data may reduce their factual errors on knowledge-based questions.Chat models should prioritize misleading-question hallucinations through alignment without substantially losing knowledge-question capability.Source: §4 · p. 9
  • Retrieval-augmented chat models should primarily address misleading-hard questions, which may represent edge cases not typically encountered during alignment.These models already combine alignment with external knowledge enhancement.Source: §4 · p. 9

5 RELATED WORK

The related work reviews representative Chinese language models, existing hallucination benchmarks, and LLM-based evaluation methods that motivate HalluQA.

5.1 CHINESE LARGE LANGUAGE MODELS

  • Chinese model families include pretrained systems such as PanGu-α, GLM-130B, and Baichuan2, alongside aligned chat models such as ChatGLM and Qwen-chat.These models differ in scale, training data, multilingual coverage, and whether they use supervised fine-tuning or reinforcement learning from human feedback.Source: §5.1 · pp. 9–10

5.2 HALLUCINATIONS AND BENCHMARKS

  • Hallucinations in LLMs primarily denote plausible-looking outputs that are inconsistent with reality, while existing benchmarks cover truthfulness, factuality, generated hallucinations, and self-knowledge.TruthfulQA, ChineseFactEval, HaluEval, and SelfAware use different datasets and evaluation procedures, with HalluQA positioned against these benchmarks.Source: §5.2 · p. 10

5.3 EVALUATION WITH LLMS

  • Prior work shows that LLMs can serve as automated evaluators, with GPT-4 reaching 80% consistency with human judgments on MT-Bench and LLM-based methods outperforming other automated approaches for open-domain QA.These findings support using natural-language instructions and LLM judgments for customized evaluation criteria.Source: §5.3 · p. 10

6 CONCLUSION

The paper introduces HalluQA, a Chinese hallucination benchmark that evaluates both imitative falsehoods and factual errors with automated LLM-based judging across 24 models.

6 CONCLUSION

  • HalluQA evaluates Chinese LLM hallucinations using questions targeting imitative falsehoods and factual errors, an LLM-based automated evaluator, and experiments on 24 models.All models achieved less than a 70% non-hallucination rate, and the paper analyzes hallucination types and priorities for different model types.Source: §6 · p. 10

A DETAILED NON-HALLUCINATION RATES OF ALL MODELS

The appendix provides a detailed table of non-hallucination rates across models and question types.

A DETAILED NON-HALLUCINATION RATES OF ALL MODELS

B ANALYSIS OF QUESTION PATTERNS IN TRUTHFULQA

This analysis examines which TruthfulQA question patterns improve after alignment or scaling and illustrates alignment-corrected examples using Llama2 evaluations.

B ANALYSIS OF QUESTION PATTERNS IN TRUTHFULQA

  • The analysis identifies three question categories with the largest alignment improvements and three with the greatest scaling improvements, then examines corrected test cases within those categories.Figure 7 presents summarized alignment-improvement patterns, while Figure 8 presents scaling-improvement patterns.Source: B ANALYSIS OF QUESTION PATTERNS IN… · p. 16

C TESTING LLAMA2 ON TRUTHFULQA

  • Llama2 TruthfulQA evaluation reuses the benchmark’s scoring setup by fine-tuning two 6.7B GPT-3 scorers and matching Llama2’s prompting and generation configurations.For chat models, the QA prompt pairs are represented as multi-turn dialogue history.Source: C TESTING LLAMA2 ON TRUTHFULQA · p. 16
  • Figure 7 gives examples of questions answered correctly after alignment.Source: Figure 7 · p. 17

D PROMPTS

The paper uses separate Chinese QA and GPT-4 evaluation prompts to standardize model outputs and classify hallucinations. These prompts retain fixed demonstrations while replacing task-specific placeholders during inference and evaluation.

D.1 CHINESE QUESTION-ANSWERING PROMPT

  • The Chinese QA prompt manually provides six Chinese question-answer examples to make pretrained models produce question-answer formatted outputs.The prompt structure appears in Figure 9.Source: §D.1 · p. 17, Figure 9 · p. 18
  • During inference, the tested question replaces the red placeholder, while the six QA demonstrations remain fixed.Source: §D.1 · p. 17
  • Figure 9 presents the Chinese question-answering prompt structure used to format model responses.Source: §D.1 · p. 17, Figure 9 · p. 18

D.2 EVALUATION PROMPT FOR GPT-4

  • The GPT-4 evaluation prompt uses a conversational format with criteria and guidance in the first turn, followed by an affirmative assistant response.Source: §D.2 · p. 19, Figure 10 · p. 19
  • In the second turn, GPT-4 receives the question, model response, correct-answer examples, and an instruction to output “Yes” or “No” for hallucination evaluation.Source: §D.2 · p. 19
  • The evaluation prompt replaces red placeholders with the question, correct-answer examples, and the model’s response being judged.Source: §D.2 · p. 19, Figure 10 · p. 19

E EXAMPLE DATA FROM HALLUQA

HalluQA examples illustrate how its data sections target hallucinations through misleading prompts and fine-grained knowledge questions. The section also provides English translations for representative examples.

F CONSISTENCY BETWEEN GPT-4 AND HUMAN

GPT-4 evaluations generally agree closely with human evaluations across models and question types, supporting their use as a lower-cost alternative. Agreement is higher for knowledge-based than misleading questions, although GPT-4 itself produces some evaluation errors.

Related papers

Loading 2310.03368v4…