PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding LLM Hallucinations Caused by Soft Errors

Jinghao Wen, Dongning Ma, Ruixuan Wang, Shaohuang Wang, Xun Jiao

arXiv:2610.22520v1cs.SEcs.AR

TL;DR

The paper addresses limited evidence on whether hardware soft errors can trigger LLM hallucinations. It studies soft-error injection with LLMEI and reports four hallucination patterns whose distributions shift with error severity, from disrupted reasoning to malformed tokens or generation collapse.

  • Problem

    Although LLM hallucinations have been widely studied from software perspectives, little is known about whether hardware soft errors contribute to them.

    Source: §1 · p. 1

  • Method

    The authors conduct a case study using LLMEI to inject bit-level soft errors into LLMs and examine resulting hallucination behavior.

    Source: §1 · p. 1, §3.2 · p. 3

  • Results

    The study identifies four hallucination types, with their distribution changing as bit error rate rises from reasoning and semantic disruption toward malformed tokens and generation collapse.

    Source: §6 · p. 6

  • Takeaways & Limitations

    The four-pattern taxonomy provides a basis for targeted error detection and mitigation strategies tailored to different failure modes.

    Source: §5.1 · p. 5

    The study targets multiple bit flips in SRAM on DNN accelerators and injects errors into model parameters.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

While hallucinations in Large Language Models (LLMs) have been extensively studied from a software perspective, limited attention has been paid to hardware-level factors. In this paper, we performed a case study to understand LLM hallucination caused by soft errors on hardware. We introduce LLMEI to inject soft errors into LLMs during inference, based on what we evaluate the impact of soft error across diverse LLM architectures and tasks, including mathematical reasoning and code generation. We summarize the taxonomy of different failure modes based on LLM hallucination behaviors and analyze their distributions under different error rates. Our find- ings underscore the hallucination-specific diagnostics within the reliability of LLM implementation on hardware.

1 Introduction

The paper addresses the limited study of hardware soft errors as a source of LLM hallucinations. It presents a case study, a four-pattern taxonomy, and LLMEI for analyzing and injecting soft errors.

1 Introduction

  • The paper investigates whether hardware soft errors can trigger hallucinations, a factor less studied than software-level causes.It describes soft errors as transient hardware-induced faults that can corrupt data during execution.Source: §1 · p. 1
  • The case study systematically examines how different scales of soft errors affect LLM generation and introduces an application-level taxonomy of induced hallucinations.The authors also report four main hallucination patterns and analyze their distributions across error rates, architectures, tasks, and model components.Source: §1 · p. 1
  • LLMEI is presented as an efficient, flexible framework for large-scale soft-error injection into LLMs.The paper reports that the framework is open-sourced, with code and documentation in the supplementary materials.Source: §1 · p. 1

2 Related Works

Prior hallucination research has mainly examined software-level causes and algorithmic mitigations. The paper identifies limited study of whether hardware soft errors can directly trigger hallucination-like behavior in LLMs.

2 Related Works

  • Prior hallucination work has focused mainly on software-level causes, including training data, alignment, decoding, retrieval, and prompt sensitivity.Mitigations are typically algorithmic, including retrieval-augmented generation, uncertainty estimation, self-consistency, verification, and detection.Source: §2 · p. 2
  • The paper identifies limited study of whether hardware soft errors can directly trigger hallucination-like behavior in LLMs.Transient hardware disturbances can cause bit flips that may propagate through model computations into downstream behavior.Source: §2 · p. 2
  • The study frames its focus as examining whether low-level hardware faults can manifest as high-level generation failures.This addresses a gap alongside prior work centered on algorithmic causes and mitigation.Source: §2 · p. 2

3 Error Injection

The paper scopes its study to bit flips in LLM parameters under modeled hardware soft errors, while describing how such faults may destabilize generation. LLMEI supports configurable error-map generation and injection into LLM targets.

3.1 Background and Scope

  • Soft errors may disrupt internal representations and, through autoregressive generation, influence later token predictions, producing semantic drift or abrupt failures.The study targets multiple bit flips in SRAM on DNN accelerators, characterized by bit error rate, and injects them into model parameters.Source: §3.1 · pp. 2–3

3.2 Workflow of LLMEI

  • LLMEI allocates binary error maps for target parameters or activations, matches user-specified configurations, generates maps, and applies them to the LLM.Users can specify a bit-flip rate or a fixed number of flips for each target.Source: §3.2 · p. 3
  • LLMEI uses tensor views and bitwise operations to speed up injection, reporting an increase in overall injection-phase speed of more than 1,000 times.The framework avoids type-casting floating-point tensors and supports configurable bit flips.Source: §3.2 · p. 3

4 Experimental Setup

The experiments cover mathematical reasoning and code generation using models from multiple architectures and scales. The setup varies bit error rates and evaluates pass@1 across repeated trials.

4 Experimental Setup

  • The evaluation uses AIME 2024 for mathematical reasoning and LiveCodeBench for code generation.The tasks respectively assess symbolic, multi-step reasoning and correctness on programming problems.Source: §4 · p. 3
  • The evaluated models span DeepSeek distilled Qwen and LLaMA variants for reasoning, plus multiple model families and sizes for code generation.The code-generation set includes base and instruction-tuned variants.Source: §4 · p. 4
  • The component profiling covers self_attn, mlp, lm_head, and embed_tokens, with mlp layers typically accounting for over 60% of parameters.The profiling supports targeted injection into model components.Source: §4 · p. 4, Figure 3 · p. 4
  • The setup situates its BER range against reported hardware conditions, including higher rates for accelerator SRAM than ECC-protected datacenter DRAM.The cited ranges are 10^-12 to 10^-11 for ECC-protected DRAM and around 10^-10 to 10^-8 for on-chip SRAM.Source: §4 · p. 4, Table 1 · p. 4
  • The study evaluates BER values from 10^-12 to 10^-4 and averages pass@1 across 10 trials per injection configuration.Pass@1 is the primary metric for functional correctness on both tasks.Source: §4 · p. 4

5 Results and Analysis

The study characterizes soft-error-induced hallucinations across tasks and model components, finding that both BER and task or component structure shape how failures emerge. Its analysis covers four failure types and examines their distributions and robustness under increasing error rates.

5.1 Hallucinations Induced by Soft Errors

  • Soft errors produce four distinct failure modes: Token Corruption, Language Drift, Logical Corruption, and Premature Termination.The study describes these as application-level patterns in corrupted outputs.Source: §5.1 · p. 5

5.2 Hallucinations Distribution across Tasks

  • Increasing BER shifts hallucinations from logical corruption toward token corruption, with mathematical reasoning degrading in stages and code generation collapsing abruptly.For code generation, Category I takes over beyond BER 10^-9; mathematical-reasoning outputs retain structure under mild corruption before collapsing as errors accumulate.Source: Figure 4 · p. 5, §5.2 · p. 5

5.3 Hallucinations Distribution across Components

  • Component-level hallucination distributions differ: self_attn and mlp shift rapidly from Category III to Category I at moderate BERs, while embed_tokens remains comparatively robust.At BER 10^-7, lm_head shows a mix of Categories II and IV; embed_tokens instead yields more diverse, less destructive failures such as Categories III or II.Source: §5.3 · p. 5, Figure 5 · p. 6

5.4 Model- and Component-Wise Error Robustness

  • Robustness varies across models and components; model size alone does not determine sensitivity, and self_attn and mlp generally lose Pass@1 earlier than embed_tokens.Across three DeepSeek-R1 distilled models, embed_tokens retains non-zero performance over a substantially wider BER range, while lm_head has intermediate sensitivity.Source: §5.4 · p. 6

6 Conclusion

The case study finds that hardware soft errors can propagate through LLM computation and appear as high-level hallucinations. The resulting failures span four types, and their distribution changes with error severity.

  • The study categorizes soft-error-induced hallucinations as Token Corruption, Language Drift, Logical Corruption, and Premature Termination.Source: §6 · p. 6
  • Relatively mild errors are more likely to disrupt reasoning and semantic consistency.Source: §6 · p. 6
  • Severe corruption can lead to malformed tokens and complete generation collapse.Source: §6 · p. 6
Loading 2610.22520v1…