Signals of AI Hallucination: Designing Hallucination-Aware Cues for Embodied Conversational Agents in VR

Motivation

• Text-based conversational agents have used color highlighting, source links, and provenance cards to expose uncertainty or evidence, and one cited study reports faster verification with additional cues [52].These approaches provide context for the paper’s comparison of cue designs in immersive VR.Source: §1 · p. 1
• In VR, persistent inline cues lack a clear anchor, while reading them aloud can lengthen turns, disrupt answer flow, and increase task load.The authors identify uncertainty and provenance as the information that hallucination-awareness interfaces need to convey.Source: §1 · p. 1

Study design

• The study compares embodied gestures and posture, visual icons, and color-coded text with inline citations against a no-cue baseline in a within-subjects study of 24 participants.The cues communicate response uncertainty and source provenance.Source: §1 · pp. 1–2

Study findings

• All three cue designs improved identification of hallucination-related information over the no-cue baseline, with different trade-offs in trust, immersion, interpretability, and disruption.Embodied cues aligned with the agent’s social behavior and supported trust and immersion; text cues were clearest, while icons preserved relatively good interpretability with less immersion disruption.Source: §1 · p. 2

2.1 LLM Hallucination and Hallucination Detection

• Hallucinations include factual, input, and context conflicts; detection may assess factuality against external evidence or faithfulness to the input, and this paper uses a faithfulness-based backend.Examples of prior methods include Chain-of-Verification and SelfCheckGPT for factuality, and QAFactEval and QuestEval for faithfulness [15] [47] [19] [61].Source: §2.1 · p. 2

2.2 Hallucination-Awareness Designs for CA

• Hallucination-awareness designs communicate uncertainty and provenance, but text-heavy cues developed for chat or dashboards may interrupt flow and reduce immersion in speech-led settings.The paper operationalizes internal model-derived signals as uncertainty and external supporting-source evidence as provenance.Source: §2.2 · p. 2

2.3 Hallucination Risk in Immersive ECAs

• Immersive ECAs share the user’s 3D space, and their speech and embodied signals may make hallucinated content feel more credible or easier to overlook; cue design in this setting remains little studied.ECA interaction combines verbal and nonverbal channels, while immersive presentation may intensify social responses and trust depending on design and task context.Source: §2.3 · pp. 2–3

3.1 Overview of ECA Setup in VR

• The VR prototype uses a GPT-4o-driven humanoid ECA, speech, and a handheld panel for source inspection; all cue conditions share detection outputs and voice, with a no-cue baseline.The setup is a classroom scenario built in Unity and runs on Meta Quest 3.Source: §3.1 · p. 3

3.3 Designs of Hallucination-Aware Cues

• The four conditions are embodied cues, icon cues, text cues, and a no-cue baseline, all using the common underlying sentence-level labels and source annotations.Source access through the handheld reference panel is held constant.Source: Figure 2 · p. 3, §3.3 · p. 3

3.2 Representing Hallucination-Related Information for Cue Design

• The design represents model-derived internal indicators as uncertainty and cited supporting evidence as provenance, with the same uncertainty labels and source-availability annotations across conditions.The baseline retains ordinary embodied appearance and conversational actions but presents no hallucination-aware cues; source webpages remain accessible through the same panel across conditions.Source: §3.3 · p. 3, §3.2 · p. 3

3.3.1 Embodied Cues

• Embodied cues use reflective and defensive postures for uncertain and unconfident content, plus a citation gesture when source evidence is available; no dedicated gesture signals confident content.Hallucination-specific gestures temporarily take precedence over default conversational actions; the authors note gesture interpretation may vary culturally.Source: §3.3.1 · pp. 3–4

3.3.2 Icon Cues

• Icon cues pair a lightbulb’s traffic-light color with confidence level and place a website icon beside it when cited source evidence is available.Green, yellow, and red correspond to confident, uncertain, and unconfident, respectively.Source: §3.3.2 · p. 4

3.3.3 Text Cues

• Text cues synchronize speech with on-screen subtitles, color-coded confidence highlights, and abbreviated website labels for externally sourced sentences.The source labels are adaptive rectangles intended to make provenance more salient.Source: §3.3.3 · p. 4

4 METHODS

• A within-subjects study compared the three cue designs with a baseline lacking hallucination-awareness cues.The study was approved by the university IRB and recruited 24 participants aged 19–35, with varied VR experience and self-reported familiarity with AI hallucinations.Source: §4 · p. 4, §4.1 · p. 4

4.2 Hallucination Identification Task Design

• Participants watched short VR lessons on four historical topics, each using a roughly 330-word script containing externally referenced and potentially hallucinated statements.Five hallucination-related segments per script were distributed at randomized positions; a pre-test checked limited prior topic knowledge and systematic differences across topics.Source: §4.2 · p. 4
• Participants flagged content they thought warranted further checking; successful detection meant flagging sentences labeled Uncertain or Unconfident, not judging whether they were definitively false.Participants could inspect cited webpages through a handheld panel but were not required to do so.Source: §4.2 · p. 4

4.3.1 Questionnaires

• Questionnaires measured perceived identification ability, trust, immersion, and task load using study items and the Trust in Automation Scale, Immersive Experience Questionnaire, and raw NASA-TLX.Perceived identification used two five-point items; the IEQ has 31 items and six subscales, while NASA-TLX covers six task-load dimensions.Source: §4.3.1 · p. 5

4.3.2 Logs on Hallucination Identification

• Button-press logs were compared with system-assigned sentence labels using a confusion matrix to calculate Precision, Recall, F1, and Accuracy.The system labels served as the reference set rather than independently verified factual ground truth.Source: §4.3.2 · p. 5

4.3.3 Semi-structured Interview

• A 30-minute semi-structured interview gathered impressions, comparisons across conditions, attention and checking strategies, trust, interaction experience, preferences, and factors shaping those preferences.The interviews were used to interpret and triangulate quantitative findings.Source: §4.3.3 · p. 5

4.4 Study Procedure

• 24 participants experienced all four conditions in a balanced Latin-square order, while historical topics remained in a fixed order.This counterbalancing varied interface order while keeping content progression consistent; no condition was tied to a single topic.Source: §4.4 · p. 5
• After cue training, participants used a controller to mark content needing further verification during four five-minute tasks, consulting cues or a source panel as desired.The system compared button-press judgments with system-assigned labels.Source: §4.4 · p. 5
• Participants completed questionnaires after each condition, took five-minute breaks, and finished with an approximately 30-minute semi-structured interview.All participants completed the tasks, and none reported severe symptoms after breaks.Source: §4.4 · pp. 5–6

4.5.1 Quantitative Data Analysis

• Questionnaire items were checked with Cronbach’s α, removing low-correlation items when reliability fell below 0.7.The team also calculated pooled reliability across conditions.Source: §4.5.1 · p. 6
• After reverse-scoring items, the researchers used repeated-measures ANOVA for normally distributed measures and Friedman tests otherwise, with corrected post hoc comparisons after significant omnibus tests.Paired t-tests or Wilcoxon signed-rank tests were used as appropriate; effect sizes were Cohen’s d or rank-biserial correlation.Source: §4.5.1 · p. 6

4.6 Qualitative Analysis

• Interview transcripts were openly coded, iteratively refined into candidate themes, and reviewed collaboratively to contextualize quantitative results and examine preferences.The analysis followed Braun and Clarke’s reflexive thematic approach.Source: §4.6 · p. 6

5.1 Quantitative Results

• All questionnaires reached acceptable reliability (α > 0.7) after the IEQ was refined; most variables failed normality tests.Frustration, trust, accuracy, and the IEQ Cognitive and Control subscales were exceptions to the normality pattern.Source: §5.1 · p. 6

5.1.1 User Ability of Hallucination Identification

• All three cue conditions significantly outperformed Baseline on Precision, Recall, F1 Score, and Accuracy, with no significant differences among cue conditions.Figure 3 presents the confusion-matrix results; the comparison used system-assigned labels as the reference set in the analysis.Source: Figure 3 · p. 6, §5.1.1 · p. 6
• Participants rated their ability to identify content needing further checking and hallucination-related cues higher in every cue condition than at Baseline.For Q1, Text Cue also scored higher than Embodied Cue; no such between-cue difference is reported here for Q2.Source: §5.1.1 · p. 6, Figure 4 · p. 7

5.1.2 User Trust

• Trust scores were significantly higher than Baseline for Embodied Cue (M = 3.6), Icon Cue (M = 3.48), and Text Cue (M = 3.55).Figure 4 presents perceived-ability and trust questionnaire results.Source: §5.1.2 · p. 6, Figure 4 · p. 7

5.1.3 Level of Immersion

• All three cue conditions scored above Baseline on Cognitive and Control immersion; Embodied Cue also exceeded Baseline on General Immersion.Figure 5 presents the Immersive Experience Questionnaire results.Source: Figure 5 · p. 7, §5.1.3 · p. 7

5.1.4 Task Load

• Text Cue had lower Mental Demand than Embodied and Icon Cues, while Baseline scores were higher than Icon and Text Cue on Performance and higher than Text Cue on Effort.No significant main effects were found for the remaining NASA-TLX subscales; Figure 6 notes that Performance is reverse-coded.Source: Figure 6 · p. 7, §5.1.4 · p. 7

5.2.1 Theme 1: Familiar cues become intuitive and make instant sense

• Familiar icon and text cues felt intuitive, supporting quick identification; unclear embodied gestures required interpretation and could distract from listening.Participants also said familiar icons could help them focus on the lecturer avatar and assess familiar source websites at a glance.Source: §5.2.1 · p. 7

5.2.2 Theme 2: Embodied cues keep immersive experience but are easy to miss and require further interpretation

• Embodied cues felt realistic and immersive but blended with social gestures, while participants preferred subtitles for locating and verifying the precise hallucinated content.Some participants suggested exaggerating gestures or narrowing highlighted text to suspicious words and phrases.Source: §5.2.2 · pp. 7–8

5.2.3 Theme 3: Cue modality and explainable hallucination awareness influences trust on ECA

• Participants associated human-like embodied cues with trust, but some distrusted gestures signaling erroneous content; clear rationales and source evidence also supported trust.Text cues enabled statement verification, and source badges offered links or screenshots for secondary confirmation.Source: §5.2.3 · p. 8

6.1 Principal Results

• All three cue conditions improved identification relative to baseline; text cues reduced Mental Demand, embodied cues increased immersion, and icon and text cues improved task performance over baseline.Embodied cues had the highest trust ratings, but differences between cue types were not statistically significant.Source: §6.1 · p. 8

6.2.1 Balance representational informativeness and visual load in hallucination-awareness design

• Text highlights and source labels made information easier to locate and interpret, but too many visual elements could clutter the interface while too few left users unsure what to check.The discussion recommends calibrating representational detail, persistence, and explicitness to context and user needs.Source: §6.2.1 · p. 8

6.2.2 Highlight the conspicuity of hallucination-related actions to enhance the awareness of embodied cues

• Embodied cues could be mistaken for ordinary emotional or conversational actions, helping explain why text cues elicited greater confidence in identifying hallucination-related information.The discussion recommends noticeable but socially integrated gestures, with clearer motion contrast or consistent gesture-to-meaning mappings; mappings may vary across cultures.Source: §6.2.2 · p. 8

6.2.3 Integrate multimodal hallucination cues to achieve natural fusion and preserve immersion

• Text cues were more explicit and inspectable, while embodied cues aligned more closely with conversational style and supported immersion; the discussion proposes coordinated multimodal cues rather than a single cue type.Embodied signals may draw attention, visual cues provide explanations, and verbal feedback offers reasoning clues; modalities should complement rather than duplicate one another.Source: §6.2.3 · pp. 8–9

• Participants had limited prior knowledge, so the study mainly captures judgments based on system cues rather than how users balance cues against their own domain knowledge.The authors call for testing users with different prior knowledge, especially when they disagree with the system or use independent verification strategies.Source: §7 · p. 9
• Judgments were compared with system-assigned uncertainty labels, not independently verified factual annotations, so the study does not establish performance when backend detection is imperfect.The authors recommend evaluation against factual reference labels and examination of how detection errors affect trust and verification.Source: §7 · p. 9
• The tightly controlled, scripted history-learning task simplified interaction and did not capture open-ended dialogue involving interruptions, follow-ups, changing goals, memory, or conversational repair.The authors recommend studying cues in more interactive settings such as tutoring, collaborative planning, or exploratory information seeking.Source: §7 · p. 9
• The embodied-cues condition had no gesture for high-confidence content, leaving the effects of selective confidence gestures on judgment calibration and overreliance untested.The authors propose independently varying gesture presence and frequency, estimated confidence, and factual correctness.Source: §7 · p. 9
• Findings came from VR and should not be assumed to transfer unchanged to AR or non-immersive interfaces; proposed directions for those media require empirical validation.The discussion offers medium-specific design directions for AR and non-immersive agents.Source: §7 · p. 9
• The three representative cue types do not exhaust the design space; prototypes presented uncertainty and provenance at response delivery and assumed relatively stable cue meanings.The authors note that gesture meanings can vary by cultural and interaction context and call for studying which representations and combinations work for whom and when.Source: §7 · p. 9

• All three cue designs improved hallucination detection and trust; embodied cues strengthened immersion, text cues improved confidence and efficiency, and icon cues balanced immersion and task efficiency.Qualitative findings linked embodied-cue trust to human–agent relationship quality and text-cue outcomes to information density and interpretability.Source: §8 · p. 9
• Qualitative findings linked embodied cues’ trust effects to stronger human–agent relationships, while text cues supported trust and detection through information density and interpretability.The authors describe these findings as an exploratory basis for designing ECA hallucination-awareness interfaces.Source: §8 · p. 9