PaperlayerView exact revision on arXiv

Source-linked AI summary

Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models

Yu Zhang, Pingrui Zhang, Xuefeng Bai, Pengfei Zhang, Yang Xiang, Kehai Chen

arXiv:2609.37568v1cs.CL

TL;DR

AVLLMs can ground answers in cues from the wrong modality, but how internal cross-modal interactions produce this failure remains insufficiently understood. The paper identifies question states as a relay for required-source evidence and interfering cues, then introduces SECRET to steer those states toward required evidence. SECRET improves results across CMM and AVHBench and shows improved source grounding in open-ended captioning.

  • Problem

    Source-confused grounding hallucination arises when cues from a non-required modality induce responses unsupported by the required modality, a challenge whose internal cross-modal origins remain insufficiently understood.

    Source: §1 · p. 1

  • Method

    SECRET uses source-conditioned pathway interventions to contrast question representations and steer original question states toward required-modality evidence.

    Source: §1 · p. 2, §3.1 · p. 5

  • Results

    SECRET consistently improves results on CMM and AVHBench across three AVLLMs, with average accuracy gains over base models of up to 18.0 and 7.1 percentage points, respectively.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    Modality-specific captioning evaluations show improved source grounding in open-ended generation.

    Source: §6 · p. 10

    The evaluation uses existing AVHBench and CMM benchmarks and modality-specific captioning inputs described in Appendix E, so the reported scope is bounded by those evaluations.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual, auditory, and linguistic information. However, recent studies show that AVLLMs face a critical challenge: $\textbf{source-confused grounding hallucination}$, where cues from the unused modality induce responses that the required modality does not support, undermining reliability in real-world applications. Existing methods have made progress in mitigating this failure, yet how it arises from internal cross-modal interactions remains insufficiently understood. To address this gap, we conduct path-intervention and representation analyses, revealing a $\textbf{question-relay}$ mechanism: question states carry interfering cues alongside required-source evidence, undermining grounding in required-modality evidence. Cutting pathways from interfering modality to question states yields greater correct-answer logit recovery than cutting those to the generation position. Motivated by these findings, we propose $\textbf{SECRET}$ ($\textbf{S}$ourc$\textbf{E}$-$\textbf{C}$onditioned $\textbf{RE}$lay s$\textbf{T}$eering), a training-free method that mitigates cross-modal interference at the question relay. Using contrasting question representations elicited through different modality-pathway interventions, SECRET steers the original question states toward required-source evidence. Experiments on two widely adopted benchmarks CMM and AVHBench across three AVLLMs show that SECRET consistently outperforms prior training-free methods, substantially mitigating source-confused grounding hallucinations (e.g., up to +18.0 and +7.1 percentage points over base models). Modality-specific captioning further demonstrates its generalizability to open-ended generation.

1 INTRODUCTION

AVLLMs can answer from cues in the wrong modality, while the internal pathways behind this failure remain insufficiently understood. The paper identifies question states as a relay for both required-source evidence and interfering cues, then introduces SECRET to steer that relay toward required-modality evidence.

1 INTRODUCTION

  • Source-confused grounding hallucination occurs when cues from a non-required modality induce responses unsupported by the required modality, undermining AVLLM reliability.For example, a visible piano can prompt a piano-music answer even when the audio contains only human speech.Source: §1 · p. 1
  • Path-intervention and representation analyses identify question states as a relay where interfering cues accompany required-source evidence and influence source-specific answers.Cutting interfering-modality pathways to question states yields greater correct-answer logit recovery than cutting pathways to the generation position.Source: §1 · p. 2
  • SECRET consistently outperforms evaluated training-free methods across three AVLLMs, improving average accuracy over base models by up to 18.0 and 7.1 percentage points on CMM and AVHBench.It also improves source grounding in modality-specific captioning under mismatched audio-video inputs.Source: §1 · p. 2
  • The paper proposes SECRET, a training-free method that constructs source-conditioned question representations through pathway interventions and steers original question states toward required-modality evidence.Its contributions include identifying the question-relay mechanism and demonstrating results across three AVLLMs and modality-specific captioning.Source: §1 · p. 2

2 UNDERSTANDING SOURCE-CONFUSED GROUNDING

The analyses trace source-confused grounding from required-modality identification through cross-modal pathways to question states. Those states relay required evidence as well as interfering cues, and cutting the interfering route to them restores correct-answer support more effectively than cutting the route to the generation position.

2 UNDERSTANDING SOURCE-CONFUSED GROUNDING

  • The progressive analyses find that hallucination is not simply misunderstanding the requested evidence source, then trace required-source evidence and interfering cues through question states.These findings motivate source-conditioned relay steering to mitigate cross-modal interference.Source: §2 · pp. 2–3

2.1 PRELIMINARIES

  • The analysis represents video, audio, and question tokens as a combined sequence and defines source-faithful answers as supported by the instruction-required modality.It examines the Video-Driven Audio Hallucination and Audio-Driven Video Hallucination subsets of AVHBench, whose questions specify audio or video evidence.Source: §2.1 · p. 3

2.2 OBSERVATION 1: AVLLM CAN RELIABLY IDENTIFY THE REQUIRED MODALITY

  • Qwen2.5-Omni-7B identifies the question’s required evidence modality correctly for 99.85% of questions, suggesting hallucinations are not simply a failure to identify that modality.The diagnostic asks the model to classify the required evidence as audio, video, or ambiguous without answering the original question.Source: §2.2 · p. 3

2.3 OBSERVATION 2: QUESTION STATES RELAY REQUIRED-SOURCE EVIDENCE

  • On source-faithful cases, predictions depend most strongly on the pathway from required-modality tokens to question tokens, identifying question states as a relay for required-source evidence.Attention-path cutting compares required-modality pathways to question tokens, interfering-modality tokens, and the generation position.Source: §2.3 · pp. 3–4, Figure 2 · p. 4

2.4 OBSERVATION 3: INTERFERING CUES IN QUESTION STATES INFLUENCE PREDICTIONS

  • Cutting interfering-modality pathways to question tokens yields markedly larger correct-answer logit recovery than cutting pathways to the generation position at every tested layer.Interfering-pathway cuts also lower target-object scores in more than 90% of examples at every tested layer, indicating wrong-source cues in question states.Source: §2.4 · pp. 4–5, Figure 3 · p. 5

3 SOURCE-CONDITIONED RELAY STEERING (SECRET)

SECRET steers question states toward required-modality evidence using contrasting representations elicited by selectively cutting modality-to-question attention pathways. It identifies the required modality from the question alone, then applies the steering to updated question states while retaining the original audio-visual states.

3.1 REQUIRED-MODALITY IDENTIFICATION

  • The AVLLM predicts the required modality from the textual question alone to construct positive and negative references for question-state steering.The positive mask blocks interfering-modality paths to question tokens while preserving required-modality paths; the negative mask does the reverse.Source: §3.1 · p. 5

3.2 QUESTION-RELAY STEERING

  • Three shared-parameter branches process the same input through the first L1 layers, producing positive, original, and negative question states under different attention masks.The positive and negative branches block attention from the interfering and required modalities, respectively, into question tokens.Source: §3.2 · p. 5, Figure 4 · p. 6
  • SECRET adds each token-wise positive–negative difference, scaled to the original state’s L2 norm, to steer original question states toward required-source evidence.The updated question states are recombined with the original audio-visual states and passed through the remaining L2 layers for generation; steering occurs during prefill.Source: §3.2 · p. 6

3 SOURCE-CONDITIONED RELAY STEERING (SECRET)

  • SECRET uses internal attention-path interventions rather than altered modality inputs, preserving the audio-visual context while steering question states toward required-source evidence.The authors contrast this with prior approaches that perturb or remove modality inputs.Source: §3 · p. 5, §3.2 · p. 6

4 EXPERIMENTS

The experiments evaluate SECRET against training-free baselines on CMM and AVHBench, then analyze intervention design, steering depth, captioning, and fine-grained CMM outcomes. SECRET leads the evaluated methods on both benchmarks, with gains also reported in open-ended captioning and duration-stratified evaluation.

4.1 EXPERIMENTAL SETUP

  • Evaluation covers AVHBench and CMM across three AVLLMs, comparing SECRET with VCD, AVCD, and MAD training-free mitigation methods.AVHBench has 3,426 question–answer pairs and CMM has 800 questions; accuracies are reported by subset and as benchmark means.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • Up to 18.0 percentage points on CMM and 7.1 on AVHBench over base models, SECRET achieves the highest overall accuracy for each backbone across both benchmarks.The benchmarks span three AVLLMs with different scales and dense or mixture-of-experts architectures.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 ANALYSIS AND DISCUSSION

  • On both tested models, SECRET outperforms Gen-Interv. and Removal-Interv.; omitting norm matching also reduces accuracy.Figure 5(a) compares the variants on CMM.Source: §4.3 · p. 7, Figure 5 · p. 8
  • VideoLLaMA2-AV-7B peaks at L1 = 21 (89.3%) and Qwen2.5-Omni-7B at L1 = 25 (86.4%); each peak coincides with lowest positive–negative similarity.The models show opposite trends in mean representation similarity as intervention depth increases.Source: Figure 5 · p. 8, §4.3 · p. 8
  • Across both models and captioning tasks, SECRET achieves the lowest D-CIDEr and highest LLM-score while maintaining competitive T-CIDEr.The evaluation covers audio-target and video-target captioning.Source: Table 2 · p. 8, §4.3 · p. 8
  • SECRET raises mean and median answer margins for both backbones and gains accuracy across common duration groups, including 14.7 percentage points for VideoLLaMA2-AV-7B on 14–16-second clips.Answer margin is the correct-answer logit minus the incorrect-candidate logit at the first answer-token prediction position.Source: §4.3 · pp. 8–9, Figure 6 · p. 9

5 RELATED WORK

Prior work addresses cross-modal hallucinations with inference-time correction and training-time alignment, while internal cross-modal interactions remain insufficiently understood. This paper studies question states as an interference relay and uses source-conditioned contrasts to steer them toward required-source evidence.

Source-confused grounding hallucination

  • Prior mitigation methods use inference-time adaptive decoding or preference-based alignment, while the internal interactions underlying source-confused hallucination remain insufficiently understood.AVHBench and CMM evaluate failures across directions of cross-modal interference (Kim et al., 2024; Leng et al., 2024a).Source: §5 · p. 9

Mechanistic understanding of multimodal information utilization

  • Mechanistic studies analyze modality reliance at generation positions and preceding instruction positions, informing attention modulation, contrastive decoding, and token pruning.Examples include pathway interventions (Selvakumar et al., 2026) and instruction-position studies (Zhang et al., 2025d; 2026b; Suharitdamrong et al., 2026).Source: §5 · p. 9
  • This work extends mechanistic analysis to how non-required audio-visual cues propagate through question states, then introduces SECRET to steer those states toward required-source evidence.The related-work discussion identifies question states as an internal relay for cross-modal interference.Source: §5 · pp. 9–10

6 CONCLUSION

The paper identifies question states as a relay for interfering and required-source cues, and proposes training-free SECRET to steer generation toward required-modality evidence. Its evaluations report improvements on two benchmarks and modality-specific captioning, while the stated scope is existing benchmarks and described captioning inputs.

6 CONCLUSION

  • Path-intervention and representation analyses identify question states as relaying interfering cues alongside required-source evidence, motivating training-free SECRET's source-conditioned steering.The method contrasts question representations elicited through pathway interventions.Source: §6 · p. 10
  • Experiments across three AVLLMs show consistent improvements on CMM and AVHBench, and modality-specific captioning evaluations show improved source grounding in open-ended generation.The tested models are VideoLLaMA2-AV, Qwen2.5-Omni-7B, and Qwen3-Omni-30B-A3B.Source: §6 · p. 10, AI USE STATEMENT · p. 10
  • The evaluation uses existing AVHBench and CMM benchmarks and modality-specific captioning inputs described in Appendix E.The paper frames its findings as a structural foundation for developing more robust and transparent AI safeguards.Source: ETHICS STATEMENT · p. 10
  • The paper documents intervention, diagnostic, steering, representation, and captioning evaluation protocols to support reproducibility.Appendices B–E specify data construction, model settings, metrics, prompts, and scoring procedures.Source: REPRODUCIBILITY STATEMENT · p. 10

A DISCUSSION AND LIMITATION

Question states relay both required-source evidence and interfering cues, motivating source-aware control at this intermediate stage. The analysis is limited to pathway-level cross-modal information flow, with attention-head roles left for future study.

A DISCUSSION AND LIMITATION

  • Question states serve as a shared relay for audio-visual evidence while also carrying interfering cues into answer generation.A cue can be semantically related to the question yet remain inappropriate evidence for the requested modality.Source: A DISCUSSION AND LIMITATION · p. 14
  • SECRET applies inference-time control at the question relay using source-conditioned representation contrasts while retaining the complete audio-visual input.The discussion presents this as an example of mechanistic analysis informing a training-free intervention.Source: A DISCUSSION AND LIMITATION · p. 14
  • The analysis focuses on pathway-level cross-modal information flow; examining individual attention heads could further clarify the mechanism.Source: A DISCUSSION AND LIMITATION · p. 14

B.1 DATASET DETAILS AND STATISTICS

The AVHBench subsets cover both directions of cross-modal interference and distinguish visible sound sources, visible silent objects, and off-camera audible sources. The appendix also describes how required modality is identified and mapped for SECRET.

B.1 DATASET DETAILS AND STATISTICS

  • AVHBench includes 2,290 audio-grounded Video-Driven Audio Hallucination questions and 1,136 video-grounded Audio-Driven Video Hallucination questions.Together, the subsets contain 3,426 question–answer pairs drawn from VALOR and AudioCaps.Source: §B.1 · p. 14
  • The subsets distinguish visible sound sources, visible but silent objects, and audible sources outside the camera view, with the latter two serving as negative examples.Source: §B.1 · p. 14

B.2 REQUIRED-MODALITY IDENTIFICATION

  • Required-modality identification prompts classify questions as AUDIO, VIDEO, or AMBIGUOUS, and scoring counts ambiguous, invalid, and incorrect labels as errors.AUDIO and VIDEO guide SECRET's positive and negative attention masks; ambiguous predictions trigger random selection of a required modality.Source: §B.2 · p. 15

B.3 DETAILS AND MORE EXPERIMENTS FOR ATTENTION-PATH CUTTING ANALYSIS

Across tested window sizes and both hallucination settings, cutting the modality-to-question pathway produces the strongest middle-layer reductions in target-answer probability. The section also defines a target-object score for comparing original and intervened question states.

B.3 DETAILS AND MORE EXPERIMENTS FOR ATTENTION-PATH CUTTING ANALYSIS

  • Cutting Xr →XQ produces substantially larger middle-layer reductions in target-answer probability than cutting Xr →XG or Xr →X¯r across tested window sizes.The strongest effects remain concentrated in the middle layers, though window size changes their magnitude and layer-wise extent.Source: Figure 7 · p. 15, Figure 8 · p. 16, §B.3 · p. 16
  • The robustness analysis repeats pathway comparisons with three- and five-layer windows against the main analysis's seven-layer window in both hallucination settings.Source: §B.3 · p. 15

B.4 OBJECT EXTRACTION AND TARGET-OBJECT SCORES

  • The target-object score is the maximum logit for the extracted object across question positions at a layer, with higher values indicating stronger support in question states.The score is computed separately for original and intervened runs using the same target object and identical inputs.Source: §B.4 · p. 16
  • Figure 3(a) reports the share of examples with a lower intervened score and the mean score for each run.Source: §B.4 · p. 16

C EXPERIMENTAL AND IMPLEMENTATION DETAILS

SECRET uses model-specific steering depths selected after modality-to-question information transfer, then updates aligned question-token states. The appendix also specifies baseline constructions and intervention variants that alter the steering location or representation contrast.

C.1 IMPLEMENTATION DETAILS OF SECRET

  • SECRET steering depths are selected after the main modality-to-question transfer stage, with depths 25, 21, and 32 for Qwen2.5-Omni-7B, VideoLLaMA2-AV-7B, and Qwen3-Omni-30B-A3B, respectively.Source: §C.1 · p. 17
  • At the selected depth, aligned positive, negative, and original question-token states are used for the update, excluding the generation position; updated states rejoin the original branch for remaining layers.Source: §C.1 · p. 17

C.2 BASELINES AND INTERVENTION VARIANTS

  • The VCD extension uses video-only, audio-only, and both-modality contrastive branches, while MAD uses full, single-modality, and question-only branches.AVCD implements attentive masking by zeroing selected token representations while retaining sequence positions.Source: §C.2 · p. 17
  • Gen-Interv. redirects pathway cuts and steering to the generation position, while Removal-Interv. replaces internal pathway cuts with modality removal and applies the contrast to original question states.Removal-Interv. retains the required modality in its positive branch and the interfering modality in its negative branch; both preserve question text.Source: §C.2 · p. 17

D REPRESENTATION ANALYSES

The representation analyses compare positive and negative question states using cosine similarity and PCA, while the captioning evaluation specifies swapped-input data, preprocessing, and target- and distractor-reference metrics.

D REPRESENTATION ANALYSES

  • Cosine similarity averages matching-token similarities, while joint two-dimensional PCA visualizes positive and negative question representations and their paired separation.Representations are compared before steering on the same CMM examples across tested depths; Figures 9 and 10 show Qwen2.5-Omni-7B and VideoLLaMA2-AV-7B across layers 21–25, respectively.Source: D REPRESENTATION ANALYSES · p. 17, Figure 9 · p. 18, Figure 10 · p. 18

E.1 DATA, GENERATION, AND PREPROCESSING

  • Captioning evaluates 400 audio-swapped examples per target modality, using modality-specific references and prompts to describe only the requested audio or video.The swapped clips retain video while replacing audio with a plausible mismatch; generated captions are truncated at the first Human: or User:, reduced to the first complete sentence, and stripped of surrounding whitespace.Source: §E.1 · p. 18

E.2 TARGET AND DISTRACTOR CIDER

  • T-CIDEr measures target-reference agreement, whereas D-CIDEr measures distractor-reference overlap; higher T-CIDEr and lower D-CIDEr indicate the desired pattern.Both use identical predictions and sample IDs with different reference corpora; low D-CIDEr alone is insufficient because empty or generic captions can avoid distractor content.Source: §E.2 · pp. 18–19

E.3 LLM-BASED GROUNDING EVALUATION

The paper evaluates modality grounding with an LLM score that combines target quality and distractor leakage, then checks pairwise preferences against human judgments. GPT–human agreement is 89% for audio-target and 87% for video-target captioning.

E.3 LLM-BASED GROUNDING EVALUATION

  • The LLM judge scores target quality and distractor leakage from the requested modality, references, and generated caption, then averages a floored score from 1 to 5.It receives references and the caption rather than raw audio or video; shared reference content is not leakage, while unrelated hallucinations reduce target quality.Source: §E.3 · p. 19

E.4 HUMAN VALIDATION OF LLM SCORES

  • Human validation compares randomized caption pairs from the same swapped input and target modality, with method identities and GPT scores hidden from annotators.GPT preferences are derived from per-caption scores; agreement tests whether score differences reflect human preferences without requiring matching absolute scores or a separate GPT pairwise prompt.Source: §E.4 · p. 20
  • GPT–human pairwise agreement is 89% for audio-target and 87% for video-target captioning, versus human–human agreement of 93% and 90%, respectively.Agreement requires matching preferences, including ties; a tie from only one judge counts as disagreement, and ties remain in the denominator.Source: §E.4 · p. 20
Loading 2609.37568v1…