PaperlayerView exact revision on arXiv

Source-linked AI summary

Multimodal Fact-Level Attribution for Verifiable Reasoning

David Wan, Han Wang, Ziyang Wang, Elias Stengel-Eskin, Hyunji Lee, Mohit Bansal

arXiv:2602.11509v2cs.CLcs.AIcs.CV

TL;DR

Reliable MLLM reasoning requires verifiable, source-grounded factual claims, yet existing grounding work does not assess realistic fact-level attribution across heterogeneous multimodal inputs. MuRGAt evaluates that capability and finds that strong models still hallucinate citations despite high question-answering accuracy.

  • Problem

    Prior multimodal grounding work includes temporal video grounding and retrieval-augmented generation, while realistic fact-level attribution in heterogeneous-input reasoning settings remains to be evaluated.

    Source: §1 · p. 1

  • Method

    MuRGAt benchmarks fact-level citations for answers with explicit reasoning across modalities and evaluates verifiable claims, atomic facts, and citation quality using MURGAT-SCORE.

    Source: §1 · p. 1, §3.2 · p. 3

  • Results

    Models often achieve high question-answering accuracy yet hallucinate citations; programmatic decoupling improves attribution by avg. +9.6 MURGAT-SCORE, while explicit grounding can degrade complex-task reasoning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The experiments show that the ability to reason does not imply the ability to ground multimodal claims.

    Source: §6 · p. 9

    The reported attribution improvement from programmatic grounding comes with an average 7.4-point drop in answer accuracy.

    Source: §5.4 · p. 9

Abstract

from arXiv · show

Multimodal large language models (MLLMs) are increasingly used for real-world tasks involving multi-step reasoning and long-form generation, where reliability requires grounding model outputs in heterogeneous input sources and verifying individual factual claims. However, existing multimodal grounding benchmarks and evaluation methods focus on simplified, observation-based scenarios or limited modalities and fail to assess attribution in complex multimodal reasoning. We introduce MuRGAt (Multimodal Reasoning with Grounded Attribution), a benchmark for evaluating fact-level multimodal attribution in settings that require reasoning beyond direct observation. Given inputs spanning video, audio, and other modalities, MuRGAt requires models to generate answers with explicit reasoning and precise citations, where each citation specifies both modality and temporal segments. To enable reliable assessment, we introduce an automatic evaluation framework that strongly correlates with human judgments. Benchmarking with human and automated scores reveals that even strong MLLMs frequently hallucinate citations despite correct reasoning. Moreover, we observe a key trade-off: increasing reasoning depth or enforcing structured grounding often degrades accuracy, highlighting a significant gap between internal reasoning and verifiable attribution.

1. Introduction

MURGAT evaluates whether MLLMs can reason over heterogeneous multimodal inputs while grounding verifiable claims in precise evidence; experiments reveal a gap between answering correctly and attributing evidence reliably.

1. Introduction

  • Existing grounding studies often emphasize direct observation or retrieval and narrow modalities, leaving reasoning across heterogeneous sources under-evaluated (Hendricks et al., 2017; Lei et al., 2021; Dong et al., 2025; Yu et al., 2025; Chen et al., 2022; Wang et al., 2025a; Lei et al., 2019).Source: §1 · p. 1
  • MURGAT asks models for explicit reasoning and modality- and time-specific citations, then evaluates verifiable claims, atomic facts, and attribution quality.Attribution is assessed for recall and precision while accounting for temporal alignment and modality.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • MURGAT-SCORE reaches Pearson correlation 0.84 with human judgments averaged across evaluation steps, versus r = 0.59 for the next-best LLM-as-judge baseline.Human annotations on WorldSense and Video-MMMU provide reference labels for the evaluation pipeline.Source: §1 · p. 2
  • Strong MLLMs often answer questions correctly yet fail to attribute evidence, frequently producing hallucinated grounding.The benchmark evaluation finds insufficient or inaccurate citations despite correct answers.Source: §1 · p. 2
  • Programmatic separation of reasoning and citation raises attribution by avg. +9.6 MURGAT-SCORE, but explicit grounding can hurt reasoning on complex tasks.Citation generation acts as a reasoning tax on simple recognition but can scaffold complex reasoning; more thinking effort helps larger models' grounding while smaller models' MURGAT-SCORE declines.Source: §1 · p. 2

2. Related Work

Prior attribution work spans text and multimodal settings, but MuRGAt targets fine-grained temporal and per-modality citations while distinguishing verifiable claims from reasoning steps.

Attribution and Grounding Benchmark

  • MuRGAt requires models to self-select fine-grained temporal and per-modality evidence, unlike video-grounding tasks that assume the target evidence is specified in the prompt.It also separates verifiable claims from reasoning steps, enabling evaluation of multi-step responses rather than treating every sub-claim as observable.Source: §2 · p. 3
  • MCiteBench (Hu et al., 2025a) and MAVIS use document-level image evidence, while MIRAGE (Martin et al., 2025) also finds citation hallucinations despite correct answers.MIRAGE uses atomic decomposition and VLM verification for multimodal RAG; these benchmarks leave temporal and audio modalities unaddressed or lack MuRGAt’s finer attribution requirements.Source: §2 · p. 3

Attribution and Grounding Methods

  • Text attribution methods use direct generation, post-retrieval evidence (Nakano et al., 2021; Menick et al., 2022), or post-generation verification and revision (Gao et al., 2023a; Chen et al., 2024; Hsu et al., 2024).Direct generation prompts models to cite from parametric knowledge; post-retrieval approaches add explicit evidence retrieval and citation-aware reasoning.Source: §2 · p. 3
  • Multimodal programmatic, agent-based, and modular reasoning frameworks improve video retrieval, temporal grounding, or interpretability, but typically prioritize answer accuracy or localization over fine-grained claim attribution.Examples include executable-step frameworks (Wang et al., 2025c; Mahmood et al., 2025; Wang et al., 2025f; Li et al., 2025) and visual programs (Sur´ıs et al., 2023; Min et al., 2024).Source: §2 · p. 3

3. Task and Evaluation

MuRGAT evaluates whether multimodal model responses ground each verifiable claim in precise, supporting source segments. Its protocol identifies citation-covered claims, decomposes them into atomic facts, and scores citation coverage and attribution quality.

3.1. MURGAT

  • For multimodal inputs and a question, MuRGAT requires sentence-level reasoning responses whose verifiable claims cite timestamped segments from a specific modality, with each claim strictly entailed by its citations.A citation can specify, for example, an audio segment from 0:42–0:46.Source: §3.1 · p. 3

3.2. Evaluation Protocol

  • The evaluation protocol identifies verifiable sentences, decomposes them into atomic facts, and evaluates citation quality; MURGAT-SCORE measures fact-level grounding without penalizing unobservable reasoning statements.Source: §3.2 · p. 3

Subtask 1: Verifiable Claim Identification.

  • The pipeline retains observable, citation-covered sentences, decontextualizes them into minimal facts while propagating sentence citations, then checks whether cited segments entail each fact and are necessary evidence.Reasoning statements that cannot be directly grounded are excluded from attribution evaluation.Source: Subtask 1: Verifiable Claim… · pp. 3–4

3.3. Evaluation Metrics

  • MURGAT evaluates grounding along two axes—Coverage and Attribution—and combines them into the holistic MURGAT-SCORE.Source: §3.3 · p. 4

3.3.1. CITATION COVERAGE

  • Citation coverage is the proportion of verifiable sentences accompanied by at least one citation.Source: §3.3.1 · p. 4

3.3.2. ATTRIBUTION QUALITY

  • Attribution quality evaluates whether citations support atomic facts using citation Precision, Recall, and their harmonic-mean F1.Precision reflects citation relevance; Recall measures whether the provided evidence fully entails the facts.Source: §3.3.2 · p. 4

3.3.3. MURGAT-SCORE

  • MURGAT-SCORE scales attribution quality by coverage, penalizing models that ground only a small subset of facts while leaving most ungrounded.Source: §3.3.3 · p. 4

4. Automatic Evaluation

The section develops an automated evaluation pipeline that simulates human annotation stages, then tests its component methods and final metric against human judgments.

4.1. Human Annotation

  • Human ratings show that models fall short on coverage and Attribution F1, with no model consistently leading on both datasets and Video-MMMU scores generally lower despite higher QA accuracy.The evaluation samples 10 examples each from Video-MMMU and WorldSense, yielding 80 responses from four MLLMs; qualitative analysis links minimalist, shot-by-shot descriptions from Gemini-2.5-Flash with higher faithfulness than fluent narrative synthesis.Source: §4.1 · p. 5, Table 1 · p. 5

4.2. Subtask 1: Verifiable Claim Identification

  • Gemini-3-Pro with JSON prompting achieves the highest verifiable-claim identification score at 84.2 BAcc, followed by its CoT prompt at 81.4 BAcc.The subtask uses a binary verifiability decision evaluated with Balanced Accuracy.Source: Table 2 · p. 5, §4.2 · p. 5

4.4. Subtask 3: Attribution Quality

  • Gemini-3-Pro with JSON prompting achieves the highest attribution-quality F1 at 73.1, while Gemini-2.5-Flash with a Simple prompt scores 72.9.Because the difference is only 0.2 points, Gemini-2.5-Flash is selected for entailment in the pipeline to maximize efficiency.Source: Table 2 · p. 5, §4.4 · p. 6

4.5. End-to-End Evaluation

  • MURGAT-SCORE outperforms all prompting-based baselines across dimensions, reaching r = 0.97 on coverage; sentence-level judging reaches r = 0.76 vs. 0.58 for response-level judging.The authors attribute the gains to fine-grained atomic fact decomposition; attribution precision and recall also show strong gains.Source: Table 4 · p. 6, §4.5 · p. 7

4.3. Subtask 2: Atomic Fact Decomposition

  • Sentence-level decomposition with Gemini-3-Pro and explicit decontextualization achieves 81.8 F1, while citation accuracy remains challenging at 86.4%.Sentence-level methods outperform response-level methods; omitting decontextualization or handling it implicitly also reduces performance.Source: §4.3 · p. 6, Table 3 · p. 6

5. Generation Experiments

Generation experiments show that multimodal models can answer correctly while failing to ground claims, and that interventions affecting reasoning or citation generation can improve one aspect of performance while hurting another. The results also show that these effects depend on the task and model.

5.1. Experimental Setup

  • The evaluation sampled 100 Video-MMMU and WorldSense examples and measured answer accuracy and MURGAT-S across audio-visual and vision-only models.It compared direct generation, citation generation, and post-hoc attribution variants.Source: §5.1 · p. 7

5.2. Main Results

  • Citations lowered Gemini-3-Pro accuracy on WorldSense (71.4% to 70.0%) but raised it on Video-MMMU (85.3% to 86.0%; Qwen3-VL-Thinking, 51.0% to 60.0%).Qwen3-Omni-Thinking gained 9.0% accuracy on Video-MMMU with citations but struggled to generate valid timestamps, yielding MURGAT-S 4.8; post-hoc attribution improved WorldSense MURGAT-S (Gemini-3-Pro, 51.7 → 65.2) but reduced Video-MMMU attribution (Gemini-2.5-Flash, 41.5 → 38.0) by force-aligning abstract reasoning to segments.Source: §5.2 · pp. 7–8
  • On Video-MMMU, Gemini-3-Pro and Gemini-3-Flash matched 86.0 accuracy with citations, but Flash had higher MURGAT-S (56.9 vs 41.8).The authors argue that correct answers can rely on parametric knowledge while supporting citations are hallucinated, making MURGAT-S an independent measure.Source: §5.2 · p. 7
  • MURGAT-S peaked at 69.2 on WorldSense and 56.9 on Video-MMMU (Gemini-3-Flash), yet even top models failed to ground roughly 30–35% of claims.Table 5 reports Coverage, Attribution, MURGAT-S, and answer accuracy across model variants.Source: §5.2 · p. 7, Table 5 · p. 8
  • On WorldSense, Qwen3-Omni-Instruct outscored Qwen3-VL-Instruct (57.0% vs 50.0%), but on Video-MMMU the ordering reversed (53.0% vs 45.0%).Vision-only models hallucinated audio citations for up to 31.6% of references, so their attribution scores could reflect visual proxies rather than auditory understanding.Source: §5.2 · p. 8

5.3. Impact of Reasoning Effort

  • Effort effects diverged: Gemini-3-Flash WorldSense MURGAT-S fell from 69.7 to 64.4 (Minimal to High), while Gemini-3-Pro gained +6.1 MURGAT-S and +7.4 accuracy (Low to High).Flash reached its Video-MMMU accuracy peak of 91.5% at Medium effort.Source: §5.3 · p. 8, Figure 2 · p. 9

5.4. Programmatic Multimodal Grounding

  • On WorldSense, program-aided methods averaged +9.6 MURGAT-S points over BASE + CITATION, with Logic Imperative highest at 76.4 versus Declarative at 74.3.The framework varies logic- versus narrative-centric reasoning and model-predicted timestamps versus retrieval-tool queries, with runtime checks that atomic operations follow input evidence; Imperative methods outperform Declarative ones.Source: §5.4 · p. 9, Figure 3 · p. 9
  • Programmatic grounding improved attribution but reduced answer accuracy by an average of 7.4 points.The authors note that explicit structuring can aid verification while constraining reasoning flexibility.Source: §5.4 · p. 9

6. Conclusion · A. Human Evaluation Details

MURGAT evaluates fact-level attribution in complex multimodal reasoning, while MURGAT-SCORE provides automatic evaluation that correlates highly with human judgments. The paper identifies a gap between reasoning and grounding, and describes a multi-stage human annotation protocol for validating its metrics.

6. Conclusion

  • MURGAT targets complex answers synthesized from video, audio, and figures with precise evidentiary support, beyond retrieval or simple observation.It is designed to evaluate fact-level attribution in multimodal large language models.Source: §6 · p. 9
  • MURGAT-SCORE is a decomposed, fine-grained automatic evaluation pipeline with high correlation to human judgments.Source: §6 · p. 9
  • The experiments reveal that MLLMs’ capability to reason does not imply their capability to ground.Source: §6 · p. 9
  • Post-hoc methods can hallucinate mappings in complex reasoning tasks.Source: §6 · p. 9
  • The paper identifies a trade-off between programmatic rigor and narrative accuracy.Source: §6 · p. 9

A. Human Evaluation Details

  • Human evaluation uses three stages: atomic decomposition, verifiable-claim identification, and attribution quality.Annotators identify verifiable claims at the atomic-fact level for comprehensiveness and future fine-grained research, whereas the evaluation framework uses sentence-level identification.Source: A. Human Evaluation Details · p. 14
  • Sentence-level verifiable-claim identification yields similar performance at much lower cost than atomic-fact identification.The comparison is detailed in Appendix B.1.Source: A. Human Evaluation Details · p. 14

A.1. Data and Models … A.5. Handling Multi-Source Sentences

The appendix details how MuRGAt’s evaluation data and annotations were assembled, from multimodal model generations through atomic-fact verification and citation assessment. Its protocol aims for granular, high-recall claim identification while judging citation support and necessity separately for each fact.

A.1. Data and Models

  • The evaluation sampled 10 examples each from Video-MMMU and WorldSense and produced 80 generations and 600 sentences across four MLLMs.The evaluated models were Gemini-2.5-Flash, Gemini-3-Pro, Qwen3-Omni-Instruct, and Qwen3-Omni-Thinking; they were prompted to provide reasoning and citations.Source: §A.1 · p. 14

A.2. Atomic Decomposition

  • Annotators decomposed complex sentences into independent atomic facts, resolving pronouns from prior context and propagating only citations relevant to each fact.They also decomposed reasoning steps, mathematical operations, and compound visual attributes; Gemini-3-Pro drafted candidate facts for manual refinement, averaging 25.6 facts per response.Source: §A.2 · p. 14

A.3. Verifiable Claim Identification Annotation

  • Verifiable claims were defined as atomic facts about specific visual or audio events, dates, locations, or object absence; meta-data, general knowledge, and subjective chitchat were excluded.The decision was made at the atomic-fact level for granular coverage.Source: §A.3 · pp. 14–15
  • The union strategy retained 15.2% of the dataset (N=216) that strict annotator consensus would have discarded, supporting a high-recall gold standard.Agreement on verifiable-claim identification was 73.7%; disagreements primarily reflected different sensitivity thresholds.Source: §A.3 · p. 15
  • Only 3 of 37 human-labeled non-verifiable sentences contained citations, an over-citation rate of 8%.Source: §A.3 · p. 15

A.4. Attribution Annotation

  • Annotators assessed attribution by checking whether cited segments jointly supported each fact, then marking only timestamps necessary to prove supported facts.This separates recall (support) from precision (necessity) and penalizes citation dumping; verification agreement reached 86.1%, compared with 82.2% reported by Liu et al. (2023a).Source: §A.4 · p. 15

A.5. Handling Multi-Source Sentences

  • For multi-source sentences, each citation is evaluated against the specific atomic fact it supports, rather than penalizing all facts collectively.In the example, the audio and visual segments jointly support F1, while audio alone supports F2; the visual citation is unnecessary for F2, yielding sentence-level recall 1.0 and precision 0.75.Source: §A.5 · pp. 15–16

A.6. Citation Placement Analysis … B.2. Atomic Fact Decomposition

The analyses validate citation propagation, motivate sentence-level evaluation, and show that explicit staged decomposition improves atomic-fact quality and citation accuracy.

A.6. Citation Placement Analysis

  • Among sentences with multiple citations, 79.8% end-dump citations and 20.2% place them inline; annotators assigned the full set to every fact in all 71 end-dumped cases.The splitting rule handles inline citations; end-dumping rates were symmetric across models (71–100%), leaving rankings unaffected, and recall is unaffected because it pools citation unions.Source: §A.6 · p. 16

A.7. Full Statistics

  • Table 7 provides sentence and atomic-fact counts alongside detailed attribution-recall and attribution-precision statistics.The supplied passage describes the table’s contents but gives no individual values.Source: §A.7 · p. 16

B.1. Verifiable Claim Identification

  • Gemini-3-Pro achieves the highest verifiable-claim identification performance at both granularities, while GPT-5.2 (CoT) trails by just 0.3 points at sentence level.BAcc is highly comparable across granularities, so sentence-level evaluation is used thereafter to avoid the higher computational cost of atomic-fact decomposition.Source: §B.1 · p. 16, Table 8 · p. 17
  • The verifiable-claim identification evaluation treats annotated verifiable claims as positive instances and all other claims as negative, and expands beyond Gemini models to Gemma-3-27b-it and GPT-5.2.Results cover sentence-level and atomic-fact granularity.Source: §B.1 · p. 16

B.2. Atomic Fact Decomposition

  • Response-level decomposition performs noticeably worse than sentence-level decomposition, while the full pipeline’s results include Gemma-3-27b-it and GPT-5.2.Table 9 reports the full results and compares the full pipeline with ablations and a single-pass method.Source: §B.2 · p. 17, Table 9 · p. 17
  • Gemini-3-Pro gains 2 F1 points with decontextualization, and separating stages yields a 3-point gain over single-pass generation.Citation accuracy is also consistently highest with the two-stage process.Source: §B.2 · p. 17

B.3. Robustness to Evaluator Choice … C.4. Implementation

The paper finds that its automated evaluator is robust to evaluator choice and reliable against human judgments, then presents a programmatic approach that separates reasoning from evidence localization. The framework explores alternative reasoning and grounding designs, verifies attribution during execution, and uses atomic tools to produce evidence-grounded answers.

B.3. Robustness to Evaluator Choice

  • The pipeline achieves perfect agreement with human rankings of Gemini outputs (τ = 1.00); GPT-5 reverses the top two rankings (τ = 0.33), with no systematic in-family bias.Gemini-3-Pro scores its own outputs at 40.0 versus 44.7 for Gemini-2.5-Flash; multi-model correlations are comparable to single-evaluator alternatives.Source: §B.3 · p. 18

B.4. Statistical Reliability of Correlations

  • Median Spearman correlation remains ≈0.84 across annotation sizes 30–80, while confidence intervals narrow with sample size; all evaluator correlations are significant at p < 0.001.The analysis uses 10,000 bootstrap resamples and reports tight 95% confidence intervals, supporting stable rank-level conclusions from the 80-response evaluation set.Source: §B.4 · p. 18

B.5. Manual Spot-Check of Pipeline Outputs

  • 47 of 50 manually reviewed entailment judgments (94%) match the pipeline; all three disagreements involve borderline temporal boundaries.The authors independently re-judged each sampled citation against the underlying video and audio evidence.Source: §B.5 · p. 19

C. Programmatic Multimodal Grounding

  • MuRGAt’s programmatic grounding framework uses a plan-then-execute paradigm in which executable modules link every claim to retrieved sources for verifiable citation assignment.The framework seeks an effective structure for faithful grounding by exploring reasoning paradigm and grounding mechanism as two design axes.Source: C. Programmatic Multimodal Grounding · p. 19

C.1. Axis 1: Reasoning Paradigm

  • Logic-centric programs query multimodal sources with imperative control flow but can obscure reasoning in opaque data structures, while narrative-centric programs produce human-readable semantic steps.The narrative-centric approach makes the reasoning trace a verifiable explanation of the final answer.Source: §C.1 · p. 19

C.2. Axis 2: Grounding Mechanism

  • Planner-defined grounding commits to timestamps during planning, whereas executor-discovered grounding delegates localization to a retrieval tool, trading planner control for tool-dependent robustness.Planner-defined citations rely on the model’s ability to localize without hallucinating; executor-discovered grounding shifts performance demands to retrieval recall.Source: §C.2 · p. 19

C.3. Refinement Mechanism

  • A runtime attribution check verifies after each execution step that function outputs are entailed by their input evidence before atomic results are aggregated.The approach is intended to improve grounding fidelity and showed improvement in early experiments.Source: §C.3 · p. 20

C.4. Implementation

  • The Python framework provides three multimodal operations: find retrieves timestamps, describe generates segment-grounded descriptions, and synthesize deduces answers from descriptions without raw-video access.This structure forces the final deduction to rely on retrieved evidence.Source: §C.4 · p. 20

C.5. Results · D. Additional Results. · D.1. Full Results

On Video-MMMU, programmatic variants can reduce answer accuracy despite curbing “correct for the wrong reasons” behavior. The full-results material covers benchmark metrics, reasoning and program-aided tasks, metric correlations with human judgments, and attribution precision by modality.

C.5. Results

  • Accuracy fell from 90.0% to 84.7% for Gemini-3-Flash on Video-MMMU; base models consistently outperform programmatic variants on this task.The passage says the plan-then-execute structure curbs “correct for the wrong reasons” behavior but may overly constrain flexibility on questions requiring holistic video understanding.Source: §C.5 · p. 20, Table 13 · p. 20

D.1. Full Results

  • Table 13 reports full WorldSense and Video-MMMU results across Coverage, Attribution precision, recall, and F1, MURGAT-SCORE, and answer accuracy.Best results within each method are shown in bold.Source: Table 13 · p. 20, §D.1 · p. 21
  • Table 14 provides program-aided WorldSense results for Gemini-3-Flash.The passage identifies Table 14 as the detailed performance metrics source for program-aided tasks.Source: Table 14 · p. 21, §D.1 · p. 21
  • Table 15 reports full reasoning results across different thinking levels.The passage identifies Table 15 as the detailed performance metrics source for reasoning tasks.Source: Table 15 · p. 21, §D.1 · p. 21
  • Table 16 reports Pearson, Spearman, and Kendall correlations with human judgments for Coverage, Attribution precision and recall, and MURGAT-SCORE.The caption says the evaluation protocol’s results are reported and best results are bolded.Source: Table 16 · p. 21, §D.1 · p. 21
  • Table 17 breaks down attribution precision by modality.The passage identifies this as the final component of the full-results material.Source: §D.1 · p. 21

D.2. Qualitative Analysis

The qualitative analyses show that attribution depends on balancing narrative synthesis with grounding precision. Post-hoc attribution helps on perceptual tasks but can hurt deductive tasks, while program-aided variants expose trade-offs among accuracy, attribution, and coverage.

Gemini-3-Pro vs Gemini-3-Flash

  • Gemini-3-Pro performs worse than Gemini-3-Flash in attribution; intricate cross-cut synthesis causes spatial hallucinations and temporal misalignment, while Gemini-2.5-Flash often benefits from shot-by-shot minimalism.Larger-model attempts to build global context across cuts or infer details absent from cited frames can undermine grounding. By prioritizing direct, verifiable observations, Gemini-2.5-Flash avoids the contextualization trap where reasoning overrules visual evidence.Source: §D.2 · p. 21

Post-hoc Attribution

  • Post-hoc attribution improves recall on perceptual WorldSense tasks but compromises precision on deductive VideoMMMU tasks by anchoring deductions to generic or irrelevant visual evidence.A second perceptual pass can capture scene elements overlooked during initial reasoning; on knowledge-intensive tasks, mapping internal deductions to visuals can produce performative citations and faithfulness noise.Source: §D.2 · pp. 21–22

Program-Aided Generation

  • Logic Imperative reaches 78.7 F1 attribution but only 60.0 accuracy versus 67.0 for BASE + POST-HOC; LOGIC DECLARATIVE balances 75.2 precision and 61.0 accuracy.Narrative Imperative achieves Recall/Coverage of 80.7/99.0 but can produce chatty output that includes irrelevant noise. Declarative prompts specify facts to check, reducing trace drift; overall, program-aided verification can find evidence yet fail to synthesize it accurately, making attribution and accuracy difficult to balance.Source: §D.2 · p. 22

E. Prompts

The appendix documents prompts for automatic evaluation and response generation, alongside comparisons of model grounding and attribution strategies.

E.1. Automatic Evaluation

  • Prompts for atomic fact decomposition and decontextualization are provided in Figures 9 and 10.Source: §E.1 · p. 22
  • Verifiability-evaluation prompts are provided for Simple, CoT, and JSON variants in Figures 12–14.Source: §E.1 · p. 22
  • Attribution-entailment prompts appear in Figures 15–17.Source: §E.1 · p. 22
  • Gemini 2.5 Flash favors minimalist, observation-first responses, while Gemini 3 Pro attempts higher-level narrative synthesis but has lower grounding precision.Source: Figure 6 · p. 23

E.2. Response Generation

  • The appendix provides prompts for baseline output generation and generation with citations in Figures 18 and 11.Source: §E.2 · p. 23
  • A post-hoc refinement prompt is provided in Figure 19.Source: §E.2 · p. 23
  • Post-hoc attribution improves Recall on WorldSense by grounding missed scene elements, but can produce “Citation Salad” on VideoMMMU by mapping technical steps to generic introductory frames.Source: Figure 7 · p. 24

Related papers

Loading 2602.11509v2…