PaperlayerView exact revision on arXiv

Source-linked AI summary

Catching Hallucinated Citations in Video-LLM Question Answering: A Self-Verification Pipeline and Verifier Ablation Study

Yogesh Kumar

arXiv:2608.15574v1cs.CVcs.AI

TL;DR

Timestamped video-LLM answers can sound grounded even when cited frames do not support their claims. GROUNDEDVQA drafts timestamped claims and independently verifies each cited frame, with a small NLI classifier catching 79% of adversarial fabrications while leaving truthful evasive answers unflagged.

  • Problem

    Video question answering systems can confidently narrate unsupported facts, while timestamp citations appear grounding evidence without checking whether cited frames support claims.

  • Method

    GROUNDEDVQA retrieves timestamped captions, drafts claim-level answers, and independently checks each cited frame before labeling claims GROUNDED or UNVERIFIED.

  • Results

    The NLI verifier catches 79% of adversarial fabricated claims, while truthful evasive answers remain unflagged.

  • Takeaways & Limitations

    A small entailment classifier provides a stable, useful verifier where direct model self-judgment fails and a general language-model judge varies with prompts.

  • Takeaways & Limitations

    Evaluation covers one animated video and 40 claims from 12 hand-constructed questions, so generalization to other footage and longer videos remains untested.

Abstract

from arXiv · show

Video question answering systems built on vision-language models often produce timestamped claims with high confidence even when unsupported by the cited frame. This deceptive hallucination arises because timestamps imply grounding without ensuring correctness, increasing user trust but not accuracy. We introduce a pipeline that closes this loop. A retrieval-augmented language model drafts answers with per-claim timestamp citations, and each cited frame is independently re-examined before being shown to the user. We compare against a plain baseline and ablate three verification designs, evaluated on both Apple Silicon (MLX) and Google Colab (HF Transformers, CUDA). Directly asking the vision model whether a frame supports a claim fails completely (0% catch rate on 40 claims) due to sycophancy. Blind re-captioning plus a general LLM judge improves results but is unstable, oscillating between 0% and 100% flagged depending on prompt phrasing. Replacing that judge with a small natural language inference model yields a stable, interpretable verifier that catches 79% of fabricated claims on adversarial false-premise questions while leaving true claims untouched. We release the full pipeline, evaluation harness, and implementations for both Apple Silicon and Colab. Code is available at https://github.com/yogesh-iitj/grounded-video-qa.

1 Introduction

Video-LLM question answering can produce confident claims whose timestamp citations are unsupported, making citations appear grounded without actually verifying the cited frame. GROUNDEDVQA addresses this gap with post hoc self-verification and controlled measurement of verifier design.

  • Problem: Video question answering systems may confidently narrate facts that are unsupported by the underlying footage.This failure occurs in direct systems and retrieval-augmented pipelines that sample and caption frames before composing answers.
  • Motivation: Timestamp citations can increase user trust despite lacking evidence that the cited frame supports the claim.The paper characterizes an unverified citation as decorative rather than grounding evidence.
  • Contribution: GROUNDEDVQA closes this gap with a post hoc self-verification loop and measures whether verification catches unsupported claims.The paper emphasizes controlled measurement of the verification step as a central contribution.
  • Contribution: The study compares verified and no-verification conditions while controlling claim drafts to avoid generation-sampling confounds.The pipeline is implemented on an 18 GB Apple Silicon laptop via MLX and a Google Colab T4 via HF Transformers.

2 Related Work

Related work spans agentic retrieval, efficient video-LLM processing, self-verification, and NLI-based consistency checking. This paper combines retrieve-before-generate video QA with post-generation cross-modal verification, while evaluating both local Apple Silicon and cloud CUDA implementations.

  • Agentic video understanding: VideoAgent retrieves additional keyframes iteratively before answering, whereas this pipeline adds post-generation verification to target citation faithfulness.The distinction is retrieval sufficiency versus checking whether cited evidence supports generated claims.
  • Efficient video-LLM processing: 65 percent reduction in computation with 97 to 99 percent of task performance is reported by query-conditioned temporal token pruning.This work reduces cost differently by sampling frames at fixed intervals.
  • Self-refinement and self-verification: Self-Refine and Chain-of-Verification reduce factual errors by having language models critique and revise their own text outputs.This paper instead verifies textual claims against visual evidence, motivating its cross-modal design questions.
  • NLI based factual consistency checking: NLI classifiers detect factual inconsistency more reliably than generative language-model judges in text summarization, a finding reproduced here cross-modally.The paper reports that a small NLI model outperforms a larger general instruction-tuned language model as a judge.
  • Efficient local and cloud inference: The implementations use 4-bit quantized Qwen models through Apple MLX locally or HF Transformers with bitsandbytes quantization on a CUDA GPU.Retrieval uses Sentence-BERT embeddings.

3 Method

GROUNDEDVQA uses offline video indexing and online retrieval-augmented question answering, then independently rechecks each cited claim against freshly extracted video evidence. It exposes each claim’s verification status with the checked frame so users can audit the result directly.

  • 3 Method: GROUNDEDVQA separates processing into offline index construction and online retrieval, answer drafting, and verification.Frames are sampled, captioned, embedded, and stored with timestamps during ingestion.
  • 3 Method: The system retrieves the top-k captions by cosine similarity and prompts a language model to return timestamped claims as a JSON array.Each claim is an independent factual statement paired with the timestamp of its retrieved segment.
  • 3 Method: Structured claim-and-timestamp output enables per-claim verification instead of loosely embedded prose citations.The method makes each citation explicit and checkable.
  • 3 Method: Each claim is independently checked using the actual video frame at its cited timestamp, optionally including neighboring sampled timestamps to tolerate small offsets.The evidence is re-extracted directly from the source video rather than taken from the cached index.
  • 3 Method: The verifier labels each claim GROUNDED or UNVERIFIED and displays the evidence frame used for the decision.This lets users or evaluators audit the verdict directly rather than trusting it blindly.

4 Implementation Details

The implementation provides reproducible Python configurations across Apple Silicon and CUDA, with deterministic evaluation settings and measured performance on an Apple M3 Pro. Ingestion dominates one-time cost, while verification is substantially cheaper than answer drafting.

  • System configuration: Python uses OpenCV for decoding and provides equivalent MLX and CUDA backends with 4-bit quantized model loading.MLX uses mlx-vlm and mlx-lm checkpoints; CUDA uses HF Transformers with bitsandbytes.
  • System configuration: The default configuration samples every 5 seconds, retrieves k = 5 items, and checks ±1 neighboring frame when the primary frame is insufficient.These settings are exposed as plain constants in one configuration module for easy adjustment.
  • Measurement setup: All measurements use an Apple M3 Pro with 18 GB unified memory and the MLX backend, reporting cold model-load time and steady-state throughput.The latency table measures performance on this hardware configuration.
  • Measured performance: 0.56 seconds per frame: captioning roughly 120 frames from a 10-minute video takes about a minute and a half, with results cached to disk.This makes ingestion the dominant one-time cost at the default sampling interval.
  • Measured performance: Verification costs about one quarter as much as drafting the whole answer per claim because it combines captioning with a small NLI forward pass.The comparison is relative to answer drafting and follows directly from the measured performance numbers.
  • Evaluation reproducibility: The evaluation harness fixes the question set, drafts each claim once, and uses greedy or deterministic settings for model loading, sampling, and NLI checks.This ensures baseline and verified conditions use identical text rather than independent generations.

5 Verifier Design Iterations

The verifier evolved from direct vision-model questioning, which missed every unsupported claim, through blind captioning with an unstable language-model judge, to a stable NLI-based design. The final design separates perception from judgment and is used for the paper’s subsequent results.

  • Direct vision-model verification: 0 of 40 claims were flagged as unsupported by directly asking the vision-language model whether each frame supported its claim.This included verifiably false claims; the model restated the claim instead of attending to the image.
  • Blind captioning with language-model judgment: 100 percent of 40 claims were flagged as unsupported when blind captions were judged by the drafting language model.The model rejected even near word-for-word caption–claim matches, demonstrating instability rather than reliable classification.
  • NLI-based verification: The final design uses blind captioning with cross-encoder/nli-deberta-v3-small as an NLI judge over entailment, contradiction, and neutral.The caption is the premise, the timestamp-stripped claim is the hypothesis, and entailment is the positive class.
  • NLI-based verification: The NLI verifier was the only design stable across qualitative spot checks and correctly accepted paraphrase matches while rejecting or remaining neutral on fabricated claims.It is the verifier used for the results in Section 7.
  • Practical takeaway: Choosing a model trained specifically for NLI appears more important than prompt wording when a pipeline needs an intermediate binary or graded judgment.The paper presents NLI as a cheap, effective substitute for prompting a general-purpose chat model to behave like a classifier.

6 Experimental Setup

The evaluation uses a 596-second Big Buck Bunny video, a 12-question set yielding 40 claims, and matched baseline-versus-verification conditions. It implements the pipeline with specified quantized models on both Apple Silicon and Colab GPU environments.

  • Video: The video is Big Buck Bunny, sampled every 5 seconds to produce roughly 120 frames.The animated short runs for 596 seconds and is licensed CC BY 3.0.
  • Hardware and implementation: The implementation runs locally on an Apple M3 Pro MacBook via MLX or on Colab’s free-tier NVIDIA T4 via HF Transformers and bitsandbytes 4-bit quantization.The two implementations share pipeline logic except for model loading.
  • Question set: The question set contains 12 fixed questions across five factual, five adversarial false-premise, and two positional questions, producing 40 total claims.The positional questions target caption-similarity retrieval’s known weakness at the video’s beginning or end.
  • Baseline comparison protocol: The baseline reports each question’s claims exactly as drafted, while verification processes the same claims through Algorithm 1 to avoid drafting-sampling confounds.Claims are drafted once for each question, and matched conditions isolate verification effects.

7 Results

The verifier catches 79% of fabricated adversarial claims while leaving factual claims unflagged, but retrieval limits positional-claim detection to 14%. Its NLI decisions are interpretable: adversarial catches are mostly neutral rather than contradiction, with no false contradictions on factual or positional claims.

  • Results: 0% catch rate on factual claims is the desired outcome because no true video-content claim was incorrectly flagged.This contrasts with V2a, which rejected every claim regardless of correctness.
  • Results: 79% catch rate on adversarial claims shows the verifier catches most timestamp-cited fabrications by checking the actual cited frame.The three adversarial claims not caught were manually judged truthful rather than hallucinated.
  • Results: 14% catch rate on positional claims reflects retrieval’s inability to encode chronological position, not a verification failure.Semantic caption similarity can retrieve a real frame from the wrong location, leaving the resulting claim truthfully grounded in that frame.
  • NLI label analysis: No factual or positional claim was labeled contradiction; the verifier instead entailed it or remained neutral in the single positional case.This argues against the NLI model simply pattern matching toward rejection.
  • NLI label analysis: Among 11 caught adversarial claims, 7 were labeled neutral and 4 contradiction, because unrelated blind captions more often fail to support than directly deny claims.Only entailment counts as grounded among the three NLI labels: entailment, contradiction, and neutral.

8 Discussion and Limitations

The verifier assesses whether a cited claim is visually supported, but cannot correct retrieval errors or judge answer relevance and exhaustiveness. Findings are indicative because the study uses small models and a narrow evaluation scope without confidence intervals.

  • Retrieval limitation: Verification cannot detect wrong-frame retrieval when the claim is true of that frame; retrieval-sensitive errors require improved retrieval, such as combining semantic similarity with position or recency signals.The verifier checks frame entailment, not whether the correct frame was retrieved.
  • Verification scope: A claim may be grounded in its cited frame yet fail to answer the question, because the verifier checks visual support rather than relevance or exhaustiveness.Several uncaught adversarial claims illustrate this limitation.
  • Model scale: All models were deliberately small for consumer and free-tier hardware; larger vision models may change the precise catch rate, but likely not the qualitative advantage of a purpose-built classifier over a prompted generative judge.The authors attribute the judge failure mode to model type rather than size within the tested regime.
  • Evaluation scope: Results cover one animated video and 40 claims from 12 hand-constructed questions, without confidence intervals, so they are indicative rather than definitive.Generalization to live action, dialogue-heavy, or longer videos remains untested.

9 Conclusion

The conclusion presents GROUNDEDVQA, a retrieval-augmented video question answering pipeline with a post hoc self-verification loop, evaluated against an unverified baseline using controlled same-claims comparisons. It identifies sycophancy as causing the direct frame-support check to fail completely and releases the full implementations, evaluation harness, question set, and raw outputs.

  • Conclusion: GROUNDEDVQA combines retrieval-augmented video question answering with a post hoc self-verification loop and controlled same-claims comparison against an unverified baseline.
  • Conclusion: The direct design of asking the frame-viewing model whether it supports the claim fails completely because of sycophancy.
  • Reproducibility: The release includes Apple Silicon MLX and Google Colab HF Transformers implementations, the evaluation harness, fixed question set, and raw evaluation outputs.The outputs include per-claim evidence frames used for manual audits in Section 5.
Loading 2608.15574v1…