PaperlayerView exact revision on arXiv

Source-linked AI summary

MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learning

Suifeng Zhao, Zida Liu, Xinyu Lei, Lei Sun, Jun Gao, Sujian Li

arXiv:2608.21808v1cs.CL

TL;DR

Multimodal RAG needs visual citations that are precise and aligned with generated answers, but current RAG and SFT-based approaches often fall short. MCite-RL combines iterative agentic evidence refinement with process- and outcome-level citation rewards, and experiments show joint gains in citation precision and answer accuracy. The framework improves visual grounding across three benchmarks while introducing computational and citation-structure scope limitations.

  • Problem

    Existing multimodal RAG approaches often produce imprecise visual citations or citations decoupled from generated answers, limiting reliable evidence grounding.

  • Method

    MCite-RL combines an agentic visual-evidence refinement workflow with citation-enhanced reinforcement learning that provides process- and outcome-level supervision.

  • Results

    Across Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc, MCite-RL improves citation precision by 26.16% average IoU over vanilla RAG and answer accuracy by 5.89% on average.

  • Takeaways & Limitations

    Explicit citation supervision supports joint optimization of visual citation precision and answer quality in multimodal RAG.

  • Takeaways & Limitations

    The method adds computational overhead and focuses on bounding-box visual citations for single images without explicitly modeling more expressive citation structures.

Abstract

from arXiv · show

Multimodal Retrieval-Augmented Generation (RAG) with visual citation is crucial for ensuring the traceability and verifiability of MLLMs. However, current RAG and SFT-based methods struggle to achieve robust cross-modal reasoning, causing imprecise visual citations or decoupling between the citation and the generated answers. To address these limitations, we propose MCite-RL, a citation-enhanced agentic reinforcement learning framework designed for reliable multimodal RAG. MCite-RL introduces an Agentic Refinement module for visual citation that employs iterative retrieval, reasoning, and recursive cropping to progressively narrow the search space, transforming citation into a dynamic, evidence-driven reasoning process rather than a static step. Furthermore, we incorporate a Citation-enhanced Reward mechanism that integrates both process-level and outcome-level feedback within a reinforcement learning paradigm to jointly optimize answer accuracy and source traceability. Extensive experiments on benchmarks such as Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc demonstrate that MCite-RL effectively achieves the joint optimization of citation precision and answer quality.

1 Introduction

Multimodal RAG with visual citation aims to make generated answers traceable and verifiable, but existing approaches often produce imprecise or answer-decoupled citations. MCite-RL addresses these challenges through iterative evidence refinement and citation-enhanced reinforcement learning, improving both citation precision and answer accuracy.

  • Visual citations improve the verifiability and traceability of multimodal RAG outputs by linking responses to underlying visual evidence.
  • Existing visual-citation benchmarks typically use vanilla RAG or SFT-based baselines that provide only basic citation capabilities.VISA introduced fine-grained visual source attribution, while FinRAGBench-V added automated evaluation for complex documents.
  • Multimodal RAG systems struggle to localize visual evidence at the right granularity, producing citations that are spatially misaligned, too coarse, or too detailed.These imprecise citations fail to provide verifiable visual support, and citations can also become misaligned with generated answers.
  • MCite-RL uses an agentic refinement workflow that progressively narrows the visual search space through multi-step evidence refinement.The workflow lets answers and corresponding citations emerge from a shared, evidence-driven reasoning process rather than a static single-step generation paradigm.
  • MCite-RL's citation-enhanced reward function jointly optimizes reasoning accuracy and citation precision by incorporating citation feedback at multiple levels.Experiments on Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc report a 26.16% average IoU gain over vanilla RAG and a 5.89% average increase in answer accuracy.

2 Related Work

Research on multimodal RAG has progressed from coarse-grained multimodal retrieval toward fine-grained visual citation, while agentic reasoning and reinforcement learning provide iterative alternatives to static grounding pipelines.

  • Early multimodal RAG systems retrieved coarse-grained context from documents or images to augment multimodal generation.
  • Visual citation grounds generated content in fine-grained image regions to support traceability and verification.VISA aligns generated text spans with detected visual regions, while FinRAGBench-V evaluates citation performance in dense financial charts and tables.
  • Most existing visual-citation approaches rely on static, passive grounding in which visual regions are pre-selected before generation and loosely aligned afterward.
  • Agentic RAG reframes retrieval as an iterative reasoning-action loop, with Search-R1 refining text queries and VRAG-RL applying reinforcement learning in multimodal settings.

3 MCite-RL

MCite-RL formulates multimodal RAG with citation as constrained answer-and-bounding-box generation, using iterative retrieval, cropping, and citation-enhanced reinforcement learning. Its two-stage training combines SFT cold-start with RL to jointly optimize answer accuracy and citation precision.

  • 3.1 Task Formulation: MCite-RL generates a textual answer and visual citation bounding boxes for a query and initial image.The citation coordinates identify visual evidence supporting the answer.
  • 3.2 Agentic Workflow for Visual Citation: The agent alternates image retrieval and visual cropping to progressively narrow the search space toward precise evidence coordinates.Its action space includes retrieval, cropping, and final answer generation with visual citations.
  • 3.2 Agentic Workflow for Visual Citation: The final answer and citation are jointly generated from the converged visual region, coupling both outputs to shared visual evidence.The citation is spatially localized within the final region, reducing misalignment risk.
  • 3.3 Citation-enhanced Reward: Citation-enhanced rewards combine process-level grounding feedback with outcome-level citation precision alongside retrieval efficiency, answer accuracy, and format validity.Process-level reward uses IoGT on the terminal crop to preserve ground-truth evidence during refinement, while outcome-level reward uses IoU for final localization.
  • 3.4 Training Method: Training first uses SFT cold-start to establish the agentic format, then citation-enhanced RL to optimize accuracy and citation precision.The SFT stage synthesizes explicit reasoning trajectories because existing multimodal datasets generally lack them.
  • 3.4 Training Method: MCite-RL achieves a synergistic improvement in precise visual citation and answer accuracy by directly optimizing its citation-enhanced rewards.

4 Experimental Settings

The experiments evaluate MCite-RL across three multimodal RAG benchmarks against proprietary models, reproduced baselines, and ablation variants, measuring answer accuracy and citation precision. The framework uses Qwen2.5-VL backbones, ColQwen retrieval, and two-stage SFT followed by citation-enhanced RL.

  • Datasets: MCite-RL is evaluated on Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc to measure performance across diverse multimodal domains.The datasets cover open-domain Wikipedia retrieval, multimodal financial charts and tables, and long-context documents.
  • Baselines and variants: The comparison includes proprietary zero-shot models, reproduced RAG and agentic baselines, and training ablations.The reproduced baselines include Vanilla RAG, ReAct, Search-R1, and VRAG-RL.
  • Baselines and variants: The ablations remove process-level rewards, RL training, or all training to isolate the contributions of citation supervision and optimization.The w/o Proc variant uses only final answer correctness, while w/o RL is SFT-only and w/o Training uses the base model directly.
  • Evaluation metrics: Performance is evaluated along two dimensions: answer accuracy and citation precision.Citation precision is measured using average Intersection over Union between predicted bounding boxes and ground-truth annotations.
  • Implementation and training: The implementation uses Qwen2.5-VL-3B-Instruct and 7B-Instruct backbones with ColQwen as the retriever.Training uses Wiki-VISA subsets, while FinRAGBench-V and MMLongBench-Doc are reserved for out-of-domain evaluation.
  • Implementation and training: Training first applies SFT to 2,417 curated samples, then uses GRPO on 1,000 samples to optimize citation-enhanced rewards.The two stages establish citation capabilities and align the agent policy with the reward objectives.

5 Experimental Results and Analysis

MCite-RL improves visual citation precision across benchmarks and generally improves answer accuracy, while ablations show that both training stages and both levels of citation reward contribute to performance. Training dynamics further indicate lower uncertainty and stronger answer–citation coupling with explicit citation rewards.

  • Main results: Vanilla RAG and inference-time agentic reasoning produce low citation accuracy, showing that retrieval and cropping without training are insufficient.GPT-4o reaches 3.40% citation accuracy on Wiki-VISA, while Qwen2.5-VL-7B-Instruct reaches 0.04%.
  • Main results: MCite-RL yields double-digit citation improvements over previous strong baselines on the Qwen2.5-VL-7B-Instruct backbone.The comparison covers all three benchmark datasets.
  • Main results: On Qwen2.5-VL-7B-Instruct, MCite-RL achieves answer accuracies of 60.00% on Wiki-VISA, 21.81% on FinRAGBench-V, and 35.04% on MMLongBench-Doc.These results correspond to 4–6% improvements, whereas citation gains do not consistently transfer to answer accuracy on the 3B model.
  • Training dynamics: MCite-RL converges to lower entropy than VRAG-RL without citation rewards, indicating more confident generation under citation constraints.The entropy comparison is reported in Figure 3(d).
  • Training dynamics: Explicit citation rewards strengthen the weak natural correlation between answering and visual grounding, including on out-of-domain data.The baseline signal plateaus early, while MCite-RL turns it into an explicit reinforcement loop.
  • Ablation analysis: Removing RL causes significant degradation in both answer and citation quality, showing that reward optimization contributes beyond SFT instruction following.SFT establishes the agentic workflow and mitigates cold-start difficulty, but the RL stage drives the reported gains.
  • Ablation analysis: Removing visual citation rewards reduces citation precision from 36.05% to 20.56% and answer accuracy from 60.00% to 54.20%.Outcome-level rewards provide precise IoU feedback, while process-level rewards constrain intermediate cropping.
  • Ablation analysis: Without outcome-level visual citation rewards, expansive crops reduce citation quality to 13.15%, demonstrating the need for both reward levels.The process-level signal refines intermediate cropping, while the outcome-level signal anchors precision.

6 Conclusion

MCite-RL addresses the challenge of accurate visual citation by combining an agentic workflow with citation-enhanced rewards. Explicit process- and outcome-level supervision jointly optimizes citation and answer quality.

  • Conclusion: MCite-RL integrates an agentic workflow with citation-enhanced rewards to improve reliability in multimodal RAG.The framework explicitly supervises citation at both process and outcome levels.
  • Conclusion: Joint optimization of citation and answer quality is the framework’s central conclusion.The conclusion frames accurate citation as essential for reliable multimodal RAG systems.

Limitations

MCite-RL improves visual citation precision through citation-aware reinforcement learning but incurs computational overhead and remains limited to bounding-box citations on single images.

  • The training process introduces additional computational overhead.
  • The citation formulation focuses on bounding-box–based visual citation for single images.
  • More expressive citation structures are left for future work.

Ethical Consideration

The work aims to improve traceability, verifiability, transparency, and reduce hallucination by grounding outputs in identifiable visual evidence. However, dataset biases and the lack of guaranteed factual correctness require caution and human oversight in high-stakes deployment.

  • Grounding outputs in identifiable visual evidence aims to reduce hallucination and improve transparency.
  • The system may inherit biases from underlying datasets, including visual representations and annotation practices.
  • Visual citation improves interpretability but does not guarantee factual correctness of retrieved content.
  • High-stakes deployment requires caution and continued human oversight.

A.1 Answer-Citation Consistency Evaluation

The evaluation supplements spatial IoU with answer-citation consistency to assess whether cited regions semantically support generated answers. MCite-RL outperforms baselines across three datasets and qualitatively benefits from citation-enhanced rewards.

  • Answer-Citation Consistency Evaluation: Answer-citation consistency evaluates whether cited evidence is sufficient, non-redundant, and directly supports the generated answer.
  • Answer-Citation Consistency Evaluation: The evaluation protocol supplies the cropped citation region, question, and generated answer to an LLM judge.
  • Answer-Citation Consistency Evaluation: Table 3 reports answer-citation consistency percentages, with higher scores indicating better semantic alignment between answers and cited regions.
  • Answer-Citation Consistency Evaluation: MCite-RL consistently outperforms baselines across all three datasets under the consistency evaluation.
  • Qualitative Comparison: Figure 4 contrasts citation-reward settings, showing irrelevant cropping and an incorrect answer without rewards versus precise evidence localization and a correct answer with MCite-RL.
  • Agentic Supervision: Agentic SFT trajectories include retrieval, visual grounding, and final answer-with-citation actions, filtered for reliable supervision.

B.2 Trajectory Filtering

Trajectory filtering removes samples with incorrect retrieval, inaccurate answers, or invalid citation regions before SFT training. The section also documents reward formatting, two-stage training, implementation settings, datasets, and a qualitative reward comparison.

  • Trajectory Filtering: Filtering retains only trajectories satisfying retrieval correctness, answer correctness, and bounding-box validity.
  • Trajectory Filtering: 28,175 candidate trajectories were generated, and 2,417 were retained after filtering.
  • Trajectory Filtering: The retained trajectories correspond to an acceptance rate of approximately 8.6%.
  • Training and Reward Design: The composite reward balances retrieval quality, reasoning correctness, and visual alignment, while format violations reduce total reward to 0.
  • Training and Reward Design: Training uses supervised fine-tuning followed by reinforcement learning, with Qwen2.5-VL-7B-Instruct fine-tuned during SFT and GRPO used during RL.
  • Datasets and Evaluation: Wiki-VISA supplies training data, while FinRAGBench-V and MMLongBench-Doc provide out-of-distribution evaluation sets.

D.5 Details of Evaluation Metrics

The evaluation follows benchmark-specific protocols, using exact-match and ROUGE-L measures for short factual answers and a GPT-4o judge for mixed short and long-form responses.

  • Benchmark-specific metrics: Wiki-VISA and MMLongBench-Doc use Relaxed Exact Match and ROUGE-L for predominantly short, fact-based answers.These datasets have limited lexical variation.
  • Benchmark-specific metrics: FinRAGBench-V uses a GPT-4o-based judge for semantic evaluation of short factual and long-form reasoning responses.Its answer set mixes short factual and long-form reasoning responses.
  • Data and citation criteria: Synthesized trajectories are retained only when they satisfy all filtering conditions, while annotation balances completeness, compactness, and relevance.These criteria concern trajectory quality and visual-citation bounding boxes.
  • Reward evaluation: Reward optimization combines process-level grounding feedback with outcome-level citation precision.The reward is tailored to the agentic interaction trajectory.
Loading 2608.21808v1…