PaperlayerView exact revision on arXiv

Source-linked AI summary

MMKG-R1: Constructing Multimodal Knowledge Graphs via Reinforcement Learning

Zongyu Wu, Yilong Wang, Xiaochen Wang, Minhua Lin, Yuan Zhong, Fenglong Ma, Xiang Zhang, Suhang Wang

arXiv:2610.19341v1cs.CV

TL;DR

Existing MMKG construction often lacks explicit graph-quality guidance, and answer-correctness rewards may provide only indirect feedback about graph quality. MMKG-R1 trains a multimodal constructor with rewards for source extractions and the assembled graph, and experiments report improved downstream question answering across datasets.

  • Problem

    Existing MMKG construction often lacks explicit quality guidance, while answer-level RL feedback may not reliably reflect whether the graph represents its source documents.

    Source: §1 · p. 2

  • Method

    MMKG-R1 trains a multimodal constructor using graph-level Deducibility Reward and source-level Source-Grounded Quality and Graph-Readiness rewards.

    Source: §6 · p. 8

  • Results

    Across multiple datasets, MMKG-R1 constructs useful MMKGs for downstream question answering and demonstrates strong generalization to unseen datasets.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The constructor’s extractions build useful MMKGs by combining feedback on evidence sufficiency, source fidelity, and usability.

    Source: §6 · p. 8

    Deducibility and Source-Grounded Quality rewards rely on external multimodal LLM judges, so their reliability depends on the quality of those assessments.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Multimodal knowledge graphs (MMKGs) organize information across different modalities to support multimodal Graph Retrieval-Augmented Generation (RAG), which aims to improve the performance of multimodal large language models (MLLMs). The effectiveness of these multimodal Graph RAG methods largely depends on the quality of the constructed MMKGs. However, most existing methods construct MMKGs without explicit guidance on graph quality, while reinforcement learning (RL)-based methods that rely on final-answer correctness may provide only indirect feedback on MMKG quality. To address these limitations, we propose MMKG-R1, a novel MMKG-quality-driven RL framework for training an MLLM as a multimodal knowledge constructor. To be specific, MMKG-R1 introduces multiple quality-aware rewards to directly evaluate both the quality of information extracted from individual knowledge sources and the quality of the assembled MMKG. These rewards are then used to guide the RL training. Experiments across multiple datasets show that MMKG-R1 constructs MMKGs that improve downstream question-answering accuracy.

1 Introduction

Multimodal Graph RAG depends on MMKG quality, yet construction often lacks explicit quality guidance, and answer-level feedback may not reflect whether a graph faithfully represents its sources. MMKG-R1 trains a multimodal constructor with rewards that assess both source-level extraction and the assembled graph.

1 Introduction

  • MMKG-based Graph RAG depends heavily on constructed-graph quality, but existing approaches often construct graphs without explicitly considering downstream RAG utility.Multimodal sources include text and images, and MMKGs connect within-modality information with cross-modal associations.Source: §1 · p. 1
  • A correct answer can coexist with an incorrect graph fact, while that noise may impair retrieval and answers to other questions.An incorrect profit value may be bypassed using a source image for one question, but may prevent retrieval of the relevant company for a question about revenue.Source: §1 · p. 2
  • MMKG-R1 uses complementary RL rewards to assess extracted information at the knowledge-source level and the assembled MMKG, rather than relying only on final-answer correctness.Deducibility Reward evaluates retrieved evidence; Source-Grounded Quality Reward assesses extraction accuracy and key-information coverage; Graph-Readiness Reward encourages usable entities and facts.Source: §1 · p. 2
  • Experiments on multiple datasets show that MMKG-R1 constructs high-quality MMKGs that effectively support multimodal Graph RAG.The paper presents this framework as a way to provide more targeted feedback on construction quality.Source: §1 · p. 2

2 Related Work

Graph RAG organizes external evidence into graph structures, and multimodal variants build MMKGs to support multimodal retrieval. Prior graph-construction RL methods provide a point of comparison for MMKG-R1’s direct quality feedback.

2.1 Graph RAG

  • Graph RAG organizes external evidence into graph structures, while multimodal methods construct MMKGs; most existing methods use pretrained models without task-specific training of construction.MMGraphRAG builds image-based entity-relation graphs and a text-based KG, then integrates them through cross-modal fusion.Source: §2.1 · p. 2

2.2 Reinforcement Learning for Graph Construction

  • AutoGraph-R1 trains a textual graph constructor with utility rewards, whereas EvoGraph-R1 optimizes query-conditioned multimodal hypergraphs using answer correctness, trajectory structure, and action cost.The passage notes that EvoGraph-R1’s rewards do not directly evaluate constructed-graph quality; MMKG-R1 instead assesses source extractions and graph readiness.Source: §2.2 · p. 3

3 Preliminaries

MMKG-R1 constructs a graph from source-specific extractions using fixed parsing and assembly modules. Construction is question-blind, while the training environment uses questions and answers to assess graph quality.

3 Preliminaries

  • The constructor processes each text passage, image, or table independently to extract typed entities and subject–relation–object triples from visible content.The resulting graph is assembled from these extractions and maintained source information.Source: §3 · p. 3
  • A fixed parser and assembler turn extractions and source metadata into a graph by assigning identifiers, attaching provenance, linking entities, and assembling the result.Each extracted triple becomes a fact node connected to its arguments and supporting source; entity linking connects compatible mentions across sources.Source: §3 · p. 3
  • Question-blind construction processes each source without access to the question, accepted answers, or other sources, allowing extractions to be reused across questions.The training environment uses questions and answers only to assess the resulting graph.Source: §3 · p. 3
  • The learning objective scores graph quality itself rather than final-answer correctness.The quality measure and optimization procedure are specified later in the paper.Source: §3 · p. 3

4 Method

MMKG-R1 trains a multimodal constructor with complementary rewards for answer-supporting evidence, source fidelity, and graph usability. Independent source extractions are assembled into candidate graphs and optimized using graph- and source-level feedback.

4 Method

  • Three complementary rewards assess whether graph evidence supports an answer, whether extractions preserve source content, and whether outputs are usable for graph construction.Deducibility is computed at the candidate-graph level; Source-Grounded Quality and Graph-Readiness are computed per source extraction.Source: §4 · pp. 3–4

4.1 Source-Level Construction and Graph Assembly

  • The constructor independently samples four responses per knowledge source, then combines outputs with matching sample indices into four candidate MMKGs per question.The candidates are retrieved and evidence-packed using a fixed pipeline; other components remain fixed during training.Source: §4.1 · p. 4

4.2 Graph-Level Deducibility Reward

  • A frozen MLLM judge scores each candidate graph’s retrieved evidence for answer support, distinguishing insufficient evidence from an incomplete or incorrect structured reasoning path.Scores are 0 for a whole-bundle gap, 0.5 for a structured-path gap alone, and 1 when neither gap exists.Source: §4.2 · pp. 4–5

4.3 Knowledge Source-Level Feedback

  • Source-level feedback complements question-specific deducibility with extraction fidelity and graph readiness; quality evaluation checks reliability and central coverage, while readiness checks format and usable content.Quality feedback is applied only to eligible groups whose candidate graphs all receive the same deducibility reward of 0.5 or 1; at most one eligible source per group is sampled.Source: §4.3 · p. 5

4.4 GRPO-Based Optimization

  • GRPO optimization combines graph-level deducibility advantages with weighted source-level advantages, applying activation and source-credit masks before maximizing a dual-clipped surrogate objective.The two source-level advantages are mean-centered across responses generated for the same source.Source: §4.4 · pp. 5–6

5 Experiments

Experiments evaluate MMKG-R1 on subsets of MMDocRAG and MMQA, with MMLongBench-Doc used to assess transfer; ablations examine its reward designs. MMKG-R1 improves answer accuracy over Base GraphRAG and F1-based RL in the reported main results and transfer evaluations.

5.1.1 Datasets

  • The experiments train constructors separately on MMDocRAG and MMQA and evaluate subsets that reduce gold-passage overlap and exclude questions answerable from text alone.MMDocRAG-trained constructors are also evaluated on a subset of MMLongBench-Doc for transferability.Source: §5.1.1 · p. 6

5.1.2 Evaluation Protocol

  • For each dataset, methods use the same prompt and evaluation criteria, with Qwen3.5-27B serving as an independent judge distinct from the model used to calculate training rewards (Team, 2026).The listed baselines are Direct Inference with Materials, Naive RAG, Base GraphRAG, and F1-based RL; evaluation assesses the same trained constructor with Dense, Subgraph, and Hybrid Retrieval.Source: §5.1.2 · p. 6, §5.1.3 · p. 6, §5.1.4 · p. 7

5.2 Main Results

  • On the MMQA subset with dense retrieval, MMKG-R1 reaches 38.13% accuracy, versus 33.75% for Base GraphRAG and 35.83% for F1-based RL.Across the three evaluation subsets, MMKG-R1 consistently outperforms both baselines; F1-based RL improvements over Base GraphRAG are not consistent across settings.Source: §5.2 · p. 7

5.3 Transferability of MMKG-R1

  • MMKG-R1 achieves higher answer accuracy across all three retrieval strategies on MMQA when evaluated with Qwen3-VL-32B-Instruct-AWQ as the answer model.Table 2 reports performance with this answer model.Source: Table 2 · p. 6, §5.3 · p. 7
  • On MMLongBench-Doc, MMKG-R1 outperforms Base GraphRAG and F1-based RL across all three retrieval strategies after training on MMDocRAG without further training.The authors say these results suggest the quality-aware rewards help the constructor learn capabilities that transfer to unseen datasets.Source: §5.3 · p. 7

5.4 Ablation Studies

  • Removing the Source-Grounded Quality Reward reduces accuracy, while binarizing Deducibility generally degrades it further; full MMKG-R1 outperforms both variants.The binary variant also removes SGQ and maps Deducibility scores of 0.5 and 1 to 1, retaining 0 unchanged.Source: §5.4 · p. 8, Figure 4 · p. 8
  • The Deducibility Reward yields a higher proportion of discriminative candidate groups than its binary variant across all reported windows.A group is discriminative when at least three valid candidates remain and their task rewards are not all identical; the nonbinary reward distinguishes incomplete from complete structured reasoning paths even when both support the answer.Source: §5.4 · p. 8
  • As training progresses, full MMKG-R1 retains more triples and has fewer output delivery failures than MMKG-R1 without SGQ.Triples are counted per source output after parsing and deduplication; failures include structural, normal-termination, or nonempty-output requirement failures.Source: §5.4 · p. 8

6 Conclusion

MMKG-R1 trains a multimodal constructor with complementary graph-level and knowledge-source-level rewards. Across multiple datasets, the trained constructor builds useful MMKGs for question answering and shows generalization to unseen datasets.

6 Conclusion

  • MMKG-R1 combines Deducibility, Source-Grounded Quality, and Graph-Readiness rewards to assess evidence sufficiency, source fidelity, and structural usability.The framework trains a multimodal constructor to extract knowledge from multimodal sources for MMKG construction.Source: §6 · p. 8
  • Across multiple datasets, the trained constructor effectively extracts knowledge to build useful MMKGs and demonstrates strong generalization to unseen datasets.Source: §6 · p. 8

Limitations

The Deducibility and Source-Grounded Quality rewards rely on external multimodal language models as judges.

Limitations

  • The reliability of the Deducibility and Source-Grounded Quality rewards depends on the quality of the external multimodal language model judges' assessments.Source: Limitations · p. 9

Ethical considerations

The work trains a multimodal language model to extract knowledge from multimodal sources for MMKG construction and bases its experiments on publicly available datasets. The authors do not anticipate significant ethical or societal risks.

Ethical considerations

A Dataset Details

The experiments use three datasets for multimodal evidence retrieval, cross-modal question answering, and long-document transfer evaluation.

  • MMDocRAG evaluates multimodal evidence retrieval and question answering over multimodal documents and is licensed under Apache License 2.0.Source: A Dataset Details · p. 11
  • MultiModalQA links Wikipedia tables with text passages and images for complex cross-modal, multi-hop question answering.Source: A Dataset Details · p. 11
  • For MultiModalQA, text and images remain in original form, tables are serialized as text, and no explicit license is specified.Source: A Dataset Details · p. 11
  • MMLongBench-Doc transfer evaluation uses answerable questions with valid supporting pages and figure or chart evidence, with the constructor trained on MMDocRAG.The dataset evaluates long-document understanding and is licensed under CC BY-NC 4.0.Source: A Dataset Details · p. 11

B More Implementation Details

Training rollouts use specified sampling settings, while verl supports the training pipeline and vLLM handles answer-model inference.

C AI Usage Disclosure

The authors disclose using generative AI tools for writing and coding, with all AI-assisted content and code reviewed and verified by the authors.

  • The authors used generative AI tools to assist with writing and coding, then reviewed and verified all AI-assisted content and code.Source: C AI Usage Disclosure · p. 11

Related papers

Loading 2610.19341v1…