Source-linked AI summary
GraMRAG: Orchestrating Multi-Agent Multi-Step Reasoning via Graph Memory with Reinforcement Learning
Zhongyu Wang
TL;DR
Existing multi-agent RAG systems have limited retrieval depth and memory structures that lose track of reasoning states. GraMRAG combines visual reasoning, dynamic graph memory, and topology-aware policy optimization; it reports consistent outperformance and state-of-the-art results on multimodal benchmarks. Its multi-agent collaboration also adds inference overhead relative to traditional RAG, potentially limiting deployment in resource-constrained environments.
Problem
Existing multi-agent RAG systems have limited retrieval depth, while linear or iterative-summary memories can lose track of reasoning states and prompt redundant retrieval.
Source: §1 · p. 1Method
GraMRAG combines vision-text bridged reasoning, a dynamic multimodal memory graph, and topology-aware policy optimization for multi-step multimodal reasoning.
Source: §1 · p. 2Results
GraMRAG consistently outperforms existing baselines and achieves state-of-the-art performance on diverse multimodal benchmarks.
Source: §6 · p. 9Takeaways & Limitations
The multimodal memory graph’s 3.3point performance improvement over iterative summarization supports structured memory for long-horizon multimodal reasoning.
Source: §5 · p. 8Multi-agent collaboration adds inference overhead relative to traditional RAG, which may hinder deployment in resource-constrained environments.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Although existing multi-agent Retrieval-Augmented Generation (RAG) systems have demonstrated promise on complex multimodal reasoning tasks, they remain fundamentally limited in reasoning depth and memory structure, suffering from inadequate retrieval and state blindness when answering knowledge-intensive questions. To address these limitations, we propose GraMRAG, a graph memory-guided multi-agent RAG framework that integrates a dynamic multimodal memory graph to enable stable, multi-step multimodal reasoning. We introduce a vision-text bridged reasoning paradigm that unifies multi-scale entity cropping with a ReAct-style visual toolchain, enhancing the long-horizon cross-modal reasoning capability. We further construct a multimodal memory graph that formalizes agent reasoning as a dynamic directed acyclic graph (DAG), explicitly modeling action-observation dependencies to mitigate state blindness and suppress redundant retrieval. Moreover, we propose Topology-Aware Policy Optimization (TAPO) that leverages graph topology for critical path identification and targeted node pruning, enabling fine-grained credit assignment across multi-step reasoning trajectories. Extensive experiments on challenging multimodal benchmarks demonstrate that our approach consistently outperforms existing baselines and achieves state-of-the-art performance on complex long-horizon reasoning tasks.
1 Introduction
Existing multi-agent RAG systems are limited by shallow retrieval, memory state blindness, and coarse credit assignment; GraMRAG combines visual reasoning, graph memory, and topology-aware optimization. Experiments on challenging multimodal benchmarks report state-of-the-art performance over existing baselines.
1 Introduction
- Existing multi-agent RAG systems are limited by insufficient retrieval rounds, memory state blindness, and coarse outcome-based rewards that impede fine-grained credit assignment.Shallow exploration limits long-horizon planning; linear or iterative-summary memories lose track of reasoning states, prompting repetitive queries and redundant retrieval.Source: §1 · pp. 1–2
- GraMRAG combines vision-text bridged reasoning, a multimodal memory graph, and TAPO to address long-horizon reasoning, state blindness, redundant retrieval, and credit assignment.The visual paradigm unifies multi-scale entity cropping with a ReAct-style toolchain; TAPO uses graph topology for critical-path identification and node pruning.Source: §1 · p. 2
- GraMRAG achieves state-of-the-art performance over existing baselines on challenging multimodal benchmarks.The introduction reports extensive experiments but provides no benchmark-level metrics in these passages.Source: §1 · p. 2
2 Related Work
Prior multi-agent multimodal RAG work motivates collaborative problem-solving, but retrieval depth and memory structure remain challenges. GraMRAG addresses them with long-horizon visual reasoning and a dynamic graph memory.
2.1 Multi-Agent Systems in Multimodal RAG
- Multi-agent architectures support collaborative problem-solving, while insufficient retrieval rounds leave trajectories too shallow for deep-research tasks requiring tens of reasoning steps.GraMRAG introduces image-description bridging with multi-scale entity cropping and a visual toolchain for long-horizon multimodal planning.Source: §2.1 · p. 2
2.2 Memory Structures of Agent Systems
- History accumulation and iterative summarization both lose historical retrieval paths as interactions grow, inducing state blindness, redundant queries, and repetitive retrieval.GraMRAG’s dynamic DAG explicitly encodes temporal and logical action-observation dependencies to mitigate these problems.Source: §2.2 · p. 2
3 Method
GraMRAG combines multi-scale visual retrieval and textual research through a vision-text bridge, records reasoning in a dynamic multimodal DAG, and uses graph topology for step-level policy credit assignment.
3.1 Vision-Text Bridged Reasoning Paradigm
- Visual exploration uses multi-scale region crops and a Hit/No Hit judge before bridging verified visual evidence into textual research and final answer validation.Unsuccessful retrieval can trigger finer recropping or query reformulation; the bridge replaces the image with a caption while preserving reasoning and verified observations for the Text Researcher.Source: §3.1 · pp. 3–4
3.2 Multimodal Memory Graph
- The memory graph is a dynamic DAG that records action-observation dependencies across visual and textual phases, preserving typed reasoning units and their cross-modal transitions.Its nodes include the query and image, visual evidence, the image-description bridge, textual evidence, and the final answer; sequential and bridge edges retain the trajectory.Source: §3.2 · pp. 4–5
3.3 Topology-Aware Policy Optimization
- TAPO uses graph topology to mask gradients for dead-end steps in correct rollouts and valuable retrieval steps in incorrect rollouts, enabling step-level credit assignment.For correct rollouts, it identifies the critical path by traversing backward from the answer node; for incorrect rollouts, it protects relevant visual or textual retrieval from negative updates.Source: §3.3 · pp. 5–6
4 Experiments
GraMRAG is evaluated across text, image, visual-document, and video benchmarks using a unified binary model-based metric. It achieves state-of-the-art average scores, and ablations report complementary contributions from long-horizon reasoning, graph memory, and TAPO.
4.1 Datasets and Evaluation Metrics
- GraMRAG is evaluated on text, image and visual-document, and long-context video benchmarks with a unified binary reward-model metric.The benchmarks include HotpotQA, SQuAD, ViDoSeek, SlideVQA, MMLongBench, LVBench, WikiHowQA, SyntheticQA, and XVBench.Source: §4.1 · p. 6
4.3 Comparison with State-of-the-Art Methods
- 48.5% and 53.5% overall average scores are reported for GraMRAG with Qwen3-VL-4B-Instruct and Qwen3-VL-8B-Instruct, respectively, with state-of-the-art performance across benchmarks.The comparison includes ReAct, Vanilla RAG, UniversalRAG, VideoRAG, MemAgent, Mem1, and VimRAG.Source: Table 1 · p. 7, §4.3 · p. 7
4.2 Implementation Details
- Training proceeds in two stages: SFT fine-tunes the language model for 3 epochs with LoRA rank 32, then RL uses TAPO for 2 epochs, with 8 agent groups and batch size 32.All experiments use 8 NVIDIA H100 GPUs.Source: §4.2 · p. 7
4.3 Comparison with State-of-the-Art Methods
- ReAct can exhaust context under visual-token overhead, task-specific methods have limited generalizability, and summarization-based memories lose retrieval paths as rounds accumulate.The authors also describe VimRAG as lacking long-horizon reasoning capability.Source: §4.3 · p. 7
4.4 Ablation Study
- The ablation supports component complementarity: a multimodal memory graph mitigates state blindness and suppresses redundant retrieval, while replacing GRPO with TAPO further improves results; the complete model averages 53.5%, over 9 points above the baseline.Source: §4.4 · p. 8
5 Analysis and Discussion
The analysis links retrieval quality to sustained reasoning depth and reports that graph memory improves performance over iterative summarization. It also reports faster training convergence for TAPO than GSPO.
Retrieval quality depends on reasoning depth
- GraMRAG's vision-text bridge supports trajectories spanning tens of steps, and its memory graph directs retrieval toward unexplored evidence; the analysis reports higher hit rate across three task categories.The passage attributes this retrieval result to long-horizon reasoning depth.Source: §5 · p. 8
Graph pruning accelerates training convergence
- TAPO converges faster throughout training than GSPO, with topology-guided pruning supplying cleaner credit-assignment signals.The passage also states that rollout preference alignment is more decisive for performance than simply scaling up the training set.Source: Figure 6 · p. 8, §5 · p. 8
Graph memory mitigates state blindness
- Replacing iterative summarization-based memory with the multimodal memory graph yields a 3.3-point performance improvement.The passage associates the graph with mitigating state blindness and suppressing redundant retrieval.Source: §5 · p. 8
6 Conclusion
GraMRAG combines vision-text reasoning, dynamic graph memory, and topology-aware policy optimization for multimodal long-horizon reasoning. The paper reports consistent gains over existing baselines and state-of-the-art performance on diverse multimodal benchmarks.
6 Conclusion
- A vision-text bridged paradigm combines multi-scale entity cropping with a ReAct-style visual toolchain for long-horizon cross-modal reasoning.The framework's three innovations also include dynamic DAG memory and topology-aware policy optimization.Source: §6 · p. 8
- GraMRAG consistently outperforms existing baselines and achieves state-of-the-art performance on diverse multimodal benchmarks.The conclusion identifies future work on scaling the framework to larger architectures and broader multimodal scenarios.Source: §6 · p. 9
- The multimodal memory graph represents agent reasoning as a dynamic DAG to mitigate state blindness and suppress redundant retrieval.Topology-Aware Policy Optimization (TAPO) leverages graph topology for fine-grained credit assignment across multi-step trajectories.Source: §6 · pp. 8–9
7 Limitations
GraMRAG's multi-agent collaboration adds inference overhead relative to traditional RAG, which may hinder deployment in resource-constrained environments.
7 Limitations
- Multi-agent collaboration adds inference overhead over traditional RAG, potentially hindering deployment in resource-constrained environments.The authors describe the trade-off as generally acceptable in deep research, where users prioritize reasoning accuracy over response speed.Source: §7 · p. 9