RACER: Reinforced Agent Collaboration for Explainable Reasoning on Knowledge Graphs

Yuwei Lou; Hao Hu; Yuzhou Jiang; Zongfei Zhang; Liang Wang; Jincai Liu; Jidong Ge; Xianping Tao

Source-linked AI summary

RACER: Reinforced Agent Collaboration for Explainable Reasoning on Knowledge Graphs

Yuwei Lou, Hao Hu, Yuzhou Jiang, Zongfei Zhang, Liang Wang, Jincai Liu, Jidong Ge, Xianping Tao

arXiv:2608.29263v1Aug 29, 2026cs.AI

TL;DR

LLMs can hallucinate and struggle with multi-hop reasoning, while KG-enhanced approaches face large search spaces and fixed prompting. RACER combines semantic-pruned reinforcement learning, shared-memory path refinement, and four-agent collaboration, achieving approximately 5% average improvement over state-of-the-art baselines on CommonsenseQA and OpenBookQA.

Problem
LLMs face hallucination and complex multi-hop reasoning challenges, while existing KG-enhanced approaches rely on single-agent path extraction and fixed prompting.
Method
RACER uses semantic-aware action pruning, teacher-guided reinforcement learning, a cross-task shared memory graph, attention-based path refinement, and four-role multi-agent collaboration.
Results
Approximately 5% average performance improvement over current state-of-the-art baselines is reported on CommonsenseQA and OpenBookQA.
Takeaways & Limitations
RACER provides traceable and verifiable reasoning pathways while supporting adaptive collaborative execution instead of a single inflexible prompt-generation step.
Takeaways & Limitations
Future work is needed for dynamic knowledge graph updates, more sophisticated role allocations, and transfer to domain-specific reasoning tasks.

Abstract

from arXiv · show

Large Language Models (LLMs) often suffer from hallucination and struggle with complex reasoning tasks requiring multi-hop domain knowledge. While integrating Knowledge Graphs (KGs) provides a structured and verifiable information source, current KG-enhanced LLM paradigms usually rely on single-agent path extraction and fixed prompting, lacking adaptability and facing huge search spaces. To address these challenges, we propose RACER, a Reinforced Agent Collaboration framework for Explainable Reasoning on knowledge graphs. RACER employs a semantic-aware action pruning and teacher-guided reinforcement learning mechanism to efficiently extract high-quality reasoning pathways from large-scale KGs. Furthermore, to mitigate single-path generation pitfalls, we introduce a cross-task accumulated shared memory graph paired with an attention-driven multi-path knowledge refinement module. Finally, RACER orchestrates these components through a four-role multi-agent collaboration system (GraphAgent, TemplateAgent, AnswerAgent, and CriticAgent) to dynamically refine prompts and evaluate answers. Extensive experiments on CommonsenseQA and OpenBookQA datasets demonstrate that RACER significantly outperforms state-of-the-art KG-enhanced LLM baselines with an average improvement of 5\%, offering robust and highly interpretable reasoning capabilities.

1 Introduction

LLMs face hallucination and degraded performance on scenarios outside training data or involving long-range dependencies, while KG-enhanced methods still struggle with large search spaces and fixed templates. RACER addresses these challenges with reinforcement learning, shared memory, multi-agent collaboration, and achieves approximately 5% improvement over state-of-the-art baselines.

LLMs often hallucinate and degrade on unfamiliar scenarios or tasks involving long-range dependencies.
Knowledge graphs support multi-hop reasoning and verifiable relational pathways, but KG-enhanced methods face enormous search spaces and fixed template constraints.
RACER combines reinforcement learning, knowledge graphs, and multi-agent collaboration to extract relevant reasoning paths and align them with task execution.
RACER uses semantic-aware action pruning and teacher-guided reinforcement learning to extract efficient paths, a shared memory graph to refine candidate paths, and four specialized agents for end-to-end reasoning.
Approximately 5% performance improvement over state-of-the-art baselines is reported across multiple benchmark datasets.

2 Related Work

Prior work uses retrieval, knowledge graphs, and multi-agent systems to improve LLM reasoning, but existing approaches remain limited by unstructured information, fixed prompting, and reliance on single-agent execution. Multi-agent frameworks divide complex reasoning into cooperative roles to improve reliability.

RAG can reduce hallucinations through external knowledge, but scattered, low-quality, and unstructured information makes accurate retrieval and reasoning difficult.
Knowledge graphs ground LLM responses in structured facts and verifiable logical pathways, complementing approaches that inject KG knowledge during model training.
KG prompting methods verify or decompose LLM reasoning with structured evidence, but commonly depend on manually designed prompt templates.
Multi-agent systems assign distinct reasoning, validation, and generation roles to cooperative agents, mitigating single-agent failure points and improving overall reliability.

3 Methodology

RACER combines semantic-pruned KG reinforcement learning, shared-memory path refinement, and specialized multi-agent orchestration. Its architecture targets efficient extraction of relevant, interpretable reasoning knowledge from large-scale graphs.

RACER comprises semantic-pruning KG reinforcement learning, a shared memory graph with dual attention, and multi-agent collaborative reasoning.
Identifying an ideal concise and relevant KG subgraph is computationally intractable because large-scale subgraph extraction is NP-hard.
RACER formulates subgraph construction as discovering relevant reasoning chains and uses semantic-aware action pruning with teacher-guided reinforcement learning.
Semantic pruning scores candidate actions by query-relation, query-node, and target-node similarities, then retains the top-K actions to reduce the search space.

Teacher-Guided Policy Gradient Training

RACER guides reinforcement-learning policies with shortest paths computed by BFS, combining teacher actions and policy sampling during training. The teacher signal supplies a positive reward intended to accelerate convergence and improve generalization.

BFS computes the shortest path from each source node to its target, providing a teacher trajectory with prior information about the optimal path.
During training, RACER follows teacher actions with probability p_teacher and samples from the reinforcement-learning policy with probability 1 − p_teacher.
The policy state combines the current node embedding, query embedding, and historical path encoding.
A fixed positive reward R_teacher = 1.0 supervises teacher-guided steps to accelerate policy convergence and enhance generalization.

Reward Function Design

RACER combines complementary reward components to guide graph traversal toward semantically relevant target entities while limiting unnecessary exploration. The design balances path efficiency, relation relevance, goal proximity, and successful target reaching.

Reward Components: A negative step penalty discourages unnecessary exploration and reduces path length.The penalty is applied at each timestep to prevent protracted exploration.
Reward Components: Relation similarity rewards edges whose traversed relations are semantically aligned with the query.The reward uses semantic similarity between the traversed relation and the query embedding.
Reward Components: Goal-oriented dense rewards guide the policy toward target nodes by rewarding increases in semantic similarity between current nodes and targets.During multiple-choice inference, each answer choice is mapped to a graph entity representation and evaluated as a candidate target.
Reward Components: A reach-goal reward provides a positive signal when the policy arrives at the target node, while hyperparameters control the relative contribution of each reward term.The framework uses λreach for the arrival reward and adjusts λstep, λrel, λgoal, and λreach to accommodate task requirements.
Reward Components: The composite reward combines step, relation-similarity, goal-oriented, and reach-goal terms to balance exploration efficiency with semantic relevance.The total reward integrates these components into a single objective for target identification.

3.2 Shared Memory Graph and Attention-Driven Knowledge Refinement

RACER's shared memory graph records historical edge performance and uses those statistics to improve path selection. Reranking actions by edge quality leverages accumulated experience and mitigates local optima during multi-path reasoning.

Shared Memory Graph: The shared memory graph addresses weak historical-path utilization and single-path local optima by persistently maintaining edge-level statistics for path selection.It is coupled with attention-driven multi-path knowledge refinement to improve the quality of auxiliary reasoning paths.
Edge Statistics: For each knowledge-graph edge, the method maintains statistics describing successful, failed, and rejected path-selection outcomes.These statistics provide global information for evaluating edge quality.
Edge Quality Scoring: An edge quality score is computed from the accumulated statistics to guide path selection.The score incorporates frequency information and uses γ to balance success against rejection rates.
Action Selection: Candidate actions are reranked by edge quality, prioritizing edges with stronger historical performance and mitigating local optima.This selection mechanism uses global statistical information during action choice.

Multi-Path Generation

RACER generates diverse candidate paths and refines them into a compact subset of query-relevant knowledge. Temperature sampling promotes diversity early, while greedy selection and dual attention preserve quality and relevance.

Candidate Path Generation: RACER generates N candidate paths from a query and starting-node set using reinforcement-learning action probabilities.The candidate set is denoted P = {P1, P2, ..., PN}.
Candidate Path Generation: Temperature sampling is used during the first half of generation to increase diversity, followed by greedy selection to ensure path quality.The temperature parameter τ controls sampling randomness.
Attention-Driven Path Refinement: A dual-attention refinement network selects relevant paths through cross-path relational modeling and query-semantic alignment.The network models complementary and redundant relationships among paths while matching each path to the query.
Attention-Driven Path Refinement: The refinement function reduces k candidate paths to a smaller subset P* that preserves information needed to answer the query.The selected subset satisfies |P*| ≪ k and is passed to the template generator as compact context.
Attention-Driven Path Refinement: A composite importance score combines attention-derived relevance and intrinsic path importance before selecting the Top-k′ paths.The score is si = λαi + (1 − λ)βi, with k′ determined through validation.

Agent Roles and Collaboration

RACER divides reasoning across four specialized agents that coordinate path extraction, knowledge templating, answer generation, and evaluation. Critic feedback can trigger alternative graph exploration and update shared memory to avoid repeated failures.

System Coordination: The four-agent system performs end-to-end optimization from graph path search through knowledge generation and final question answering.Its roles are GraphAgent, TemplateAgent, AnswerAgent, and CriticAgent.
GraphAgent: GraphAgent extracts knowledge-triplet sequences with semantic-aware pruning and teacher-guided reinforcement learning, then refines them with attention.It distills high-quality knowledge triplets for downstream agents.
TemplateAgent: TemplateAgent converts refined triplet sequences into logical paths and natural-language templates using the shared memory graph.The templates are designed to be comprehensible to large language models.
AnswerAgent: AnswerAgent uses TemplateAgent’s outputs to produce task-specific answers and records their correctness for inspection.Generated answers are subsequently examined by CriticAgent.
CriticAgent: CriticAgent returns ACCEPT or UNSURE with confidence, and UNSURE activates alternative semantic exploration while penalizing failed trajectories in shared memory.This feedback loop supports adaptive self-correction and prevents redundant cycles.

4 Experiment

RACER is evaluated on CommonsenseQA and OpenBookQA against fine-tuned, KG-enhanced, zero-shot, and KG-prompting baselines using multiple backbone LLMs. It consistently outperforms comparison methods, while ablations show that semantic pruning and teacher guidance contribute strongly to performance.

Experimental Setup: RACER is evaluated on CommonsenseQA and OpenBookQA, both using ConceptNet as the background knowledge graph.CommonsenseQA contains 12,102 questions, while OpenBookQA contains 5,957 elementary-science questions.
Experimental Setup: The baselines span fine-tuned language models, KG-enhanced language models, zero-shot LLMs, and LLMs using KG prompting.Examples include BERT, QA-GNN, GPT-4, and KnowGPT.
Implementation: RACER uses multiple backbone LLMs, including GPT-5, Qwen, GLM, and Gemini 3, with few-shot prompts and a maximum of three collaboration turns.If consensus is not reached within max_turns = 3, AnswerAgent returns the candidate with the highest initial confidence.
Main Results: 84.7%: RACER (GPT-4) achieves this accuracy on CommonsenseQA, surpassing KnowGPT.This is one example of RACER outperforming existing KG-prompting methods.
Main Results: 98.0%: RACER (GPT-5) reaches this accuracy on OpenBookQA, closely approaching human expert levels.The reported state-of-the-art result occurs with an advanced backbone model.
Ablation Studies: −4.6% / −3.7%: removing semantic pruning produces the largest reported performance drop across the evaluated settings.Removing teacher guidance causes the next-largest reported decrease, at −3.2% / −2.4%.

5 Conclusion

RACER combines reinforcement learning, knowledge graphs, and multi-agent collaboration to address LLM hallucinations and inference limitations. Its evaluation reports improved benchmark performance and transparent logic verification, while future work targets broader adaptability and dynamic knowledge updates.

RACER combines reinforcement learning, multi-agent collaboration, and knowledge graphs to address hallucinations and inference limitations in LLMs.
A cross-task accumulated shared memory graph preserves and reuses path-finding statistics to dynamically refine knowledge retrieval.
RACER uses a four-role multi-agent collaboration mechanism to replace inflexible prompt generation with an adaptive collaborative execution pipeline.
Approximately 5% average performance improvement over current state-of-the-art baselines was achieved on CommonsenseQA and OpenBookQA.
Ablation studies validate the criticality of multi-agent collaboration and reinforcement-learning-guided path extraction components.
Future work will integrate dynamic knowledge-graph updates, explore more sophisticated role allocations, and transfer RACER to domain-specific reasoning tasks.