Source-linked AI summary
TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph
Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, Hanwang Zhang
TL;DR
Multi-hop QA requires precise reasoning across entity relations represented as labels or text, while prior path-based and graph-feature methods face optimization or interpretability limitations. TransferNet unifies both relation forms by transferring entity scores across question-conditioned relations, and it substantially outperforms prior models, achieving 100% accuracy on MetaQA 2-hop and 3-hop questions. Its visualized intermediate results provide transparent reasoning steps.
Problem
Multi-hop QA must reason across entity relations at multiple steps, but existing approaches are difficult to optimize or lack interpretable intermediate reasoning.
Method
TransferNet attends to different question parts, scores relations, and differentiably transfers entity scores across activated relations in a unified label-text framework.
Results
100% accuracy is achieved on MetaQA 2-hop and 3-hop questions, while TransferNet significantly outperforms state-of-the-art models on WebQSP, CompWebQ, and text-form MetaQA.
Takeaways & Limitations
TransferNet provides an effective framework for multi-hop QA across knowledge-graph and text-formed relation graphs with interpretable intermediate results.
Abstract
from arXiv · showhide
Multi-hop Question Answering (QA) is a challenging task because it requires precise reasoning with entity relations at every step towards the answer. The relations can be represented in terms of labels in knowledge graph (e.g., \textit{spouse}) or text in text corpus (e.g., \textit{they have been married for 26 years}). Existing models usually infer the answer by predicting the sequential relation path or aggregating the hidden graph features. The former is hard to optimize, and the latter lacks interpretability. In this paper, we propose TransferNet, an effective and transparent model for multi-hop QA, which supports both label and text relations in a unified framework. TransferNet jumps across entities at multiple steps. At each step, it attends to different parts of the question, computes activated scores for relations, and then transfer the previous entity scores along activated relations in a differentiable way. We carry out extensive experiments on three datasets and demonstrate that TransferNet surpasses the state-of-the-art models by a large margin. In particular, on MetaQA, it achieves 100\% accuracy in 2-hop and 3-hop questions. By qualitative analysis, we show that TransferNet has transparent and interpretable intermediate results.
1 Introduction
Multi-hop QA over relation graphs remains difficult because reasoning must traverse multiple entity relations, which may appear as constrained labels or free text. TransferNet addresses both forms in one transparent framework and reports strong results across several datasets.
- Multi-hop QA requires reasoning over entity relations at multiple steps, unlike simple single-hop questions.
- Relation graphs represent edges either as constrained predicates in knowledge graphs or as free text extracted from corpora.The text form is described as more economical and practical because label-form graphs can be expensive and incomplete.
- TransferNet tackles label-form and text-form multi-hop questions within a unified framework.
- Existing path-based methods face convergence and search-space problems, while graph-neural methods provide limited interpretability.The path-based approach is also mainly designed for label-form graphs, whereas text-form relations have a larger search space.
- TransferNet transfers entity scores across activated relations while attending to question words at successive reasoning steps.Relation scores are converted into transfer probabilities between entity pairs.
- 100% accuracy is achieved on MetaQA 2-hop and 3-hop questions, with significant improvements over state-of-the-art models on WebQSP, CompWebQ, and text-form MetaQA.The experiments cover label, text, and mixed relation forms, and visualization supports interpretable intermediate results.
2 Related Work
Prior work addresses multi-hop QA through graph retrieval, semantic parsing, explicit paths, or implicit graph-neural reasoning. These approaches are often specialized to one graph form or provide limited visibility into intermediate reasoning.
- Mixed-form systems such as GraftNet and PullNet retrieve question-specific subgraphs and use graph convolutional networks to infer answers implicitly.
- GCN-based methods generally cannot produce the intermediate reasoning paths needed for interpretable multi-hop QA.
- Label-form QA methods include information retrieval and semantic parsing, which learn question-graph representations or parse questions into logical forms.
- VRN and SRN learn explicit reasoning paths with reinforcement learning, but related soft-distribution methods are difficult to extend to text-form relations.
- Text-corpus QA systems commonly construct entity graphs and apply graph neural networks for implicit multi-hop reasoning.
3 Methodology
TransferNet represents entities as nodes and labels or texts as relations, then performs multi-hop reasoning by attending to question parts and transferring entity scores across activated relations. Additional modules control score growth, resolve ambiguous text relations, and reduce computation while preserving differentiability and interpretability.
- Relation graphs: TransferNet models a relation graph with entities as nodes and labels, texts, or both as edges.A question starts from a topic entity and traverses relations toward answer entities.
- Multi-hop transfer: At each hop, TransferNet attends to question words, scores relations, and transfers previous entity scores along activated edges.Entity scores start at 1 for the topic entity and 0 for all others; relation scores form an adjacency matrix of transfer probabilities.
- Multi-hop transfer: After T steps, TransferNet combines intermediate entity-score vectors using a learned hop distribution and selects the highest-scoring entity as the answer.The hop distribution supports questions from 1-hop through T-hop without manually specifying the hop number.
- Training: TransferNet remains differentiable, allowing question attention, relation scores, and intermediate entity scores to be learned from a simple L2 objective.The target vector marks the golden answer entities, and training minimizes the Euclidean distance between predicted and target scores.
- Additional modules: Score truncation keeps entity scores within [0, 1] after each transfer step to prevent excessive values and gradient explosion at larger hop counts.The truncation operation is designed to preserve differentiability while maintaining the score range required by the loss.
- Additional modules: A language mask uses the question embedding to disambiguate identical text relations, while relation selection limits computation to at most ω relations per step.The language mask is needed for text-form graphs, whereas label-form predicates are treated as unambiguous; candidate relations are filtered using threshold τ and top-ω selection.
4 Experiments
The experiments evaluate TransferNet on label- and text-formed relation graphs across MetaQA, WebQSP, and CompWebQ, using dataset-specific graph construction and model settings. TransferNet reaches perfect 2-hop and 3-hop MetaQA accuracy and substantially improves text-form results over prior models.
- Datasets: MetaQA contains more than 400k movie-domain questions with up to 3 hops, 43k entities, 9 predicates, and 135k triples.The questions are generated from dozens of templates.
- Datasets: WebQSP contains 1.8 million entities, 572 predicates, and 5.7 million triples after pruning to mentioned predicates and nearby triples.Only its label form is used because of the knowledge graph’s large scale.
- Datasets: CompWebQ extends WebQSP with more hops and constraints; its PageRank-retrieved subgraphs average 1948 entities with 64% recall.
- Baselines: TransferNet is compared with memory, reinforcement-learning, graph-neural-network, probability-transfer, and embedding-based baselines across the evaluated datasets.The cited prior methods include KVMemNN, VRN, SRN, GraftNet, PullNet, ReifKB, and EmbedKGQA.
- Implementation: TransferNet uses three reasoning steps for MetaQA and two for WebQSP and CompWebQ, with GRU encoders for MetaQA and pretrained BERT for the latter datasets.MetaQA text relations use a second bidirectional GRU relation encoder; WebQSP and CompWebQ do not use hop annotations.
5 Results
TransferNet achieves strong results across label-formed and text-formed relation graphs, while its visualized intermediate activations support transparent reasoning. It is also data-efficient and converges quickly on MetaQA.
- Label-Formed Graph: 100% accuracy is achieved on 2-hop and 3-hop label-formed MetaQA questions.TransferNet reaches 97.5% on 1-hop MetaQA, with errors attributed to ambiguous entities.
- Label-Formed Graph: 71.4% accuracy on WebQSP surpasses previous state-of-the-art models at 68.1%.WebQSP contains more predicates and triples but fewer training examples than MetaQA.
- Label-Formed Graph: 48.6% accuracy on CompWebQ exceeds PullNet’s 47.2%.
- Text-Formed Graph: Text-form accuracy improves from 81.0% to 98.1% for 2-hop questions and from 78.2% to 94.3% for 3-hop questions.Unlike graph-feature aggregation methods, TransferNet provides intermediate relation paths through attention-based reasoning.
- Mixed Graph: Adding 50% label relations yields 94.7% accuracy versus PullNet’s 85.2% in the mixed setting.The added labels improve performance by about 0.4% over the pure text form because some relations are missing from the text corpus.
- Interpretability and Ablation: The language mask filters incorrect activations when similar text descriptions activate both the target release year and an incorrect genre entity.The ablation study also identifies score truncation and language masking as important, especially for text-form relations.
- Model Efficiency: TransferNet reaches the same performance with 10% of the training data and reaches optimal results after two epochs.Figure 4 compares average hits@1 across partial training data and training epochs on label-formed MetaQA.
6 Conclusions
TransferNet is presented as an effective and transparent framework for multi-hop QA over knowledge graphs and text-formed relation graphs. It achieves perfect 2-hop and 3-hop MetaQA accuracy and significantly outperforms state-of-the-art models on several harder settings.
- TransferNet supports multi-hop QA over knowledge graphs and text-formed relation graphs with transparent intermediate reasoning.
- TransferNet achieves 100% accuracy on 2-hop and 3-hop label-formed MetaQA questions and significantly outperforms state-of-the-art models elsewhere.The paper’s qualitative analysis also reports good interpretability.