Source-linked AI summary
Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals
Gaole He, Yunshi Lan, Jing Jiang, Wayne Xin Zhao, Ji-Rong Wen
TL;DR
Multi-hop KBQA lacks supervision for intermediate reasoning steps because training data typically provides only question-answer pairs. The paper uses a teacher-student framework with bidirectional reasoning to learn intermediate entity supervision, and experiments on three benchmark datasets demonstrate effectiveness, especially when training data is limited.
Problem
Multi-hop KBQA typically lacks supervision for intermediate reasoning steps because datasets provide question-answer pairs rather than relation paths.
Method
A student network finds answers while a teacher network uses forward and backward reasoning to learn intermediate entity distributions for supervision.
Results
Experiments on three benchmark datasets demonstrate the approach’s effectiveness, especially for cases lacking training data.
Takeaways & Limitations
Bidirectional reasoning enables the teacher to produce more reliable intermediate supervision signals and alleviate spurious reasoning.
Takeaways & Limitations
The current approach adopts NSM as its student network, with extension to other neural architectures left for future work.
Abstract
from arXiv · showhide
Multi-hop Knowledge Base Question Answering (KBQA) aims to find the answer entities that are multiple hops away in the Knowledge Base (KB) from the entities in the question. A major challenge is the lack of supervision signals at intermediate steps. Therefore, multi-hop KBQA algorithms can only receive the feedback from the final answer, which makes the learning unstable or ineffective. To address this challenge, we propose a novel teacher-student approach for the multi-hop KBQA task. In our approach, the student network aims to find the correct answer to the query, while the teacher network tries to learn intermediate supervision signals for improving the reasoning capacity of the student network. The major novelty lies in the design of the teacher network, where we utilize both forward and backward reasoning to enhance the learning of intermediate entity distributions. By considering bidirectional reasoning, the teacher network can produce more reliable intermediate supervision signals, which can alleviate the issue of spurious reasoning. Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our approach on the KBQA task. The code to reproduce our analysis is available at https://github.com/RichardHGL/WSDM2021_NSM.
1 INTRODUCTION
Multi-hop KBQA requires finding answers through multiple reasoning steps, but training data usually provides only final answers, leaving intermediate steps unsupervised. The paper proposes a teacher-student framework with bidirectional reasoning to learn intermediate supervision and reduce spurious paths.
- Multi-hop KBQA extends question answering by requiring multiple reasoning hops through a knowledge base.
- Training datasets typically pair questions with answers rather than relation paths, so algorithms receive feedback only at the final answer.
- Reinforcement-learning approaches rely heavily on terminal rewards, with reward shaping and action dropout introduced to reduce spurious paths.
- The proposed framework separates roles: a student finds answers, while an auxiliary teacher learns relevant intermediate entity distributions as supervision signals.
- The teacher jointly models forward reasoning from topic entities and backward reasoning from answers, using their correspondence to improve intermediate supervision.
- Experiments on three benchmark datasets report effectiveness, especially for cases lacking training data, and identify the framework as the first to explicitly learn intermediate supervision in this setting.
2 RELATED WORK
Related work spans KBQA methods, multi-hop reasoning, and teacher-student learning. The paper distinguishes its approach by applying teacher-student learning to spurious reasoning caused by weak supervision in multi-hop KBQA.
- Knowledge Base Question Answering: KBQA methods include semantic parsing systems that execute intermediate logic forms and retrieval-based systems that directly retrieve answers from a knowledge base.
- Knowledge Base Question Answering: Recent multi-hop KBQA methods use classical neural architectures, external corpora, or enriched knowledge-graph embeddings to reason across the knowledge base.
- Knowledge Base Question Answering: Existing multi-hop KBQA methods commonly optimize final prediction performance as their only objective.
- Multi-hop Reasoning: Multi-hop reasoning research decomposes complex queries, performs sequential attention-based reasoning, or conducts graph reasoning with reinforcement learning.
- Teacher-student Framework: Knowledge distillation uses a teacher-student framework in which teacher predictions serve as soft labels for training a student model.
- Teacher-student Framework: Prior teacher-student applications include web question answering, visual question answering, and reading comprehension; this work targets spurious reasoning in multi-hop KBQA.
3 PRELIMINARY
The paper formalizes knowledge bases as relational triples and defines KBQA as identifying answer entities for a natural-language question from the entities available in the knowledge base.
- A knowledge base is represented as triples connecting head entities, relations, and tail entities.
- Entity neighborhoods contain both incoming and outgoing triples, with reverse triples used to simplify neighborhood notation.
- Entity and relation embeddings are represented as d-dimensional column vectors in embedding matrices.
- KBQA asks for answer entities in the knowledge base given a natural-language question and its available candidate entity set.
- Entities mentioned in a question are designated as topic entities.
4 THE PROPOSED APPROACH
This section introduces the proposed approach for multi-hop KBQA under a teacher-student framework.
- The proposed approach addresses multi-hop KBQA within a teacher-student framework.
- The section presents the approach as a method for the multi-hop KBQA task.
- The approach is introduced as the paper’s proposed framework.
4.1 Overview
The framework addresses missing intermediate-step supervision in multi-hop KBQA by pairing an answer-seeking student with a teacher that infers intermediate entity distributions.
- The main difficulty is that only answer entities are labeled, leaving intermediate reasoning steps without direct supervision.
- The student network performs multi-hop KBQA, while the teacher network provides inferred intermediate entity distributions as pseudo-supervision.
- The student is based on Neural State Machine and maintains an entity distribution that is gradually updated during reasoning over the KB graph.
4.2 Neural State Machine for Multi-hop KBQA
The adapted Neural State Machine uses question-conditioned instruction vectors and graph-based reasoning to update entity representations and distributions across hops.
- NSM contains an instruction component that sends vectors to a reasoning component, which infers entity distributions and learns entity representations.
- At different reasoning steps, instruction vectors attend to specific question parts while dynamically incorporating previous instruction information.
- The reasoning component combines the current instruction with prior entity distributions and embeddings to produce the next entity distribution and embeddings.
- Initial entity embeddings aggregate embeddings of relations involving each entity rather than using the entity’s original embedding.
- Relation-based initialization emphasizes relational semantics, reduces noisy-entity influence, and supports unseen entities with known contextual relations.
- The model aggregates relation-matching messages from neighboring triples, weighted by entity probabilities from the previous reasoning step.
4.3 The Teacher Network
The teacher network learns intermediate supervision by synchronizing forward reasoning from topic entities with backward reasoning from answer entities through distribution correspondence.
- Bidirectional Reasoning: The teacher uses correspondence between forward and backward intermediate distributions as constraints, without labeled intermediate entity distributions.
- Bidirectional Reasoning: Forward reasoning searches from topic entities to answers, whereas backward reasoning searches from answers back toward topic entities.
- Bidirectional Reasoning: The two processes synchronize at intermediate steps so their entity distributions become more reliable than distributions learned in one direction.
- Reasoning Architectures: Two teacher architectures are proposed: parallel reasoning with separate NSMs and hybrid reasoning with shared instruction and cycled information flow.
- Reasoning Architectures: Hybrid reasoning feeds the final forward state into backward reasoning, enabling backward reasoning to better trace the forward path despite different semantics.
4.4 Learning with the Teacher-Student Framework
The teacher is trained with directional reasoning and correspondence losses, then supplies averaged intermediate distributions to supervise the student’s reasoning.
- Teacher training combines reasoning loss for accurate entity prediction with correspondence loss for consistency between the two reasoning processes.
- The correspondence loss uses Jensen-Shannon divergence to measure symmetric differences between intermediate distributions.
- The teacher’s total loss is L_t = L_f + λ_bL_b + λ_cL_c, with λ_b and λ_c controlling component weights.
- After convergence, the teacher averages forward and backward distributions to create intermediate supervision signals for the student.
- The student adds a loss matching its intermediate entity distributions to the teacher’s supervision signal alongside its reasoning loss.
- The framework learns intermediate distributions without additional intermediate-step labels by enforcing correspondence between bidirectional reasoning processes.
5 EXPERIMENT
Experiments evaluate the approach on three multi-hop KBQA benchmarks using Hits@1 and F1, comparing it with retrieval, graph, embedding, reinforcement-learning, and model variants. The teacher-student variants are especially effective on the difficult WebQuestionsSP and CWQ datasets.
- 5.1 Datasets: Evaluation uses MetaQA, WebQuestionsSP, and Complex WebQuestions, covering questions requiring up to 3, 2, and 4 reasoning hops, respectively.CWQ includes composition, conjunction, comparative, and superlative questions.
- 5.2 Evaluation Settings: Models are evaluated as answer-ranking systems with Hits@1 and F1, where Hits@1 measures whether the top-ranked answer is correct.Models are trained on the training set and tuned on the validation set.
- 5.2 Compared Methods: Comparisons include KV-Mem, GraftNet, PullNet, SRN, EmbedKGQA, and three NSM variants differing in teacher-network usage and reasoning strategy.NSM omits the teacher, NSM+p uses parallel reasoning, and NSM+h uses hybrid reasoning.
- 5.3 Results: Among baselines, EmbedKGQA and PullNet perform best overall, while KV-Mem performs worst and struggles with complex reasoning steps.Most methods perform well on MetaQA-1hop and MetaQA-2hop, whereas WebQuestionsSP and CWQ are more difficult.
- 5.3 Results: NSM outperforms competitive baselines in most cases, while NSM+p and NSM+h substantially improve over NSM on WebQuestionsSP and CWQ.The two teacher-network variants have similar results, with hybrid reasoning slightly better on WebQuestionsSP and CWQ.
5.4 Detailed Performance Analysis
The analysis examines overall performance, teacher-network variants, parameter choices, one-shot learning, and a qualitative case study. Results support bidirectional teacher reasoning, correspondence loss, and teacher-guided intermediate supervision as important components.
- Overall Performance: The approach overall outperforms the compared KBQA methods in Table 2.The table reports Hits@1 in percent and identifies the best and second-best methods.
- Teacher-Network Ablation: Bidirectional teacher reasoning consistently outperforms unidirectional forward or backward reasoning.Both parallel and hybrid variants perform better than NSM+f and NSM+b in the ablation study.
- Teacher-Network Ablation: Removing the correspondence loss substantially reduces performance, indicating that forward and backward reasoning mutually enhance one another.The comparison removes the correspondence loss from both bidirectional teacher variants.
- Parameter Tuning: Hybrid reasoning works well with small λ, whereas parallel reasoning works better with relatively large λ; λ_c=0.01 and λ_b=0.1 are reported as good choices.The teacher influence coefficient λ is tuned over {0.01, 0.05, 0.1, 0.5, 1.0}.
- Intermediate-Entity Retrieval: The teacher network is much better than the student network at finding intermediate entities, although it performs slightly worse at the second hop.The comparison uses Precision, Recall, and F1 on MetaQA-3hop; third-hop results are omitted because it is the last hop.
- One-Shot Evaluation: Under one-shot training, the approach still works very well and improves more substantially over basic NSM.The one-shot dataset samples a single training case for every question template.
6 CONCLUSION
The paper presents a teacher-student approach for multi-hop KBQA, using bidirectional reasoning to learn intermediate supervision signals and improve the student’s reasoning. Experiments on three benchmark datasets show superior effectiveness over previous methods, while the current implementation uses NSM as the student network.
- The student network focuses on answering the query, while the teacher network learns intermediate supervision signals to improve the student’s reasoning.The teacher learns intermediate entity distributions using correspondence between forward and backward reasoning states.
- The teacher integrates forward and backward reasoning and supports their integration through two reasoning architectures.
- Experiments on three benchmark datasets show that the proposed model is superior to previous methods in multi-hop KBQA effectiveness.
- The current approach adopts NSM as the student network and may be extended to other neural architectures or graph-learning strategies.Future work also considers improving entity embeddings with KB embedding methods and obtaining better intermediate supervision signals.