Source-linked AI summary
Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning
Yuncheng Hua, Yuan-Fang Li, Gholamreza Haffari, Guilin Qi, Wei Wu
TL;DR
Complex KBQA must handle diverse questions while existing retrieval approaches require manually collected annotations and may be disconnected from the answering model. MARL alternately and jointly trains a retriever and programmer from answer-performance feedback, achieving state-of-the-art performance on CQA.
Problem
Existing KBQA models face distributional bias from diverse question types, while supervised retrieval requires costly annotations and can be trained independently of the meta-learner.
Method
MARL alternately optimizes a retriever and programmer, selecting secondary questions and using adaptive-programmer answer rewards as weak supervision for retrieval.
Results
MARL achieves state-of-the-art overall macro and micro F1 on CQA and reaches competitive results using only 1% of the training data for meta-learning tasks.
Takeaways & Limitations
Joint retriever-programmer training lets the system learn support sets from question-answering performance while adapting programmers to new questions.
Takeaways & Limitations
The reward feedback does not disentangle task-specific knowledge from task-agnostic knowledge in the programmer’s correct answers.
Abstract
from arXiv · showhide
A compelling approach to complex question answering is to convert the question to a sequence of actions, which can then be executed on the knowledge base to yield the answer, aka the programmer-interpreter approach. Use similar training questions to the test question, meta-learning enables the programmer to adapt to unseen questions to tackle potential distributional biases quickly. However, this comes at the cost of manually labeling similar questions to learn a retrieval model, which is tedious and expensive. In this paper, we present a novel method that automatically learns a retrieval model alternately with the programmer from weak supervision, i.e., the system's performance with respect to the produced answers. To the best of our knowledge, this is the first attempt to train the retrieval model with the programmer jointly. Our system leads to state-of-the-art performance on a large-scale task for complex question answering over knowledge bases. We have released our code at https://github.com/DevinJake/MARL.
1 Introduction
Complex question answering maps natural-language questions to executable action sequences, but diverse datasets challenge one-size-fits-all models and manually supervised retrieval. MARL addresses this by jointly and alternately training a retriever and programmer using answer-performance feedback.
- Motivation: CQA maps natural-language questions to logical-form action sequences executed on a knowledge base, including aggregation actions such as intersection, counting, and min/max.Answers may be entities or numbers.
- Motivation: One-size-fits-all KBQA models struggle when question diversity reduces shared generic knowledge across less common question types.CQA questions vary substantially in length and complexity, from one select action to select, intersection, and count.
- Prior limitations: Existing approaches either train separate models by question category or retrieve similar questions using annotations, limiting transfer or imposing manual-labeling costs.S2A also trains its retriever independently of the meta-learner, making support-set quality difficult to evaluate through answering performance.
- MARL: MARL jointly optimizes the retriever and programmer in two stages to adapt a programmer for each primary question using selected secondary questions.The retriever selects top-N similar questions, while the programmer adapts from their trial trajectories and rewards.
- MARL: The retriever learns weakly supervised support-set selection from adaptive programmers’ answer rewards rather than teacher-forced similarity labels.Positive reward differences increase the probability of selecting a question set, while negative differences decrease it.
- Results: MARL achieves state-of-the-art overall macro and micro F1 on the large-scale CQA dataset, using only 1% of training data to form meta-learning tasks.It outperforms standard imitation-learning and reinforcement-learning methods.
2 Alternate Meta-learning for Complex Question Answering over Knowledge Bases
MARL alternates meta-reinforcement learning for a programmer with weakly supervised reinforcement learning for a retriever, using sampled secondary questions to adapt to primary questions. The method combines task-specific knowledge from similar questions with task-agnostic knowledge in a generic policy, while jointly optimizing both networks from answer rewards.
- Method overview: Each primary question forms a task, and the retriever selects top-N secondary questions to help rapidly adapt a unique programmer to that task.The selected questions serve as a support set for meta-learning on an unseen or new primary question.
- Method overview: MARL trains an encoder-decoder programmer and a retriever jointly to transform questions into executable programs and select analogous secondary questions.The programmer produces programs executed on the knowledge base, while the retriever provides support questions for adaptation.
- Alternate training: The retriever learns weakly from answer rewards rather than teacher-forced annotations, favoring support sets that improve the adapted programmer’s primary-question reward.Support sets are sampled, adapted programmers are evaluated, and the resulting feedback guides retriever optimization.
- Filter softmax: The filter softmax restricts retrieval candidates by question type and weights eligible questions using their semantic similarity to the primary question.DSSM computes the semantic similarity, while the filter excludes questions with different types.
- Knowledge decomposition: MARL separates task-specific knowledge from task-agnostic knowledge, using retrieved similar questions for the former and a generic policy for the latter.The distinction motivates retrieval for local task features and meta-learning for reusable features across tasks.
- Alternate training: MARL divides each epoch into alternating stages: the programmer learns broadly applicable knowledge, then the retriever is updated using the programmer’s performance.The first stage fixes the retriever while optimizing the programmer; the second fixes the programmer while updating the retriever.
3 Evaluation
MARL is evaluated on the large-scale CQA benchmark against established baselines and model variants, with results emphasizing performance under category imbalance and the value of joint retriever learning.
- Dataset and setup: CQA contains 944K, 100K, and 156K question-answer pairs for training, validation, and testing, respectively, across seven answer-based categories.Accuracy is used for Verification, Quantitative (Count), and Comparative (Count), while other categories use F1; results are presented as F1 for consistency.
- Dataset and setup: MARL uses approximately 1% pseudo-annotated training data, 2,072 pseudotask samples, and jointly learns the programmer and retriever with MAML.The pseudo-gold action sequences are generated by breadth-first search, while the Vanilla comparison model is a one-size-fits-all BiLSTM trained with reinforcement learning.
- Performance comparison: MARL achieves 66.96% macro F1 and 77.71% micro F1, outperforming KVmem, CIPITR-All, and CIPITR-Sep overall.Its macro-F1 advantages over KVmem, CIPITR-All, and CIPITR-Sep are 47.51, 47.14, and 19.26 percentage points, respectively.
- Performance comparison: MARL achieves the best or second-best performance across all seven categories and leads on the three hardest categories.Comparative Reasoning and Comparative (Count), two categories representing less than 5% of the training set each, are among those where MARL performs best.
- Ablation and analysis: MAML-based Random, Jaccard, and MARL variants outperform the non-MAML Vanilla model, supporting task-specific adaptation.CIPITR-All performs worse than CIPITR-Sep across categories, while MARL adapts to different categories with one model trained once.
- Ablation and analysis: MARL improves over Vanilla by 2.99 percentage points and over Jaccard by 1.40 micro-F1 points, exceeding random and fixed retrieval variants.Random improves over Vanilla by 0.97 points, and Jaccard improves by 1.59 points; the authors attribute MARL’s additional gain to alternating joint optimization.
4 Related Work
Related work addresses distributional variation in complex question answering through category-specific models and meta-learning, while MARL targets transfer across question types with learned adaptation.
- Complex question answering: CIPITR uses neural program induction without annotations and trains either separate models for each question category or one model across all categories.The separate and unified variants are denoted CIPITR-Sep and CIPITR-All, respectively.
- Complex question answering: Unlike one-size-fits-all category models, the paper aims to discover knowledge specific to a new task and adapt the programmer to that task.This framing emphasizes transfer between question types rather than independently fitting each type.
- Meta-learning: Meta-learning uses inductive biases learned from similar tasks so models can learn new tasks quickly, commonly by learning how to update underlying-model parameters.MAML is cited as a prominent approach to this parameter-update direction.
5 Conclusion
The paper concludes that MARL alternately and jointly trains a retriever and programmer, enabling adaptation from similar questions using weak supervision and improving CQA performance.
- Conclusion: MARL jointly and alternately optimizes a retriever that selects questions and a programmer that adapts to them for answering primary questions.The approach is framed as a meta-learning method for complex question answering over knowledge bases.
- Conclusion: The model adapts quickly to new questions using similar questions and weak supervision from question-answering performance.This avoids relying on manually labeled similar-question retrieval targets.
- Conclusion: MARL outperforms state-of-the-art models on the large-scale CQA dataset.The authors identify distributional bias and the cost of manually labeling similar questions as challenges addressed by the method.
- Conclusion: The authors plan to extend MARL to other domains and tasks requiring manual support-set construction.