Source-linked AI summary
Question Answering with Subgraph Embeddings
Antoine Bordes, Sumit Chopra, Jason Weston
TL;DR
Open-domain question answering requires mapping natural-language questions to answers in large knowledge bases while avoiding extensive hand-crafted resources. The paper learns joint embeddings and scores questions against structured candidate-answer representations, extending inference to longer paths and richer subgraphs. It achieves competitive performance on WebQuestions, with an ensemble improving the state of the art.
Problem
Open-QA systems require expert-crafted lexicons, grammars, and knowledge-base schemas, while the task must handle broad natural-language questions over large knowledge bases.
Method
The model learns low-dimensional embeddings for question words and knowledge-base constituents, scoring questions against candidate answers represented by paths and surrounding subgraphs.
Results
The approach is competitive on WebQuestions, outperforming several published systems, performing similarly to one, and achieving a state-of-the-art improvement through an ensemble.
Takeaways & Limitations
Longer-path inference and richer subgraph answer representations extend the earlier embedding model to more complicated questions without lexicons, rules, or parsing systems during training.
Takeaways & Limitations
The task assumes questions contain one identified knowledge-base entity, using plain string matching when that entity is not given.
Abstract
from arXiv · showhide
This paper presents a system which learns to answer questions on a broad range of topics from a knowledge base using few hand-crafted features. Our model learns low-dimensional embeddings of words and knowledge base constituents; these representations are used to score natural language questions against candidate answers. Training our system using pairs of questions and structured representations of their answers, and pairs of question paraphrases, yields competitive results on a competitive benchmark of the literature.
1 Introduction
Open-domain question answering queries large, growing knowledge bases with natural-language questions, but remains difficult because of scale and language interpretation. Existing systems often depend on hand-crafted resources, motivating an embedding approach that handles more complex questions with minimal manual engineering.
- Motivation: Open-domain question answering aims to query large structured knowledge bases using natural-language questions across domains.Knowledge bases organize diverse answers in structured formats, but their scale makes efficient querying challenging.
- Related work: Existing open-QA systems primarily use information retrieval or semantic parsing to identify answers from knowledge bases.Retrieval systems generate candidates and apply detection heuristics, whereas semantic parsers convert questions into formal queries.
- Limitations of prior systems: These approaches require experts to hand-craft lexicons, grammars, and knowledge-base schemas, limiting convenient scaling across databases, vocabularies, and languages.The paper identifies this human intervention as a non-negligible constraint on generality.
- Contributions: The paper improves an earlier embedding model with efficient longer-path inference and answer representations encoding question-answer paths and surrounding knowledge-base subgraphs.The earlier model considered only answers directly connected to the question entity.
- Contributions: The approach is competitive on WebQuestions without lexicons, rules, or additional part-of-speech, syntactic, or dependency-parsing systems during training.This contrasts with the resources used by many other systems.
2 Task Definition
The task trains an open-QA system from question-answer pairs and a knowledge base whose structure links candidate answers. Training combines structured knowledge-base-derived questions with less rigid paraphrase data, and evaluation uses WebQuestions built on Freebase.
- Task definition: The system assumes candidate answers are knowledge-base entities and questions contain one identified knowledge-base entity.When the entity is absent, the paper uses plain string matching for entity resolution.
- Evaluation data: WebQuestions contains 5,810 Freebase question-answer pairs, split into 3,778 training and 2,032 testing examples, with 1,000 training questions held out for validation.Answers are Freebase entities, and ambiguous entity matches are resolved by retaining the most popular entity.
- Knowledge base: Freebase represents facts as subject–relation–object triples, and the paper’s subset contains 14M triples, 2.2M entities, and 7k relation types.Triples are converted into automatically generated questions whose answers are the object entities.
- Training data: ClueWeb extractions add 2M text-linked triples to diversify training beyond Freebase’s fixed lexicon and vocabulary.The extracted subject and object entities are linked to Freebase and converted into questions using simple patterns and types.
- Training data: Paraphrase pairs supplement generated questions because their semi-automatic wording and rigid syntax do not adequately model natural language.WikiAnswers supplies an indirect supervision signal based on question paraphrases.
- Scope: The Freebase restriction does not simplify the task because WebQuestions itself contains approximately 2k entities.The paper explicitly notes this scope boundary for its knowledge-base subset.
3 Embedding Questions and Answers
The model scores questions against candidate answers by embedding both the question and a structured answer representation, then learning these representations with ranking and paraphrase objectives. Inference expands candidate paths beyond direct connections while balancing coverage and ranking efficiency.
- Embedding Questions and Answers: The model learns low-dimensional embeddings for question words and Freebase entities and relation types, scoring candidate answers through a learned function S(q, a).Question and answer representations are composed from embeddings of their individual words and symbols.
- Representing Candidate Answers: Candidate answers can be represented as a single entity, a 1- or 2-hop path, or a path plus the surrounding connected-entity subgraph.The subgraph representation distinguishes path entities from surrounding entities using separate embedding representations.
- Representing Candidate Answers: The subgraph representation adds connected entities and relation types around the answer, encoding more information than the entity or path alone.The authors motivate this richer representation because a single entity embedding may not feasibly encode all relevant information, while a path omits other answer-entity information.
- Training and Loss Function: Training minimizes a margin-based ranking loss so correct answers score above sampled incorrect candidates by at least margin m, fixed to 0.1.Incorrect candidates are sampled from connected entities or by replacing the answer with a random entity; optimization uses stochastic gradient descent with Hogwild! and unit-ball constraints.
- Training and Loss Function: Paraphrase prediction and Freebase-name mapping are multitask objectives that encourage similar embeddings for paraphrased questions and entity identifiers with their names.Paraphrase training alternates with question-answer scoring and reuses the same embedding matrix.
- Inference: At test time, the model ranks a question-specific candidate set, using direct triples or beam-search-selected 2-hop paths to avoid scoring the entire knowledge base.C1 uses direct connections, while default C2 keeps 2-hop candidates whose relations appear among the top 10 ranked relation types and weights 1-hop scores by 1.5.
4 Experiments
On WebQuestions, the system outperforms several published baselines, matches one comparable method, and improves further when combined with that method. Ablations show that both candidate-answer selection and richer subgraph representations materially affect performance.
- The approach outperforms,, and [5] on WebQuestions, while performing similarly to.
- Replacing C2 with directly connected candidates causes a large performance drop, whereas using all 2-hop candidates is detrimental because ranking becomes slower and more confused.
- The Subgraph answer representation improves over the Path representation, which corresponds to the earlier model’s setting with C1 candidates.
5 Conclusion
The paper presents an embedding model for open-domain question answering trained from question-answer pairs and a knowledge base structure. It achieves promising performance on the competitive WebQuestions benchmark.
- The model learns to perform open-domain question answering from questions paired with answers and a knowledge base linking those answers.
- Its training data consists of question-answer pairs, while the knowledge base supplies structure among candidate answers.
- The system achieves promising performance on the competitive WebQuestions benchmark.