Source-linked AI summary
Simple Question Answering by Attentive Convolutional Neural Network
Wenpeng Yin, Mo Yu, Bing Xiang, Bowen Zhou, Hinrich Schütze
TL;DR
SimpleQA asks systems to answer single-relation Freebase questions by identifying the relevant entity and fact. The paper combines simple entity linking with character- and word-level CNN matching, including attentive maxpooling for predicate patterns. Its system outperforms prior systems and achieves optimal performance in the reported FB2M and FB5M experiments.
Problem
SimpleQA remains difficult because many paraphrases express the same single-relation question, while entity linking can have candidate-collection limitations when entities share names.
Method
The system ranks entity candidates with simple coverage and position factors, then uses char-CNN for entity–mention matching and attentive word-CNN for predicate–pattern matching.
Results
The system outperforms the state-of-the-art system, while active-entity-linker AMPCNN has optimal performance on FB2M and FB5M.
Takeaways & Limitations
Simple entity linking and attentive convolution provide an effective fact-selection approach for SimpleQA and also show outstanding performance on relation classification.
Takeaways & Limitations
Surface-form entity linking is limited when distinct entities have the same names, constraining candidate collection.
Abstract
from arXiv · showhide
This work focuses on answering single-relation factoid questions over Freebase. Each question can acquire the answer from a single fact of form (subject, predicate, object) in Freebase. This task, simple question answering (SimpleQA), can be addressed via a two-step pipeline: entity linking and fact selection. In fact selection, we match the subject entity in a fact candidate with the entity mention in the question by a character-level convolutional neural network (char-CNN), and match the predicate in that fact with the question by a word-level CNN (word-CNN). This work makes two main contributions. (i) A simple and effective entity linker over Freebase is proposed. Our entity linker outperforms the state-of-the-art entity linker over SimpleQA task. (ii) A novel attentive maxpooling is stacked over word-CNN, so that the predicate representation can be matched with the predicate-focused question representation more effectively. Experiments show that our system sets new state-of-the-art in this task.
1 Introduction
The paper frames SimpleQA as selecting a single Freebase fact whose subject and predicate match the question. It proposes lightweight entity linking and CNN-based matching for entity mentions and predicate-focused question patterns.
- Task motivation: SimpleQA answers single-relation questions by selecting a Freebase fact matching the question’s entity and topic.The target fact has the form (subject, predicate, object).
- Entity linking: Entity linking reduces Freebase’s overwhelming fact inventory to a small candidate subset.The proposed linker searches entity names and ranks candidates using coverage and mention-position factors.
- Fact selection: Fact selection matches each candidate fact’s subject with the question mention and its predicate with the question pattern.The system treats surface-form matching and predicate-pattern matching as distinct signals.
- Neural architecture: A character-level CNN handles entity–mention surface forms, while a word-level CNN with attentive maxpooling handles predicate–pattern matching.Character-level representations are intended to tolerate typos, spaces, and other character violations; attentive pooling emphasizes predicate-like phrases.
- Overall result: The simple architecture outperforms a more complicated state-of-the-art system on the task.The full pipeline uses top-N entity candidates, then applies char-CNN and word-CNN matching to select a fact.
2 Related Work
Prior SimpleQA research used retrieval, relation extraction, distributed representations, semantic parsing, and CNN matching. This work positions its approach as CNN-based fact selection with distinct surface-form and predicate-pattern modeling.
- Research landscape: Factoid QA research distinguishes single-relation questions from multi-relation questions commonly addressed with semantic parsing.Multi-relation systems often map questions to formal meaning representations.
- Limitations: Surface-form entity linking can fail during candidate collection when distinct entities share the same name.Matching entity descriptions with another word-CNN did not improve results in the reported experiment.
- Candidate retrieval: Earlier approaches retrieve candidate answers from knowledge bases using relation extraction or distributed representations.These methods form alternative routes to candidate generation and answer selection.
- CNN approaches: Prior CNN work matched entity mentions and predicate patterns, but used the same word-hashing-based CNN architecture for both.The present work instead separates character-level entity matching from word-level predicate-pattern matching.
- Benchmarks: The SimpleQuestions benchmark framed SimpleQA as a single lookup in memory and was also studied with embedding-based Memory Networks.Training could combine SimpleQuestions with WebQuestions and Reverb datasets.
3 Task Definition and Data Introduction
SimpleQuestions contains single-relation questions paired with ground-truth Freebase facts, where the object is the default answer. Evaluation requires recovering both the correct subject and predicate.
- Freebase: Freebase represents entities connected by directional predefined predicates, with each triple (h, p, t) describing a fact.For example, a route can be connected to a city through a major-cities predicate.
- Benchmark: The SimpleQuestions benchmark provides single-relation questions, each paired with a ground-truth fact whose object is the default answer.The benchmark is split into train, dev, and test sets.
- Data scale: SimpleQuestions includes FB2M and FB5M Freebase subsets with millions of entities, thousands of predicates, and millions of atomic facts.FB2M contains 2,150,604 entities and FB5M contains 4,904,397 entities.
- Task formulation: The task assumes that a question can be answered by querying Freebase with a single subject and predicate.Only the tuple (h, p) is used for matching, and accuracy requires both subject and predicate to match ground truth.
4 Entity Linking
The entity-linking stage identifies candidate Freebase entities and the question span referring to the entity. It supports passive and active procedures, with LCCS-based scoring and mention construction.
- Entity linking goals: Entity linking must identify candidate Freebase entities and the question span that refers to the entity.The order of these operations yields passive and active entity linkers.
- Passive Entity Linker: The passive linker retrieves entities containing question words, computes word-level LCCS overlap, and retains top-N ranked candidates.Its score combines question coverage, entity-name coverage, and mention position through factors a, b, and c.
- Passive Entity Linker: The passive linker prefers candidates covering more question or entity words and candidates whose matching span occurs near the question’s end.The authors report that it outperforms other state-of-the-art SimpleQuestions entity linkers and runs unsupervised and fast.
- Mention detection: Mention detection can expand word-level LCCS matches, fall back to character-level LCCS, and create a pattern by replacing the mention with <e>.Passive mention detection can produce multiple mention–pattern pairs, whereas active detection produces one.
- Active Entity Linker: The active linker detects the entity mention first, then searches for entity candidates using the detected span.It produces one mention–pattern pair per question and requires entity-labeled training questions.
5 Fact Selection
Fact selection matches each question with candidate Freebase facts using character-level entity matching and predicate-focused word-level matching. Attentive maxpooling reweights question subsequences according to their similarity to the predicate before extracting the pattern representation.
- Candidate generation: The entity linker supplies top-N entity candidates, whose subject-linked facts form the fact-search pool.Fact selection then ranks these candidate facts against the question.
- Two-part matching: The system uses char-CNN for entity–mention surface-form matching and word-AMPCNN for predicate–pattern matching.The two CNNs model complementary aspects of the question–fact match.
- CNN framework: The CNN framework represents each input as embedded entries, applies sliding n-gram convolutions, and uses shared weights for the two text inputs.Entries may be characters or words, depending on the model.
- Traditional maxpooling: Traditional maxpooling takes dimension-wise maxima across n-gram representations without using predicate context.This produces the same pooled representation for different predicate representations.
- Attentive maxpooling: Attentive maxpooling computes predicate–pattern similarities, reweights pattern features, and retrieves maxima from the original feature map.Higher-similarity subsequences therefore receive greater influence in the final pattern representation.
- Attentive maxpooling: AMP is designed to extract n-gram features related to the predicate, unlike TMP’s context-free feature selection.The predicate guides which question subsequences are emphasized during pooling.
6 Experiments
Experiments evaluate entity linking, SimpleQuestions fact selection, and attentive maxpooling through comparisons with ablations and established baselines. The system achieves strong results while using a comparatively simple CNN-based architecture.
- 6.1 Training Setup: Training uses fact pools from top-N linked entities, with 99 negative facts sampled per ground-truth fact during training and all candidates retained for development and testing.The model uses Adagrad, L2 regularization, diversity regularization, and development-set hyperparameter tuning.
- 6.2 Entity Linking: The entity linker outperforms the baseline by large margins and exceeds its reranked scores by 2–3 percent.All three ranking factors matter, with factor a dominant at small N and factor c contributing less as N increases.
- 6.2 Entity Linking: The active entity linker performs significantly better than both the baseline linker and the passive linker.The experiments compare passive and active linkers and release the resulting entity-linking outputs.
- 6.3 SimpleQuestions: AMPCNN with the active entity linker achieves optimal performance on both FB2M and FB5M.FB5M performance is slightly lower, mainly because entity-linking coverage is about 2% below FB2M; removing attention still yields competitive performance.
- 6.4 Effect of Attentive Maxpooling: AMPCNN outperforms the best relation-classification baseline, APCNN, by 0.8%.AMPCNN also uses fewer parameters and runs faster than APCNN.
7 Conclusion
The paper concludes that CNNs support SimpleQA through improved entity linking and attentive predicate–question matching. Its model performs strongly on both SimpleQA and relation classification.
- 7 Conclusion: The work contributes a simple entity linker that provides higher ground-truth entity coverage.It also contributes attentive maxpooling above the convolution layer to model predicate–question-pattern relationships more effectively.
- 7 Conclusion: The model shows outstanding performance on both SimpleQA and relation classification.The conclusion summarizes the system-level outcome of the proposed CNN approach.