Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks

1 Introduction

• KBs support compositional reasoning but are highly incomplete, while text offers abundant facts in an unstructured form that lacks KB-style compositional reasoning.Combining the sources is challenging because their structures differ, and distant-supervision alignment does not capture language's many forms for expressing a fact.Source: §1 · p. 1
• Universal schema jointly embeds KB facts and textual facts in a uniform representation, allowing information to propagate between the sources.The introduction illustrates the complementarity with a question about Barack Obama as the first African-American president.Source: §1 · p. 2
• The model uses observed universal-schema cells as memory components in an end-to-end QA model trained on question-answer pairs.This adapts memory networks, which have an external memory component for encoding context.Source: §1 · p. 2
• 8.5 F1 points over the state-of-the-art semantic-parsing baseline on SPADES, with universal schema a better QA knowledge source than KB or text alone.The analysis also finds that the individual sources help fill each other's weaknesses.Source: §1 · p. 2

2 Background

• The task is to fill a question's blank with an answer entity using a knowledge base and text, where the question contains at least one entity.Source: §2 · p. 2
• Universal schema represents entity pairs as rows and KB relations or textual patterns as columns, with embeddings learned by low-rank matrix factorization.This model uses distributed sentence representations for textual relations, enabling reasoning over sentences with similar meanings but different surface forms.Source: §2 · p. 2
• Memory networks use external differentiable memory; prior variants store KB triples or sentences, while key-value MemNNs attend to keys and use values to predict answers.The cited prior key-value work stores either KB triples or sentences, not multiple distinct sources as this paper does.Source: §2 · pp. 2–3

3 Model

• Each KB fact becomes a key formed by concatenating subject and relation embeddings, with the object embedding as its value.Textual facts use a sentence key with the subject retained and the object position replaced by a blank token; the object is the value, giving KB and text facts a unified representation.Source: §3 · p. 3
• The question is encoded as a distributed representation, and attention scores memory cells by the dot product of each key with the current contextual vector.Only cells containing at least one entity in the question are considered.Source: §3 · p. 3
• Attention-weighted memory values update the context across iterations, enabling multi-hop reasoning without an explicit symbolic query representation.The initial context is the question embedding; the update uses a projection matrix and a weight matrix for prior context and current values.Source: §3 · p. 3
• The final context selects the answer from among the dataset's 1.8 million entities by highest inner product.Source: §3 · p. 3

4.4 Results and Discussions

• UNISCHEMA beats the evaluated alternatives on SPADES and exceeds the state of the art by 8.5 F1 points.The reported comparison places it ahead of the paper's other models and the semantic-parsing baseline.Source: Table 1 · p. 4, §4.4 · pp. 4–5

4.1 Evaluation Dataset

• SPADES supplies 93K web cloze sentences and 1.8M entities, but its Freebase-linked entity-pair filter skews evaluation toward Freebase; text facts come from training sentences.Source: §4.1 · p. 4

4.2 Models

• The baselines isolate KB-only and text-only memories, ENSEMBLE combines their scores and evidence, and UNISCHEMA stores both kinds of facts jointly.Source: §4.2 · p. 4

4.3 Implementation Details

• UNISCHEMA starts from trained ONLYKB parameters; minibatch normalization aligns textual-fact statistics to KB-memory statistics and stabilizes training.Source: §4.3 · p. 4

4.4 Results and Discussions

• UNISCHEMA answers KB failures involving absent Freebase relations, intentions, comparative predicates, and answer-type constraints.Source: §4.4 · pp. 4–5
• Attention favors KB facts in 58.7% of cases, text in 34.8% even when the fact is already in the KB, and splits evenly in 6.5%.Source: §4.4 · p. 5
• With fewer than 16 KB facts per entity, UNISCHEMA beats ONLYKB by a wide margin, while ONLYKB is very sensitive to coverage.The test-time experiment varies the number of randomly selected KB facts available for each entity.Source: Figure 2 · p. 5, §4.4 · p. 5

• Most cited work improves KB inference with text features or text inference with KB signals, rather than exploiting both jointly.Source: §5 · p. 5
• Gardner and Krishnamurthy (2017) generate open-vocabulary logical forms and rank answers by their likelihood in Freebase and text; this paper uses question-answer supervision without logical-form annotations.Source: §5 · p. 5
• Unlike methods using curated databases with OpenIE triples, this approach needs no explicit database query and retains complete sentential context in its text facts.Source: §5 · p. 5

6 Conclusions

• Universal schema is a more promising QA knowledge source than either KB or text alone.Source: §6 · p. 5
• When the KB contains the fact of interest, KB is preferred over text.Source: §6 · p. 5
• Although many queries still attend to text, the combination of text and KB is superior to KB alone.Source: §6 · pp. 5–6