Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention

1 Introduction

• Supervised relation extraction needs extensive training data that is expensive to obtain, limiting its use for Web-scale tasks.Relation extraction classifies the semantic relationship between two entities in a sentence.Source: §1 · p. 1
• Distant supervision labels sentences mentioning an entity pair as positive examples, but missing knowledge-base facts can make this assumption noisy.The task is framed as multi-instance multi-label learning because entity pairs may have multiple sentences and relation labels.Source: §1 · p. 1
• In Riedel2010, 50% of sentences had 40 or more words, motivating attention to focus on words that express the relation.The paper notes that not all words in these long sentences contribute to expressing the given relation.Source: §1 · p. 1
• Noisy test labels, including false negatives when knowledge-base facts are missing, impede model comparison; GDS is introduced to address this evaluation problem.The authors state that GDS makes automatic evaluation in this setting more reliable.Source: §1 · p. 1
• The paper proposes BGWA, an entity-centric attention model (EA), and a weighted ensemble combining new and existing relation-extraction models.The introduction describes BGWA as BiGRU-based and presents the ensemble as a way to combine complementary models.Source: §1 · p. 2

2.1 Background

• PCNN splits convolution outputs into the left, between-entity, and right regions, max-pools each, and concatenates them into a fixed-size representation.Source: §2.1 · p. 2

2.2 Bi-GRU based Word Attention Model (BGWA)

• BGWA encodes both directions with a Bi-GRU, learns relation-conditioned attention over words, and piecewise max-pools the attention-weighted representations.Source: §2.2 · pp. 2–3

2.3 Entity Attention (EA) Model

• EA pairs entity-specific attention over word/entity embeddings with PCNN features, adapting to bags by selecting the highest-probability sentence and using PCNN rather than CNN.Source: §2.3 · p. 3

2.4 Bring it all together: Ensemble Model

• On sampled GDS sets, attention models contribute when PCNN is wrong; development-set weighted voting combines them and outperformed a jointly learned neural ensemble.Source: Figure 5 · p. 4, §2.4 · p. 4

• Earlier distant-supervision datasets can contain incorrect labels from missing knowledge-base facts and instance sets with no sentence supporting the assigned relation, complicating evaluation.Source: §3 · p. 4
• GDS guarantees that each labelled relation is correct and each instance set contains at least one sentence expressing its assigned relation.Source: §3 · p. 4
• GDS is built from a human-judged Google relation-extraction corpus by retrieving documents and adding entity-pair snippets of at most 500 words to instance sets.The corpus contains five binary relations; each new set combines the original sentence with retrieved snippets.Source: §3 · p. 4
• The GDS split assigns 60% of data to training, 10% to development, and 30% to testing, with no entity-pair overlap between splits.The development set supports principled model selection.Source: §3 · p. 4
• Although smaller than Riedel2010, GDS avoids manual validation of the top 1,000 confident predictions and makes automated evaluation possible.Source: §3 · p. 4

4 Experiments and Results

• The experiments use Riedel2010-b, formed by splitting the Riedel2010 training set 80/20 for training and development, and the GDS dataset.Models are evaluated on test sets with held-out precision-recall curves and compared with PCNN and NRE.Source: §4 · p. 5

4.1 Results

• BGWA performs well on Riedel2010-b, while EA performs better on GDS, where the datasets also differ in relation count and test-label noise.Riedel2010-b has 53 relations; GDS has five and no test-label noise. The paper describes the models' errors as uncorrelated and complementary.Source: §4.1 · p. 5
• The weighted ENSEMBLE is consistently strong across recall ranges on both datasets and gains over 2–3% precision on Riedel2010-b.The comparison appears in the precision-recall curves for Riedel2010-b and GDS.Source: §4.1 · pp. 5–6, Figure 7 · p. 6, Figure 8 · p. 6
• For the “location in” example, entity attention focuses on place-indicating words and the entity “maryland,” while word attention uses Bi-GRU hidden representations.The paper notes that entity attention is relation-specific and that a Bi-GRU representation at a time point summarizes correlated sequence timepoints.Source: §4.1 · p. 6

5 Related Work

• Riedel et al. introduced an expressed-at-least-once assumption in a factor graph to aggregate mention-level predictions, followed by related MIML work.The paper frames distant-supervision relation extraction as multi-instance, multi-label learning.Source: §5 · p. 6
• CNN-based relation extraction was followed by PCNN and instance attention for selecting sentences.PCNN uses piecewise max-pooling; the cited account says it uses one sentence per instance set for prediction and backpropagation.Source: §5 · p. 6
• Other approaches leverage inter-sentence information for entity pairs, relation connections, or background knowledge and consistency signals [Weston et al., 2013; Nagarajan et al., 2017].The cited background-knowledge approaches include joint text-KB embeddings, Markov logic networks, and inter-instance-set couplings.Source: §5 · p. 6
• Attention mechanisms have been applied to distant-supervision relation extraction, and this work focuses on selecting relevant words using word- and entity-based attention.Source: §5 · p. 6

6 Conclusion

• The proposed models are BGWA, EA, and a weighted voting ensemble that combines complementary models for relation extraction.Source: §6 · pp. 6–7
• GDS is introduced as a dataset that removes test-data noise present in previous distant-supervision benchmarks, supporting credible automatic evaluation.Source: §6 · p. 7
• Combining the proposed methods with attention-based sentence selection is left for future work, and the authors plan to release code and datasets.Source: §6 · p. 7