Source-linked AI summary
Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks
Matthew Francis-Landau, Greg Durrett, Dan Klein
TL;DR
Entity linking must resolve mentions whose referents vary with context, while earlier context heuristics are coarse and difficult to calibrate. The paper uses multi-granularity CNNs to compare source context with candidate entities and combines those features with sparse signals, achieving strong performance across several datasets. The authors also note interpretability and comparison constraints in their evaluation.
Problem
Entity linking must resolve contextually polysemous mentions, while heuristic context methods are difficult to calibrate and capture structure coarsely.
Method
The model uses CNNs at multiple text granularities to compare source-document and candidate-entity topic vectors, then combines these features with sparse linking signals.
Results
The combined CNN and sparse-feature system achieves state-of-the-art performance across several datasets and outperforms prior systems.
Takeaways & Limitations
CNN-derived semantic information and sparse indicator features provide complementary information for entity linking.
Takeaways & Limitations
The system is less interpretable than its purely indicator-based variant, and one baseline comparison could not be reproduced without the original Wikipedia dump.
Abstract
from arXiv · showhide
A key challenge in entity linking is making effective use of contextual information to disambiguate mentions that might refer to different entities in different contexts. We present a model that uses convolutional neural networks to capture semantic correspondence between a mention's context and a proposed target entity. These convolutional networks operate at multiple granularities to exploit various kinds of topic information, and their rich parameterization gives them the capacity to learn which n-grams characterize different topics. We combine these networks with a sparse linear model to achieve state-of-the-art performance on multiple entity linking datasets, outperforming the prior systems of Durrett and Klein (2014) and Nguyen et al. (2014).
1 Introduction
The paper addresses contextually polysemous mentions in entity linking, where the same mention can refer to different entities. It replaces coarse heuristic use of context with learned semantic modeling and combines CNNs with an existing linker.
- 1 Introduction: Contextually polysemous mentions can refer to different entities depending on the surrounding document.The example “Germany” may denote a nation, its government, or a soccer team.
- 1 Introduction: Earlier approaches often used nearby linked mentions and topical similarity to help resolve ambiguous mentions.These approaches use collective entity linking signals from neighboring mentions.
- 1 Introduction: Document-only approaches commonly relied on tf-idf heuristics, which are difficult to calibrate and capture structure more coarsely than learning-based methods.
- 1 Introduction: The paper models semantic similarity between source-document context and candidate entities with convolutional neural networks.It uses multiple input granularities to capture different notions of semantic context.
- 1 Introduction: Combining CNN features with a preexisting entity linking system achieves state-of-the-art performance across several datasets.
2 Model
The model represents source-document and candidate-entity text at multiple granularities, compares their learned topic vectors, and combines the resulting semantic features with sparse linking signals. A latent query variable further supplies candidate entities through mention-subset generation.
- 2 Model: The model uses CNNs to distill topic semantics from document and entity text at multiple granularities.Source inputs include the mention, its immediate context, and the full document; target inputs include the title and article text.
- 2 Model: The log-linear model assigns distributions over target entities using CNN-derived features for a mention and its containing document.The CNN feature vector is parameterized by learned filters.
- 2 Model: Each CNN feature is a cosine similarity between a source-document topic vector and a candidate-entity topic vector.Distinct CNNs operate over different subsets of relevant text.
- 2 Model: The system compares all source–target topic-vector pairs and combines their semantic features with sparse features in a final logistic regression layer.The CNN filters are learned end to end through backpropagation.
- 2.2 Integrating with a Sparse Model: Sparse features encode query and entity signals including link counts, lexical matches, and tf-idf similarities, complementing the CNN features.The complete model is trained by maximizing labeled-corpus log likelihood.
- 2.2 Integrating with a Sparse Model: The sparse integration introduces a latent query variable representing which mention subset is resolved and uses generated queries to define candidate entities.Query generation may remove stop words, suffixes, punctuation, or leading and trailing words.
3 Experimental Results
Experiments across four entity linking datasets compare the full sparse-plus-CNN system with prior systems and ablations. Results show complementary benefits from CNN topic features, multiple context granularities, and learned representations, while dataset access and interpretability impose limits.
- Datasets: The experiments cover four datasets: ACE, CoNLL-YAGO, WP, and Wikipedia.The Wikipedia dataset contains 10,000 sampled articles, while CoNLL-YAGO includes rarer entities.
- Overall results: The full system outperforms Durrett and Klein (2014) and Nguyen et al. (2014), while CNN features generally outperform sparse features alone.Combining CNN and sparse features produces further gains because they capture complementary information.
- Complementary signals: CNN features capture nuanced document context, whereas sparse features capture target-link frequency and lexical query information.The full model can therefore select less common targets when topic indicators support them.
- Multiple granularities: Neither document-and-article context nor mention-and-title context matches the performance of the complete six-feature CNN set.The two restricted systems represent coarse and fine context extremes, whose combination is less effective than using all granularities.
- Embedding sources: Wikipedia-trained word vectors outperform Google News vectors because the latter have much higher out-of-vocabulary rates.The comparison uses the full sparse-and-convolutional model on development sets.
- Learned convolutions: The learned convolutional filters often identify topic-specific n-grams, but some filters remain difficult to interpret.The authors suggest such filters may have low impact, reflect moderate rather than maximal activations, or combine topics that do not cooccur.
4 Conclusion
The model uses multi-granularity convolutional networks to compare source-document and candidate-entity semantics, achieving strong standalone performance and improving a sparse-feature entity linker.
- Multi-granularity convolutional features provide strong entity-linking performance independently.The approach evaluates compatibility between a mention’s context and several potential link targets.
- These semantic-similarity features improve a pre-existing system based on sparse indicator features.The results show that CNN-derived information and sparse indicators are complementary.