Source-linked AI summary
End-to-End Neural Entity Linking
Nikolaos Kolitsas, Octavian-Eugen Ganea, Thomas Hofmann
TL;DR
Entity linking traditionally separates mention detection from entity disambiguation, despite their mutual dependency. The paper introduces a neural end-to-end model that scores entity candidates for all possible spans and jointly makes both decisions. It reports strong Gerbil results when training and test characteristics align, while cross-convention settings favor coupling the disambiguation model with a traditional NER system.
Problem
Entity linking methods commonly separate mention detection and entity disambiguation, ignoring dependencies that can cause detection errors to propagate into linking.
Method
The model considers all spans with entity candidates, learns context-aware mention–entity compatibility scores using neural embeddings, and jointly optimizes mention detection and disambiguation.
Results
The system significantly outperforms popular Gerbil systems when training and test data share characteristics, while cross-convention testing is strongest or near-strongest with a traditional NER system.
Takeaways & Limitations
Joint neural optimization can deliver state-of-the-art entity-linking performance and neural embeddings can almost completely replace engineered features.
Abstract
from arXiv · showhide
Entity Linking (EL) is an essential task for semantic text understanding and information extraction. Popular methods separately address the Mention Detection (MD) and Entity Disambiguation (ED) stages of EL, without leveraging their mutual dependency. We here propose the first neural end-to-end EL system that jointly discovers and links entities in a text document. The main idea is to consider all possible spans as potential mentions and learn contextual similarity scores over their entity candidates that are useful for both MD and ED decisions. Key components are context-aware mention embeddings, entity embeddings and a probabilistic mention - entity map, without demanding other engineered features. Empirically, we show that our end-to-end method significantly outperforms popular systems on the Gerbil platform when enough training data is available. Conversely, if testing datasets follow different annotation conventions compared to the training set (e.g. queries/ tweets vs news documents), our ED model coupled with a traditional NER system offers the best or second best EL accuracy.
1 Introduction and Motivation
Entity linking must detect ambiguous entity mentions and connect them to knowledge-base entities, but conventional systems separate mention detection from disambiguation. This paper proposes a neural end-to-end model that jointly makes both decisions by scoring candidate entities for all possible spans.
- Entity linking extracts potentially ambiguous entity mentions and links them to corresponding knowledge-base entities for text understanding and information extraction.
- Separate mention detection and entity disambiguation pipelines ignore their dependency, allowing errors from mention detection to propagate into linking.
- Joint modeling can improve disambiguation through richer mention context and improve mention boundaries through knowledge of the entities present.
- The model generates all candidate-bearing spans, computes context-aware mention–entity compatibility scores, and jointly optimizes entity linking and mention decisions.It uses word, entity, and mention embeddings with neural attention and global voting, without requiring manually annotated negative non-linkable mentions.
- Neural embeddings almost completely replace engineered features for the joint mention detection and entity disambiguation task.
- On Gerbil, the method gains significantly when training and testing share domain characteristics; under different annotation schemes, it remains effective when coupled with a traditional NER system.
2 Related Work
Related entity-linking systems generally treat mention detection and entity disambiguation separately, while joint approaches remain limited by weak task coupling, engineered features, or computational complexity.
- Most entity-linking systems address mention detection or named-entity recognition and entity disambiguation as separate subtasks.
- Traditional NER commonly uses sequence labeling with engineered features or bi-LSTMs that learn lexical and syntactic features.
- Neural entity-disambiguation methods rely on context-aware word, span, and entity embeddings with neural similarity functions.
- Prior joint approaches include NER over-generation, semi-CRFs, and parse-tree graphical models, but use engineered features or weakly capture mutual task dependency.
3 Neural Joint Mention Detection and Entity Disambiguation
The model jointly performs mention detection and entity disambiguation by scoring candidate entities for all potentially linkable spans. It combines contextual mention and entity representations with local and global coherence scores during training and inference.
- Task formulation: EL maps document spans to knowledge-base entities, while ED receives mentions and predicts their corresponding entities.EL may also operate on queries or tweets; ED assumes the mention list is given.
- Mention representation: Context-aware word embeddings support both mention boundary detection and entity disambiguation by incorporating local context through a bi-LSTM.Word-character embeddings are formed by combining character-level bidirectional-LSTM features with pretrained word vectors before contextual encoding.
- Mention representation: Each mention representation combines its boundary words and soft head, then projects them through a feedforward network to the entity-embedding dimension.The soft head uses attention over mention word embeddings but only marginally improves results because most mentions are at most two words long.
- Candidate scoring: For each span-candidate pair, the final local score combines embedding similarity with a probabilistic entity-mention prior, and optional attention adds long-range context.Candidate sets contain up to s entities selected using a probabilistic map derived from hyperlink and dictionary resources.
- Joint training: The model considers all potentially overlapping spans with at least one candidate and trains gold pairs to outrank incorrect candidates and invalid mentions.Without annotated non-linkable negatives, spans and candidates absent from the gold annotations are treated as non-linkable.
- Global disambiguation: A global layer filters high-scoring local pairs and promotes document-level coherence using cosine similarity among candidate entity embeddings.The combined global score supplies the final annotations, while validation selects the linking threshold that maximizes micro F1.
4 Experiments
Experiments evaluate the model on Gerbil datasets using InKB F1 for EL and ED, including strong and weak matching. The system leads baselines on AIDA, while cross-domain testing favors coupling its ED model with a state-of-the-art NER system.
- Datasets and metrics: Experiments use Wikipedia 2014 as the knowledge base and Gerbil for evaluation against popular and state-of-the-art EL and ED systems.The study reports micro and macro InKB F1 scores for EL and ED, with strong and weak matching for EL.
- Experimental settings: EL training uses all potentially linkable spans, whereas ED training uses only gold spans.The two settings share model hyper-parameters but differ in the span set used for training.
- Main results: Our system outperforms all baselines on end-to-end EL for both AIDA-A development and AIDA-B test datasets.The paper identifies these as the biggest publicly available EL datasets.
- Main results: Weak-matching EL results are comparable with strong-matching results, indicating effective mention-boundary detection.Weak matching accepts overlapping spans linked to the correct entities, while strong matching requires exact boundaries.
- Generalization: When test data differs in statistics or annotation conventions, the method works best with a state-of-the-art NER system.This pattern is reported for the ED global model running on Stanford NER detections.
- Limitations and analysis: ED-only results are slightly behind recent top-performing systems, while the unified architecture is not suited to expensive global ED strategies.The authors leave bridging this ED gap for future work.
- Ablation and error analysis: For rare entities, removing the p(e|m) prior improves results, but doing so is impractical because the gold entity is unknown at test time.The prior helps unambiguous mentions but introduces noise when gold entities are infrequent.
- Ablation and error analysis: Qualitative analysis found neither systematic over-generation nor under-generation of mentions, while many additional mentions matched real knowledge-base entities absent from gold annotations.The analysis was conducted on a sample document from AIDA.
5 Conclusion
The paper presents the first neural end-to-end entity-linking model and reports benefits from jointly optimizing entity recognition and linking. On Gerbil, it achieves state-of-the-art EL performance on AIDA/CoNLL and generalizes to differing datasets when combined with Stanford NER.
- Conclusion: The paper presents the first neural end-to-end entity-linking model and shows benefits from jointly optimizing entity recognition and linking.Its representation uses word, entity, and mention embeddings.
- Conclusion: The model largely replaces engineered features with modern neural networks built from word, entity, and mention embeddings.The authors report this as a central design outcome.
- Conclusion: The system achieves state-of-the-art EL performance on the AIDA/CoNLL dataset and generalizes to different datasets when combined with Stanford NER.The code is publicly available.