PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Entity Linking: A Survey of Models Based on Deep Learning

Ozge Sevgili, Artem Shelmanov, Mikhail Arkhipov, Alexander Panchenko, Chris Biemann

arXiv:2006.00575v4cs.CL

TL;DR

Entity linking resolves ambiguous mentions and connects them to knowledge-graph information useful in information extraction and natural language understanding. This survey organizes neural EL systems around a generic architecture and common model directions, finding that neural methods generally outperform classical approaches, with deep-learning disambiguation improving AIDA accuracy by around 10 percentage points.

  • Problem

    Entity linking matters because it resolves ambiguous entity mentions and connects them to knowledge-graph information useful for information extraction and natural language understanding.

  • Method

    The survey analyzes neural EL systems through a generic architecture covering candidate generation, entity ranking, and mention/entity encoding, then groups modifications into four common directions.

  • Results

    Neural entity linking generally achieves higher accuracy than classical methods; on AIDA, deep-learning disambiguation improved the state of the art by around 10 percentage points in accuracy.

  • Takeaways & Limitations

    Global models outperform local ones, while joint mention detection and disambiguation trails disambiguation-only systems; entity linking also supports knowledge-enriched language models.

  • Takeaways & Limitations

    Global linking improves disambiguation accuracy but has combinatorial entity assignments, high inference time complexity, and NP-hard training and exact inference.

Abstract

from arXiv · show

This survey presents a comprehensive description of recent neural entity linking (EL) systems developed since 2015 as a result of the "deep learning revolution" in natural language processing. Its goal is to systemize design features of neural entity linking systems and compare their performance to the remarkable classic methods on common benchmarks. This work distills a generic architecture of a neural EL system and discusses its components, such as candidate generation, mention-context encoding, and entity ranking, summarizing prominent methods for each of them. The vast variety of modifications of this general architecture are grouped by several common themes: joint entity mention detection and disambiguation, models for global linking, domain-independent techniques including zero-shot and distant supervision methods, and cross-lingual approaches. Since many neural models take advantage of entity and mention/context embeddings to represent their meaning, this work also overviews prominent entity embedding techniques. Finally, the survey touches on applications of entity linking, focusing on the recently emerged use-case of enhancing deep pre-trained masked language models based on the Transformer architecture.

1. Introduction

Entity linking connects ambiguous textual mentions to knowledge-graph entities, supporting information extraction and language understanding. This survey systematizes neural EL models since 2015, their architectures, representations, applications, and evaluation.

  • Motivation: Entity linking resolves lexical ambiguity by matching textual mentions to contextually appropriate knowledge-graph records.The resulting links support semantic processing and applications including information extraction, biomedical text processing, semantic parsing, and question answering.
  • Motivation: Neural EL models emerged as a recent wave of systems that generally outperform classical machine-learning approaches.Neural models benefit from distributed semantic representations, whereas classical systems often rely on shallow architectures and hand-crafted features.
  • Contributions: The survey distills a generic neural EL architecture and systematizes candidate generation, mention-context encoding, entity ranking, and model variations.It also organizes methods through taxonomies and provides references to publicly available implementations.
  • Contributions: The survey reviews distributed entity representations that encode graph structure, definitions, or co-occurrence statistics to improve EL generalization.It also discusses entity linking as a way to introduce knowledge-graph information into deep pre-trained language models.
  • Scope and novelty: Its scope emphasizes neural EL literature published from 2015 onward and addresses gaps in earlier surveys, including zero-shot, cross-lingual, and language-model applications.The review uses a conceptual inclusion framework rather than a strict article-collection algorithm.

2. Task Description

Entity linking detects mentions in text and assigns them to entities or concepts in a knowledge graph. The task is commonly divided into mention detection and entity disambiguation, while large and incomplete graphs create central modeling challenges.

  • Task definition: Entity linking identifies textual mentions, resolves their ambiguity, and links them to corresponding knowledge-graph entries.For example, context distinguishes which person a common name such as Scott Young denotes.
  • Knowledge graphs: Knowledge graphs represent entities, relations, and facts as structured triples or typed multi-relational graphs.Typed adjacency matrices encode different predicates such as occupation or founded.
  • Task decomposition: Mention detection finds entity spans, whereas entity disambiguation assigns detected mentions to knowledge-graph entities.Most EL studies assume mentions are already supplied or detected by a named-entity recognition system.
  • Challenges: Large knowledge graphs make EL training highly unbalanced because some entities or mentions may have no training examples.Models therefore need to learn mappings despite sparse supervision across hundreds of thousands or millions of entities.
  • Challenges: Knowledge-graph incompleteness requires systems to identify mentions that cannot be linked to any known entity, often through a NIL assignment.Some approaches add a separate unlinkability function, while others include NIL among disambiguation outputs.

3. Neural Entity Linking

The survey organizes neural entity linking around a general architecture and then examines major modifications, including joint, global, domain-independent, and cross-lingual approaches.

  • Organization: The survey begins with a general neural EL architecture and extends it to joint detection-linking, global context, domain-independent, and cross-lingual models.This organization provides a common framework for comparing the diverse neural EL literature.

3.1. General Architecture

Neural EL commonly uses a ranking pipeline that detects mentions, generates candidate entities, encodes context and mentions, and scores candidate correspondences. Candidate generation combines surface forms, aliases, and prior probabilities, while contextual encoding increasingly relies on recurrent or self-attentive representations.

  • General pipeline: Neural EL is commonly formulated as ranking rather than large-scale multiclass classification, reducing the burden of treating every entity as a class.The pipeline first identifies mention boundaries, then produces candidate entities and scores their correspondence with the mention.
  • Candidate generation: Candidate generation supplies probable entities for each ambiguous mention before neural ranking selects the best correspondence.The candidate list can also support a later decision that a mention is unlinkable.
  • Candidate generation: Three common candidate-generation strategies use surface-form matching, alias expansion, or precomputed mention-entity priors.Systems often combine these strategies; alias resources can recover abbreviations and synonyms that direct surface matching misses.
  • Mention-context encoding: Context encoding constructs a dense contextualized mention vector, with recurrent networks and self-attention dominating recent approaches.Earlier systems used convolutional encoders or candidate-word attention, while newer systems frequently use pretrained BERT representations.

3.1.3. Entity Encoding

Neural EL systems encode entities using textual, graph-based, or neural representations designed to capture semantic relatedness. Ranking then compares mention and candidate representations, often using dot or cosine similarity, to select the contextually appropriate entity.

  • Embedding behavior: Semantic entity embeddings place related entities near one another, illustrated by football-related neighbors for one Scott Young sense and writer-related neighbors for another.The survey visualizes these spaces with PCA reduced to two dimensions.
  • Entity encoding approaches: Entity encoders use three broad strategies: text-based embeddings, knowledge-graph relations, or neural encoders over descriptions and other entity information.These approaches may also align words, mentions, and entities in a shared vector space.
  • Text-based embeddings: Text-based methods learn entity embeddings from entity–word co-occurrences, hyperlink contexts, or distant supervision that jointly represents words and entities.Ganea and Hofmann use a max-margin objective with negative sampling; Newman-Griffis et al. expand word2vec using terminology-based distant supervision.
  • Knowledge-graph embeddings: Graph-based entity embeddings include DeepWalk, which preserves vertex proximity through random walks and skip-gram training, and TransE, which embeds entities and relations compatibly with graph facts.DeepWalk learns from generated walks, whereas TransE models head–relation–tail correspondences.
  • Neural entity encoders: Other encoders represent entities from descriptions, titles, categories, or types using neural networks such as convolutional, feedforward, tensor, recurrent, and BERT-based models.Masked entity prediction can train entity representations alongside standard word representations.
  • Entity ranking: Entity ranking compares a mention representation with each candidate representation using a similarity score, then usually converts candidate scores into a softmax probability distribution.Systems may combine these scores with mention–entity priors and features such as string matching or entity types.

3.1.5. Unlinkable Mention Prediction

Entity linking must handle mentions whose referents are absent from the knowledge graph. The survey describes NIL prediction as a reject-option classification problem with several decision strategies.

  • NIL prediction: NIL prediction handles mentions whose referent is absent from the knowledge graph, such as an unrepresented Scott Young cricket-player sense.The system must predict that the mention has no suitable linked entity in the graph.
  • Decision strategies: NIL prediction is a classification task with a reject option implemented through candidate absence, score thresholds, a special NIL entity, or a post-ranking classifier.The post-ranking classifier can use linking scores, mention detection signals, and other features.

3.2. Modifications of the General Architecture

The survey groups improvements to the generic neural EL architecture into notable modifications, including joint mention detection and disambiguation. Joint systems increase task difficulty but may benefit from interactions between the two stages.

  • Architecture modifications: The survey categorizes notable modifications and improvements to the general neural EL architecture.These modifications are summarized in a dedicated taxonomy.
  • Joint linking: Joint entity linking systems detect mentions and disambiguate their entities simultaneously rather than separating the two stages.The joint formulation predicts mentions and entities for all mentions in a context.
  • Joint linking: Although joint detection and disambiguation make EL more challenging, interactions between the stages can improve the overall pipeline because the stages are mutually dependent.The passage presents this as a potential benefit of solving the tasks together.

3.2 - Modifications of the General Architecture

The survey organizes neural EL modifications around joint detection and linking, global coherence, domain-independent methods, and cross-lingual architectures. These variations extend the general pipeline through alternative candidate generation, entity representations, contextual modeling, and inference strategies.

  • 3.2.1. Joint Entity Mention Detection and Linking: Joint models combine mention detection and entity linking, using span enumeration, multitask networks, sequence labeling, masked-language-model objectives, or constrained generation.Some approaches jointly optimize detection, ranking, and NIL prediction, while others omit explicit candidate generation or generate markup linking mentions to knowledge-graph entities.
  • 3.2.2. Global Context Architectures: Global entity disambiguation resolves multiple mentions interdependently by incorporating entity coherence and broader context.Consistency scores model relationships among candidate entities, such as mutually compatible national-team and footballer interpretations.
  • 3.2.2. Global Context Architectures: Global methods improve disambiguation with document-level coherence but face combinatorial assignments, NP-hard training and inference, and potentially misleading coherent errors.Approximate message passing and beam search are proposed to reduce complexity or correct uncertain local decisions.
  • 3.2.3. Domain-Independent Architectures: Embedding-based domain-independent methods represent mentions and entity descriptions in shared spaces for candidate retrieval or ranking.Simple embedding similarity can support one- or two-stage retrieval, while BERT bi-encoders and cross-encoders trade retrieval efficiency against richer joint context-entity encoding.
  • 3.2.4. Cross-lingual Architectures: Cross-lingual systems use multilingual representations and transfer learning to link entities across languages, including massively multilingual retrieval over almost 20 million candidates.Reported approaches benefit from mention-entity priors, resource-rich source languages, and multilingual training, with the largest improvements in zero-shot and few-shot settings.

3.3. Methods that do not Fit the General Architecture

Some entity-linking methods do not follow the survey’s general architecture because they use entity typing as an intermediate mechanism before final disambiguation. These approaches exploit typed candidate filtering or distant supervision from Wikipedia resources.

  • Entity Typing: Entity typing can replace direct disambiguation as an intermediate step that narrows candidates for a final linking model.Raiman and Raiman train a type system and an intermediate mention-type classifier to refine the candidate set.
  • Entity Typing: Distant supervision from Wikipedia pages and categories trains fine-grained entity typing whose soft predictions and candidate types support final disambiguation.Onoe and Durrett use typing information derived from Wikipedia at test time.

3.4. Summary

The survey organizes neural entity linking models around shared architectural features and highlights major design trends across encoders, candidate generation, supervision, and linking scope.

  • Encoder types: Mention encoders have shifted toward self-attention and deep pre-trained models such as BERT.The survey also lists ELMo, fastText, word2vec-based, convolutional, recurrent, feedforward, tensor-network, and attention-based encoders.
  • Candidate generation: Most studies still rely on external knowledge for candidate generation, using methods such as surface matching, aliases, priors, type classifiers, BM25, nearest neighbors, Google Search, or character-level models.The survey distinguishes explicit candidate-generation strategies from models without an explicit candidate-generation step.
  • Learning settings: Zero-shot domain adaptation has become more common, while zero-shot joint entity mention detection and linking remains unaddressed.The survey separately records supervised, unsupervised, weakly supervised, and zero-shot learning types.
  • Linking scope and comparison: Several studies report that cross-encoders outperform models with separate mention and entity encoders, while global context is widely used and local-only linking receives less recent attention.The table also tracks cross-lingual systems and distinguishes global solutions from joint mention-detection and disambiguation models.
  • Architectural organization: Neural entity linking models are compared by architectural features including encoder type, candidate generation, global inference, joint mention detection and disambiguation, NIL prediction, and entity embeddings.Table 2 provides the survey’s feature-based comparison framework.
  • Documentation and categorization: The survey documents model design features and publicly available implementations in comparative tables, with footnotes clarifying categorization and feature annotations.Some systems use entity description pages but are categorized as unstructured-text methods because their training follows word2vec principles.

4. Evaluation

The survey evaluates neural entity linking across standard monolingual and cross-lingual benchmarks, separating entity disambiguation from joint mention detection and disambiguation. Neural models substantially outperform classical baselines, with global models and Transformer-based encoders generally leading.

  • Evaluation settings: The survey separates entity disambiguation with provided mention boundaries from joint mention detection and disambiguation evaluated from plain text.Joint-model scores accumulate mention-detection errors, so the settings are reported separately.
  • Evaluation settings: Evaluation covers English benchmarks including AIDA, TAC KBP 2010, MSNBC, AQUAINT, ACE2004, CWEB, and WW, plus Spanish and Chinese TAC KBP datasets.AIDA-B is used for the AIDA test-set results.
  • Entity disambiguation results: Deep neural models improve AIDA entity-disambiguation accuracy by around 10 percentage points over the classical state of the art.The comparison concerns models evaluated with mention boundaries provided.
  • Entity disambiguation results: The best ED model on AIDA is Yamada et al., whose results are top or near top across most datasets; no non-neural baseline achieves the best result on any dataset.This indicates broad robustness across the reported benchmarks rather than a single-dataset advantage.
  • Local-global models: Global models generally outperform local models, and Yamada et al.'s global model attributes its gains to masked entity prediction and multi-step global disambiguation.The masked objective fine-tunes pretrained BERT for contextualized entity embeddings.
  • Mention-context encoders: Transformer-based mention-context encoders now lead the reported ED results, while RNNs remain popular and self-attention encoders have gained popularity.Yamada et al.'s top-performing model uses Transformers; Shahbazi et al.'s next model uses an ELMo bidirectional-LSTM encoder.
  • Joint mention detection and disambiguation: Joint MD+ED models trail disambiguation-only systems because their self-produced mention boundaries are noisy, although neural models outperform classic models by up to around 10 points.Chen et al. report the best AIDA result, while EBERT provides the best comparable AIDA result; De Cao et al. lead MSNBC.
  • Entity relatedness: Entity-relatedness results favor Huang et al. for nDCG@1 and MAP, Shi et al. for nDCG@5 and nDCG@10, and cosine-similarity methods over KG-relation methods at nDCG@10.The strongest entity representations combine multiple information sources, including entity types, relations, text, and knowledge-graph information.

5. Applications of Entity Linking

Entity linking supports established text-mining, knowledge-graph population, information-retrieval, and question-answering applications, while neural models enable its integration into larger language-model architectures. These integrations combine entity and graph information with language representations and improve several knowledge-oriented tasks.

  • Established applications: Entity linking resolves ambiguous mentions in text, making it a foundational step for text mining and biomedical or clinical information extraction.Biomedical and clinical settings are especially challenging because of terminology variability, complex ontologies, and scarce annotated resources.
  • Established applications: Entity linking identifies concepts in raw text and connects them to graph nodes before new facts are added during knowledge-graph population.TAC KBP provides evaluation workshops and benchmarks for knowledge-graph population subsystems, including entity linking.
  • Established applications: In information retrieval and question answering, entity linking adds semantic information, resolves query ambiguity, and restricts the search space.Applications include enriching biomedical literature search with genes, diseases, and drugs.
  • Neural language-model applications: Neural entity linkers can be integrated into larger neural networks, enabling joint training with additional entity-linking objectives.The survey describes integrations with BERT and other deep language models for representation learning and transfer learning.
  • Neural language-model applications: Knowledge-enhanced language models improve knowledge-oriented tasks, including question answering, entity typing, and relation extraction.The surveyed systems use structured knowledge graphs to enrich language representations; entity linking serves as the bridge between the two.

6. Conclusion

The survey finds that neural entity-linking systems generally outperform classical methods and organizes them around a generic architecture and recurring design trends. It highlights external candidate generation, self-attentive pretrained encoders, zero-shot adaptation, cross-encoder advantages, and leading joint or local models.

  • Overall conclusions: Neural entity-linking models generally achieve higher accuracy than classical methods, and the survey provides a generic architecture covering candidate generation, entity ranking, and mention and entity encoding.Taxonomies and feature tables organize prominent variations and model characteristics.
  • Design trends: Most studies still rely on external knowledge for candidate generation, while mention encoders increasingly use self-attention and pretrained contextual models such as BERT.Zero-shot domain adaptation methods use only target-domain entity descriptions rather than annotated target-domain data.
  • Design trends: Cross-encoder architectures are reported as superior to models with separate mention and entity encoders, while global context is widely used.The survey notes comparatively few recent studies focused only on local entity linking.
  • Model comparisons: E-BERT and De Cao et al.'s BART-based autoregressive model lead published approaches that jointly perform mention detection and entity disambiguation.Among local disambiguation models, the survey reports the best results for approaches by Shahbazi et al. and Wu et al.

7. Future Directions

The survey identifies future work centered on more end-to-end and zero-shot systems, better standardized evaluation, broader language-model applications, and integration of entity-linking losses into other neural networks. It also points to reasoning over linked knowledge as a further opportunity.

  • End-to-end and zero-shot EL: Future systems should reduce reliance on explicit candidate-generation resources through more end-to-end entity-linking models.The survey identifies resource completeness and ambiguity as factors affecting candidate-generation recall and precision.
  • End-to-end and zero-shot EL: Zero-shot entity linking remains a promising direction for emerging entities, but comparable evaluation requires more standardized benchmarks and processes.The proposed direction also includes global coherence, NIL prediction, and jointly combining mention detection with entity disambiguation.
  • Knowledge-enriched language models: More use-cases are expected for language models enriched with entity-linking information, particularly in downstream tasks.The survey reports that existing studies incorporate knowledge graphs into deep models and observe improvements over contextual language models.
  • Broader neural integration: Integrating entity-linking losses into neural models beyond language models is presented as technically straightforward because end-to-end entity linking is itself a neural network.The survey suggests applying this strategy to other neural architectures.
  • Broader neural integration: Linking textual mentions to knowledge graphs also opens the possibility of using reasoning elements in the final task.This connects disambiguation with structured graph information rather than treating linking as an endpoint.

Appendix A. Public Implementations of Neural Entity Linking Models

Appendix A provides a table of publicly available implementations for the neural models presented in Table 2.

  • Public implementations: Table 7 lists publicly available implementations of the neural models presented in Table 2.Implementations are either provided in the original paper or available through PapersWithCode.com.
Loading 2006.00575v4…