Source-linked AI summary
REL: An Entity Linker Standing on the Shoulders of Giants
Johannes M. van Hulst, Faegheh Hasibi, Koen Dercksen, Krisztian Balog, Arjen P. de Vries
TL;DR
Entity linking is widely used but existing toolkits lack a single solution that is modular, updateable, efficient, independent of external sources, and state of the art. REL addresses this gap with an open-source toolkit built from neural NLP components and delivered as a Python package and web API. The paper evaluates REL against established systems and a state-of-the-art linker on standard benchmarks.
Problem
Existing entity-linking toolkits have limitations in maintenance, long-text efficiency, external-source dependence, Wikipedia updating, throughput, and use of recent neural NLP approaches.
Method
REL combines modular mention detection and entity disambiguation with state-of-the-art NLP components, Wikipedia-based resources, and replaceable deployment components.
Results
The paper reports experimental comparisons of REL with established entity-linking systems and a state-of-the-art end-to-end linker on standard benchmarks.
Takeaways & Limitations
REL provides an open-source entity-linking toolkit that can be deployed as a Python package or web API and updated to newer Wikipedia versions.
Takeaways & Limitations
REL is currently optimized for annotating documents and short texts, with query linking left for future training on annotated queries.
Abstract
from arXiv · showhide
Entity linking is a standard component in modern retrieval system that is often performed by third-party toolkits. Despite the plethora of open source options, it is difficult to find a single system that has a modular architecture where certain components may be replaced, does not depend on external sources, can easily be updated to newer Wikipedia versions, and, most important of all, has state-of-the-art performance. The REL system presented in this paper aims to fill that gap. Building on state-of-the-art neural components from natural language processing research, it is provided as a Python package as well as a web API. We also report on an experimental comparison against both well-established systems and the current state-of-the-art on standard entity linking benchmarks.
1 INTRODUCTION
REL addresses shortcomings in existing entity-linking toolkits by combining modern NLP methods with modular, updateable, lightweight deployment. It is offered as an open-source toolkit and can be used as a Python package or web API.
- Motivation: Existing toolkits may be unmaintained, inefficient for long text, dependent on external sources, tied to dated Wikipedia versions, or lacking neural approaches.The paper frames these limitations as barriers to maintaining and deploying a modern entity linker.
- Contribution: REL is an open-source entity-linking toolkit built as an ensemble of state-of-the-art methods and packages from NLP research.REL stands for Radboud Entity Linker.
- Design: REL separates mention detection from entity disambiguation in a modular architecture, allowing the NER component to be replaced for different precision, recall, or throughput needs.The design explicitly supports choosing an NER method appropriate to the deployment context.
- Design: REL reports 700 ms for an average 300-word document and is designed to run on an average laptop or desktop without a GPU.The stated throughput includes NER, which can be replaced with a more efficient option.
- Deployment: REL is trained on the 2019-07 Wikipedia dump and includes scripts intended to simplify updates to newer Wikipedia versions.It is also distributed as a Python package and exposed through a RESTful API.
2 ENTITY LINKING IN REL
REL follows a three-stage entity-linking pipeline: mention detection, candidate selection, and entity disambiguation. Its components combine NER, prior- and context-based candidate generation, and document-level coherence modeling.
- Mention Detection: REL detects linkable text spans with Flair, a contextualized-word-embedding NER system using bidirectional character-level language modeling and sequence labeling.The NER component produces coarse-grained entity types for detected mentions.
- Mention Detection: The mention detector is replaceable, allowing REL to use another NER tagger such as spaCy or a dictionary-based approach.This modularity supports different precision–recall and throughput trade-offs.
- Candidate Selection: REL selects up to 7 candidates per mention: 4 from the mention–entity prior and 3 from similarity to the mention’s context.The prior combines Wikipedia and CrossWikis hyperlink counts with a uniform YAGO-based probability.
- Candidate Selection: Context-based candidate selection uses Wikipedia2Vec entity and word embeddings to compare entities with a 50-word local context.The similarity expansion considers 30 entities with the highest P(e|m) prior before adding the top 3 candidates.
- Entity Disambiguation: Entity disambiguation combines local compatibility with coherence across linking decisions in the document.REL uses the Ment-norm method, whose coherence function incorporates relations between document mentions.
- Entity Disambiguation: The coherence model uses latent relations, entity embeddings, diagonal relation matrices, and mention-context representations from a single-layer neural network.The relation weights are normalized over other mentions.
- Entity Disambiguation: Max-product loopy belief propagation optimizes the disambiguation objective, after which a two-layer network combines P(e|m) with document-level max-marginal probabilities.The ED model is trained with max-margin loss and calibrated with a logistic function for posterior probabilities.
3 IMPLEMENTATION AND USAGE
REL is implemented as a lightweight, modular toolkit that can be deployed locally or accessed through an API, with support for updating its Wikipedia resources.
- Implementation Details: REL stores embeddings and mention-entity priors in SQLite, reducing API memory use to 1.8GB without preloading embeddings.
- Implementation Details: REL does not require a GPU during inference because its disambiguation model is a feed-forward network.
- Implementation Details: REL separates mention detection, entity disambiguation, and P(e|m) index generation into replaceable components.
- Usage: Users can deploy REL with a new Wikipedia dump to keep entities current or target a specific Wikipedia version.
- Usage: The API returns mention positions, text, linked entities, disambiguation confidence, and available mention-detection confidence and type.
4 EVALUATION
The evaluation compares REL versions with established systems and a state-of-the-art NLP linker on GERBIL benchmarks, including entity-linking, disambiguation, and efficiency results.
- Evaluation Setup: REL is evaluated against SOTA NLP, DBpedia-Spotlight, and WAT using GERBIL micro and macro InKB F1 scores for EL and ED.
- Evaluation Setup: The comparison reports strong matching results, which require exact prediction of gold mention boundaries.
- Results: The evaluation includes local entity-disambiguation results reported from prior work.
- Efficiency: REL efficiency is measured in seconds on 50 AIDA-B documents longer than 200 words, averaging 323 (± 105) words and 42 (± 19) mentions per document.
- Efficiency: The implementation design targets a more efficient NER approach within the entity-linking pipeline.
5 CONCLUSION
REL is an open-source entity-linking toolkit built from state-of-the-art neural NLP components and offered as both a Python package and a web API; its current scope is documents and short texts.
- REL builds on state-of-the-art neural components from NLP research and is available as a Python package and web API.
- REL is currently optimized for annotating documents and short texts.
- The authors plan to train REL on annotated queries for entity linking in queries.