Source-linked AI summary
Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation
Hongzhao Huang, Larry Heck, Heng Ji
TL;DR
Entity disambiguation must link ambiguous mentions to knowledge-base entities while modeling topical coherence among candidates. This paper introduces DSRM, which combines DNNs with semantic KGs to map heterogeneous entity knowledge into a latent space. DSRM improves entity-disambiguation performance over the Milne and Witten (2008a) relatedness approach on two public datasets.
Problem
Entity disambiguation requires linking ambiguous mentions to knowledge-base entities, while existing Wikipedia anchor-link relatedness can be unreliable, popularity-biased, and based on an overly strong shared-link assumption.
Method
DSRM uses DNNs to encode structured and textual knowledge from semantic KGs and train entity representations so semantically related entities are closer in latent space.
Results
DSRM significantly outperforms NGD and VSP for relatedness measurement and entity disambiguation, while KG-based NGD and VSP outperform their Wikipedia-anchor-link variants (p ≤ 0.05).
Takeaways & Limitations
Semantic KGs provide better resources than Wikipedia anchor links for relatedness measurement, and DNNs are more effective than NGD and VSP in this setting.
Takeaways & Limitations
The approach assumes that entity mentions are given as input, for example by a named entity recognition system.
Abstract
from arXiv · showhide
Entity Disambiguation aims to link mentions of ambiguous entities to a knowledge base (e.g., Wikipedia). Modeling topical coherence is crucial for this task based on the assumption that information from the same semantic context tends to belong to the same topic. This paper presents a novel deep semantic relatedness model (DSRM) based on deep neural networks (DNN) and semantic knowledge graphs (KGs) to measure entity semantic relatedness for topical coherence modeling. The DSRM is directly trained on large-scale KGs and it maps heterogeneous types of knowledge of an entity from KGs to numerical feature vectors in a latent space such that the distance between two semantically-related entities is minimized. Compared with the state-of-the-art relatedness approach proposed by (Milne and Witten, 2008a), the DSRM obtains 19.4% and 24.5% reductions in entity disambiguation errors on two publicly available datasets respectively.
1 Introduction
Entity disambiguation links ambiguous mentions to knowledge-base entities, using topical coherence to measure which candidates belong to the same context. The paper proposes DSRM, which combines semantic KGs and DNNs to represent heterogeneous entity knowledge for relatedness modeling.
- Entity disambiguation links ambiguous mentions to referent entities in a knowledge base such as Wikipedia, with topical context helping distinguish candidates such as basketball teams from cities.
- Wikipedia anchor-link relatedness can be unreliable, favor popular entities, and assume that semantically related entities share anchor links.The paper gives the example that NBA is judged more related to Chicago than Chicago Bulls.
- DSRM combines deep neural networks with semantic knowledge graphs to address limitations of anchor-link relatedness.The proposed model uses structured and contextual KG knowledge, including entity facts, types, and descriptions.
- DSRM maps heterogeneous entity knowledge into a latent space by training neural models to minimize distances between semantically related entities.
- Experiments on publicly available news and tweet datasets show that DSRM outperforms competitive baselines for relatedness measurement and entity disambiguation quality.
2 Related Work
Prior work treats entity relatedness as important evidence for disambiguation, spanning individual and collective approaches. This paper extends deep distributed representation learning from textual objects to semantic KGs and applies it to global topical coherence.
- Earlier relatedness approaches rely on classic similarity measures and resources such as WordNet or Wikipedia.
- Existing entity-disambiguation methods use entity relatedness as evidence, ranging from non-collective resolution to collective disambiguation based on global topical coherence.
- The paper extends distributed representation learning to large-scale semantic KGs by combining structured and contextual knowledge for entity representations.
- Unlike work measuring local context similarity for entity disambiguation, this paper measures entity relatedness for global topical coherence modeling.
- Semantic KGs have also supported external knowledge mining and entity representations for tasks including extraction, linking, semantic parsing, link prediction, and question answering.
3 A Deep Semantic Relatedness Model (DSRM)
The DSRM learns latent semantic representations of entities from heterogeneous knowledge-graph information to measure entity relatedness. It uses automatically mined related and unrelated pairs to train a deep neural model whose output vectors support cosine-based relatedness scoring.
- 3.1 The DSRM Architecture: DSRM encodes heterogeneous knowledge from knowledge graphs into latent entity representations for semantic relatedness modeling.The model uses structured and contextual entity information as input to a deep neural network.
- 3.1 The DSRM Architecture: Connected entities, relations, and entity types provide the DSRM's input features, with word hashing reducing the dimensionality of large bag-of-words representations.Connected entities are represented as bag-of-words, while relations and entity types use binary one-hot vectors.
- 3.1 The DSRM Architecture: The network applies nonlinear hidden layers to produce an entity semantic vector, and cosine similarity measures relatedness between two output vectors.The hidden layers learn semantic features through back propagation with an objective designed for entity relatedness.
- 3.2 Learning the DSRM: Training data are automatically mined from knowledge graphs and nearby Wikipedia anchor links, producing positive and negative entity pairs for discriminative learning.The procedure uses linked entity pairs as positives and sampled candidate entities as negatives.
- 3.2 Learning the DSRM: The model minimizes a softmax-based loss over related and non-related entities, with smoothing selected on held-out data and parameters tuned using cross validation.Training uses mini-batch stochastic gradient descent; the reported configuration has two hidden layers with 300 units each.
4 Topical Coherence Modeling with Unsupervised Graph Regularization
The framework models topical coherence through an unsupervised relational graph connecting mention–candidate pairs. Graph regularization then refines candidate rankings by combining initial scores with smoothness over semantically related nodes.
- 4 Topical Coherence Modeling with Unsupervised Graph Regularization: The framework is designed as a completely unsupervised alternative because manually labeled data are challenging to obtain for entity disambiguation.The implementation omits local features used by Huang et al. (2014) because those features were mainly designed for mention detection.
- 4.1 Relational Graph Construction: The method constructs a relational graph whose nodes pair each mention with an entity candidate and carry candidate ranking scores.Each ranking score represents the probability that the candidate is the mention's referent.
- 4.1 Relational Graph Construction: Edges connect relevant mentions and semantically related candidates when the neighboring candidate is among the k nearest nodes, with edge weights given by entity relatedness.Tweet relevance uses social-network connectivity, whereas news relevance is modeled within a single document.
- 4.2 Seed Selection: Initial rankings combine prior popularity and context similarity, while unambiguous mentions and highly confident candidates provide labeled seed nodes.The confidence heuristic requires prior popularity of at least 0.95 and agreement with context similarity.
- 4.3 Graph Regularization: Graph regularization simultaneously refines unlabeled-node rankings using seed scores, initial rankings, and the graph's weight matrix.The objective penalizes differences between rankings of strongly connected nodes through a regularization parameter controlling initial-score fidelity versus graph smoothness.
5 Experiments
Experiments evaluate semantic relatedness methods and their effects on entity disambiguation, showing that DSRM improves relatedness quality and disambiguation over established alternatives.
- 5.1 Data and Scoring Metric: The experiments use Wikipedia and Freebase resources, a relatedness benchmark with 3,314 queries, and news and tweet disambiguation datasets.Wikipedia is reduced to 4 million entities after filtering, and candidate generation considers the top 30 entities by prior popularity.
- 5.3 Quality of Semantic Relatedness Measurement: DSRM significantly outperforms M&W for semantic relatedness, and adding more knowledge types improves relatedness quality because the four types complement each other.The comparison is statistically significant at p ≤0.05.
- 5.3 Quality of Semantic Relatedness Measurement: M&W favors cities and other popular entities, whereas DSRM assigns higher relatedness to sports teams associated with NBA and NFL.The examples indicate that DSRM avoids the popularity bias observed in M&W.
- 5.4 Impact on Entity Disambiguation: GraphRegu combined with M&W achieves competitive performance on both datasets, supporting its effectiveness for modeling topical coherence.The regularization parameter µ is set to 0.8 from a held-out CoNLL 2003 set.
- 5.4 Impact on Entity Disambiguation: 24.5% and 19.4% relative reductions in disambiguation errors are achieved by DSRM1234 versus M&W on news and tweet datasets, respectively.DSRM1234 significantly outperforms strong baselines on both datasets, with all comparisons at p ≤0.05.
- 5.5 Discussion: Semantic KGs outperform Wikipedia anchor links for relatedness and disambiguation, while DNN-based DSRM outperforms NGD and VSP on both tasks.The reported comparisons are significant at p ≤0.05; KGs are attributed to containing cleaner semantic knowledge than anchor links.
6 Conclusions and Future Work
The paper concludes that DSRM combines deep neural networks and semantic knowledge graphs to measure entity relatedness, improving entity disambiguation through topical coherence modeling. It also identifies future directions involving semantic-path encoding and joint, context-aware modeling.
- DSRM encodes diverse knowledge from semantic knowledge graphs through multilayer nonlinear transformations to represent entities with useful semantic features.
- Graph regularization uses DSRM for topical coherence modeling and achieves significantly better performance than state-of-the-art entity disambiguation approaches.
- Future work will encode semantic paths directly in neural networks and jointly model entity disambiguation with entity relatedness for dynamic, context-aware scores.