Source-linked AI summary
SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions
Han Xiao, Minlie Huang, Xiaoyan Zhu
TL;DR
Text-aware knowledge graph embedding is needed to capture semantic relevance and express triples precisely, but prior models inadequately model strong text–triple correlations. SSP jointly embeds symbolic triples and textual descriptions in semantic subspaces, and experiments report substantial improvements over state-of-the-art baselines across knowledge graph completion and entity classification.
Problem
Existing embedding models need textual descriptions to capture semantic relevance and precise semantic expression, while current text-aware models inadequately characterize strong correlations between texts and triples.
Method
SSP jointly learns symbolic triples and textual descriptions by restricting each triple’s embedding to a semantic subspace represented by a hyperplane.
Results
SSP outperforms all baselines across knowledge graph completion and entity classification, with SSP (Joint) improving 12.4% against TransE and 20.9% against DKRL(BOW).
Takeaways & Limitations
The results support modeling interactions between symbolic triples and textual descriptions as central to effective triple embedding.
Takeaways & Limitations
Direct comparison with one method is unfair because that method uses substantially more extra text corpus.
Abstract
from arXiv · showhide
Knowledge representation is an important, long-history topic in AI, and there have been a large amount of work for knowledge graph embedding which projects symbolic entities and relations into low-dimensional, real-valued vector space. However, most embedding methods merely concentrate on data fitting and ignore the explicit semantic expression, leading to uninterpretable representations. Thus, traditional embedding methods have limited potentials for many applications such as question answering, and entity classification. To this end, this paper proposes a semantic representation method for knowledge graph \textbf{(KSR)}, which imposes a two-level hierarchical generative process that globally extracts many aspects and then locally assigns a specific category in each aspect for every triple. Since both aspects and categories are semantics-relevant, the collection of categories in each aspect is treated as the semantic representation of this triple. Extensive experiments justify our model outperforms other state-of-the-art baselines substantially.
Introduction
Knowledge graph embeddings benefit from textual descriptions because they capture semantic relevance and provide more precise semantic expression. SSP addresses weak text–triple correlation by projecting triple embeddings into semantic subspaces and outperforms baselines on two benchmark tasks.
- Motivation: Textual descriptions capture semantic relevance and provide precise semantic expression beyond conventional symbolic triples.These properties help recognize difficult true triples and distinguish candidates with similar symbolic evidence.
- SSP approach: Projecting a triple’s loss onto a semantic hyperplane uses entity-description relevance to assess triple plausibility.A triple is accepted when the projected loss has sufficiently small l2-norm.
- SSP approach: Precise semantic expression distinguishes competing triples by projecting their loss vectors onto topic-specific semantic hyperplanes.The politician and lawyer candidates for Daniel Sturgeon become reasonably distinguishable after projection.
- Research gap: Existing text-aware models such as DKRL and “Jointly” leave weak correlations between textual and triple information insufficiently modeled.SSP instead models strong correlations by performing embedding in a semantic subspace.
- Evaluation: SSP consistently outperforms other baselines on two tasks across three WordNet and Freebase benchmark datasets.The evaluated tasks are knowledge graph completion and entity classification.
Related Work
Related work divides knowledge graph embedding into triple-only and text-aware approaches. Triple-only models transform entities through relations or alternative geometric spaces, while text-aware models incorporate names, corpora, or textual information.
- Taxonomy: Embedding methods are categorized as triple-only models using symbolic triples and text-aware models employing textual descriptions.This distinction organizes prior knowledge graph embedding research into two broad branches.
- Triple-only Embedding Models: TransE represents a fact by translating the head entity toward the tail with a relation vector, h + r = t.Its L2 loss norm scores triple plausibility, with smaller scores preferred.
- Triple-only Embedding Models: Later triple-only variants use manifolds, relation-specific hyperplanes, relation-related matrices, or additional structural information.Examples include ManifoldE, TransH, TransR, TransG, TransD, and TransM.
- Text-Aware Embedding: Text-aware models incorporate entity names, word embeddings, corpus alignment, or joint knowledge and language embedding.Alignment requirements can limit practical applicability and performance.
Methodology
SSP integrates knowledge-graph triples with textual semantics by projecting triple losses onto semantic hyperplanes generated from entity descriptions. It combines embedding and topic objectives, with Standard and Joint training settings, while retaining efficiency comparable to TransE.
- SSP uses knowledge triples and entity descriptions, adopting Freebase entity descriptions and WordNet definitions as textual information.
- Model Description: The score function projects the loss onto the semantic hyperplane and uses λ to balance the projected component with the loss norm; lower scores indicate more plausible triples.The projection component is negative, so a smaller score corresponds to less loss.
- Semantic Vector Generation: Entity descriptions are converted into topic-distribution semantic vectors, which are composed by normalized addition to represent the union of head and tail topics.The paper treats each entity description as a document and uses a topic model to obtain its semantic vector.
- Correlation Perspective: SSP restricts each translated triple loss h′ − t to a semantic hyperplane defined by the associated textual semantics, strengthening interaction between triples and descriptions.The translated head is h′ = h + r; the projection is designed to encode semantic relevance and precise semantic expression.
- Objectives & Training: SSP optimizes embedding and topic-specific objectives jointly, balancing them with μ; Standard fixes pretrained semantic vectors, whereas Joint learns semantics and embeddings simultaneously.The embedding objective uses rank-based hinge loss over golden and negative triples, while the topic objective uses NMF with L2 loss.
- 0.36s versus 0.28s per round in Link Prediction shows SSP has comparable running time to TransE in the reported setting.The paper gives theoretical complexity as O(ν × O(TransE)), with a small constant ν from projection and topic calculation.
Experiments
SSP is evaluated on knowledge graph completion and entity classification using benchmark subsets of WordNet and Freebase. Across the reported tasks, SSP outperforms the baselines, with analyses attributing gains to interactions between triples and textual descriptions, semantic relevance, and precise semantic expression.
- Overall results: SSP outperforms all baselines across the reported knowledge graph completion and entity classification tasks.The experiments use benchmark datasets derived from WordNet and Freebase.
- Knowledge Graph Completion: SSP(Joint) improves more than SSP(Std.) when textual semantics and symbolic triples are learned jointly.The comparison is reported for the knowledge graph completion results.
- Knowledge Graph Completion: SSP(Joint) improves 12.4% against TransE and 20.9% against DKRL(BOW).TransE represents the triple-only setting, whereas DKRL(BOW) represents the text-only setting in this comparison.
- Entity Classification: In entity classification, SSP yields the best accuracy, while FB20K evaluates a difficult zero-shot setting using only descriptions.The task is multi-label classification over 50 classes, evaluated with MAP.
- Semantic Relevance Analysis: The rank-statistics analysis reports triples whose rank is larger than m in TransE and less than n in SSP.For example, 601 denotes triples ranked below 100 by SSP(S.) but above 500 by TransE.
- Precise Semantic Expression Analysis: SSP correctly distinguishes 82.0% of hard triples in the standard setting and 83.2% in the joint setting, although TransE misclassifies them.These triples were scored slightly better than the golden triples by TransE; SSP uses precise semantic expression to distinguish them.
Conclusion
The paper proposes SSP to jointly learn from symbolic triples and textual descriptions by modeling their strong correlations. Extensive experiments show substantial improvements over state-of-the-art baselines.
- SSP jointly learns knowledge graph embeddings from symbolic triples and textual descriptions.
- SSP characterizes strong correlations between fact triples and textual descriptions.
- Extensive experiments show SSP achieves substantial improvements against state-of-the-art baselines.