Source-linked AI summary
Interaction Embeddings for Prediction and Explanation in Knowledge Graphs
Wen Zhang, Bibek Paudel, Wei Zhang, Abraham Bernstein, Huajun Chen
TL;DR
Existing knowledge graph embeddings generally overlook bi-directional crossover interactions between entities and relations, despite their relevance to selecting information for new-triple prediction. CrossE models these interactions with triple-specific embeddings and achieves state-of-the-art link-prediction results while generating more reliable prediction explanations than other methods.
Problem
Existing knowledge graph embeddings do not formally model bi-directional crossover interactions affecting entities and relations, while explanations for predictions are valuable for reliability in real applications.
Method
CrossE learns general entity and relation embeddings plus triple-specific interaction embeddings generated through a relation-specific interaction matrix, then compares a combined head-relation embedding with the tail.
Results
CrossE achieves state-of-the-art results on link prediction with complex datasets and generates more reliable explanations than other methods.
Takeaways & Limitations
Modeling crossover interactions supports both accurate link prediction and reliable explanations, although the paper presents explaining prediction results as a first step.
Takeaways & Limitations
The evaluation assumes that better explanations correspond to higher Recall and AvgSupport when selecting the same numbers of similar entities and relations.
Abstract
from arXiv · showhide
Knowledge graph embedding aims to learn distributed representations for entities and relations, and is proven to be effective in many applications. Crossover interactions --- bi-directional effects between entities and relations --- help select related information when predicting a new triple, but haven't been formally discussed before. In this paper, we propose CrossE, a novel knowledge graph embedding which explicitly simulates crossover interactions. It not only learns one general embedding for each entity and relation as most previous methods do, but also generates multiple triple specific embeddings for both of them, named interaction embeddings. We evaluate embeddings on typical link prediction tasks and find that CrossE achieves state-of-the-art results on complex and more challenging datasets. Furthermore, we evaluate embeddings from a new perspective --- giving explanations for predicted triples, which is important for real applications. In this work, an explanation for a triple is regarded as a reliable closed-path between the head and the tail entity. Compared to other baselines, we show experimentally that CrossE, benefiting from interaction embeddings, is more capable of generating reliable explanations to support its predictions.
1 INTRODUCTION
CrossE addresses the unmodeled bidirectional interactions between entities and relations in knowledge graph embedding by generating triple-specific interaction embeddings. It is evaluated on link prediction and prediction explanation, achieving strong results on challenging datasets and more reliable explanations than other methods.
- Motivation: Existing knowledge graph embeddings had not formally modeled bidirectional crossover interactions between entities and relations, although these interactions help select relevant information for predicting new triples.Prior methods typically learn general embeddings, while some learn multiple embeddings for only entities or relations.
- Approach: CrossE learns general embeddings plus multiple triple-specific interaction embeddings for both entities and relations through a relation-specific interaction matrix.For a triple (h,r,t), it generates hI and rI, combines them, and compares the result with the tail embedding.
- Results: CrossE achieves state-of-the-art results on link prediction tasks involving complex and more challenging datasets.The evaluation compares CrossE with various other KGEs across three benchmark datasets and reports a modest parameter size.
- Prediction Explanation: The paper adds an evaluation scheme that assesses whether embeddings can provide explanations for predicted triples, addressing a limitation of accuracy-only link prediction evaluation.Explanations are treated as reliable paths between the head and tail entities.
- Prediction Explanation: CrossE generates more reliable explanations than other methods, suggesting that interaction embeddings capture entity-relation similarities across different triple contexts.Explanation quality is evaluated using Recall and Average Support.
2 RELATED WORK
Prior KGE research includes general-embedding methods, methods with multiple representations for entities or relations, and methods incorporating extra information. CrossE is positioned against the limitation that existing approaches do not jointly model both directions of entity-relation interaction.
- Overview: Knowledge graph embedding learns low-dimensional distributed representations for entities and relations, sometimes incorporating logical rules, text, or hierarchical type information.These representations support knowledge graph applications such as completion and question answering.
- Overview: Prior KGE methods can be grouped into general-embedding methods and methods learning multiple representations for either entities or relations.This classification reflects the need for multiple representations arising from crossover interactions.
- General Embeddings: General-embedding methods such as TransE, RESCAL, HOLE, and RDF2Vec learn one general embedding for each entity and relation while using different modeling assumptions.TransE represents each relation as a single translation vector, whereas RESCAL learns relation matrices through tensor factorization.
- Multiple Embeddings: Methods with multiple embeddings include SE, ORC, TransH, TransR, and CTransR, which introduce relation- or role-specific representations under different structural assumptions.These methods primarily model effects from relations to entities and generally retain general embeddings for relations.
- Extra Information: Other KGE methods use path rules, physical or logical rules, external text, or hierarchical type information as additional constraints or signals.These approaches augment training from knowledge graph triples with extra information.
3 CrossE: MODEL DESCRIPTION
CrossE explicitly models bi-directional crossover interactions by generating triple-specific interaction embeddings for entities and relations from general embeddings and a relation-specific interaction matrix. Its four-stage score function combines these embeddings, while training uses negative sampling and regularization.
- Model overview: CrossE learns general entity and relation embeddings plus multiple triple-specific interaction embeddings for both entities and relations.Interaction embeddings preserve specific properties produced by crossover interactions.
- Interaction embeddings: A relation-specific interaction matrix generates entity interaction embeddings through Hadamard products with general entity embeddings.Each relation determines an interaction embedding for the entity.
- Interaction embeddings: Relation interaction embeddings are formed by a Hadamard product between the head entity’s interaction embedding and the general relation embedding.Each head entity therefore has an interaction embedding of the relation.
- Scoring: The score function combines head and relation interaction embeddings nonlinearly, then compares the combined representation with the general tail embedding using a bounded similarity score.The combination uses tanh, while the final score applies a sigmoid to the dot product.
- Training: CrossE is trained with a log-likelihood objective using positive and negative samples, L2 regularization, and gradient-based optimization.The regularization applies to E, R, C, and b.
- Model benefits: CrossE uses fewer extra parameters than learning multiple independent embeddings for every entity and relation, while inverse relations support head-entity prediction.Its total parameter count is (ne +2nr +1)×d before adding inverse-relation embeddings for head prediction.
4 EXPLANATIONS FOR PREDICTIONS
CrossE explains predicted triples by searching for reliable closed paths between their head and tail entities and finding similar structures that support those paths. Embedding-based candidate selection prunes the search space before exhaustive path and support checks.
- 4.1 Background: Explanations are meaningful paths between the head and tail entities that support a predicted triple.For example, a spouse-to-child path can support a father-child prediction.
- 4.1 Background: Embedding-based explanation search aims to improve predicted-result reliability while providing a perspective for evaluating embedding quality.Candidate selection uses embedding similarity to reduce search space before explanations are generated.
- 4.1 Background: Explanation reliability is measured by the number of similar knowledge-graph structures supporting the inference.Similar structures use the same relations with different specific entities, and more supports indicate greater reliability.
- 4.2 Embedding-based explanation search: The search first selects relations similar to the target relation, restricting paths to those likely to describe the same entity aspect.This pruning step avoids unrelated paths despite their potentially large number of supports.
- 4.2 Embedding-based explanation search: The method searches paths between the head and tail using direct searches for shorter structures and bidirectional searches for length-two structures.The procedure considers paths containing one or two relations and six corresponding structure types.
- 4.2 Embedding-based explanation search: It then selects similar entities to the head and retains only paths whose corresponding structures exist for those entities as supports.A path is accepted as an explanation only when it has at least one support in the knowledge graph.
5 EXPERIMENTAL EVALUATION
The evaluation covers link prediction and explanation generation using three established benchmark datasets from WordNet and Freebase. Dataset statistics are reported separately in Table 1.
- Datasets: CrossE is evaluated on WN18, FB15k, and FB15k-237, benchmark datasets derived from WordNet or Freebase.WN18 and FB15k were introduced in earlier work, while FB15k-237 was proposed separately.
- Tasks: The experimental evaluation considers two tasks: link prediction and generating explanations for predicted triples.Table 1 provides statistics for the benchmark datasets.
5.1 Evaluation I: Link Prediction
CrossE is evaluated on canonical head and tail link-prediction tasks using standard ranking metrics and filtered or raw settings. It performs especially strongly on complex datasets and relation types, while gains are smaller on simpler WN18.
- Evaluation protocol: Link prediction evaluates tail prediction (h,r, ?) and head prediction (?,r,t) by ranking candidate entities.MR, MRR, and Hit@N summarize the resulting ranks; MRR and Hit@k are reported as percentages.
- Evaluation protocol: CrossE is compared with published results from 21 baseline KGEs on filtered Hit@10 for WN18 and FB15k.Models using external information are excluded for fair comparison.
- Dataset results: On WN18, CrossE has Hit@10 comparable to the best baselines and outperforms most baselines on MRR, Hit@1, and Hit@3.WN18 is simpler than FB15k; all methods exceed 90% Hit@3 on WN18, whereas the best FB15k score is around 80%.
- Dataset results: On FB15k, CrossE achieves state-of-the-art results on most metrics, including MRR, Hit@3, and Hit@10.CrossE outperforms ANALOGY on Hit@3 and Hit@10, although ANALOGY is best on Hit@1.
- Dataset results: On FB15k-237, CrossE achieves state-of-the-art results with significant improvements over all baselines on every evaluation metric.The dataset removes redundant triples that cause inverse-relation leakage, making diverse semantic encoding more important.
- Relation-type analysis: CrossE performs much better than CrossES on all three datasets, isolating crossover-interaction modeling as the key architectural difference.Across relation types, it significantly outperforms other methods except for tail prediction on N-1 relations, with an average improvement of 11.7% on specified harder cases.
5.2 Evaluation II: Generating Explanations
The explanation evaluation measures how often KGEs can find supporting paths and how reliable those explanations are. CrossE’s interaction embeddings produce stronger explanation behavior than TransE and ANALOGY, though prediction accuracy and explanation quality are not directly aligned.
- Evaluation metrics: Explanations are evaluated by Recall and Average Support, measuring coverage and the reliability of found explanations.Higher Average Support is treated as indicating a more reliable explanation.
- Experimental setup: The experiment uses FB15k and compares CrossE with TransE and ANALOGY using the embeddings from their link-prediction experiments.CrossE uses interaction embeddings for triple-specific selection, while the baselines use general embeddings.
- Results: With ten similar entities and three similar relations, Recall ranges from 0.26 to 0.43 and Average Support from 5 to 566 across the three methods.ANALOGY has the best Recall but few examples per explanation, whereas TransE has the lowest Recall and roughly ten times ANALOGY’s Average Support.
- Results: CrossE’s Recall rises slightly while Average Support rises substantially as more similar relations or entities are selected.Its Average Support increases much faster than for TransE and ANALOGY.
- Structure analysis: Type 5 structures provide the most Average Support for TransE and CrossE, whereas ANALOGY’s high shares of types 1 and 2 have very low Average Support.The paper characterizes type 5 as the most natural path because both path relations point in the same direction as the target relation.
- Conclusion: KGE design and vector-space assumptions affect which path explanations models can provide.Link-prediction performance and explanation capability are not directly related, so both prediction accuracy and explanation quality matter.
6 CONCLUSION
CrossE captures crossover interactions and achieves state-of-the-art link-prediction results on complex linked datasets. The paper also treats reliable explanations as an important complement to prediction accuracy, while identifying broader explanation coverage as future work.
- CrossE captures crossover interactions between entities and relations when modeling knowledge graphs.
- CrossE achieves state-of-the-art results on link prediction with complex linked datasets.
- The paper evaluates embedding reliability through explanations for prediction results, alongside high-accuracy prediction.
- Future work includes enabling explanations for all predicted triples and improving missing-triple prediction and explanation reliability.