Source-linked AI summary
TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs
Zequn Sun, Jiacheng Huang, Wei Hu, Muchao Chen, Lingbing Guo, Yuzhong Qu
TL;DR
Existing KG embedding models use relation-level representations that cannot fully capture diverse relational structures. TransEdge contextualizes relations for specific head-tail pairs, forming edge embeddings used as translations, and reports state-of-the-art entity-alignment results with competitive link prediction. Its gains are smaller on some datasets whose KGs already have aligned relational structures.
Problem
Relation-level KG embeddings use one representation per relation and cannot distinguish the diverse contexts of different head-tail entity pairs.
Method
TransEdge contextualizes relation representations for specific head-tail pairs, forming edge embeddings and edge translations between entity embeddings.
Results
TransEdge achieves the best reported entity-alignment results across five datasets and very competitive link-prediction performance, including best Hits@1 on both tasks.
Takeaways & Limitations
The experiments support TransEdge as a general KG embedding approach for preserving relational structure across entity alignment and link prediction.
Takeaways & Limitations
On DBP-WD, TransEdge-CP improves Hits@1 against BootEA by 0.040, smaller than its gains on DBP15K because the KGs have aligned relational structures.
Abstract
from arXiv · showhide
Learning knowledge graph (KG) embeddings has received increasing attention in recent years. Most embedding models in literature interpret relations as linear or bilinear mapping functions to operate on entity embeddings. However, we find that such relation-level modeling cannot capture the diverse relational structures of KGs well. In this paper, we propose a novel edge-centric embedding model TransEdge, which contextualizes relation representations in terms of specific head-tail entity pairs. We refer to such contextualized representations of a relation as edge embeddings and interpret them as translations between entity embeddings. TransEdge achieves promising performance on different prediction tasks. Our experiments on benchmark datasets indicate that it obtains the state-of-the-art results on embedding-based entity alignment. We also show that TransEdge is complementary with conventional entity alignment methods. Moreover, it shows very competitive performance on link prediction.
1 Introduction
KG embeddings encode entities and relations as vectors, but relation-level representations cannot distinguish entity-specific contexts in complex knowledge graphs. TransEdge addresses this by learning edge embeddings and achieves strong results on entity alignment and link prediction.
- Motivation: Knowledge graphs represent entities and labeled directed edges as relational triples, enabling embedding-based inference tasks.A KG is defined by entities, relations, and triples of the form (head, relation, tail).
- Motivation: Relation-level models assign one representation to each relation, limiting their ability to reflect diverse head-tail contexts.The same relation may connect different entity pairs, while one pair may hold multiple relations.
- Motivation: TransE can make distinct entities appear similar because the same relation translation is applied across different entity pairs.This can harm entity alignment and produce incorrect link predictions in complex relations.
- TransEdge: TransEdge contextualizes relation representations for specific head-tail pairs and treats them as edge translations between entity embeddings.These context-specific representations are called edge embeddings.
- Results: TransEdge achieves state-of-the-art entity-alignment results and competitive link-prediction performance across benchmark datasets.The paper reports the best Hits@1 performance on both embedding-based entity alignment and link prediction.
2 Related Work
Related work includes translational, bilinear, and neural KG embedding models, alongside embedding-based methods for entity alignment. Existing approaches differ in how they model relational structure and incorporate alignment information.
- KG embedding models: KG embedding models evaluated by link prediction are grouped into translational, bilinear, and neural categories.The categories reflect different ways of computing relational compatibility or energy.
- KG embedding models: Translational models represent relations as transformations between entity embeddings, including translation vectors, hyperplane projections, and space projections.TransE uses translations, while TransH, TransR, and TransD use relation-specific transformations.
- KG embedding models: Bilinear models compute relational compatibility through operations such as Hadamard products, circular correlation, analogical constraints, or tensor decomposition.Examples include DistMult, ComplEx, HolE, Analogy, SimplE, and CrossE.
- Entity alignment: Embedding-based entity-alignment methods combine vector-space transformations, unified spaces, self-training, attributes, bootstrapping, descriptions, or graph structure.Examples include MTransE, IPTransE, JAPE, BootEA, KDCoE, and GCN-Align.
- Relation-contextualized embeddings: Relation-contextualized embeddings represent edge-specific relation information using general entity and relation embeddings plus contextualized edge embeddings.The contextualization operator combines entity interaction embeddings with relation information.
3 Edge-centric Knowledge Graph Embedding
TransEdge replaces one relation embedding with edge embeddings contextualized by specific head-tail entity pairs, using these as translations between entity embeddings. It defines context-compression and context-projection operations, then trains the resulting energy model with positive and negative triples.
- Edge-centric formulation: TransEdge computes an edge embedding from the head-tail context and relation label, then uses it as a translation between entity embeddings.The contextualization operator takes combined head and tail interaction embeddings with the relation embedding as input.
- Edge-centric formulation: Separate general and interaction embeddings represent entity geometry while encoding each entity’s participation in edge-embedding computation.The separation is intended to avoid interference between these two kinds of information.
- Context compression: The context-compression operation combines head-relation and tail-relation representations through multilayer perceptrons.It encodes [hc; r] and [r; tc] separately before a further MLP combines the resulting representations.
- Context projection: The context-projection operation projects the relation representation onto a hyperplane determined by the head-tail context.A unit-normal vector for the hyperplane is computed by an MLP from concatenated head and tail interaction embeddings.
- Training objective: Training assigns low energy to observed triples and higher energy to unobserved negative triples under a local-closed world assumption.Negative samples replace either the head or tail with random neighbors of the corresponding positive triple.
- Applications and complexity: TransEdge supports entity alignment by merging KGs through shared seed-alignment embeddings and bootstrapping additional likely aligned pairs.At test time, counterpart candidates are ranked by cosine similarity; link prediction ranks candidate triples by energy.
- Applications and complexity: The model learns two embeddings per entity with complexity O(2ned + nrd), exceeding TransE but remaining below TransD.Its parameter complexity grows linearly with the number of entities and embedding dimension.
4 Experiments
The experiments evaluate TransEdge on entity alignment and link prediction using multilingual and monolingual KG benchmarks, comparing it with established embedding-based methods. The study reports tuned model settings and evaluates against separate competitor sets for the two tasks.
- Evaluation scope: TransEdge is evaluated on entity alignment between two KGs and link prediction within one KG.The source code is reported as available online.
- Datasets: Entity-alignment experiments use DBP15K multilingual datasets and DWY100K monolingual datasets sampled from DBpedia, Wikidata, and YAGO3.DBP15K includes Chinese-English, Japanese-English, and French-English datasets with 15 thousand aligned entity pairs each.
- Baselines: Entity-alignment comparisons include MTransE, IPTransE, JAPE, BootEA, AlignE, and GCN-Align.Models requiring additional textual or attribute resources are excluded from the stated problem setting.
- Baselines: Link-prediction comparisons include translational, bilinear, and neural models such as TransH, TransR, TransD, HolE, SimplE, ProjE, and ConvE.HolE represents the bilinear group because ComplEx is described as very similar to it.
- Training settings: The reported TransEdge-CC and TransEdge-CP settings use embedding dimension d = 75 with distinct margins, negative-sample weights, and alignment thresholds.The selected settings are γ1 = 0.3, γ2 = 2.0, α = 0.3, s = 0.75 for TransEdge-CC and γ1 = 0.2, γ2 = 2.0, α = 0.8, s = 0.7 for TransEdge-CP.
Experimental Settings.
TransEdge consistently performs strongly on entity alignment across DBP15K and DWY100K, with semi-supervised training and context-based variants improving results. Its advantage is largest on datasets containing noisy, non-counterpart entities.
- TransEdge consistently achieves the best entity-alignment metrics across five datasets.
- On DBPZH-EN, TransEdge-CP (w/o semi) improves Hits@1 by 0.187 against AlignE and by 0.030 against BootEA.
- On DBP15K, TransEdge-CC and TransEdge-CP outperform BootEA, while TransEdge-CP performs better than TransEdge-CC.
- Semi-supervised training raises TransEdge-CP's DBPZH-EN Hits@1 from 0.659 to 0.735.
- TransEdge's improvement is smaller on DWY100K, where DBP-WD gains only 0.040 Hits@1 over BootEA.
- TransEdge has a large advantage on datasets with noisy entities, whereas DWY100K is relatively simple because entities are one-to-one aligned.
- TransR performs well only on datasets containing relation alignment and fails on the other three datasets.
Datasets.
The link-prediction evaluation uses the difficult FB15K-237 and WN18RR benchmarks and compares TransEdge with a broad set of published and reproduced baselines under filtered ranking evaluation.
- FB15K-237 and WN18RR are harder successors to FB15K and WN18RR because inverse relations are removed from testing sets.
- The link-prediction comparison includes five translational, seven bilinear, and five neural embedding models.
- Hyperparameters are selected by grid search, with d=200 for FB15K-237 and d=500 for WN18RR.
- Evaluation uses filtered ranking lists and reports average head- and tail-prediction results.
Link Prediction Results.
TransEdge performs strongly on link prediction by distinguishing relation contexts, achieving the best Hits@1 on both evaluated datasets while using lower complexity than some competing models.
- TransEdge-CP achieves the best Hits@1 scores on both FB15K-237 and WN18RR.
- TransEdge significantly outperforms TransE, TransH, TransR, and PTransE on link prediction.
- TransEdge-CC obtains the best MR result on WN18RR.
- TransEdge's Hits@10 and MRR scores trail ConvKB and RotatE, but its model complexity is lower.
4.3 Analysis on Complex Relational Structures in KGs
The complex-relational-structure analysis tests whether TransEdge remains stable when entity pairs receive multiple relations and visualizes how its contextual translations differ from relation-level translations.
- The double-relation experiment adds a dummy relation for every existing relation without changing KG structure, while intensifying multi-relation pairs.
- TransEdge shows less performance variation than MTransE under doubled relations.
- PCA visualizations show that TransEdge assigns different translation directions to the same relation across contexts.
- MTransE's translation vectors are almost parallel, unlike TransEdge's flexible and robust relational structures.
Multiple Entity Pairs with one Relation.
Conventional entity alignment methods use literals or OWL logics, while TransEdge is evaluated alone and in combination with LogMap on DWY100K.
- Conventional entity alignment methods exploit literal attributes such as names and comments, or OWL logics, to identify similar entities.
- Figure 4 visualizes country or state entities and their capitals, with green arrows denoting translation vectors between entities.
- Figure 5 compares TransEdge, LogMap, and their combination on DWY100K.
- The authors compare TransEdge-CP with LogMap and combine their predicted alignments by voting based on predicted similarity.They report conventional precision, recall, and F1-score results for this comparison.
5 Conclusion and Future Work
The paper concludes with a relation-contextualized KG embedding model that represents relations using context-specific embeddings and edge translations. Experiments support its effectiveness on entity alignment and link prediction, while future work targets richer multi-hop contexts and additional proximity measures.
- The proposed model represents relations with context-specific embeddings and builds edge translations between entities to preserve KG structures.
- Experiments on standard datasets demonstrate the model’s effectiveness on entity alignment and link prediction.
- Future work will study language models for multi-hop relation contexts and incorporate proximity measures such as attribute similarity.