Source-linked AI summary
Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
Ke Liang, Yue Liu, Sihang Zhou, Wenxuan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, Xinwang Liu
TL;DR
Contrastive KGE methods often struggle to construct reliable positive pairs from complex KG structure, especially when language models misestimate entity semantics. KGE-SymCL mines relation-symmetrical positions, aligns their embeddings, and improves existing KGE models across link prediction and entity classification settings. The paper concludes that future gains may come from constructing high-confidence negative pairs.
Problem
Complex KG structures make appropriate contrastive-pair construction difficult, while existing methods often rely on language models instead of fully mining graph structure.
Method
KGE-SymCL extracts relation-symmetrical structures, treats entities in symmetrical positions as positive pairs, and applies a self-supervised alignment loss.
Results
Experiments show that KGE-SymCL can be adopted across KGE models and benchmark datasets for different downstream tasks, with promising and superior performance.
Takeaways & Limitations
Relation-symmetrical structural information provides a reusable basis for improving the expressive and discriminative ability of KGE models.
Takeaways & Limitations
The framework does not study high-confidence negative contrastive-pair construction, which the paper identifies as a direction for further improving discriminative capacity.
Abstract
from arXiv · showhide
Knowledge graph embedding (KGE) aims at learning powerful representations to benefit various artificial intelligence applications. Meanwhile, contrastive learning has been widely leveraged in graph learning as an effective mechanism to enhance the discriminative capacity of the learned representations. However, the complex structures of KG make it hard to construct appropriate contrastive pairs. Only a few attempts have integrated contrastive learning strategies with KGE. But, most of them rely on language models ( e.g., Bert) for contrastive pair construction instead of fully mining information underlying the graph structure, hindering expressive ability. Surprisingly, we find that the entities within a relational symmetrical structure are usually similar and correlated. To this end, we propose a knowledge graph contrastive learning framework based on relation-symmetrical structure, KGE-SymCL, which mines symmetrical structure information in KGs to enhance the discriminative ability of KGE models. Concretely, a plug-and-play approach is proposed by taking entities in the relation-symmetrical positions as positive pairs. Besides, a self-supervised alignment loss is designed to pull together positive pairs. Experimental results on link prediction and entity classification datasets demonstrate that our KGE-SymCL can be easily adopted to various KGE models for performance improvements. Moreover, extensive experiments show that our model could outperform other state-of-the-art baselines.
1 INTRODUCTION
KGE-SymCL addresses unreliable contrastive-pair construction in knowledge graphs by mining relation-symmetrical structures, where structurally symmetrical entities provide positive pairs. The plug-and-play framework and alignment loss improve KGE representations across downstream tasks and models.
- Motivation: Language-model-based contrastive KGE can construct inaccurate pairs because KG relationships may assign opposite semantics to entities that language models consider similar.This dependence can sharply reduce performance when the language model is not finely trained.
- Motivation: Relation-symmetrical structures offer a graph-based criterion for finding entities with similar semantics, unlike ordinary KG neighborhoods whose neighbor semantics may differ.The paper connects this property to the symmetry underlying positive-pair construction in homogeneous graph contrastive learning.
- Proposed framework: KGE-SymCL takes entities in relation-symmetrical positions as positive pairs and uses a self-supervised alignment loss to pull those pairs together.The framework is designed as a plug-and-play enhancement for existing KGE models.
- Contribution: The paper presents relation-symmetrical structures as a structural-semantic basis for contrastive KGE and identifies the approach as the first to mine these symmetrical patterns in KGs.This contribution is positioned as a way to use information underlying KG structure rather than relying primarily on language models.
- Evaluation: The framework integrates with typical KGE baselines and is evaluated on link prediction and entity classification tasks.The experiments are intended to assess both effectiveness and generalization.
- Evaluation: Experiments report promising performance, generalization across KGE models and datasets, and superiority over other KGE models.The paper attributes these outcomes to improved expressive and discriminative representations.
2 RELATED WORK
Prior work spans KGE, graph contrastive learning, and early contrastive KGE methods. The paper identifies language-model similarity and limited structural generalization as unresolved issues that KGE-SymCL addresses with symmetry-based semantics.
- KGE: KGE models include translational distance, semantic matching, and GNN-based approaches for representing entities and relations.Examples include TransE and HAKE, DisMult and ConvE, and RGCN and COMPGCN.
- Graph contrastive learning: Contrastive learning mines hidden intra-data information self-supervised, while graph contrastive learning has become an active research direction.The related work discusses mutual-information, redundancy-reduction, asymmetrical, and learnable-augmentation strategies.
- Contrastive KGE: Existing contrastive KGE methods such as SimKGC commonly use language-model-estimated semantic similarity to construct positive pairs.This approach differs from directly mining structural information in the graph.
- Open gap: Language models may misestimate KG semantics because structurally opposite entities can share similar surface forms, while structural semantics are rarely used in contrastive KGE.The paper also notes that task-specific models such as KGCL have poor generalization to other tasks.
3 METHOD
The method consists of relation-symmetrical structure extraction followed by relation-symmetrical contrastive learning. These components identify structurally corresponding entities and use them to enhance KGE representations.
- Method overview: KGE-SymCL is organized into Relation-Symmetrical Structure Extraction and Relation-Symmetrical Contrastive Learning.The framework first obtains structural positives and then applies contrastive learning to their embeddings.
3.1 Preliminary
The preliminary method defines relation sequences over a knowledge graph and its inverse graph, providing the basis for identifying relation-symmetrical structures.
- Knowledge graph notation: A knowledge graph is represented as fact triplets (e_u, r_t, e_v), with entities as nodes and relations as edge labels.The inverse graph reverses each triplet's head and tail entities.
- Relation sequence extraction: Relation Sequence Extraction obtains the relation sequence along the i-th path between two entities in the union of the KG and its inverse graph.The operation is defined only when that path exists.
- Relation sequence extraction: The extracted sequence records the relations encountered along a path, with each relation connecting consecutive entities in the inverse-augmented graph.The notation includes the path index, entity positions, and relation labels.
3.2 Relation-Symmetrical Structure Extraction
Relation-symmetrical structures identify entities occupying corresponding positions around a pivot, whose relation sequences and hop lengths match. The extraction module traverses the KG to collect these structurally symmetrical target entities for each anchor.
- Motivation: Unlike ordinary KG neighbors, entities in relation-symmetrical positions can provide positive pairs with similar semantics.The method reframes contrastive-pair construction as extracting these structures.
- Definition: Relation-symmetrical structures place an anchor and target around a pivot with matching relation sequences on both sides.Both paths contain k hops.
- Extraction: The extraction module traverses structures from anchor entity e_a and retains those satisfying the relation-symmetry definition.It takes the KG, anchor entity, and upper-bound hyper-parameter K as inputs.
- Extraction: For each anchor, the target set P_ea contains entities identified as relation-symmetrical within structures of at most K hops.The procedure selects all relation-symmetrical entities from the retained target structures.
3.3 Relation-Symmetrical Contrastive Learning
KGE-SymCL encodes entities with a selected KGE model, samples relation-symmetrical targets as positive candidates, and aligns their embeddings. The framework uses structural information without language models or negative pairs.
- Knowledge Graph Encoding: A selected KGE encoder maps each entity e to an embedding h_e, allowing the framework to plug into multiple KGE models.Examples include RDF2Vec, RGCN, COMPGCN, HAKE, and CompLEX-DURA.
- Contrastive Positive Pair Construction: Entities in the target set P_ea are positive candidates for anchor e_a, with m candidates randomly sampled for efficiency.The sampled entities are fed into the selected KGE model with the KG.
- Symmetrical Contrastive Loss: A self-supervised alignment loss pulls the anchor and positive embeddings together in latent space.The loss uses an MSE-style alignment objective and L2-normalized embeddings.
- Framework Properties: KGE-SymCL constructs contrastive samples from KG structure without language models, negative samples, or graph-augmented views.The framework treats relation-symmetrical entities as positive pairs but does not assume structurally unrelated entities are reliable negatives.
- Framework Properties: Its deterministic structural rules make positive-pair construction more explainable than embedding-similarity construction based on language models.The contrastive pairs directly reflect a specific attribute in the original KG.
3.4 Training Objective
The training objective combines the task-specific KGE loss with the symmetrical contrastive loss. KGE-SymCL therefore operates as an auxiliary module coupled to supervised task training.
- Objective: The overall objective is L = L_task + α · L_contrastive, combining task and contrastive losses.α is the trade-off hyper-parameter.
- Objective: KGE-SymCL is coupled with task-specific losses for link prediction or entity classification rather than used as a standalone objective.The task loss can use ranking, logistic, sampled multi-class, binary cross-entropy, or cross-entropy formulations depending on the task.
4 EXPERIMENT AND ANALYSIS
The experiments evaluate KGE-SymCL across link prediction and entity classification using seven benchmark datasets and multiple KGE baselines. The setup also compares against existing KGE models and reports standard evaluation procedures.
- Datasets: Experiments cover link prediction on FB15k-237, WN18RR, and NELL-995, plus entity classification on AIFB, AM, MUTAG, and BGS.The train/validation/test splits follow previous work.
- Implementation: The implementation uses PyTorch, one NVIDIA TITAN XP GPU, and an Intel Core i9-9900K CPU.KGE encoder parameters follow the original papers, while K, m, and α are tuned over specified ranges.
- Evaluation: Entity classification is evaluated with accuracy, while link prediction uses rank-based measures including Mean Reciprocal Rank.Candidate triples are ranked by model score against corrupted head or tail entities.
- Baselines: The comparison includes translational, semantic-matching, GNN-based, and contrastive KGE models.The link-prediction baselines include fourteen KGE models and SimKGC; entity classification uses established benchmark baselines.
4.2 Performance Comparison
KGE-SymCL improves typical KGE encoders across link prediction and entity classification, and it outperforms competing state-of-the-art models across the evaluated datasets. The reported gains support its plug-and-play generalization across encoders, tasks, and datasets.
- Effectiveness of KGE-SymCL: 3.1%, 3.0%, 5.2%, and 3.2% are the average NELL-995 improvements for RGCN and COMPGCN on MRR, Hit@1, Hit@3, and Hit@10, respectively.On FB15K-237, DisMult and HAKE also improve by 1.4%, 1.8%, 1.2%, and 1.3% on those metrics.
- Effectiveness of KGE-SymCL: 0.71% and 1.0% are the average improvements on MRR and Hit@10 when SymCL is added to SimKGC.This indicates that the framework extends to another contrastive KGE model.
- Effectiveness of KGE-SymCL: 1.92% is the average accuracy improvement over KGE baselines on entity classification.A Student’s t-test reports p-values below 0.05 for MRR in the tested COMPGCN and RGCN comparisons.
- Superiority of KGE-SymCL: KGE-SymCL outperforms other KGE models on all evaluated datasets, with especially strong results on FB15k-237 and NELL-995.WN18RR shows fewer improvements because its simpler, smaller dataset can already support promising performance from basic KGE models.
- Superiority of KGE-SymCL: 1.25% is the average accuracy improvement over previous state-of-the-art KGE models for entity classification.COMPGCN-SymCL performs better on the BGS and AM datasets.
- Discussion: The framework generalizes across different KGE encoders, benchmark datasets, and downstream tasks.The authors attribute the stronger representations to the structural information leveraged by SymCL.
4.3 Relation-Symmetrical Structure Analysis
The analysis finds relation-symmetrical structures across benchmark KGs and uses examples, runtime measurements, and hyper-parameter studies to assess their feasibility and practical behavior. These structures are common, while extraction is performed once and reused.
- Statistical Analysis: 1-hop and 2-hop relation-symmetrical structures occur in all three analyzed link-prediction datasets: WN18RR, FB15K-237, and NELL-995.The authors use their counts and proportions to assess the universality of the defined structures.
- Statistical Analysis: NELL-995 contains more relation-symmetrical structures than WN18RR and FB15K-237.The paper suggests this may relate to NELL-995 showing more apparent improvements from KGE-SymCL.
- Structure Samples: Entities in relation-symmetrical positions are commonly similar and correlated in the illustrated benchmark examples.The examples include one 1-hop and one 2-hop structure for each dataset.
- Running Time Analysis: Relation-symmetrical extraction runs once per dataset, and the resulting dictionary files are reused during experiments.The authors report acceptable computational costs for typical KG benchmarks.
- Hyper-parameter Analysis: Performance shows no great fluctuation as hop K and sampling number m vary over the tested ranges.The study varies K in {1, 2, 3}, with different m ranges for link prediction and entity classification.
5 CONCLUSION
KGE-SymCL uses relation-symmetrical structures to enhance KGE models and can be adopted across KGE models. The work leaves high-confidence negative contrastive-pair construction for future study.
- KGE-SymCL leverages symmetrical structural information in knowledge graphs to enhance the discriminative ability of KGE models.
- Extensive experiments verify that the framework can be easily adopted by other KGE models to enhance their discriminative and expressive ability.
- High-confidence negative contrastive-pair construction is not studied and remains a direction for improving KGE-SymCL.