Source-linked AI summary
Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces
Jiahang Cao, Jinyuan Fang, Zaiqiao Meng, Shangsong Liang
TL;DR
Knowledge graphs store structured factual information but are difficult to manipulate because their large-scale graph structures contain intricate relationships. This paper systematically surveys KGE methods through algebraic, geometric, and analytical representation-space perspectives, finding that non-Euclidean models typically capture diverse KG structures better in low dimensions, while matching Euclidean models in high dimensions.
Problem
Knowledge graphs effectively represent structured factual information but are difficult to manipulate because their large-scale structures contain intricate and complex relationships.
Method
The paper provides a comprehensive survey and fine-grained categorisation of KGE methods according to algebraic, geometric, and analytical structures.
Results
Non-Euclidean models typically capture various KG structures better than Euclidean models in low dimensions, while both show comparable performance in high dimensions.
Takeaways & Limitations
Hyperbolic space supports low-dimensional embeddings of tree-like structures with minimal distortion, while spherical space can represent ring structures.
Takeaways & Limitations
The survey does not further explore the intricacies of KGE model complexity, which depends on multiple factors including model size, mapping method, and acceleration algorithms.
Abstract
from arXiv · showhide
Knowledge graph embedding (KGE) is an increasingly popular technique that aims to represent entities and relations of knowledge graphs into low-dimensional semantic spaces for a wide spectrum of applications such as link prediction, knowledge reasoning and knowledge completion. In this paper, we provide a systematic review of existing KGE techniques based on representation spaces. Particularly, we build a fine-grained classification to categorise the models based on three mathematical perspectives of the representation spaces: (1) Algebraic perspective, (2) Geometric perspective, and (3) Analytical perspective. We introduce the rigorous definitions of fundamental mathematical spaces before diving into KGE models and their mathematical properties. We further discuss different KGE methods over the three categories, as well as summarise how spatial advantages work over different embedding needs. By collating the experimental results from downstream tasks, we also explore the advantages of mathematical space in different scenarios and the reasons behind them. We further state some promising research directions from a representation space perspective, with which we hope to inspire researchers to design their KGE models as well as their related applications with more consideration of their mathematical space properties.
1 Introduction
Knowledge graphs encode factual knowledge but become difficult to manipulate as their structures grow complex. This survey organizes KGE methods by representation spaces to relate spatial properties to the patterns they can capture and guide model design.
- Knowledge graphs store entities and relations as factual triples and support applications including question answering, recommendation, and information retrieval.A triple contains a head entity, relation, and tail entity.
- Existing KGE methods model relational and structural patterns using spaces such as Euclidean, hyperbolic, spherical, probability, and group-theoretic spaces.Examples include translation-based Euclidean embeddings, hyperbolic hierarchy modeling, Gaussian uncertainty representations, and group-based models.
- Representation space matters because its mathematical properties determine which relational and structural patterns KG embeddings can capture and preserve.
- The survey addresses the lack of a systematic mathematical-space review by classifying KGE methods into Algebraic, Geometric, and Analytical Structures.It also introduces mathematical-space definitions and their relationships to KGE components.
- KGE is also positioned as a structured factual-knowledge resource that can help address hallucination, domain-generalization, and explainability shortcomings of LLMs.The cited examples include incorporating KGs into language-model pretraining and using domain-specific KGs for biomedical tasks.
- The paper surveys space properties, compares KGE models from spatial perspectives, provides space-selection guidance, and proposes future research directions.The stated guidance includes choices related to loss functions and optimization.
2 Preliminaries
The preliminaries introduce the paper’s notation, KGE overview, and foundational mathematical spaces before the survey’s main categorization.
- Section 2 presents notation, an overview of KGE methods, and basic mathematical spaces together with their relationships.
2.1 Notations and Mathematical Background
This section establishes notation for mathematical spaces, scalars, vectors, matrices, and knowledge graphs represented as entity-relation triples.
- Mathematical spaces are denoted with blackboard-bold characters, while R and C denote the real and complex number fields.
- Scalars use normal characters, vectors use bold lowercase characters, and matrices use bold uppercase characters.The notation examples include x∈R, z∈R^n, and X∈R^m×n.
- A knowledge graph is represented as G = {E, R, T}, containing entities, relations, and relational facts stored as triples (h,r,t).The example <Beijing, isCapitalOf, China> represents a factual relation.
2.2 Knowledge Graph Embedding
KGE maps entities and relations into low-dimensional spaces, scores triple plausibility, and trains embeddings to distinguish observed facts from sampled negatives.
- KGE learns a mapping that projects entities and relations into dense, low-dimensional spaces while preserving graph structure and attributes for relationship inference.
- The embedding-mapping component represents entities and relations in spaces whose mathematical operations model patterns such as transformation, symmetry, and antisymmetry.TransE uses Euclidean space, whereas RotatE uses complex vector space.
- A score function evaluates triple plausibility by assigning higher scores to positive triples and lower scores to negative triples.
- Representation training maximizes positive-triple scores and minimizes negative-triple scores, commonly treating observed triples as positive and sampled unobserved triples as negative.
- Margin-ranking and cross-entropy losses are widely used objectives for training entity and relation representations.Cross-entropy assigns labels y_hrt∈{−1,1}, with 1 indicating positive and −1 indicating negative triples.
2.3 Fundamental Mathematical Spaces
Fundamental mathematical spaces are organized by progressively weaker structure, from inner-product vector spaces through normed and metric spaces to topological spaces. The section defines their operations, norms, distances, completeness, and geometric generalizations.
- Space relationships: The inclusion hierarchy is Inner Product Vector Space ⊂ Normed Vector Space ⊂ Metric Space ⊂ Topological Space, with each category carried by the next.Inner products express vector lengths and angles, whereas later spaces retain fewer measurable concepts.
- Topological and metric spaces: A topological space specifies open sets, while a metric space additionally introduces distances between elements.The topology supports concepts such as convergence, connectivity, and continuity.
- Vector spaces: A vector space defines addition and scalar multiplication over a real or complex scalar field, subject to algebraic operation conditions.The zero vector and additive inverses are also part of the vector-space structure.
- Normed spaces: A normed space assigns each vector a nonnegative norm satisfying norm axioms, and induces the metric d(x, y) = ∥x − y∥.The induced distance obeys nonnegativity, identity, symmetry, and the triangle inequality.
- Complete spaces: A Banach space is a complete normed space, while a Hilbert space is a complete inner product space.Completeness requires every Cauchy sequence to converge.
- Geometric spaces: Euclidean space is a finite-dimensional real inner product space, whereas manifolds generalize spaces locally through neighborhoods homeomorphic to open Euclidean subsets.Manifolds describe structures such as spheres and curved surfaces using topological properties.
3 Representation Spaces in Knowledge Graph Embedding
KGE methods use diverse representation spaces to preserve complicated knowledge-graph structures and desirable properties. The survey organizes these spaces and models into algebraic, geometric, and analytical structures to clarify how spatial properties affect embedding methods.
- Motivation: Complex KG structures, including 1-to-N, N-to-N, and hierarchical relationships, motivate embedding entities and relations in different representation spaces.The spaces are selected to better preserve complicated structural information.
- Space diversity: Mathematical spaces in KGE have diverse structures and complicated relationships, and some spaces originate from different mathematical structures rather than forming a single hierarchy.Spherical space and probability space are given as examples that cannot be discussed in the same category.
- Categorization: The survey categorizes KGE representation spaces and models according to Algebraic Structure, Geometric Structure, and Analytical Structure.The authors state that most KGE models fall under these three structures.
- Survey procedure: For each mathematical structure, the survey introduces definitions and properties, presents representative KGE methods, and summarizes how spatial advantages serve embedding needs.This connects mathematical-space properties with the intended behavior of KGE models.
3.1 Algebraic Structure
The algebraic perspective groups KGE models by the operations and structures used in their representation spaces. Vector, complex, group, and manifold-based approaches use different transformations to preserve relational and structural properties.
- Algebraic Structure: Algebraic structures are nonempty sets equipped with finite operations satisfying axioms, including vector spaces, groups, and rings.Vector addition, scalar multiplication, and other operations underpin many KGE methods.
- Vector Space: Vector-space models project entities and relations into shared or distinct spaces, using translations, projections, mappings, and linear operations to score triples.TransE uses translations in R^k, TransH uses relation-specific hyperplanes, and TransR maps entity embeddings into relation spaces.
- Vector Space: Linear and complex embeddings model relational interactions through bilinear or Hermitian operations, with ComplEx distinguishing entity ordering to capture antisymmetric relations.Real-space linear models match latent semantics, while complex embeddings retain dot-product efficiency and support asymmetric scoring.
- Vector Space: Complex-space rotations and quaternion extensions encode patterns such as symmetry, antisymmetry, inversion, and composition through angular or Hamilton-product transformations.RotatE defines relations as rotations using the Hadamard product, while quaternion systems provide richer algebraic representations.
- Group and Ring: Group-based models use rotation, reflection, composition, and scaling operations, while MöbiusE embeds entities and relations on a Möbius-ring surface.DihEdral models relation compositions with finite non-Abelian groups; MöbiusE inherits TorusE properties and adds manifold expressiveness.
3.2 Geometric Structure
The survey organizes geometric KGE models by Euclidean, hyperbolic, and spherical spaces, whose operations and distance functions shape the patterns and structures they can represent. Models use coordinate systems, curvature, or geometric objects to capture translations, hierarchies, temporal changes, and heterogeneous structures.
- Geometric perspectives: Geometric KGE models are analyzed through Euclidean, Hyperbolic, and Spherical Geometry, with operations summarized for each space.The survey distinguishes geometric structure from coordinate systems and examines models built on these three geometric perspectives.
- Euclidean Geometry: Euclidean models use Cartesian, polar, or spherical coordinates to represent transformations, hierarchies, and temporal changes.TransE models translation in Cartesian coordinates, HAKE uses radial and angular coordinates for hierarchy, and STKE uses radial, azimuth, and polar components for temporal graphs.
- Cartesian Coordinate: Cartesian models encode relations through translations, rotations, projections, and other transformations to capture relational patterns.Examples include TransE, RotatE, QuatE, PairRE, TripleRE, InterHT, TranS, HousE, and CompoundE.
- Hyperbolic Geometry: Hyperbolic models represent hierarchical data using spaces whose increasing radius provides more capacity for hierarchy information than Euclidean space.MuRP embeds hierarchical multi-relational data in the Poincaré ball, while ATTH uses relation-specific curvature and hyperbolic isometries to capture logical patterns and hierarchies.
- Spherical Geometry: Spherical and manifold-based models represent concepts, instances, or entities as geometric objects to model inclusion, uncertainty, noise, and flexible mappings.TransC uses spheres for concepts, HypersphereE extends spheres to account for uncertainty, and ManifoldE uses high-dimensional spheres to distinguish true facts.
- Mixed-curvature models: Mixed-curvature modeling addresses heterogeneous knowledge-graph structures that single-curvature spaces may overlook.M2GNN is designed for graphs containing both hierarchical and cyclic typed structures by using a mixed-curvature space.
3.3 Analytical Structure
Analytical KGE models are organized around probability and Euclidean spaces, using measures such as uncertainty or distance to represent knowledge. They include distributional embeddings and differential-equation-based dynamics for uncertain, multi-semantic, continuous, and temporal representations.
- Analytical perspectives: Analytical structure is defined through measurable operations, and the survey divides analytical KGE models into Probability Space and Euclidean Space.Metrics support operations such as integration and differentiation, while probability spaces provide probabilistic measures.
- Probability Space: KG2E represents each entity and relation with a multidimensional Gaussian N(μ, Σ), using the mean for position and covariance for uncertainty.KG2E is identified as the first density-based KGE model in the survey; related methods use KL-divergence or expected likelihood.
- Probability Space: TransG models multiple relation semantics with a Bayesian non-parametric infinite mixture model that generates multiple translation components.The model automatically selects the best match among the head, tail, and relation representations.
- Euclidean Space: FieldE represents entities as trajectories and relations as vector fields, using ordinary differential equations to preserve continuous changes in embedding space.Its ODE is de(t)/dt = fθr(e(t)); TANGO applies Neural ODEs to continuous-time temporal knowledge graphs.
- Analytical methods: Analytical methods support dynamic and continuous entity and relation representations while improving memory efficiency and adaptive computation.The survey connects these benefits to FieldE and temporal extensions such as TANGO.
4 Downstream Tasks of Knowledge Graph Embedding
The survey evaluates KGE models across downstream tasks by relating their performance, efficiency, and application suitability to mathematical representation spaces. It finds that space choice helps determine which relational and structural patterns embeddings capture, while also imposing computational trade-offs.
- Link Prediction: Link prediction ranks candidate entities by scoring completed triples against learned entity and relation embeddings.For an incomplete triple, each entity is considered as a candidate answer and assigned a plausibility score.
- Hierarchy Acquisition: On hierarchical datasets, non-Euclidean models generally outperform Euclidean models in low dimensions, whereas both become comparable in high dimensions.The reported datasets are WN18RR and FB15K-237, with curvature metrics ξG of -2.54 and -0.65, respectively.
- Hierarchy Acquisition: Hyperbolic space represents tree-like hierarchies with minimal distortion in two dimensions, while spherical space is suited to circular structures.The survey links these advantages to the geometric properties of the respective spaces.
- Model Complexity: Vector-based KGE models are generally more efficient than matrix-based models, while non-Euclidean models have marginally different complexity but can require more training time.Complexity also varies with model size, mapping method, acceleration algorithms, and geometric operations.
- Pattern Inference: Product operations, including Hadamard and Hamilton products, help models capture symmetry, antisymmetry, inversion, and composition concurrently.The survey recommends exploring additional algebraic operations under suitable conditions.
- Knowledge Infusion To Enhance Other Domain Applications: The survey covers KGQA, knowledge reasoning, recommendation, information retrieval, biomedicine, and integration with large language models as downstream application areas.It identifies combining KGs and LLMs for reasoning, interpretability, and knowledge enrichment as promising future directions.
5 Future Directions
The paper identifies future directions spanning algebraic, geometric, analytical, and integrated mathematical perspectives for more flexible KGE models. Key challenges include hybrid structures, computational cost, and underdeveloped analytical optimization.
- Algebraic Perspective: Algebraic operations can compensate for model shortcomings when handling complex 1-N, N-1, and N-M relations, motivating operations that support multiple tasks.CompoundE is cited as an example of combination operations.
- Geometric Perspective: Geometric methods capture properties such as spheres and hyperbolic hierarchies, but existing approaches can perform poorly on knowledge graphs with hybrid structures.Mixture-curvature and geometry-interactive methods are proposed to improve geometric embedding.
- Geometric Perspective: Future geometric models should combine comprehensive representation with lower energy consumption because complex geometric embeddings require more training time on large-scale knowledge graphs.A fewer-parameter RotH-like model is noted, but the efficiency challenge remains open.
- Analytical Perspective: Analytical properties including convergence, stability, complexity, and derivability affect downstream performance but remain poorly studied in KGE.Differentiability is required for gradient-descent training, while compact Lie groups can prevent divergence and remove the need for regularization.
- Integrated Perspectives: Because algebraic, geometric, and analytical perspectives are complementary, multi-perspective models are a promising direction.FieldE uses neural ODE trajectories to continuously represent several underlying geometries, including Euclidean, Poincare Ball, Hyperboloid, and Spherical spaces.
6 Conclusion
The survey reviews state-of-the-art KGE techniques through algebraic, geometric, and analytical perspectives of representation spaces. It compares their embedding spaces, scoring functions, and optimization properties, and reports that hyperbolic models are competent at assimilating hierarchical information.
- Survey Scope: The survey classifies KGE techniques according to algebraic, geometric, and analytical perspectives of representation spaces.It first introduces fundamental mathematical spaces and their relationships before comparing KGE methods.
- Comparative Analysis: The review compares KGE techniques across embedding space, scoring function, and optimization properties.It also summarizes unique characteristics of specific spaces through experimental results.
- Observed Strengths: Hyperbolic-based models manifest competence in assimilating hierarchical information.
- Future Directions: The survey highlights efficient geometric embedding and rational optimization as future directions.