Source-linked AI summary
Message Passing for Hyper-Relational Knowledge Graphs
Mikhail Galkin, Priyansh Trivedi, Gaurav Maheshwari, Ricardo Usbeck, Jens Lehmann
TL;DR
Hyper-relational KGs require models that represent qualifiers without losing their distinction from main-triple relations, while existing link-prediction benchmarks contain important flaws. The paper proposes StarE, a message-passing encoder, and WD50K, a Wikidata-based dataset; experiments show generally stronger link-prediction performance for StarE-based models. The paper also identifies limitations in benchmark quality and notes that gains cannot all be attributed to StarE’s encoding ability.
Problem
Triple-based representations do not naturally capture additional fact information, and existing hyper-relational link-prediction benchmarks have design flaws.
Method
StarE extends GNN message passing to encode hyper-relational KGs with arbitrary qualifier counts while preserving the roles of qualifiers and main-triple relations.
Results
StarE-based models generally outperform other approaches on hyper-relational link prediction across the evaluated benchmarks.
Takeaways & Limitations
Qualifier-enriched facts provide useful signal for link prediction, and WD50K offers a Wikidata-based benchmark designed to better capture hyper-relational task complexity.
Takeaways & Limitations
The reported gains cannot all be attributed to StarE’s hyper-relational encoding because Transformer without StarE also performs competitively.
Abstract
from arXiv · showhide
Hyper-relational knowledge graphs (KGs) (e.g., Wikidata) enable associating additional key-value pairs along with the main triple to disambiguate, or restrict the validity of a fact. In this work, we propose a message passing based graph encoder - StarE capable of modeling such hyper-relational KGs. Unlike existing approaches, StarE can encode an arbitrary number of additional information (qualifiers) along with the main triple while keeping the semantic roles of qualifiers and triples intact. We also demonstrate that existing benchmarks for evaluating link prediction (LP) performance on hyper-relational KGs suffer from fundamental flaws and thus develop a new Wikidata-based dataset - WD50K. Our experiments demonstrate that StarE based LP model outperforms existing approaches across multiple benchmarks. We also confirm that leveraging qualifiers is vital for link prediction with gains up to 25 MRR points compared to triple-based representations.
1 Introduction
Hyper-relational KGs represent facts with qualifiers that add disambiguating information, addressing limits of triple-based KGs. The paper proposes StarE, a message-passing encoder that preserves qualifier roles, introduces WD50K to address benchmark flaws, and reports stronger link-prediction performance.
- Motivation: Triple-based KGs make it difficult to represent additional information that disambiguates facts between entities.For example, qualifiers can distinguish the degrees Albert Einstein obtained at different universities.
- Motivation: Existing approaches can lose the distinction between main-triple relations and qualifier relations when representing hyper-relational facts.The paper identifies n-ary composition and decomposition into quintuples as examples of this problem.
- Approach: StarE extends GNN message passing to encode hyper-relational KGs with arbitrary numbers of qualifiers while preserving the semantic roles of triples and qualifiers.The encoder learns latent representations of graph nodes and edges.
- Dataset: Existing WikiPeople and JF17K benchmarks contain design flaws that make them ineffective for hyper-relational link prediction.The paper proposes WD50K, a Wikidata-based dataset containing statements with varying amounts of qualifiers.
- Results: STARE-based models generally outperform other approaches on hyper-relational link prediction, while qualifier pairs provide additional useful signal.The experiments evaluate both model performance and the contribution of qualifier information.
2 Related Work
Earlier hyper-relational KG methods simplify or decompose qualifier structure, often losing the attribution of entities and relations. StarE instead directly augments relation representations with qualifiers while supporting variable qualifier counts and unrestricted fact ordering.
- Prior approaches: Early methods derived from triple-based embeddings simplify qualifier attributes or convert hyper-relational facts into n-ary representations.These transformations can lose entity-relation attribution or use an abstract relation for the combined fact.
- Prior approaches: GETD expects one relation per fact and cannot process facts with different arities in a single dataset.Facts of different arities must be split and trained separately.
- Prior approaches: NaLP supports multiple entities and relations but treats the main triple and qualifier key-value pairs equally.Its artificial decomposition therefore does not distinguish relation-specific qualifiers from the main triple.
- Prior approaches: HINGE retains hyper-relational facts but represents them through triple-quintuple operations that lack granular relation-instance representations.It also requires sequential curriculum training after sorting facts by arity.
- StarE: StarE directly augments relation representations with any number of qualifiers while separating auxiliary entities and relations from the main triple.The approach imposes neither an input-order restriction nor a limit on qualifiers per fact.
- Hypergraph distinction: Hypergraph approaches are conceptually different because their abstract hyperedges lose attribution between main-triple entities and qualifiers.Combining main and qualifying relations into typed hyperedges can create a combinatorial explosion when qualifier counts are unrestricted.
3 Preliminaries
This section introduces message passing for graph neural networks and extends it to directed, multi-relational and hyper-relational knowledge graphs. It also formalizes hyper-relational facts as main triples with qualifier pairs.
- Message passing iteratively learns node representations by aggregating messages from neighboring nodes.Each node has a vector representation and a neighborhood from which messages are collected.
- GCNs transform neighbor representations with a shared weight matrix, combine them, and apply a nonlinearity such as ReLU.
- GCN, GAT, and GIN do not explicitly model relation embeddings and therefore require modifications for multi-relational knowledge graphs.
- Directed multi-relational graphs represent edges as ordered relation-labeled links, with inverse and self-looping relations added for bidirectional flow and state updates.
- CompGCN addresses relation modeling by learning specific edge-type vectors and composing each neighbor representation with its relation.
- A hyper-relational fact is represented as a main triple (s, r, o) plus a set of qualifier relation–value pairs.For example, an education fact can include academic degree and academic major qualifiers.
4 STARE
STARE extends message passing to encode hyper-relational facts by combining main-triple relations with aggregated qualifier representations while preserving their semantic roles. Its link-prediction architecture uses the updated embeddings to encode queries and predict entities.
- STARE incorporates qualifier pairs into message passing alongside the main triple by combining a relation embedding with a fixed-length qualifier vector.
- The combination function γ controls how qualifier information is merged with the main relation representation, including weighted-sum formulations controlled by α.
- Qualifier vectors are composed from qualifier relations and entities, aggregated by position-invariant summation, and projected through Wq.
- STARE supports an arbitrary number of qualifier pairs while distinguishing entities and relations in the main triple from those in qualifiers.
- For link prediction, STARE updates entity and relation embeddings through message passing, then a Transformer, pooling, and fully connected layers produce scores over entities.The fixed-dimensional output is compared with updated entity embeddings and passed through a sigmoid to obtain a probability distribution.
- STARE is presented as a general representation-learning framework, while this work focuses its evaluation on link prediction.Other downstream tasks, such as node classification, are left for future work.
5 WD50K Dataset
The paper argues that WikiPeople and JF17K inadequately benchmark hyper-relational link prediction because qualifiers are sparse or test leakage allows memorization. It introduces WD50K, sampled from Wikidata and cleaned to provide statements with meaningful qualifier coverage.
- WikiPeople contains few usable qualifiers after removing literals, so triple-only facts dominate its overall performance.About 13% of statements contain literals, and fewer than 3% of the remaining statements contain qualifier pairs.
- JF17K has major test leakage: about 44.5% of test statements share their main triple with training statements.This overlap allows triple-based models to memorize subjects and objects appearing in the test set.
- WD50K is proposed as a Wikidata-based dataset to alleviate the qualifier sparsity and leakage problems of earlier benchmarks.Its construction filters literals and removes train or validation statements sharing test main triples.
- WD50K contains 236,507 statements describing 47,156 entities and 532 relations, with about 14% of statements having at least one qualifier pair.
6 Experiments
The experiments evaluate STARE for hyper-relational link prediction, compare it with existing models and triple-only baselines, and vary qualifier coverage and statement length. STARE-based models generally perform competitively or better, with larger gains as qualifier information increases, although performance saturates after two qualifiers.
- 6.1 Evaluating STARE on the LP Task: STARE (H) + Transformer (H) outperforms other hyper-relational models on WikiPeople and JF17K, including a gain of 11.3 MRR points on JF17K.The reported JF17K gains are 11.3 (25%) MRR, 13 (33%) H@1, and 7.8 (12%) H@10 points over the next-best approach.
- 6.1 Evaluating STARE on the LP Task: On WD50K, STARE-based approaches provide effective but non-trivial baselines, with MRR 0.349, H@1 0.271, and H@10 0.496.The authors state that the task remains far from solved.
- 6.1 Evaluating STARE on the LP Task: Transformer (H) without STARE also performs competitively to HINGE, so the gains cannot all be attributed to STARE’s encoding ability.The paper nevertheless reports that adding STARE is consistently advantageous across the datasets.
- 6.2 Impact of Ratio of Statements with and Without Qualifier Pairs: As the ratio of qualifier-containing facts increases, STARE (H) + Transformer (H) increasingly outperforms Transformer (H).The H@1 differences are 4.1, 6.8, and 8.9 points on WD50K (33), WD50K (66), and WD50K (100), respectively.
- 6.3 Impact of Number of Qualifiers per Statement: For all datasets, performance saturates after two qualifier pairs; STARE remains slightly improving or stable as statements lengthen, while Transformer (H) degrades.The authors leave open whether saturation reflects dataset characteristics or limited aggregation of longer statements.
- 6.4 Comparison to Triple Baselines: Triple-only baselines perform competitively on JF17K and WikiPeople because leakage and qualifier sparsity dominate those benchmarks.Cleaning JF17K produces a performance drop of about 20 MRR points in both models.
7 Conclusion
The conclusion presents STARE as a message-passing encoder for hyper-relational KGs and introduces WD50K after identifying flaws in existing link-prediction datasets.
- STARE applies message passing to representation learning over hyper-relational KGs.The authors identify future applications to node and graph classification and to large-scale KGs.
- Experimental results show that STARE performs competitively against existing hyper-relational approaches and greatly outperforms triple-only baselines.
- WD50K is a Wikidata-based hyper-relational dataset proposed to address flaws in existing link-prediction datasets.The authors characterize WD50K as closer to real-world graphs and better able to capture link-prediction complexity.
A Further details on WD50K
WD50K is built from Wikidata and combines broad entity and relation coverage with qualifier-rich statements, sparse graph structure, leakage controls, and sparse storage.
- WD50K uses Wikidata because it is actively maintained, receives contributions from companies, and serves as a reference KG for recent NLP tasks and datasets.
- WD50K contains 47,156 entities and 532 relations, including 5,460 entities and 45 relations found only in qualifier pairs.
- 99.9% of 236,393 statements contain up to five qualifiers, while 114 long-tail statements contain up to 20 qualifiers.
- The dataset removes training and validation instances whose main triples occur in test instances; fewer than 4% of test instances retain direct or semantic inverse leakage.The inverse-leakage estimate is 1.6k out of 46k test instances.
- Two sparse COO matrices represent main triples and qualifiers, linking each fact’s qualifier columns through a shared index k.This yields O(|E| + |Q|) memory consumption, linear in the total number of qualifiers.
C Hyperparameters
The reported configuration uses a two-layer STARE encoder with a Transformer decoder, while experiments were trained on a Tesla V100 under dataset-specific resource demands.
- STARE with a Transformer decoder was tuned on validation data using the hyperparameters reported in Table 5.The decoder functions include DistMult, circular correlation, and RotatE implementations for φq and φr.
- The selected configuration uses two STARE layers, 200-dimensional embeddings, Adam with learning rate 0.0001, label smoothing 0.1, and 0.3 dropout.Qualifier aggregation uses summation, and γ(·) is a weighted sum with α of 0.8.
- The Transformer decoder has two layers, hidden dimension 512, four attention heads, and 0.1 dropout.
- Training STARE (H) plus Transformer (H) on WD50K requires approximately two days on one Tesla V100 GPU and uses 10.8M parameters.The corresponding JF17K model uses 7.1M parameters and takes about 10 hours; the WikiPeople configuration is also reported separately.
- Reduced WD50K variants require 20 hours with 9M parameters, 9 hours with 6.8M parameters, or 5 hours with 5M parameters.
D Decoders
The decoder study compares convolutional and Transformer-based alternatives for qualifier-aware link prediction on WD50K, with the default Transformer generally performing best.
- The study evaluates StarE with a default Transformer, two CNN-based decoders, and another Transformer decoder using the same encoder hyperparameters.
- StarE + ConvE reshapes stacked entity and relation embeddings into an H × W image before applying 2D convolution, ReLU, and flattening.
- StarE + MskTrf inserts an explicit [MASK] token at the object position, processes the query with Transformer layers, and scores the resulting mask representation.
- The default StarE + Trf decoder generally achieves the strongest results on WD50K variants, with its largest gains on H@1.
E Relation-Qualifiers Aggregation
The experiment compares three functions for aggregating relation and qualifier representations across four WD50K datasets. All settings show similar performance, indicating model stability with respect to the aggregation function.
- The experiment evaluates concatenation, element-wise multiplication, and a weighted sum for aggregating relation and qualifier representations.The weighted sum uses a fixed α of 0.8.
- The comparison uses STARE (H) + Transformer (H) models across four WD50K datasets.
- All three aggregation settings have similar performance, indicating model stability with respect to the choice of γ(·) function.