PaperlayerView exact revision on arXiv

Source-linked AI summary

Embedding Uncertain Knowledge Graphs

Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, Carlo Zaniolo

arXiv:1811.10667v2cs.AIcs.CL

TL;DR

Embedding uncertain knowledge graphs is challenging because models must preserve confidence information and estimate uncertainty for unseen facts, which matters for knowledge-driven applications. UKGE learns embeddings from fact confidence scores, augments training with probabilistic soft logic, and consistently outperforms baselines across three tasks.

  • Problem

    Uncertain KG embedding must preserve confidence information and estimate the uncertainty of unseen relation facts, unlike deterministic KG embedding.

  • Method

    UKGE learns embeddings from uncertain relation-fact confidence scores, uses probabilistic soft logic for unseen facts, and evaluates two variants.

  • Results

    UKGE consistently outperforms baseline models on confidence prediction, relation fact ranking, and relation fact classification across three real-world uncertain KGs.

  • Takeaways & Limitations

    UKGE effectively preserves relation facts and uncertainty information in the embedding space across the evaluated tasks.

  • Takeaways & Limitations

    The method assumes confidence scores lie in [0, 1], requiring normalization for uncertain KGs whose original scores fall outside that range.

Abstract

from arXiv · show

Embedding models for deterministic Knowledge Graphs (KG) have been extensively studied, with the purpose of capturing latent semantic relations between entities and incorporating the structured knowledge into machine learning. However, there are many KGs that model uncertain knowledge, which typically model the inherent uncertainty of relations facts with a confidence score, and embedding such uncertain knowledge represents an unresolved challenge. The capturing of uncertain knowledge will benefit many knowledge-driven applications such as question answering and semantic search by providing more natural characterization of the knowledge. In this paper, we propose a novel uncertain KG embedding model UKGE, which aims to preserve both structural and uncertainty information of relation facts in the embedding space. Unlike previous models that characterize relation facts with binary classification techniques, UKGE learns embeddings according to the confidence scores of uncertain relation facts. To further enhance the precision of UKGE, we also introduce probabilistic soft logic to infer confidence scores for unseen relation facts during training. We propose and evaluate two variants of UKGE based on different learning objectives. Experiments are conducted on three real-world uncertain KGs via three tasks, i.e. confidence prediction, relation fact ranking, and relation fact classification. UKGE shows effectiveness in capturing uncertain knowledge by achieving promising results on these tasks, and consistently outperforms baselines on these tasks.

1 Introduction

Existing KG embeddings mainly target deterministic knowledge, while uncertain KGs attach confidence scores to facts and remain difficult to embed. UKGE addresses this gap by learning from confidence scores, using probabilistic soft logic for unseen facts, and outperforming baselines across three tasks.

  • 1 Introduction: Uncertainty is important because it naturally characterizes knowledge and supports applications including short-text understanding, question answering, and named entity recognition.
  • 1 Introduction: Uncertain KG embedding remains unresolved because models must preserve confidence information and estimate uncertainty for unseen relation facts.Existing methods often treat unseen facts as false, limiting their ability to model subtle uncertainty.
  • 1 Introduction: UKGE learns entity and relation embeddings from confidence scores to preserve both structural and uncertainty information.Unlike binary-classification approaches, it models the confidence scores of uncertain relation facts directly.
  • 1 Introduction: UKGE uses probabilistic soft logic to infer confidence scores for unseen relation facts during training and proposes two model variants.
  • 1 Introduction: UKGE consistently outperforms baseline models on confidence prediction, relation fact ranking, and relation fact classification across three real-world uncertain KGs.

2 Related Work

Prior work largely develops embeddings for deterministic KGs, while uncertain KGs associate facts with confidence scores and have received limited embedding research. Existing uncertain-network methods do not generalize to relational KGs, motivating UKGE's probabilistic reasoning component.

  • 2 Related Work: The paper presents uncertain KG embedding as an early research direction, noting no previous work on learning embeddings for uncertain KGs to its knowledge.
  • 2 Related Work: Deterministic KG embedding research includes translational, bilinear, neural, and circular-correlation-based models.
  • Uncertain Knowledge Graphs: Uncertain KGs attach confidence scores to relation facts and support applications such as short-text understanding.ConceptNet, Probase, and NELL obtain uncertainty estimates through crowdsourcing, relation extraction, or semi-supervised learning.
  • 2 Related Work: A prior uncertain-network embedding method cannot generalize to uncertain KGs because it models node proximity without explicit relations and produces only node embeddings.
  • Probabilistic Soft Logic: Probabilistic soft logic represents confidence values in [0, 1] as soft truth values and is adopted here to enhance embedding performance on unseen relation facts.

3 Problem Definition

An uncertain KG is defined as weighted relational triples over entities and relations, with each weight representing the fact's confidence. The embedding problem is to preserve both graph structure and confidence scores in a low-dimensional space.

  • 3 Problem Definition: An uncertain KG consists of weighted triples (h, r, t) over entities and relations, with each triple assigned a confidence score s_l in [0, 1].
  • 3 Problem Definition: The confidence score is interpreted as a probability for probabilistic soft logic, so scores outside [0, 1] require normalization.
  • 3 Problem Definition: The uncertain KG embedding problem is to encode each entity and relation in a low-dimensional space that preserves structure information and relation-fact confidence scores.

4 Modeling

UKGE models uncertain relation facts by preserving both KG structure and confidence information, then uses probabilistic soft logic to estimate confidence for unseen facts. Two variants differ in their confidence-score mapping functions.

  • 4 Modeling: UKGE learns embeddings from confidence scores while preserving structural and uncertainty information for observed and unseen relation facts.It models triple plausibility with entity and relation embeddings, then maps plausibility to confidence.
  • 4.1 Embedding-based Confidence Score Modeling for Relation Facts: The embedding plausibility function uses the inner product between relation embeddings and the element-wise product of head and tail embeddings.This DistMult-style function measures relatedness under a relation without additional parameter complexity.
  • The Joint Objective Function: The joint objective combines MSE on observed confidence scores with squared distances to satisfaction for PSL ground rules on unseen relations.A prior rule also penalizes predicted confidence for unseen facts, while logical rules can encourage scores supported by observed facts.
  • The Joint Objective Function: UKGErect and UKGElogi are two variants that differ in the choice of the confidence-score function f(l).UKGErect uses Equation (4), whereas UKGElogi uses Equation (3).

5 Experiments

The experiments evaluate UKGE on three uncertain knowledge-graph datasets spanning commonsense, web-extracted, and protein-interaction knowledge. Evaluation covers confidence prediction, relation ranking, and relation classification.

  • 5 Experiments: UKGE is evaluated on confidence prediction, relation fact ranking, and relation fact classification across three uncertain KG datasets.The datasets are CN15k, NL27k, and PPI5k, extracted from ConceptNet, NELL, and STRING.
  • 5.1 Datasets: CN15k contains 15,000 entities and 241,158 uncertain relation facts from ConceptNet, with scores normalized to [0.1, 1.0].Scores are first bounded to [0.1, 3.0] and then min-max normalized after taking logarithms.
  • 5.1 Datasets: NL27k contains 27,221 entities, 404 relations, and 175,412 uncertain relation facts extracted from NELL.Normalization to [0.1, 1] yielded the best results for this dataset.
  • 5.1 Datasets: PPI5k contains 271,666 uncertain protein-interaction facts involving 4,999 proteins and 7 interactions.The dataset is a STRING subset whose protein interactions are labeled with occurrence probabilities.

5.2 Experimental Setup

The experimental setup uses fixed train, validation, and test splits, adds negative links for testing, and constructs PSL rules from validated length-2 paths. Training and model selection use validation-based optimization procedures.

  • 5.2 Experimental Setup: Each dataset is split into 85% training, 7% validation, and 8% testing, with equal numbers of negative links added to the test sets.The added negatives test whether models correctly interpret negative links.
  • 5.2 Experimental Setup: Adam optimization and validation-selected hyperparameters are used, with L2 regularization fixed at 0.005 and early stopping based on validation performance.The search varies learning rate, embedding dimensionality, and batch size.
  • 5.2 Experimental Setup: Candidate PSL rules are created from length-2 paths and validated by hit ratio before grounding.Only observed strong relation facts are used in rule bodies; the final rule counts are 3 for CN15k, 4 for NL27k, and 1 for PPI5k.

5.4 Baselines

The comparison includes deterministic KG embeddings, an uncertain graph model, and simplified UKGE variants. These baselines isolate the roles of relation modeling, negative sampling, and PSL reasoning.

  • 5.4 Baselines: The baselines comprise TransE, DistMult, ComplEx, URGE, UKGEn−, and UKGEp−.They represent deterministic KG embeddings, uncertain graph embedding, and simplified versions of UKGE.
  • 5.4 Baselines: Deterministic KG models train on high-confidence facts and cannot predict confidence scores, so they are compared only on ranking and classification.Their validation early stopping uses mean reciprocal rank for these tasks.
  • 5.4 Baselines: URGE produces only node embeddings and cannot handle multiple relation types, so relation types are ignored when applying it here.The first-order proximity version is used because the tasks focus on edge relations.
  • 5.4 Baselines: UKGEn− removes negative sampling, whereas UKGEp− removes PSL reasoning and uses MSE for unseen relation facts.These variants test the contributions of negative links and PSL reasoning.

5.5 Confidence Prediction

Confidence prediction evaluates whether embeddings can estimate scores for unseen uncertain relation facts. UKGErect and UKGElogi outperform the baselines, with UKGErect achieving the lowest MSE and MAE among the variants.

  • 5.5 Confidence Prediction: The task predicts confidence scores for unseen relation facts using MSE and MAE as evaluation metrics.For each test fact, the predicted confidence score is compared with its target score.
  • 5.5 Confidence Prediction: UKGErect and UKGElogi outperform URGE, UKGEn−, and UKGEp− on confidence prediction.URGE lacks rich relation modeling, while UKGEn− omits negative sampling; PSL improves UKGErect over UKGEp−.
  • 5.5 Confidence Prediction: UKGErect achieves smaller MSE and MAE than UKGElogi.The comparison is reported across the confidence-prediction evaluation.

5.6 Relation Fact Ranking

Relation fact ranking evaluates how well models order candidate tails by confidence-weighted relevance. Both UKGE variants outperform all baselines, while UKGElogi achieves higher nDCG than UKGErect on all three datasets.

  • 5.6 Relation Fact Ranking: Mean nDCG evaluates the ranking of all vocabulary entities as candidate tails, using linear and exponential confidence-based gains.Exponential gain emphasizes highly relevant results more strongly.
  • 5.6 Relation Fact Ranking: UKGErect and UKGElogi outperform all baselines under every ranking setting, with UKGElogi higher on all three datasets.The models preserve both high plausibility for observed facts and their confidence-based ordering.
  • 5.6 Relation Fact Ranking: ComplEx performs best among deterministic KG embedding models because it handles asymmetric relations better than DistMult and TransE.TransE performs worse on 1-to-N relations, while ComplEx uses complex-domain embeddings.
  • 5.6 Relation Fact Ranking: UKGElogi produces top-ranked tails that include reasonable unseen relation facts, such as hotel is used for relaxing.Table 5 shows the top four predicted and true tails for example queries, with unobserved facts marked N/A.

5.7 Relation Fact Classification

Relation fact classification tests whether models distinguish strong facts from weak or false ones using confidence thresholds. UKGErect and UKGElogi consistently outperform all baseline models on F-1 and accuracy.

  • 5.7 Relation Fact Classification: The task labels a relation fact strong when its confidence score exceeds a KG-specific threshold τ.Test cases combine KG facts with randomly sampled negative links, followed by logistic regression on predicted confidence scores.
  • 5.7 Relation Fact Classification: UKGErect and UKGElogi consistently outperform all baseline models in relation fact classification.The evaluation reports F-1 scores and accuracies for distinguishing strong facts from weak or false facts.
  • 5.7 Relation Fact Classification: Deterministic KG models distinguish observed facts from negative links but do not use confidence information to identify high-confidence facts.UKGEn− fits confidence scores but cannot correctly interpret negative links as false.

6 Conclusion and Future Work

The paper presents UKGE as an approach for preserving relation facts and uncertainty in uncertain KG embeddings, evaluated through three tasks. Future work targets systematic logical-rule generation and uncertain knowledge extraction from text.

  • 6 Conclusion and Future Work: UKGE preserves both relation facts and uncertainty information in the KG embedding space.The authors describe this as the paper’s central model outcome.
  • 6 Conclusion and Future Work: The two UKGE variants are evaluated on confidence prediction, relation fact ranking, and relation fact classification.These experiments are conducted across the paper’s three evaluation tasks.
  • 6 Conclusion and Future Work: Future work will study systematic logical-rule generation and extend UKGE to uncertain knowledge extraction from text.The paper also proposes testing the impact of generated rules on embedding quality.
Loading 1811.10667v2…