Source-linked AI summary
Uncertain Knowledge Graph Completion via Semi-Supervised Confidence Distribution Learning
Tianxing Wu, Shutong Zhu, Jingting Wang, Ning Xu, Guilin Qi, Haofen Wang
TL;DR
UKG completion is hindered by extremely imbalanced triple-confidence distributions, which weaken embedding learning for missing triples and confidences. ssCDL combines confidence distribution learning with meta self-training over labeled and pseudo-labeled unseen triples. It achieves the best reported performance against state-of-the-art baselines on real-world UKG datasets and verifies its component strategies through ablation studies.
Problem
Existing UKG completion methods neglect extremely imbalanced confidence distributions, leaving lower-confidence samples insufficiently represented during embedding learning.
Source: §1 · pp. 1–2, §2 · p. 3Method
ssCDL transforms triple confidences into distributions and iteratively trains a relational learner with labeled data and pseudo-labeled unseen triples generated by a meta-optimized pseudo-confidence generator.
Source: §1 · p. 2, §4.3 · p. 6, §4.4 · p. 6Results
ssCDL achieves the best performance on real-world UKG datasets compared with state-of-the-art baselines, with ablations verifying the effectiveness of its strategies.
Source: §1 · p. 3, §5.4 · p. 9, §6 · p. 10Takeaways & Limitations
Confidence distributions and meta self-training provide a semi-supervised framework for addressing imbalanced confidence distributions in UKG embedding learning.
Source: §6 · pp. 9–10On CN15k, most ConceptNet triples are generally correct, so high- and low-confidence triples have little reliability distinction; better benchmark datasets are needed.
Source: §5.2 · p. 8, D Limitation · p. 13
Abstract
from arXiv · showhide
Uncertain knowledge graphs (UKGs) associate each triple with a confidence score to provide more precise knowledge representations. Recently, since real-world UKGs suffer from the incompleteness, uncertain knowledge graph (UKG) completion attracts more attention, aiming to complete missing triples and confidences. Current studies attempt to learn UKG embeddings to solve this problem, but they neglect the extremely imbalanced distributions of triple confidences. This causes that the learnt embeddings are insufficient to high-quality UKG completion. Thus, in this paper, to address the above issue, we propose a new semi-supervised Confidence Distribution Learning (ssCDL) method for UKG completion, where each triple confidence is transformed into a confidence distribution to introduce more supervision information of different confidences to reinforce the embedding learning process. ssCDL iteratively learns UKG embedding by relational learning on labeled data (i.e., existing triples with confidences) and unlabeled data with pseudo labels (i.e., unseen triples with the generated confidences), which are predicted by meta-learning to augment the training data and rebalance the distribution of triple confidences. Experiments on two UKG datasets demonstrate that ssCDL consistently outperforms state-of-the-art baselines in different evaluation metrics.
1 Introduction
UKGs improve knowledge representation by attaching confidence scores to triples, but completion methods struggle with highly imbalanced confidence distributions. ssCDL addresses this through confidence distributions and meta self-training over labeled and unlabeled data, outperforming state-of-the-art baselines across UKG completion tasks.
Motivation
- Existing UKG embedding methods neglect extremely imbalanced confidence distributions, limiting their ability to fit lower-confidence samples for completion.Most stored NELL triples have confidences above 0.9, while lower-confidence triples are often omitted as potentially erroneous.Source: §1 · pp. 1–2, Figure 1 · p. 2
Confidence Distribution Learning
- Confidence Distribution Learning transforms each triple confidence into a distribution so neighboring, few, or unseen confidences provide additional supervision.For example, confidence 0.78 can introduce supervision from nearby values such as 0.76, 0.77, and 0.79.Source: §1 · p. 2
Method
- ssCDL combines a confidence-distribution relational learner with a pseudo-confidence generator that meta-optimizes pseudo labels for unseen triples.The method iteratively learns embeddings from labeled and pseudo-labeled data through meta self-training.Source: §1 · p. 2
- Meta self-training augments training data with reliable pseudo confidences, helping rebalance the confidence distribution and address unlabeled-data reinforcement.The approach targets the challenge of generating reliable confidences for negative-sampled unseen triples.Source: §1 · p. 2
Empirical Results
- Experiments show that ssCDL outperforms baselines across different evaluation metrics and UKG completion tasks.The reported tasks include confidence prediction and link prediction, while ablations verify the effectiveness of confidence distribution learning and meta self-training.Source: §1 · p. 3
2 Related Work
Prior UKG completion research uses relational learning and related semi-supervised or few-shot approaches for confidence and link prediction. The paper identifies imbalanced confidence distributions as a remaining limitation and proposes ssCDL to use labeled and unlabeled data.
Scope
- UKG completion comprises confidence prediction and link prediction, commonly solved by embedding entities and relations while preserving graph structure and confidence information.Few-shot relational learning additionally addresses long-tail relation distributions, but that is not this paper’s focus.Source: §2 · p. 3
Normal Relational Learning
- Normal relational-learning methods include UKGE [3], PASSLEAF, BEUrRE [2], UPGAT, UKGsE, GTransE, FocusE [16], and UKRM, which differ in scoring, embeddings, subgraphs, or language modeling.Some methods target both completion tasks, whereas GTransE and FocusE [16] focus on link prediction.Source: §2 · p. 3
Few-Shot Relational Learning
- Few-shot methods such as GMUC and GMUC+ [23] use metric learning, while unKR re-implements few-shot and normal relational-learning approaches.The paper distinguishes these methods from its main focus on imbalanced confidence distributions.Source: §2 · p. 3
Research Gap
- Existing relational-learning methods neglect extremely imbalanced confidence distributions, motivating ssCDL’s use of labeled and unlabeled data to reinforce UKG embedding learning.The related-work discussion excludes query-answering reasoning methods because they do not focus on UKG completion.Source: §2 · p. 3
3 Preliminaries
UKGs represent factual triples with confidence scores, and completion predicts missing confidences or entities. Confidence Distribution Learning extends this setting by estimating a distribution over confidence labels rather than relying on a single confidence value.
3.1 Problem Definition
- An uncertain knowledge graph is a set of quadruples (h, r, t, s), where s ∈ [0, 1] measures the likelihood that triple (h, r, t) is true.Entities and relations are drawn from sets E and R, respectively.Source: §3.1 · p. 3
- UKG completion has two sub-tasks: confidence prediction estimates a missing triple confidence, while link prediction estimates a missing entity.The definitions describe queries with one unknown component, including missing confidence, tail, or head entities.Source: §3.1 · p. 3
3.2 Confidence Distribution Learning
- Confidence Distribution Learning estimates a confidence distribution for each given triple and applies label distribution learning to UKG completion.Unlike ordinary label prediction, label distribution learning also quantifies each label’s degree of relevance.Source: §3.2 · p. 3
- In CDL, triple confidences are transformed into distributions to capture supervision from few or unseen confidence values in labeled data.This extends supervision beyond the single observed confidence associated with a triple.Source: §3.2 · p. 4
- The confidence distribution is discrete over the [0, 1] interval, with each label assigned a description degree and all degrees summing to one.The construction uses ordered confidence labels and a distribution vector for each triple.Source: §3.2 · p. 4
- The distribution is generated from a Gaussian centered at the triple confidence, with 101 confidence labels when n = 100.The confidence is the mean and therefore receives the highest description degree.Source: §3.2 · p. 4
4 Methodology
ssCDL combines confidence-distribution learning with relational learning and meta-generated pseudo labels to improve UKG embedding training. Its CDL-RL predicts confidences and links, while PCDG supplies pseudo confidence distributions through iterative meta self-training.
4.1 Overview
- ssCDL transforms labeled triple confidences into distributions and jointly trains CDL-RL with labeled and PCDG-generated pseudo-labeled data.The framework iteratively improves embeddings while incorporating additional confidence supervision.Source: §4.1 · p. 4, §4.2 · p. 4
4.2 CDL-based Relational Learner
- CDL-RL predicts confidence distributions from concatenated entity-relation embeddings and learns them using KL divergence together with expectation-based MSE.Its confidence-prediction network uses Softmax over an (n + 1)-dimensional output, while the objective combines distribution similarity and confidence expectation matching.Source: §4.2 · pp. 4–5
- CDL-RL also optimizes link prediction with a sigmoid rank score and margin-based ranking loss, while uncertainty weights balance confidence and link-prediction objectives.The loss includes observation-noise parameters and limits the influence of the relatively large number of negative samples through ϕ.Source: §4.2 · p. 5
4.3 Pseudo Confidence Distribution Generator
- PCDG generates pseudo confidence distributions for negatively sampled unlabeled triples and is meta-optimized according to whether they improve CDL-RL on labeled data.After a one-step CDL-RL update using labeled and temporary pseudo-labeled data, PCDG minimizes the updated learner’s labeled-data loss.Source: §4.3 · pp. 5–6
4.4 Meta Self-training
- Meta self-training delays pseudo-label use until PCDG quality is sufficient, then repeatedly selects high-quality pseudo labels and trains CDL-RL with them.The schedule first stabilizes CDL-RL, trains PCDG, and only later incorporates selected pseudo-labeled data into CDL-RL.Source: §4.4 · p. 6
5 Experiments
Experiments evaluate ssCDL on two UKG datasets for confidence prediction and link prediction, including setup, ablations, and low-confidence analysis. ssCDL achieves stronger confidence and link prediction results than baselines, while both confidence distribution learning and meta self-training contribute to performance.
5.1 Experimental Setup
- Experiments use NL27k and CN15k, partitioned into 85% training, 7% validation, and 8% testing, with MSE and MAE for confidence prediction and Hits@1 and WMRR for link prediction.Baselines include UKGE, PASSLEAF, UKGsE, BEUrRE, and UPGAT.Source: §5 · p. 7, §5.1 · p. 8
5.2 Confidence Prediction
- ssCDL reduces MSE and MAE against the best baseline by 52.6% and 17.6% on NL27k, and by 63.8% and 53.2% on CN15k.These comparisons concern confidence prediction.Source: §5.2 · p. 8, Table 2 · p. 8
5.3 Link Prediction
- ssCDL achieves the best Hits@1 and WMRR among baselines for link prediction on NL27k and CN15k.The improvement is less significant than for confidence prediction because link prediction benefits only indirectly from the confidence-focused optimization.Source: §5.3 · p. 8, Table 2 · p. 8
5.4 Ablation Study
- Removing confidence distribution learning causes a significant performance decline on both datasets, showing that confidence distributions capture supervision from different confidence levels.The ablation compares the full method with the w/o cdl variant.Source: §5.4 · p. 9, Table 3 · p. 9
- Removing meta self-training also reduces performance, while w/o mst outperforms w/o cdl, indicating labeled-data supervision has greater influence than potential unlabeled-data information.The w/o mst variant trains CDL-RL only with labeled data, whereas the full method uses pseudo labeled data generated by PCDG.Source: §5.4 · p. 9, Table 3 · p. 9
5.5 Low-Confidence Triples Analysis
- The cited passage describes ssCDL as reinforcing UKG embedding learning under an imbalanced confidence distribution, while Figure 3 compares confidence prediction on low-confidence triples in NL27k and CN15k.Source: §5.5 · p. 9, Figure 3 · p. 9
6 Conclusion
The paper proposes ssCDL, combining a CDL-based relational learner with a pseudo confidence distribution generator trained through meta self-training. Experiments show the method performs best on real-world UKG datasets, while future work targets rule learning, reasoning, and large-language-model applications.
6 Conclusion
- ssCDL combines a CDL-based relational learner and pseudo confidence distribution generator trained iteratively through meta self-training.The framework addresses imbalanced triple-confidence distributions in UKG embedding learning.Source: §6 · pp. 9–10
- ssCDL achieves the best performance on real-world UKG datasets compared with state-of-the-art baselines, and ablations verify its component effectiveness.The conclusion attributes the result to the semi-supervised framework and triple confidence distributions.Source: §6 · p. 10
- Future work will study rule learning and reasoning, apply large language models to UKG completion, and use UKGs for reliable retrieval-augmented generation.These are proposed directions rather than evaluated results.Source: §6 · p. 10
A Baselines
ssCDL is compared with state-of-the-art embedding-based UKG completion methods spanning several modeling strategies and established variants.
A Baselines
- The baselines include UKGE, PASSLEAF, UKGsE, BEUrRE, and UPGAT, covering relational, semi-supervised, language-model, box-embedding, and graph-attention approaches.PASSLEAF also has DistMult, ComplEx, and RotatE variants.Source: A Baselines · p. 12
B Implementation Details.
The implementation tunes optimization, architecture, and pseudo-labeling parameters separately for NL27k and CN15k, with validation-based evaluation during training.
B Implementation Details
- The search varied batch size, embedding size, margin γ, and MSE weight β, while sensitivity analysis varied σ, pseudo-label weight w_p, and the selection threshold.Adam was used for SGD training.Source: B Implementation Details · p. 12
- NL27k used σ = 0.6, w_p = 0.7, β = 1, α = 0.001, γ = 0.1, threshold 0.03, batch size 4096, and embedding size 128.Validation was conducted every ten epochs, with a maximum of 500 epochs.Source: B Implementation Details · p. 12
- CN15k used σ = 0.6, w_p = 0.3, β = 1, α = 0.001, γ = 0.1, threshold 0.015, batch size 4096, and embedding size 512.Validation was conducted every ten epochs, with a maximum of 300 epochs.Source: B Implementation Details · p. 12
C Parameter Sensitivity
Sensitivity experiments on NL27k and CN15k show that ssCDL performs best at intermediate settings for confidence-distribution spread, pseudo-label weighting, and selection threshold.
C Parameter Sensitivity
- Figure 4 evaluates σ, w_p, and the pseudo-label selection threshold using MAE for confidence prediction and WMRR for link prediction on both datasets.The figure organizes the sensitivity analysis across the two UKG benchmarks and both tasks.Source: C Parameter Sensitivity · p. 12, Figure 4 · p. 13
- σ = 0.6 gives the best results on both datasets, while smaller values limit global feature capture and larger values weaken confidence discrimination.The performance first improves and then deteriorates as σ increases.Source: C Parameter Sensitivity · p. 12
- w_p = 0.7 on NL27k and w_p = 0.3 on CN15k are optimal, indicating that balancing labeled and pseudo-labeled data matters.Performance improves initially and then decreases as w_p grows.Source: C Parameter Sensitivity · p. 13
- The pseudo-label selection threshold is optimal at 0.03 on NL27k and 0.015 on CN15k, because extreme thresholds respectively admit noisy data or restrict high-quality pseudo-label use.Performance follows a first-increasing-then-decreasing trend as the threshold rises.Source: C Parameter Sensitivity · p. 13
D Limitation
The evaluation is limited by the choice of CN15k, whose mostly correct triples provide little reliability distinction between high- and low-confidence facts.
D Limitation
- CN15k offers limited confidence variation because most ConceptNet triples are generally correct, motivating better UKG completion benchmarks in future work.The authors report that their current experiments include CN15k.Source: D Limitation · p. 13