Source-linked AI summary
From Local Recovery to Unified Generation: Discovering Triple Set in Knowledge Graphs via Observation-Anchored Latent Diffusion
Jiaqi Wang, Yisen Gao, Huayang Shi, Junhao Hou, Wengen Li, Jihong Guan, Yichao Zhang, Shuigeng Zhou, Yangqiu Song, Hong Cheng
TL;DR
Triple set prediction seeks to recover missing-triple sets, but subgraph-based methods can fragment relational context and restrict candidate coverage. LatentDiff performs conditional generation in a shared latent space using an autoencoder and observation-anchored diffusion. It achieves the best F_TSP on all three datasets, exceeding the strongest baseline by 5.4 percentage points on average.
Problem
Subgraph-based completion can break cross-subgraph dependencies, discard boundary context, and limit candidate coverage when predicting missing-triple sets.
Source: §1 · p. 1Method
LatentDiff learns a relation-aware residual-quantized latent space and performs observation-anchored diffusion there to recover missing relational information.
Source: §1 · p. 2, §4.1 · p. 3Results
5.4 percentage points: LatentDiff exceeds the strongest baseline on F_TSP on average and achieves the best F_TSP on all three datasets.
Source: §5.2 · p. 5Takeaways & Limitations
The experiments support unified latent generation as an effective approach to triple set prediction.
Source: §6 · p. 8Systematic investigation of scalability to larger knowledge graphs is left for future work.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Triple set prediction (TSP) is to recover missing relational triples in a knowledge graph by predicting the triple set instead of isolated links. Existing approaches predominantly rely on the subgraph-based completion paradigm, where the original graph is first partitioned into local subgraphs which are then independently completed. Such a paradigm suffers from fragmented structural reasoning, limited cross subgraph dependency modeling, and missing predictions across subgraphs. This paper proposes LatentDiff, a novel unified latent generative framework for TSP, which reformulates knowledge graph completion as latent space graph generation. LatentDiff first learns a compact latent representation using a relation-aware residual-quantized knowledge graph autoencoder, which encodes the whole graph into relation-aware latent embeddings, quantizes them with chunked residual codebooks, and reconstructs triples through a factorized decoder. On top of this latent space, we introduce an observation-anchored latent graph diffusion model, where the target latent graph representation is perturbed toward the observed latent. The reverse process recovers missing triples through progressive latent denoising. Experiments on three benchmark datasets show that LatentDiff outperforms the SOTA in terms of F_{tsp} by 5.4 percentage points on average. Code: https://anonymous.4open.science/r/LKGDiff-44A5.
1 Introduction
Triple set prediction recovers missing-triple sets, but subgraph-based completion can fragment cross-subgraph reasoning and context. LatentDiff instead generates in a shared latent space, using a relation-aware autoencoder and observation-anchored diffusion.
1 Introduction
- Triple set prediction recovers a set of missing triples conditioned on an observed graph, unlike conventional methods that predict missing links independently.The task is framed as structured generation and holistic graph completion.Source: §1 · p. 1
- Subgraph partitioning can break cross-subgraph dependencies, discard boundary context, and yield missing or inconsistent predictions after local completions are merged.For example, a relation between nodes in different subgraphs cannot be recovered by isolated local models.Source: §1 · p. 1
- LatentDiff performs completion in a compact shared latent space, allowing triples to be generated without requiring their endpoints to co-occur in a local subgraph.This compresses the generation state rather than restricting the prediction domain through graph partitioning.Source: §1 · p. 2
- LatentDiff uses a relation-aware residual-quantized knowledge graph autoencoder with relational-view attention, chunked residual quantization, and a factorized triple decoder.These components construct a latent space for relational structure and reconstruct triples.Source: §1 · p. 2
- Its observation-anchored latent graph diffusion perturbs target latents toward observed latents, then uses a dual-view denoiser to progressively recover missing relational information.The denoiser captures dependencies along both relation and entity axes.Source: §1 · p. 2
- Extensive experiments show that LatentDiff outperforms state-of-the-art baselines across multiple datasets.The passage does not specify metric values.Source: §1 · p. 2
2 Related Work
Prior TSP work includes dedicated evaluation, LLM-based generation, and discrete diffusion, while many methods predict within partitioned subgraphs. Graph-diffusion research has largely addressed generic graph tasks or link prediction rather than conditional triple-set recovery in multi-relational graphs.
2.1 Triple Set Prediction
- GPHT introduces TSP evaluation protocols and metrics, LLMTSP and SMuRT use LLMs to generate triple sets, and DiffTSP [17] uses discrete diffusion.Subgraph partitioning can separate target entities and fragment relational evidence needed for connecting triples.Source: §2.1 · p. 2
2.2 Diffusion Models for Graphs
- Graph diffusion studies [33] [42] focus mostly on generic graph generation or prediction, while knowledge-graph diffusion methods primarily evaluate link prediction.TSP instead recovers missing relational facts from an observed multi-relational graph.Source: §2.2 · p. 2
3 Problem Definition
A knowledge graph is represented by entities, relations, and factual triples. In triple set prediction, the goal is to recover the target graph's missing triples from an observed incomplete graph.
3 Problem Definition
- A knowledge graph is G = (E, R, T), where triples in T are drawn from E × R × E and encode head, relation, and tail.E and R denote the entity and relation sets.Source: §3 · p. 2
- The observed graph contains Tobs, a subset of target triples Ttar, and the missing set is Tmiss = Ttar \ Tobs.The task is to recover those missing triples.Source: §3 · p. 2
- The predicted graph combines observed triples with the predicted missing set, and predictions already in Tobs are filtered before evaluation against the missing set.The learning objective is for the predicted set to approximate the ground-truth missing set.Source: §3 · p. 3
4 Method
LatentDiff uses a two-stage unified latent-generation framework: a relation-aware residual-quantized autoencoder builds a reconstructive KG representation, then observation-anchored diffusion generates the target latent for triple decoding. Its encoder, quantizer, decoder, and denoiser respectively model relational context, discretize embeddings, reconstruct triples, and recover missing information.
4.1 Overview
- LatentDiff first learns a compact KG latent space with a relation-aware residual-quantized autoencoder, then generates the target latent conditioned on the observed latent.The forward corruption uses the observed representation as its mean-reversion reference, and iterative latent recovery yields a continuous estimate for triple decoding.Source: §4.1 · p. 3
4.2.1 Variable-Density KG Construction.
- Stage I randomly sparsifies the training graph at each iteration and trains the autoencoder to reconstruct that same masked graph.The sparsification ratio is sampled uniformly from 0 to 1.Source: §4.2.1 · p. 3
4.2.2 Relation-aware Graph Encoder.
- Relational view attention models dependencies among relation-specific embeddings for each head entity, with a residual connection preserving the projected representation.Each relation-specific embedding is a projection of that head-relation pair’s sparse tail-indicator vector.Source: §4.2.2 · p. 3
4.2.3 Chunked Residual Vector Quantization.
- Chunked residual vector quantization divides each latent embedding into chunks and progressively quantizes them with residual codebooks.The quantized chunks are concatenated to form the latent embedding used by the decoder.Source: §4.2.3 · p. 3
4.2.4 Factorized Triple Decoder.
- The factorized decoder reconstructs triples from quantized latents, while Stage I training combines triple reconstruction and vector-quantization objectives.The reconstruction loss preserves information needed to reconstruct triples; the VQ loss aligns continuous encoder outputs with the discrete residual-codebook space.Source: §4.2.4 · p. 3, §4.2.5 · p. 4
4.3.1 Latent Priors Construction.
- Stage II encodes observed and target KGs with the trained Stage-I encoder and quantizer, which remain frozen during diffusion training.Training constructs nested graph pairs with T_obs ⊆ T_tar ⊆ T_train.Source: §4.3 · p. 4, §4.3.1 · p. 4
4.3.2 Mean-Reverting Forward Process.
- The mean-reverting forward process perturbs the target latent toward the observed latent, making diffusion residual denoising around the observed representation.At small steps the noisy latent remains close to the target; near the final step it is centered mainly around the observed latent with Gaussian noise.Source: §4.3.2 · p. 4
4.3.3 Dual-view Latent Graph Denoiser.
- The dual-view denoiser predicts the clean target latent from the noisy latent, observed latent, and diffusion step, using attention across relation and entity axes.Relation-wise attention exchanges information among relations for the same head entity; node-wise attention aggregates across head entities under the same relation. Training predicts the clean target latent, and inference iteratively denoises from the observed latent plus noise before decoding candidate triple scores and filtering observed triples.Source: §4.3.3 · p. 5, §4.3.4 · p. 5, §4.3.5 · p. 5
5 Experiments
LatentDiff is evaluated on three benchmark datasets using the established TSP partitions and metrics, with the main comparison favoring LatentDiff and ablations and analyses examining its design and cross-subgraph recovery. The experiments also assess hyperparameters and inference efficiency.
5.1.1 Datasets. / 5.1.2 Evaluation Metrics.
- We evaluate TSP on Wiki79k, Wiki143k, and CFamily using GPHT’s original data partitions and definitions, with JPrecision, STRecall, and F_TSP; F_TSP differs from conventional F1.Source: §5.1.1 · p. 5, §5.1.2 · p. 5
5.3 Ablation Study
- Removing quantization, relational-view attention, factorized decoding, Stage II, mean-reverting or iterative diffusion, or either-axis attention reduces performance.The matched-budget VQ-VAE comparison controls for codebook parameter count; standard Gaussian diffusion lowers the average score to 0.542.Source: §5.3 · pp. 5–7, Table 4 · p. 7
5.2 Main Results
- 5.4 percentage points on average: LatentDiff achieves the best F_TSP on all three datasets, surpassing the strongest baseline.The main results use F_TSP as the primary comparison metric.Source: §5.2 · p. 5, Table 2 · p. 6, Table 3 · p. 6
5.4.1 Effect of Codebook Size. / 5.4.2 Effect of CRVQ Configuration.
- Larger codebooks improve Stage-I reconstruction, while TSP performance saturates at size 1024; more chunks improve reconstruction and moderate residual-codebook counts benefit TSP.The CRVQ configuration study varies chunk count and residual quantization levels.Source: §5.4.1 · p. 7, Figure 3 · p. 7, §5.4.2 · p. 7
5.4.3 Effect of Diffusion Steps.
- 2 diffusion steps perform clearly worse than 10 or 50; F_TSP gains saturate after 50 or 100 steps, with no consistent gain at 200.Source: §5.4.3 · p. 7, Figure 5 · p. 7
5.5 Case Study / 5.6 Analysis of Cross Subgraph Recovery
- LatentDiff recalls representative cross-subgraph test triples that the compared subgraph-based methods do not generate; partition fragmentation varies by dataset.Wiki79k has 5.43% uncovered training triples and 21.40% partition-boundary test triples, while CFamily has 1.02% boundary test triples.Source: §5.5 · p. 7, Table 6 · p. 8, §5.6 · p. 8
5.7 Efficiency Study
- LatentDiff has faster inference than DiffTSP and GPHT under the same hardware environment, while head-wise chunking reduces peak latent memory without graph partitioning.Source: §5.7 · p. 8, Table 8 · p. 8
5.6 Analysis of Cross Subgraph Recovery
- On partition-boundary triples, GPHT and DiffTSP recover none, while LatentDiff reaches STRecall_bd of 0.229 on Wiki79k and 0.195 on Wiki143k.Boundary evidence may be sparse or structurally distant, and the threshold is selected by overall validation F_TSP rather than boundary-specific performance.Source: Table 7 · p. 8, §5.6 · p. 8
6 Conclusion
LatentDiff uses a two-stage conditional latent generative framework for TSP, completing the graph in a globally shared latent space rather than through subgraph-level prediction. The appendix describes its dual-view attention and complexity-management strategies.
6 Conclusion
- LatentDiff learns a relation-aware residual-quantized latent space, then uses observation-anchored diffusion to recover missing relational residuals in a globally shared space.The conclusion reports state-of-the-art performance and leaves systematic scalability investigation on larger knowledge graphs for future work.Source: §6 · p. 8
A.1 Dual-view Linear Attention
- Relation-wise attention operates over relation tokens per entity, while node-wise attention operates over entity tokens per relation, using independent projections.Both branches use multi-head attention and concatenate their outputs.Source: §A.1 · p. 10
A.2 Complexity Analysis
- Latent generation uses O(NRd) memory instead of O(N^2R) raw-adjacency memory, and head-wise chunking reduces active memory without partitioning the graph.Chunks share the encoder, codebooks, denoiser, and decoder; key-value statistics can preserve global node-wise interactions across chunks.Source: §A.2 · p. 10
A.3 Stage-I Reconstruction Fidelity Metrics
The appendix describes how Stage-I reconstruction fidelity is evaluated and how comparison baselines are selected. Reconstruction diagnostics use test-containing evaluation graphs, while TSP baseline thresholds are selected on validation data.
A.3 Stage-I Reconstruction Fidelity Metrics
- Stage-I reconstruction scores are diagnostic: the encoder receives an evaluation graph containing test triples, but these inputs and scores are excluded from TSP training and hyperparameter selection.Training triples are removed from the reconstructed set before reconstruction metrics are computed against test triples.Source: §A.3 · p. 10
B.1 Baselines
- The comparison includes TSP systems GPHT, LLMTSP, SMuRT, and DiffTSP, alongside link-prediction methods modified to score candidate triples for TSP.For modified link-prediction methods, a validation-set grid search selects the threshold.Source: §B.1 · p. 11
B.2 Evaluation Metrics
The appendix specifies the TSP metrics, validation-based threshold selection, and implementation details. F_TSP is the primary comparison metric, and the same threshold-selection protocol is applied to LatentDiff and its baselines.
B.2 Evaluation Metrics
- F_TSP is the harmonic mean of JPrecision and STRecall and is the primary comparison metric; JPrecision and STRecall are reported as complementary metrics.JPrecision accounts for predicted-set size, while STRecall measures test-triple recovery with a square-root transformation.Source: §B.2 · p. 11
B.3 Triple Set Decoding and Threshold Selection
- Each method’s scores are thresholded over candidate triples after observed training triples are filtered out, with the threshold chosen on validation data for each method and dataset.The selected threshold is then fixed for test-set prediction, applying the protocol consistently across LatentDiff and the baselines.Source: §B.3 · p. 11
B.4 Implementation Details
- Experiments use one NVIDIA A100 GPU, Adam, and validation-set grid search; head-wise chunking reduces peak memory without partitioning the graph into subgraphs.The chunks process latent tensors along the head-entity dimension.Source: §B.4 · p. 11
B.5 Ablation Study Details
The Stage-I ablations compare reconstruction components and show that reconstruction fidelity alone does not determine TSP performance. The Vanilla AE has slightly higher reconstruction fidelity but substantially worse TSP performance.
B.5 Ablation Study Details
- Replacing CRVQ with VQ-VAE causes the largest drop in reconstruction fidelity, while removing relational-view attention or replacing the factorized decoder also reduces reconstruction quality.The VQ-VAE comparison matches total learnable codebook parameter budget and keeps the encoder, decoder, latent dimensionality, and training protocol identical.Source: §B.5 · p. 11
- The Vanilla AE achieves slightly higher reconstruction fidelity but substantially worse TSP performance, indicating that reconstruction fidelity alone is insufficient for effective latent generation.The paper describes quantization’s regularizing role as consistent with this contrast; reconstruction fidelity is not a strict upper bound because Stage II decodes continuous latent predictions without re-quantization.Source: §B.5 · p. 11
B.6 Sensitivity to the Observed Ratio
The appendix examines how the observed-triple ratio and CRVQ configuration affect LatentDiff’s performance. Results favor a moderate observed ratio and show improvements as either CRVQ configuration size increases.
B.6 Sensitivity to the Observed Ratio
- Performance is relatively stable across observed ratios, with the best results around 0.7–0.8; the default ratio is 0.8.The observed ratio is the proportion of triples retained as the conditioning graph.Source: §B.6 · pp. 11–12
- At high observed ratios, fewer target triples remain and supervision on the missing part may be reduced, making the prediction task less informative.Source: §B.6 · p. 12
B.7 Effect of CRVQ Configurations
- Increasing either the number of chunks or residual codebooks improves Stage-I reconstruction fidelity and TSP performance on Wiki79k.Source: §B.7 · p. 12
B.8 Inference Efficiency and Memory Cost
Inference time increases with the number of diffusion steps, while peak memory remains unchanged because reverse denoising runs sequentially. The efficiency evaluation uses one A100 GPU.
B.8 Inference Efficiency and Memory Cost
- Inference time increases with the number of diffusion steps because DDPM sampling requires iterative reverse denoising.Source: §B.8 · p. 12, Table 11 · p. 12
- Inference efficiency is evaluated on one A100 GPU.Source: §B.8 · p. 12
- Peak memory remains unchanged as diffusion steps increase because reverse denoising runs sequentially without storing activations across steps.Source: §B.8 · p. 12, Table 11 · p. 12