Source-linked AI summary
LLM-Based Knowledge Graph Completion Combining Discrete Structural Coding with Similar Entity Information
Jiaqi Wang, Dongying Lin, Yang Yang, Yinan Liu, Bin Wang, Xiaochun Yang
TL;DR
LLM-based KGC must use textual and relational information, while prior methods have largely separated discrete structural coding from candidate refinement. CoSC combines these directions by generating an initial code-based ranking and refining it with similar-entity information, outperforming existing baselines on MRR and Hits@10 on FB15k-237 while remaining competitive on Hits@1.
Problem
Existing LLM-based KGC methods have largely studied discrete structural coding and restricted candidate refinement separately, despite KGC requiring textual and relational information.
Method
CoSC uses an LLM to rank retrieved candidates from their names and discrete structural codes, then refines that ranking with relation-specific evidence from structurally similar entities.
Results
CoSC outperforms existing baselines on MRR and Hits@10 while remaining competitive on Hits@1 on FB15k-237.
Takeaways & Limitations
Integrating discrete structural coding with similar entity information yields better KGC performance on MRR and Hits@10 while preserving competitiveness on Hits@1.
Abstract
from arXiv · showhide
Knowledge graph completion requires models to use both textual descriptions and relational structure. Existing LLM-based methods either encode KG structure as discrete tokens or refine a restricted set of candidate entities, and these two directions have largely been studied separately. We propose CoSC for LLM-based KGC, which combines discrete structural coding with similar entity information. Specifically, an LLM generates an initial candidate entity ranking from discrete structural codes, after which information from entities with structures similar to that of the query entity refines the ranking. Experiments on FB15k-237 show that CoSC outperforms existing baselines on MRR and Hits@10 while remaining competitive on Hits@1.
1. Introduction
LLM-based KGC methods have separately emphasized discrete structural coding or candidate-constrained prediction. CoSC combines both by using similar-entity information to refine an initial code-based ranking, improving key benchmark metrics.
- LLM-based KGC methods mainly either represent entity structure as discrete tokens or restrict prediction to retrieved candidate entities.The first direction determines how KG information is represented, while the second adds retrieval, filtering, verification, or structural evidence to prediction.
- CoSC integrates discrete entity codes with information from entities structurally similar to the query entity.This combines the two previously separated directions for LLM-based KGC.
- AdaProp first retrieves ranked candidates, and an LLM generates an initial ranking from candidate names and discrete structural codes.CoSC then uses similar entities' relation-specific information to refine that ranking.
- Similar entities provide local evidence for refining candidates when their structures resemble the query entity's structure.The method uses relation-specific information from those entities during ranking refinement.
- CoSC outperforms existing baselines on MRR and Hits@10 while remaining competitive on Hits@1 on FB15k-237.
2. Related Work
Related work has progressed from text-focused KGC toward methods that connect graph structure with language models and candidate-constrained reranking. These approaches motivate combining structural representations with restricted candidate decisions.
- Early language-model-based KGC methods focused on textual descriptions of entities and relations.KG-BERT converts triples into text, while SimKGC learns text-based entity representations through contrastive training.
- Later methods explicitly connect graph representations to language models through prefixes, graph adapters, or cross-layer alignment.KoPA uses knowledge graph embeddings as learnable prefixes, KG-Adapter introduces graph adapters, and MAKI aligns linguistic and structural representations.
- Candidate-constrained methods reduce open-ended generation to decisions over retrieved entity sets.Their pipelines use graph-aware contexts, candidate knowledge, structural filtering, retrieval, reasoning, reranking, embedding filtering, or LLM verification.
- OMNIA combines graph-derived candidate triples, embedding-based filtering, and LLM verification of remaining alternatives.
3. Method
CoSC ranks retrieved candidates with discrete structural codes, then refines the ranking using neighborhood evidence from structurally similar heads. The final score combines both signals without expanding the candidate set.
- Method: CoSC has two components: candidate ranking via discrete structural codes and ranking refinement using similar entity information.Figure 1 presents the complete procedure.
- Candidate Ranking via Discrete Structural Codes: For query q=(h, r, ?), AdaProp retrieves an ordered candidate list, and each candidate is represented by its name plus Lcode tokens.The query, names, and codes are provided to an adapted Meta-Llama model.
- Candidate Ranking via Discrete Structural Codes: The LLM generates up to Ptarget code sequences; exact code matches move candidates to the front while unmatched candidates retain AdaProp's relative order.This updates leading positions without discarding the retrieval ranking of remaining entities.
- Ranking Refining using Similar Entity Information: CoSC selects M similar heads by shared outgoing relation–tail pairs and uses their query-relation triples as structural support.The evidence is constructed from the training graph with inverse relations included.
- Ranking Refining using Similar Entity Information: Candidates receive greater neighborhood support when several similar heads connect to them through the query relation.The accumulated support is normalized to limit the effect of large values.
- Ranking Refining using Similar Entity Information: CoSC combines code-based rank and normalized neighborhood support, sorts candidates by the resulting score, and reorders rather than expands the retrieved set.The neighborhood weight α is selected on validation data.
4. Experiments
CoSC is evaluated on all 22,850 FB15k-237 test queries using MRR, Hits@1, and Hits@10. Retrieval covers 79.72% of queries, and the complete procedure improves MRR and Hits@10 over existing baselines while remaining competitive on Hits@1.
- 22,850 FB15k-237 test queries are evaluated with MRR, Hits@1, and Hits@10.
- MRR 0.5021, Hits@1 0.3674, and Hits@10 0.7434 are obtained across all test queries.These values measure the complete candidate retrieval and reranking procedure.
- CoSC outperforms existing baselines on MRR and Hits@10 while remaining competitive on Hits@1.
5. Conclusion
CoSC integrates discrete structural coding with similar entity information for LLM-based knowledge graph completion. On FB15k-237, it outperforms existing baselines on MRR and Hits@10 while remaining competitive on Hits@1.
- CoSC integrates discrete structural coding with similar entity information for LLM-based knowledge graph completion.
- On FB15k-237, CoSC outperforms existing baselines on MRR and Hits@10 while remaining competitive on Hits@1.
- Future work will evaluate CoSC on additional datasets and under fully controlled baseline settings.
Declaration of use of Generative AI
The authors used OpenAI Codex for language and formatting assistance, then reviewed and edited the content. The tool did not generate scientific figures or experimental results.
- OpenAI Codex was used to paraphrase and reword author-prepared text and assist with section restructuring.
- The tool also assisted with writing style, grammar, spelling, and LaTeX formatting.
- The authors reviewed, verified, and edited the content after using the tool and retained responsibility for the publication.
- No scientific figures or experimental results were generated using OpenAI Codex.