PaperlayerView exact revision on arXiv

Source-linked AI summary

LKD-KGC: Domain-Specific KG Construction via LLM-driven Knowledge Dependency Parsing

Jiaqi Sun, Shiyou Qian, Zhangchi Han, Wei Li, Zelin Qian, Dingyu Yang, Jian Cao, Guangtao Xue

arXiv:2505.24163v1cs.CLcs.AI

TL;DR

Domain-specific KG construction faces hierarchical dependencies, technical specificity, and limited reference knowledge, while existing LLM methods can rely on predefined schemas or external sources. LKD-KGC orders repository documents and uses their hierarchical context to generate schemas for extraction. Across the reported datasets and base LLMs, it improves precision and recall over state-of-the-art baselines.

  • Problem

    Domain-specific corpora present hierarchical knowledge dependencies, dense technical terminology, and limited external references, while existing LLM-based KGC methods can require predefined schemas or external knowledge.

    Source: §1 · pp. 2–3

  • Method

    LKD-KGC orders repository documents using LLM-driven dependency evaluation, builds context-aware summaries, and uses document and summary information to generate entity schemas for triple extraction.

    Source: §1 · p. 3

  • Results

    10%–20% improvements in both precision and recall are reported over state-of-the-art baselines across three domain-specific datasets and two base LLMs.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Experiments on public and private datasets support LKD-KGC’s capability to handle domain-specific complexities while outperforming state-of-the-art methods in precision and recall.

    Source: §5 · p. 14

    The dependency-evaluation module assumes the input repository contains a multi-layered hierarchical structure.

    Source: Dependency Evaluation Module · p. 7

Abstract

from arXiv · show

Knowledge Graphs (KGs) structure real-world entities and their relationships into triples, enhancing machine reasoning for various tasks. While domain-specific KGs offer substantial benefits, their manual construction is often inefficient and requires specialized knowledge. Recent approaches for knowledge graph construction (KGC) based on large language models (LLMs), such as schema-guided KGC and reference knowledge integration, have proven efficient. However, these methods are constrained by their reliance on manually defined schema, single-document processing, and public-domain references, making them less effective for domain-specific corpora that exhibit complex knowledge dependencies and specificity, as well as limited reference knowledge. To address these challenges, we propose LKD-KGC, a novel framework for unsupervised domain-specific KG construction. LKD-KGC autonomously analyzes document repositories to infer knowledge dependencies, determines optimal processing sequences via LLM driven prioritization, and autoregressively generates entity schema by integrating hierarchical inter-document contexts. This schema guides the unsupervised extraction of entities and relationships, eliminating reliance on predefined structures or external knowledge. Extensive experiments show that compared with state-of-the-art baselines, LKD-KGC generally achieves improvements of 10% to 20% in both precision and recall rate, demonstrating its potential in constructing high-quality domain-specific KGs.

1 Introduction

Domain-specific corpora combine hierarchical dependencies, technical specificity, and limited external references, making efficient KG construction difficult. LKD-KGC addresses gaps in prior LLM-based methods by ordering repository documents and generating context-aware schemas; reported results improve precision and recall over state-of-the-art baselines.

1 Introduction

  • Domain-specific corpora pose challenges from hierarchical knowledge dependencies, dense technical terminology, and limited access to external reference knowledge.Manual KG construction is inefficient for massive, complex data and requires domain expertise.Source: §1 · p. 2
  • Prior LLM-based KGC methods face reliance on predefined schemas or external knowledge and may miss cross-document entity relationships.RAP requires a predefined schema, AutoKG relies on public web retrieval, and EDC generates schema from individual documents.Source: §1 · pp. 2–3
  • 10%–20% improvements in both precision and recall are reported over state-of-the-art baselines across three domain-specific datasets and two base LLMs.The framework combines dependency evaluation, schema definition, and triple extraction.Source: §1 · p. 3

2 Related Work

Prior LLM-based KGC includes fine-tuning, schema-guided extraction, and reference-knowledge integration, but each has constraints relevant to domain-specific corpora. The section describes these approaches and their limits, especially dependence on schemas or external knowledge and insufficient modeling of cross-document dependencies.

2 Related Work

  • The related-work overview frames fine-tuning, schema guidance, and reference integration as three LLM-based KGC directions, each with limitations for real-world domain-specific corpora.These approaches build on LLM contextual reasoning capabilities.Source: §2 · p. 3

2.1 KGC based on Fine-tuned Language Models

  • Fine-tuned KGC methods generate triples directly or predict tail entities, but domain adaptation remains a challenge for rapidly updated, sparsely annotated corpora.The section identifies COMET and REBEL as examples of generative approaches.Source: §2.1 · p. 3

2.2 KGC based on Schema Guidance

  • Schema-guided methods use schemas to support extraction and alignment, but some require predefined schemas and EDC derives schema from within-document relationships alone.AdaKGC adjusts schemas but cannot create them from scratch.Source: §2.2 · p. 4

2.3 KGC based on Reference Knowledge Integration

  • Reference-integration methods draw on search engines, public KGs, or domain corpora, but rely on external knowledge sources that may not cover private-corpus terminology and concepts.Examples include SAC-KG, TKGCon, AutoKG, and KBTE; private resources may also lack annotated ground-truth triples.Source: §2.3 · pp. 4–5

3 The LKD-KGC Framework

LKD-KGC prioritizes global knowledge dependencies by processing corpora hierarchically, using context from preceding segments. It is presented as not relying on predefined schemas or external reference knowledge while addressing global context.

Design principles

  • LKD-KGC uses hierarchical knowledge access to prioritize global dependencies and leverage context from preceding corpus segments, without predefined schemas or external reference knowledge.The framework is described as addressing global knowledge context through knowledge-dependency awareness.Source: Design principles · p. 5

Observation

The design motivation rests on latent access order in domain corpora and the value of summarization for condensing entity types. The Prometheus example illustrates both observations.

Observation

  • Prometheus documentation is organized as a directory tree, and the proposed access order moves from Introduction to Concepts and then downstream Best Practices.The sequence is presented as helping establish architecture, understand components and terminology, and then address application cases.Source: Observation · p. 6
  • Summarizing text can condense entity types by emphasizing significant entities and relations while removing redundancy.The Prometheus Metric Types example yields four primary types: Counter, Gauge, Histogram, and Summary.Source: Observation · p. 6

Workflow

LKD-KGC processes a repository through dependency evaluation, schema definition, and schema-guided triple extraction. The example corpus is an internal monitoring system called IMS, for which little public information is available.

Workflow

  • The workflow runs in three stages: determine knowledge-access order, construct an entity schema, then extract triples guided by that schema.Source: Workflow · p. 6
  • Dependency Evaluation orders documents and builds context-aware summaries, while Schema Definition derives and refines entity types and their definitions.Source: Workflow · p. 6
  • The framework illustration uses an internal application monitoring system corpus, codenamed IMS, with almost no publicly available Internet information.Source: Figure 2 · p. 7

Dependency Evaluation Module

The Dependency Evaluation Module first summarizes documents and directories, then recursively determines access order and generates document summaries using prior knowledge. For large repositories, vector retrieval limits the context passed to the model.

Dependency Evaluation Module

  • For unstructured long text without a directory structure, chunks can be treated as documents in a single-layer structure; the described setting assumes a multilayer repository.Source: Dependency Evaluation Module · p. 7
  • The module first summarizes individual documents without cross-document references, then aggregates summaries bottom-up into directory summaries containing key entities and relationships.Source: Dependency Evaluation Module · pp. 7–8
  • It traverses the hierarchy top-down to assess access order, recursively evaluates subdirectories, and incorporates prior knowledge into summaries of subsequently accessed documents.This autoregressive process progressively builds a knowledge repository through ordered knowledge integration.Source: Dependency Evaluation Module · p. 8
  • For large repositories, it retrieves the top 10 semantically nearest summaries by default, adapting referenced context to the model’s context-length limit.Document summaries are embedded and stored in a vector database; new documents serve as retrieval queries.Source: Dependency Evaluation Module · p. 8

Schema Definition Module

The Schema Definition Module extracts entity types from documents and contextual summaries, clusters and deduplicates them, then uses relevant summaries to produce definitions for the final schema.

Schema Definition Module

  • The module extracts entity types from individual documents using summaries that carry contextual information, then clusters and deduplicates the types to form the schema.Source: Schema Definition Module · p. 8
  • It filters out entity types appearing only once, embeds the remaining types, and groups semantically similar ones with K-means clustering.Source: Schema Definition Module · pp. 8–9
  • The cluster count k is selected automatically using the silhouette coefficient, where a higher score indicates stronger within-cluster cohesion and clearer separation.The coefficient uses intra-cluster distance a(i) and nearest-cluster distance b(i).Source: Schema Definition Module · p. 9
  • The LLM deduplicates semantically equivalent types within each cluster and defines them using relevant summaries retrieved from the summary knowledge base.The resulting schema contains refined entity types with definitions.Source: Schema Definition Module · p. 9
  • In the illustrated prompt, entity types such as Configuration and Configures are merged and defined using reference knowledge from a related guide summary.Source: Schema Definition Module · p. 9

Triple Extraction Module

The Triple Extraction Module uses the entity schema to extract valid entities from each document, then identifies relations among those entities to form triples. It does not define relation types because their possible number grows rapidly with the entity types.

Triple Extraction Module

  • The module extracts valid entities under the schema first, then identifies potential relations among them to construct the knowledge graph.The example prompt requests triples in subject–predicate–object form.Source: Triple Extraction Module · p. 10
  • Because potential relationship types between entity types grow exponentially, defining all relation types is impractical; the Triple Extraction Module therefore extracts valid entities first, then potential relations based on them.Source: Triple Extraction Module · p. 10

4 Experiments

Across three datasets and two base LLMs, LKD-KGC outperforms three unsupervised LLM-based baselines in precision and recall. The evaluation spans public and private domain corpora.

4 Experiments

  • Across three datasets and two base LLMs, LKD-KGC outperforms state-of-the-art methods in both precision and recall.The experiments compare it with three LLM-based methods that unsupervisedly extract knowledge graphs.Source: §4 · p. 10

Datasets

  • Tests used Prometheus (62 documents), 15 Windows Re-DocRED documents, and 46 internal IMS documents, comparing EDC, AutoKG, and KBTE with LLM-judged precision.Prometheus and IMS recall used true triples per document; Re-DocRED used semantic-equivalence judgments against ground-truth triples.Source: Datasets · pp. 10–11

Public Corpora

  • LKD-KGC reduces false positives on Prometheus by extracting schema-valid entities first, then discovering relationships among them.The corpus contains extensive technical terminology; LKD-KGC extracted 6,580 valid entities, averaging 110.5 per document, and achieved an advantage in recall number.Source: Public Corpora · p. 12
  • F1 approached 30 on Re-DocRED, where LKD-KGC significantly outperformed all baselines in precision and recall.The evaluation used the Microsoft Windows subset; the passage contrasts this with LLM F1 scores around 20 reported for the dataset.Source: Public Corpora · pp. 12–13, Table 2 · p. 12

Private Corpora

  • LKD-KGC achieved superior precision and recall on IMS, identifying 41 entity types in the less extensive corpus with simpler concepts.It extracted 4,984 valid entities; its document-access ordering supports more efficient use of knowledge contexts, while AutoKG's online retrieval introduced irrelevant concepts and reduced accuracy.Source: Private Corpora · p. 13, Table 3 · p. 13

5 Conclusion

LKD-KGC constructs domain-specific knowledge graphs without supervision by inferring hierarchical document-processing order and generating schemas from cross-document context. Experiments on public and private datasets show superior precision and recall against state-of-the-art methods.

  • LKD-KGC infers hierarchical corpus-processing order and integrates cross-document contexts to generate entity schemas, then demonstrates superior precision and recall on public and private datasets.The framework is proposed for unsupervised domain-specific knowledge graph construction.Source: §5 · p. 14

Related papers

Loading 2505.24163v1…