Generative Knowledge Graph Construction: A Review

1 Introduction

• KGC pipelines use task-specific discriminators for entity discovery, linking, relation extraction, and event extraction, but face error propagation and poor adaptability.High-quality knowledge graphs also rely almost exclusively on human-curated structured or semi-structured data.Source: §1 · p. 1
• Generative KGC casts extraction as sequence-to-sequence generation, with tagged linearized triples converted back into structured knowledge.The review notes that this paradigm has shown appealing performance on benchmark datasets.Source: §1 · p. 1
• The review organizes generative KGC work by model generation target and task level, and summarizes recent progress.It also maintains a public repository for research convenience.Source: §1 · p. 2
• The review offers a comprehensive taxonomy, theoretical and empirical analysis of paradigm advantages and disadvantages, and coverage of emerging areas and future directions.Its related-work distinction is a focus on generative KGC rather than broader KGC, multimodal KGC, or text-generation surveys.Source: §1 · p. 2

2.1 Knowledge Graph Construction

• KGC covers extracting structural information from text, including named entities, relations, events, entity links, and completion of missing triples.It is framed as structure prediction from input data to an output structure sequence.Source: §2.1 · p. 2

2.2 Discrimination and Generation Methodologies

• Discriminative methods predict labels or tag sequences, whereas generative methods autoregressively produce linearized triples and can be fine-tuned with sequence-to-sequence models.The review describes maximizing the likelihood of the generated linearized triples using cross-entropy loss.Source: §2.2 · p. 3

2.3 Advantages of the Generation Methods

• Sequence-to-sequence KGC could support universal architectures across tasks and settings; structurally consistent pretraining can facilitate knowledge sharing and generative methods address nested labels and overlapping triples.The review cautions that discriminative and generative methods are not simply superior or inferior, and that paradigm choice depends on task form.Source: §2.3 · pp. 3–4

3.1 Copy-based Sequence

• Copy-based generation copies entities from the input and restricts relations to special tokens, avoiding ambiguous or hallucinative entities.The reviewed work also explores generation order, entity representations, and TOPk copying for document-level extraction.Source: §3.1 · p. 4

3.2 Structure-linearized Sequence

• Structure-linearized outputs serialize extracted structures, with event-schema constraints keeping outputs semantically and structurally legitimate.The paradigm also covers document-level event associations and flexible triplet decomposition for relation extraction.Source: §3.2 · pp. 4–5
• Extensions address nested NER, context-conditioned entity retrieval, and task-agnostic generation for heterogeneous information extraction.The cited work also uses contrastive learning with dynamic attention masking to address unreliable generated sequences.Source: §3.2 · pp. 4–5

3.3 Label-augmented Sequence

• Label-augmented outputs copy input words and mark entities or relations with labels, but can skip gold labels in document-level tasks when targets exceed input length.Natural-language labels can leverage pretrained-model knowledge; the length mismatch limits document-level use.Source: §3.3 · p. 5

3.4 Indice-based Sequence

• Indice-based generation outputs input-token positions directly and encodes class labels as label indices, constraining entity indices to words present in the input.The reviewed applications include overlapping relation tuples, flat, nested, and discontinuous NER, and role-filler extraction.Source: §3.4 · p. 5

3.5 Blank-based Sequence

• Blank-based generation fills role placeholders in event-type templates with arguments, with variants targeting low-resource, cross-lingual, and repeated-role extraction.Document-level event argument extraction can be conditioned on a template and document-level information.Source: §3.5 · pp. 5–6

3.6 Comparison and Discussion

• Blank-based methods have a clear advantage in semantic utilization through fluent manual templates, while indice-based decoding is bounded by input length and some structure-based methods share copy-based decoding space.The comparison treats search space as the vocabulary searched by the decoder.Source: §3.6 · p. 6
• Structure-, label-, and blank-based architectures are judged to offer cross-task migration, whereas blank templates cost more labor; the authors call for universal architectures and more efficient decoding.Most other paradigms rely on linear concatenation rather than complex template design.Source: §3.6 · p. 6

4.1 Theoretical Insight

• Generative KGC’s output-space design must balance optimization speed against semantic use: small output spaces can converge faster but may miss information in labels or text.The review frames this as a trade-off between parametric optimization and semantic utilization.Source: §4.1 · pp. 6–7
• Inference depends on decoding choices, including schema-guided decoding and non-autoregressive parallel generation; these choices affect speed and quality.The review calls for sophisticated, efficient decoding strategies, including guidance from the knowledge graph.Source: §4.1 · p. 7

4.2 Empirical Analysis

• On NYT, structure-based and label-based methods achieve extraction performance similar to discrimination models, while non-autoregressive generation is comparable to advanced discriminative models with better decoding and training efficiency.The review also reports that generative methods improved more vastly in recent years.Source: §4.2 · pp. 7–8, Table 2 · p. 8
• On ACE-2005, generation methods obtain performance comparable to discrimination models on event extraction tasks.The review attributes this partly to structure-based supervision and schema constraints that preserve structural and semantic legitimacy; blank-based templates help PLMs represent task and label knowledge in natural language.Source: Table 3 · p. 8, §4.2 · p. 8

Generation Architecture

• Neuro-symbolic models and architectures such as spiking networks, dynamic networks, ordinary differential equations, and diffusion models are proposed as alternatives to Transformer homogenization.The review links neuro-symbolic designs to interpretability.Source: §5 · pp. 8–9

Generation Quality

• Generation quality could be controlled through strategies spanning control codes, decoding feedback, discriminators, loss and prompt design, retrieval augmentation, editing, and diffusion.Source: §5 · p. 9

Training Efficiency

• To reduce annotation and training costs, the review proposes freezing most model parameters, prompt learning, or knowledge-decoupling interventions to reduce parameter redundancy.Source: §5 · p. 9

Universal Deployment

• A unified model spanning tasks and modalities could make improvements less exclusive to a single task, domain, or dataset and support widespread KGC tasks with one model.The review points to UIE as an example of a unified KGC view.Source: §5 · p. 9

Inference Speed

• Because autoregressive decoding generates tokens sequentially and is not parallelizable, the review recommends faster inference, including non-autoregressive, semi-autoregressive, and adaptive-computation approaches.It notes that non-autoregressive KGC can address exposure bias and overfitting, while semi-autoregressive decoding preserves within-block autoregression for consistency.Source: §5 · p. 9

• The review sees generative KGC as potentially unifying tasks and using semantic information better, with future work drawing on NLG methods and KG structure to guide text generation.Source: §6 · p. 9

• The review omits technical model details and mainly covers work from the past five years, especially papers from major NLP and AI venues.The authors plan to add related work and provide more detailed analysis.Source: §7 · p. 9

• Table 4 organizes existing generative KGC papers chronologically to help newcomers understand the paradigms’ evolution.Source: A Timeline Analysis · p. 16, Table 4 · p. 17
• Each paper’s timeline date uses its first arXiv version when available, or an estimated submission time otherwise.Source: Table 4 · p. 17
• Table 4 color-codes works by paradigm: copy-based, structure-linearized, label-augmented, indice-based, and blank-based sequence methods.Source: Table 4 · p. 17