Agentic schema-guided extraction of materials process knowledge from scientific literature

1 Introduction

• Materials records are difficult to reuse computationally when information is scattered across document elements, chemical forms vary, and measurements lack process-specific context.Quantitative values must remain attached to the correct process instance, quantity kind, and unit.Source: §1 · p. 2
• Rubric-based judges identify unsupported or missing information and turn those assessments into actionable revision instructions rather than terminal scores.The approach uses an extract → reflect → feedback → refine loop for deeply structured records.Source: §1 · pp. 2–3
• ALD tests the workflow with records spanning precursor chemistry, process conditions, film properties, and device properties, including complex IGZO supercycles.IGZO deposition may combine In2O3, Ga2O3, and ZnO subcycles whose ordering and ratios affect composition and electronic properties.Source: §1 · p. 3
• The study poses five questions covering chemical canonicalization, agentic verification, deep-schema extraction, transfer beyond ALD, and quality–cost trade-offs.It evaluates exact precision, recall, and F1 because textual resemblance alone cannot validate chemically, numerically, or relationally incorrect values.Source: §1 · pp. 3–4
• SciKGExtract combines schema-guided extraction, chemical normalization, rubric-based evaluation, and feedback-driven refinement for experimental process records.The framework also supports direct extraction, self-refinement, multi-judge refinement, and evaluator–critic debate.Source: §1 · pp. 3–4
• The contributions include comparisons across extraction models and agentic configurations, an expert-curated full-schema ZnO and IGZO evaluation, and transfer and cost assessments.The authors also report that more judges or elaborate interaction do not guarantee better extraction quality.Source: §1 · p. 4

2.1 A deep experimental schema changes the extraction problem

• The full-schema task represents one ALD process through 65 leaf properties and 155 QUDT-grounded quantity nodes, requiring correct process and property assignments.Quantities retain quantity kind, numerical value, and unit.Source: §2.1 · p. 4, Figure 1 · p. 5

2.2 Chemical normalization improves exact agreement across models

• PubChem normalization improved F1 for every tested model in both systems; best normalized exact F1 was 0.805 for ZnO and 0.344 for IGZO.The comparison evaluates 162 ZnO and 14 IGZO papers on deposited material, precursor, and co-reactant.Source: §2.2 · p. 5, Table 1 · p. 7

2.3 Agentic refinement helps, but more agents are not always better

• 0.805 normalized exact-match F1 for ZnO under debate exceeded direct extraction's 0.591, while IGZO peaked at 0.344 with two judges.For IGZO, debate with GPT-5 and DeepSeek R1 reached 0.331, below the two-judge result.Source: §2.3 · p. 6, Table 2 · p. 8

2.4 Expert evaluation exposes the difficulty of full experimental reconstruction

• Reflection improved some full-schema scores but not uniformly, while coverage analysis found over-generation and inconsistent recovery of expert-annotated process instances.IGZO Mistral Large rose from 0.20 to 0.32 F1, whereas GPT-4o fell from 0.48 to 0.45; one-shot outputs averaged about 75 populated IGZO properties versus 61.8 in expert annotations.Source: §2.4 · p. 8, Table 3 · p. 9

2.5 Transfer experiments test portability beyond ALD

• 0.2834 BioRED and 0.7253 BC5CDR exact F1 were the best observed transfer results, with different refinement strategies used for the two benchmarks.The orchestration can be reused by changing the external schema and typed output model rather than rewriting ALD-specific extraction logic.Source: §2.5 · p. 9, Table 4 · p. 9

2.6 Extraction quality is not proportional to model cost

• Llama 3.3 70B produced the best normalized ZnO result while remaining among the lowest-cost extractors; judge-heavy workflows added calls without guaranteed gains.Average extractor request costs ranged from approximately $0.00061 for Gemma 4 26B to $0.020 for Claude Sonnet 4.6 in the executed runs.Source: §2.6 · pp. 9–10

3 Discussion

• Schema, chemical normalization, and agentic reflection address distinct error sources and are complementary rather than interchangeable.The schema defines what can be represented; normalization resolves chemical surface-form variation; reflection targets unsupported, missing, or structurally inconsistent content.Source: §3 · p. 10
• Identifier linking improves agreement across tested models by resolving equivalent chemical names or formulas that otherwise fail exact matching.The discussion recommends integrating identifier linking into extraction rather than postponing it to database cleaning.Source: §3 · p. 10
• Agentic verification gains vary: ZnO improves through debate, while IGZO and some full-schema evaluations do not benefit from additional refinement.Reflection is most useful when evaluators identify correctable errors and provide feedback aligned with structured correctness, process identity, and property assignment.Source: §3 · pp. 10–11
• IGZO’s multicomponent subcycles, supercycle ratios, and composition-dependent measurements make process reconstruction more difficult; its best exact F1 after normalization is 0.344.The discussion links this difficulty to long-range document interpretation and assigning properties to the appropriate process instance.Source: §3 · p. 11
• Full-schema evaluation exposes process segmentation, nested-property alignment, and dense numerical assignment as difficulties beyond identifying materials and reactants.Expert annotation also revealed an additional ZnO process only after visual information was considered; the study is text-centred and identifies multimodal evidence integration as a next step.Source: §3 · p. 11
• The deep-schema gold standard covers two papers and 13 text-derived process instances, limiting variance estimates and property-level error analysis.The discussion also recommends exact structured evaluation, canonical identifiers, process-level deep-schema evaluation, and empirically selective verification for interoperable materials knowledge bases.Source: §3 · p. 11

4.1 SciKGExtract architecture

• An orchestrator coordinates Extract & Normalize, Reflect, Feedback, and KG Integration while maintaining shared article, schema, model, candidate, evaluation, and feedback state.Refinement strategies mainly vary the configuration of evaluation and feedback, not the extraction target or overall pipeline.Source: §4.1 · p. 12
• Extract & Normalize generates schema-conforming process instances, then parses and validates typed outputs, cleans empty structures, and optionally enriches selected values with external identifiers.The schema governs property names, types, nesting, and cardinality; optional few-shot examples demonstrate how evidence maps into the structure.Source: §4.1 · p. 12
• Reflect evaluates correctness and completeness, and Feedback turns judge ratings and rationales into revision instructions for re-evaluation.Depending on the experiment, evaluation uses a single judge, multiple independent judges, or an evaluator–critic configuration; candidates can be revised before acceptance.Source: §4.1 · p. 12
• KG Integration prepares accepted process records for graph-oriented resources, while configuration allows pipeline stages and model providers to vary independently.Integration is separated from extraction and quality control so only records that pass configured validation, normalization, and reflection proceed downstream.Source: §4.1 · p. 12

4.2 ALD schema and task representation

• The expert-validated ALD schema represents each process through nested system, reactant, parameter, material-property, device-property, and experimental-information descriptions.Its programmatic target has 65 leaf properties and 155 QUDT-grounded quantity nodes.Source: §4.2 · p. 13
• A correct extraction must identify process instances and assign observations to the right schema branch, since textually correct values can be structurally misplaced.For example, deposition temperature belongs under process parameters, while mobility belongs under device properties.Source: §4.2 · p. 13

4.3 ALD literature corpus

• The core-property evaluation uses 176 AtomicLimits-linked papers: 162 ZnO papers and 14 IGZO papers.AtomicLimits manually curates deposited-material and reactant information from ALD literature and major reviews.Source: §4.3 · p. 13
• ZnO represents a mature binary-oxide system, whereas IGZO descriptions may contain multiple subcycles and supercycle ratios.PDFs were converted to machine-readable text and tables with Docling and stored as Markdown for SciKGExtract.Source: §4.3 · p. 13

4.4 Chemical normalization

• Normalization targets surface-form variation because the same material or reactant may appear as a systematic name, formula, abbreviation, or author-specific variant.Such forms can denote the same entity while making computational reuse difficult.Source: §4.4 · p. 14
• The normalizer standardizes a form and queries PubChem lookup components, using a cache and local synonym index for efficient resolution and fallback.An LLM may propose an alternative representation for another lookup attempt when identifier resources do not resolve the original form.Source: §4.4 · p. 14
• Resolved values retain the original extracted string and gain an external identifier, separating naming variation from errors such as wrong-reactant selection or process assignment.This addresses exact-match disagreement between strings denoting the same chemical entity without replacing the source wording.Source: §4.4 · p. 14

4.5 Refinement strategies and models

• Four workflow settings are compared: direct extraction, self-refinement, multi-judge refinement, and debate refinement.Direct extraction uses one schema-guided pass; the other settings add feedback through self-evaluation, independent judges, or evaluator–critic pairs.Source: §4.5 · p. 14
• The 176-paper ALD comparison tests five extraction models and uses distinct judge and critic configurations for refinement.The extractors are GPT-4o, Llama 3.3 70B, Mistral Large, Gemma 4 26B, and Ministral 3B; GPT-5 and DeepSeek R1 serve as two-judge models, with Qwen3 30B Thinking added in the three-judge setting.Source: §4.5 · p. 14
• A separate controlled model sweep measures reconstruction of the complete 65-leaf-property schema rather than ranking the newer model set.The sweep uses GPT-4o, Mistral Large, Qwen3 30B, and Qwen3 235B.Source: §4.5 · p. 14

4.6 Expert full-schema annotation

• Two representative ALD studies were annotated into eight thermal-ALD ZnO and five plasma-enhanced-ALD IGZO process instances.The text-based annotations cover reactor conditions, reactant parameters, cycle structure, growth characteristics, composition, and device properties; an additional figure-derived ZnO process is excluded from the text-only evaluation.Source: §4.6 · pp. 14–15

4.7 Transfer benchmarks

• BioRED and BC5CDR test framework portability by changing the target schema and typed output model while keeping SciKGExtract orchestration unchanged.The experiments assess portability, not direct state-of-the-art performance against systems trained under different assumptions [40, 41].Source: §4.7 · p. 15

4.8 Evaluation metrics

• Exact precision, recall, and F1 are primary metrics, with correctness requiring a match to the gold-standard representation under evaluation normalization rules.The paper does not base its conclusions on embedding similarity because similar serialized values may not be scientifically equivalent, particularly for numbers and chemical identifiers.Source: §4.8 · p. 15

4.9 Cost measurements

• Per-request costs and token counts are averaged by model and characterize the tested workflows, not fixed price estimates.Provider pricing and token usage vary over time.Source: §4.9 · p. 15

4.10 Use of generative AI in manuscript preparation

• Generative AI supported manuscript language editing, phrasing, and clarity, with the authors reviewing and validating the resulting text.The authors remain responsible for the scientific content.Source: §4.10 · p. 15

Code availability

• SciKGExtract is available as open-source software in the project repository.The passage provides the repository address.Source: Code availability · p. 16
• An archived software release is available through Zenodo.The passage provides the release DOI.Source: Code availability · p. 16
• The repository contains the workflow, normalization components, schemas, prompts, evaluation code, and runnable examples.Source: Code availability · p. 16

• The AWASES project, funded by Merck and Intel, partly supported the work.Source: Funding · p. 21
• The KISSKI AI Service Center supported the large language model services used in the work through German federal funding.The grant number was 01IS22093C.Source: Funding · p. 21