Ontology-to-tools compilation for executable semantic constraint enforcement in LLM agents

• The listed web address is https://como.ceb.cam.ac.uk/.Source: CoMo GROUP · p. 2

• Structured scientific-literature extraction supports hypothesis generation, data-driven modelling, and synthesis prediction.In reticular chemistry, it can also support synthesis prediction, property estimation, and identifying gaps in chemical space.Source: §1 · p. 6
• Scientific papers report roles, units, and chemical identities inconsistently, while downstream records need normalized values and unambiguous canonical identifiers.The same chemical may appear under multiple names or abbreviations, and experimental conditions may use inconsistent unit forms.Source: §1 · p. 6
• Ontologies specify domain concepts and relations, while LLMs can interpret context and produce schema-shaped records using prompt-based role definitions.Ontologies commonly separate a T-Box of concepts and relations from an A-Box of concrete instances and facts.Source: §1 · p. 6
• Ontology-conformant extraction requires valid types and relations, normalized values and units, and grounded identifiers, often enforced through bespoke pipeline logic or post-hoc processing.This concentrates expert effort in code that must be revised as schemas and scope change.Source: §1 · pp. 6–7
• Tool calling and MCP provide a common interface for exchanging typed inputs and outputs, making it possible to enforce structure and domain constraints during extraction.Source: §1 · p. 7
• The framework compiles ontologies into LLM-callable tools that enforce constraints during extraction, construct knowledge graphs with structured feedback, and generate lexical grounding tools for identifier alignment.The authors demonstrate the framework on metal–organic polyhedra synthesis literature in The World Avatar.Source: §1 · p. 7

• The demonstration uses 30 MOP-related publications as full-text inputs, including tables and supplementary information when available.The task is to convert synthesis literature into grounded, ontology-consistent knowledge-graph instances.Source: §2 · p. 8
• The representation combines OntoSynthesis for synthesis events, OntoSpecies for chemical identities, OntoMOPs for products and CBUs, and OM-2 for quantities and units.Chemical species are linked at the usage-instance level to canonical OntoSpecies entries.Source: §2 · p. 8
• Coordinating procedural knowledge, canonical chemical identity, product structures, and provenance across ontologies makes MOP extraction challenging.Roles, amounts, and qualifiers must remain associated with the appropriate chemical usages and synthesis steps, while CBU derivation connects evidence to product representations and external data.Source: §2 · p. 8
• For each paper, the framework produces a structured synthesis recipe, grounded chemical species instances, and derived CBUs linked to the reported MOP product and synthesis.Species occurrences retain contextual qualifiers such as role, amount, and units; CBU derivation combines extracted evidence with chemical and crystallographic data.Source: §2 · pp. 8–9
• Figure 1 shows one representative synthesis instance as both an A-Box subgraph and a compact canonical slot–value record, integrating ordered steps, step-scoped reagent usage, reported MOP product, yield, representations, and selected canonical species links.Source: §2 · p. 9

• The framework separates ontology-to-tool preparation from document-level instantiation by an agent.Preparation generates aligned prompts, scripts, and MCP tool interfaces; instantiation applies them to create and link graph individuals and relations.Source: §3 · p. 9
• Tool calls return validation feedback, and violations such as missing required fields, type mismatches, or invalid units prompt corrected inputs and retries.Agents may retrieve missing evidence or query external resources before trying again.Source: §3 · p. 9
• The ontology T-Box acts as a machine-readable contract specifying allowed classes, relations, attributes, and constraints.Generated tools define inputs, outputs, and validation behaviour, and are the only way to create or modify structured instances.Source: §3 · p. 9
• During instantiation, agents use tools to construct synthesis steps, usage instances, species links, and product entities, while integrating external chemical and crystallographic resources.The resulting entities are instantiated in a knowledge graph under ontology-defined constraints.Source: §3 · pp. 9, 11
• Figure 1 illustrates the instance output as an A-Box subgraph and a compact canonical slot–value projection of the same synthesis.Source: Figure 1 · p. 10

4.1 Ontology-compiled MCP tools enable semantically valid and content-correct end-to-end graph instantiation

• 0.826 micro-F1 (precision 0.844; recall 0.808) summarizes graph-recoverable performance; synthesis steps lead categories at F1 0.843, while reaction chemicals have precision 1.000 but recall 0.582.A record is recoverable only when its content is instantiated with the required entity types, relations, and fields for fixed SPARQL queries.Source: Figure 3 · p. 13, §4.1 · p. 16
• Macro precision 0.865, macro recall 0.735, and macro F1 0.785 accompany wide per-paper variability, with sparse or uneven evidence associated with remaining errors.Recall variability drives the lower-performing tail in some domains.Source: §4.1 · p. 16

4.2 Constraint feedback improves completeness of instantiated synthesis steps

• Removing constraint feedback substantially lowers synthesis-step F1, mainly through recall loss and fewer complete instances recoverable by the fixed queries.Without feedback, outputs more often have inconsistent or missing step numbers, unnecessary fragmentation, and placeholder or default values.Source: Figure 4 · p. 14, §4.2 · p. 17

4.3 Dominant synthesis-step error modes and improvement priorities

• A small set of synthesis-step fields, including order, action descriptions, and key parameters, concentrates errors; recall-limited fields call for coverage and normalization, while precision-limited fields call for constraints and filtering.Errors include both missing required values and extra values absent from the ground truth.Source: §4.3 · p. 17
• Under a conservative fix-by-field assumption, correcting a few highest-impact fields yields a large share of achievable synthesis-step F1 improvement, with diminishing returns thereafter.The roadmap estimates improvement from correcting ranked fields one at a time.Source: Figure 5 · p. 15, §4.3 · p. 17
• Synthesis-step errors are unevenly distributed across papers, with a minority accounting for a large fraction and targeted analysis complementing field-level fixes.The best–worst paper contrasts suggest reporting detail affects what the system can reliably extract and reconstruct.Source: §4.1 · p. 16, §4.3 · pp. 17–18

5 Discussion

• Ontology-to-tools compilation operationalises ontologies as runtime constraints that shape agent behaviour through interaction, rather than static context or post-hoc validation targets.Source: §5 · p. 18

5.1 Limitations

• Evaluation covers one MOP ontology and document collection; chemical-species extraction has lower recall amid naming variation, while curated ground truth for new domains or schemas is costly.Source: §5.1 · p. 18

5.2 Future work

• Future work will test ontology changes and additional domains, and expand to larger, more heterogeneous corpora while targeting recall-limited tasks without reducing current precision.Source: §5.2 · pp. 18–19

6.1 Preparation stage

• Preparation compiles the T-Box and reusable meta-prompts into a static JSON plan, ontology-specific MCP tools, and prompts encoding soft interpretation rules alongside formal constraints.Source: §6.1 · pp. 19–21

6.2 Instantiation stage

• The ReAct agent follows the plan, incrementally updates persistent Turtle memory, and uses tool validation errors to repair missing or invalid inputs.Source: §6.2 · pp. 21–22

6.3 Grounding stage

• Grounding uses ontology- and endpoint-derived lookup tools to select canonical entities from cached labels, then materializes mappings by rewriting IRIs or adding owl:sameAs links.Source: §6.3 · pp. 22–23

6.4 Preparation of evaluation dataset

• The benchmark contains 30 MOP synthesis and characterisation papers with 6,705 annotated items across synthesis steps, reaction chemicals, characterisation entities, and CBUs.Source: §6.4 · p. 23

6.5 Evaluation metrics and methodology

• Evaluation retrieves ontology instances through fixed SPARQL queries and matches JSON records by slot-level exact match; CBU evaluation is an end-to-end grounding task.Source: §6.5 · pp. 23–24

6.6 CBU derivation for metal-organic polyhedra

• The CBU agent combines CCDC crystallographic parsing, graph-based node and linker classification, PubChem enrichment, and ontology-aligned instantiation in the shared knowledge graph.Source: §6.6 · pp. 24–25

6.7 Models and inference settings

• $5.70 for script and prompt generation and $100.56 for processing 30 papers ($3.35 per article); tool calls use temperature=0 and schema validation.The models assigned were gpt-4.1 for extraction, gpt-4o for instantiation, gpt-5 for CBU derivation and generation, and gpt-4o-mini for the query interface.Source: §6.7 · p. 25

7.1.1 LLM-based agents and ReAct-style reasoning

• ReAct-style agents alternate reasoning and tool actions, using tool observations to refine decisions and partial outputs across repeated cycles.Source: §7.1.1 · p. 28

7.1.2 Tool-calling interfaces and the Model Context Protocol

• MCP standardizes tool and data-source access through machine-readable server capabilities, structured invocations, streaming results, and error reporting across hosts and models.Source: §7.1.2 · p. 28

7.1.3 The World Avatar and its chemistry ontologies

• TWA’s reticular-chemistry stack combines OntoSpecies, OntoSyn, and OntoMOPs to represent species, synthesis procedures, MOPs, and their building units.Source: §7.1.3 · pp. 28–29

7.1.4 LLM-based knowledge instantiation and ontology grounding

• Prior approaches separate schema discovery, instance validation, or ontology grounding from instance creation, whereas this work compiles ontologies into executable interfaces and runtime constraints.Source: §7.1.4 · pp. 29–30

7.2 Supplementary Methods

• The supplementary material expands on dataset curation, evaluation and matching rules, MCP specifications, and runtime iteration, error-handling, and repair policies.Source: §7.2 · p. 30
• The workflow figure shows preparation converting a T-Box and domain-agnostic meta-prompts into ontology-specific MCP tools and task prompts for runtime knowledge-graph construction.Source: Figure 7.6 · p. 31
• The task-specification class diagram represents iterations with KG-building steps, file flows, optional sub-iterations, and MCP tool configurations.Source: Figure 7.8 · p. 33

OM T-Box

• The OM temperature tool looks up the unit, validates the numeric value, and creates linked temperature and unit individuals that satisfy T-Box constraints.The example also shows invalid values returning a validation message.Source: OM T-Box · p. 34

7.2.1 Iteration 1: System Meta Prompt

• The system meta prompt directs iteration 1 to create only top-level entities, link each to its source document, and apply ontology identification and constraint rules.It calls for concise, tool-oriented, domain-agnostic instructions for the MCP agent.Source: §7.2.1 · p. 35

7.2.2 Iteration 1: User Meta Prompt

• The user meta prompt reinforces iteration 1’s top-level-only scope and requires ontology rules, MCP tool guidance, identification handling, and domain-agnostic instructions.It includes tool invocation, IRI management, error handling, placeholder policies, and termination conditions.Source: §7.2.2 · pp. 36–37

7.2.3 Extension Ontology: System Meta Prompt

• The extension system meta prompt requires actionable instructions for comprehensively populating an extension A-Box, reusing main-ontology IRIs, and avoiding fabrication.It directs the prompt generator to extract requirements, including cardinality constraints, from T-Box comments and connect the extension A-Box to the main A-Box.Source: §7.2.3 · p. 37

7.2.4 Extension Ontology: User Meta Prompt

• The extension user meta prompt asks for a stepwise route to extend the main A-Box using MCP tools, reusing existing IRIs and following T-Box requirements without fabricating information.It also requests any tool-specific handling notes required by the T-Box or domain.Source: §7.2.4 · p. 38

7.2.5 Illustrative trace of constraint-triggered repair

• A rejected temperature unit C prompted the agent to normalize it to degree Celsius and retry, producing a valid instance.The ontology allowed degree Celsius and kelvin; the tool returned a structured validation error listing those values.Source: §7.2.5 · p. 40
• The tool returned ok=true and validated=true for a TemperatureCondition instance at 120 °C in reaction_step_3.The output includes an instance IRI beginning twa:TemperatureCondition_0f3a….Source: §7.2.5 · p. 40

7.3 Supplementary Evaluation Results

• Tables 7.4–7.6 report per-paper results against full ground truth for synthesis steps, formula-only chemical building units (CBUs), and characterisation entities, with TP, FP, FN, precision, recall, F1, and aggregate rows.Table 7.2 gives the overall evaluation summary by category.Source: §7.3 · p. 41, Table 7.2 · p. 42
• The benchmark profile covers 30 papers, with ground-truth positives computed as TP+FN from Table 7.2.Source: Table 7.1 · p. 41

7.3.1 Detailed instance visualisation for a UMC-1 synthesis

• The UMC-1 visualisation shows the complete instantiated synthesis subgraph under OntoSyn and OntoMOPs, including typed steps, chemical inputs, and the product entity.Unlike the main-text example, it omits abstraction and condensation; the RDF-derived layout was manually edited for presentation and page fitting.Source: §7.3.1 · p. 41, Figure 7.10 · p. 47