PaperlayerView exact revision on arXiv

Source-linked AI summary

Ontology-Grounded, Reasoner-Verified Benchmarks for Evaluating LLM Reasoning in Scientific AI

Nishtha N. Vaidya, Stephan Grimm, Thomas Hubauer, Thomas A. Runkler

arXiv:2610.00682v1cs.AIcs.LG

TL;DR

LLM reasoning over structured scientific knowledge remains unreliable, while manual benchmark construction scales poorly and LLM-generated benchmarks may reproduce reasoning flaws. The paper automates ontology-grounded MCQ creation with reasoner-verified distractors; six LLMs score 41.1–76.8% against a 25% random-guessing baseline.

  • Problem

    LLM reasoning over scientific ontologies can produce unacceptable factual inaccuracies, while manual benchmark construction is labour-intensive and LLM-generated benchmarks risk hallucinating errors.

    Source: §1 · pp. 1–2

  • Method

    The pipeline generates ontology-grounded MCQs from sufficiently axiomatised OWL 2 ontologies, deriving correct answers from class definitions and verifying distractors with an OWL reasoner.

    Source: §1 · p. 2

  • Results

    Six zero-shot LLMs achieve 41.1–76.8% accuracy, above the 25% random-guessing baseline.

    Source: §4 · p. 8

  • Takeaways & Limitations

    Distractors span semantic categories that support diagnostic evaluation of specific reasoning failures.

    Source: §1 · p. 2

    Operators that guarantee fixed distractor categories remain future work because they must account for ontology entailments and may not yield enough candidates.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Large language models (LLMs) increasingly underpin scientific AI applications that reason over structured knowledge, from biomedical question answering to materials informatics. However, their logical reasoning often falls short, producing factual inaccuracies unacceptable in these settings. Reliable evaluation remains challenging: manual dataset construction scales poorly, and LLM-based generation risks embedding the very flaws it aims to measure. High-quality benchmarks must ground both correct and incorrect labelled examples in explicit background knowledge, formally verifiable by a standard reasoner. We propose a pipeline that automatically generates ontology-grounded multiple-choice question (MCQ) benchmarks from any sufficiently axiomatised OWL 2 ontology, with correct answers grounded in the ontology by design. Distractors are generated by perturbing the right-hand-side class expressions of class definition axioms, and their incorrectness is formally verified by an OWL reasoner via entailment checks. We evaluate the pipeline on three ontologies: Pizza (small, academic), PMDco (complex, materials science), and DOID (large, biomedical), generating 112, 2,491, and 15,216 MCQs respectively. Distractors span four semantic categories from class unsatisfiability to weakened subsumptions, enabling diagnostic evaluation of specific reasoning failures. Items meet natural language quality standards: mean LLM judge scores of 4.02, 4.36, and 3.36 out of 5 confirm fluency, and correct-answer-to-distractor similarity above 0.8 shows that wrong options cannot be dismissed on surface form alone. Six LLMs evaluated zero-shot achieve 41.1-76.8% accuracy, well above the 25% random-guessing baseline, indicating the benchmarks are challenging and discriminative. This work is a step towards more reliable benchmarks for assessing logical reasoning in scientific AI.

1 Introduction

Reliable scientific AI requires LLMs to reason correctly over structured ontologies, yet benchmark construction is difficult to scale and LLM-generated data may inherit reasoning flaws. The paper proposes an ontology-grounded, reasoner-verified benchmark-generation pipeline.

1 Introduction

  • LLM reasoning over scientific ontologies remains unreliable, with factual inaccuracies posing risks for biomedical retrieval, clinical decision support, and materials informatics.Prompt engineering, retrieval-augmented generation, and multi-step reasoning pipelines do not eliminate this problem.Source: §1 · p. 1
  • Useful ontology benchmarks must ground examples in explicit knowledge and express them in logic that a reasoner can verify.Manual expert verification is labour-intensive and does not scale, while LLM generation risks embedding the reasoning flaws being evaluated.Source: §1 · p. 2
  • The proposed pipeline generates ontology-grounded MCQs with correct answers derived from class definitions and distractors verified incorrect by an OWL reasoner.It targets sufficiently axiomatised OWL 2 ontologies and provides both natural-language and symbolic forms.Source: §1 · p. 2

2 Related Works

Prior benchmarks variously lack formal verification of distractors, leave their construction unspecified, reduce questions to named-entity recall, or require costly expert annotation.

2 Related Works

  • Existing benchmarks fail at least one of three requirements: ontology grounding, formal verification of answers and distractors, or scalable automated construction.The surveyed approaches differ in which requirement they do not meet.Source: §2 · p. 2
  • Onto-LAMA derives positive subsumption samples with a reasoner, but its negative samples rely on assumed disjointness and are not formally verified [He et al., 2023].Source: §2 · p. 2
  • LLM-ORBench verifies ontology-derived true examples but does not specify false-example construction, while OntoURL uses structurally related distractors without verifying their incorrectness [Zhang et al., 2025].OntoURL uses named classes as distractors, reducing its task to named-entity recall.Source: §2 · p. 2
  • MALLS produces silver-label NL–FOL pairs, while FOLIO's 1,435 examples required about 980 expert annotation hours and target general FOL over Wikipedia [Yang et al., 2024].Source: §2 · p. 2

3 A Method for Generating MCQs from Ontology Class Definition Axioms

The pipeline turns sufficiently axiomatised OWL 2 class-definition axioms into objective MCQs, using complex class expressions as distractors to test ontological reasoning rather than named-entity recall.

3.1 Generating Concept Distractors by Perturbation

  • Perturbations alter a class filler, cardinality bound, or restriction kind while preserving the surrounding expression; guards and output caps constrain some substitutions.Source: §3.1 · pp. 4–5
  • Reasoner checks classify candidates as contradiction, weakening, non-entailed-satisfiable, or incomparable; equivalent and strengthening candidates are discarded.Source: §3.1 · p. 5

3.2 Verbalisation of Concept Expressions

  • DEEPONTO verbalises expressions deterministically, then an LLM smooths the wording while the underlying description-logic expression anchors its logical content.Source: §3.2 · p. 6

3.3 Metrics

  • Operator count keeps distractors structurally matched, while embedding similarity ranks their smoothed verbalizations; grain-boundary child swaps score 0.976, 0.966, and 0.954.Operator count is a syntactic approximation; description-tree alternatives cover EL rather than full SROIQ expressivity.Source: §3.3 · p. 6, §3.4 · p. 6

3.4 Overall Pipeline

  • Axiom filters, inference expansion, and per-class caps define the working set; only axioms with at least MINDISTR valid candidates proceed.Source: §3.4 · p. 6
  • Category priority selects reasoning types before similarity ranks distractors, and an LLM judge filters assembled MCQs for faithfulness, clarity, and domain relevance.The pipeline generates template and scenario stems and blocklists option-content tokens from scenario wording.Source: §3.4 · pp. 6–7

4 Experiments

Across three ontologies, the pipeline produced ontology-grounded MCQs with reasoner-verified distractors and varied outputs by ontology. Six zero-shot LLMs scored above random guessing, while quality measures indicate fluent, non-obvious options.

Pipeline Productivity

  • 112 Pizza, 2,491 PMDco, and 15,216 DOID MCQs were generated, and every item was grounded in an ontology axiom.Table 2 summarizes pipeline results across the three ontologies.Source: §4 · p. 7, Table 2 · p. 8

Formal Quality

  • Every correct answer was a class definition axiom, and every distractor was verified incorrect by HermiT; all four distractor categories appeared in each ontology.Pizza’s 77 disjointness declarations coincided with a 43.5% contradiction rate, while sparse-disjointness PMDco and DOID were dominated by other categories.Source: §4 · p. 7

Natural-Language Quality

  • Mean distractor similarity was 0.922 for Pizza, 0.810 for PMDco, and 0.906 for DOID, so options could not be rejected on surface form alone.Mean LLM judge scores were 4.02, 4.36, and 3.36 out of 5 for Pizza, PMDco, and DOID, respectively.Source: §4 · pp. 7–8

Expert Check

  • A domain expert’s mean score of 3.33 and the LLM judge’s mean of 4.0 on 20 PMDco items both exceeded the scale midpoint.Their mean absolute difference was 1.33; the authors identify a larger human study with formal inter-rater agreement as future work.Source: §4 · p. 8

LLM Evaluation

  • 41.1–76.8% accuracy across six zero-shot LLMs exceeded the 25% random-guessing baseline, indicating challenging and discriminative benchmarks.The models included frontier, open-source, reasoning, and non-reasoning systems.Source: §4 · p. 8, Table 3 · p. 8

5 Conclusions

The conclusion presents an ontology-grounded MCQ pipeline as a response to limited scalable, logically sound benchmark construction. It reports generated benchmark sizes and identifies derived axioms as a promising extension.

Conclusion

  • The pipeline automatically converts sufficiently axiomatised OWL 2 ontologies into MCQs with class-definition-axiom answers and distractors verified incorrect by an OWL reasoner.It produced 112 Pizza, 2,491 PMDco, and 15,216 DOID items without manual annotation.Source: §5 · p. 8

Future Work

  • The authors identify extending generation to derived axioms as the most promising direction for probing deeper reasoning and separating it from prior knowledge of public ontologies.Initial Pizza experiments using πF2 yielded 191 additional MCQs.Source: §5 · p. 8
  • Future work also includes extending distractor operators, strengthening DeepOnto verbalisation templates, and quantifying verbalisation and smoothing effects on benchmark quality.The conclusion also describes potential use of the benchmarks for diagnosis, fine-tuning signals, and neuro-symbolic supervision.Source: §5 · p. 8

LLM Usage Disclosure

LLMs serve as core pipeline components and are also evaluated as subjects of the study.

LLM Usage Disclosure

  • LLMs support the pipeline and are evaluated; an additional assistant was used only for writing, editing, or formatting, not for the core methodology or research originality.Model roles, parameters, and prompts are described in Section 4 and the appendix.Source: LLM Usage Disclosure · p. 11

Full Pipeline Implementation Details

The implementation fixes operator limits, reasoner settings, and model configurations, with a timeout that excludes overly slow axioms.

Reasoning and Selection

  • The implementation caps axiom and operator candidates, uses a seeded shuffle for tied similarity scores, and skips axioms whose HermiT calls exceed 180 seconds.HermiT 1.3.8 runs through Owlready2 0.50; BFO classes serve as PMDco primitives but not Pizza primitives.Source: Full Pipeline Implementation Details · p. 11

Model Configuration

  • GPT-4.1 performs Stage 2 smoothing and Stage 4 stem generation, while GPT-5.4 judges quality and text-embedding-3-small supports Stage 3 similarity.All models were selected based on availability during the research period and accessed through Azure OpenAI.Source: Full Pipeline Implementation Details · p. 11

Overall Pipeline Algorithm

The overall pipeline is organized into four stages, detailed in Algorithm 1.

Overall Pipeline Algorithm

  • Algorithm 1 details the pipeline’s four stages.The passage identifies the algorithm as the overview of ontology-grounded MCQ benchmark generation.Source: Overall Pipeline Algorithm · p. 11

Concept Distractor Generation Algorithm

The benchmark-generation overview proceeds through formal record generation, verbalisation and smoothing, distractor ranking, and MCQ assembly with quality filtering.

Concept Distractor Generation Algorithm

  • The overview divides benchmark generation into four stages, ending with assembled MCQs that pass a judge-score threshold.Records are retained when mean(scores) meets or exceeds QUALITYTHRESH.Source: Concept Distractor Generation Algorithm · p. 12
  • Formal records are produced by selecting ontology axioms and perturbing and classifying their expressions.The algorithm overview names SELECTAXIOMS and PERTURBANDCLASSIFY as the Stage 1 operations.Source: Concept Distractor Generation Algorithm · p. 12
  • Correct and distractor expressions are verbalised with DEEPONTO and then smoothed by an LLM.The smoothed expressions are stored for subsequent distractor ranking and question assembly.Source: Concept Distractor Generation Algorithm · p. 12
  • Distractors are ordered first by category preference and then by descending similarity within each category before the top k are selected.TOPK controls how many candidates proceed to assembly.Source: Concept Distractor Generation Algorithm · p. 12
  • Template and scenario stems are paired with shuffled options, and an LLM judge filters the assembled questions.The scenario stem is generated from the class label and domain description; accepted items meet QUALITYTHRESH.Source: Concept Distractor Generation Algorithm · p. 12

Smoothing Prompt (Stage 2)

The appendix describes prompt-based smoothing and question construction alongside distractor classification, quality rubrics, benchmark-record schema, and zero-shot model evaluation.

System message

  • The smoothing prompt asks an LLM to produce one fluent sentence faithful to the DL expression, using the expression when the rule-based verbalisation is erroneous or artifact-laden.It gives rules for rendering conjunctions, disjunctions, and restrictions in natural language.Source: Smoothing Prompt (Stage 2) · p. 12, System message · pp. 12–13
  • The distractor algorithm uses HermiT checks to classify perturbed expressions and retains records only when the distractor pool meets MINDISTR.Its categories include contradiction, non-entailed satisfiable, weakening, and incomparable expressions.Source: System message · p. 13

User message

  • The user prompt supplies a class, domain, and optional definition, while the stem instruction forbids hinting at the correct constraint.It requests one concise professional-context question ending with a specified “which of the following” form.Source: User message · p. 14, Stem Generation Prompt (Stage 4) · p. 14

Stem Generation Prompt (Stage 4)

Judge Prompt (Stage 4)

  • The judge scores option faithfulness, stem clarity, and relevance using five-point rubrics.The rubrics distinguish exact preservation of the DL expression, clear non-hinting stems, and domain relevance.Source: Judge Prompt (Stage 4) · p. 14, User message · p. 15

Benchmark Schema

  • Table 4 is identified as the complete schema for each benchmark record.The overview says assembled questions include stems, options, and an answer before quality filtering.Source: Benchmark Schema · p. 16, Table 4 · p. 16

LLM Evaluation Setup

Loading 2610.00682v1…