Source-linked AI summary
APOLO: Automatic Prompt Optimization for Ontology Learning
Huu Tan Mai, Roman Kochnev, Cuong Xuan Chu, Lukas Lange, Heiko Paulheim, Daria Stepanova
TL;DR
Ontology learning from text is challenged by scarce annotated text-ontology pairs and the difficulty of adapting LLMs effectively. APOLO generates such pairs from existing ontologies and optimizes greedy and autoregressive learners with GEPA; experiments on DOID and PO show improvements across nearly all tested combinations, especially for autoregressive learners.
Problem
Ontology learning from text has limited annotated training data, and adapting LLMs to perform the task effectively remains challenging.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The results suggest that combining automatic data generation with prompt optimization is a promising path toward better ontology learners and could extend to multi-agent architectures and arbitrary ontology domains.
Source: §4 · p. 5The evaluation does not penalize redundant axioms, and underlying domain ontologies may have appeared in backbone-model pretraining, potentially inflating performance estimates.
Source: §3 · p. 5
Abstract
from arXiv · showhide
Ontology Learning (OL) from text has advanced with the emergence of Large Language Models (LLMs), but it remains challenging due to the limited availability of annotated training data and the difficulty of adapting LLMs to perform OL effectively. We address this via APOLO - Automatic Prompt Optimization for Ontology Learning, by casting OL as an explicit prompt optimization problem over LLM modules. To obtain training data, we employ a multi-agent system that generates text-ontology pairs from existing expert-curated ontologies. We then propose two ontology learner architectures: a greedy and an autoregressive learner, and optimize both using GEPA, a greedy evolutionary prompt optimizer built on DSPy. Experiments on two ontologies - a biomedical (DOID) and a plant ontology (PO) show consistent improvements after optimization across nearly all model and mode combinations, with autoregressive learners achieving the largest gains. Our results demonstrate that prompt optimization is a viable and lightweight alternative to fine-tuning for OL, and that the autoregressive formulation better captures ontological structure than the greedy approach.
1. Introduction
Ontology learning from text could reduce the expertise-intensive cost of building domain ontologies, but finding effective LLM setups and prompts is time-consuming and costly. APOLO frames ontology learning as prompt optimization and reports improvements across nearly all tested model and method combinations.
1. Introduction
- Manual prompt search and fine-tuning LLM pipelines are costly, motivating automatic prompt optimization as a weight-free alternative.The paper describes automatic prompt optimization as cheaper than supervised fine-tuning or reinforcement learning.Source: §1 · p. 1
- The paper formalizes ontology learning as an explicit optimization problem over LLM prompts using text-ontology pairs.Source: §1 · p. 1
- APOLO proposes greedy and autoregressive learners, both optimized with GEPA, an evolutionary prompt optimizer built on DSPy.Source: §1 · p. 1
- APOLO’s optimization loop evaluates a predicted ontology against ground truth and uses the score and actionable feedback to revise the prompt while keeping the underlying LLM fixed.Source: Figure 1 · p. 2
- Prompt-optimized APOLO improves ontology learning across nearly all tested LLM and method combinations on a challenging benchmark.Source: §1 · p. 2
2. Approach
APOLO optimizes a learner’s prompt against chunk-level text-ontology pairs while holding the underlying model weights fixed. It combines ontology scoring and actionable feedback with greedy or autoregressive chunk learners, with the latter conditioning on prior chunks.
2. Approach
- Ontology learning is framed as maximizing evaluation scores over document-ontology pairs, or over corresponding text chunks and grounded subontologies when chunking is needed.Source: §2 · p. 2
- GEPA iteratively mutates prompt candidates and uses a reflection LLM to propose improvements from failure cases, without updating the learner’s weights.Source: §2 · pp. 2–3
- Greedy learners predict each chunk separately and concatenate the outputs, whereas autoregressive learners also condition each chunk on axioms predicted for preceding chunks.During optimization, the autoregressive learner uses ground-truth preceding axioms as context to prevent error accumulation.Source: Figure 2 · p. 3, §2 · p. 3
- Entity precision and recall assess recovery of classes and individuals, while entailment-based axiom precision and recall assess axiom correctness and completeness.Source: §2 · p. 3
- The score is zero for inconsistent or unparsable predictions and otherwise combines entity and axiom F-β scores using a weighting parameter.Actionable feedback can diagnose inconsistency or parsing errors, missing or unsupported axioms, and missing entities.Source: §2 · pp. 3–4
- Training examples pair text chunks generated from ontology subdomains with corresponding entities and axioms grounded in supporting text quotes.Source: Figure 3 · p. 4
3. Experiments
Experiments use generated text–ontology pairs to optimize learners evaluated on DOID and PO. Prompt optimization improves performance across nearly all model/mode combinations, with the largest reported gain for the autoregressive Qwen3.5-27B learner; the evaluation has limitations involving redundant axioms and possible pretraining exposure.
Setup
- The multi-agent system generates and refines text–ontology pairs from 12 public ontologies; 10 are used for prompt optimization and DOID and PO are held out for testing.Planner and Verifier handle ontology decomposition and consistency checking, while Writer and Critic generate and refine textual snippets.Source: §3 · p. 4
Results
- The strongest baseline on DOID, Bakker et al. (B) at AxP = 0.667, falls to AxP = 0.018 on PO, indicating weak cross-domain generalization.Most baselines have near-zero axiom-level scores, and one times out.Source: §3 · p. 4
- 47% improvement in AxP on DOID for Qwen3.5-27B AR after optimization, from AxP = 0.569 to AxP = 0.838.The optimized learner surpasses all off-the-shelf baselines across most axiom-level metrics.Source: §3 · p. 4, Table 1 · p. 5
- Improvements after GEPA optimization are consistent across nearly all model/mode combinations.The two held-out test ontologies are DOID and PO; Table 1 reports results from a single run.Source: §3 · p. 4, Table 1 · p. 5
Limitations
- The scoring method counts entailed redundant axioms as correct, while possible backbone pretraining exposure to source ontologies could inflate performance estimates.Verbose optimized prompts may also lead to overgenerated predicted ontologies.Source: §3 · p. 5
4. Conclusion
The paper frames ontology learning as prompt optimization over LLM modules and evaluates greedy and autoregressive learners optimized with GEPA on generated text–ontology pairs. On DOID and PO, optimization improves nearly all tested model/mode combinations, with autoregressive learners benefiting most.
4. Conclusion
- The paper formalizes ontology learning as explicit prompt optimization over LLM modules.It proposes greedy and autoregressive learners, both optimized with GEPA using generated text–ontology pairs.Source: §4 · p. 5
- Prompt optimization improves performance across nearly all tested model/mode combinations without weight updates, with autoregressive learners benefiting most.Prompt-optimized Qwen3.5-27B AR surpasses tested off-the-shelf baselines across most axiom-level metrics.Source: §4 · p. 5
- The authors describe combining automatic data generation with prompt optimization as a promising path that scales naturally to multi-agent architectures and arbitrary ontology domains.Future work will explore synthetic ontology generation to reduce data-leakage concerns and more robust metrics for complex OWL 2 constructs.Source: §4 · p. 5