PaperlayerView exact revision on arXiv

Source-linked AI summary

OWL2Vec*: Embedding of OWL Ontologies

Jiaoyan Chen, Pan Hu, Ernesto Jimenez-Ruiz, Ole Magnus Holter, Denvar Antonyrajah, Ian Horrocks

arXiv:2009.14654v2cs.AI

TL;DR

Existing embedding methods largely target knowledge graphs, while expressive OWL ontologies combine graph structure with logical constructors and lexical information. OWL2Vec* extracts multiple ontology views and learns entity and word embeddings, achieving stronger results than established methods on two prediction tasks across three real-world ontologies. Its supervised setting is less effective for zero-shot prediction of new classes.

  • Problem

    Existing knowledge-graph embeddings do not fully address OWL ontologies, which combine graph structure with logical constructors and rich lexical information.

  • Method

    OWL2Vec* walks ontology graph forms to generate structure-logic, lexical, and combined documents, then trains word embeddings for entities and words.

  • Results

    OWL2Vec* significantly outperforms state-of-the-art methods on class membership and class subsumption prediction across three real-world ontologies.

  • Takeaways & Limitations

    OWL2Vec* provides a general framework for embedding multiple OWL semantic aspects and supports ontology-completion prediction tasks.

  • Takeaways & Limitations

    In a zero-shot setting for a new class without known instances, the class-membership recall drops to 65.6% for OWL2Vec*.

Abstract

from arXiv · show

Semantic embedding of knowledge graphs has been widely studied and used for prediction and statistical analysis tasks across various domains such as Natural Language Processing and the Semantic Web. However, less attention has been paid to developing robust methods for embedding OWL (Web Ontology Language) ontologies which can express a much wider range of semantics than knowledge graphs and have been widely adopted in domains such as bioinformatics. In this paper, we propose a random walk and word embedding based ontology embedding method named OWL2Vec*, which encodes the semantics of an OWL ontology by taking into account its graph structure, lexical information and logical constructors. Our empirical evaluation with three real world datasets suggests that OWL2Vec* benefits from these three different aspects of an ontology in class membership prediction and class subsumption prediction tasks. Furthermore, OWL2Vec* often significantly outperforms the state-of-the-art methods in our experiments.

1 Introduction

OWL ontologies express graph structure alongside logical constructors and rich lexical metadata, but existing embedding methods do not fully capture these semantics. OWL2Vec* addresses this gap and outperforms established methods across two ontology-completion tasks on three real-world ontologies.

  • Motivation: OWL ontologies extend RDF-style graph structures with logical constructors such as disjointness and quantification, plus synonyms, definitions, and comments.These ontologies are used in bioinformatics, the Semantic Web, and Linked Data.
  • Motivation: Existing knowledge-graph embeddings generally target RDF triples and do not handle the broader semantics of expressive OWL ontologies.Earlier ontology-schema methods also rely on many facts and do not support several expressive OWL constructors.
  • Approach: OWL2Vec* generates three documents covering graph structure and logic, lexical information, and their combination, then learns entity and word embeddings.The framework uses word embedding over ontology-derived corpora and is compatible with different word-embedding settings.
  • Evaluation: OWL2Vec* was evaluated for class membership and class subsumption prediction using three large-scale real-world ontologies.The case studies use HeLis, FoodOn, and the Gene Ontology, with ablation and visualization analyses also reported.
  • Results: OWL2Vec* outperforms current ontology embedding methods and classic knowledge-graph embedding methods on both ontology-completion tasks.The reported comparisons include state-of-the-art ontology embeddings, RDF2Vec, TransE, and DistMult.

2 Preliminaries

OWL ontologies combine description-logic axioms, instance assertions, graph relations, and annotation metadata. Semantic and ontology embedding methods differ in how they explore graph neighborhoods, logical structure, and lexical information.

  • OWL Ontologies: OWL ontologies are based on SROIQ description logic and contain TBox axioms, ABox assertions, concepts, roles, individuals, and logical constructors.Examples include conjunction, disjunction, existential and universal restrictions, subsumption, role inclusion, and equality assertions.
  • OWL Ontologies: OWL represents concept assertions as class-membership relations and general concept inclusions as class-subsumption relations, while complex axioms can be serialized as RDF triples.These triples may use built-in RDF, RDFS, and OWL properties alongside bespoke object properties.
  • OWL Ontologies: Ontology annotations provide names, descriptions, definitions, and other metadata through annotation properties.Such lexical information is represented using RDF triples and supplements formal axioms and assertions.
  • Semantic Embedding: Word-embedding approaches explore entity and relation neighborhoods before learning embeddings from sequences such as random walks.RDF2Vec adapts this paradigm to directed, labeled RDF graphs for large-scale knowledge graphs.
  • Semantic Embedding: OWL2Vec* targets OWL ontologies rather than typical knowledge graphs and jointly preserves graph structure, lexical information, and logical constructors.The paper notes that existing knowledge-graph embedding methods do not jointly explore these ontology-specific aspects.
  • Ontology Embedding: Earlier ontology embedding methods encode logical axioms or use ontology text, but may omit lexical information or incompletely explore graph relations.OWL2Vec* also creates embeddings for words in lexical information, not only ontology entities.

3 Methodology

OWL2Vec∗ extracts ontology corpora that jointly represent graph structure, logical constructors, lexical information, and their correlations, then trains word embeddings for target entities. It supports RDF graph transformations, entailment reasoning, random walks, lexical processing, and downstream prediction.

  • Framework: OWL2Vec∗ uses corpus extraction and word-embedding training to produce configurable vectors for selected ontology entities.Its corpus contains structure, lexical, and combined documents.
  • From OWL Ontology to RDF Graph: Two ontology-to-RDF strategies either preserve complex OWL constructs through blank-node triples or approximate them with projection rules that avoid blank nodes.The projection strategy reduces possible embedding noise but does not retain exact logical relationships.
  • From OWL Ontology to RDF Graph: Entailment reasoning can classify the TBox and realize the ABox before graph construction, making some hidden knowledge explicit.For example, subclass information can yield an additional instance-membership triple.
  • Structure Document: Structure documents combine random walks over the RDF graph, optional Weisfeiler–Lehman subgraph identities, and Manchester Syntax sequences for ontology axioms.WL kernels encode local subgraph structure into identities used in walks.
  • Lexical and Combined Documents: Lexical documents replace entity IRIs with tokenized English labels and incorporate annotation literals, while combined documents preserve IRI–word correlations.The combined document uses structure sentences as a backbone and supports IRI and word representations.
  • Downstream Prediction: For membership prediction, concatenated instance and class embeddings are classified into a plausibility score, then candidate classes are ranked by normalized scores.Higher scores indicate more plausible membership relations.

4.1 Experimental Setting

The experiments evaluate OWL2Vec∗ on three real-world ontologies using membership and subsumption prediction, with controlled data splits, ranking metrics, baselines, and analyses of modeling choices. The settings also compare classifiers and a Transformer based on entity text.

  • Datasets and Tasks: Three ontologies support two tasks: membership prediction on HeLis, and subsumption prediction on FoodOn and Gene Ontology.The ontologies differ in their available membership and subsumption axioms.
  • Evaluation Protocol: Explicit membership or subsumption axioms are randomly split into 70% training, 10% validation, and 20% testing.Predicted candidates are ranked against the ground-truth class.
  • Evaluation Protocol: Performance is measured with Hits@1, Hits@5, Hits@10, and MRR, where higher values indicate better ranking performance.The Hits metrics measure top-k recall, while MRR averages reciprocal ground-truth ranks.
  • Experimental Analyses: The evaluation compares documents, IRI and word embeddings, graph-walking and transformation settings, reasoning, pre-training, and classifiers.Classifiers include RF, MLP, Logistic Regression, and Support Vector methods.
  • Experimental Analyses: A supervised Transformer classifier is additionally compared with OWL2Vec∗ plus RF using contextual text from head and tail entities.The comparison addresses possible effects of naming surface forms on prediction.

4.2 Comparison with Baselines

OWL2Vec∗ outperforms the evaluated ontology- and KG-embedding baselines, with especially large Hits@1 gains over RDF2Vec and OPA2Vec. Textual information is important, while Transformer-based surface-form modeling is inconsistent across ontologies.

  • OWL2Vec∗ outperforms all evaluated baselines, with statistically significant comparisons at p-value ≪0.05.The comparison includes ontology embedding, KG embedding, and other baseline methods.
  • 325.6%, 146.6%, and 126.7% higher Hits@1 than RDF2Vec on HeLis and OPA2Vec on FoodOn and GO, respectively.
  • 6.0%, 56.6%, and 38.2% higher MRR than pre-trained Word2Vec on HeLis, FoodOn, and GO, respectively.The ontology-tailored corpus combines graph structure and logical axioms with lexical information.
  • Textual information plays an important role because entity and class names often share relevant words, synonyms, or word variants.
  • Transformer classifiers using surface-form contexts perform well on HeLis but much worse than other methods on FoodOn and GO.The results indicate partial predictive information on HeLis and little predictive information on FoodOn and GO.
  • HeLis membership prediction is less challenging than FoodOn and GO subsumption prediction because it has fewer candidate classes.

4.3 Analysis of OWL2Vec∗Settings

OWL2Vec∗ benefits from combining lexical information with structure and from using word embeddings, while graph exploration choices and IRI features have task- and ontology-dependent effects. Reasoning has limited impact in the reported experiments.

  • Lexical Information: 26.9%, 18.8%, and 22.1% higher MRR on HeLis, FoodOn, and GO when the lexical document is merged with the structure document using IRI embeddings.With both IRI and word embeddings, the corresponding gains are 169.7%, 31.8%, and 44.2%.
  • Lexical Information: The best MRRs for combined random or traversal documents are close to the lexical-plus-structure setting for membership prediction but worse for FoodOn and GO subsumption prediction.The reported membership MRRs are 0.951 and 0.953 versus 0.952 for the lexical-plus-structure setting.
  • Lexical Information: Word embeddings strongly improve performance: with Ds,l, Viri,word Hits@1 is 0.934, 0.133, and 0.068 versus 0.295, 0.120, and 0.048 for Viri on HeLis, FoodOn, and GO.
  • Lexical Information: The IRI embedding helps HeLis membership prediction but can hurt FoodOn and GO subsumption prediction when concatenated with word embeddings.For Ds,l on HeLis, Viri,word has 1.5% higher MRR than Vword; on FoodOn and GO it is often close to or worse than Vword.
  • Graph Structure: Graph transformation and walking settings are ontology-dependent, while the WL sub-graph kernel generally achieves higher top MRR than raw random walks on HeLis and FoodOn.The WL kernel also typically reaches its best performance at smaller walking depths.
  • Logical Constructors: OWL2Vec∗’s logical structure performs poorly alone but has a positive impact when combined with graph structure.The comparison attributes part of the gains over RDF2Vec to Manchester Syntax axiom sentences and projection rules, depending on the ontology.
  • Logical Constructors: Reasoning has limited impact: MRR values with and without entailment reasoning are quite close for the evaluated OPA2Vec and OWL2Vec∗ settings.

4.4 Classifiers

Using OWL2Vec∗ embeddings with different binary classifiers yields competitive results, with MLP generally close to Random Forest and better than the reported baselines.

  • The classifier comparison uses OWL2Vec∗ with Ds,l + Vword because this setting performs best on FoodOn and GO subsumption prediction and competitively on HeLis membership prediction.
  • MLP with a single hidden layer has performance quite competitive with Random Forest, especially on HeLis and FoodOn.
  • OWL2Vec∗ with MLP performs better than all baselines in Table 4 on each ontology.
  • SVC works for HeLis and GO but has lower MRR and Hits@1 than RF and MLP.

4.5 Interpretation and Visualization

The embedding visualizations show that OWL2Vec∗ can cluster instances or subclasses near their corresponding classes, although this structure is weaker for some settings and ontologies. These geometric differences matter especially for generalization beyond supervised class examples.

  • Average Euclidean distances compare positive and corrupted negative memberships or subsumptions used to train the classifiers.
  • OWL2Vec∗ with Ds,l and Vword clusters each class’s instances compactly and places them close to their corresponding class in HeLis.
  • For OPA2Vec and Ds + Viri, instances can be farther from their class than from another class despite remaining clustered by class.This pattern can support supervised membership prediction but dramatically impacts generalization in zero-shot learning.
  • For FoodOn, Ds,l and Vword mostly clusters subclasses near one another and closer to their class than to other classes, but less strongly than in HeLis.
  • Ds,l and Vword produces more significant subclass clustering than Word2Vec, OPA2Vec, and Ds + Viri, consistent with better subsumption-prediction performance.Examples include shorter distances between Fish and its subclasses and tighter clustering around Yogurt Food Product.

5 Discussion and Outlook

OWL2Vec* is evaluated as a robust embedding framework for OWL ontology completion and is reported to outperform state-of-the-art methods. The discussion highlights lexical information, specialized text modeling, zero-shot limitations, and broader ontology-quality applications.

  • Evaluation: OWL2Vec* significantly outperforms state-of-the-art methods on class membership and class subsumption prediction across three real-world ontologies.The evaluated ontologies are HeLis, FoodOn, and GO.
  • Ontology Text Understanding: Lexical information plays a very important role in both class membership prediction and class subsumption prediction.Entity names and ontology annotations can express relationships to surrounding entities.
  • Ontology Text Understanding: Word-embedding architectures and training methods should be tailored to the textual information typically present in state-of-the-art ontologies.Ontology text patterns differ from those of normal natural-language text.
  • Ontology Completion via Prediction: In supervised class membership prediction, OWL2Vec* recalls 93.2% of test instances, but recall drops to 65.6% for a zero-shot new-class scenario.The zero-shot setting has no known instances for the newly defined class.
  • Industrial Applications: OWL2Vec* embeddings can support ontology alignment and other ontology design and quality-assurance problems beyond the evaluated prediction tasks.The paper describes using embeddings of classes from two ontologies for mapping prediction.
Loading 2009.14654v2…