PaperlayerView exact revision on arXiv

Source-linked AI summary

Surprising Effectiveness of Self-Demonstrations in Enhancing Schema-Ontology Mapping with LLMs

Siddhesh Thombre, Manasi Patwardhan, Sunita Sarawagi

arXiv:2609.13776v1cs.AI

TL;DR

Mapping heterogeneous relational databases to a centralized ontology is difficult, and direct LLM prompting and prior mapping approaches have limitations. The paper combines neuro-symbolic decomposition with pattern-guided, dependency-aware self-demonstrations, achieving state-of-the-art results on three challenging RODI scenarios with about 25 percentage points of F1 improvement over prior automated baselines.

  • Problem

    Mapping heterogeneous relational databases to a centralized ontology is challenging, while traditional systems can miss semantic correspondences and direct LLM prompting performs poorly on complete mappings.

    Source: §1 · p. 1, §5.3 · p. 11

  • Method

    The approach decomposes mapping into focused stages and guides LLMs with demonstrations generated from domain-agnostic patterns and prior-stage outputs.

    Source: §1 · p. 3, §4.3 · p. 7

  • Results

    About 25 percentage points in F1 improvement over direct prompting and prior automated baselines is reported across three challenging RODI scenarios, with slightly better results than manually curated few-shots.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The automatic self-demonstration strategy matches or slightly exceeds manually curated few-shots while eliminating manual curation overhead.

    Source: §5.3 · p. 13

    When the ontology omits the requirement to normalize values to meters using wlbCoreIntervalUom, the LLM cannot generate the correct SQL view.

    Source: §5.3 · p. 14

Abstract

from arXiv · show

Integrating heterogeneous relational databases into a centralized ontology remains a persistent challenge in enterprise knowledge representation, primarily due to semantic heterogeneity, cryptic schema naming, missing metadata, and the abstraction gap between relational schemas and ontological models. Although large language models (LLMs) offer strong semantic reasoning capabilities, we show that directly applying them through one-shot prompting or naive multi-stage pipelines leads to poor performance for schema-ontology mapping. This paper presents a self-demonstration-driven approach that combines a neuro-symbolic task decomposition with a novel mechanism for automatically generating pattern-guided, dependency-aware demonstrations to address this integration challenge. Our approach incorporates two key strategies to achieve substantial accuracy gains over existing LLM-based schema integration methods: (i) a neuro-symbolic decomposition of the task into cascaded sub-tasks, where symbolic constraints structure the search space and LLMs perform semantic reasoning within each focused sub-task, and (ii) self-generated demonstrations guided by domain-agnostic patterns to supervise each sub-task. Experiments on three of the most challenging scenarios from the RODI benchmark show that our approach achieves state-of-the-art performance, substantially outperforming (25 percentage points F1 improvements) both traditional schema-to-ontology mapping techniques and recent LLM-based schema-to-ontology and schema matching approaches. Ablation studies further reveal the significant benefits of pattern-guided self-demonstrations and the complementary benefits of neuro-symbolic task decomposition.

1. Introduction

Mapping heterogeneous relational databases to a centralized ontology is difficult, and direct LLM prompting and manual demonstration curation have important limitations. The paper combines neuro-symbolic decomposition with pattern-guided, dependency-aware self-demonstrations and reports substantial gains on challenging RODI scenarios.

1. Introduction

  • Traditional systems rely on lexical similarity and templates, while one-shot LLM mapping performs poorly and manually curated demonstrations are labor-intensive and difficult to scale.The introduction also notes that errors can accumulate across stages in zero-shot pipelines.Source: §1 · p. 2
  • Self-demonstrations are generated from reusable domain-agnostic patterns, with earlier stage outputs carried into downstream examples to make them diverse and less noisy.Examples include different class-to-table matches and object-property matches through foreign keys or junction tables.Source: §1 · p. 2
  • Because mappings are reused across downstream queries and applications, the approach prioritizes accuracy and semantic correctness over execution efficiency or compute cost.The paper characterizes database-to-ontology mapping as typically a one-time integration process.Source: §1 · p. 3
  • About 25 percentage points in F1 separates the approach from direct prompting and prior automated baselines on three challenging RODI scenarios; it also slightly exceeds manually curated few-shots.The reported scenarios are geographical, conference, and oil and gas.Source: §1 · p. 3
  • The approach decomposes mapping into stages that combine symbolic structure with LLM reasoning, and supplies each stage with automatically constructed, task-specific demonstrations.The contributions describe one-time, domain-agnostic pattern curation and detailed ablations of decomposition and self-demonstrations.Source: §1 · p. 3

2. Related Work

Prior mapping systems often rely on lexical or structural similarity and templates, while recent LLM methods may provide alignment without targeted guidance for mapping. The paper positions its approach as combining symbolic decomposition with semantic reasoning and stage-specific self-demonstrations.

2. Related Work

  • Traditional mapping tools can degrade with cryptic names, incomplete metadata, or semantic correspondences that exceed string similarity, while some LLM methods stop at alignment or retain template-driven mapping.Earlier systems that derive an ontology from a database before mapping it are described as an unrealistic setting.Source: §2 · p. 3
  • Schema matching is typically simpler than DB-to-ontology mapping because the latter bridges different abstraction levels and granularities.The passage says adapting Matchmaker’s automated few-shot generation produces sub-optimal results in this setting.Source: §2 · p. 4
  • The paper combines symbolic decomposition and pruning with LLM semantic reasoning, guiding mapping stages with automatically synthesized, diverse, scenario-specific self-demonstrations.The related-work discussion identifies diversity and dependency awareness as important properties of automated few-shots for this task.Source: §2 · pp. 3–4

3. Background and Problem Definition

The task takes an ontology and a relational database schema as inputs and seeks mappings between their concepts and entities. The target output is an R2RML mapping whose SQL views generate RDF triples that populate the ontology from database data.

3. Background and Problem Definition

  • The input ontology contains classes, subclasses, data properties, and object properties, while the database schema contains tables, columns, and their data values.The passage gives Pipeline, OilPipeline, pipelineDimension, and pipelineOperator as ontology examples, and pipeline and pipmedium as database examples.Source: §3 · p. 4
  • Typical correspondences link classes to tables or filtered subsets, subclasses to column-value filters, data properties to columns, and object properties to foreign keys or junction-table keys.The mapping patterns may involve one-hop neighboring tables for data properties and object properties.Source: §3 · p. 4
  • The goal is to generate R2RML mappings consisting of SQL views and subject/object columns so execution produces RDF triples that populate the ontology from the database.The passage identifies R2RML as the Relational to RDF Mapping Language.Source: §3 · p. 4

4. Self-Demonstration Driven Approach

The approach divides DB-to-ontology mapping into matching and mapping stages, using symbolic constraints to narrow candidates for LLM reasoning. It generates demonstrations from reusable patterns and carries earlier decisions into downstream demonstration synthesis.

4.1. Overview of Neuro-Symbolic Decomposition

  • Matching proceeds through class, subclass, data-property, and object-property stages, pruning candidates by ontology structure and prior matches.For the NPD pair, symbolic pruning reduces the candidate space by approximately 95% for data-property matching and 85% for object-property matching.Source: §4.1 · p. 5
  • Mapping generates SQL views and the R2RML subject/object columns for matched concepts, including filters for subclasses and links through relation columns or junction tables for object properties.Per-concept outputs are aggregated into the final R2RML mapping file.Source: §4.1 · p. 6

4.2. Why Pattern-guided Context-aware Self-demonstrations.

  • The approach seeks automated few-shots because manual curation requires domain and task expertise, while adapted schema-matching methods may produce examples with limited semantic variation for this harder mapping task.The passage attributes the greater variation to the abstraction and granularity mismatch between relational schemas and ontologies.Source: §4.2 · p. 6

4.3. Synthesis of Self-Demonstrations

  • The pattern library covers lexical, semantic, structural, value-constrained, and SQL-mapping behaviors, and prompts the LLM to span pattern families rather than select random examples.Examples include foreign-key or junction-table relations, filtered subclasses, joins, and value normalization.Source: §4.3 · p. 7
  • Demonstrations are synthesized sequentially using relevant prior-stage outputs, so downstream examples use established class-table alignments as structural anchors.For instance, class matching outputs feed subclass, data-property, and object-property matching demonstrations; matching outputs also feed data- and object-property mapping demonstrations.Source: §4.3 · pp. 7–8
  • Dependency-aware context focuses examples on candidates consistent with earlier decisions and makes downstream demonstrations more consistent, while the automatic approach uses at most one example per applicable pattern.The same fixed domain-agnostic pattern set is used across input pairs; experts may optionally add rare missing patterns.Source: §4.3 · pp. 7–8

5. Experimentation

Experiments on three challenging RODI scenarios compare the proposed approach with traditional and LLM-based systems, testing neuro-symbolic decomposition and automated demonstrations. The full approach achieves state-of-the-art average performance, while the results also expose mapping errors when required semantics are not specified in the ontology.

5.1. RODI Benchmark

  • RODI has 18 test scenarios across three domains; evaluation uses query-pair F-measures, averaged across queries and selected scenarios.The experiments select Mondial, NPD-Atomic, and Conference-no-FK as the most challenging scenario in each domain; Conference-no-FK lacks foreign-key definitions.Source: §5.1 · pp. 8–9

5.3. Results and Discussion

  • LLM4VKG (0.21), Table-to-KG (0.18), and Magneto (0.23) trail Milan (0.31), while single-pass matching-plus-mapping achieves only 0.10 average F1.The baselines include traditional DB-to-ontology systems, LLM-based mapping systems, and a schema-matching system adapted to this task.Source: §5.2 · p. 9, §5.3 · p. 11
  • 0.56 average score establishes state-of-the-art performance on Conference-no-fk, Mondial, and NPD-Atomic with self-demonstrations enabled.The approach matches or slightly exceeds manually curated few-shots while eliminating manual curation overhead.Source: Table 2 · p. 11, §5.3 · p. 13
  • 0.79 on Conference-no-fk and Mondial and 0.51 on NPD-Atomic are the method's best scenario results, substantially exceeding Milan and A4MO in average performance.The symbolic pipeline separates matching from SQL-view generation, while LLMs use ontology and schema context to disambiguate cryptic elements and generate expressive SQL.Source: Table 2 · p. 11, §5.3 · p. 12
  • Dropping the matching stage or replacing LLM mapping with programmatic mapping reduces performance, showing both stages contribute distinct capabilities.The paper attributes the improvement over earlier LLM systems to sequential symbolic sub-stages that constrain candidates and to expressive SQL generation beyond template-based mapping.Source: §5.3 · pp. 11–12
  • Matchmaker-style automatic few-shot selection relies on evaluated examples, but its selected demonstrations tend to miss difficult mapping patterns and inter-stage dependencies.The experiment compares top-scoring ontology concepts and matches selected for each pipeline stage.Source: §5.2 · p. 11, §5.3 · pp. 12–13
  • Pattern-guided demonstrations yield an average score of 0.56 versus 0.38 without them, and match or slightly exceed manually curated few-shots.Matchmaker-style selection provides limited, inconsistent improvement; its examples underrepresent difficult mapping patterns and lack dependency-aware context across stages.Source: §5.3 · pp. 12–13
  • NPD-Atomic error analysis includes SQL-view failures when ontology requirements are unstated, such as converting core intervals to meters or excluding a production-year value.The interval example requires unit-dependent conversion, but the ontology does not specify normalization to meters; the productionForField example lacks a stated reason to exclude 44576.Source: §5.3 · p. 14

6. Conclusion

The paper concludes that accurate schema-to-ontology mapping requires structured neuro-symbolic decomposition together with pattern-guided, dependency-aware self-demonstrations. On three challenging RODI scenarios, the combined approach achieves state-of-the-art performance, with approximately 25 percentage-point F1 improvements over prior automated baselines.

6. Conclusion

  • Symbolic constraints narrow the search space across structured stages, focusing LLMs on decisions that improve mapping accuracy.The conclusion states that decomposition alone is insufficient to address the mapping challenges.Source: §6 · p. 15
  • On three challenging RODI scenarios, the combined method achieves state-of-the-art performance, with ∼25 percentage-point F1 improvements over prior automated baselines.The approach combines neuro-symbolic decomposition with pattern-guided, dependency-aware self-demonstrations.Source: §6 · p. 15
  • Pattern-guided, dependency-aware self-demonstrations supervise each stage with diverse examples and preserve consistency across stages, limiting error propagation.The demonstrations are generated from reusable, domain-agnostic mapping patterns and conditioned on earlier-stage outputs.Source: §6 · p. 15
  • Future work proposes replacing some LLM reasoning with programmatic templates for repetitive patterns and returning ranked matches with confidence scores for expert review.The proposed workflow would let experts validate a small set of plausible alternatives rather than perform exhaustive manual verification.Source: §6 · p. 15

Declaration on Generative AI

The authors used Microsoft Copilot to paraphrase and reword human-authored text, then reviewed and edited the content and retained responsibility for the publication.

Declaration on Generative AI

  • Microsoft Copilot was used for paraphrasing and rewording; the authors reviewed and edited the text and take full responsibility for the publication.The declared tool contribution was linguistic refinement of human-authored text.Source: Declaration on Generative AI · p. 15

Related papers

Loading 2609.13776v1…