Source-linked AI summary
UniDataAgent: An Ontology-Grounded Agent for Enterprise Question-to-Report Automation
Yutai Duan, Yahui Zhao, Zhangti Li, Yu Ma, Zhenfeng Qi, Shaoyang Yuan, Jing Fan, Jie Liu
TL;DR
Enterprise data agents must preserve business semantics across questions, calculations, and reports, not merely generate queries. UniDataAgent builds a reusable ontology and uses it to execute question-specific analysis with skills and data tools. Its deployment-specific comparison reports improvements over document RAG across task types, with the largest gains on regional comparison and cross-metric diagnosis.
Problem
Enterprise agents need to preserve business semantics, but without an explicit ontology they repeatedly infer business knowledge and cannot consistently apply it across questions.
Source: §I · p. 1Method
UniDataAgent separates ontology acquisition and validation from online question-to-report execution, linking them through a typed ontology contract.
Source: §I · p. 2Results
UniDataAgent improves performance across all three task types, with the largest gains over document RAG on regional comparison and cross-metric diagnosis.
Source: B. Results and Failure Analysis · p. 4Takeaways & Limitations
The deployment-specific results indicate that explicit ontology grounding is particularly valuable when questions require complete coverage or business-consistent interpretation beyond isolated facts.
Source: B. Results and Failure Analysis · p. 4
Abstract
from arXiv · showhide
Enterprise data agents must preserve organization specific semantics, not just translate questions into queries. We present ChinaUnicom DataAgent (UniDataAgent), an ontology grounded system for reusable question-to-report analysis that separates semantic acquisition from online execution. Ontology Acquisition and Validation stage (OAV) builds versioned enterprise ontologies from metadata, business knowledge, and supporting materials through expert authored business skills, constrained generation, question verification, and selected expert review. Question-to-Report Execution (QRE) stage retrieves semantic contracts for each question, coordinates skills and data tools, validates results, and produces evidence linked reports. Across 27 enterprise tables and roughly thousands of metric types, ontology construction took a few hours instead of about one week manually. It took just a few minutes to generate the reports, instead of several working days. Ontology grounding achieved 95.0\% strict accuracy on real business questions, versus 72.5\% for document RAG, especially on structured and compositional tasks. The system has already been deployed to generate cost savings and has the potential to be replicated in other enterprises.
I. INTRODUCTION
Enterprise question-to-report automation must preserve business semantics across scoping, calculation, and reporting, not merely generate executable SQL. UniDataAgent addresses this with a reusable ontology and an online agent, with reported gains over document RAG on structured and compositional tasks.
I. INTRODUCTION
- Enterprise reporting requests require cross-role scoping, authoritative calculation, and report preparation, a process that can take days and drift from the original intent.The central challenge is preserving end-to-end business semantics rather than merely generating SQL.Source: §I · p. 1
- Without an explicit ontology, an LLM agent must repeatedly infer business knowledge and cannot consistently apply it across questions.The paper identifies ontology construction from fragmented metadata, documents, and expert knowledge as costly.Source: §I · p. 1
- The introduction frames three challenges: scalable semantic acquisition, operationalizing expert analysis, and producing reports traceable to executed evidence.Reports must preserve metric caliber and provenance, distinguish accepted evidence from execution artifacts, and constrain factual claims to observations.Source: §I · pp. 1–2
- UniDataAgent separates reusable semantic acquisition from question-specific analysis through OAV and QRE, linked by a typed ontology contract.OAV acquires and verifies reusable enterprise semantics; QRE retrieves and executes them in analysis.Source: §I · p. 2, Figure 1 · p. 2
- The system demonstration covers reusable ontology construction, grounding ambiguous questions in executable definitions and constraints, and producing evidence-backed reports.The ontology is reused across requests while QRE turns retrieved semantics into a validated analytical workflow.Source: §I · p. 2
- UniDataAgent improves performance across all three task types, with the largest gains on regional comparison and cross-metric diagnosis.The comparison is against document RAG.Source: §I · p. 2
II. SYSTEM OVERVIEW
UniDataAgent couples offline ontology construction with online question-to-report analysis. The typed ontology contract connects reusable enterprise semantics to evidence-linked report delivery.
II. SYSTEM OVERVIEW
- OAV transforms fragmented enterprise materials into a versioned, reusable ontology, while QRE uses it for question-specific analysis and evidence-linked report delivery.The system overview presents these as the two stages addressing the three challenges.Source: §II · p. 2
- An explicit ontology contract makes offline knowledge directly executable online.Source: §II · p. 2
- QRE consumes the ontology for online analysis and delivery of reports linked to evidence.Source: §II · p. 2
A. OAV: Constructing the Semantic Foundation
OAV compiles fragmented enterprise materials into a versioned ontology through Business-Skill grounding, semantic planning, constrained generation, and question-based verification. Failures trigger repair or replanning, with expert review recommended for consequential decisions.
A. OAV: Constructing the Semantic Foundation
- OAV interprets metadata, business knowledge, and supplementary materials with an expert-authored Business Skill, then checks a semantic plan for coverage, conflicts, unsupported mappings, and source support.The plan specifies domains, entity and relation types, report classes, mappings, provenance requirements, and prohibited content.Source: A. OAV: Constructing the Semantic… · p. 2
- Ontology candidates are generated under the approved plan, linked to supporting sources, and verified for coverage, consistency, provenance, mappings, and question answerability.Verification failures trigger localized repair or replanning; passing candidates are released as versioned ontologies.Source: A. OAV: Constructing the Semantic… · p. 2, Figure 2 · p. 3
- Expert approval is recommended for unresolved or high-impact changes, so experts review consequential decisions rather than authoring every ontology item manually.Source: A. OAV: Constructing the Semantic… · p. 3
B. QRE: From Questions to Evidence-Backed Reports
QRE grounds each question in retrieved ontology context, coordinates analytical skills and data tools, and validates observations before producing evidence-backed reports. Factual report claims are anchored to executed observations.
B. QRE: From Questions to Evidence-Backed Reports
- QRE identifies query attributes and retrieves ontology context that maps user expressions to canonical entities and metrics, dimensions, mappings, aggregation rules, constraints, and lineage.It requests clarification when required business parameters are missing or ambiguous; business documents and procedural rules supplement the ontology.Source: B. QRE: From Questions to… · p. 3
- QRE runs a plan–act–observe loop, selecting skills or tools and validating observations against caliber, units, scope, time, provenance, and prior results.Failed observations can trigger revision, cross-source verification, recomputation, or replanning.Source: B. QRE: From Questions to… · p. 3
- Once sufficient evidence is gathered, QRE synthesizes findings and evidence-backed suggestions into reports, anchoring factual claims to executed observations.The front end renders tables and charts with data views, diagnostic analysis, caliber definitions, actionable insights, and evidence citations.Source: B. QRE: From Questions to… · p. 3
III. DEMONSTRATION
The demonstration presents an inspectable enterprise ontology as the semantic foundation for UniDataAgent’s question-to-report workflow. It shows how a natural-language request becomes charts, candidate suggestions, and evidence-linked findings in approximately 200 seconds.
III. DEMONSTRATION
- Hundreds of domain concepts and data objects and thousands of semantic relations form an inspectable ontology in a few hours.The ontology provides explicit, traceable grounding for downstream agents.Source: Figure 3 · p. 3, §III · p. 3
- Defined entities, metrics, rules, and data objects link through the ontology, letting users inspect the semantic basis before analysis runs.This grounding supports question answering from defined business semantics rather than ungrounded inference.Source: §III · p. 3
- For a natural-language request, the system retrieves records under source, temporal, metric, and geographic-granularity constraints, then organizes results into charts and candidate suggestions.The charts cover revenue rankings, year-over-year changes, and user-development trends; suggestions draw on cross-period comparisons.Source: §III · p. 3
- The report presents evidence-linked charts, candidate suggestions for analyst review, and findings grounded in retrieved results.The demonstration data were irreversibly transformed, with original numerical values, locations, and identifiers omitted.Source: Figure 5 · p. 4
- Approximately 200 seconds covered data screening, crosstable reconciliation, chart generation, and report synthesis.Source: §III · p. 4
A. Evaluation Setup
The evaluation uses 40 real enterprise questions from six operational domains, categorized by primary semantic requirement. It compares UniDataAgent with document RAG in the same enterprise data environment.
A. Evaluation Setup
- The evaluation covers 40 real enterprise questions across six operational domains, assigned to metric and scope grounding, complete regional comparison, or cross-metric business diagnosis.The task groups contain 22, 11, and 7 questions, respectively.Source: A. Evaluation Setup · p. 4
- Each question is paired with a reference answer and a checklist covering required results, scope, conditions, and metric caliber; the evaluation used DeepSeek-V4.Source: A. Evaluation Setup · p. 4
- Document RAG retrieves business documents but infers definitions, aggregation rules, dimensions, and physical mappings at query time, whereas UniDataAgent retrieves an ontology contract with executable constraints.Source: A. Evaluation Setup · p. 4
B. Results and Failure Analysis
UniDataAgent improves performance over document RAG across all three task types, with the largest gains on regional comparison and cross-metric diagnosis.
B. Results and Failure Analysis
- UniDataAgent improves performance across all three task types, with the largest gains on regional comparison and cross-metric diagnosis.Source: B. Results and Failure Analysis · p. 4
- Document RAG can retrieve relevant values yet omit required provinces, apply inconsistent metric calibers, or form unsupported relations between indicators.Source: B. Results and Failure Analysis · p. 4
- The ontology contract specifies valid mappings, scopes, and calculation constraints before execution, while evidence checks constrain the final analysis to supported observations.Source: B. Results and Failure Analysis · p. 4
V. CONCLUSION
The conclusion frames UniDataAgent as a persistent, reviewed semantic foundation for enterprise question-to-report workflows, combining offline acquisition with online execution.
V. CONCLUSION
- UniDataAgent builds an enterprise question-to-report workflow on a persistent, reviewed semantic foundation.Source: §V · p. 4
- Offline semantic acquisition and online skill-driven execution close the report loop with traceable outputs and evidence-constrained synthesis.Source: §V · p. 4
- Real-world deployment and comparative tests demonstrate less semantic rework, higher analytical accuracy, lower delivery costs, and cross-enterprise reuse potential.Source: §V · p. 4