Source-linked AI summary
From Question-First to Analyst-First: Domain-Expert Skills and Verified Knowledge Compilation for Proactive Enterprise Analytics
Harmohit Singh, Rahul Sharma
TL;DR
Enterprise conversational analytics often assumes users know what to ask, leaving fresh schemas and tenants without useful starting points. This paper presents an analyst-first architecture that combines auto-selected domain-expert skills with offline, critic-gated knowledge compilation and execution-verified reports. In a single illustrative deployment, the system onboarded a tenant without human intervention, produced reports and questions before the first query, and suppressed unreproducible metrics.
Problem
Enterprise analytics interfaces assume a well-formed user question, while fresh tenants lack the logs or curated expertise needed to decide what to ask.
Method
The system packages domain expertise as deterministically selected, cross-cutting skills and compiles validated dataset knowledge offline to drive reports, questions, and verified deep dives.
Results
The single-tenant deployment explored all ten tables, proved twenty-two joins, admitted seven reports, and generated roughly thirty domain questions before the first query.
Takeaways & Limitations
The architecture’s supported consequence is a proactive loop in which published numbers seed answerable questions and clicks launch deep dives grounded in the same compiled knowledge.
Takeaways & Limitations
The evidence is limited to one illustrative deployment with no controlled user study, A/B test, or public-benchmark leaderboard, and verification does not establish semantic metric correctness.
Abstract
from arXiv · showhide
Conversational analytics systems assume the user already has a well-formed question, leaving a non-expert facing a blank query box on an unfamiliar enterprise schema. Commercial 'proactive' tools narrow this gap only by detecting statistical anomalies over analyst-curated metric layers, and academic next-question recommenders depend on query logs that a fresh dataset lacks. We describe a production analytics system that inverts the interaction model from question-first to analyst-first through two coupled architectural ideas. First, a pluggable domain-expert 'skill' abstraction: a folder-based, database-free subject-matter pack (a manifest, per-stage prompt facets, keyword-routed references, report templates, and optional compute) auto-selected per (client, dataset) by deterministic schema matching and spliced as a cross-cutting concern into every stage of an agentic pipeline, the schema explorer, and the report engines, degrading to a strict no-op when absent. Because a skill is a self-contained folder resolved deterministically, the catalogue is open-ended: an extensible marketplace of domain experts. Second, an offline knowledge-compilation loop: an agent probes the dataset's parquet via DuckDB (zero load on production), runs critic-gated per-table convergence with self-healing retries, and data-validates joins by value overlap, producing durable schema knowledge that drives standing expert reports whose every published metric is re-verified by re-executing its evidence SQL, plus suggested questions that mirror the report agenda. These close a proactive loop: reports surface numbers, the numbers seed questions, and a click launches a verified deep dive, all before the query box is used. We give a formal model and report illustrative single-tenant evidence. We make no user-study or benchmark claims; the contribution is the architecture and its defensibility.
1 Introduction
The paper addresses the blank-query-box problem by making domain expertise and compiled dataset knowledge drive analysis before users ask questions. Its production architecture combines pluggable skills, offline verification, and proactive reports with an explicit single-tenant evidence boundary.
- Motivation: Enterprise text-to-SQL systems assume users already know what to ask, leaving analysts facing a blank query box on unfamiliar schemas.This is identified as the cold-start problem and a documented barrier to self-service business-intelligence adoption.
- Motivation: Log-based recommenders cold-start without tenant query logs, while statistical insight miners lack portable domain expertise.The paper distinguishes statistical interestingness from questions a domain expert would act on.
- Thesis: The proposed analyst-first system couples auto-selected domain-expert skills with offline compilation of durable, verified dataset knowledge.Skills are self-contained folders injected across the analytics pipeline and offline engines; compilation probes data, validates joins, and persists knowledge.
- Thesis: The fused architecture produces standing expert reports with re-verified metrics and suggested questions grounded in the same validated knowledge.Reports surface numbers, those numbers seed questions, and clicks launch verified deep dives before the query box is used.
- Contribution: The contribution is the deployed fusion of schema-selected expertise, data-validated knowledge, and execution-grounded reporting across the analytics stack.The paper characterizes each ingredient alone as incremental and emphasizes their combination as the contribution.
- Scope and honesty: The paper reports no controlled user study, A/B test, or public-benchmark leaderboard, and its deployment figures are illustrative observations from one live tenant.Its verification guarantee covers agreement with declared evidence SQL within tolerance, not semantic correctness of business definitions.
- Contribution: The skill abstraction injects expertise throughout the pipeline, schema explorer, and report engines while becoming an exact no-op when inactive.This allows the base product to remain unchanged when no skill is selected.
- Contribution: Skill selection, offline compilation, execution-grounded reports, and the proactive loop form an automatic onboarding chain without query logs or per-client engineering.The architecture includes deterministic selection, critic-gated exploration, evidence-query verification, and shared support for reports, questions, and chat.
2 Related Work
Related work supplies question refinement, statistical insight discovery, agentic infrastructure, knowledge injection, and verification techniques, but the paper positions its system around their fusion. Its claimed white space is proactive analysis over an uncurated schema using portable expertise and execution verification.
- Text-to-SQL and translation framing: Text-to-SQL has matured, but enterprise workflows still require schema scouting and external knowledge beyond translation.Spider 2.0 is cited as evidence that real enterprise workflows involve many tables, dialects, and external knowledge.
- Stated versus prior question: Question-refinement systems resolve ambiguity after a question exists, whereas this paper targets deciding what to ask first.NaLIR, DataTone, and Eviza are presented as complementary systems for phrasing and ambiguity management.
- Cold-start and next-question recommendation: Log- and session-dependent recommenders cold-start on fresh datasets, while tabular question generation offers the closest prior art.The paper distinguishes its generator by using a portable domain-expert prior and validated knowledge.
- Automated insight discovery and mixed-initiative analytics: Insight miners and visualization recommenders surface statistically salient patterns, whereas this system sets its agenda through domain expertise and data verification.The distinction is framed as statistical interestingness versus an expert-defined, verified agenda.
- LLM data-science and multi-agent systems: Agentic and multi-agent systems provide the substrate, but this paper targets the closed loop deciding which artifacts to create and re-verifying their numbers.Related systems are described as targeting reasoning, data science, or artifact generation separately.
- Domain-knowledge injection: Knowledge injection, semantic layers, skills, and subagents inform the architecture, but semantic layers require human metric curation per organization.The paper contrasts upstream knowledge supply and modular capability packaging with its downstream evidence-SQL publish gate.
- Verifier-gated loops and agent memory: Verifier-gated loops and persistent agent memory motivate the system’s execution-grounded critics and durable knowledge base.The paper specifically cites concerns that intrinsic self-correction is unreliable.
- Commercial proactive analytics: The claimed white space is proactivity over an uncurated schema, for a skill-covered domain, driven by portable expertise and backed by execution verification.This positioning distinguishes the system from question-first, metric-bound, and domain-agnostic commercial products.
3 Problem Framing and Formalization
The formalization defines a multi-tenant platform with reactive and proactive regimes, then specifies skill selection, prompt injection, offline knowledge compilation, join validation, and report admissibility. Together these mechanisms produce durable, verified reports and question-driven deep dives.
- 3 Problem Framing and Formalization: The platform separates a reactive live pipeline from proactive offline artifact generation, with formal contracts for skills, compilation, and report publication.Everything is scoped by client identifier, and the formalization makes the three contracts precise.
- 3.1 Sets and objects: Each client-dataset pair has a schema of tables, columns, and sampled values, and the live pipeline is an ordered sequence of six stages.The route may bypass scouting and collation, but skill injection applies per executed stage.
- 3.1 Sets and objects: A skill is a database-free registry artifact containing detection rules, per-node facets, references, routing fields, report templates, deterministic libraries, and configuration.The registry is defined by folders containing SKILL.md, making the catalogue a pure function of disk.
- 3.2 Selection: deterministic and fail-open: Selection requires a table-or-value hit and a score threshold, allowing distinctive cell values to identify opaque schemas while blocking isolated coincidental columns.Selection is evaluated over sampled values offline; live chat scores over names only.
- 3.2 Selection: deterministic and fail-open: The active skill is chosen by a deterministic lexicographic rule with caching and error-to-empty behavior, making resolution independent of registry order and fail-open.The resolved slug is written once into the run context.
- 3.3 The injection operator and its no-op property: Prompt injection splices skill facets and keyword-selected references into stage-specific anchors, while absent skills render as empty text.The coder combines methodology and chart facets, a selected reference, and a reasoning-discipline block; offline reports admit more references.
- 3.3 The injection operator and its no-op property: When no skill is active or resolution fails, the injected prompts equal the baseline prompts and the augmented pipeline is unchanged on every executed route.This is the formal no-op additivity and backward-compatibility property.
- 3.4 Offline knowledge compilation: Offline compilation probes each parquet table through read-only DuckDB, uses deterministic coverage criticism and optional fail-open depth criticism, and keeps a shrinking work list.A round cap and keep-best policy ensure termination without regression.
4 The Skill Abstraction
The skill abstraction packages domain expertise as a portable folder that is deterministically selected per client and dataset, then injected across the analytics stack. Its no-op behavior and file-based registry make expert packs independently extensible and safe to add.
- Skill pack anatomy: A skill is a version-controlled folder containing domain expertise that influences every reasoning step without requiring a prompt, fine-tune, or database row.The pack includes a manifest, prompt facets, references, report templates, and optional deterministic code.
- Skill pack anatomy: The manifest declares routing patterns, keyword-to-reference mappings, core references, KPI vocabulary, and review configuration.Detection can use table names, column names, and distinctive cell values for opaque schemas.
- Deterministic selection: Selection resolves one active skill per client and dataset using explicit assignments or deterministic, source-agnostic schema scoring with stable persistence.The resolver can inspect cached parquet schemas or unified connector configuration across uploaded files and database systems.
- Cross-cutting injection: Small render helpers inject the active skill into routing, per-table exploration, coding, charting, narration, reports, and question generation, while remaining no-ops when absent.This preserves the base product when no skill is active.
- Toward a marketplace: Because packs are self-contained, deterministically routed, and safely no-op when unmatched or malformed, new domain experts can be added by content authors rather than code changes.The framework is therefore designed as an extensible catalogue or marketplace of expert packs.
5 Persistent Agentic Schema Exploration
The offline explorer compiles durable schema knowledge by probing each table over read-only DuckDB-on-parquet with deterministic coverage gates and bounded self-healing. It also validates cross-table joins against observed value overlap before retaining them.
- Compilation overview: Knowledge compilation performs a slower second pass that profiles columns, tests encoding hypotheses, and validates joins after an initial onboarding description.The explorer follows an agentic tool-use pattern with independent feedback.
- Per-table probe loop: Each table receives a bounded probe agent with schema context, tools, and an optional domain-expert lens; actions execute against parquet through DuckDB without loading the production database.The agent emits JSON actions such as profiling, sampling, SQL execution, and finding recording.
- Per-table probe loop: A deterministic coverage critic, rather than the model alone, gates termination by checking column profiling, typing, primary key, grain, and expected domain entities.Missing coverage produces a specific gap and forces the loop to continue.
- Convergence and durability: Self-healing retries use anti-runaway guards and a keep-best policy, while the optional depth critic is fail-open; the run is labeled deep only when every table converges.These mechanisms bound work and prevent a retry from regressing a converged table.
- Join discovery and persistence: Join candidates are generated from names, skill bindings, identifier-like columns, and semantic proposals, then retained only after DuckDB proves value overlap on the data.Completed findings are persisted per client and dataset and can be resumed or exported.
6 Execution-Verified Expert Report Synthesis
Standing expert reports are pre-computed views whose metrics are published only when independent re-execution reproduces their evidence SQL. Failed views are suppressed or reports deferred, while the surviving artifact becomes a durable refreshable recipe.
- Report planning: Standing expert reports provide pre-computed domain-expert views for users to read before submitting a question.Their latency-independent presentation makes correctness the primary design requirement.
- Report planning: An adaptive planner expands a report template into verifiable cuts supported by the actual dataset rather than treating the template as a fixed script.Section analysts then define durable SQL steps and attach evidence queries to emitted metrics.
- Execution verification: Every metric is independently re-executed and tolerance-matched against its evidence query, with mismatches, broken references, and evidence errors blocking publication.A non-numeric result also blocks a metric that declares a numeric unit.
- Failure handling: Unverifiable views are suppressed and reports with no surviving view are deferred, so the system withholds figures it cannot reproduce rather than surfacing raw errors.This is a suppress-or-defer guarantee, not a claim of zero hallucination or universal correctness.
- Verification boundary: Verification certifies arithmetic agreement with declared evidence SQL, but does not establish that the metric’s business definition is semantically correct.The boundary protects evidence faithfulness rather than definitional validity.
- Durable recipe: Publication saves a SQL-step DAG, metric definitions, evidence SQL, deterministic charts, drill-down tables, filters, provenance, and confidence tiers as a re-runnable recipe.Refreshes, filtering, and chart-click drill-down reuse the recipe with zero LLM and zero kernel cost, while consistent step rows keep report views aligned.
7 Proactive Question Generation and the Loop
The system closes a proactive loop by generating domain-grounded questions from verified reports before users query the dataset, while connecting onboarding automatically to deep dives.
- Suggested questions combine the matched skill’s expert agenda with validated knowledge about entities, grains, and joins.
- Four automatic trigger edges connect fast-pass onboarding, deep exploration, skill persistence, report generation, and question regeneration.
- The loop completes before any prior user query is required.
- A clicked number returns to the live pipeline as an on-topic deep-dive query grounded in the same compiled knowledge.
- Users land on finished analyses with visualizations, then can drill down without an LLM or launch a verified natural-language deep dive.
8 System and Implementation
The implementation uses durable, replayable pipeline infrastructure and shared DuckDB-over-parquet access, with deterministic report artifacts and fail-open, backward-compatible extensions.
- Runs are detached durable tasks whose events are stored in per-run append-only logs, while HTTP responses replay and tail those logs.
- Parquet over DuckDB is shared by exploration, reporting, and the live coder, imposing zero load on the client’s production database.
- Extensions are additive behind flags, preserve document compatibility, and degrade to prior behavior or a no-op on error.
9 Case Study: Single-Tenant Deployment
A single live-tenant deployment qualitatively demonstrated end-to-end onboarding and verified report generation, but its observations are illustrative rather than benchmark evidence.
- The deployment used Tally accounting data materialized to parquet and was observed from one live tenant.
- The evidence comprises illustrative single-deployment counts, not precision/recall or report-correctness measurements, because no ground truth was collected.
- The loop ran end to end without a human in the path, recognizing accounting data from distinctive ledger-group strings despite opaque table names.
- Exploration converged on all ten tables and proved twenty-two joins before the tenant ran a single query.
- Seven reports passed the verification gate and roughly thirty domain questions were generated from report KPIs and validated knowledge.
10 Discussion: Novelty and Positioning
The paper positions its novelty as the deployed fusion of portable domain expertise, schema-derived proactivity, and execution-grounded verification. This combination supplies questions over un-curated schemas while constraining published numbers to declared evidence queries.
- Table 2 compares systems along who supplies the question, what proactive means, and what guarantees the number.
- Who supplies the question: The system supplies questions from an auto-selected domain-expert prior available without logs or per-client engineering.Its questions are generated from the same auto-explored and validated knowledge used to answer them.
- What “proactive” means: Its proactivity operates over un-curated schemas, with expert templates setting the agenda and actual columns determining supported analytical cuts.
- What guarantees the number: Every published metric carries evidence SQL that an independent deterministic critic re-executes and tolerance-matches, while joins are proven on the data.The expert sets the agenda but cannot fabricate the figure.
- The honest differentiator: The claimed differentiator is the deployed fusion of cross-cutting, schema-selected expertise, schema-derived analysis, and end-to-end execution verification.The system is described as fail-open and closed into a fully automatic onboarding chain.
- The trade-off we accept: The architecture trades organization-specific metric curation for portability, recovering arithmetic but not definitional trust through evidence re-execution.Ingesting an existing metric layer as an additional skill input is identified as future work.
11 Limitations and Threats to Validity
The paper’s evidence is limited to one deployment without controlled evaluation, while its heuristic skills, verification boundary, persona bias, and offline cost constrain generalization.
- Evidence comes from one tenant, with no controlled study, user study, public-benchmark numbers, or measured deployment wins.The deployment figures are illustrative, and undeployed packs evidence structural generality rather than results.
- Human-authored skills form a small catalogue, and heuristic auto-detection can miss or mis-route on truly generic schemas.Explicit assignment and fail-open-to-no-skill mitigate but do not eliminate this risk.
- Verification certifies arithmetic agreement between a published number and its evidence query within tolerance, not semantic correctness of the metric definition.The stated failure mode is suppression rather than a certificate of universal correctness.
- Opinionated experts may bias framing even when deterministic grounding constrains the numbers.Evidence re-execution, typed formatting, and deterministic rendering do not necessarily constrain emphasis.
- Exploration and report generation are token-heavy offline jobs whose cost and staleness remain validity concerns.
12 Conclusion
The paper concludes that pluggable expertise and offline knowledge compilation create a proactive analytics loop that authors agendas and re-verifies numbers before users query. Its broader trajectory is an extensible expert marketplace, alongside semantic verification, continuous revalidation, and rigorous user evaluation.
- The coupled architecture turns reactive question-answering into proactive analysis by understanding data, authoring an agenda, and suggesting what users should ask.This occurs before the query box is used.
- Published numbers are re-verified against evidence SQL, making execution-grounded trust part of the closed proactive loop.
- Self-contained, deterministically selected, fail-open skill packs support a potential marketplace of domain experts automatically drawn on when data is connected.
- Future work includes a larger partly auto-authored catalogue, semantic metric verification, continuous knowledge revalidation, and a rigorous user study.The current paper deliberately does not claim the user study.