PaperlayerView exact revision on arXiv

Source-linked AI summary

Intent Graph: Navigating the Analytical Reasoning Space for Exploratory Data Analysis

Junran Yang, Shruti Badrish, Teanna Barrett, Leilani Battle

arXiv:2610.11025v1cs.HCcs.AI

TL;DR

EDA analysts prioritize possible analyses using domain knowledge and hypotheses, but unstructured LLM suggestions do not make the explored space or their grounding clear. DAG-EDA links a grammar-governed intent graph with a knowledge graph to support inspection and navigation; a walkthrough illustrates how alternative paths can converge, while evaluation with analysts remains future work.

  • Problem

    LLM responses can suggest relevant analytical directions but are unstructured, leaving analysts unable to see what has been explored, what is missing, or why a direction fits the available data.

    Source: §1 · p. 1

  • Method

    DAG-EDA combines a grammar-governed intent graph for progressively refining analytical questions with a knowledge graph that externalizes domain knowledge and grounds concepts in dataset variables.

    Source: §1 · p. 2, §5.1 · p. 8

  • Results

    In the walkthrough, distinct readings of a question converge on the same analysis, and the analyst can see that convergence and retain multiple dashboards positioned by their reasoning paths.

    Source: §4 · pp. 6–8

  • Takeaways & Limitations

    The walkthrough illustrates how analysts can inspect alternatives, make their own narrowing choices, and compare dashboards through the reasoning that produced them.

    Source: §4 · p. 8

    DAG has not been evaluated with analysts, so the chapter’s claims rest on design rationale and a walkthrough rather than evidence about analyst behavior.

    Source: §7.2 · p. 14

Abstract

from arXiv · show

Exploratory data analysis (EDA) is rarely open-ended in practice: analysts work from high-level domain questions toward the concrete analyses that can answer them, prioritizing directions with domain knowledge and prior hypotheses. Large language models (LLMs) can supply such knowledge, but their responses are unstructured, leaving analysts no way to see what has been explored, what is missing, or why one direction was chosen over another. We present DAG-EDA, a system that lets analysts and an LLM co-navigate the space of possible analyses through two linked structures. An intent graph, governed by a grammar of analytical intent, decomposes an ambiguous natural-language question into progressively concrete analysis tasks, keeping alternative framings open and letting analysts branch, backtrack, and compare paths. A multi-layered knowledge graph externalizes the LLM's domain knowledge, linking domain concepts to the dataset variables that can measure them, so analysts can inspect and contest how their question is grounded in the data. Both graphs are constructed from only the dataset and the analyst's question, and the analyses the analyst reaches are rendered as interactive dashboards. We illustrate the system through a usage scenario and describe a user study design for examining whether the system scaffold analysts' reasoning and navigation.

1 Introduction

EDA often requires analysts to prioritize among many possible analyses using domain knowledge and hypotheses, while LLM assistance remains difficult to inspect and navigate. DAG-EDA pairs a navigable intent graph with a knowledge graph to help analysts move from high-level questions toward concrete analyses.

1 Introduction

  • Analysts must prioritize among vast variable combinations without a principled basis or natural stopping condition, despite limited time.In practice, analysts guide exploration with domain knowledge, prior hypotheses, and high-level intents.Source: §1 · p. 1
  • LLMs can suggest relevant variables and directions, but their unstructured responses do not show what has been explored, what is missing, or why a direction fits the available data.The paper asks how users and LLMs can transparently co-navigate possible analyses.Source: §1 · p. 2
  • DAG-EDA uses an intent grammar to constrain how analytical intents are instantiated, propagated, and refined between high-level goals and low-level data operations.Natural-language parsing initializes the intent specification, while the interface makes its rules navigable and interactive.Source: §1 · p. 2
  • The intent specification provides shared language for goals, while a knowledge graph exposes LLM-generated domain knowledge for analyst inspection and agreement.The two structures are presented as interdependent: navigation becomes meaningful through shared context, and that representation becomes actionable through progressive refinement.Source: §1 · p. 2
  • The system’s contributions include a graph-based representation of analytical intent, a multilayer knowledge graph, and LLM methods that construct both from the dataset and question.The methods combine bottom-up grouping of data columns into domain concepts with top-down extraction of concepts from the question, without a curated domain ontology.Source: §1 · p. 2
  • The interface supports iterative intent refinement and dashboard creation from terminal analyses, but the chapter reports design rationale and a walkthrough rather than an empirical evaluation of analyst use.Reading dashboard interactions to update the intent graph is described as future work.Source: §1 · pp. 2–3

2 Related Work

Related work spans formalizations of analytical intent, dashboard recommendation, analysis-space navigation, and LLM support for open-ended analysis. DAG-EDA distinguishes itself by making an intent decomposition inspectable and navigable while grounding it in explicit domain knowledge.

Characterizing analytical intent

  • Prior work formalizes intent at several granularities, while DAG’s prescriptive grammar constrains semantic gaps filled by an LLM and composes claim types into a dynamic graph of refinements.Unlike a descriptive grammar that enumerates parseable hypotheses, this grammar constrains language-model interpretation while preserving refinement paths.Source: §2 · p. 3

Dashboard recommendation

  • Dashboard recommendation systems use designer knowledge or learn composition from data, but often limit the available intents and leave domain knowledge implicit.DAG generates decompositions per dataset and question and constructs an explicit conceptual knowledge graph alongside them.Source: §2 · p. 3

Navigating the space of analyses

  • Analysis-space tools include recommenders, multiverse analysis, insight networks, and shared diagrams, but differ in whether they assume a settled specification, comparable outcomes, or make the full reasoning space visible.DAG’s grammar makes each intent-graph node a well-formed claim and each edge a legal refinement that can constrain generation and be checked by the analyst.Source: §2 · p. 3

Language models for open-ended analysis

  • LLM systems for visualization, insight generation, and open-ended analysis often return single views, summarize data, or struggle with ambiguity; DAG instead exposes and records its decomposition for validation and redirection.Suggestions can be traced to grammar rules and overridden rather than merely re-prompted.Source: §2 · p. 3

3 Design Considerations

The design aims to help analysts reach executable hypotheses through a process that preserves alternatives and makes interpretive choices visible, justified, and reversible. Its considerations address shared intent, dataset grounding, navigable analysis paths, verification, and comparison of dashboard designs.

Design Considerations

  • Analysts should be able to account for system choices and encounter alternatives, with interpretive choices visible, justified, and reversible and live branches easy to revisit.The design avoids silent narrowing and premature convergence, and ranks or offers moves by how much they distinguish downstream analyses.Source: §3 · pp. 4–5

DC1: Provide a Shared Representation of Analysis Intent Between the User and System

  • A human-readable grammar should provide a shared representation of intent that structures valid outputs and lets users verify whether LLM suggestions match expressed intent.The grammar steers generation and supplies a verifiable contract for inspecting and overriding suggestions.Source: §3 · p. 4

DC2: Verify That the Selected Dataset Is Appropriate for Answering the User’s Analysis Question

  • Dataset appropriateness should be checked by grounding abstract concepts in an LLM-generated domain graph, then refining them toward concrete data variables.The two-stage process separates semantic interpretation from operationalization in data, reducing the risk of mismatching a question to an inappropriate dataset.Source: §3 · p. 4

DC3: Enable Both the User and the System to Reason About the Space of All Possible Analysis Paths

  • The system should make the space of valid analysis paths enumerable and navigable, so users can inspect, question, redirect, refine, or accept paths.This contrasts with fixed top-k recommendations and limited LLM suggestions that can obscure alternatives.Source: §3 · p. 4

DC4: Enable the User to Verify How Any Given Analysis Step Is Grounded in Data Science Best Practices and the Relevant Domain Context

  • Users should be able to verify how each analysis step is grounded in domain context and data-science practice, including why variables and methods fit.The design consideration links trust in suggested analyses to understanding both domain context and method appropriateness for selected data types.Source: §3 · p. 4

DC5: Enable the User to Compare Multiple Dashboard Designs Simultaneously

  • Users should compare dashboard designs by their reasoning paths, so they can see how different interpretations of one question yield distinct analytical paths.This supports principled selection among alternatives rather than comparison by visual appearance alone.Source: §3 · p. 4

4 The Interface in Use

A usage scenario follows analyst Anna as she explores what makes movies successful. DAG-EDA keeps alternative readings and their paths visible while connecting domain concepts to dataset columns, rather than requiring her to settle on a meaning before examining the data.

The interface

  • The canvas presents candidate claims and available moves, while the knowledge graph links domain concepts to data variables; dashboards open from grounded nodes.Anna accumulates dashboards over the session rather than receiving a running recommendation.Source: §4 · p. 5, Figure 1 · p. 6

The question

  • Anna’s question yields three readings with confidence scores, while “success” remains unresolved and the system commits to none.The relation reading is ranked at 94%, the comparison at 38%, and the trend at 22%.Source: §4 · p. 5

The knowledge graph answers a question about the dataset

  • The knowledge graph shows which dataset columns can operationalize each meaning of success, including when a rating is only a proxy for critical judgment.Anna treats IMDB and Rotten Tomatoes ratings separately after learning that IMDB is an audience score standing in for critics’ judgment.Source: §4 · p. 5

Grounding a branch to columns

  • Anna grounds a branch incrementally, with each intermediate claim preserved until its required slots reach columns and it becomes renderable.In the Box Office Success branch, the path proceeds from concept to production budget and then to production budget against worldwide gross.Source: §4 · p. 5

A commitment reshapes the claim

  • Binding genre to an open relation reshapes the claim into a comparison rather than forcing a category into a measure slot.The system follows the selected slot binding, turning the relation into a comparison of Rotten Tomatoes rating across genre.Source: §4 · pp. 5–6

Two paths, one claim

  • The relation and comparison readings converge on one Rotten Tomatoes-rating-by-genre claim, despite reaching it through different moves.The shared node records both readings as parents, making their convergence visible.Source: §4 · p. 6, Figure 2 · p. 7

Dashboards, and what is left open

  • Anna compares dashboards as distinct readings of success, each traceable to the moves that produced it, without a system-selected answer.The three dashboards cover box-office measures, the two ratings, and Rotten Tomatoes rating across major genre.Source: §4 · p. 7

What the session shows

  • Unpursued readings remain visible, but DAG does not yet feed interactions with rendered dashboards back into the intent graph.The trend reading remains untouched, and available moves are shown as counts at the selected node.Source: §4 · p. 8

5 System Design

DAG-EDA turns an ambiguous analytical question into a navigable space of analyses, using a claim grammar and linked graphs to refine intent, ground it in dataset columns, and render dashboards.

5 System Design

  • The intent graph organizes candidate analyses as claims and lets analysts refine them while preserving convergent paths and earlier branches.Nodes represent claims rather than routes; returning to a branch restores its unchanged state.Source: §5 · p. 8, §5.2 · p. 10

5.1 The Grammar

  • Claims use at most two typed slots, with slot types determining the claim shape; referential and relational choices can be made independently.Grounding required slots makes a claim executable, while unsupported concepts remain visible rather than being silently dropped.Source: §5.1 · pp. 8–9

5.2 The Intent Graph

  • Intent-graph nodes represent claims, and legal refinement moves connect them; convergent routes share a node while provenance records how each route arrived.Offers are checked against production rules, and previews can show likely next steps without materializing them.Source: §5.2 · p. 10

5.3 The Knowledge Graph

  • The knowledge graph separates question-related concepts from dataset columns, connecting them through bridges tagged as available, proxy, or missing.The concept layer and bridges are built for a question and dataset, while the data layer is reusable across questions on that dataset.Source: §5.3 · pp. 10–11

5.4 How the Two Graphs Work Together

  • The intent graph proposes what could fill a claim’s slot, and the knowledge graph supplies candidate columns, so grounding links the two structures.The parser and graph builder use generated knowledge; claim enumeration and validity checks then follow deterministic grammar rules.Source: §5.4 · p. 11

5.5 Composition

  • Composition combines bounded claims into moderation, mediation, or comparison analyses, supplying designations that a single claim’s slots cannot express.Terminal claims can map to several recommended dashboards, allowing analysts to branch across dashboard designs.Source: §5.5 · p. 12

6 User Study Design

The study design examines how analysts use DAG to explore their own questions on two datasets, focusing on reasoning, graph use, perceived effort, and performance.

6 User Study Design

  • The planned study examines whether DAG’s intent and knowledge graphs scaffold analysts’ reasoning rather than automate or confuse it, using four study questions.The questions address alignment and agency, knowledge-graph utility, scaffold versus menu use across domains, and cognitive load relative to perceived performance.Source: §6 · p. 12, §5.6 · p. 12

5.6 Implementation Notes

  • The canvas presents the intent graph and concept trees alongside dashboards; the design also describes composite layouts for analyses involving multiple variables.Examples include a Condition over a Comparison yielding a two-way CAT2 crosstab and a Compare over Trends sharing a time axis yielding a CH2 dual-series chart.Source: §5.6 · p. 12, Table 2 · p. 13

6.1 Participants

  • The study targets experienced dashboard builders or EDA practitioners across domains and plans to recruit 10–12 participants.Eligible participants are invited to a 90-minute session and compensated with a $45 gift card.Source: §6.1 · p. 13

6.2 Setup

  • Sessions are remote and collect recordings, questionnaire and interview responses, and interaction logs to reconstruct analysts’ graph paths alongside their explanations.Logs capture intent-graph paths and branches, chart brushing, knowledge-graph interactions, and dashboard bookmarking.Source: §6.2 · p. 13

6.3 Procedure

  • After a warm-up, participants explore movies and NHANES in two 25-minute tasks, each beginning with their own stated question and allowing question refinement.Task order is fixed, with the movies task following the warm-up on the same dataset and NHANES second.Source: §6.3 · pp. 13–14
  • Post-task measures adapt NASA-TLX dimensions and ask about perceived exploration success, system effort, graph usefulness, and whether the resulting analysis answers the starting question.A closing interview asks for specific instances of alignment, divergence, useful features, confusion, and help or hindrance to reasoning.Source: §6.3 · p. 14

7 Discussion

DAG’s discussion frames its grammar, editable knowledge graph, and information-oriented ranking as ways to make analytical reasoning inspectable without resolving every ambiguity. It also identifies limits in evaluation, domain coverage, and graph legibility, alongside extensions for analysis outputs and adaptive feedback.

7.1 Implications

  • The intent grammar makes model suggestions checkable by constraining them to well-formed claims and legal refinements, so analysts can identify wrong bindings or refinements.This differs from free-form suggestions, where an unhelpful response can only be re-prompted.Source: §7.1 · p. 14
  • Because dataset grounding cannot verify all domain knowledge, the knowledge graph is editable so analysts can correct claims they—not the system—are authoritative about.The paper gives differing rating-system voter pools and proxy-versus-measure distinctions as examples of claims without a dataset oracle.Source: §7.1 · p. 14
  • DAG ranks offers by how much they distinguish downstream analyses, rather than by estimated analysis quality, which is undefined for competing interpretations.The ordering is a property of the space, not a verdict on an analysis.Source: §7.1 · p. 14

7.2 Limitations and Future Work

  • DAG has not been evaluated with analysts, so its design rationale and walkthrough do not establish how practitioners behave when using it.The proposed next step is a study with practitioners exploring unfamiliar datasets under questions they formulate themselves.Source: §7.2 · p. 14
  • The LLM-generated knowledge graph’s domain coverage depends on model knowledge and training-data representation, while the growing claim space has an unestablished human-legibility limit.Specialized vocabulary is especially likely to yield a plausible-looking graph practitioners may not recognize; editability shifts mitigation effort to the analyst.Source: §7.2 · pp. 14–15
  • Grounded terminal states could drive editable notebook regressions or bound a Boba multiverse using ruled-out candidates to delimit what it need not enumerate.Composable notebook cells could let terminal paths merge into an environment mirroring the analyst’s traversal.Source: §7.2 · p. 15
  • Future work could measure hypothesis-space coverage and containment, and rank options by directly computing divergence in what candidates unlock downstream.The paper notes that the cited grammar raises the possibility of such measures but does not itself implement them in a system.Source: §7.2 · p. 15
  • Evidence-backed commitments and deliberate path removals could inform ranking from both pursued and rejected directions, with demotions visible and reversible.How far removals should generalize and how strongly they should propagate remain open questions.Source: §7.2 · p. 15
Loading 2610.11025v1…