A Survey of Data Agents: Emerging Paradigm or Overstated Hype?

• Data agents are comprehensive LLM-powered architectures that orchestrate Data + AI ecosystems across management, preparation, and analysis tasks.They operate on data within environments such as database systems, code interpreters, and APIs to produce task-specific outputs.Source: A. The Dawn of Data Agents · p. 2
• Their specialized capabilities include interactive data exploration, data-tool invocation, and adaptive handling of noise, inconsistencies, scalability constraints, and updates.Source: A. The Dawn of Data Agents · p. 2
• Recent systems range from database maintenance and preparation tools to natural-language database interfaces, visualization systems, and versatile operations across heterogeneous data lakes.Source: A. The Dawn of Data Agents · p. 2
• The term “data agent” is applied inconsistently, conflating systems with substantially different autonomy, reliability, and complexity.Source: B. The Terminological Ambiguity of… · p. 2
• This ambiguity can produce expectation mismatches, unclear accountability for harms, and barriers to industry development and technology adoption.Source: B. The Terminological Ambiguity of… · p. 3

Survey framework

• The survey applies the taxonomy through a structured literature review and roadmap covering autonomy progression, evolutionary gaps, capability boundaries, accountability, and future generative systems.The authors emphasize the L2-to-L3 transition and distinguish their review from surveys organized mainly around architectures or scenarios.Source: D. Comparison and Our Contributions · pp. 3–4, C. A Hierarchical Taxonomy for Data… · p. 3

Taxonomy levels

• The taxonomy spans L0 manual operations through L5 full autonomy, progressively transferring dominance and responsibility from humans to data agents.Humans move from hands-on operators to supervisors and eventually disengage, while agents evolve toward autonomous, generative data scientists.Source: A. The Overview of Hierarchical Taxonomy · p. 4

L0–L1

• L0 assigns all data management, preparation, and analysis to humans, whereas L1 provides stateless, query-responsive assistance under human control.L1 agents respond to queries, but humans retain responsibility for interacting with environments, integrating outputs, and verifying results.Source: A. The Overview of Hierarchical Taxonomy · p. 4

L2–L3

• L2 agents perceive environments and execute adaptive procedures, while L3 agents are expected to orchestrate diverse data pipelines under human supervision.This progression moves from human-designed workflows toward agent dominance over task execution.Source: A. The Overview of Hierarchical Taxonomy · p. 4

L4–L5

• L4 removes human supervision through proactive issue discovery and pipeline orchestration, while L5 envisions agents inventing novel methods without human involvement.L5 extends beyond applying existing methods toward advancing data management, preparation, and analysis.Source: A. The Overview of Hierarchical Taxonomy · p. 4

Evolutionary transitions

• The L2-to-L3 leap transfers task dominance from humans defining pipelines to agents autonomously orchestrating versatile and comprehensive data-related tasks.This transition is identified as critical because it changes the human role from workflow owner to supervisor.Source: B. The Evolutionary Leaps Between Levels · p. 4

Review organization

• The survey organizes its literature review by increasing autonomy, using the taxonomy to analyze data agents and the challenges involved in transitions between levels.The following review begins with manual processes and the initial introduction of L1 agents.Source: B. The Evolutionary Leaps Between Levels · p. 4, §III · p. 4

L0: Manual operations

• At L0, humans manually orchestrate and execute data-management, preparation, and analysis pipelines without agent involvement.Examples include database tuning and diagnosis, scripted cleaning and integration, SQL analysis, visualization curation, and interpretation.Source: A. L0: Manual Labor in Early Ages · pp. 4–5

Motivation

• The traditional workflow is labor-intensive and time-consuming, requiring substantial domain knowledge and technical skill that creates barriers for non-experts.LLMs have begun providing preliminary assistance for data-related tasks.Source: A. L0: Manual Labor in Early Ages · p. 5

• L1 agents respond to user queries with organized answers or code snippets, while humans retain pipeline orchestration and execution.Source: B. L1: Preliminary Assistance · p. 5

Configuration Tuning

• Configuration-tuning systems use LLMs to generate recommendations or candidates, sometimes combined with Bayesian optimization or cost-model-based selection.Source: C. L1 Assistance in Data Management · p. 5

Query Optimization

• Query-optimization agents apply LLM knowledge to logical rewriting and physical plan selection, using prompting, fine-tuned models, and list-wise cost modeling.Source: C. L1 Assistance in Data Management · pp. 5–6

System Diagnosis

• Diagnosis agents compare regressed plans with efficient instances or retrieve historical and domain-specific context to generate configuration recommendations.Source: C. L1 Assistance in Data Management · p. 6

• Because L1 agents lack real-time monitoring and environmental interaction, they cannot adaptively diagnose or recover systems without human integration and verification.Source: B. L1: Preliminary Assistance · p. 5, C. L1 Assistance in Data Management · p. 6

• L1 agents support preparation tasks including profiling, annotation, cleaning, schema matching, entity resolution, and data discovery.Source: D. L1 Assistance in Data Preparation · p. 6

Data Cleaning

• Cleaning agents either generate direct answers, such as missing-value imputations, or produce intermediate code and formal quality rules for users or traditional systems to execute.Source: D. L1 Assistance in Data Preparation · p. 6

Profiling

• Prompt-based profiling combines statistical, semantic, domain-specific, or task-specific information to produce dataset summaries and interpretable views.Source: D. L1 Assistance in Data Preparation · p. 6

Annotation and Grounding

• Retrieval and knowledge-graph grounding improve robustness, relevance, and semantic consistency for profiling and annotation.Source: D. L1 Assistance in Data Preparation · pp. 6–7

Structured Data Analysis

• For structured analysis, L1 systems use prompt engineering, instruction fine-tuning, and neuro-symbolic parse-then-execute workflows for TableQA and NL2SQL.Source: E. L1 Assistance in Data Analysis · p. 7

• L1 prompt-based assistance significantly improves developer efficiency by offloading trivial and routine operations.Source: F. Progress and Limitations of L1 Data… · p. 8
• L1 agents remain stateless and cannot autonomously perceive or interact with external environments or data systems.Source: F. Progress and Limitations of L1 Data… · p. 8
• Human analysts must manually execute, integrate, verify, and optimize outputs, keeping user oversight responsible for correctness and procedure management.Source: F. Progress and Limitations of L1 Data… · p. 8

• L2 agents connect to real-world data systems, explore sources, execute operations, invoke tools, and adaptively optimize outputs.Source: A. Partial Automation: From Responder… · p. 8
• L2 interaction alleviates the repetitive manual cycle of integrating agent outputs and interacting with the environment.Source: A. Partial Automation: From Responder… · p. 8
• ReFoRCE exemplifies L2 by exploring database columns and translating and optimizing natural-language queries into SQL through database interaction.Source: A. Partial Automation: From Responder… · p. 8
• Table III compares representative L2 agents across capabilities such as perception, planning, memory, tool invocation, reflection, and data complexity.Source: Table III · p. 9
• L2 autonomy remains partial because agents operate within human-orchestrated pipelines and cannot independently devise workflows for diverse, comprehensive tasks.Source: A. Partial Automation: From Responder… · p. 9

Data Management

• L2 data agents iteratively optimize configuration tuning, query plans, system diagnoses, cleaning strategies, and integration processes through execution feedback, monitoring, or structured knowledge.These mechanisms include database performance metrics, external verifiers, code interpreters, memory modules, and evolving diagnostic knowledge graphs.Source: B. L2 Data Agents in Data Management · pp. 9–10, C. L2 Data Agents in Data Preparation · pp. 10–11

Query Optimization

• Query-optimization agents refine alternative queries or execution plans through database feedback, external rewrite engines, equivalence verifiers, and reward signals.These systems improve candidate plans iteratively but rely on human-designed transformation rules, pipelines, and evaluation mechanisms.Source: B. L2 Data Agents in Data Management · p. 10

System Diagnosis

• System-diagnosis agents collect evidence, localize database issues, and generate actionable recommendations using execution feedback and graph-based diagnostic experience.DBdoctor monitors SQL execution and resource usage, while more advanced systems coordinate specialized agents or retrieve relevant knowledge subgraphs.Source: B. L2 Data Agents in Data Management · p. 10

Limitations

• Across data-management tasks, L2 agents cannot independently initiate or redesign procedures beyond predefined diagnostic paths, knob sets, and strategic pipelines.They remain partially autonomous because humans specify the workflow, available procedures, or collaboration mechanisms.Source: B. L2 Data Agents in Data Management · p. 10

Data Cleaning

• Data-cleaning agents use execution environments, internal reflection, repair-detection cycles, and downstream model performance to refine cleaning actions.Despite these feedback loops, their actions remain confined to human-designed workflows and task-specific architectures.Source: C. L2 Data Agents in Data Preparation · pp. 10–11

TableQA

• L2 analysis agents use execution feedback, state transitions, specialized APIs, and multi-agent collaboration to make TableQA adaptive and support iterative planning, correction, and replanning.Systems such as ReAcTable and Chain-of-Table expose intermediate tables as feedback, while role-based agents coordinate planning, execution, evaluation, and revision.Source: D. L2 Data Agents in Data Analysis · p. 12

Limitations

• Across analysis tasks, L2 systems remain limited by predefined tools, operators, APIs, and procedural workflows that restrict strategic reasoning and broader data-lifecycle planning.Their feedback can be superficial or tactical, producing unproductive loops and limiting adaptation to unforeseen problems.Source: D. L2 Data Agents in Data Analysis · pp. 12–13

NL2SQL

• L2 NL2SQL agents interact with databases, refine queries using execution feedback, and decompose complex SQL reasoning through designed planning and checking pipelines.These systems combine schema exploration, query-plan reasoning, divide-and-conquer decomposition, and tool-assisted validation.Source: D. L2 Data Agents in Data Analysis · p. 12

NL2VIS

• L2 visualization agents execute code, inspect visual outputs, and iteratively debug or refine charts through multimodal feedback, specialized agents, or reinforcement-learning rewards.Their workflows support code correction, visual error detection, divide-and-conquer planning, and closed-loop improvement.Source: D. L2 Data Agents in Data Analysis · p. 12

Unstructured Data Analysis

• For unstructured documents, L2 agents navigate long or multimodal sources, retrieve information under uncertainty, and verify outputs through structured representations and self-assessment.These methods address context-window limits, factual grounding, retrieval relevance, and analysis cost.Source: D. L2 Data Agents in Data Analysis · pp. 12–13

Dependence on human-defined pipelines

• L2 data agents depend on human-defined pipelines and cannot independently orchestrate workflows tailored to high-level or evolving task goals.CleanAgent, for example, follows a human-crafted architecture rather than designing the complete cleaning process itself.Source: E. The Glass Ceiling of L2 Data Agents · p. 13

Task-specific rigidity

• Task-specific rigidity limits L2 agents’ generalizability across data integration, management, preparation, and analysis.Specialized systems such as Chase-SQL are designed for narrow subtasks and cannot readily transfer to broader data-related problems.Source: E. The Glass Ceiling of L2 Data Agents · p. 13

The L2 autonomy ceiling

• Together, pipeline dependence and task specificity leave L2 agents as reactive executors rather than self-directed orchestrators of comprehensive data solutions.The section characterizes this boundary as a glass ceiling on L2 autonomy.Source: E. The Glass Ceiling of L2 Data Agents · p. 13

L3: Conditional Autonomy

• L3 data agents autonomously orchestrate and optimize comprehensive pipelines spanning data management, preparation, and analysis while humans retain supervisory responsibility.The formalization assigns pipeline orchestration and execution to the agent, with human supervision over both.Source: A. The Critical Leap: From “Executor”… · p. 14
• No existing system fully realizes the versatile, self-directed orchestration required for a complete L3 data agent.The survey therefore characterizes current efforts as Proto-L3 systems rather than fully autonomous L3 systems.Source: A. The Critical Leap: From “Executor”… · p. 14

Emerging Efforts and Proto-L3 Systems

• Proto-L3 efforts aim to transform manually designed procedures into autonomously orchestrated and optimized pipelines for diverse, complex data tasks.Data Interpreter uses hierarchical task and action graphs with iterative refinement based on executor feedback.Source: B. Emerging Efforts and Proto-L3 Systems · p. 14
• Emerging systems broaden orchestration toward data lakes, heterogeneous sources, multi-agent workflows, and industrial data platforms, but coverage remains uneven and often analysis-focused.Examples include SiriusBI, iDataLake, AOP, Snowflake Cortex, BigQuery, ByteDance Data Agent, and TabTab.Source: B. Emerging Efforts and Proto-L3 Systems · p. 14
• Proto-L3 systems still rely on predefined operators, tools, collaboration mechanisms, constrained task scopes, tactical reasoning, and limited self-evolution.Tool Evolution and LLM-generated operators begin addressing these constraints, but the survey identifies true L3 autonomy as unresolved.Source: B. Emerging Efforts and Proto-L3 Systems · pp. 14–15

Challenges

• Proto-L3 systems remain constrained by predefined tools, narrow data-lifecycle coverage, limited advanced reasoning, and static-environment operation.These gaps motivate research toward true L3 autonomy and beyond.Source: C. Challenges and Research… · p. 15
• Current agents can become trapped in unproductive loops because they address failed-step symptoms without reassessing overarching strategies or root causes.Proposed directions include causal reasoning, meta-reasoning, and memory for abstract strategic knowledge.Source: C. Challenges and Research… · p. 15

L4 Vision

• L4 data agents are envisioned to monitor and explore data lakes independently, identify valuable emerging tasks, and shift humans from supervisors to passive recipients.This autonomy requires autonomous problem discovery, trustworthy self-governance, and long-horizon views.Source: A. L4: High Automation · pp. 15–16
• L4 agents must discover latent data issues, manage responsibilities and resources securely, and optimize strategic trade-offs across extended periods.The vision combines curiosity about data, reliable generalist capabilities, and holistic long-term planning.Source: A. L4: High Automation · p. 16

L5 Vision

• L5 data agents would create new knowledge by recognizing when existing methods are insufficient and developing novel theories, algorithms, paradigms, or visualization grammars.Examples include new sampling theories and federated data-preparation frameworks.Source: B. L5: The Ultimate Vision of… · p. 16
• At L5, human engagement is envisioned as unnecessary while the agent independently performs data management, preparation, and analysis as a generative intellect.The proposed formalization has the agent identify a task, invent a process, produce an output, and leave the human disengaged.Source: B. L5: The Ultimate Vision of… · p. 16

Research Odyssey

• The progression from Proto-L3 through L3, L4, and L5 requires fundamental breakthroughs in autonomous orchestration, versatility, reasoning, motivation, planning, and safety.The survey characterizes L4 and L5 as long-term visions while identifying extended autonomous management with reduced human intervention as an achievable step.Source: B. L5: The Ultimate Vision of… · p. 16
• The survey’s six-level taxonomy organizes autonomy shifts and reviews technical gaps toward proactive and generative data agents.It emphasizes the L2-to-L3 transition while outlining future opportunities toward L4–L5.Source: §VII · p. 16, Figure 8 · p. 16