Source-linked AI summary
A Survey of Data Agents: Emerging Paradigm or Overstated Hype?
Yizhang Zhu, Liangwei Wang, Chenyu Yang, Xiaotian Lin, Boyan Li, Wei Zhou, Xinyu Liu, Zhangyang Peng, Tianqi Luo, Yu Li, Chengliang Chai, Chong Chen, Shimin Di, Ju Fan, Ji Sun, Nan Tang, Fugee Tsung, Jiannan Wang, Chenglin Wu, Yanwei Xu, Shaolei Zhang, Yong Zhang, Xuanhe Zhou, Guoliang Li, Yuyu Luo
TL;DR
Data agents lack a shared terminology that distinguishes simple responders from sophisticated autonomous systems, creating expectation and accountability challenges. This survey introduces a six-level autonomy taxonomy, reviews data-agent research through it, and analyzes gaps in the L2-to-L3 transition. It concludes with a roadmap toward proactive, generative data agents while noting that current L1 and emerging L2/L3 systems remain limited in autonomy and strategic reasoning.
Problem
Inconsistent use of “data agent” conflates systems with different autonomy, creating expectation mismatches, accountability challenges, and barriers to industry development.
Method
The survey proposes a hierarchical L0–L5 taxonomy, conducts a structured review organized by autonomy, and analyzes evolutionary gaps with emphasis on L2-to-L3 progression.
Results
The survey identifies significant gaps toward true L3 autonomy, including the need for causal reasoning, meta-reasoning, and memory architectures for strategic knowledge.
Takeaways & Limitations
The taxonomy clarifies capability boundaries and accountability while providing a roadmap toward proactive and generative data agents.
Takeaways & Limitations
L1 agents remain stateless and require human execution, integration, verification, and optimization, limiting them to atomic and static subtasks.
Abstract
from arXiv · showhide
The rapid advancement of large language models (LLMs) has spurred the emergence of data agents, autonomous systems designed to orchestrate Data + AI ecosystems for tackling complex data-related tasks. However, the term "data agent" currently suffers from terminological ambiguity and inconsistent adoption, conflating simple query responders with sophisticated autonomous architectures. This terminological ambiguity fosters mismatched user expectations, accountability challenges, and barriers to industry growth. Inspired by the SAE J3016 standard for driving automation, this survey introduces the first systematic hierarchical taxonomy for data agents, comprising six levels that delineate and trace progressive shifts in autonomy, from manual operations (L0) to a vision of generative, fully autonomous data agents (L5), thereby clarifying capability boundaries and responsibility allocation. Through this lens, we offer a structured review of existing research arranged by increasing autonomy, encompassing specialized data agents for data management, preparation, and analysis, alongside emerging efforts toward versatile, comprehensive systems with enhanced autonomy. We further analyze critical evolutionary leaps and technical gaps for advancing data agents, especially the ongoing L2-to-L3 transition, where data agents evolve from procedural execution to autonomous orchestration. Finally, we conclude with a forward-looking roadmap, envisioning proactive, generative data agents.
I. INTRODUCTION
Data agents are emerging as LLM-powered systems for orchestrating complex data tasks, but inconsistent terminology conflates systems with different autonomy and creates trust, accountability, and industry-development challenges.
- Data agents are comprehensive LLM-powered architectures that orchestrate Data + AI ecosystems across management, preparation, and analysis tasks.They operate on data within environments such as database systems, code interpreters, and APIs to produce task-specific outputs.
- Their specialized capabilities include interactive data exploration, data-tool invocation, and adaptive handling of noise, inconsistencies, scalability constraints, and updates.
- Recent systems range from database maintenance and preparation tools to natural-language database interfaces, visualization systems, and versatile operations across heterogeneous data lakes.
- The term “data agent” is applied inconsistently, conflating systems with substantially different autonomy, reliability, and complexity.
- This ambiguity can produce expectation mismatches, unclear accountability for harms, and barriers to industry development and technology adoption.
C. A Hierarchical Taxonomy for Data Agents
The survey proposes a six-level data-agent taxonomy inspired by SAE J3016, tracking increasing autonomy and the transfer of task responsibility from humans to agents. It uses this framework to classify existing systems, clarify capability boundaries and accountability, and identify research directions toward proactive, generative agents.
- Survey framework: The survey applies the taxonomy through a structured literature review and roadmap covering autonomy progression, evolutionary gaps, capability boundaries, accountability, and future generative systems.The authors emphasize the L2-to-L3 transition and distinguish their review from surveys organized mainly around architectures or scenarios.
- Taxonomy levels: The taxonomy spans L0 manual operations through L5 full autonomy, progressively transferring dominance and responsibility from humans to data agents.Humans move from hands-on operators to supervisors and eventually disengage, while agents evolve toward autonomous, generative data scientists.
- L0–L1: L0 assigns all data management, preparation, and analysis to humans, whereas L1 provides stateless, query-responsive assistance under human control.L1 agents respond to queries, but humans retain responsibility for interacting with environments, integrating outputs, and verifying results.
- L2–L3: L2 agents perceive environments and execute adaptive procedures, while L3 agents are expected to orchestrate diverse data pipelines under human supervision.This progression moves from human-designed workflows toward agent dominance over task execution.
- L4–L5: L4 removes human supervision through proactive issue discovery and pipeline orchestration, while L5 envisions agents inventing novel methods without human involvement.L5 extends beyond applying existing methods toward advancing data management, preparation, and analysis.
B. The Evolutionary Leaps Between Levels
Data-agent evolution is framed as a sequence of shifts from manual, labor-intensive data work to increasingly capable assistance and autonomy. The central transition is from agents executing human-designed procedures to agents orchestrating broader workflows.
- Evolutionary transitions: The L2-to-L3 leap transfers task dominance from humans defining pipelines to agents autonomously orchestrating versatile and comprehensive data-related tasks.This transition is identified as critical because it changes the human role from workflow owner to supervisor.
- Review organization: The survey organizes its literature review by increasing autonomy, using the taxonomy to analyze data agents and the challenges involved in transitions between levels.The following review begins with manual processes and the initial introduction of L1 agents.
- L0: Manual operations: At L0, humans manually orchestrate and execute data-management, preparation, and analysis pipelines without agent involvement.Examples include database tuning and diagnosis, scripted cleaning and integration, SQL analysis, visualization curation, and interpretation.
- Motivation: The traditional workflow is labor-intensive and time-consuming, requiring substantial domain knowledge and technical skill that creates barriers for non-experts.LLMs have begun providing preliminary assistance for data-related tasks.
B. L1: Preliminary Assistance
L1 data agents provide prompt-response assistance for discrete data tasks, generating answers, recommendations, code, or candidate configurations. Their stateless operation leaves execution, validation, and adaptive optimization under human control.
- L1 agents respond to user queries with organized answers or code snippets, while humans retain pipeline orchestration and execution.
- Configuration Tuning: Configuration-tuning systems use LLMs to generate recommendations or candidates, sometimes combined with Bayesian optimization or cost-model-based selection.
- Query Optimization: Query-optimization agents apply LLM knowledge to logical rewriting and physical plan selection, using prompting, fine-tuned models, and list-wise cost modeling.
- System Diagnosis: Diagnosis agents compare regressed plans with efficient instances or retrieve historical and domain-specific context to generate configuration recommendations.
- Because L1 agents lack real-time monitoring and environmental interaction, they cannot adaptively diagnose or recover systems without human integration and verification.
D. L1 Assistance in Data Preparation
L1 data agents assist data preparation through prompt-based profiling, semantic annotation, cleaning, integration, discovery, and structured analysis. They primarily generate answers, metadata, code, or rules in static, user-directed workflows.
- L1 agents support preparation tasks including profiling, annotation, cleaning, schema matching, entity resolution, and data discovery.
- Data Cleaning: Cleaning agents either generate direct answers, such as missing-value imputations, or produce intermediate code and formal quality rules for users or traditional systems to execute.
- Profiling: Prompt-based profiling combines statistical, semantic, domain-specific, or task-specific information to produce dataset summaries and interpretable views.
- Annotation and Grounding: Retrieval and knowledge-graph grounding improve robustness, relevance, and semantic consistency for profiling and annotation.
- Structured Data Analysis: For structured analysis, L1 systems use prompt engineering, instruction fine-tuning, and neuro-symbolic parse-then-execute workflows for TableQA and NL2SQL.
F. Progress and Limitations of L1 Data Agents
L1 assistance improves efficiency by offloading trivial and routine data operations, but remains limited to atomic, static subtasks. Humans must execute, integrate, verify, and optimize the generated outputs.
- L1 prompt-based assistance significantly improves developer efficiency by offloading trivial and routine operations.
- L1 agents remain stateless and cannot autonomously perceive or interact with external environments or data systems.
- Human analysts must manually execute, integrate, verify, and optimize outputs, keeping user oversight responsible for correctness and procedure management.
IV. L2: PERCEIVE THE ENVIRONMENT
L2 data agents perceive and interact with databases, files, and APIs, executing operations and invoking external tools within human-designed workflows. This reduces manual output integration, but L2 agents still cannot independently devise workflows for diverse, comprehensive tasks.
- L2 agents connect to real-world data systems, explore sources, execute operations, invoke tools, and adaptively optimize outputs.
- L2 interaction alleviates the repetitive manual cycle of integrating agent outputs and interacting with the environment.
- ReFoRCE exemplifies L2 by exploring database columns and translating and optimizing natural-language queries into SQL through database interaction.
- Table III compares representative L2 agents across capabilities such as perception, planning, memory, tool invocation, reflection, and data complexity.
- L2 autonomy remains partial because agents operate within human-orchestrated pipelines and cannot independently devise workflows for diverse, comprehensive tasks.
B. L2 Data Agents in Data Management
L2 data agents use environmental feedback, tools, and structured knowledge to iteratively optimize data-management procedures. Their autonomy remains bounded by predefined workflows, strategic paths, and task-specific designs.
- Data Management: L2 data agents iteratively optimize configuration tuning, query plans, system diagnoses, cleaning strategies, and integration processes through execution feedback, monitoring, or structured knowledge.These mechanisms include database performance metrics, external verifiers, code interpreters, memory modules, and evolving diagnostic knowledge graphs.
- Query Optimization: Query-optimization agents refine alternative queries or execution plans through database feedback, external rewrite engines, equivalence verifiers, and reward signals.These systems improve candidate plans iteratively but rely on human-designed transformation rules, pipelines, and evaluation mechanisms.
- System Diagnosis: System-diagnosis agents collect evidence, localize database issues, and generate actionable recommendations using execution feedback and graph-based diagnostic experience.DBdoctor monitors SQL execution and resource usage, while more advanced systems coordinate specialized agents or retrieve relevant knowledge subgraphs.
- Limitations: Across data-management tasks, L2 agents cannot independently initiate or redesign procedures beyond predefined diagnostic paths, knob sets, and strategic pipelines.They remain partially autonomous because humans specify the workflow, available procedures, or collaboration mechanisms.
- Data Cleaning: Data-cleaning agents use execution environments, internal reflection, repair-detection cycles, and downstream model performance to refine cleaning actions.Despite these feedback loops, their actions remain confined to human-designed workflows and task-specific architectures.
D. L2 Data Agents in Data Analysis
L2 data-analysis agents move beyond static querying by interacting with tables, databases, documents, tools, and visual outputs through iterative feedback. Yet their reasoning and adaptation remain bounded by predefined workflows, tools, and task-specific architectures.
- TableQA: L2 analysis agents use execution feedback, state transitions, specialized APIs, and multi-agent collaboration to make TableQA adaptive and support iterative planning, correction, and replanning.Systems such as ReAcTable and Chain-of-Table expose intermediate tables as feedback, while role-based agents coordinate planning, execution, evaluation, and revision.
- Limitations: Across analysis tasks, L2 systems remain limited by predefined tools, operators, APIs, and procedural workflows that restrict strategic reasoning and broader data-lifecycle planning.Their feedback can be superficial or tactical, producing unproductive loops and limiting adaptation to unforeseen problems.
- NL2SQL: L2 NL2SQL agents interact with databases, refine queries using execution feedback, and decompose complex SQL reasoning through designed planning and checking pipelines.These systems combine schema exploration, query-plan reasoning, divide-and-conquer decomposition, and tool-assisted validation.
- NL2VIS: L2 visualization agents execute code, inspect visual outputs, and iteratively debug or refine charts through multimodal feedback, specialized agents, or reinforcement-learning rewards.Their workflows support code correction, visual error detection, divide-and-conquer planning, and closed-loop improvement.
- Unstructured Data Analysis: For unstructured documents, L2 agents navigate long or multimodal sources, retrieve information under uncertainty, and verify outputs through structured representations and self-assessment.These methods address context-window limits, factual grounding, retrieval relevance, and analysis cost.
E. The Glass Ceiling of L2 Data Agents
L2 data agents achieve partial autonomy by executing and optimizing within human-designed workflows, but they remain reactive, task-specific systems rather than self-directed orchestrators.
- Dependence on human-defined pipelines: L2 data agents depend on human-defined pipelines and cannot independently orchestrate workflows tailored to high-level or evolving task goals.CleanAgent, for example, follows a human-crafted architecture rather than designing the complete cleaning process itself.
- Task-specific rigidity: Task-specific rigidity limits L2 agents’ generalizability across data integration, management, preparation, and analysis.Specialized systems such as Chase-SQL are designed for narrow subtasks and cannot readily transfer to broader data-related problems.
- The L2 autonomy ceiling: Together, pipeline dependence and task specificity leave L2 agents as reactive executors rather than self-directed orchestrators of comprehensive data solutions.The section characterizes this boundary as a glass ceiling on L2 autonomy.
V. L3: STRIVING FOR AUTONOMOUS DATA AGENT
L3 data agents are defined as conditionally autonomous systems that orchestrate and optimize comprehensive data pipelines under human supervision. Emerging Proto-L3 systems explore this direction, but none yet achieves full L3 autonomy because predefined components, limited scope, and insufficient self-evolution remain.
- L3: Conditional Autonomy: L3 data agents autonomously orchestrate and optimize comprehensive pipelines spanning data management, preparation, and analysis while humans retain supervisory responsibility.The formalization assigns pipeline orchestration and execution to the agent, with human supervision over both.
- L3: Conditional Autonomy: No existing system fully realizes the versatile, self-directed orchestration required for a complete L3 data agent.The survey therefore characterizes current efforts as Proto-L3 systems rather than fully autonomous L3 systems.
- Emerging Efforts and Proto-L3 Systems: Proto-L3 efforts aim to transform manually designed procedures into autonomously orchestrated and optimized pipelines for diverse, complex data tasks.Data Interpreter uses hierarchical task and action graphs with iterative refinement based on executor feedback.
- Emerging Efforts and Proto-L3 Systems: Emerging systems broaden orchestration toward data lakes, heterogeneous sources, multi-agent workflows, and industrial data platforms, but coverage remains uneven and often analysis-focused.Examples include SiriusBI, iDataLake, AOP, Snowflake Cortex, BigQuery, ByteDance Data Agent, and TabTab.
- Emerging Efforts and Proto-L3 Systems: Proto-L3 systems still rely on predefined operators, tools, collaboration mechanisms, constrained task scopes, tactical reasoning, and limited self-evolution.Tool Evolution and LLM-generated operators begin addressing these constraints, but the survey identifies true L3 autonomy as unresolved.
C. Challenges and Research Opportunities Towards True L3
Current Proto-L3 data agents remain limited in orchestration, lifecycle coverage, reasoning, and adaptation, while L4 envisions proactive, self-governing systems that discover tasks and plan holistically.
- Challenges: Proto-L3 systems remain constrained by predefined tools, narrow data-lifecycle coverage, limited advanced reasoning, and static-environment operation.These gaps motivate research toward true L3 autonomy and beyond.
- Challenges: Current agents can become trapped in unproductive loops because they address failed-step symptoms without reassessing overarching strategies or root causes.Proposed directions include causal reasoning, meta-reasoning, and memory for abstract strategic knowledge.
- L4 Vision: L4 data agents are envisioned to monitor and explore data lakes independently, identify valuable emerging tasks, and shift humans from supervisors to passive recipients.This autonomy requires autonomous problem discovery, trustworthy self-governance, and long-horizon views.
- L4 Vision: L4 agents must discover latent data issues, manage responsibilities and resources securely, and optimize strategic trade-offs across extended periods.The vision combines curiosity about data, reliable generalist capabilities, and holistic long-term planning.
B. L5: The Ultimate Vision of Ubiquitous and Generative Data Agents
L5 is the survey’s ultimate vision of a fully autonomous, generative data scientist that discovers tasks and invents new methods without human engagement. Reaching this vision requires fundamental breakthroughs beyond incremental improvements, with the taxonomy and roadmap providing a structured path forward.
- L5 Vision: L5 data agents would create new knowledge by recognizing when existing methods are insufficient and developing novel theories, algorithms, paradigms, or visualization grammars.Examples include new sampling theories and federated data-preparation frameworks.
- L5 Vision: At L5, human engagement is envisioned as unnecessary while the agent independently performs data management, preparation, and analysis as a generative intellect.The proposed formalization has the agent identify a task, invent a process, produce an output, and leave the human disengaged.
- Research Odyssey: The progression from Proto-L3 through L3, L4, and L5 requires fundamental breakthroughs in autonomous orchestration, versatility, reasoning, motivation, planning, and safety.The survey characterizes L4 and L5 as long-term visions while identifying extended autonomous management with reduced human intervention as an achievable step.
- Research Odyssey: The survey’s six-level taxonomy organizes autonomy shifts and reviews technical gaps toward proactive and generative data agents.It emphasizes the L2-to-L3 transition while outlining future opportunities toward L4–L5.