Source-linked AI summary
DeepEye: A Steerable Self-driving Data Agent System
Boyan Li, Yiran Peng, Yupeng Xie, Sirong Lu, Yizhang Zhu, Xing Mu, Xinyu Liu, Yuyu Luo
TL;DR
Existing data-agent systems struggle with heterogeneous sources, context explosion, and opaque linear workflows. DeepEye addresses these gaps with standardized multimodal nodes, context-isolated hierarchical reasoning, and a validated, optimized workflow engine. Its demonstration shows multimodal outputs alongside transparent, parallel, and human-in-the-loop execution.
Problem
Current data-agent systems are limited by siloed handling of structured and unstructured data, context explosion, opaque execution, and missed workflow parallelism.
Method
DeepEye uses a workflow-centric architecture with a Unified Node Protocol, context-isolated AgentNodes and deterministic ToolNodes, and a Compiler-Validator-Optimizer-Executor workflow engine.
Results
The demonstration orchestrates heterogeneous data analysis into Data Videos, Dashboards, and Analytical Reports, while supporting schema validation and transparent execution traces.
Takeaways & Limitations
DeepEye demonstrates a steerable data-agent workflow that combines multimodal synthesis with automated optimization and human-in-the-loop reliability.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have revolutionized natural language interaction with data. The "holy grail" of data analytics is to build autonomous Data Agents that can self-drive complex data analysis workflows. However, current implementations are still limited to linear "ChatBI" systems. These systems struggle with joint analysis across heterogeneous data sources (e.g., databases, documents, and data files) and often encounter "context explosion" in complex and iterative data analysis workflows. To address these challenges, we present DeepEye, a production-ready data agent system that adopts a workflow-centric architecture to ensure scalability and trustworthiness. DeepEye introduces a Unified Multimodal Orchestration protocol, enabling seamless integration of structured and unstructured data sources. To mitigate hallucinations, it employs Hierarchical Reasoning with context isolation, decomposing complex intents into autonomous AgentNodes and deterministic ToolNodes. Furthermore, DeepEye incorporates a database-inspired Workflow Engine (comprising a Compiler, Validator, Optimizer, and Executor) that guarantees structural correctness and accelerates execution via runtime topological optimization. In this demonstration, we showcase DeepEye's ability to orchestrate complex workflows to generate diverse multimodal outputs -- including Data Videos, Dashboards, and Analytical Reports -- highlighting its advantages in transparent execution, automated optimization, and human-in-the-loop reliability.
1 Introduction
DeepEye addresses the limitations of linear ChatBI systems in heterogeneous, complex data analysis by introducing a steerable, workflow-centric data agent. Its demonstration combines multimodal analysis with transparent, parallelizable, and human-refinable execution.
- Motivation: Existing systems remain siloed or linear, limiting trustworthy production deployment for complex multi-step data workflows.Specialized Text-to-SQL and general-purpose agent frameworks do not adequately handle complex workflows or provide transparent execution logic.
- Challenges: Heterogeneous analysis requires joint reasoning over structured databases and unstructured documents, which current systems do not unify.Text-to-SQL primarily handles databases, while RAG primarily handles documents.
- Challenges: Context explosion and sequential execution make complex agent workflows difficult to validate, debug, optimize, and run efficiently.Global contexts accumulate tool descriptions, observations, and reasoning histories, while sequential execution misses parallelism and increases latency.
- Approach: DeepEye uses Unified Multimodal Orchestration, Hierarchical Reasoning, and a database-inspired Workflow Engine to address these challenges.The system combines standardized AgentNodes and ToolNodes, context-isolated sub-agents, and static validation with runtime topological optimization.
- Demonstration: The Global Sales Performance Analysis demonstration produces Data Videos, Dashboards, and Reports while supporting node inspection and Validator-assisted manual refinement.The demonstration also highlights explicit context binding, parallel database and document processing, and human-in-the-loop reliability.
2 System Architecture
DeepEye represents analytical capabilities as standardized nodes and coordinates them through a planner and database-inspired workflow engine. Context isolation, static validation, runtime parallelization, feedback-driven replanning, and cloud-native execution support scalable workflows.
- Architecture: DeepEye organizes data-agent analysis around transparent workflow DAGs within a workflow-centric, cloud-native architecture.The architecture bridges flexible LLM reasoning with rigid data engineering.
- Unified Node Protocol: The Unified Node Protocol models each capability as a node with semantic descriptions, typed input and output ports, configuration, and execution logic.Descriptions support planner retrieval and tool selection, while schemas standardize artifacts exchanged between nodes.
- Node Types: ToolNodes execute deterministic rule-based operations, whereas AgentNodes perform probabilistic reasoning or nested sub-workflows in private contexts.ToolNodes require no reasoning context, while AgentNodes use local context windows and internal tools.
- Isolation and Planning: Context Isolation exposes only standardized interfaces to the global planner, shielding it from local intermediate reasoning and mitigating context overflow.The planner manages global dependencies while sub-agents handle local reasoning.
- Isolation and Planning: The planner retrieves schemas, documentation, and verified workflows before execution, then uses runtime feedback to re-plan failed workflows.Successful workflows can be archived automatically, while experts can save refined workflows as SOP templates.
- Workflow Engine: The Workflow Engine compiles plans, validates graph structure and schemas, optimizes independent nodes into parallel layers, and executes them asynchronously.Its phases include DAG construction, cycle and schema checks, Kahn-based topological optimization, scheduling, retries, shared storage, and logging.
- Infrastructure: DeepEye deploys containerized microservices with asynchronous scheduling, unified structured and unstructured storage, and sandboxed ToolNode execution.The stack uses Docker, FastAPI, Celery, Redis, PostgreSQL, and MinIO/S3.
3 Demonstration
The demonstration applies DeepEye to global sales analysis across uploaded records and domain documents. It shows parallel multimodal synthesis, workflow inspection, schema-error prevention, and auditable execution feedback.
- Scenario Setup: The Global Sales Performance Analysis scenario covers heterogeneous data ingestion through multimodal output generation.Its stated goal is to autonomously orchestrate heterogeneous sources and optimize workflow execution for diverse outputs.
- Scenario Setup: Users upload sales records and knowledge documents, then bind selected contexts to a natural-language request with the “@” feature.Selecting @Sales Database and @Financial Metrics grounds the query before planning.
- Automated Multimodal Analysis: The workflow places independent knowledge-search and database-read nodes in one execution layer for parallel processing.The Planner creates a logical plan, which the Workflow Engine compiles into a DAG displayed on the Workflow Canvas.
- Automated Multimodal Analysis: Successful execution synthesizes structured and unstructured results into a data video, dashboard, and analytical report.This demonstrates multimodal content generation from the combined analysis.
- Human-in-the-Loop Refinement: Users can inspect and intervene in workflow nodes, including reviewing SQL-generator inputs, outputs, and parameters.The scenario integrates DeepEye-SQL for query generation.
- Human-in-the-Loop Refinement: The Validator immediately raises a “Schema Mismatch” error when a manually edited SQL node connects incompatibly downstream.This supports safer manual refinement of the workflow.
- Human-in-the-Loop Refinement: Real-time logs and Planner “Thought-Action” traces make the rerun transparent and auditable.The Process Monitor displays execution feedback after the refined workflow is rerun.