PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management

Zherui Yang, Fan Liu, Yansong Ning, Hao Liu

arXiv:2606.03841v1cs.AI

TL;DR

Existing autonomous data science agents rely on fixed workflows and struggle to acquire reusable skills and manage long-horizon context. EvoDS combines autonomous skill acquisition, adaptive context compression, and hierarchical multi-agent reinforcement learning, outperforming state-of-the-art open-source agents across four benchmarks. However, it still trails proprietary baselines on the hardest scientific discovery tasks.

  • Problem

    Existing autonomous data science agents rely on fixed workflows and action spaces, limiting systematic accumulation of reusable experience across tasks and iterations.

  • Method

    EvoDS combines autonomous skill acquisition and adaptive context compression within a hierarchical multi-agent architecture trained through a two-stage multi-agent scheme.

  • Results

    EvoDS significantly outperforms state-of-the-art open-source agents across four benchmarks and demonstrates the effectiveness of its proposed components.

  • Takeaways & Limitations

    EvoDS provides a framework for accumulating reusable skills and regulating long execution contexts while solving complex data science tasks end to end.

  • Takeaways & Limitations

    EvoDS remains behind proprietary baselines on the most difficult scientific discovery tasks, where performance is constrained by the underlying model’s scientific knowledge.

Abstract

from arXiv · show

Recent progress in Large Language Model (LLM) agents has enabled promising advances in automated data science. However, existing approaches remain fundamentally limited by their static action sets and lack of principled long-horizon context management, hindering their ability to accumulate reusable experience across tasks and operate reliably in multi-stage, iterative data science pipelines. To address these challenges, we introduce EvoDS, a self-evolving autonomous data science agent that learns to expand its skills and adaptively managing long-term context through agentic reinforcement learning. Specifically, EvoDS introduces two key strategies: (1) Autonomous Skill Acquisition (ASA) mechanism, which enables agents to synthesize, validate, and reuse executable skills; and (2) Adaptive Context Compression (ACC) strategy, which treats context management as a learned control problem rather than passive truncation. These strategies are orchestrated within a two-stage multi-agent training scheme, enabling EvoDS to autonomously improve over time. Theoretically, we prove that EvoDS's hierarchical design reduces tool-selection error, and its optimization objective aligns with an information bottleneck principle, ensuring efficient context use. Empirically, EvoDS outperforms state-of-the-art open-source data science agents by an average of 28.9% across four diverse benchmarks while eliminating out-of-token failures. Our code and data are available at https://github.com/usail-hkust/EvoDS.

1 Introduction

EvoDS addresses the fixed workflows and action spaces of existing autonomous data science agents by combining reusable skill acquisition with active long-term context regulation. Its hierarchical multi-agent design and two-stage training scheme support experience-driven improvement while reducing tool-selection errors and operating under context constraints.

  • Motivation: Existing autonomous data science agents remain constrained by predefined workflows and fixed action spaces, limiting exploratory and experience-driven problem solving.Systems including DS-Agent, AutoKaggle, and DeepAnalyze orchestrate LLMs with code execution and tool calling for data science tasks.
  • Theory and objectives: The hierarchical architecture theoretically reduces tool-selection errors, while the training scheme supports progressive improvement under context constraints.These properties address the challenges of acquiring reusable skills and managing explosive context over multi-stage workflows.
  • EvoDS: A Manager Agent coordinates specialized agents for data handling, modeling, visualization, and debugging, with scope-specific skills for decomposed executable subtasks.The architecture integrates skill acquisition and context regulation across specialized agents.
  • EvoDS: EvoDS combines Autonomous Skill Acquisition and Adaptive Context Compression in a unified hierarchical multi-agent framework.The framework enables agents to acquire reusable operational skills while actively controlling long-term context.
  • Training: EvoDS uses a two-stage multi-agent training scheme that jointly optimizes task execution, skill acquisition, and context regulation.This scheme is intended to help the agent improve with experience while remaining robust under context constraints.

2 Related Works

Related work on autonomous data science agents spans workflow-based automation, inference-time self-evolution, context compression, and agent optimization through supervised or reinforcement learning. Existing methods respectively rely on predefined pipelines, iterative adaptation without parameter updates, heuristic context retention, and expert imitation or environment feedback.

  • Autonomous Data Science Agents: Autonomous data science agents automate exploratory analysis, feature construction, predictive modeling, and result interpretation, with early systems relying on predefined workflow pipelines.DS-Agent retrieves relevant cases and iteratively generates and debugs code within a predefined pipeline.
  • Self-Evolving LLM Agents: Inference-time self-evolving agents improve prompts, memories, tools, skills, or frameworks without updating LLM parameters.Prompt-based methods refine prompts or reasoning templates from task feedback, while memory-based methods accumulate and retrieve execution experiences.
  • Context Compression: Context compression supports long-horizon reasoning through threshold-triggered summarization or task decomposition that retains task-critical information.Summarization reduces memory usage while preserving coarse-grained information; decomposition folds intermediate execution processes and keeps relevant content.
  • Agent Optimization: Agent optimization commonly uses SFT for stable initialization from expert demonstrations and RL for feedback-driven exploration, robustness, and adaptability.Most prior work focuses on optimizing single-agent systems.

3 Preliminary

The preliminary section formalizes agent skills, contexts, and data science agents, then defines data science tasks as multi-step interactions evaluated by task-specific outcome functions.

  • Core Definitions: An agent skill is a named, described, and executable code-based capability, represented as a = ⟨n,d,c⟩.Skills are treated as executable tools that agents can invoke as actions.
  • Core Definitions: An agent’s context C_t contains the task description and interaction history accumulated during the preceding t−1 steps.
  • Core Definitions: A data science agent π_θ is an LLM-based policy that selects actions from A using its current context, receives environmental feedback, and updates that context.Example actions include code execution, tool invocation, and textual response.
  • Task Formulation: A data science task pairs a natural-language description q with a dataset D, requiring the agent to generate a solution through multi-step environmental interaction.
  • Task Formulation: Task quality is measured by R_outcome = F(ŷ), using reference comparison for ground-truth tasks or ranking and LLM-as-a-judge for open-ended tasks.Open-ended examples include machine learning and visualization.

4 Methodology

EvoDS combines a hierarchical multi-agent architecture with Autonomous Skill Acquisition and Adaptive Context Compression to manage specialized execution, reusable capabilities, and long-horizon context. A unified reinforcement-learning strategy jointly optimizes orchestration, subtask execution, skill acquisition, and context management.

  • Hierarchical Multi-Agent Architecture: A hierarchical architecture assigns a Manager Agent to global reasoning, task decomposition, and coordination while specialized agents execute domain-specific subtasks in localized skill spaces.The Manager uses minimal code-execution tools and delegates to Cleaner, Featurizer, Modeler, Visualizer, and Debugger agents, restricting decisions to domain-specific operations.
  • Autonomous Skill Acquisition: Autonomous Skill Acquisition lets sub-agents synthesize task-agnostic executable skills when assigned subtasks exceed their existing capabilities, then validate them through execution feedback and output validity.Effective skills are retained for reuse, whereas failed or invalid skills are discarded; the mechanism incrementally addresses capability gaps across evolving tasks.
  • Adaptive Context Compression: Adaptive Context Compression summarizes successful outcomes and failed-execution causes while preserving task progress, key decisions, and critical intermediate results relevant to the global objective.Sub-agent-level abstraction and Manager-level compression control context growth, mitigating token-budget limitations and lost-in-the-middle effects in multi-step workflows.
  • Reinforcement Learning Training: The Manager’s hybrid reward balances final solution quality with subtask completion, while penalizing trajectory context length and interaction turns relative to their budgets.The outcome reward measures solution quality, and the context and turn penalties encourage efficient coordination.
  • Reinforcement Learning Training: A unified training objective jointly optimizes main and segmented trajectories, using reward broadcasting and GRPO-based clipped-surrogate optimization for orchestration, execution, skill acquisition, and context management.This training strategy is designed to support robust long-horizon reasoning with adaptive skills and effective context management.

5 Theoretical Analysis

The theoretical analysis establishes that EvoDS’s hierarchical tool selection has a lower error bound than a flat framework and that the Manager Agent’s objective is equivalent to an Information Bottleneck problem. These results justify structured tool selection and context compression that preserves task-critical information while reducing irrelevant content.

  • Hierarchical Tool Selection: Hierarchical tool selection reduces uncertainty by decomposing a large decision space into smaller, structured sub-problems, improving selection robustness.This explanation accompanies Theorem 5.1’s tighter error bound.
  • Information Bottleneck Context Compression: The Manager Agent minimizes task-irrelevant information while preserving task-critical signals, theoretically justifying context compression that balances efficiency and decision quality.Here, C denotes global context, Z compressed context, and Y the final agent output.

6 Experiments

EvoDS is evaluated across four diverse data science benchmarks and generally establishes strong open-source performance, with gains attributed to skill acquisition, context compression, and joint reinforcement learning. Its advantages extend to long-horizon execution and skill reuse, although difficult scientific-discovery and domain-specialized tasks remain challenging.

  • Benchmark Evaluation: EvoDS is evaluated on DABench, DA-Code, ScienceAgentBench, and MLE-Dojo, covering data wrangling, exploratory analysis, and predictive modeling.DA-Code visualization is assessed using an LLM-as-a-judge strategy because EvoDS generates plots through tools rather than executable code.
  • Overall Performance: EvoDS establishes new state-of-the-art performance among open-source backbones across all four benchmarks, while EvoDS-evo consistently outperforms EvoDS.The reported comparison identifies Autonomous Skill Acquisition as the mechanism behind the EvoDS-evo improvement.
  • Proprietary Comparisons: 3.9%: EvoDS exceeds the best DeepSeek-V3.1-based baseline on average, while 15.5% separates it from the strongest GPT-4o-based baseline.EvoDS achieves the second-best average performance overall, surpassed only by ReAct with the o4-mini backbone.
  • Long-Horizon Tasks: 0.311: EvoDS’s MLEDojo score exceeds the best open-source baseline’s 0.136 and surpasses proprietary ReAct with o4-mini on this difficult long-horizon benchmark.The passage highlights especially large gains on DA-Code and MLE-Dojo, where EvoDS coordinates full end-to-end workflows.
  • Failure Cases: EvoDS remains limited on difficult scientific-discovery tasks and professional quantitative-finance tasks, where insufficient domain knowledge can produce flawed execution logic and failure.The scientific-discovery gap reflects requirements for deep domain knowledge and abstract reasoning beyond procedural tool use.
  • Ablation Study: Removing Adaptive Context Compression causes the most severe degradation among trained variants, while removing Autonomous Skill Acquisition also noticeably reduces performance.The ablations indicate that context management is essential for stable long-horizon execution and synthesized-tool reuse improves agent capability.
  • Skill Reuse: 2.9% and 9.3%: Skill reuse improves performance on DA-Code and ScienceAgentBench, respectively, including under substantial distribution shifts.These gains show that skills synthesized from earlier tasks provide reusable capabilities rather than only task-specific procedures.

7 Conclusion

EvoDS is presented as a self-evolving autonomous data science agent addressing reusable-skill acquisition and long-horizon context-management limitations through hierarchical multi-agent reinforcement learning.

  • Conclusion: EvoDS targets existing LLM-based data science systems’ inability to acquire reusable skills from experience and manage long-horizon execution contexts.These limitations motivate the proposed agent.
  • Conclusion: EvoDS integrates a hierarchical multi-agent architecture with autonomous skill acquisition and adaptive context compression.These mechanisms are central components of the presented system.
  • Conclusion: EvoDS uses a joint reinforcement learning strategy for multi-role agents to support continuous improvement.The supplied passage identifies joint reinforcement learning as part of the system design.

A Theoretical Analysis · A.1 Notations

The section formalizes EvoDS’s tool-selection setting by defining task context, agents, available tools, policies, and tool utility. It distinguishes direct global-tool selection from hierarchical selection through partitioned tool subsets and a manager agent.

  • A.1 Notations: Task context C denotes information accumulated by the agent, while A denotes an LLM-based agent and T = {t_1, …, t_K} the global tool set.These notations are used throughout the theoretical analysis.
  • A.1 Notations: Given context C, tool selection chooses the most appropriate tool from T to solve the task.This is the formal definition of the tool-selection problem.
  • A.1 Notations: The selected tool is defined by t̂ = π_θ(C, T), where π_θ is the agent policy parameterized by θ.The policy selects tools using an internal scoring function conditioned on context representation ϕ(C).
  • A.1 Notations: A base agent selects tools directly from the global tool set T.This provides the non-hierarchical reference setting.
  • A.1 Notations: A hierarchical agent partitions T into N disjoint subsets, with subset T_j having size k_j, and a Manager Agent first selects a sub-agent indexed by j(C).The selected sub-agent then chooses a tool from the corresponding subset T_j(C).
  • A.1 Notations: Each tool t has true utility u(t|C) ∈ [0, 1], and the optimal tool is t*(C) = arg max_t∈T u(t|C).Utility is defined relative to the given context C.

A.2 Tool Selection Error Bound for Base Agent

The base agent selects the highest-scoring tool under independent Gaussian scoring noise whose variance depends on context uncertainty. Its tool-selection error is bounded using the minimum utility margin and a union bound over incorrect candidates.

  • Model: The base agent scores each tool as utility plus independent Gaussian noise, with context-dependent variance, and selects the argmax score.The noise variance is σ^2(ϕ(C)), reflecting uncertainty induced by the context representation.
  • Margin: The minimum utility margin Δbase(C) is the smallest gap between the optimal tool and any alternative, assumed positive almost surely.This margin determines how difficult it is for scoring noise to reverse the optimal choice.
  • Error Bound: The overall error bound follows by applying the union bound across all K−1 incorrect candidate tools.The supplied passage states that this aggregation yields the lemma’s result, but does not include the displayed bound.
  • Error Bound: For each incorrect tool, an error occurs when the difference between its noise and the optimal tool’s noise exceeds Δbase(C).The noise difference is Gaussian with variance 2σ^2(ϕ(C)).

A.3 Tool Selection Error Bound for Hierarchical Agent

The hierarchical agent decomposes tool selection into manager-level sub-agent choice and local tool choice, yielding a strictly smaller error-probability upper bound than a base agent under the stated assumptions. The bound is governed by manager and sub-agent margins, with localized context reducing variance for the optimal sub-agent.

  • Hierarchical Tool Selection: Hierarchical selection first chooses a sub-agent and then selects a tool from that sub-agent’s local tool set.The manager selects the sub-agent maximizing sub-agent utility, while the selected sub-agent chooses from its local tools.
  • Margin Decomposition: The manager margin Δ_M(C) measures the utility gap between the optimal sub-agent and the closest alternative, while Δ_S(C) measures the optimal sub-agent’s tool-selection gap.The global second-best tool either lies within the optimal sub-agent’s tool set or in another sub-agent’s set, so the relevant margin is determined by these two cases.
  • Context Specialization: Localized context for the optimal sub-agent has variance no greater than the full context, reflecting context specialization.Assumption 3 states σ^2(ϕ_j∗(C)) ≤ σ^2(ϕ(C)).
  • Error Bound: The hierarchical agent admits a strictly smaller upper bound on tool-selection error than the base agent under Assumptions 1–5.Its error occurs either when the manager selects a nonoptimal sub-agent or when the optimal sub-agent selects a nonoptimal tool.

A.4 Information-Theoretic Interpretation of the Optimization Objective

The Manager Agent’s context-compression objective trades task reward against compressed-context length. Under assumptions linking performance to mutual information and token cost to entropy, optimizing this objective is equivalent to an Information Bottleneck problem.

  • Optimization Objective: The relaxed objective maximizes expected task reward while penalizing the expected length of compressed context Z = g(C).It is defined as J(g) = E[R(Y)] − γ·E[|Z|].
  • Assumptions: Assumption 5 models expected task reward as an increasing function of mutual information I(Z;Y).Specifically, E[R(Y)] = f(I(Z;Y)) with f′(·) > 0.
  • Assumptions: Assumption 6 models expected context token cost as proportional to the entropy H(Z).The relationship is E[|Z|] = κH(Z), where κ > 0.
  • Information Bottleneck Equivalence: Under Assumptions 5–6, optimizing the Manager Agent’s objective is equivalent to the Information Bottleneck problem min p(z|c) I(Z;C) − λI(Z;Y).Because Z is a deterministic function of C, I(Z;C) = H(Z), yielding the Information Bottleneck form with λ > 0.

B Benchmarks • … E Prompts Used for EvoDS

The paper defines its evaluation benchmark, baseline agents, hierarchical executable skill suite, and prompts for coordinating EvoDS’s multi-agent data science workflow. These components cover benchmark task structure, agent specialization, tool use, fallback skill creation, and synthesized-tool configuration.

  • B Benchmarks •: DABench evaluates end-to-end LLM-agent data analysis using 257 questions from 52 real-world CSV files and executable-environment interaction.Its format-prompting strategy standardizes open-ended outputs into a closed form for automatic evaluation.
  • C Baselines •: Baselines include an AutoGen-based data science agent constructed following DSBench, alongside ReAct, LATM, and ML-Master2 approaches.AutoGen supports multi-agent collaboration, LATM generates reusable executable tools, and ML-Master2 targets ultra-long-horizon machine-learning engineering.
  • D Skills Used for EvoDS: EvoDS’s predefined executable skills are organized across a hierarchical multi-agent architecture covering the Manager Agent and specialized sub-agents.The suite is designed to support coordinated execution of data science workflows.
  • D.1 Manager Agent: The Manager Agent decomposes tasks, controls execution, coordinates agents, runs Bash, SQL, and Python, and compresses long interaction histories.It routes cleaning, feature engineering, modeling, visualization, and debugging tasks to specialized agents.
  • D.2 Cleaner Agent: The Cleaner Agent provides preprocessing skills for imputation, missing-data removal, outlier handling, deduplication, type conversion, and datetime formatting.These operations target data cleaning and quality improvement.
  • D.3 Featurizer Agent: The Featurizer Agent transforms, encodes, selects, scales, and reduces dataset features using methods such as one-hot, label, frequency, and target encoding.Its selection tools include correlation- and variance-based feature selection.
  • D.4 Modeler Agent / D.5 Visualizer Agent: The Modeler Agent trains, tunes, and evaluates models, while the Visualizer Agent generates plots for exploration and result analysis.Listed modeling skills include logistic and linear regression, random forests, XGBoost, and LightGBM; visualization skills include line, bar, histogram, boxplot, scatter, heatmap, pie, and pair plots.
  • E Prompts Used for EvoDS: EvoDS prompts define Manager and sub-agent roles, preserve global task context, constrain execution, specify input templates, and formalize synthesized-tool configurations.If existing cleaning tools cannot solve a subtask, the Cleaner prompt directs creation of a new tool; modeling prompts require loading, designing, training, and validating a model.
Loading 2606.03841v1…