PaperlayerView exact revision on arXiv

Source-linked AI summary

Yunjue Agent Tech Report: A Fully Reproducible, Zero-Start In-Situ Self-Evolving Agent System for Open-Ended Tasks

Haotian Li, Shijun Yang, Weizhen Qi, Silei Zhao, Rui Hua, Mingzhu Song, Xiaojian Yang, Chao Peng

arXiv:2601.18226v2cs.AI

TL;DR

Open-ended environments expose the limits of agents built around static tools, offline supervision, and rigid capability boundaries. The paper introduces in-situ self-evolution through tool synthesis, refinement, and reuse, with Parallel Batch Evolution for efficiency. Across five benchmarks, zero-start evaluations outperform proprietary baselines, warm-start evaluations transfer knowledge to novel domains, and the authors identify tool-evolution stability and broader memory/workflow evolution as remaining boundaries.

  • Problem

    Conventional agents rely on static toolsets or offline supervision, limiting adaptation when task distributions drift and external supervision is scarce.

  • Method

    Yunjue Agent treats sequential interactions as experience, synthesizes and refines reusable tools, and uses Parallel Batch Evolution to consolidate concurrent tool generation.

  • Results

    Across five benchmarks, zero-start evaluations outperform proprietary baselines, while warm-start evaluations show accumulated general knowledge transfers to novel domains.

  • Takeaways & Limitations

    Tool evolution provides a route to zero-start learning and transferable capabilities in open-ended environments.

  • Takeaways & Limitations

    The framework currently validates self-evolution primarily through tools, leaving co-evolution of memory architectures and workflow policies for future work.

Abstract

from arXiv · show

Conventional agent systems often struggle in open-ended environments where task distributions continuously drift and external supervision is scarce. Their reliance on static toolsets or offline training lags behind these dynamics, leaving the system's capability boundaries rigid and unknown. To address this, we propose the In-Situ Self-Evolving paradigm. This approach treats sequential task interactions as a continuous stream of experience, enabling the system to distill short-term execution feedback into long-term, reusable capabilities without access to ground-truth labels. Within this framework, we identify tool evolution as the critical pathway for capability expansion, which provides verifiable, binary feedback signals. Within this framework, we develop Yunjue Agent, a system that iteratively synthesizes, optimizes, and reuses tools to navigate emerging challenges. To optimize evolutionary efficiency, we further introduce a Parallel Batch Evolution strategy. Empirical evaluations across five diverse benchmarks under a zero-start setting demonstrate significant performance gains over proprietary baselines. Additionally, complementary warm-start evaluations confirm that the accumulated general knowledge can be seamlessly transferred to novel domains. Finally, we propose a novel metric to monitor evolution convergence, serving as a function analogous to training loss in conventional optimization. We open-source our codebase, system traces, and evolved tools to facilitate future research in resilient, self-evolving intelligence.

1. Introduction

Yunjue Agent addresses the rigidity of conventional agents by enabling in-situ self-evolution through tool synthesis and reuse without external supervision. Across five benchmarks, it reports zero-start gains, cross-domain transfer, convergence monitoring, and economical operation.

  • Conventional agents remain constrained by predefined operational boundaries and often rely on offline supervision or narrow domains.
  • The In-situ Self-Evolving framework treats task interactions as a continuous experience stream for accumulating generalized knowledge from a zero-start phase.
  • Yunjue Agent uses multiple specialized agents to synthesize bespoke Python primitives when existing capabilities are insufficient.
  • Parallel Batch Evolution distills high-quality tools from concurrent executions to improve evolutionary efficiency and accelerate convergence.
  • Evaluations across five benchmarks report zero-start gains over proprietary baselines, transfer to novel domains, convergence monitoring, and economical operation.

2. In-situ self-evolving agents

The paper frames an agent as workflow, context, and tools, then proposes continuously updating these components in situ without external supervision. Its focused formulation fixes workflow and context while evolving the toolset as the system processes queries.

  • An agent system is represented as M = ⟨W, C, T⟩, comprising workflow, context, and available tools.
  • Existing self-evolving methods commonly require iterative training or remain confined to specific components or domains.
  • In-situ self-evolving continuously updates agent components while operating without external supervision signals.
  • The general formulation evolves M from M0 to MT after each query by modifying workflow, context, or tools using internal feedback or experience.
  • For focused study, Yunjue fixes workflow and context and reduces the evolving state to Mt = ⟨W0, C0, Tt⟩.

3. Methodology

Yunjue evolves capabilities by retrieving, synthesizing, refining, and accumulating tools, while Parallel Batch Evolution combines concurrent execution with shared tool knowledge. Batch aggregation and merging reduce redundancy and support throughput and stability.

  • 3.1. In-situ self-evolving via tool accumulation: Yunjue treats continuous toolset evolution as a prerequisite for generalist agents and implements it through in-situ tool accumulation.
  • 3.1. In-situ self-evolving via tool accumulation: For each query, the agent retrieves relevant tools, synthesizes missing capabilities, executes the task, and refines tools using errors and execution traces.
  • 3.1. In-situ self-evolving via tool accumulation: The repository expands in breadth through new capabilities and in depth through iterative refinement, approaching convergence except for outlier queries.
  • 3.2. Parallel batch evolution: Parallel Batch Evolution processes query batches concurrently while preserving a cohesive evolutionary trajectory and shared knowledge accumulation.
  • 3.2. Parallel batch evolution: The tool-absorbing mechanism clusters semantically similar local and global tools, then merges them into a compact pool that filters quality and redundancy.
  • 3.2. Parallel batch evolution: Batch processing improves throughput, reduces evolutionary stochasticity through merging, and provides multiple parallel tool-creation rollouts for selection.
  • 3.1. In-situ self-evolving via tool accumulation: The multi-agent architecture separates management, execution, tool development, integration, aggregation, and merging for in-situ evolution.

4. Experiment setup

The evaluation spans five complementary benchmarks and compares Yunjue Agent with proprietary and open-source systems across static and self-evolving paradigms. The agent is tested from an empty toolset, with tool access adjusted for benchmark-specific fairness.

  • Datasets: The study evaluates generalizability across five benchmarks covering diverse domains and task complexities.
  • Datasets: The benchmark suite includes HLE, DeepSearchQA, and xBench’s xSciQA subset, targeting expert reasoning, deep search, and professional science questions.
  • Baselines: Baselines include proprietary and open-source systems spanning static and self-evolving agent paradigms.

5. Evaluation on open-ended evolution

Yunjue Agent achieves strong zero-start performance across heterogeneous benchmarks while accumulating reusable tools, transferring capabilities across domains, and converging toward a compact, general tool library. Its evolutionary mechanisms improve execution success and efficiency relative to independent Python-interpreter processing, while backend-model choice remains flexible.

  • Zero-start performance: Yunjue Agent achieves state-of-the-art performance on DSQA, FSC, and xSciQA, while remaining competitive on HLE and xDS.On HLE it scores 48.0 versus 45.8 for the backend, and on xDS it scores 59.7.
  • Tool evolution and transfer: xSciQA performance improved from 76.5 to 80.2 during warm-start transfer, while new tool synthesis fell 55% on FSC and 100% on xSciQA and xDS.The HLE-evolved toolset was used to initialize evaluations on DSQA, FSC, xSciQA, and xDS.
  • Tool evolution and transfer: 97 tools were generated across the 2,500-query HLE corpus, after which tool growth across secondary datasets exhibited near-zero slopes.The late-HLE synthesis surge reflects shifts from mathematics toward social sciences and other disciplines.
  • Tool evolution and transfer: 71% of semantically clustered tools overlapped between zero-start and warm-start DeepSearchQA runs, including exact and functionally equivalent utilities.Examples include exact recovery of read excel and semantic alignment between send http request and fetch api data.
  • Convergence monitoring: Evolutionary Generality Loss declines as tool creation falls below its historical average, with transient spikes corresponding to domain-specific synthesis bursts.EGL is introduced as a real-time convergence indicator analogous to training loss.
  • Ablation and efficiency: 47.5–51.5 versus 40.0: batch-evolved agents outperform the independent Python-interpreter baseline, alongside tool success above 99% versus 81.8%.The baseline also consumes 518 tokens per invocation, while increasing batch size reduces latency and marginally improves accuracy.
  • Ablation and efficiency: Terminal tool-library sizes remain comparable across batch sizes, while token expenditures stay below the baseline and approach an asymptotic minimum.This suggests convergence toward a consistent capability equilibrium despite different batch sizes.
  • Backend robustness: GPT-5 and GPT-5-mini backends retain strong performance, indicating that the framework operates across backend architectures and model scales.With GPT-5, the agent surpasses the majority of established baselines; GPT-5-mini remains competitive despite reduced parameter scale.

6. Related work

Prior self-evolving agents adapt workflows, context, or tools, but related systems often depend on offline supervision, narrow domains, or limited autonomous environmental adaptation. Yunjue Agent builds on tool evolution for broader open-ended operation.

  • Self-evolving agents refine workflows, model parameters, context, or tools without necessarily relying on human intervention.
  • Tool-evolution systems synthesize executable utilities for embodied control and software-engineering tasks, while clustering functional equivalents can consolidate redundant tools.
  • Existing tool-reuse approaches preserve successful tools, but their optimization trajectories remain tied to particular settings.
  • Open-ended agent systems pursue cross-domain operation, yet scalable toolsets or execution-experience summaries may still lack autonomous environmental adaptation or sufficient knowledge.

7. Discussion and future work

The discussion interprets convergent tool evolution and zero-/warm-start consistency as evidence for reusable agentic knowledge, while identifying limitations and optimization directions for broader, more stable adaptation.

  • Paradigm parallels: towards system-level pre-training for agentic systems: Zero-start and warm-start consistency, together with convergent tool-library curves, suggests that task-solving capability can be learned and distilled as generalizable patterns.
  • Paradigm parallels: towards system-level pre-training for agentic systems: A future agentic pre-training paradigm would distill a foundation toolset before deployment, reducing reliance on test-time evolution for novel tasks.
  • Co-evolution of memory and workflow: Tool generation is necessary but insufficient for personalized assistants and complex process management, which also require evolving memory structures and workflow policies.
  • Evolutionary stability and regularization: LLM-generation stochasticity causes toolset variance across runs, motivating regularization methods for more deterministic evolutionary convergence.
  • Optimization of parallel batch evolution: Query ordering affects convergence because delaying foundational primitives can impede system maturation.
  • Optimization of parallel batch evolution: Low intra-batch diversity can support Best-of-N-style selection through redundancy, creating a trade-off between quality assurance and evolution efficiency.
  • Optimization of parallel batch evolution: Smaller early batches may consolidate general knowledge, whereas larger post-convergence batches may improve throughput for corner cases.

8. Conclusion

The conclusion presents In-Situ Self-Evolving and Parallel Batch Evolution as enabling autonomous adaptation through tool-based knowledge crystallization. It reports robust zero-start learning, transferable capabilities, and state-of-the-art performance across heterogeneous benchmarks.

  • The In-Situ Self-Evolving framework enables LLM-based agents to adapt and evolve within open-ended environments.
  • Yunjue Agent and Parallel Batch Evolution treat tools as dynamic vehicles for knowledge crystallization, supporting zero-start learning and generalized, transferable capabilities.
  • The approach achieves state-of-the-art performance across heterogeneous benchmarks and transfers accumulated generalized knowledge to novel domains in warm-start experiments.
  • The authors release the codebase, evaluation and system traces, and identify unified evolution of tools, memory, and workflows as future work.

A. Mathematical proof

The proof establishes a sufficient condition for the Evolutionary Generality Loss metric to decrease at the next step by transforming the ratio inequality into a comparison of marginal and historical synthesis rates.

  • The proof begins by targeting the monotonic-decrease condition Lt+1 < Lt for the EGL metric.
  • Because Ut is strictly positive, the ratio inequality can be cross-multiplied without reversing its direction.
  • The cross-multiplied condition is (Ct + ct+1) · Ut < Ct · (Ut + ut+1).
  • Expanding produces CtUt + ct+1Ut < CtUt + Ctut+1, after which the common CtUt term is removed.
  • The resulting descent condition compares the current marginal synthesis-to-invocation ratio ct+1/ut+1 with the historical cumulative ratio Ct/Ut.

A.2. Proof of theorem 5.2

The proof analyzes EGL under bounded knowledge and continuous activity assumptions, showing that it converges to zero at rate O(1/t) as system stability is achieved.

  • Assumptions: Knowledge boundedness assumes synthesized tools converge to a finite upper bound Cmax, so marginal tool generation vanishes asymptotically.The assumption requires a finite set of optimal primitives sufficient to resolve potential queries.
  • Assumptions: Continuous activity assumes newly synthesized tools are invoked and cumulative invocations Ut grow linearly with evolutionary steps.The invocation count is assumed to fluctuate around a stationary mean µ ≥1.
  • Convergence: EGL functions mathematically as a loss metric, converging to zero at a rate of O(1/t) as system stability is achieved.The derivation relies on Assumptions A.1 and A.2 and positive constants Cmax and µ.
  • Pipeline: The execution and evolution pipeline processes query batches in parallel, synthesizing local tools when the Manager identifies capability gaps.The Executor uses the provisioned local toolset and returns answers while updating the global library.

C. Experiment setup

The experiment setup evaluates Yunjue Agent across five benchmarks using standard dataset protocols, reported statistics, and controlled implementation settings.

  • Datasets: The evaluation covers five benchmarks, with dataset statistics summarized in Table 4.The supplied materials identify HLE, DeepSearchQA, FinSearchComp, and xBench evaluation protocols.
  • Evaluation protocols: Standard evaluation protocols use official scripts for HLE, DeepSearchQA, and xBench, while FinSearchComp uses judging prompts.The protocols are described as following each dataset’s released evaluation procedure.
  • Metrics: Table 5 reports zero-start performance, with accuracy computed as the mean and standard deviation over three LLM-as-judge runs.Token denotes average token consumption, Exec. Ratio denotes Executor-token proportion, and Cost denotes monetary expense.
  • Baselines: Baseline results are collected from official reports or leaderboards and cited source publications.The listed baselines include proprietary systems and results reported by their respective organizations or authors.
  • Implementation: All system nodes use Gemini 3 Pro, with batch size B = 16, Codex powering Tool Developer, and temperature fixed at 0.7.Image processing is encapsulated as a dedicated tool for multimodal functionality.

D. More evaluation results

Additional evaluations examine detailed zero-start performance, EGL behavior across benchmarks and batch sizes, and model-dependent execution patterns.

  • Benchmark results: Table 5 provides detailed zero-start results across five datasets, reporting accuracy means and standard deviations over three runs.Accuracy is evaluated using an LLM-as-judge.
  • EGL analysis: EGL trajectories across DSQA and FSC, together with batch-size analyses, validate asymptotic stability across experimental conditions.Figure 8 tracks DSQA and FSC EGL dynamics, while Figure 9 examines batch-size effects.
  • Execution pipeline: The execution pipeline integrates generation and execution as the agent synthesizes, deploys, and iteratively refines tools for each query.Figure 10 presents this end-to-end process.
  • Technical insights: Gemini 3 Pro tends to use fewer tools, while GPT-5 and GPT-5-mini typically invoke more tools, planning, and reasoning steps.The GPT-5 series creates more tools and shows stronger instruction-following under added prompt constraints.

E. Case study

The case study illustrates runtime tool creation, tool consolidation, and contrasts the approach with a Python-only baseline’s context contamination.

  • Tool evolution: When an initial tool fails, Yunjue Agent detects the error, refines its strategy, and derives the final answer.The pipeline delegates tool creation to the Tool Developer and supports self-correction during execution.
  • Tool absorbing: After batch execution, functionally analogous tools are clustered and merged into a compact generalized toolset, pruning redundancy.The mechanism is illustrated in Figure 11 and can consolidate multiple web scrapers into one robust tool.
  • Baseline comparison: The Python-only baseline accumulates erroneous execution traces, causing context contamination and reasoning degradation in complex tasks.The case study highlights library-dependency problems and high-friction debugging loops in medical diagnosis.
  • Tool similarity: Independent evolutionary runs converge on similar HTTP functionality despite different tool names and initialization settings.The compared tools are send_http_request in zero-start and fetch_api_data in warm-start DeepSearchQA.
  • Tool orchestration: The tool-generation protocol prioritizes available atomic tools and requests new tools only when existing capabilities cannot satisfy the task.It discourages composite tools that merely combine available atomic capabilities.
Loading 2601.18226v2…