Source-linked AI summary
AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents
Jiabin Tang, Tianyu Fan, Chao Huang
TL;DR
Existing LLM-agent frameworks typically require programming expertise, limiting access despite broad interest in automated agents. AutoAgent addresses this gap with a natural-language, self-developing agent operating system, and evaluations report strong performance on generalist and RAG tasks. The paper also identifies semantic-matching and dynamic-web-search issues in the GAIA evaluation protocol.
Problem
Current LLM-agent development remains dependent on programming expertise, although only 0.03% of the global population possesses the necessary skills.
Method
AutoAgent uses an autonomous, modular agent operating system to create and customize agents, tools, and workflows through natural language without coding.
Results
AutoAgent secured a stable top-two position on GAIA and significantly outperformed state-of-the-art RAG approaches.
Takeaways & Limitations
AutoAgent bridges high-level natural-language requirements with practical multi-agent systems and workflows for non-programming users.
Takeaways & Limitations
GAIA evaluation is limited by strict string matching that ignores semantic equivalence and by dynamic anti-automation mechanisms during web searches.
Abstract
from arXiv · showhide
Large Language Model (LLM) Agents have demonstrated remarkable capabilities in task automation and intelligent decision-making, driving the widespread adoption of agent development frameworks such as LangChain and AutoGen. However, these frameworks predominantly serve developers with extensive technical expertise - a significant limitation considering that only 0.03 % of the global population possesses the necessary programming skills. This stark accessibility gap raises a fundamental question: Can we enable everyone, regardless of technical background, to build their own LLM agents using natural language alone? To address this challenge, we introduce AutoAgent-a Fully-Automated and highly Self-Developing framework that enables users to create and deploy LLM agents through Natural Language Alone. Operating as an autonomous Agent Operating System, AutoAgent comprises four key components: i) Agentic System Utilities, ii) LLM-powered Actionable Engine, iii) Self-Managing File System, and iv) Self-Play Agent Customization module. This lightweight yet powerful system enables efficient and dynamic creation and modification of tools, agents, and workflows without coding requirements or manual intervention. Beyond its code-free agent development capabilities, AutoAgent also serves as a versatile multi-agent system for General AI Assistants. Comprehensive evaluations on the GAIA benchmark demonstrate AutoAgent's effectiveness in generalist multi-agent tasks, surpassing existing state-of-the-art methods. Furthermore, AutoAgent's Retrieval-Augmented Generation (RAG)-related capabilities have shown consistently superior performance compared to many alternative LLM-based solutions.
1 Introduction
AutoAgent addresses the accessibility gap in LLM-agent development by enabling natural-language, zero-code creation and customization. Its self-developing framework combines modular agent utilities, actionable planning, file management, and self-play customization, and performs strongly across generalist and RAG evaluations.
- Motivation: Only 0.03% of the global population possesses the programming expertise needed to build and customize current LLM agents.Existing frameworks require codebase navigation, API integration, and prompt-engineering skills.
- Approach: AutoAgent reframes agent development as a fully automated, language-driven process requiring zero programming expertise.Its stated capabilities include natural-language multi-agent building, self-managing workflow generation, and intelligent resource orchestration.
- Architecture: The framework combines foundational multi-agent utilities, an LLM-powered Actionable Engine, a self-managing file system, and self-play customization.These components support tool integration, workflow creation, and iterative agent improvement.
- Evaluation: AutoAgent secured second place on GAIA and significantly outperformed state-of-the-art RAG approaches in standardized evaluations.Case studies additionally examined self-development across diverse real-world scenarios.
2 Related Work and Preliminaries
The paper situates AutoAgent at the intersection of generalist multi-agent systems and workflow-driven coordination. It targets the expert knowledge and engineering burden involved in translating high-level requirements into executable agents and workflows.
- Related Work: Existing agent frameworks demonstrate capabilities in cooperative communication, customizable LLM combinations, and complex workflow automation.The related work includes systems such as CAMEL, AutoGen, MetaGPT, and workflow-oriented methods.
- Generalist MAS: Generalist multi-agent systems coordinate specialized agents through an orchestrator to solve multifaceted tasks collaboratively.Agents delegate subtasks through transfer actions and return results to the orchestrator.
- Workflow Design: Workflow-driven mechanisms provide stable coordination for deterministic processes, while generalist MAS offer greater flexibility for open-ended tasks.Generalist systems can incur higher costs, complexity, and compounding errors.
- Workflow Design: Common workflow patterns include routing, parallelization, and evaluator-optimizer coordination between agents.These patterns organize conditional transfers between agents for efficient task solving.
- Research Gap: AutoAgent’s central challenge is fully automating the translation from high-level user requirements to effective multi-agent systems and workflows.The proposed direction uses natural-language interaction to reduce dependence on expert knowledge and engineering expertise.
3 The AutoAgent Framework
AutoAgent is an operating-system-like, modular architecture for natural-language agent development and task execution. It coordinates specialized agents, tools, files, models, and actions through an orchestrator and an LLM-powered engine.
- System Architecture: AutoAgent combines Agentic System Utilities, an LLM-powered Actionable Engine, a Self-Managing File System, and Self-Play Agent Customization.Together, these components support agent development, resource coordination, data retrieval, and tailored workflows without coding.
- Specialized Agents: Specialized web, coding, and file agents provide extensible capabilities for browsing, code-driven tasks, and local data analysis.The architecture abstracts diverse operations behind agent-specific tools and environments.
- Agent Coordination: The orchestrator receives user tasks, decomposes them into subtasks, delegates them to suitable agents, and iterates until completion.It uses handoff tools to coordinate sub-agents and return results.
- Coding Agent: The Coding Agent executes code-related operations in an interactive terminal and runs them inside a secure Docker sandbox.The sandbox protects local data during code execution and supports third-party sandboxing integration.
- Actionable Engine: The LLM-powered Actionable Engine understands natural language, generates plans, and coordinates tasks across agents.LiteLLM standardizes requests across more than 100 models, while action-observation histories provide the agent state.
- Tool Use: AutoAgent supports both direct tool use and transformed XML-based tool use for generating executable actions.The transformed paradigm converts tool invocation into structured code generation that is parsed for arguments and related information.
3.3 Self-Managing File System
AutoAgent’s self-managing file system turns uploaded documents and archives into retrievable vector-database content. Its workflow-generation tools then create and coordinate agents through analyzed requirements, structured workflows, and constraint-based regeneration.
- File System: Uploaded text files and archives are normalized into text and stored in user-defined vector-database collections.Agents retrieve and generate from this managed memory using tools such as query_db and answer_query.
- Agent Generation: AutoAgent generates sophisticated agent systems and workflows from high-level descriptions, reducing the need for specialized domain expertise.The system uses available tools, agents, and workflows to construct the required components.
- Agent Generation: The profiling process analyzes user requirements and existing components before selecting tools and structuring agents.This analysis determines which available capabilities can be leveraged for the requested system.
- Tool Creation: The Tool Editor Agent integrates third-party APIs and tests generated tools, debugging them automatically when execution fails.The reported RapidAPI integration spans 145 APIs across 8 categories.
- Multi-Agent Workflows: For multi-agent requests, AutoAgent creates the required agents and an orchestrator that connects them using the Orchestrator-Workers pattern.Users can also specify desired tasks so the system generates agents and workflows tailored to collaborative objectives.
- Workflow Construction: AutoAgent models agent tasks as events and uses listening and triggering mechanisms instead of rigid graph structures.Workflow forms are checked against system constraints and regenerated with feedback when invalid.
4 Evaluation
AutoAgent is evaluated as a generalist multi-agent system, a retrieval-augmented generator, and a natural-language agent-development framework. Across these evaluations, it achieves strong benchmark performance and demonstrates automated creation of agents, tools, and workflows.
- Generalist Agent Evaluation: 466 test and 165 validation questions across three difficulty levels evaluate GAIA abilities including reasoning, multimodality handling, web browsing, and tool use.AutoAgent is evaluated on the GAIA validation set using success rate as the primary metric.
- Generalist Agent Evaluation: AutoAgent secures a stable top-two GAIA ranking and exceeds 70% accuracy on Level 1 tasks, outperforming all open-source baselines.The authors attribute this performance to System Utilities and stable interactions between basic agents and their environments.
- Generalist Agent Evaluation: AutoAgent outperforms Magentic-1 and FRIDAY by emphasizing stable sub-agent–environment interactions and precise tool definitions.The authors report that these design choices allow the Orchestrator Agent to achieve better results with simple prompts and handoff tools.
- Generalist Agent Evaluation: GAIA evaluation is limited by strict string matching and dynamic anti-automation mechanisms that can penalize semantically equivalent answers or obstruct web searches.The authors identify these issues as motivation for more semantically aware evaluation.
- RAG Evaluation: MultiHop-RAG evaluates multi-source retrieval using Accuracy for consistency with expected answers and Error for confident but incorrect responses.The reported RAG setup uses gpt-4o-mini, text-embedding-3-small, 256-token chunks, and top-6 retrieval.
- RAG Evaluation: AutoAgent significantly outperforms LangChain on RAG because it orchestrates workflows dynamically instead of relying on predefined workflows and tools.The authors associate this flexibility with more efficient and accurate file-search results.
- Open-Ended Agent Development: Natural-language case studies show AutoAgent creating a DaVinci image agent, composing tools through XML specifications, and coordinating financial-analysis agents.The examples include tool creation, agent composition, local-document analysis, external-data research, and report generation.
5 Conclusion
The conclusion presents AutoAgent as a framework for making LLM-powered agent technology accessible to non-programmers. It connects natural-language requirements with the implementation of agents, tools, and workflows through modular system components.
- Conclusion: AutoAgent targets non-programming users by connecting high-level natural-language requirements with practical multi-agent systems and workflows.The framework is described as supporting agent, tool, and workflow creation, customization, and deployment without substantial technical expertise.
- Conclusion: Its modular architecture and Agentic System Utilities support the framework’s broader goal of democratizing LLM-powered agent technology.The passage identifies these components as part of AutoAgent’s contribution to accessible agent development.
6 Appendix
The appendix provides a detailed technical description of the Agentic System Utilities implementation within AutoAgent.
- Appendix: The supplementary materials contain a detailed technical description of AutoAgent’s Agentic System Utilities implementation.The appendix serves as the location for this implementation description.
6.1 System-level Tools
AutoAgent equips its system-level agents with predefined tool categories spanning coding, browsing, file management, and the creation and execution of tools, agents, and workflows.
- System-level Tools: Seven predefined tool categories support coding, web browsing, file management, tool creation, agent creation, workflow creation, and document question answering.The detailed tool names and descriptions are presented in Table 4.
- System-level Tools: The system-level tool interface includes listing, creating, deleting, and running plugin tools.These operations are represented by list_tools, create_tool, delete_tool, and run_tool.
6.2 Web Agent
The Web Agent uses browser-interaction tools and a system prompt that selects the next action from the page state until the user’s task is complete.
- The Web Agent provides tools for clicking, scrolling, navigating history, searching the web, entering text, waiting, visiting URLs, and extracting page markdown.
- Its system prompt requires reviewing the current page and available information before choosing the next executable action.
- The agent can convert media-containing or text-heavy pages into markdown for more detailed analysis.
6.3 Local File Agent
The Local File Agent manages files under /workplace through markdown-oriented file tools, page navigation, and visual question answering for images.
- It operates as a file surfer that handles local files through a tailored toolkit and system prompt.
- The Local File Agent uses tools to open local files, navigate converted markdown, search within pages, answer visual questions, and return control to the orchestrator.
- The agent can access only files in /workplace and must use absolute paths rooted there.
- The open_local_file tool reads supported documents and media as markdown text and accepts common formats including HTML, spreadsheets, presentations, audio, PDF, DOCX, and text files.
- Images are not handled by open_local_file and require the visual_question_answering tool instead.
- Converted markdown pages can be navigated with page-up, page-down, and find tools, while unresolved file-opening problems can be escalated to the orchestrator or Coding Agent.
6.4 Coding Agent
The Coding Agent is designed to solve complex tasks by creating, editing, inspecting, and executing code within the /workplace directory.
- The Coding Agent provides tools for generating project structures, executing commands, reading and writing files, listing files, creating directories, running Python, and navigating terminal output.
- Its system prompt directs it to create project structures and write clean, efficient, documented code using file and directory tools.
- The agent runs Python scripts and can page through long terminal output to locate meaningful content.
- It can perform complex computation, make API requests, and handle other tasks achievable through code.
- After completing the requested work, it transfers the conversation back to the System Orchestrate Agent.
- The Coding Agent is explicitly intended to complete tasks through coding rather than conversation alone.
6.5 Orchestrator Agent
The Orchestrator Agent routes user requests among specialized Local File, Web, and Coding Agents based on the current task state, continuing transfers until completion.
- The Orchestrator Agent has transfer tools for the Local File Agent, Web Agent, and Coding Agent.
- Its system prompt requires selecting the agent best suited to the user’s request under the current context.
- The orchestrator continues transferring the conversation until the user’s request is completed.
- The Local File Agent handles local-file opening and browsing, the Web Agent handles website and content browsing, and the Coding Agent writes code for especially complex tasks.
6.6 Detailed Implementation of “Self-Play Agent Customization” in AutoAgent
AutoAgent’s self-play customization converts natural-language requirements into structured single- or multi-agent forms, iteratively refining profiles, tools, and agents. The process supports workflow creation while acknowledging that documented patterns may omit mandatory elements.
- Customization pipeline: The customization pipeline accepts requirements, existing tools and agents, an optional task, and a maximum iteration count before generating agent and tool configurations.It profiles the request, parses a creation form, and repeats parsing until success or the iteration limit.
- Tool creation: When new tools are required, the Tool Editor Agent automatically creates and tests them, repeating the process until success or the iteration limit.The algorithm then continues with subsequent agent-editing stages after tool creation.
- Agent profiling: An Agent Profiling Agent analyzes user requests and generates structured creation forms for either single- or multi-agent systems.The form uses an <agents> root containing system inputs, outputs, agent definitions, tools, and optional shared variables.
- Form structure: Single-agent forms align system input and output with the agent’s input and output, while multi-agent forms cover all routed inputs and unify agent responses.Each agent specifies an identifier, purpose, instructions, available tools, and one-key input and output descriptions.
- Illustrative configurations: Example configurations show natural-language requests becoming agents that combine retrieval, query modification, answering, email, sales, recommendation, and service tools.The examples include a document-based OpenAI-product helper and a purchase-or-refund system for a named user.
- Implementation boundary: The documented workflow patterns are incomplete because context-length limits omit some mandatory elements, so complete workflows require reconstructing the pattern logic.This is an explicit implementation boundary for the presented patterns.
6.7 Supplementary Experimental Findings
Supplementary cases show AutoAgent generating agents, workflows, and tools from natural-language requirements, then applying them to image creation, investment analysis, and multi-model reasoning tasks. These demonstrations include iterative image refinement, a Microsoft–Apple allocation recommendation, and majority-voting solutions that resolve disagreements among models.
- Case of ‘DaVinci Agent’: AutoAgent generated a DaVinci Agent that creates images, evaluates them with predefined criteria, and iteratively refines them.The generated workflow integrated newly created image-generation and refinement tools with an existing visual-question-answering tool.
- Case of ‘Majority Voting’ workflow: AutoAgent’s majority-voting workflow compared multiple model solutions and selected answers supported by two solvers after analyzing disagreements and verifying the reasoning.For the modulo problem, the workflow selected 13 after identifying support from GPT-4 and Claude and verifying −11213141 ≡13 (mod 18).
- Case of ‘Majority Voting’ workflow: For geometry reasoning, the workflow favored Claude’s solution because it correctly treated the relevant angles as supplementary and applied the corresponding cosine sign relationship.The workflow rejected GPT-4’s equal-angle assumption and DeepSeek’s complementary-angle interpretation.
- Case of ‘Majority Voting’ workflow: In the tangent-circle example, the workflow selected Claude’s r = 14/3 solution because it matched the stated tangency relationship r + r + r = R.The analysis contrasted this result with GPT-4’s r = 7 and DeepSeek’s 14(√2 −1), judging the latter alternatives inconsistent with the described configuration.
- Case of ‘Financial Agent’: The financial-agent trajectory demonstrates coordinated document management, market research, and orchestration agents producing detailed and actionable investment analysis.The tested system processed AAPL and MSFT 10-K reports, analyzed financial trends, and generated allocation, rationale, monitoring, risk-management, and implementation recommendations.