AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents

Problem

• Only 0.03% of the global population possesses the programming expertise needed to build and customize current LLM agents, limiting personalized AI assistance.Existing frameworks require codebase, API-integration, and prompt-engineering skills, creating a barrier for non-technical users.Source: §1 · p. 2

Contribution

• AutoAgent enables natural-language construction of collaborative multi-agent systems, adaptive workflows, and unified tool and resource orchestration without programming.It operates as an autonomous Agent Operating System that converts high-level task descriptions into executable agent systems and workflows.Source: §1 · p. 2

Framework

• AutoAgent integrates Agentic System Utilities, an LLM-powered Actionable Engine, a Self-Managing File System, and Self-Play Agent Customization.These components support specialized web, code, and file agents; flexible tool-use across LLM providers; queryable vector databases; and iterative workflow self-improvement.Source: §1 · pp. 2–3

Results

• AutoAgent achieved second place on GAIA and significantly outperformed state-of-the-art RAG approaches, while case studies demonstrated robust self-development across real-world scenarios.The evaluation combined standardized benchmarks with practical case studies of automated agent development.Source: §1 · p. 3

Related Work

• Frameworks including LangChain, AutoGPT, CAMEL [4], MetaGPT [5], and OpenAgent demonstrate diverse agent capabilities but remain inaccessible to many non-technical users because they require coding and domain expertise.The proposed direction is automatic generation, customization, and orchestration through natural-language interactions.Source: §2 · p. 3

LLM-Empowered Agent

• LLM agents can be modeled as an MDP over states, actions, observations, transitions, and environments, with Tool-Use and ReAct [8] as two action-generation paradigms.At each step, the agent observes its state, selects an action, interacts with the environment, and updates its context.Source: §2 · p. 3

Generalist Multi-Agent System

• Generalist multi-agent systems coordinate specialized agents through a central Orchestrator to combine capabilities and knowledge for complex, varied tasks.In the Orchestrator-Workers paradigm [9], the Orchestrator delegates subtasks and Workers return their results through transfer actions.Source: §2 · p. 3

Workflow Design in Generalist MAS

• Workflow-driven systems such as GPTSwarm and specialized mathematical-reasoning [12] and code-generation [13] systems support applications including GUI interaction [14] [15], software development [17], web browsing [19], and embodied tasks [20].These systems model or apply workflows to achieve strong capabilities across deterministic or specialized tasks.Source: §2 · p. 4
• Generalist MAS offer flexibility for open-ended tasks but incur higher cost, complexity, and compounding-error risks, whereas fixed workflows can be more stable for deterministic, expert-driven processes.Workflow patterns include Routing, Parallelization, and Evaluator-Optimizer.Source: §2 · p. 4

Fully-Automated Generalist MAS

• AutoAgent targets fully automated construction and execution of MAS and workflows by bridging high-level user requirements to practical implementations through natural-language interactions.Its language-driven architecture comprises Agentic System Utilities, an LLM-powered Actionable Engine, a Self-Managing File System, and a Self-Play Agent Customization module.Source: §2 · p. 4, Figure 2 · p. 4

3 The AutoAgent Framework

• AutoAgent combines agentic utilities, an LLM-powered actionable engine, file management, and self-play customization for natural-language agent development and task execution.Together, these components support autonomous solutions across diverse applications without coding requirements.Source: §3 · p. 4

3.1 Agentic System Utilities

• Specialized web, coding, and file agents coordinate through an orchestrator, providing an extensible foundation for diverse personal-assistant capabilities.The modular architecture integrates browsing, information retrieval, data analysis, and code execution.Source: §3.1 · pp. 4–5

3.1.1 Orchestrator Agent

• The Orchestrator decomposes user requests, delegates subtasks through handoff tools, and iteratively assigns follow-up work until completion without complex planning prompts.It serves as the primary user interface and receives completed subtask results from sub-agents.Source: §3.1.1 · p. 5

3.1.2 Web Agent

• The Web Agent performs searches, page navigation, content browsing, and file downloads through an extensible set of high-level tools.Its tools abstract web behaviors such as click, web_search, and visit_url; the implementation combines BrowserGym low-level actions.Source: §3.1.2 · p. 5

3.1.3 Coding Agent

• The Coding Agent handles code-driven tasks including analysis, calculations, machine learning, automation, and administration in a secure execution environment.It provides 11 tools and runs operations in a Docker sandbox, with support for third-party systems such as E2B.Source: §3.1.3 · p. 5

3.1.4 Local File Agent

• The Local File Agent converts and analyzes text, video, audio, spreadsheet, and other multimodal files through unified tools.An interactive Markdown Browser presents converted files in paginated form for analyzing long or complex documents.Source: §3.1.4 · pp. 5–6

3.2 LLM-powered Actionable Engine

• The LLM-powered actionable engine understands natural language, generates plans, and coordinates cross-agent tasks for human-agent collaboration and completion.LiteLLM standardizes requests across 100+ models, while action-observation pairs up to time t form state s_t for selecting the next action.Source: §3.2 · p. 6

3.2.1 Generating Actionable Reflections

• AutoAgent generates actionable reflections either through direct tool use or structured XML tool-use transformation, supporting both commercial and open-source LLMs.Direct tool use reduces parsing errors but depends on platform optimization; transformed tool use improves weaker tool-use models and enables greater flexibility.Source: §3.2.1 · p. 6

3.3 Self-Managing File System

• The file system converts uploaded text files, archives, and folders into normalized documents stored in user-defined vector-database collections for agent retrieval and generation.Agents manage memory through save_raw_docs_to_vector_db, query_db, and answer_query; tool definitions appear in Tab 4.Source: §3.3 · p. 6

3.4 Self-Play Agent Customization

• AutoAgent supports code-driven customization that uses constraints, error handling, and workflows to generate tools, agents, and multi-agent systems in two creation modes.The modes are agent creation without workflow and agent creation with workflow.Source: §3.4 · p. 6

3.4.1 Agent Creation without Workflow

• Without requiring specialized knowledge such as financial regulation or healthcare protocols, AutoAgent profiles requirements and available components before structuring agents and generating XML specifications.The process analyzes requirements, evaluates existing and needed tools, and records functionality, dependencies, and interactions in structured XML.Source: §3.4.1 · pp. 6–7
• Workflow-based generation lets users describe desired agents and functions in natural language, after which AutoAgent generates agents and coordinating workflows using the framework’s current components.This mode targets sophisticated agent systems for users with minimal domain expertise.Source: §3.4.1 · pp. 6–7
• The Tool Editor Agent integrates APIs including LangChain, RapidAPI, and Hugging Face, supporting 145 RapidAPI APIs across 8 categories and models across 9 Hugging Face categories.Future plans include integrating Composio.Source: §3.4.1 · p. 7
• Generated tools undergo syntax checking, test-case design, execution, and automatic debugging until successful, producing a customized rather than bloated tool set.For multi-agent requirements, AutoAgent can create an orchestrator that connects specialized agents using the Orchestrator-Workers MAS pattern.Source: §3.4.1 · p. 7

3.4.2 Agent Creation with Workflow

• For specified tasks and workflows, AutoAgent generates agents and coordination logic through event-driven collaboration, avoiding the rigid graph structures associated with graph-based methods [27] [28].The Workflow Form Agent analyzes requirements and components, generates XML, and the Workflow Editor Agent creates needed agents, constructs the workflow, and executes it after constraint validation or regeneration.Source: §3.4.2 · pp. 7–8

4.1 Evaluation for a Generalist Agent System

• On the GAIA validation set, AutoAgent significantly outperforms all open-source agent systems, approaches h2oGPTe Agent v1.6.8, ranks stably in the top two, and exceeds 70% accuracy on Level 1 tasks.Evaluation uses success rate as the primary metric, measuring the percentage of tasks successfully completed; Table 1 reports results published on the GAIA leaderboard.Source: §4.1 · p. 8, Table 1 · p. 8
• GAIA error analysis identifies strict string matching and dynamic anti-automation mechanisms as limitations requiring more semantically aware evaluation.String matching can reject semantically equivalent answers, while web-search defenses can obstruct task completion.Source: §4.1 · p. 9

4.2 Evaluation of AutoAgent on the Retrieval-Augmented Generation (RAG) Task

• On MultiHop-RAG, AutoAgent significantly outperforms LangChain because it dynamically orchestrates workflows instead of relying on predefined search workflows and tools.Accuracy measures consistency with expected answers, while Error counts confident but incorrect responses.Source: §4.2 · p. 9, Table 2 · p. 9

4.3 AutoAgent’s Performance on Open-Ended Tasks

• AutoAgent automatically creates a DaVinci image agent by generating and integrating tools for image creation, evaluation, storage, and iterative refinement from natural-language requirements.The Tool Editor Agent creates the image-generation and refinement tools, while the Agent Editor Agent composes and executes the tailored agent, even with a smaller model under limited local resources.Source: §4.3 · p. 10
• For a financial-assistance request, AutoAgent decomposes the requirements into Document Manager and Market Research agents and coordinates them through a Financial Analysis Orchestrator.It creates tools for financial statements and analysis, combines local-document research with external data, and self-debugged a SyntaxError before completing the report.Source: §4.3 · p. 10
• AutoAgent also generates workflows that parallelize multiple model-based agents and aggregate their answers through majority voting for mathematical problem solving.This demonstration targets the barrier of manually constructing test-time-compute workflows.Source: §4.3 · p. 10, Table 3 · p. 11

Conclusion

• AutoAgent enables users without substantial technical expertise to create, customize, and deploy multi-agent systems, tools, and workflows.It bridges high-level natural-language requirements with practical multi-agent implementation.Source: §5 · p. 11
• Its modular architecture combines Agentic System Utilities and an LLM-powered Actionable Engine to automate agent development and task execution.Source: §5 · p. 11
• The Self-Organizing File System and Self-Play Agent Customization enable dynamic agent evolution and task-specific optimization.Source: §5 · p. 11
• Extensive evaluations demonstrate AutoAgent’s superior performance and its potential to broaden access to LLM capabilities.Source: §5 · p. 11

• The appendix details the implementation of AutoAgent’s Agentic System Utilities.Source: §6 · p. 15

6.1 System-level Tools

• Seven predefined system-level tool categories support coding, web browsing, file management, and natural-language question answering over documents.They also support creating new tools, agents, and workflows.Source: §6.1 · p. 15
• Table 4 lists the detailed information for the system-level tools.The table presents their names and comprehensive descriptions.Source: Table 4 · p. 15, §6.1 · p. 15
• The plugin-tool interface supports listing, creating, deleting, and running tools with supplied code.These operations are exposed through list_tools, create_tool, delete_tool, and run_tool.Source: §6.1 · p. 17

6.2 Web Agent

• The Web Agent provides tools for clicking, navigation, history control, web search, text input, URL visits, page-to-markdown conversion, sleeping, and orchestration transfer.Its tool set also includes page_down, page_up, history_forward, and history_back.Source: §6.2 · p. 18
• The system prompt instructs the agent to inspect the current page state and other information before selecting and formatting the next executable action.This action-selection process is explicitly tied to accomplishing the assigned goal.Source: §6.2 · p. 18
• For media pages or detailed text analysis, the Web Agent must use get_page_markdown to convert page information into markdown.The prompt specifically mentions YouTube, Wikipedia, and other pages containing media content.Source: §6.2 · p. 18
• Downloaded files are placed in /workplace/downloads and cannot be opened directly; the agent must transfer them to the System Orchestrate Agent for Local File Agent handling.The prompt also requires transfer_back_to_orchestrate_agent after completing the assigned task, rather than stopping early.Source: §6.2 · p. 18

Tools

• The Local File Agent uses tools for opening local files, navigating markdown pages, searching within pages, answering visual questions, and transferring control to the orchestrate agent.Its listed tools are open_local_file, page_up_markdown, page_down_markdown, find_on_page_ctrl_f, find_next, visual_question_answering, and transfer_back_to_orchestrate_agent.Source: §6.3 · p. 18

System Prompt

• It can access only files under /workplace and must use absolute paths rooted there when opening files.The system prompt specifies /workplace as the accessible folder and requires paths such as /workplace/....Source: §6.3 · p. 18

File Handling

• open_local_file reads files as markdown text and supports HTML, spreadsheets, presentations, audio, PDF, DOCX, and other text files.Supported extensions include .html, .htm, .xlsx, .pptx, .wav, .mp3, .flac, .pdf, and .docx.Source: §6.3 · p. 19
• Because open_local_file does not handle images, the agent must use visual_question_answering to inspect them.This limitation is explicitly stated in the system prompt.Source: §6.3 · p. 19

Navigation

• For multi-page converted markdown, the agent can move between pages and search using page_up, page_down, find_on_page_ctrl_f, and find_next.These navigation tools apply when the converted markdown exceeds one page.Source: §6.3 · p. 19

Agent Handoff

• After completing the assigned task, the agent transfers the conversation back to the System Orchestrate Agent; if it cannot open a file, the Coding Agent may handle it by coding.The prompt requires handoff only after task completion, while failed file access permits escalation to coding support.Source: §6.3 · p. 19

6.4 Coding Agent

• The Coding Agent provides file, directory, code-generation, command-execution, Python-running, terminal-navigation, and orchestrator-handoff tools.Its listed tools include gen_code_tree_structure, execute_command, read_file, create_file, write_file, list_files, create_directory, run_python, terminal paging, and transfer_back_to_orchestrate_agent.Source: §6.4 · p. 19
• The system prompt constrains development to the /workplace folder and requires absolute paths, file inspection before edits, nonempty files, testing decisions, dependency commands, and Python scripts instead of notebooks.Python scripts must use run_python, while other programming languages are written to files and run with execute_command.Source: §6.4 · p. 19
• The agent can perform complex computation and API requests through coding, using terminal paging for long output and transferring control only after completing the assigned task.The prompt directs it to reuse existing code where possible and return the conversation to the System Orchestrate Agent when finished.Source: §6.4 · p. 20

Orchestrator tools

• The orchestrator can transfer conversations to Local File, Web, and Coding Agents through three dedicated tools.These tools are transfer_to_local_file_agent, transfer_to_web_agent, and transfer_to_coding_agent.Source: §6.5 · p. 20

Routing policy

• Its system prompt instructs it to select the best-suited agent under the current context and keep transferring until the task is completed.The specialized agents respectively handle local-file browsing, website browsing, and code-based solutions, especially complex tasks.Source: §6.5 · p. 20

6.6.1 Agent Creation without Workflow

• The creation algorithm iteratively profiles requirements and parses agent forms, then creates and tests required tools before running newly created agents on the optional task.Each stage retries up to M−1 times and stops when the preceding result is successful.Source: §6.6.1 · pp. 20–21
• Agent Profiling Agent converts user requests into structured forms containing agents, system input/output, agent definitions, optional global variables, and precise agent input/output schemas.Single-agent systems require system and agent interfaces to match, while multi-agent systems define complete system inputs and unified outputs across specialized agents.Source: §6.6.1 · pp. 21–22
• The form-generation guidelines require focused responsibilities, minimal sufficient tool selections, actionable instructions, precise interfaces, and shared context through global variables.Examples cover a document-grounded single agent with email capability and a purchase/refund multi-agent system using shared user context.Source: §6.6.1 · pp. 22, 23, 25
• Tool Editor Agent manages plugin tools by checking existing tools first, consulting API or Hugging Face documentation when needed, implementing reusable tools, and testing them through run_tool.API keys must come from retrieved documentation and remain embedded in tool bodies; visual tasks must use the existing visual_question_answering plugin through run_tool.Source: §6.6.1 · pp. 25, 27
• 6.6.1 Agent Creation without Workflow: The Agent Editor Agent parses XML agent forms, manages agents, orchestrates multi-agent systems, executes tasks, and handles dependencies and errors.Single-agent requests use create_agent followed by run_agent, while multi-agent requests require creating all agents, creating an orchestrator, and running it.Source: §6.6.1 · pp. 27–28

6.6.2 Agent Creation with Workflow

• 6.6.2 Agent Creation with Workflow: Algorithm 2 profiles a workflow, parses its form, retries parsing up to M attempts, then creates and runs it with iterative workflow-editor retries after failures.The process takes requirements, existing tools, agents, workflows, an optional task, and a maximum attempt count M as inputs.Source: §6.6.2 · p. 28
• 6.6.2 Agent Creation with Workflow: Workflow forms define system inputs and outputs, agents, optional shared variables, event inputs and tasks, and RESULT, ABORT, or GOTO actions with conditions.The on_start event passes through system input, sequential flows use RESULT, and conditional flows use mutually exclusive outputs with appropriate actions.Source: §6.6.2 · p. 29
• 6.6.2 Agent Creation with Workflow: The implementation provides if-else branching, parallel execution that joins after all listened events complete, and evaluator-optimizer loops that regenerate content below a quality threshold.The evaluator-optimizer pattern routes approved content onward and sends content needing improvement back to generation.Source: §6.6.2 · pp. 31–33
• The workflow patterns support mutually exclusive branching, parallel events with aggregator synchronization, and evaluator-optimizer iteration using GOTO actions, explicit criteria, retry paths, and optional iteration limits.If-Else workflows allow only one RESULT output, parallel branches may use different agent models, and evaluator-optimizer workflows distinguish success from retry outcomes.Source: §6.6.2 · p. 34
• A worked example creates a Wikipedia-like article by chaining web search, outline generation, outline evaluation, revision on failure, and article writing.The example defines Web Surfer, Outline, Evaluator, and Article Writer agents, with the evaluator sending unsuccessful outlines back to the outline stage.Source: §6.6.2 · pp. 34, 36
• The XML workflow excerpts show on_start and on_outline event blocks assigning claude-3-5-sonnet-20241022 to Web Surfer, Outline, and Article Writer agents.Source: §6.6.2 · pp. 35–37
• Workflow construction guidelines require clear event interfaces, conditional outcome handling, proper listen chaining, review steps for quality control, and RESULT or ABORT action types.The supplied patterns are explicitly incomplete because context-length limitations omit some mandatory elements, so complete workflows require following their underlying logic rather than copying them verbatim.Source: §6.6.2 · pp. 33, 37
• The workflow editor parses XML-formatted workflow forms, creates required agents, validates and creates workflows, and executes them through dedicated management functions.Its process includes checking agent existence, using empty tool lists when unspecified, validating workflow structure, monitoring execution, and handling errors.Source: §6.6.2 · pp. 37–38

6.7.1 Case of ‘DaVinci Agent’

• The DaVinci Agent generates images from natural-language descriptions, evaluates them with predefined criteria, and iteratively refines them.Its profile includes image-generation, visual-question-answering, and refinement capabilities.Source: §6.7.1 · pp. 38–39

6.7.2 Case of ‘Financial Agent’

• The 2025 recommendation allocates 60% to Microsoft and 40% to Apple within the technology allocation, citing Microsoft’s AI and cloud prospects and 37.61% profit margin versus 15.52%.Apple’s supporting rationale includes 43.01% return on capital, strong services growth potential, and valuable ecosystem lock-in.Source: §6.7.2 · pp. 41–43

6.7.3 Case of ‘Majority Voting’ workflow

• The Majority Voting workflow coordinates multiple math solvers, including GPT-4 and Claude, and determines the final solution through majority voting.The supplied passage describes the generated workflow profile and its solver and vote-aggregation agents, but does not provide the referenced comparison results.Source: §6.7.3 · pp. 43–44
• Majority voting selected 13 for the modulo problem because GPT-4 and Claude agreed, while DeepSeek produced 5.The workflow verified that −11213141 ≡ 13 (mod 18) with 0 ≤ 13 < 18.Source: §6.7.3 · p. 47
• For the trigonometric problem, the workflow favored Claude’s −24/25 answer because it correctly treated the angles as supplementary.GPT-4 assumed equal angles, whereas DeepSeek incorrectly applied complementary-angle reasoning.Source: §6.7.3 · p. 48
• For the tangent-semicircle geometry problem, the workflow selected Claude’s 14/3 result using the 3r = 14 relationship.The analysis rejected GPT-4’s r = 7 as causing overlap and considered DeepSeek’s 14(√2 − 1) unnecessarily complex for the configuration.Source: §6.7.3 · p. 48

6.7.4 Detailed Trajectory of Creating ‘Financial Agent’

• AutoAgent created Document Manager, Market Research, and Financial Analysis Orchestrator agents, tested on AAPL and MSFT documents to analyze financial metrics and trends and provide investment advice.The orchestrator was queried to compare AAPL and MSFT using their past five years of 10-K reports and available data; the agents processed 10-K reports from both companies.Source: §6.7.4 · pp. 53, 56, 58