PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoFlow: Automated Workflow Generation for Large Language Model Agents

Zelong Li, Shuyuan Xu, Kai Mei, Wenyue Hua, Balaji Rama, Om Raheja, Hao Wang, He Zhu, Yongfeng Zhang

arXiv:2407.12821v1cs.CLcs.AIcs.LG

TL;DR

Manually designing workflows for reliable LLM-agent task solving takes substantial effort and domain knowledge. AutoFlow generates and optimizes natural-language workflows through fine-tuning-based and in-context-based methods; on OpenAGI, its average score exceeds baselines, with gains over the best baseline CoRE for both Mixtral and GPT-4 interpreters.

  • Problem

    Manually designing agent workflows requires substantial effort and domain knowledge, making large-scale agent development and deployment difficult.

    Source: §1 · pp. 1–2

  • Method

    AutoFlow generates natural-language agent workflows and iteratively optimizes them using fine-tuning-based or in-context-based methods with reinforcement-learning rewards.

    Source: §1 · p. 2

  • Results

    On OpenAGI, AutoFlow's average score over tasks was significantly better than baselines, improving over best baseline CoRE by over 40% with Mixtral and over 5% with GPT-4 as interpreter.

    Source: §1 · p. 2, §5.5 · p. 7

  • Takeaways & Limitations

    Compared with manually designed workflows, AutoFlow-generated workflows can achieve better performance while significantly reducing human labor.

    Source: §6 · p. 8

    The reinforcement-learning workflow-generator training may be less efficient than gradient-based or few-shot methods, and other learning paradigms may be worth evaluating.

    Source: §6 · p. 8

Abstract

from arXiv · show

Recent advancements in Large Language Models (LLMs) have shown significant progress in understanding complex natural language. One important application of LLM is LLM-based AI Agent, which leverages the ability of LLM as well as external tools for complex-task solving. To make sure LLM Agents follow an effective and reliable procedure to solve the given task, manually designed workflows are usually used to guide the working mechanism of agents. However, manually designing the workflows requires considerable efforts and domain knowledge, making it difficult to develop and deploy agents on massive scales. To address these issues, we propose AutoFlow, a framework designed to automatically generate workflows for agents to solve complex tasks. AutoFlow takes natural language program as the format of agent workflow and employs a workflow optimization procedure to iteratively optimize the workflow quality. Besides, this work offers two workflow generation methods: fine-tuning-based and in-context-based methods, making the AutoFlow framework applicable to both open-source and closed-source LLMs. Experimental results show that our framework can produce robust and reliable agent workflows. We believe that the automatic generation and interpretation of workflows in natural language represent a promising paradigm for solving complex tasks, particularly with the rapid development of LLMs. The source code of this work is available at https://github.com/agiresearch/AutoFlow.

1 Introduction

LLM agents use language models and external tools for complex tasks, but reliable task-solving workflows are often manually designed and costly to scale. AutoFlow generates and iteratively optimizes natural-language workflows, with experiments reporting stronger performance than manually designed workflows while retaining readability.

1 Introduction

  • Manually crafted workflows require substantial effort and domain knowledge, creating a barrier to large-scale agent development and deployment.Workflows guide agents' task-solving procedures and are critical to effective and reliable operation.Source: §1 · pp. 1–2
  • AutoFlow automatically generates natural-language workflows for agents and iteratively refines their quality to address manual-design challenges.Natural-language programs are used as the workflow format.Source: §1 · p. 2
  • AutoFlow workflows outperform manually designed ones while remaining readable, and experiments report higher valid-plan rates and overall performance.The framework is also described as producing high-quality workflows for reliable complex-task performance.Source: §1 · p. 2
  • AutoFlow offers fine-tuning-based and in-context-based workflow generation methods, incorporating reinforcement learning for open-source and closed-source LLMs.In the described loop, an interpreter evaluates generated workflows on a dataset, and its performance reward updates the generator.Source: §1 · p. 2

2 Related Work

Related work covers LLM agents and workflow-based approaches to reasoning, planning, and coding, alongside AutoML methods for reducing manual machine-learning design work. AutoFlow follows the AutoML controller–child-model training paradigm, assigning those roles to a workflow generator and an interpreted workflow.

2.1 LLM Agents and Workflow

  • LLM agents are autonomous entities that make decisions and execute actions; related work covers chain-of-thought reasoning, sampled-path voting, and structured reasoning paths.The cited approaches include tree and graph structures, while self-consistency selects among sampled paths by voting.Source: §2.1 · p. 3
  • Planning research addresses non-executable or invalid plans through tool-chain post-processing, finite-state-machine control, and external tools such as search engines.The paper states that it builds on these ideas to improve generated workflow executability.Source: §2.1 · p. 3
  • Coding work includes workflow-based methods and natural-language programming; CoRE unifies natural language, pseudo-code, and workflow programming with an LLM interpreter.AutoFlow follows the workflow concept in natural-language programming to reduce human labor.Source: §2.1 · p. 3

2.2 Automated Machine Learning

  • AutoML approaches automate model selection, feature engineering, or neural architecture search, with NAS methods reported to match or outperform human-designed architectures across tasks.The cited examples include Auto-sklearn and Auto-WEKA for model selection, and Data Science Machine, ExploreKit, and VEST for feature engineering.Source: §2.2 · p. 3
  • AutoFlow adopts the AutoML controller–child-model training pattern, using the workflow generator as controller and the generated workflow plus interpreter as child model.The controller samples model selections, while the child model comprises the model parameters used for the task.Source: §2.2 · p. 3

3 Preliminary and Background

The section presents CoRE natural-language programs as executable workflows and motivates automatically generating them to reduce the human labor and domain expertise workflow construction requires.

3.1.1 CoRE Language Syntax

  • CoRE represents each workflow step with a name, type, natural-language instruction, and connection, using ::: to delimit the components.Source: §3.1.1 · p. 4
  • Its three step types are process, decision, and terminal, supporting progression, conditional branching, and program completion.Source: §3.1.1 · p. 4

3.1.2 LLM as Interpreter for Workflow Execution

  • An LLM interpreter executes workflow steps by retrieving relevant memory, prompting on the instruction, optionally using external tools, and choosing the next step.Source: §3.1.2 · p. 4

3.2 Motivation

  • CoRE unifies natural-language, pseudocode, and workflow programming, but constructing its workflows still requires substantial human labor and domain expertise.Source: §3.2 · pp. 4–5

4 The AutoFlow Framework

AutoFlow iteratively improves workflows using execution feedback, with a fine-tuning approach for open-source generators and an in-context approach for closed-source generators.

4.1 Fine-tuning Method for Workflow Generation with Open-source LLMs

  • Open-source workflow generators receive a task description and example workflow; examples help them learn CoRE grammar and task descriptions convey the application scenario.Source: §4.1 · p. 5
  • For open-source models that do not reliably produce valid grammar, GPT-4 parses and revises generated workflows before evaluation.Source: §4.1 · p. 5
  • The open-source method executes each workflow on validation data, uses average validation metrics as an RL reward to update the generator's LoRA adapter, and stops when consecutive rewards differ by less than a threshold.Source: §4.1 · p. 6

4.2 In-context Learning Method for Workflow Generation with Closed-source LLMs

  • For closed-source generators, AutoFlow avoids parameter fine-tuning: GPT-4 uses the example to generate a workflow, which is evaluated and followed by a new prompt containing its reward.Source: §4.2 · p. 6

5 Experiments

AutoFlow is evaluated on OpenAGI against several planning schemas using Mixtral and GPT-4 as interpreters; the experiments report better average and task-level scores than baselines.

5.1 Backbone Large Language Model (LLM)

  • The experiments use GPT-4-1106-preview as the closed-source model and Mixtral-8x7B, a 46.7-billion-parameter sparse mixture-of-experts model, as the open-source model.Source: §5.1 · p. 6

5.2 Planning Schema of LLMs

  • Planning comparisons include zero-shot, chain-of-thought, few-shot, manually designed CoRE, and AutoFlow workflows.Source: §5.2 · p. 6

5.3 Benchmark Datasets

  • OpenAGI tasks are grouped into three task types, evaluated with CLIP Score for text-to-image, BERT Score for text outputs, and ViT Score for image outputs.Source: §5.3 · pp. 6–7

5.5 Experimental Analysis

5.4 Implementation Details

  • The generator runs for 30 iterations; Mixtral fine-tuning uses REINFORCE with training-set average score as reward, Adam at 0.001, and LoRA rank 8.Source: §5.4 · p. 7

6 Conclusions and Future Work

The paper concludes that automatically generated workflows perform better than manually designed ones while reducing human labor, and identifies possible improvements to its learning and collaboration paradigms.

6. Conclusions and Future Work

  • AutoFlow generates effective agent workflows through fine-tuning for open-source generators and in-context learning for closed-source generators.Source: §6 · p. 8
  • Compared with manually designed workflows, generated workflows achieve better performance and reduce human labor, increasing automation.Source: §6 · p. 8
  • The authors note that RL may be less efficient than gradient-based or few-shot alternatives, and suggest evaluating other learning and collaboration paradigms.Source: §6 · p. 8

Related papers

Loading 2407.12821v1…