Source-linked AI summary
AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
Mengkang Hu, Pu Zhao, Can Xu, Qingfeng Sun, Jianguang Lou, Qingwei Lin, Ping Luo, Saravan Rajmohan
TL;DR
AgentGen targets the limited diversity and scale of agent-training data caused by manually designed environments and planning tasks. It automatically generates environments and conditioned task curricula using inspiration text and bidirectional evolution, and its tuned models outperform strong baselines, including GPT-3.5 and GPT-4 comparisons reported by the paper.
Problem
Existing agent-training studies mainly use manually designed environments and tasks, making diverse and extensive trajectory generation labor-intensive.
Method
AgentGen uses LLMs to generate environments from domain-specific inspiration text and applies BI-EVOL to evolve conditioned planning tasks toward easier and harder directions.
Results
AgentGen-tuned Llama-3.1-8B surpassed GPT-3.5 overall, while the tuned Llama-3.1-70B model achieved state-of-the-art planning results and exceeded GPT-4 in the reported comparison.
Takeaways & Limitations
Automatically generated environments and task curricula improved LLM planning on both in-domain and out-of-domain tasks.
Takeaways & Limitations
The evaluation focuses on planning capability, and the paper states that broader generalization to other LLM-agent scenarios remains potential rather than established.
Abstract
from arXiv · showhide
Large Language Model-based agents have garnered significant attention and are becoming increasingly popular. Furthermore, planning ability is a crucial component of an LLM-based agent, which generally entails achieving a desired goal from an initial state. This paper investigates enhancing the planning abilities of LLMs through instruction tuning, referred to as agent training. Recent studies have demonstrated that utilizing expert-level trajectory for instruction-tuning LLMs effectively enhances their planning capabilities. However, existing work primarily focuses on synthesizing trajectories from manually designed planning tasks and environments. The labor-intensive nature of creating these environments and tasks impedes the generation of sufficiently varied and extensive trajectories. To address this limitation, this paper explores the automated synthesis of diverse environments and a gradual range of planning tasks, from easy to difficult. We introduce a framework, AgentGen, that leverages LLMs first to generate environments and subsequently generate planning tasks conditioned on these environments. Specifically, to improve environmental diversity, we propose using an inspiration corpus composed of various domain-specific text segments as the context for synthesizing environments. Moreover, to increase the difficulty diversity of generated planning tasks, we propose a bidirectional evolution method, Bi-Evol, that evolves planning tasks from easier and harder directions to synthesize a task set with a smoother difficulty curve. The evaluation results derived from AgentBoard show that AgentGen greatly improves LLMs' planning ability, e.g., the AgentGen instruction-tuned Llama-3.1-8B surpasses GPT-3.5 in overall performance. Moreover, the AgentGen-tuned Llama-3.1-70B model achieves state-of-the-art results in planning tasks. Project page: https://agent-gen.github.io/.
1 Introduction
AgentGen addresses the labor-intensive preparation of environments and planning tasks for agent training by automatically generating diverse environments and gradually difficult tasks. Its instruction-tuned models show strong in-domain and out-of-domain planning improvements.
- Motivation: Existing agent-training methods rely mainly on manually designed environments and tasks, limiting the diversity and scale of synthesized trajectory data.Creating rich environments requires domain scenarios, implementation effort, and expert programming, while task sets also need gradual difficulty progression.
- AgentGen: AgentGen uses LLMs to generate environments from domain-specific inspiration text and then produces planning tasks conditioned on those environments.Environment specifications define actions, restrictions, and other details before code is generated; an environment library provides in-context examples.
- AgentGen: BI-EVOL evolves seed planning tasks toward both simpler and more difficult variants, producing a smoother difficulty curve for planning-skill acquisition.This bidirectional process is intended to increase difficulty diversity across generated task sets.
- Results: AGENTGEN instruction-tuning improved Llama-3.1-8B on in-domain tasks by more than tenfold, from 3.0 to 33.3, surpassing GPT-3.5 overall; its tuned 70B model exceeded GPT-4.Evaluations were zero-shot, and the 70B result established a new state-of-the-art in planning tasks.
- Data and evaluation: The framework produced 592 environments with 20 tasks each and 7,246 high-quality trajectories for instruction tuning.Evaluation used zero-shot AgentBoard tasks, distinguishing PDDL-based in-domain tasks from tasks implemented in other programming languages.
- Results: AgentGen improved performance beyond in-domain tasks, with reported average success-rate gains of 10.0% for the 8B model and 3.7% for the 70B model on out-of-domain tasks.The paper presents these results as evidence of both effectiveness and generalization.
2 Preliminary
The paper formulates deterministic planning as goal-directed interaction between an environment and a task, then describes common implementations and LLM-based agents. These agents use a language model to map perceived states to actions while pursuing specified goals.
- 2.1 Planning Problems: A deterministic planning problem is represented as P = (T, E), where E is the environment and T is the task.The environment defines action and state spaces plus transitions, while the task specifies initial states and goal conditions.
- 2.2 Planning Problem Implementation: PDDL separates a planning problem into a domain file representing the environment and a problem file representing the task.The domain defines predicates, actions, preconditions, effects, and transitions; the problem file defines initial states and goals.
- 2.2 Planning Problem Implementation: Python implementations commonly encode transitions in a step or update method, goals as rewards, and initial states in reset.This provides an alternative to domain-specific languages such as PDDL.
- 2.3 Large Language Model based Agent: An LLM-based agent forms a policy π : S → A by perceiving state S and generating actions A that pursue the task’s goal.The transition function remains fixed by the environment, while the language model selects actions based on its interpretation of the task and state.
3 Methodology
AgentGen formalizes planning-task synthesis as a two-stage process that first generates environments and then generates tasks conditioned on them. Its environment pipeline uses inspiration and validation, while BI-EVOL expands tasks toward both simpler and harder goals.
- 3 Methodology: AgentGen maps an input instruction to an environment and planning task, generating the environment first and the task conditionally afterward.The stages are represented as E_i = g_E(i) and T_i = g_T(i, E_i).
- 3.1 Environment Generation: The environment pipeline generates a specification, implements corresponding code, and stores verified environments in an expanding library.The library accumulates verified environments across iterations, supporting continued refinement and diversity.
- 3.1 Environment Generation: An inspiration corpus supplies diverse text segments that condition environment specifications, addressing the limited diversity of randomly generated environments.The specification includes the environment, actions with preconditions and effects, and restrictions; this paper uses LIMA as the inspiration corpus.
- 3.2 Task Generation: Task generation first produces unconditioned tasks for each environment and then applies BI-EVOL to create a broader difficulty range.The resulting task set incorporates both the initial tasks and their evolved versions.
- 3.2 Task Generation: BI-EVOL evolves planning tasks in both directions: easy-evol simplifies goal conditions, whereas hard-evol makes goals more complex and lengthens required action sequences.The bidirectional design supports learning from easier goals while also extending planning capability through harder tasks.
4 Experiments
The experiments evaluate AgentGen-trained models on PDDL in-domain tasks and partially observable out-of-domain tasks, using success and progress rates. AgentGen improves performance across model scales and task settings, including comparisons with GPT baselines and gains after training.
- 4.1 Experimental Setup: The evaluation covers four PDDL in-domain environments and three out-of-domain partial-observable tasks, using success rate and progress rate as planning metrics.The out-of-domain set includes Alfworld, BabyAI, and Jericho; the in-domain set includes Blocksworld, Gripper, Tyreworld, and Barman.
- 4.1 Experimental Setup: The training data contains 592 generated environments and 7,246 planner-produced trajectories, with evaluation performed on AgentBoard.Tasks are generated through unconditioned sampling followed by BI-EVOL refinement, and FastDownward supplies the trajectories.
- 4.2 Evaluation on In-Domain Tasks: AgentGen-tuned Llama-3.1-8B reaches 33.3 overall progress rate versus GPT-3.5’s 25.0, while the 70B model reaches 81.5 versus GPT-4’s 81.2.The tuned models also outperform similarly scaled models across four in-domain tasks.
- 4.3 Robustness: Training with AgentGen improves all three tested 7–8B foundation models, with Llama-3-8B gaining 10.0 in success rate and CodeLlama-7B gaining 9.9 in progress rate.These results test the dataset’s effectiveness across different foundation models.
- 4.4 Evaluation on Out-of-Domain Tasks: On out-of-domain tasks, AgentGen raises average progress rate by 13.1% for Llama-3.1-8B and 5.0% for Llama-3.1-70B.The 8B model also outperforms GPT-3.5 and similarly scaled general and agent-tuned models on both tasks.
5 Related Work
Prior work enhances LLM-based agents through planning, trajectory training, and environment or task generation, but AgentGen targets broader environment diversity and task difficulty coverage. The related literature motivates this shift from manually bounded training settings toward automated generation.
- Planning with Large Language Models: LLM planning research spans open-loop and closed-loop execution, with this paper focusing on closed-loop planning because it supports error correction, human interaction, and environmental grounding.The paper studies training rather than introducing a new planning framework or relying primarily on prompt engineering.
- Planning with Large Language Models: Planning is defined here as determining executable actions from an initial state to a goal under constraints, distinguishing it from ungrounded plans used only as problem-solving guidance.The paper considers planning one capability among broader agent functions such as memorization and tool use.
- Agent Training: Existing agent-training approaches use trajectory data from a small number of environments, including GPT-4-generated trajectories, reasoning and tool-use traces, and verified plans.These methods include AgentTuning, FireAct, Agent LUMOS, LLM-Modulo, and generate-test trajectory synthesis.
- Agent Training: AgentGen differs from prior agent-training methods by generating environments as well as trajectories, producing over 500 environments instead of the fewer than 10 typically used previously.This extends trajectory synthesis beyond the task environments manually or narrowly selected in earlier work.
- Environment and Task Generation with Large Language Models: Prior environment and task-generation methods often rely on predefined specifications, test-set seeds, or scene-level layouts, limiting mechanism-level diversity, task-difficulty diversity, or risking data leakage.AgentGen instead uses diverse text segments to generate environment code without predefined definitions.
6 Conclusion
AgentGen automates the generation of diverse environments and planning tasks for agent training, using inspiration-guided environment synthesis and bidirectional task evolution. Its tuned Llama models outperform key baselines, including GPT-3.5 and prior planning-task results.
- AgentGen produced 592 environments and 7,246 trajectories, with tuned Llama-3.1-8B surpassing GPT-3.5 and Llama-3.1-70B achieving state-of-the-art planning performance.The dataset and training outcomes support the framework’s effectiveness across model scales.
- The framework uses domain-specific inspiration text to synthesize diverse environments and BI-EVOL to evolve tasks from easier and harder directions.These components target environmental diversity and a smoother range of task difficulty.
A More Implementation Details
The implementation details cover the evaluated instruct models and a GPT-4-assisted procedure for converting structured PDDL actions and predicates into natural-language mappings.
- A.1 Models: The study evaluates instruct versions of the models, with each model’s detailed configuration listed in Table 4.The supplied materials identify Table 4 as the model-configuration reference.
- A.2 Natural Language Mapping: GPT-4 generates a Python-dictionary mapping from PDDL predicates and actions to natural-language expressions, with heuristic serialization used when generation fails.The mapping preserves parameter order and requires the number of placeholders to match each predicate or action’s parameters.
- A.2 Natural Language Mapping: The Tower of Hanoi example maps predicates such as clear, on, and smaller, plus move actions, while retaining their PDDL preconditions, effects, and domain restrictions.The example describes legal moves using disc clearance, peg clearance, and size constraints.
B.1 Environment Specification
The environmental specifications are moderately long and concentrated mainly in the 300–699-token range across the 592 generated environments.
- 473.55 tokens is the average environmental-specification length, while the median is 467.00 tokens across 592 specifications.These statistics characterize the typical specification length in the generated environment collection.
- 207 to 934 tokens is the observed specification-length range, with most specifications concentrated between 300 and 699 tokens.Figure 4 depicts this distributional concentration.
- The specification-length distribution indicates substantial variation around a central range rather than a single fixed prompt size.This summarizes the reported minimum, maximum, average, median, and concentration range.
B.2 Environment Implementation
Environment complexity is generally associated with the number of actions and predicates, so a useful training library should span multiple difficulty levels.
- B.2 Environment Implementation: More actions and predicates generally indicate a more complex environment.
- B.2 Environment Implementation: A training environment library is preferable when it contains a wider range of difficulty levels.
- B.2 Environment Implementation: The generated datasets show significant diversity in the numbers of actions and predicates.
B.3 Diversity Analysis
AgentGen evaluates environment diversity through token, action-predicate, and semantic-similarity analyses, while the examples section points to detailed cases in Figures 2 and 3.
- B.3 Diversity Analysis: Figure 4 shows the token distribution of generated environment specifications, while Figure 5 shows action and predicate frequencies.
- B.3 Diversity Analysis: Average cosine similarity among 100 sampled environment specifications is 0.176, indicating substantial semantic diversity.
- C Examples: The examples section provides specific details for the cases depicted in Figures 2 and 3.
C.1 Environment Specification
The example environment specifies a healthy-recipe domain with explicit actions, preconditions, effects, restrictions, and a corresponding STRIPS-style implementation.
- C.1 Environment Specification: The healthy-recipe environment includes an office, computer, kitchen, ingredients, and a nutritionist creating a recipe book.
- C.1 Environment Specification: Its action sequence researches an ingredient, develops a recipe, tests it with ingredients, and finalizes it after feedback and computer charging.
- C.1 Environment Specification: Action restrictions require appropriate locations, prior research, necessary ingredients, recipe testing, feedback, and a charged computer.
- C.2 Environment Implementation: The implementation encodes the domain with STRIPS requirements, predicates, parameters, preconditions, and effects for the recipe actions.
- C.2 Environment Implementation: The implementation models researched peanut butter and computer-charged status as predicate conditions, while an action’s effects add a recipe draft and remove the computer-charged status.
C.3 Trajectory Data
The trajectory example presents a nutritionist task with explicit observations, goals, and action responses grounded in the generated recipe environment.
- C.3 Trajectory Data: The trajectory goal requires a charged computer and a tested almond_butter_bars recipe.
- C.3 Trajectory Data: The initial observation includes ingredients, a recipe draft, prior peanut-butter research, and the nutritionist in the kitchen.
- C.3 Trajectory Data: The assistant first tests the recipe, after which the observation records the recipe as tested before a subsequent develop action.