Source-linked AI summary
Agent Workflow Memory
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, Graham Neubig
TL;DR
Language-model agents struggle with long-horizon tasks because fixed examples do not provide reusable workflows across changing contexts. AWM induces and stores reusable routines from annotated examples or successful online experiences, improving success on Mind2Web and WebArena while generalizing across task, website, and domain shifts.
Problem
Existing language-model agents lack robustness to changing task contexts and environments because they primarily use fixed examples rather than reusable workflows.
Method
AWM induces fine-grained, context-abstracted workflows from experiences and integrates them into agent memory offline from training examples or online from evaluated successful predictions.
Results
AWM improves relative success rate by 24.6% on Mind2Web and 51.1% on WebArena, while achieving 8.9–14.0 absolute-point gains across Mind2Web website and domain shifts.
Takeaways & Limitations
AWM provides a dynamic memory mechanism that supports workflow reuse and generalization across tasks, websites, and domains.
Takeaways & Limitations
AWM can guide agents toward actions that fit induced workflows but not the current environment state, making it difficult to identify when to diverge.
Abstract
from arXiv · showhide
Despite the potential of language model-based agents to solve real-world tasks such as web navigation, current methods still struggle with long-horizon tasks with complex action trajectories. In contrast, humans can flexibly solve complex tasks by learning reusable task workflows from past experiences and using them to guide future actions. To build agents that can similarly benefit from this process, we introduce Agent Workflow Memory (AWM), a method for inducing commonly reused routines, i.e., workflows, and selectively providing workflows to the agent to guide subsequent generations. AWM flexibly applies to both offline and online scenarios, where agents induce workflows from training examples beforehand or from test queries on the fly. We experiment on two major web navigation benchmarks -- Mind2Web and WebArena -- that collectively cover 1000+ tasks from 200+ domains across travel, shopping, and social media, among others. AWM substantially improves the baseline results by 24.6% and 51.1% relative success rate on Mind2Web and WebArena while reducing the number of steps taken to solve WebArena tasks successfully. Furthermore, online AWM robustly generalizes in cross-task, website, and domain evaluations, surpassing baselines from 8.9 to 14.0 absolute points as train-test task distribution gaps widen.
1 INTRODUCTION
Current language-model agents perform well on familiar action sequences but lack robustness across changing task contexts and environments. AWM addresses this by learning reusable workflows online or offline, improving benchmark performance and generalization.
- Current agents rely on fixed training or in-context examples, limiting robustness when task contexts or environments change.
- AWM continuously induces reusable workflows from new task experiences and applies them to guide subsequent actions.It can build increasingly complex workflows by composing newly learned routines with previously acquired ones.
- AWM operates offline from annotated examples or online without annotations by inducing workflows from self-generated predictions judged successful by an evaluator.
- 51.1% relative success rate improvement over the top published autonomous method was achieved on WebArena, while Mind2Web cross-task step-wise success improved 24.6%.
- 8.9–14.0 absolute points over baseline were achieved on Mind2Web cross-website and cross-domain splits, with larger margins as train-test distribution gaps widened.
2 AGENT WORKFLOW MEMORY
AWM represents reusable workflows as descriptions paired with executable trajectories, induces them from agent experiences, and adds them to memory for future task solving. It supports offline induction from training examples and online induction during test-time streaming.
- Workflow induction: An experience contains a natural-language task instruction and an action trajectory, while workflow induction maps past experiences to reusable workflows added to agent memory.The induction module is written as I(E) → W, and the resulting workflows guide subsequent task solving.
- Workflow representation: Each workflow pairs a natural-language description of its high-level goal with a trajectory of steps that completes that process.
- Workflow representation: Each workflow step records the observed environment state, the agent’s reasoning, and an executable action program.The representation includes state descriptions such as an order being shown and executable actions such as stop().
- Workflow induction: LM-based induction extracts fine-grained sub-routines shared across experiences and abstracts example-specific values to improve workflow generality.For example, a concrete product value can be replaced with a placeholder such as {product-name}.
- Offline scenario: Offline AWM induces workflows from all training examples before inference and uses the resulting shared workflow memory for test instructions.
- Online scenario: Online AWM processes test queries in a stream, inducing workflows from successful evaluated trajectories and integrating them before the next query.
3 EXPERIMENTS
Experiments on WebArena and Mind2Web show that AWM improves task success, supports cross-template and cross-domain generalization, learns rapidly from few examples, and builds increasingly complex workflows.
- WebArena main results: AWM achieves the best published WebArena results, surpassing BrowserGym by 12.0 absolute points and 51.1% relative overall success rate.It improves performance across all five website splits by 11.8–30.7 absolute points.
- WebArena main results: AWM uses about 2.0 fewer steps per WebArena example than BrowserGym while maintaining high success rates.Compared with Autoeval, AWM uses 40.8 fewer steps on average.
- Efficient learning from small amounts of data: AWM’s online learning curve rises rapidly within the first 0–40 examples and substantially improves performance with merely tens of examples.The agent first acquires essential workflows, then learns more advanced workflows as success rates stabilize.
- Cross-template workflow generalization: AWM achieves the highest performance overall and on each WebArena website split in the cross-template evaluation.This indicates that induced workflows generalize across examples instantiated from different task templates.
- Mind2Web main results: AWM improves Mind2Web step-wise and task-wise success rates over baselines by 4.0–8.9% relative and 0.4–2.8 absolute points.The gains mainly come from more accurate element selection, which increases by 5.0–9.0 points.
- Online AWM generalization: AWM improves Mind2Web step success by 7.4–8.9, 3.6–3.8, and 14.0–16.9 absolute points in cross-task, cross-website, and cross-domain scenarios.Online AWM generalizes more strongly than offline AWM on unseen websites and domains as train-test gaps widen.
4 EXPLORING OPTIMAL WORKFLOW REPRESENTATIONS
The experiments compare workflow induction methods, workflow representations, and environment-state formats. Abstract LM-induced workflows generally improve reuse and efficiency, while text and code perform similarly and natural-language state descriptions outperform filtered HTML.
- Sub-routine and abstract formats: Rule- and LM-based induction have nearly identical WebArena success rates, but LM-induced workflows use 0.4 fewer steps.Manual analysis attributes the efficiency difference to finer-grained workflows that avoid unnecessary steps.
- Sub-routine and abstract formats: AWM_lm improves Mind2Web cross-task performance over AWM_rule by a 2.8 margin.Its abstract workflow representation reduces bias toward elements resembling those in example trajectories.
- Sub-routine and abstract formats: Frequently used sub-routines are more flexibly reusable across test examples than full example trajectories.The results associate this abstract, reusable workflow structure with AWM_lm efficacy.
- Text versus code workflows: Text workflows slightly improve element selection accuracy and step success rate by 0.6 and 0.3 points, respectively, but reduce task success rate by 1.2.Overall, text and code workflows show no substantial performance variance.
- Environment abstraction: Replacing natural-language state descriptions with filtered HTML decreases step success rate by 0.8 points.Combining both formats performs worse, potentially because of longer contexts and irrelevant HTML items that can contradict descriptions.
5 EXPLORING WORKFLOW UTILIZATION IN CONTEXT AND IN ACTION
AWM can use workflows either as memory guidance or as callable additions to the agent’s action space. The action-space variant provides only small extra gains because agents rarely invoke the newly added workflow actions.
- Workflow actions: AWM_AS wraps induced workflows as high-level functions that agents can call alongside primitive actions.A workflow action executes its predetermined sequence of primitive actions.
- Results: AWM_AS improves step success rate by 1.3 points while matching AWM’s overall success rate of 3.2.Agents call workflow actions in only 18.5% of tasks, indicating limited use of the expanded action space.
- Workflow actions: Dynamic environment changes can challenge workflow action utilization.Figure 7 presents an example of such environment changes.
6 RELATED WORK
Related work spans web-agent benchmarks, action-space and memory enhancements, and methods for learning reusable procedures from experience. AWM combines reusable workflow induction with flexible use as contextual guidance.
- Web Agent Benchmarks: Web-agent benchmarks progressed from MiniWob and MiniWob++ to WebShop, WebArena, and VisualWebArena, increasing realism and task coverage.WebArena adds multiple websites and execution-based evaluation.
- Enhancing Agents for Complex Tasks: Existing agents are improved through constrained action search, self-feedback, human-designed actions, and in-context example demonstrations.These approaches modify action selection or augment agent memory.
- Learning Common Procedures from Experiences: Full example trajectories can entangle agents with example-specific contexts and hinder extrapolation to other tasks or domains.Other work extracts reusable sub-routines using rule-based or language-model-based methods.
- Learning Common Procedures from Experiences: AWM explores both rule- and LM-based induction to create reusable workflows used as context guidance without environment grounding issues.This positions workflow memory between example-based context augmentation and auxiliary-skill approaches.
7 CONCLUSION
The paper proposes Agent Workflow Memory for inducing and using reusable workflows in offline or online settings. Across web-navigation benchmarks, AWM improves task success and generalizes across tasks, websites, and domains.
- Contribution: AWM induces, augments, and uses workflows either offline from available examples or online at inference time.The method is designed for dynamic memory building and agent adaptation.
- Results: AWM achieves 24.6% and 51.1% relative increases in task success rate on Mind2Web and WebArena, respectively.These are the paper’s reported benchmark-level improvements.
- Generalization: AWM demonstrates generalization across tasks, websites, and domains.The conclusion frames this as evidence for adapting agents to varied digital tasks.
A LM-BASED WORKFLOW INDUCTION
The workflow induction module prompts language models to extract reusable abstract sub-routines from task examples. The experiments use the same prompt across WebArena and Mind2Web.
- A LM-BASED WORKFLOW INDUCTION: The module prompts language models to generate abstract sub-routine workflows from examples treated as experience.The section examines the prompt, induced exemplars, and workflow quality.
- A LM-BASED WORKFLOW INDUCTION: The WebArena and Mind2Web experiments use the same model prompt.
- A LM-BASED WORKFLOW INDUCTION: Each workflow must capture a commonly reused sub-routine spanning multiple web-navigation tasks without similar or overlapping workflows.The prompt also requires workflows to contain at least two steps.
A.2 EXAMPLE WORKFLOWS
The induced workflows encode reusable multi-step routines for shopping, entertainment, forums, repositories, maps, and travel. Examples vary from short site-specific actions to longer task procedures with parameterized elements.
- WebArena: Site workflows cover editing and saving fields, navigating Reddit forums, and finding GitLab contributor details after repository search.
- map: A map workflow fills origin, destination, and transportation mode before requesting travel time and distance.
- travel: A travel workflow enters departure and destination airports or cities and selects the best popup options.
- shopping: Shopping workflows search for products and sort results, or browse products by hovering over a category and clicking a subcategory.
- entertainment: An IMDb workflow searches for a term and selects the best match through a submit action followed by a result click.
A.3 WORKFLOW QUALITY ANALYSIS
Workflow quality is assessed through workflow count, reuse, overlap, and coverage, with dataset-specific evaluation constraints. Model-induced workflows are widely used, while coverage varies with train-test task distribution differences.
- A.3 WORKFLOW QUALITY ANALYSIS: The quality analysis measures workflow count, coverage of trajectory steps, and overlap between workflows.Fewer workflows, higher coverage, and lower overlap are treated as preferable quality signals.
- A.3 WORKFLOW QUALITY ANALYSIS: WebArena coverage is not evaluated because canonical trajectories are unavailable, while Mind2Web cross-website and cross-domain coverage are excluded because training workflows lack domain overlap.
- A.3 WORKFLOW QUALITY ANALYSIS: 7.3–7.4 workflows per example are induced by neural-based induction on the evaluated datasets.
- A.3 WORKFLOW QUALITY ANALYSIS: 0.94 of WebArena test examples use induced workflows, with only 0.08 overlap between workflow steps.
- A.3 WORKFLOW QUALITY ANALYSIS: Mind2Web workflows have a 0.91 utility rate but 0.40 coverage over test examples, which the authors relate to task-distribution variance.
B RULE-BASED WORKFLOW INDUCTION
The rule-based induction module first deduplicates experiences and then filters invalid actions. The resulting cleaned trajectories are used as rule-based workflows.
- B RULE-BASED WORKFLOW INDUCTION: Rule-based workflow induction consists of experience deduplication and invalid action filtering.
- B RULE-BASED WORKFLOW INDUCTION: Deduplication groups experiences by action sequence and, on WebArena, additionally by task template before selecting examples.The default selection is n = 1 experience per group.
- B RULE-BASED WORKFLOW INDUCTION: Invalid-action filtering removes CLICK and TYPE steps whose first arguments are not string-formatted integer element identifiers.The filtered trajectories are taken as rule-based workflows.
C INTEGRATING AWM OFFLINE AND ONLINE
The combined AWMoff +on approach uses offline workflows to warm-start task solving and online workflows to adapt to test distributions. However, the two workflow sets are not fully compatible, yielding intermediate performance across three Mind2Web generalization splits.
- AWMoff +on injects relevant training workflows to warm-start task solving while aggregating increasingly more online-induced workflows for test-distribution adaptation.
- Table 11 evaluates success rate on Mind2Web cross-task, cross-website, and cross-domain generalization using GPT-4.The table uses EA for element accuracy and AF1 for action F1.
- AWMoff +on scores between AWMoffline and AWMonline across the three test splits rather than producing an additive effect.
- Offline workflows can impair the generative quality and utility efficacy of online workflows, resulting in medium overall results.