Source-linked AI summary
From Review to Reuse: How Post-Task Workflow Can Support Human-AI Agent Interaction
Zekun Wu, Xinru Wang, Rock Yuren Pang, Chenglong Wang, Anna Maria Feit
TL;DR
Agents can complete complex tasks from high-level prompts, but users still need to understand and validate their actions. This paper develops interactive post-task workflows from completed execution traces and finds they support review and can be adapted for related tasks. In the study, workflow adaptation matched prompt adaptation in success, time, and difficulty.
Problem
Users must connect distributed execution information to understand and validate agent actions, while it remains unclear what post-task workflows add beyond prompts, traces, and results.
Source: §1 · pp. 2–3, §2.3 · p. 5Method
The authors analyzed 10,803 n8n templates and developed Trace2Flow, which translates completed agent traces into interactive post-task workflows for a user study.
Source: §1 · pp. 3–4Results
Post-task workflows improved understanding and error detection; for related tasks, workflow adaptation did not differ significantly from prompt adaptation in success, completion time, or perceived difficulty.
Source: §1 · pp. 3–4Takeaways & Limitations
A completed execution can be represented as a descriptive workflow that organizes actions and dependencies for users to understand, validate, and reuse.
Source: §1 · p. 3The study used compact tasks with limited complexity and follow-up tasks requiring little modification, limiting conclusions about larger workflows or broader adaptations.
Source: §7.4 · p. 23
Abstract
from arXiv · showhide
AI agents can automate tasks by turning a single natural-language request into a multi-step process spanning tools, files, and applications. Users are often left to judge that process from fragmented execution information and the final output. To make the completed process easier to understand, validate, and reuse, we investigate post-task workflows: editable, graph-based representations of an agent's completed execution. We first analyzed 10,803 public workflow templates from n8n to characterize real-world automation practice, then developed Trace2Flow, a research probe that translates agent execution traces into interactive post-task workflows. In a study, participants (N = 20) reviewed agent executions with prompt or agent errors. We found that post-task workflows improved their understanding and error detection over a prompt-only condition, and that validation succeeded mainly when users cross-checked across multiple evidence sources. For follow-up tasks, adapting the workflow matched adapting the prior prompt in success, time, and difficulty, and was often preferred.
1 Introduction
Natural-language prompts let agents carry out complex tasks, but leave users to understand and validate how the agent acted. This paper examines editable post-task workflows as a way to organize that oversight and support reuse.
1 Introduction
- Users must review how agents interpreted intent, verify actions and outputs, and identify errors, shifting their role from task execution to oversight.Existing interfaces expose traces, tool calls, intermediate outputs, and results, but users may still need to connect information distributed across many actions and tools.Source: §1 · p. 2
- The study found that workflow access encouraged process examination and helped users recognize prompt and agent errors that final-output checking alone could miss.For related tasks, workflow reuse and prompt adaptation did not differ significantly in task success, completion time, or perceived difficulty.Source: §1 · p. 3
- The paper characterizes real-world workflow practice using 10,803 public n8n templates, informing post-task workflow design and study-task construction.The characterization covers task domains, structural patterns, and node functions.Source: §1 · p. 4
- Trace2Flow translates completed prompt-based agent execution traces into interactive post-task workflows.The representation is descriptive rather than prescriptive, retaining high-level prompting while organizing actions, intermediate artifacts, and dependencies.Source: §1 · pp. 3–4
- Post-task workflows improved users’ understanding, inspection, and validation of completed executions, while workflow adaptation was a viable alternative to prompt reuse for related tasks.Preferences varied with prior workflow experience, adaptation scope, and prompt availability.Source: §1 · p. 4
2 Related Work
Prior work uses editable workflows to organize multi-step automation and to make agent executions more inspectable. This paper studies workflows derived after task completion, addressing what they add for review and reuse.
2.1 Workflow Representations for Task Automation
- Low-code workflow tools organize operations across applications and services; combining AI operations with editable workflow structure makes this practice relevant to post-task workflows.Platforms such as n8n, Node-RED, and Zapier let users compose and configure operations from applications and APIs.Source: §2.1 · p. 4
2.2 AI Agent Process Transparency
- Transparency design must balance an overview with step-level traceability, while limiting visibility and intervention that can add cognitive and interaction costs.Prior work also notes that verbose raw logs can hinder diagnosis, and that post-hoc review is prominent in developers’ oversight of coding agents.Source: §2.2 · pp. 4–5
2.3 Workflows for Agent Execution Review and Reuse
- Prior work derives workflows from agent executions or generates them from user inputs, but less is known about what post-task workflows add for users’ review and reuse.Figure 2 describes the analyzed n8n templates’ task-domain distribution and examples; customer operations was the largest category, with 3,061 workflows (28%).Source: §2.3 · pp. 5–6, Figure 2 · p. 6
3 Workflow Representations in Real-World Practice
An analysis of 10,803 n8n templates characterizes their task domains, structures, and node functions. These patterns ground the study in real-world workflow practice.
3.1 Task domains
- Customer operations was the largest domain, with 3,061 workflows (28%); data analytics and personal assistance each accounted for approximately 22%, content production 19%, and Other 9%.The classification scheme was developed from a manual review of 100 templates and applied to all 10,803 using an LLM.Source: §3.1 · p. 6
3.2 Workflow complexity
- Workflows had a median of 17 nodes, 12 edges, and an 8-step longest path; 49% had exactly one branch or merge, while 24% had a cycle.Median average node degree was 1.4; 17% were purely sequential, and the median number of cycles was zero.Source: §3.2 · p. 7, Figure 3 · p. 7
3.3 Node functions
- Task-action nodes appeared in 94% of workflows, workflow-control nodes in 87%, data-processing nodes in 82%, and AI-supported operations in 51%.Common nodes included Code (54%), Set (52%), HTTP Request (50%), and If (45%); the categories work together in multi-step processes such as review sentiment classification.Source: §3.3 · pp. 7–8, Figure 4 · p. 8
4 Trace2Flow: A Research Probe for Post-Task Workflows
Formative surveys identified review and reuse needs that informed Trace2Flow’s design. The probe links prompt-based agent execution to an editable workflow and translates execution traces into executable workflow graphs.
4.1 Formative Study and Design Goals
- Surveys of 42 AI-agent users and 39 workflow-automation users found needs around reviewing completed executions and adapting workflows for related tasks.Agent users reported difficulty understanding actions, verifying results, and locating errors; workflow users said reuse suited similar tasks, while natural-language delegation remained preferable for substantially different or exploratory tasks.Source: §4.1 · pp. 8–9
- Trace2Flow’s design goals are to make execution traceable, support workflow adaptation and reuse, and complement rather than replace natural-language delegation.The workflow exposes steps, tool calls, inputs, and outputs; users can rerun steps or change inputs, steps, and tools while preserving the rest of a process.Source: §4.1 · p. 9
4.2.1 Interface Experience.
- The chat interface supports prompt-based delegation and progress visibility, while the workflow canvas lets users inspect, edit, and rerun a completed execution.Users can modify node parameters, add or remove nodes, change connections, and rerun the full workflow for an adapted task.Source: §4.2.1 · pp. 9–10
4.2.2 Implementation: Trace-to-Workflow Translation.
- Trace2Flow grounds candidate operations in the execution trace, uses an LLM to generate an intermediate representation, then compiles it into an executable graph.Grounding retains operation inputs, outputs, and observed example outputs; compilation creates input bindings and graph edges and checks node references and input-output compatibility.Source: §4.2.2 · p. 10, Figure 6 · p. 12
5 User Study
The study used five related review and follow-up task pairs to compare reviewing executions and completing follow-up tasks with or without access to a post-task workflow. Participants encountered reproduced executions containing prompt or agent errors.
5.1.1 Study Task Pairs.
- Five task pairs adapted from real n8n workflows covered data analytics, customer operations, and content production across varied tools and data sources.Reference workflows had longest paths of five to ten steps and included different structural patterns.Source: §5.1.1 · p. 12, §5 · p. 12
- Each follow-up task retained most of its review task while changing a limited part, such as a candidate profile or how results were organized.The five pairs are summarized in Table 1.Source: §5.1.1 · p. 13, Table 1 · p. 13
5.1.2 Prompt Construction and Agent Execution.
- For each review task, two authors constructed prompts from task contexts specifying goals, available inputs and tools, constraints, expected outputs, and process requirements.They made three to five prompts per task and recorded the resulting execution traces and outputs.Source: §5.1.2 · p. 13
5.1.3 Error cases.
- The selected cases included two prompt errors, in T2 and T3, and three agent errors, in T0, T1, and T4.Prompt errors omitted or misexpressed requirements; agent errors involved misinterpreting or failing to follow the prompt.Source: §5.1.3 · p. 13, Table 2 · p. 14
5.2 Study Design
- In the within-participant design, chat-condition participants reviewed through chat and wrote a new prompt; workflow-condition participants could access and adapt the post-task workflow.Condition order and task-pair assignment were counterbalanced.Source: §5.2 · p. 14
5.2.1 Participants Recruitment.
- Twenty active AI-agent users completed the study; seven had recent workflow-automation experience and 13 did not.The analysis included 20 of 22 recruited participants, aged 18–32.Source: §5.2.1 · p. 15
5.2.2 Study Procedure.
- After a tutorial, participants completed two or three sessions, judged task success, described execution steps and tools, and rated understanding.A Wizard-of-Oz setup reproduced the same execution and error for participants assigned the same task; interviews followed the task sessions.Source: §5.2.2 · p. 15
Appendix. 5.3 Measures and Data Analysis
The analysis combined participant task responses and ratings with interaction recordings, interviews, and follow-up artifacts. Quantitative measures were tested according to whether they were continuous ratings or binary outcomes.
RQ1
- RQ1 measures covered reported steps and tools, process-comprehension accuracy, self-reported understanding, and success in validating task completion.Screen recordings and interviews were analyzed for review actions and how participants combined inputs, outputs, prompts, and workflow information when judging execution.Source: Appendix. 5.3 Measures and Data Analysis · pp. 15–16
RQ2
- RQ2 assessed follow-up task success, completion time, and perceived difficulty, alongside how participants approached tasks and preferred prompting or workflow reuse.Qualitative analysis considered submitted prompts, adapted workflows, recordings, and interviews, including the relation of preferences to prior workflow experience and adaptation scope.Source: Appendix. 5.3 Measures and Data Analysis · p. 16
Quantitative analysis
- Non-binary measures were compared with two-sided Wilcoxon signed-rank tests; binary validation and task-success outcomes were modeled across 46 sessions using logistic mixed effects.The binary models included condition, task, and session number, with participant as a random intercept.Source: Appendix. 5.3 Measures and Data Analysis · p. 16
6 Results
In the review task, workflow representations improved participants’ understanding and validation of agent executions, with successful validation most common when participants cross-checked evidence sources. Follow-up task performance showed no significant condition differences, while reuse preferences varied with adaptation scope and participants’ workflow experience.
6.1.1 Understanding the Agent’s Execution Process.
- Validation succeeded in 15 of 23 workflow sessions (65%) versus 7 of 23 chat sessions (30%), with a significant condition effect.Validation success was also significantly higher for T3 than T4.Source: §6.1.1 · p. 17, Figure 9 · p. 18
- Workflow validation gains varied with error visibility: T2’s misrouted article groups yielded 17% vs. 83%, while T3 success rose from 67% to 100%.T1 showed the smallest difference (33% vs. 50%): its fictional-location error required questioning CV content, and the workflow offered little extra process evidence.Source: §6.1.1 · p. 17, Table 3 · p. 18
- Validation success was 74% when participants cross-checked sources overall and 85% when workflow-condition participants combined output checks, workflow checks, and cross-checking.Output-only checking succeeded in 1 of 10 sessions (10%); neither workflow-only session produced a correct judgment, while separate output and workflow checks succeeded in 4 of 8 sessions (50%).Source: §6.1.1 · pp. 18–19, Table 4 · p. 18
6.1.3 Interview Results.
- Seventeen of 20 participants found workflows helpful for review, often because they exposed process structure and let users inspect step inputs, outputs, parameters, and tool-call results.Among participants who correctly identified errors in two tasks, 70% explicitly said the workflow helped them notice the problem when it appeared in execution structure.Source: §6.1.3 · p. 19
6.2.1 Follow-up Task Outcomes: Accuracy, Efficiency, and Perceived Difficulty.
- Follow-up task success was 17 of 23 workflow sessions (74%) versus 13 of 23 chat sessions (57%), with no significant condition effect; time and perceived difficulty also did not differ significantly.Participants took M= 6 min 12 sec in workflow sessions versus M= 5 min 24 sec in chat sessions.Source: §6.2.1 · pp. 19–20, Figure 10 · p. 20
6.2.2 Interview Results.
- Nine participants preferred adapting the workflow, six preferred editing the prior prompt, and five said their preference depended on the situation.Workflow advocates valued locating changes and inspecting or testing steps; prompt advocates cited workflow learning costs and tedious edits when changes were numerous or spread out.Source: §6.2.2 · pp. 20–21
- Adaptation scope and workflow experience shaped preferences: small local changes favored workflows, broader restructuring favored prompts, and six of seven recently experienced workflow users preferred workflows.The authors describe the experience pattern as exploratory given the small subgroups.Source: §6.2.2 · p. 21
7 Discussion and Conclusion
Post-task workflows organized execution information to support understanding and cross-checking, while their usefulness depended on whether errors were visible in the process structure or required judging underlying content. Workflow reuse matched prompt reuse on measured outcomes, but the study’s task scope and Trace2Flow’s research-probe role limit broader conclusions.
7.1 Post-Task Workflows as Post Hoc Procedural Transparency
- Workflow-condition participants recalled more major steps and tools and showed higher process comprehension; successful validation especially involved cross-checking multiple evidence sources.Participants could move between the prompt, output, workflow, and individual nodes rather than relying mainly on the final output.Source: §7.1 · pp. 21–22
7.2 What Workflows Reveal—and What They Do Not
- Participants still judged incorrectly in 8 of 23 sessions (35%); workflows helped less when errors depended on content correctness, requiring direct inspection of task data or outputs.The workflow was not sufficient on its own for validating every aspect of execution.Source: §7.2 · pp. 22–23
7.3 Post-Task Workflows as a Persistent Artifact
- Workflow reuse showed no significant differences from prompt reuse in task success, completion time, or perceived difficulty, while preferences varied with experience, available artifacts, and adaptation scope.Localized changes mapped more easily to workflow steps, while broader restructuring favored prompting; prompt preference depended substantially on having the previous prompt available.Source: §7.3 · p. 23
7.4 Limitations and Future Works
- The study’s small, experienced-user sample and compact tasks with structurally similar follow-ups constrain conclusions about novices, larger workflows, and broader adaptations.Only seven participants had recent workflow-automation experience, and the relationship between that experience and reuse remains exploratory.Source: §7.4 · p. 23
- Trace2Flow is a research probe, and its evaluation on the study task set does not establish generalization across broader tasks, tools, or agents.Future work could evaluate translation across more diverse tasks and compare effects of translation approaches on understanding, validation, and reuse.Source: §7.4 · pp. 23–24