PaperlayerView exact revision on arXiv

Source-linked AI summary

PROV-AGENT: Unified Provenance for Tracking AI Agent Interactions in Agentic Workflows

Renan Souza, Amal Gueroudji, Stephen DeWitt, Daniel Rosendo, Tirthankar Ghosal, Robert Ross, Prasanna Balaprakash, Rafael Ferreira da Silva

arXiv:2508.02866v3cs.DCcs.DB

TL;DR

Agent interactions and their downstream effects are difficult to trace because existing provenance methods do not connect agent artifacts with broader workflow context. PROV-AGENT extends W3C PROV with MCP concepts and captures agent interactions alongside workflow data and tasks. The evaluation demonstrates queries for tracing agent behavior and supporting accountability, debugging, and iterative improvement.

  • Problem

    Existing provenance approaches do not explicitly represent agent artifacts such as prompts and responses together with workflow context and downstream effects.

    Source: §I · p. 1, A. Provenance for Tracking AI Agents… · p. 2

  • Method

    PROV-AGENT extends W3C PROV and incorporates MCP concepts to connect agent interactions and model invocations with workflow tasks and data.

    Source: §III · p. 3

  • Results

    The evaluation demonstrates queries that support end-to-end analysis of agent behavior, accountability, debugging, and iterative improvement.

    Source: B. Preliminary Evaluation · p. 6

  • Takeaways & Limitations

    A connected provenance graph can trace outputs and decisions through agent reasoning, prompts, input data, system context, and execution metadata.

    Source: §III · p. 3

Abstract

from arXiv · show

Large Language Models (LLMs) and other foundation models are increasingly used as the core of AI agents. In agentic workflows, these agents plan tasks, interact with humans and peers, and influence scientific outcomes across federated and heterogeneous environments. However, agents can hallucinate or reason incorrectly, propagating errors when one agent's output becomes another's input. Thus, assuring that agents' actions are transparent, traceable, reproducible, and reliable is critical to assess hallucination risks and mitigate their workflow impacts. While provenance techniques have long supported these principles, existing methods fail to capture and relate agent-centric metadata such as prompts, responses, and decisions with the broader workflow context and downstream outcomes. In this paper, we introduce PROV-AGENT, a provenance model that extends W3C PROV and leverages the Model Context Protocol (MCP) and data observability to integrate agent interactions into end-to-end workflow provenance. Our contributions include: (1) a provenance model tailored for agentic workflows, (2) a near real-time, open-source system for capturing agentic provenance, and (3) a cross-facility evaluation spanning edge, cloud, and HPC environments, demonstrating support for critical provenance queries and agent reliability analysis.

I. INTRODUCTION

Agentic workflows use autonomous agents that make adaptive decisions across heterogeneous platforms, but their non-determinism and error propagation make transparency and reliability difficult. The paper introduces PROV-AGENT and a runtime capture system to connect agent interactions with workflow provenance.

I. INTRODUCTION

  • Agentic workflows differ from static deterministic workflows because near-real-time data, adaptive decisions, and evolving interactions can produce dynamic, cyclic behavior.Agent outputs may inform later decisions or feedback loops.Source: §I · p. 1
  • Agents can produce incorrect outputs that propagate across agents and downstream tasks, complicating assessment of results and motivating provenance that captures prompts, responses, model invocations, and context.Tracing these interactions supports root-cause analysis of unexpected behavior.Source: §I · p. 1
  • A unified provenance graph treating AI agent actions as first-class workflow components supports queries tracing inputs, decisions, affected outputs, and the origin and propagation of erroneous data.The paper connects these queries to interpreting results, debugging, and improving prompts or model tuning.Source: §I · p. 1
  • PROV-AGENT extends the W3C PROV standard with Model Context Protocol (MCP) concepts to represent agent actions and their connections to data and workflow tasks.The authors also present an open-source system for data observability and runtime agentic provenance capture in workflows, plus a preliminary evaluation using a cross-facility agentic workflow involving edge devices, cloud services, and HPC systems.Source: §I · p. 2

A. Provenance for Tracking AI Agents in Dynamic Cross-Facility Workflows

Cross-facility workflows move near-real-time experimental data across edge, cloud, and HPC systems while agents interact with that data. Existing agent records are often disconnected from workflow provenance, limiting context for tracing downstream effects.

A. Provenance for Tracking AI Agents in Dynamic Cross-Facility Workflows

  • Experimental data move in near real time from edge facilities to HPC systems for simulations, analytics, or machine-learning training, requiring coherent tracking of agent interactions across sites.The workflow figure depicts results feeding back into upstream tasks, with agentic tools alongside traditional tasks.Source: A. Provenance for Tracking AI Agents… · p. 2, Figure 1 · p. 2
  • MCP-based frameworks may record prompts, responses, and model invocations, but these records are typically isolated from workflow provenance and lack explicit representations of agent artifacts and dynamic reasoning.The proposed definition of agentic provenance tracks agent tasks and their influence on downstream non-agent tasks and data.Source: A. Provenance for Tracking AI Agents… · p. 2
  • Agentic provenance unifies agent tasks and their effects on downstream tasks and data in a queryable graph, supporting traceability, root-cause analysis, and continuous agent improvement.The passage names prompt refinement and model-parameter tuning as examples of improvement.Source: A. Provenance for Tracking AI Agents… · p. 2

B. W3C PROV and Extensions for Workflows, AI, and Agents

The paper builds on W3C PROV, a standardized way to describe how data were produced, by whom, and through which processes. Existing PROV extensions cover workflow, machine-learning, or human-action provenance, while prior agent extensions lack concepts for agentic workflows and their broader workflow relations.

B. W3C PROV and Extensions for Workflows, AI, and Agents

  • W3C PROV represents provenance through information about how data were produced, by whom, and through which processes.Its core classes are Agent, Entity, and Activity.Source: B. W3C PROV and Extensions for… · p. 2, Figure 2 · p. 2
  • PROV supports specialized extensions: PROV-DfA captures human actions, ProvONE adds workflow metadata, PROV-ML models machine-learning artifacts, and FAIR4ML takes a model-centric approach.The passage describes these approaches as complementary to representing agents that steer workflows.Source: B. W3C PROV and Extensions for… · pp. 2–3
  • Earlier PROV extensions for agents and multi-agent systems, [25] predate agentic workflows and lack core agentic AI concepts and their relations to the broader workflow [26].Source: B. W3C PROV and Extensions for… · p. 3

III. A PROVENANCE MODEL FOR AGENTIC WORKFLOWS

PROV-AGENT extends W3C PROV with MCP concepts to represent agent interactions, model invocations, and their relationships to conventional workflow tasks and data. Its connected provenance graph captures workflow, execution, and agent context for tracing decisions and outputs.

III. A PROVENANCE MODEL FOR AGENTIC WORKFLOWS

  • PROV-AGENT extends W3C PROV and incorporates MCP concepts to make agents and traditional workflow components first-class elements in a model of agent interactions and their links to tasks and data.Source: §III · p. 3, Figure 3 · p. 3
  • Campaigns, workflows, and tasks are modeled as PROV Activities; tasks use and generate domain data and also generate scheduling and telemetry metadata.Scheduling data records execution location, while telemetry records runtime resource metrics such as CPU, GPU, and disk usage.Source: §III · p. 3
  • AIAgent is a subclass of PROV Agent, and the model can represent multiple agents with distinct tools and reasoning paths in collaborative or parallel workflows.Source: §III · p. 3
  • Agent tools can be informed by model invocations that use prompts and model metadata and generate responses attributed to the agent; PROV links connect these artifacts to data and workflow context.The model is modality-agnostic for foundation models that follow a prompt-invocation-response pattern.Source: §III · p. 3
  • The connected graph supports tracing a final output or decision back through agent reasoning, prompts, input data, system context, and execution metadata.Source: §III · p. 3

A. Implementation

The implementation extends Flowcept to capture agent-tool and LLM activity as provenance linked to workflow data and tasks. It also provides a natural-language interface for querying provenance at runtime.

A. Implementation

  • Flowcept consolidates heterogeneous provenance during workflow execution into a persistent database using a W3C PROV-extended model.It accepts data from instrumented scripts, observability hooks, streaming services, and storage layers.Source: A. Implementation · pp. 3–4
  • The @flowcept_agent_tool decorator records each MCP tool execution as an AgentTool activity linked to its agent, inputs, and outputs.The linked inputs and outputs use PROV-AGENT relationships.Source: A. Implementation · p. 4
  • FlowceptLLM records prompts, responses, model metadata, and optional response-time telemetry for supported LLM calls.Each call becomes an AIModelInvocation linked to its model, prompt, response, and any AgentTool activity informed by it.Source: A. Implementation · p. 4, Figure 4 · p. 4
  • A Streamlit MCP agent lets users query and explore the provenance database using natural language at runtime.The figure illustrates an MCP tool invoking an LLM to assess physics-model outputs, with provenance captured for both tool and LLM invocation.Source: A. Implementation · p. 4, Figure 4 · p. 4

B. Preliminary Evaluation

The evaluation demonstrates how PROV-AGENT unifies agent decisions, LLM interactions, and workflow tasks in a provenance graph. Queries use this graph to trace inputs, decisions, and downstream effects in an additive-manufacturing workflow.

B. Preliminary Evaluation

  • The additive-manufacturing use case links layer-by-layer sensor data at the edge with HPC simulations and cloud-hosted gpt-4o agent decisions.Each layer's decision informs the next; the direct live sensor-to-simulation connection remains under development.Source: B. Preliminary Evaluation · p. 4
  • The provenance graph links sensor data, physics-model results, evaluation scores, agent tools, LLM prompts and responses, and layer-specific decisions.Activities also carry location metadata for execution across edge, HPC, and cloud; some schema components are omitted from the figure for clarity.Source: Figure 5 · p. 5, B. Preliminary Evaluation · p. 6
  • A decision-lineage query traces an agent decision through its tool, model evaluation, and physics model to the originating sensor data and experiment setup.Another query retrieves the scores, control results, and LLM response that contextualized a layer's decision.Source: B. Preliminary Evaluation · p. 6
  • After a surprising layer-2 decision, a query retrieves its linked tool, LLM invocation, prompt, and response.Other queries recursively follow decision relationships to later layers and trace faulty decisions backward to inputs and forward to affected outputs.Source: B. Preliminary Evaluation · p. 6
  • PROV-AGENT enables end-to-end analysis of agent behavior within workflows, supporting accountability, debugging, and iterative improvement.The evaluation demonstrates tracing erroneous outputs to upstream prompts, inputs, and prior decisions.Source: B. Preliminary Evaluation · pp. 4, 6

V. CONCLUSION

PROV-AGENT extends W3C PROV with agent artifacts linked to workflow data and tasks, and the authors demonstrate its implementation in a distributed workflow with feedback loops. They present it as a foundation for tracing agent behavior and supporting analysis and improvement.

V. CONCLUSION

  • PROV-AGENT extends W3C PROV to represent agent tools, prompts, responses, and model invocations alongside workflow data and tasks.The authors describe it as the first provenance framework designed to track agent actions and decisions in agentic workflows, to their knowledge.Source: §V · p. 6
  • The open-source implementation is demonstrated in an agentic workflow on distributed facilities, with feedback loops and AI agents steering execution.The authors characterize the work as an early step and a foundation for further research and practice.Source: §V · p. 6
  • The unified provenance approach supports root-cause analysis and comparison of decisions across prompt refinements or model-parameter fine-tuning.The authors identify hallucination detection and remediation as directions to explore, not demonstrated outcomes.Source: §V · p. 6

Related papers

Loading 2508.02866v3…