PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems

Dany Moshkovich, Hadar Mulian, Sergey Zeltyn, Natti Eder, Inna Skarbovsky, Roy Abitbol

arXiv:2503.06745v1cs.AIcs.MA

TL;DR

Agentic systems’ variable, context-sensitive behavior challenges outcome-only evaluation. The paper proposes observability and analytics taxonomies and a trace-log-based benchmark for agent analytics; experiments and a practitioner study report variability and unmet diagnosis needs.

  • Problem

    Black-box benchmarking does not capture the variable execution flows, interactions, and outputs of agentic systems that matter for evaluating their behavior.

    Source: §1 · p. 2, §3.1 · p. 3

  • Method

    The paper develops observability and analytics taxonomies and benchmarks analytics systems using agent runtime trace logs as the evaluation unit.

    Source: §1 · p. 2, §5 · p. 6, §4.4 · p. 6

  • Results

    The paper reports variability in execution and outputs, while 80% of surveyed practitioners identified non-deterministic execution as a major issue.

    Source: §3.4 · p. 4, §6 · p. 8

  • Takeaways & Limitations

    The authors conclude that analytics benchmarks and observability taxonomies provide a foundation for systematic evaluation, but need refinement for interpretability and actionability.

    Source: §6 · p. 8

    Agent behavior remains sensitive to prompt context and preceding interactions, with multi-agent dependencies adding coordination challenges.

    Source: §3.5 · p. 4

Abstract

from arXiv · show

The rise of agentic AI systems, where agents collaborate to perform diverse tasks, poses new challenges with observing, analyzing and optimizing their behavior. Traditional evaluation and benchmarking approaches struggle to handle the non-deterministic, context-sensitive, and dynamic nature of these systems. This paper explores key challenges and opportunities in analyzing and optimizing agentic systems across development, testing, and maintenance. We explore critical issues such as natural language variability and unpredictable execution flows, which hinder predictability and control, demanding adaptive strategies to manage input variability and evolving behaviors. Through our user study, we supported these hypotheses. In particular, we showed a 79% agreement that non deterministic flow of agentic systems acts as a major challenge. Finally, we validated our statements empirically advocating the need for moving beyond classical benchmarking. To bridge these gaps, we introduce taxonomies to present expected analytics outcomes and the ways to collect them by extending standard observability frameworks. Building on these foundations, we introduce and demonstrate novel approach for benchmarking of agent evaluation systems. Unlike traditional "black box" performance evaluation approaches, our benchmark is built from agent runtime logs as input, and analytics outcome including discovered flows and issues. By addressing key limitations in existing methodologies, we aim to set the stage for more advanced and holistic evaluation strategies, which could foster the development of adaptive, interpretable, and robust agentic AI systems.

1 INTRODUCTION

Agentic systems’ variable behavior makes outcome-only benchmarking inadequate for understanding execution and interactions. The paper proposes behavioral benchmarking, observability and analytics taxonomies, and a benchmark dataset for agent analytics.

1 INTRODUCTION

  • Natural-language phrasing, LLM stochasticity, and system interdependencies can change execution flows and outputs, limiting conventional black-box evaluation.The paper argues evaluation should capture execution patterns, decision-making, interactions, variability, and system dynamics.Source: §1 · p. 2
  • The authors identify gaps in traditional benchmarking and report validating their impact across development, testing, and maintenance.They use empirical experiments and a user study to examine these gaps.Source: §1 · p. 2
  • The paper introduces Agentic System Behavioral Benchmarking, core agentic-system elements, and semantic conventions for observability and analytics taxonomies.Source: §1 · p. 2
  • The proposed agent-analytics benchmarking methodology is accompanied by the ABBench dataset, which follows that methodology.Source: §1 · p. 2

2 RELATED ART

Related work traces observability from unstructured logs to standardized monitoring and specialized tools for LLM applications. Existing frameworks and benchmarks vary in capabilities and evaluation dimensions.

2 RELATED ART

  • As software systems grew more distributed and complex, monitoring moved from unstructured logs toward standardized observability frameworks such as OpenTelemetry.Source: §2 · p. 2
  • Early conversational AI exposed monitoring challenges around interaction data, contributing to modern AI-system traceability approaches.Source: §2 · p. 2
  • LLM observability tools extend monitoring to non-deterministic applications and complex interactions, with offerings such as OpenLLMetry and LangSmith.The passage describes token usage, cost, latency, traces, debugging, evaluation, and testing among their capabilities.Source: §2 · p. 2
  • Frameworks differ in integration, security, flexibility, and specialization, while benchmarking spans quantitative and qualitative measures and deployment concerns.Examples include execution time, resource use, success rates, decision quality, behavioral consistency, scalability, and regulatory compliance.Source: §2 · p. 2

3 AGENTIC SYSTEM ANALYSIS: GAPS AND REQUIREMENTS

The section characterizes variability in agentic execution, measures it in calculator experiments, and reports practitioners’ observability needs. It argues that black-box benchmarking alone is insufficient given context, temporal, and multi-agent dependencies.

3.1 Unique Characteristics of Agentic Systems

  • Agentic systems can diverge in execution paths even for identical inputs, while natural-language phrasing and context can also alter outcomes.Source: §3.1 · p. 3

3.2 Illustrative Example: A Calculator Agentic Systems

  • The calculator example handles nested expressions and parallel or distributed computation, with a LangGraph-based planner sequencing and validating decomposed work.It illustrates dynamic flows, multi-agent interactions, and integration with multiple frameworks and LLMs.Source: §3.2 · p. 3, Figure 2 · p. 3

3.3 Experimentation Setup and Results

  • The calculator experiment tested 50 examples five times each, measuring accuracy with MSE and repeated-flow variation with graph-edit distance.It also tracked cost and execution time and calculated coefficients of variation across two example variations.Source: §3.3 · p. 3

3.3.1 Flow Variability:

  • 63% mean CV across pure-math examples indicates substantial variation in task counts and execution-path structure.The paper assesses flow differences using edit distances between traces for identical inputs.Source: §3.3.1 · p. 3

3.3.2 NL Variability:

  • 19% mean CV in accuracy on natural-language math problems shows that identical inputs can yield inconsistent outputs across runs.Table 1 is captioned as reporting coefficients of variation across performance dimensions.Source: §3.3.2 · p. 3, Table 1 · p. 4

3.4 User Study

  • Among 38 practitioners, 80% identified non-deterministic execution as a major issue, and 77% struggled with root-cause diagnosis.76% prioritized understanding agentic flows as their top analytics need.Source: §3.4 · p. 4

3.5 Limitations and the Need for a Comprehensive Agentic System Approach

  • Context sensitivity, prior interactions, and multi-agent dependencies contribute to unpredictable outcomes and amplify variability.Source: §3.5 · p. 4
  • Trace complexity, data scalability, and explainability hinder observability; the authors argue black-box benchmarking needs supplementation with agent analytics.Their proposed approach combines enhanced observability, adaptive evaluation, and metrics and optimizations tailored to multi-agent LLM systems.Source: §3.5 · p. 4

4 AGENT OBSERVABILITY AND ANALYTICS APPROACH

The section extends benchmarking from final outcomes to agent behavior and component interactions, then outlines the entities, observability conventions, and analytics needed to analyze and optimize agentic systems. Its taxonomy spans passive, exploratory, and interventional analytics.

4.1 Towards Behavioral Benchmarking

  • Behavioral benchmarking evaluates both system behavior and component interactions, not only outcomes, and requires standardized observability data and analytical methods.The proposed framework is intended to support evaluation and optimization.Source: §4.1 · p. 4

4.2 Core Entities of AI Agentic Systems

  • Tasks are discrete units executed by tools, and the paper extends the task concept to all agentic tool invocations for a unified execution model.Tasks can be decomposed into subtasks, depend on one another, and be scheduled or prioritized.Source: §4.2 · p. 5

4.3 Observability of Agentic Systems

  • GenAI Events attached to spans record agent-entity lifecycle changes and issues, enabling standardized collection of runtime behavior for analytics.Events can record creation, updates, starts, ends, suspension, abortion, failure, and deletion; developers instrument them manually.Source: §4.3 · p. 5

4.4 Agentic System Analytics Taxonomy

  • Agentic-system analytics comprises passive, exploratory, and interventional approaches, offering different levels of insight, control, and optimization.The taxonomy includes task-flow analysis, which represents subtasks and dependencies as a directed acyclic graph with metrics attached to tasks.Source: §4.4 · p. 5
  • Exploratory optimization patterns include decomposition, parallel execution, and merging to manage quality, latency, and cost trade-offs.Decomposition can improve quality while increasing cost and latency; parallel execution can reduce latency, while merging can reduce redundant computation.Source: §4.4 · p. 6
  • Interventional analytics changes system behavior only with middleware support and explicit developer approval; just 16% of user-study participants trusted automation.Enhanced monitoring can raise performance and cost, so the paper describes activating it selectively for fault detection or audits.Source: §4.4 · p. 6

5 BENCHMARKING AGENT ANALYTICS TECHNOLOGIES

The benchmark evaluates agent analytics using trace logs and ground-truth analytics outputs, rather than task prompts alone. Its calculator case study reports incomplete issue detection, while the benchmark provides logs, task flows, metrics, and failure annotations for comparison.

Benchmark design

  • Agent analytics benchmarking uses a trace log of an agent’s task attempt as its basic unit, with ground truth describing correct analytics outputs for that log.Ground truth includes task flow, trace metrics and failure summaries, and detailed failure lists.Source: §5 · pp. 6–7
  • ABBench provides 30 structured calculator logs and corresponding ground-truth analytics outcomes, including task flows and failure lists.The logs cover numerical and natural-language inputs, distributed calculations, syntax errors, correct answers, and incorrect answers.Source: §5 · p. 7
  • Analytics outputs can be compared with ground truth using identical metric-and-failure summaries, text similarity for failure lists, and graph-edit distance for task flows.The benchmark is presented as a basis for evaluating agent analytics systems; TAMAS is the case-study system.Source: §5 · p. 7

Benchmark results

  • TAMAS correctly identified some logged issues but missed validator failures, including cases where correct actions failed validation.The five benchmark cases include both detected and undetected issues.Source: §5 · p. 8
  • 60% of benchmark cases had TAMAS summaries matching ground truth; the remaining cases exposed missed validator failures and failures on malformed initial inputs.The five-case comparison is summarized in Table 2.Source: §5 · p. 8, Table 2 · p. 8

6 DISCUSSION AND FUTURE WORK

The discussion presents AABench and the observability taxonomies as a foundation for systematic evaluation, while emphasizing that interpretability, actionability, and real-world applicability need further work. It identifies broader behavioral coverage, dynamic evaluation, and practical analytics and optimization tools as future priorities.

Discussion and future work

  • AABench is intended to evaluate agent analytics systems, but the paper says its benchmarking approach and observability taxonomies need refinement for greater interpretability and actionability.The proposed scope for future taxonomies includes context-dependent decisions, long-horizon dependencies, and multi-agent interactions.Source: §6 · p. 8
  • Dynamic evaluation scenarios are proposed to improve benchmark applicability to real-world deployments, alongside standardized criteria across domains and use cases.The paper identifies practical analytics tools and optimization techniques as work that remains.Source: §6 · p. 8
  • Future work includes automated prompt engineering to mitigate variability, causal analysis to identify bottlenecks, and targeted recommendations for performance optimization.The paper also calls for further exploration of real-time monitoring and adaptive optimization strategies.Source: §6 · p. 8

A DATASETS USED IN THE EXPERIMENT

The appendix describes datasets built around mathematical expressions and natural-language problems, and survey objectives spanning system characteristics, practitioner challenges, and tooling gaps.

A DATASETS USED IN THE EXPERIMENT

  • The experiment datasets use standard mathematical operations, while input types range from pure expressions to mixed natural-language and mathematical problems and fully natural-language problems.The mixed inputs were formed by replacing a numerical value with a natural-language problem; the pure natural-language set derives from GSM-plus [12].Source: A DATASETS USED IN THE EXPERIMENT · p. 9

B.1 Survey Objectives

  • The survey examines agentic-system management across development, testing, and deployment, including respondent backgrounds and usage patterns.These background data were intended to help assess whether perspectives differ with experience or tool use.Source: §B.1 · p. 9
  • Survey questions address perceptions of non-deterministic behavior, runtime insight needs, quality-latency-cost trade-offs, and gaps in current analytics tools.The tooling questions cover needs in development, testing, and maintenance.Source: §B.1 · p. 9

B.2 Methodology

The survey recruited 38 practitioners with varied agentic-system expertise, roles, frameworks, and tools. Its four-section questionnaire covered background, analytics needs, Likert-scale statements, and open-ended pain points.

B.2.1 Participants.

  • 38 practitioners, predominantly affiliated with research departments, were recruited across roles, use cases, expertise levels, frameworks, and observability tools.Recruitment sought input from prompt researchers, evaluation analysts, developers, engineers, and other relevant roles.Source: §B.2.1 · p. 9
  • The questionnaire used six background questions and five multiple-selection questions about work with agentic systems and analytics-tool requirements.It also included nine Likert-scale statements and two open-ended questions on pain points and general comments.Source: §B.2.1 · pp. 9–10
  • The survey was online and unmoderated, open at participants’ convenience without time limits, and conducted in January 2025.Questions were optional, and participants consented to research use of their data.Source: §B.2.1 · p. 10

B.3 Survey results

  • The results section first characterized practitioners’ experience, frameworks, tools, and expectations through questions organized around three primary areas.The supplied methodology passage introduces the landscape-characterization questions but does not specify the other two areas.Source: §B.3.1 · p. 10

B.3.1 Quantitative Data:

The survey describes experienced practitioners who use varied observation tools and report needs around complex agent characteristics, runtime behavior, and analytics. Responses also identify difficulties understanding execution and diagnosing root causes.

Practitioner profile

  • 92% of respondents identified as intermediate or expert in the Gen AI domain.The paper characterizes the cohort as highly experienced.Source: §B.3.1 · p. 10, Figure 8 · p. 10

Frameworks and tools

  • Respondents primarily used LangGraph, LangChain, and CrewAI, while reporting similar problems and missing features across varied observation tools.The paper interprets the shared tool gaps as widespread rather than limited to a specific tool.Source: §B.3.1 · p. 10, Figure 9 · p. 10, Figure 10 · p. 10

Analytics-tool objectives

  • 73% of respondents were developers, while only a few worked in testing or production maintenance.The paper describes this distribution as reflecting the domain’s early adoption phase.Source: §B.3.1 · p. 10, Figure 11 · p. 10

System characteristics and runtime aspects

  • Respondents highlighted hierarchical tasks, recursive calls, parallel flows, and long-term memory as system characteristics needing enhanced observability.LLM interactions and multi-agent coordination were among the runtime aspects drawing attention beyond traditional observability tools.Source: §B.3.1 · pp. 10–11, Figure 12 · p. 11, Figure 13 · p. 11

Analytics and issue priorities

  • Enhanced analytics and issue-identification requirements ranked among respondents’ top priorities in nearly every aggregated response.The survey asked about desired analytics types and issues to identify; the supplied passages do not provide their specific rankings.Source: §B.3.1 · p. 11, Figure 14 · p. 11, Figure 15 · p. 11

Reported challenges

  • Practitioners reported wanting to understand internal runtime behavior and finding root-cause identification challenging, alongside concerns about non-deterministic flow and tool adequacy.The supplied passages list these Likert statements but do not give their agreement percentages.Source: §B.3.1 · pp. 11–12, Figure 18 · p. 12

B.3.2 Qualitative Data:

Open-ended responses grouped practitioners’ concerns into themes spanning execution understanding, trust, instruction-following, optimization, and tooling choices. The accounts describe both the difficulty of interpreting agent behavior and the challenges of developing dependable workflows.

Debugging, Execution Understanding, and Observability Gaps

  • Practitioners described difficulty understanding execution behavior, debugging failures, finding patterns across runs, and extracting insights from extensive logs.They also reported that existing observability tools lack support for agentic systems.Source: §B.3.2 · p. 12

Trust, Transparency, and Governance

  • Trust and transparency concerns included making system behavior clear to users and developers while maintaining reliability in production.Respondents also wanted confidence that outputs were accurate and up to date.Source: §B.3.2 · p. 12

Prompt Adherence, Response Control, and LLM Behavior

  • Practitioners reported difficulty ensuring prompt adherence, predictable tool use, and consistent structured outputs.One response described format variation despite requesting JSON and providing few-shot prompts.Source: §B.3.2 · p. 12

Optimization of Multi-Agent and Complex Systems

  • Practitioners found multi-agent workflow optimization difficult and wanted insight into how inputs affect task or node outcomes and how planning structures should evolve.They highlighted the importance of producing high-quality, semi-deterministic responses.Source: §B.3.2 · p. 12

Rapidly Evolving Tech Landscape and Tooling Choices

  • Rapid field evolution made framework and technology-stack choices difficult, including uncertainty about which tools would remain sustainable.Source: §B.3.2 · p. 12

B.4 Two-dimensional Analysis

The authors analyzed 65 survey features using Spearman rank correlations and controlled false discovery rate for significance testing. Reported significant relationships span tool use, runtime interests, system characteristics, and human-in-the-loop analysis.

Feature construction and correlation method

  • The analysis encoded 65 features across experience, tool use, Likert responses, and multiple-choice requirements, then computed Spearman rank correlations.Missing Likert responses were replaced with Neutral before analysis.Source: §B.4 · pp. 12–13

Significance testing

  • Benjamini–Hochberg correction controlled false discovery rate at α=0.1 because the correlation matrix contained more than 2,000 entries.The authors report significant correlations after applying this procedure.Source: §B.4 · p. 13

Significant relationships

  • Significant pairs included joint use of Instana, Traceloop, and LangTrace; interest in LLM interactions and tool calls; and associations involving system distribution, dynamic structure, and human-in-the-loop analysis.The authors also report a relation between Bee Agent Framework users and interest in optimization and balancing system characteristics; LangFuse users showed interest in human-in-the-loop runtime analysis.Source: Table 3 · p. 13, §B.4 · p. 13

B.5 Cluster Analysis

K-means separated respondents into two groups with similar experience and primary tool-use objectives but distinct patterns of tool use, analytics priorities, and system requirements.

B.5 Cluster Analysis

  • Cluster 2 selected more tools and requirements; its >0.3 binary-feature differences favored LlamaIndex and OpenAI/Antropic use.Source: §B.5 · p. 13
  • The analysis used 65 features and selected K=2 using WCSS, yielding clusters of 27 and 11 participants without significant differences in experience or primary tool-use objective.Source: §B.5 · p. 13
  • Cluster 2 prioritized agent optimization and identifying cycles or deviations, alongside parallel, recursive, dynamic systems with log-term memory.Source: §B.5 · p. 14
  • Cluster 2 favored prompt optimization, task/agent decomposition, and human-as-judge interactions; no >0.3 binary feature favored Cluster 1.Source: §B.5 · p. 14
  • Cluster 2 more strongly wanted runtime insight and restructuring help, and reported terminology friction; the authors describe its requirements as more diversified and advanced.The passage connects this profile with work across diverse applications and models.Source: §B.5 · p. 14
Loading 2503.06745v1…