Source-linked AI summary
Graphectory Viewer: A Tool for Process-Centric Analysis of Agentic Software Trajectories
Charlie Jyu, Shuyang Liu, Reyhaneh Jabbarvand
TL;DR
Existing trajectory viewers offer limited support for connecting low-level execution steps with higher-level problem-solving phases and comparing runs. Graphectory Viewer addresses this gap with interactive phase-aware graphs and aggregate phase-transition summaries, increasing task accuracy from 8% to 84% in a preliminary study of five trajectory-forensics tasks.
Problem
Existing trajectory viewers provide limited support for higher-level analysis and comparison across executions because they present trajectories mainly as raw logs or step-by-step transcripts.
Method
Graphectory Viewer normalizes heterogeneous trajectories, assigns context-sensitive phase labels, and constructs interactive phase-aware graphs with corpus-level phase-transition summaries.
Results
Aggregate task accuracy increased from 8% to 84% versus the command-line viewer in a preliminary within-subject study of five participants across five trajectory-forensics tasks.
Takeaways & Limitations
The phase-aware representation made recurring localization and validation loops and mismatches between thought length and reasoning complexity more visible across trajectory collections.
Takeaways & Limitations
Generalizability is limited by the small convenience sample, fixed interface order, and task design targeting queries that may favor visual search.
Abstract
from arXiv · showhide
We present Graphectory Viewer, a web-based tool for interactive, process-centric analysis of software-agent trajectories. Building on the Graphectory representation introduced in our previous work, Graphectory Viewer transforms heterogeneous raw trajectories into phase-aware graphs that connect low-level execution details with higher-level behavioral structures. The tool supports trajectories from multiple agent frameworks and provides interactive graph construction; node-level inspection of thoughts, actions, and observations; search and filtering over large trajectory collections; and Sankey-style summaries of problem-solving phase transitions. These capabilities enable researchers and practitioners to inspect individual executions, identify recurring behavioral patterns, compare successful and failed runs, and analyze large trajectory corpora beyond final task outcomes. To support reproducibility and further research, we release Graphectory Viewer as an open-source artifact together with documentation, precomputed graphs, and the large-scale trajectory corpus.
1 Introduction
Graphectory Viewer addresses the limits of raw trajectory inspection by turning agent executions into interactive, phase-aware graphs that connect low-level evidence with higher-level behavioral structures. The paper also reports a preliminary within-subject evaluation against the SWE-agent command-line viewer.
- Motivation: Existing trajectory inspection tools mainly expose raw logs or step-by-step transcripts, limiting higher-level analysis of localization, exploration, validation, and strategy differences.The motivating questions include whether agents localized bugs before editing, revisited files, validated patches, and differed across successful and failed runs.
- Prior work: Graphectory previously mapped low-level agent actions into semantic phases and graph structures but primarily supported offline analysis through custom scripts.The prior framework offered limited interactive exploration of graph structures.
- Tool contribution: Graphectory Viewer normalizes framework-specific actions, assigns context-sensitive phase labels, and constructs interactive phase-aware directed graphs for process-centric trajectory analysis.Users can inspect node-associated thoughts, actions, and observations and connect low-level execution evidence to higher-level behavioral structures.
- Evaluation: The preliminary evaluation uses a within-subject study with five participants, comparing Graphectory Viewer against the SWE-agent command-line viewer on five trajectory-forensics tasks.The supplied contribution statement reports an increase beginning with “increased aggreg,” but the passage is truncated before the result is specified.
2 Related Work
Prior work evaluates agent behavior through complementary metrics and protocols and provides tools for trajectory inspection, diagnostics, debugging, and steering. Graphectory Viewer extends this landscape by representing heterogeneous software-agent trajectories graphically for process-centric analysis and corpus-level phase-transition comparison.
- Software Agents: Recent agent-evaluation frameworks assess instruction following, alignment with intended goals, and reward design alongside task-level success.These approaches provide metrics and protocols for evaluating agent capabilities.
- Agent Trajectory Visualization: Framework-specific viewers and tools such as SWE-agent Trajectory Inspector, mini-SWE-agent viewer, and SeaView primarily replay and inspect individual executions as formatted text.AgentDiagnose instead provides general-purpose trajectory diagnostics through execution traces and embedding-based visualizations.
- Agent Trajectory Visualization: AGDebugger and related systems support visual analysis, debugging, and steering of LLM-based agent systems.These tools address analysis and debugging needs for agent trajectories and systems.
- Agent Trajectory Visualization: Graphectory Viewer maps heterogeneous software-agent trajectories into a graphical, process-centric representation and supports corpus-level comparison through aggregate phase-transition summaries.This contrasts with tools focused primarily on replay, inspection, diagnostics, debugging, or steering.
3 Tool Overview
Graphectory Viewer provides a browser interface for exploring agent trajectories as phase-aware, process-centric graphs. It combines searchable instance navigation, detailed trajectory inspection, file-footprint views, and configurable phase-transition summaries.
- Browser interface: The browser interface accepts trajectories and reports, then populates a searchable instance list with status badges and trajectory counts.Users begin in the Data source panel before selecting an instance for visualization.
- Trajectory inspection: Selecting an instance loads its Graphectory into the main canvas for trajectory-level analysis.The illustrated SWE-agent run connects execution steps to localization activity across the graph.
- Trajectory inspection: Clicking a node reveals its raw thought-action-observation tuple, while tabs expose each step when multiple steps reference the node.A file footprint menu visualizes files viewed and edited during the trajectory, including when each file was interacted with.
- Phase summaries: The Phase Sankey Diagram summarizes runs as meaningful phase transitions with status filters, transition-depth controls, minimum-flow suppression, and phase-category toggles.These controls support focused views of selected trajectory flows and phase categories.
- Backend processing: The backend converts trajectory directories and OpenHands-style output.jsonl files into a shared process-centric graph representation using Bash-aware command normalization and context-sensitive phase labeling.The pipeline maps commands and framework-specific tool calls into structured records, then assigns localization, patch, validation, or general labels.
4 User Evaluation
A preliminary within-subject study compared Graphectory Viewer with a command-line viewer on five trajectory-forensics tasks. Graphectory achieved substantially higher task accuracy and was preferred by participants, while the small, convenience-sampled, fixed-order study limits generalizability.
- Study Design: Five participants with varied software-agent experience used both viewers to complete the same five trajectory-forensics tasks.Less experienced participants received a brief introduction to software agents, SWE-agent trajectories, and viewer fields.
- Study Design: The tasks targeted repeated commands, thought continuation, command loops, unusually long thoughts, and unusually long observations.Specific thresholds included 20 repetitions, three repetitions per loop, 700-character thoughts, and 20,000-character observations.
- Results: 84% correct answers were achieved with Graphectory versus 8% with the command-line viewer across 25 attempts.Three participants completed all five tasks with Graphectory in 5:13.80, 7:10, and 7:58; no participant completed all five within 10 minutes using the command-line viewer.
- Results: Participants rated Graphectory 8–9 out of 10 and consistently preferred its visual presentation, citing phase colors, node and arrow encodings, raw-step inspection, and the Sankey view.Suggested improvements included a legend, more accurate arrow scaling, and a tutorial, which were subsequently added.
- Limitations: The findings provide preliminary evidence of improved performance, but generalizability is limited by the small convenience sample, fixed interface order, and visually targeted task design.The authors caution that the tasks may favor visual search.
5 Utility and Quality
Graphectory Viewer compresses large, heterogeneous agent trajectories into compact phase-aware graphs while preserving behavioral distinctions across agents and models. At corpus scale, it exposes recurring localization and validation loops and shows that thought length alone does not reliably indicate reasoning complexity.
- Corpus-scale compression: The released corpus contains 3,973 trajectories across eight collections, with raw trajectories averaging 35,664 lines and 3,047,249 characters versus Graphectories averaging 34.53 nodes and 49.60 edges.The corpus covers four SWE-agent and four OpenHands runs over SWE-bench Verified tasks; 27 of 4,000 runs produced empty trajectories.
- Corpus-scale compression: Compression remains consistent across agents and models while revealing behavioral differences, including especially compact SWE-agent graphs for DeepSeek-V3 and much larger logs for Claude Sonnet 4.DeepSeek-V3 averages 14.99 nodes and a shortest execution path of 7.50 from 15,895-line trajectories, whereas Claude Sonnet 4 averages 144,587 lines and over 10 million characters.
- Behavioral patterns: Across large trajectory collections, agents frequently enter repeated localization or validation loops, revisiting actions without advancing to new phases or making meaningful progress.The process-centric representation also indicates that thought length alone is an unreliable proxy for reasoning complexity.
6 Conclusion
Graphectory Viewer transforms heterogeneous software-agent trajectories into phase-aware graphs and aggregate transition summaries for process diagnosis and strategy analysis. Releasing its implementation, documentation, and precomputed corpus supports reproducible analysis and debugging of complex agent executions.
- Graphectory Viewer transforms heterogeneous software-agent trajectories into phase-aware graphs and aggregate transition summaries.The system provides an interactive representation for analyzing agent executions beyond raw logs.
- Preliminary evaluation suggests the viewer can support process diagnosis over raw-log inspection in the studied tasks.The conclusion characterizes this evidence as preliminary and task-specific.
- The corpus analysis illustrates how phase-aware representations can expose agent problem-solving strategies.
- Releasing the implementation, documentation, and precomputed corpus aims to support reproducible analysis and debugging of increasingly complex software-agent executions.
7 Data Availability Statement
Graphectory Viewer and its code are publicly available, alongside an archived trajectory corpus, documentation, scripts, and a live demo to support access and reproducibility.
- The code and viewer are publicly available under the University of Illinois/NCSA Open Source License.
- The raw trajectory corpus is archived in Zenodo under DOI 10.5281/zenodo.17364210.
- The repository includes installation instructions, usage examples, sample inputs, graph-export and trajectory-metric scripts, a screencast, and a live demo.The live demo is available at https://graphectory-viewerdemo.vercel.app/.