PaperlayerView exact revision on arXiv

Source-linked AI summary

TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces

Dehai Min, Daoan Zhang, Yiming Zeng, Huayi Zhang, Ziyi Chen, Yan Zhang, Qinbo Bai, Mengyuan Chao, Jing Ning, Qiyue Hua, Huiyi Chen, Hanrong Zhang, Henry Peng Zou, Jie Yang, Wei Xu, Philip S. Yu

arXiv:2609.33295v1cs.AIcs.CL

TL;DR

Fixed suites target predefined behaviors, while deployment traces capture problems that auditing tools do not convert into on-demand benchmarks. TraceDance builds behavior-specific tests from traces by grading an evaluated model’s next turn at a recorded decision point. It fulfills 95.3% of build-target requests, and evaluation of frontier LLMs reveals weaknesses in appropriate behavior at those points.

  • Problem

    Fixed benchmark suites generally test predefined behaviors, and trace-auditing tools do not convert deployment records into on-demand benchmarks for newly encountered problems.

    Source: §1 · p. 2

  • Method

    TraceDance retrieves and confirms trace occurrences, then evaluates an LLM’s next turn from a recorded pre-decision context using a behavior-specific rubric without replay or a reference answer.

    Source: §1 · p. 2, §3.1 · pp. 3–4

  • Results

    95.3% of build-target requests were fulfilled, and evaluation of nine frontier LLMs showed that appropriate behavior at decision points remains challenging.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Behavior-specific results reveal weaknesses obscured by overall rankings and provide concrete targets for agent improvement.

    Source: §1 · p. 2, §6 · p. 9

    TraceDance targets behaviors with observable signals for programmable retrieval; behaviors requiring inspection of every session fall outside its deployment-scale scope.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

An agent can complete a task while exhibiting undesirable behavior during execution. Developers need tests for the specific behaviors encountered in deployment, beyond fixed benchmark suites. We present TraceDance, an agent system that constructs targeted benchmarks from deployment traces for user-specified undesirable behaviors. For efficient construction, Anchor-and-Confirm combines programmable retrieval with candidate-level confirmation by a Flash large language model (LLM), while the Anchor Synthesis Loop generates and revises specifications for custom behaviors. The benchmarks use decision-point continuation to evaluate an LLM's next turn at a recorded decision point with a behavior-specific rubric, without a reference answer or environment replay. Experiments in coding and general tool use draw on 252,557 sessions and produce 107 benchmarks with 4,125 instances, fulfilling 95.3% of build-target requests. Both human annotators confirm the requested behavior in 84% of sampled instances, and the automated grader's agreement with human pass/fail judgments is comparable to that between the annotators. Nine frontier LLMs achieve a mean pass rate of only 26.7%, showing that they still struggle to respond appropriately at the evaluated decision points. Analysis across behavior-specific benchmarks further reveals weaknesses in how current LLMs behave as agents. By turning deployment problems into targeted benchmarks, TraceDance could serve as a key component of the recursive self-improvement (RSI) loop.

1 Introduction

Task completion can miss harmful execution behavior, while fixed suites and auditing tools do not provide on-demand benchmarks for newly encountered deployment problems. TraceDance constructs behavior-specific tests from deployment traces, and its evaluation finds both high build-request fulfillment and persistent weaknesses in frontier LLM behavior.

1 Introduction

  • Fixed benchmarks generally target predefined behaviors, while auditing tools identify trace problems but do not turn them into on-demand benchmarks.Task completion alone can miss behaviors such as leaking access tokens, disabling failing tests, or running destructive commands without confirmation.Source: §1 · pp. 1–2
  • Decision-point continuation grades an evaluated LLM’s next turn from the recorded context before an observed behavior-critical turn, using a behavior-specific rubric.It requires neither a reference answer nor environment replay, allowing evaluation on traces involving non-public tools or MCP servers.Source: §1 · p. 2
  • Anchor-and-Confirm uses programmable trace retrieval followed by Flash LLM confirmation, while the Anchor Synthesis Loop creates and revises anchors for unmatched queries.Manually reviewed anchors cover predefined behavior families.Source: §1 · p. 2
  • 95.3% of build-target requests were fulfilled across 252,557 sessions, producing 107 benchmarks with 4,125 instances.The evaluation covered coding and general tool use.Source: §1 · p. 2
  • 22.9%–33.5% was the overall pass-rate range across nine frontier LLMs; mean pass rates were 67.9% for well-formed tool calls and 8.1% when checks were required.The checks include inspecting an error log after a failed background job.Source: §1 · p. 2

2 Related Work

Related work addresses task outcomes, execution behavior, trace auditing, and automated benchmark construction through distinct approaches. TraceDance connects trace analysis to reusable tests evaluated from recorded pre-decision contexts.

2 Related Work

  • Outcome benchmarks test task completion, while process and safety suites examine execution behavior; TraceDance grades an LLM’s next turn before an observed undesirable behavior.Prior continuation approaches use interaction histories [21] [26], synthesized decision snapshots [67], or saved execution states [36].Source: §2 · p. 2
  • The workflow selects or synthesizes behavior specifications, retrieves and confirms occurrences, then constructs context instances with behavior-specific rubrics.The figure depicts the construction pipeline for user-specified behaviors.Source: Figure 1 · p. 3
  • Agent auditing finds failures or violations in traces, while benchmark construction generates questions, interactions, executable scenarios, or reconstructed tasks.TraceDance connects these lines by turning observed behaviors into reusable tests with recorded pre-decision contexts as inputs.Source: §2 · p. 3

3 TraceDance

TraceDance turns user-specified undesirable behaviors in deployment traces into decision-point continuation benchmarks, using behavior-specific rubrics to grade an evaluated LLM’s next turn. It reuses reviewed specifications or synthesizes custom ones, then retrieves and confirms occurrences before constructing instances.

3.1 Problem Formulation

  • Users provide a natural-language behavior query, deployment traces, and a benchmark-size range; the system returns a benchmark within range or a rejection reason.Instances are retained only when the source turn exhibits the behavior, the context supports choosing a response, and the decision remains open.Source: §3.1 · pp. 3–4
  • Action, failure, and claim frames cut traces differently, preserving decision-relevant context while withholding the source LLM’s decision for next-turn rubric grading.The evaluated model produces one next assistant turn; its actions are not executed and the trace is not continued.Source: §3.1 · pp. 3–4
  • TraceDance targets behaviors with observable signals for programmable retrieval; behaviors requiring per-session LLM or human review fall outside scope because that cost is prohibitive at deployment scale.Source: §3.1 · p. 4

3.2 Selecting and Synthesizing Behavior Specifications

  • Specifications combine an executable retrieval anchor, confirmation criterion, frame-specific cut contract, and behavior-specific rubric; known behaviors reuse reviewed entries while custom ones undergo iterative validation.The Anchor Synthesis Loop checks code safety and runtime behavior, probes candidates, and revises anchors using confirmation rates and examples.Source: §3.2 · p. 4

3.3 Occurrence Retrieval and Instance Construction

  • Anchor-and-Confirm scans traces with executable anchors on CPUs, then applies Fast Model confirmation to candidates before constructing instances from confirmed occurrences.Instances retain recorded context but remove source-LLM reasoning; construction stops at Kmax or when candidates are exhausted.Source: §3.3 · pp. 4–5

3.4 Rubric-Based Grading

  • A three-LLM judge panel scores each next turn against a behavior-specific 0–5 rubric; responses pass when the mean score is at least 4.Lower scores correspond to responses exhibiting the requested behavior, while higher scores indicate appropriate responses.Source: §3.4 · p. 5

4 Deployment-Trace Analysis and Predefined Behaviors

The deployment-trace analysis covers coding and general tool use across 252,557 sessions collected over six weeks, with separate sessions reserved for behavior discovery. The authors also construct a catalog of 28 predefined behavior families from recurring, observable findings.

4.1 Two Deployment Settings: Coding and General Tool Use

  • The analysis covers 252,557 sessions collected over six weeks from Claude Code for coding and OpenClaw for general tool use.The sessions were de-identified and sanitized before research use.Source: §4.1 · p. 5
  • Figure 3 organizes system validation and construction workload into expected build/reject outcomes, per-query workload, and human review.Its panels describe mean scans, candidates reviewed, and instances constructed, alongside validation of instances, rubrics, and automated grading.Source: Figure 3 · p. 6
  • The authors reserve 10,000 sessions per setting for trace analysis and discovery, excluding them from benchmark construction.Source: §4.1 · p. 6

4.2 Predefined Behavior Discovery

  • For predefined behavior discovery, the authors extract behaviors and evidence, group equivalent findings, then manually select recurring families with observable trace events.Source: §4.2 · p. 6
  • The resulting catalog evaluation retains 28 behavior families: 13 action-frame, 10 failure-frame, and 5 claim-frame behaviors.Examples include Hallucinated command, Unchanged retries after failure, and Test-pass claim without a test run.Source: §4.2 · p. 6

5 Experiments

The experiments evaluate TraceDance’s query handling and benchmark validity, then use its constructed benchmarks to expose behavior-specific weaknesses in frontier LLMs. Construction largely succeeds, human review supports the instances and rubrics, and the behavior-focused results add a perspective beyond task completion.

5.1 Evaluation Setup

  • The evaluation uses 139 test queries, including 107 expected benchmark builds and 32 expected rejections, and evaluates nine LLMs on the same instance contexts.Two of eight annotators independently assessed each of 100 behavior queries and 100 constructed instances.Source: §5.1 · pp. 6–7

5.2 Does TraceDance Handle Queries as Expected?

  • TraceDance builds benchmarks for 102 of 107 build-target queries (95.3%) and rejects all 32 rejection-target queries, producing 107 benchmarks with 4,125 instances.Annotators judged 78% of 100 query interpretations accurate and 17% partially correct.Source: §5.2 · p. 7
  • Programmable anchors scan an average of 128,297 sessions per successful query without LLM calls, then pass 706 candidates to the Fast Model for confirmation.Successful queries use a mean of 552 LLM calls across construction stages and yield an average of 40.4 instances.Source: §5.2 · p. 7

5.3 Are the Constructed Benchmarks Valid?

  • Both annotators confirm the requested behavior in 84 of 100 sampled instances, and judge-panel pass/fail agreement with annotators is 81.0%.Both annotators rate rubric quality at least 4/5 in 90% of instances, with a mean rating of 4.75/5.Source: §5.3 · p. 7

5.4 What Do the Benchmarks Reveal?

  • GPT-5.6-Sol does not outperform DeepSeek-V4-Pro or GLM-5.2 on TraceDance’s behavior-focused benchmarks despite higher reported task-oriented benchmark scores.The comparison illustrates that task completion scores and behavior-focused results can differ.Source: §5.4 · p. 7, Table 2 · p. 8
  • Nine LLMs average a 67.9% pass rate for Valid call but only 8.1% for Check first, with Secret protection at 6.9%, Commit hygiene at 0.9%, and Failure-log inspection at 0.6%.The results indicate that models more often make well-formed tool calls than perform required checks; a task can still complete while exposing credentials or including unwanted files in a commit.Source: §5.4 · p. 8, Figure 4 · p. 8, Figure 5 · p. 9
  • Responses beginning with TodoWrite pass at 4.6%, versus 30.8% for other first actions on the same instances, a pattern reported for all nine LLMs.Planning-related tools and skill invocation are associated with lower pass rates, while direct actions and information requests are associated with higher rates; these comparisons are not causal.Source: Figure 4 · p. 8, §5.4 · p. 9
  • Claude Opus 4.8 ranks first overall, yet Kimi-K3 leads it on Error-guided correction at 57.8% versus 27.8%, while GPT-5.6-Sol leads in 8 of 28 behavior families.GPT-5.6-Sol leads on Test-claim grounding at 60.2%, versus at most 39.5% for other LLMs.Source: Figure 4 · p. 8, §5.4 · p. 9, Figure 5 · p. 9

6 Conclusion

TraceDance turns user-specified deployment problems into decision-point continuation benchmarks, and the paper concludes that these benchmarks could support agent improvement and regression testing. The conclusion also notes that evaluation within a recursive self-improvement loop remains future work.

6 Conclusion

  • TraceDance converts deployment problems into decision-point continuation benchmarks, and evaluation of nine frontier LLMs finds appropriate behavior at those decision points challenging.The authors say behavior-specific results expose weaknesses hidden by overall rankings and could provide concrete targets for agent improvement.Source: §6 · p. 9
  • The authors leave evaluation of TraceDance within a recursive self-improvement loop to future work.They describe the system as a potential evaluation component in an agent data flywheel that could guide improvements, evaluate progress, and detect regressions.Source: §6 · pp. 9–10

AI use statement

  • The paper reports using generative AI for writing assistance and literature discovery, and LLMs within TraceDance for behavior discovery, specification synthesis, candidate confirmation, rubric generation, and grading.LLMs also drafted test queries and generated and named session-description clusters; the authors say they reviewed and revised AI-assisted code and queries.Source: AI use statement · p. 10

Ethics statement

  • The eight annotators were compensated under local labor regulations, and production traces were de-identified and sanitized before research use.The traces were collected under the respective products’ terms of service and are not released.Source: Ethics statement · p. 10

Reproducibility statement

  • The exact benchmarks cannot be rebuilt from public data because the deployment traces are not released, though the pipeline can be applied to other trace collections.The reproducibility statement points to the pipeline description, prompts, settings, annotation protocol, and robustness analyses.Source: Reproducibility statement · p. 10

A.1 Frame-Specific Cuts and Query Handling

  • TraceDance’s AND and OR query operators let users test joint requirements or evaluate behaviors separately, while requests involving three or more behaviors are rejected.Queries requiring a behavior parameter prompt clarification when neither the query nor system defaults supply a value.Source: §A.1 · p. 16

A.2 Benchmark Construction Details

Benchmark construction combines model-assisted catalog and domain checks with reviewed custom anchors. The appendix also documents prompt templates and the distinction between user-authored and harness-injected session inputs.

A.2 Benchmark Construction Details

  • For catalog matching, the Strong Model makes three family selections and reuses a catalog specification only when all three equivalence checks agree; otherwise, TraceDance synthesizes a custom specification.Domain-restricted candidates pass a domain check before behavior confirmation, and a Reviewer Model assesses query fit, examples, and rubric precision.Source: §A.2 · p. 16, Table 4 · p. 16
  • Model calls use high reasoning effort wherever supported, with reasoning disabled for session summaries and domain checks; outputs are capped at 32,768 tokens.Evaluated LLMs receive the recorded system prompt and tool definitions without modification through each provider’s standard tool-calling API.Source: §A.2 · p. 16
  • A custom anchor for dependency-manifest behavior was revised to test the decision-point state rather than require a later edit, increasing matched sessions from 3 to 32.Generated anchor code must pass programmatic safety checks and code review, then runtime and cut-position checks.Source: §A.2 · pp. 16–17

A.3 Selected Prompt Templates

  • The appendix presents condensed prompt templates for catalog matching, candidate confirmation, and response grading, with examples and detailed output schemas omitted.For response-order bias control, responses are randomized per instance and the same order is used across the three judges.Source: §A.3 · pp. 17–18, Table 5 · p. 17, Table 6 · p. 18

B.1 Session Statistics and Input Sources

  • Messages marked with the user role can be harness-generated inputs; user-authored turns account for 74.8% of Claude Code inputs and 30.8% of OpenClaw inputs.The paper distinguishes input source from message role.Source: §B.1 · p. 19, Table 8 · p. 19

B.2 Domain and Task Clustering

The appendix describes LLM-assisted clustering of session domains and task types, a catalog of predefined behavior families, and a test-query set designed to probe varied requests and rejection cases.

B.2 Domain and Task Clustering

  • Domain and task clustering uses LLM-generated descriptions from 10,000 sessions per setting, separately clustered with k-means at k = 20 and reviewed for coherence.The Strong Model names the clusters after the Fast Model generates short descriptions and the Seed Embedding Model embeds them.Source: §B.2 · p. 19

B.3 Predefined Behavior Catalog and Naming

  • The catalog retains 28 predefined behavior families, and its family-level results use 87 single-behavior benchmarks with pass rates based on mean judge scores of at least 4.Those pass rates measure appropriate responses, not the frequency of undesirable behaviors.Source: §B.3 · p. 19, Table 9 · p. 19

C.1 Test Query Construction

  • Test queries cover catalog and custom behaviors, single and two-behavior AND/OR requests, domain or trace-property constraints, and varied wording; rejection cases include missing parameters and unsupported requests.The Strong Model drafts queries to meet coverage requirements, and authors manually review the drafts.Source: §C.1 · p. 21, Table 10 · p. 21, Table 11 · p. 21

C.2 Human Annotation and Grading Reliability

The judge panel’s pass/fail decisions align with human ratings at least as well as annotators align with each other, though judges score more leniently. Model rankings and behavior-group comparisons remain stable in the reported robustness checks, with analysis subsets varying by question.

C.2 Human Annotation and Grading Reliability

  • 81.0% pass/fail agreement with humans and Cohen’s κ of 0.40 for the judge panel compare with 0.31 between annotators.The panel recovers 60.7% of human passes, while 44.7% of its passes are also human passes; judges’ mean scores exceed the human mean.Source: §C.2 · pp. 22–23, Table 14 · p. 23
  • GPT-5.6-Sol shows a 0.22-point relative self-preference, while Claude Opus 4.8 shows no clear self-preference; removing either model’s judge leaves its rank unchanged.The comparison uses 3,945 responses from each judge’s own model and 31,560 from other models.Source: §C.2 · p. 23, Table 15 · p. 23, Table 16 · p. 24
  • Across alternative passing rules, Valid call remains highest and Check first lowest, with a gap of 52.4 to 64.4 percentage points.Counting a mean score of at least 3 as passing raises Check first from 8.1% to 26.8%, without changing group order or Claude Opus 4.8’s first-place rank.Source: §C.2 · p. 24, Table 17 · p. 24
  • Single judges also rank Valid call highest and Check first lowest, with gaps of 49.3 to 65.0 percentage points.GPT and Opus preserve the panel’s full group order; Gemini-3.5-Flash places Honest claim slightly above Handle failure (53.8% vs. 51.5%).Source: §C.2 · p. 24, Figure 7 · p. 24

C.3 Uncertainty in Evaluated Model Rankings

  • Claude Opus 4.8 ranks first in 99.9% of benchmark resamples and leads the second LLM by 4.2 percentage points (95% CI [1.7, 6.6]).The nine models fall into three groups; differences between adjacent models within each group are not significant, and resampling instances as well leaves group differences significant.Source: §C.3 · p. 25, Table 18 · p. 25

C.4 Source-Model Family and Evaluated LLMs

  • Doubao-Seed-2.1-Pro shows no family-specific disadvantage; all nine models’ confidence intervals include zero, ruling out disadvantages larger than about 3 percentage points.The conclusion is unchanged when Doubao-sourced instances are limited to those from Doubao Seed 2.0 models.Source: §C.4 · p. 25, Table 19 · p. 26

C.5 Instances Used in Each Analysis

  • Analysis subsets differ by question: cross-model comparisons require valid responses from all nine models, while behavior-group and family analyses apply additional exclusions.Behavior-group analyses exclude five AND benchmarks; family-level results also exclude benchmarks built from query-specific specifications.Source: §C.5 · p. 26, Table 20 · p. 27

D Benchmark Construction Results

TraceDance builds benchmarks through catalog or query-specific specifications, with most requested construction outcomes achieved. The section also tests behavior-group robustness and reports low pass rates associated with planning-related first actions.

D Benchmark Construction Results

  • Seven predefined-behavior queries used synthesized specifications, while eight built custom-behavior queries yielded 15 benchmarks with query-specific specifications.Of 94 built predefined-behavior queries, 87 used catalog specifications; the seven synthesized cases arose from nonmatching specifications, related-only families, or a trace condition and long context.Source: D Benchmark Construction Results · p. 27, Table 21 · p. 28, Table 22 · p. 28
  • The median benchmark contains 49 instances, and 53 of 107 benchmarks reach the requested maximum of 50.Construction-workload statistics cover 134 queries with complete logs, including construction rounds and repeated anchor scans but excluding later model evaluation and grading.Source: D Benchmark Construction Results · p. 28, Table 23 · p. 28

E.1 Behavior Groups and First-Action Comparisons

  • Across four alternative groupings, Check first pass rates remain between 7.7% and 9.4%, about 60 percentage points below Valid call.The variants move Delivery recovery into Check first and repeat both groupings using only predefined specifications.Source: §E.1 · p. 29, Table 25 · p. 29
  • Of 457 TodoWrite-first responses, 416 (91.0%) contain no other tool call; the 41 with a later non-planning call pass at 7.3%.First-action comparisons use equal response weights, and differences describe associations rather than causal effects.Source: §E.1 · pp. 29–30, Table 26 · p. 30

E.2 A Real Example of Benchmark Construction and Evaluation

The worked Error-guided correction instance illustrates how an anchor narrows trace retrieval and a Flash Model confirms the behavior. At the cut point, four of nine evaluated LLMs pass by applying a fix named in the error.

E.2 A Real Example of Benchmark Construction and Evaluation

  • Evaluation stops before the source LLM’s next response and grades the evaluated model’s single next turn at that recorded cut point.The illustrated context and cut point are recorded in Table 27; Table 28’s anchor match and confirmation records are construction-only, not evaluation input.Source: §E.2 · pp. 30–31, Table 27 · p. 31, Table 28 · p. 32
  • The anchor flags a retry lacking the required parameter, then the Fast Model confirms the case because the source LLM adopted none of the error’s stated fixes.The anchor uses text matching to narrow the search; the Fast Model reviews a condensed trace and confirms on different grounds.Source: §E.2 · p. 32, Table 28 · p. 32
  • A response passes only when the mean score across the three judges is at least 4.The rubric’s 0-point and 5-point anchors distinguish retrying without the required fix from implementing every explicit fix and prohibition precisely.Source: §E.2 · p. 33, Table 30 · p. 33
  • Four of nine evaluated LLMs pass the instance by applying one of the fixes named in the error; Kimi-K3 scores 4.33 and Claude Opus 4.8 scores 2.00.Kimi-K3 passes and Claude Opus 4.8 does not under the judge panel.Source: §E.2 · p. 33, Table 31 · p. 33

F Limitations

TraceDance measures responses at recorded decision points rather than full executions, so its results complement rather than replace task-level evaluation. Its pass rates describe behavior at contexts where the requested behavior has already occurred, not how often it occurs in deployment.

F Limitations

  • Decision-point continuation cannot observe what happens after the graded turn, so a passing response may still fail when executed.A response that begins by planning or gathering information may act appropriately later; the evaluation uses the recorded context without replaying the environment.Source: F Limitations · p. 34
  • Because each instance comes from a context where the source LLM exhibited the behavior, pass rates do not measure its frequency in deployment.The rubrics credit only responses meeting the full rubric, and automated construction and grading are imperfect; the judge panel also scores more leniently than humans.Source: F Limitations · p. 34

Related papers

Loading 2609.33295v1…