Source-linked AI summary
Signals: Trajectory Sampling and Triage for Agentic Interactions
Shuguang Chen, Adil Hafeez, Salman Paracha
TL;DR
Post-deployment improvement is difficult because agent trajectories are numerous, non-deterministic, and expensive to review, while existing evaluation methods do not scale to live interactions. The paper introduces lightweight, model-free signals for triaging trajectories across interaction, execution, and environment behaviors. On τ-bench, signal-based sampling reaches 82% informativeness versus 74% for heuristic sampling and 54% for random sampling, with a 1.52× efficiency gain.
Problem
Agent trajectories are voluminous and non-deterministic, while manual or auxiliary-LLM review is too costly for scalable post-deployment improvement.
Method
The paper computes lightweight signals without model calls and organizes them into interaction, execution, and environment categories for trajectory triage.
Results
82% informativeness and a 1.52× efficiency gain are achieved by signal-based sampling, compared with 74% for heuristic sampling and 54% for random sampling.
Takeaways & Limitations
Signal-based sampling provides practical infrastructure for trajectory triage and can support preference-data construction by selecting failures and successful exemplars.
Takeaways & Limitations
The taxonomy does not assess semantic correctness or domain-specific policy violations, so fluent but factually wrong trajectories may not be surfaced.
Abstract
from arXiv · showhide
Agentic applications based on large language models increasingly rely on multi-step interaction loops involving planning, action execution, and environment feedback. While such systems are now deployed at scale, improving them post-deployment remains challenging. Agent trajectories are voluminous and non-deterministic, and reviewing each one, whether through human review or auxiliary LLMs, is slow and cost-prohibitive. We propose a lightweight, signal-based framework for triaging agentic interaction trajectories. Our approach computes cheap, broadly applicable signals from live interactions and attaches them as structured attributes for trajectory triage, identifying interactions likely to be informative without affecting online agent behavior. We organize signals into a coarse-grained taxonomy spanning interaction (misalignment, stagnation, disengagement, satisfaction), execution (failure, loop), and environment (exhaustion), designed for computation without model calls. In a controlled annotation study on $τ$-bench, a widely used benchmark for tool-augmented agent evaluation, we show that signal-based sampling achieves an 82\% informativeness rate compared to 74\% for heuristic filtering and 54\% for random sampling, with a 1.52x efficiency gain per informative trajectory. The advantage is robust across reward strata and task domains, confirming that signals provide genuine per-trajectory informativeness gains rather than merely oversampling obvious failures. These results show that lightweight signals can serve as practical sampling infrastructure for agentic systems, and suggest a path toward preference data construction and post-deployment optimization.
1 Introduction
Post-deployment improvement is difficult because agent trajectories are large, non-deterministic, and costly to review, while existing evaluation approaches do not scale to real-world usage. The paper proposes lightweight, model-free signals and validates signal-based trajectory sampling for identifying informative interactions.
- Agent trajectories are voluminous and non-deterministic, making manual review and reliable detection of unproductive loops or user frustration difficult.
- Offline benchmarks miss the long tail of real-world usage, while evaluating every trajectory with an auxiliary LLM is cost-prohibitive.
- The framework attaches model-free signals to trajectories, covering interaction, execution, and environment patterns for downstream triage.
- The paper introduces a signal taxonomy and sampling framework that prioritizes failures and successful exemplars without computing quality scores.
- 82% informativeness and a 1.52× efficiency gain are achieved by signal-based sampling, compared with 74% for heuristic and 54% for random sampling.
2 Related Work
Prior work provides mechanisms for agent improvement, preference learning, and behavioral quality assessment, but agentic settings still face a costly preference-data bottleneck. This paper addresses that bottleneck by triaging trajectories with lightweight signals rather than evaluating every trajectory with model calls.
- Modern agentic systems interleave LLM reasoning with tool execution, while post-deployment improvement remains an open challenge.
- RLHF and DPO are established preference-based training methods, but sourcing preference pairs remains a bottleneck for agentic settings.
- LLM judges can exceed 80% agreement with human preferences, yet evaluating every deployed trajectory remains cost-prohibitive.
- Dialogue and information-retrieval research derives quality proxies from behavioral data when explicit feedback or ground-truth rewards are unavailable.
3 Signal Taxonomy
The paper organizes observable trajectory behavior into interaction, execution, and environment signals, separating learning-oriented patterns from diagnosis-oriented system conditions. Lightweight detectors instantiate these signals from language and runtime events without model calls.
- Signal Taxonomy: The taxonomy uses data layer and downstream utility to distinguish interaction, execution, and environment signals.
- Interaction Signals: Interaction signals describe user–assistant discourse dynamics without asserting internal agent state or semantic correctness.
- Interaction Signals: Misalignment captures intent mismatch, while stagnation captures discourse that continues without visible progress.
- Interaction Signals: Disengagement marks withdrawal of cooperative intent, including exit requests, negative stances, and observable abandonment.
- Interaction Signals: Interaction detectors use normalization, typotolerant phrase matching, local similarity checks, and discourse repetition heuristics.
- Execution Signals: Execution signals derive deterministically from structured runtime events and capture agent decision-making behavior.
- Execution Signals: Failure signals identify non-advancing actions, whereas loop signals identify repetitive execution patterns without progress.
- Execution Signals: Execution detection classifies non-advancing tool outcomes and analyzes invocation sequences for repeated calls and multi-tool cycles.
4 Experiment
The experiment evaluates signal-based trajectory sampling as a fixed-budget mechanism for surfacing developer-informative agent interactions. On τ-bench, signal sampling is compared with random and length-based heuristic sampling across overall, reward-stratified, domain, and category outcomes.
- Sampling Strategies: Signal sampling selects 100 trajectories using aggregated interaction and execution signals, while heuristic sampling uses a 10-user-message threshold and random sampling is uniform.Environment signals are excluded because they serve a diagnosis-only role.
- Main Results: 82.0% of signal-sampled trajectories were developer-informative, compared with 74.0% for heuristic sampling and 54.0% for random sampling.The signal–random difference was highly significant (p < 0.001), whereas the signal–heuristic difference was not significant (p = 0.232).
- Reward-stratified Analysis: 66.7% of successful trajectories selected by signal sampling were informative, compared with 50.0% for heuristic sampling and 41.3% for random sampling.Among failed trajectories, signal sampling reached 96.2% informativeness; successful trajectories exposed subtler issues such as policy violations and inefficient tool use.
- Reward-stratified Analysis: 77.6% was signal sampling’s reward-standardized informativeness rate, versus 62.7% for heuristic sampling and 54.0% for random sampling.Standardization used random sampling’s reward distribution of 63% success and 37% failure, reducing signal sampling’s advantage by 4.4 points.
- Annotation Efficiency: Signal sampling yielded 82 informative trajectories per 100 annotations and a 1.52× efficiency gain over random sampling.Each informative trajectory cost 1.22 labels with signal sampling, versus 1.35 for heuristic and 1.85 for random sampling.
- Category Distribution: Among informative trajectories, action/tool-use issues comprised 57–60% and conversation issues 38–43% across all three strategies.The distribution of reason categories was stable, with small numbers of success exemplars.
- Domain Robustness: Signal sampling achieved its clearest domain separation in retail, with 78% informativeness versus 66% for heuristic and 35% for random sampling.Airline rates were higher and closer together, ranging from 86% to 96%.
5 Limitations
The framework’s evaluation is bounded by τ-bench’s two simulated-user domains, while its coarse behavioral taxonomy does not assess factual correctness or domain-specific policy violations. Its deterministic lexical detectors may also miss subtle patterns, creating a trade-off between recall and lightweight always-on deployment.
- Scope: τ-bench covers only airline and retail domains with LLM-simulated users, so generalization to broader domains and real users remains open.Simulated users may under-represent the variability of real disengagement and satisfaction patterns.
- Coverage: The coarse-grained behavioral taxonomy does not assess semantic correctness or domain-specific policy violations.Fluent trajectories that are factually wrong may therefore remain unsurfaced.
- Detection: Deterministic rules and lexical heuristics may have lower recall for subtle misalignment or implicit frustration without explicit lexical markers.Model-based detectors could improve recall but would sacrifice some of the lightweight computation enabling always-on deployment.
6 Conclusion
The paper presents a lightweight signal-based framework for triaging agentic interaction trajectories and prioritizing informative samples without quality scoring. On τ-bench, it achieves an 82% informativeness rate and a 1.52× efficiency gain over random sampling, while remaining practical for large-scale deployment and future preference-data construction.
- Conclusion: The framework combines a coarse signal taxonomy with sampling that prioritizes human review without computing quality scores.Signals span interaction, execution, and environment patterns, while sampling includes failures and successful exemplars.
- Conclusion: 82% informativeness rate and 1.52× efficiency gain over random sampling demonstrate the framework’s empirical effectiveness on τ-bench.The advantage remains robust across reward strata and task domains rather than merely oversampling obvious failures.
- Practicality: Deterministic, model-free signal computation gives the framework negligible overhead and scalability for large collections of interaction traces.This makes it suitable as a first-stage sampler in broader preference-data construction pipelines.
- Future work: End-to-end preference-data construction using selected trajectories and counterfactual continuations remains future work.The paper leaves the design and evaluation of that pipeline open.