PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent Behavior Maps: Locating Agents by What They Would Do, Without Running Them

Shota Fujikawa, Takuya Tamura, Kosuke Akimoto, Daichi Haraguchi, Masafumi Oyamada

arXiv:2610.30014v1cs.LGcs.AIcs.CL

TL;DR

LLM agents with similar success rates can act differently, and measuring run-time behavior directly requires new rollouts. Agent Behavior Maps score LLM–harness pairs on shared reference trajectories; across environments, their coordinates support performance prediction and behavioral comparisons, subject to limits on the guarantees and their scope.

  • Problem

    Whether action probabilities measured on fixed reference histories capture run-time trajectory distributions and behavior is unclear, despite the cost of measuring those properties with new rollouts.

    Source: §1 · p. 1

  • Method

    Agent Behavior Maps use centered, teacher-forced action log-likelihoods on shared reference trajectories to represent LLM–harness combinations without new target-agent rollouts.

    Source: §1 · p. 1

  • Results

    Across four environments, ALFWorld-based coordinates support success-rate prediction and behavioral-neighbor comparisons, with accuracy and similarity varying by environment and measure.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The map represents both performance-related information and how close agents are in what they do, while identifying behavioral neighbors that take similar action sequences when run.

    Source: §6 · p. 8

    Guarantees and behavioral correspondence concern close pairs; the distance has not been shown to measure behavioral difference for far-apart pairs.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM agents with similar success rates can follow different action sequences. An Agent Behavior Map represents LLM-harness combinations using centered action log-likelihoods on shared reference trajectories, without requiring new environment rollouts from the target agent. We derive error bounds relating squared coordinate distances to trajectory KL under explicit assumptions, and across eight cross-scored models the distances track the directly measured trajectory KL in rank order. Across 100 LLMs and four environments, ALFWorld-based coordinates support performance prediction and identify behavioral similarities. In ALFWorld, these similarities persist after controlling for success rates and model size, including across model families. As an extension, harness-component displacements approximately compose, with accuracy varying across components and environments.

1 Introduction

The paper asks whether shared, off-policy reference trajectories can locate LLM agents by run-time behavior without new target-agent rollouts. It introduces Agent Behavior Maps and evaluates their theoretical and empirical relationship to trajectory divergence, performance, behavior, and harness changes.

1 Introduction

  • Agent Behavior Maps represent each LLM–harness pair with centered action log-likelihoods on shared reference trajectories, so new agents can be located without collecting their environment trajectories.The harness rebuilds each input from the shared external history, and recorded actions are teacher-forced under that input.Source: §1 · p. 1
  • Under policy-locality and reference-mismatch assumptions, squared coordinate distances admit error bounds for trajectory KL; cross-scoring eight models finds high rank correlation with directly measured KL.The error decomposes into local approximation, reference mismatch, centering, and finite-sample sources.Source: §1 · pp. 1–2
  • Across four environments, ALFWorld-based coordinates support success-rate prediction and identify behavioral neighbors; in ALFWorld, behavioral similarity persists against success-rate, size, and other-family comparisons.Prediction accuracy and behavioral similarity vary by environment and measure.Source: §1 · p. 2
  • Harness-component displacements approximately compose within an LLM, with accuracy depending on the component and environment.Source: §1 · p. 2

2 Agent Behavior Maps

The section defines Agent Behavior Maps from teacher-forced scores on shared trajectories and analyzes when their distances approximate trajectory KL. It distinguishes step and trajectory embeddings, then adds finite-reference sampling error to the approximation bounds.

2.1 Problem: comparing agents by what they do when run

  • The comparison target is trajectory KL between agents’ run-time trajectory distributions, whose direct measurement requires rollouts and O(n^2) cross-scorings for n agents.Source: §2.1 · p. 3

2.2 Scoring agents on shared reference trajectories

  • Teacher forcing scores recorded actions on shared reference trajectories without running the environment, but reference mismatch and finite trajectory counts introduce costs.The trajectories may come from different sources, and their distribution need not be known.Source: §2.2 · p. 3, §2.1 · p. 3

2.3 Step and trajectory embeddings

  • Under locality and reference-mismatch assumptions, the step and trajectory coordinate distances have bounds on their error in estimating trajectory KL.The bounds also extend to symmetrized KL; the trajectory bound has a factor of about T more than the step bound for each error source.Source: §2.3 · p. 3, Table 1 · p. 3

2.4 Total error: approximation plus sampling

  • The total error combines approximation and sampling terms; the trajectory bound carries an extra factor of T, while sampling error decays as N^-1/2.The bounds apply only to close pairs with small ε and λ.Source: §2.4 · p. 4

3 What does the map capture about an agent?

The map’s coordinate distances track on-policy trajectory KL in rank order, predict success rates from reference-trajectory scores, and identify agents with similar action sequences. Behavioral-neighbor effects are strongest and most robust in ALFWorld, including after controls for performance and size.

3.1 Does distance track the on-policy divergence?

  • Pearson r = 0.90 and Spearman ρ = 0.95 for centered step estimates versus Ksym, compared with 0.86 and 0.94 for trajectory estimates, preserving trajectory-KL rank order.Both estimates overstate KL level, so the paper uses them as proxies for its ordering rather than its magnitude.Source: Figure 2 · p. 4, §3.1 · p. 4

3.2 Performance: predicting without running

  • Across 100 models, ALFWorld kNN predicted success from 500 reference trajectories at r = 0.88 versus size’s 0.63; combining size and coordinates reached a four-benchmark average of 0.83 versus 0.74.Prediction used leave-one-model-out evaluation; the combined predictor also reached 0.81 versus 0.72 for size on the three-environment average.Source: Table 2 · p. 5, §3.2 · p. 5

3.3 Behavior: do neighbors act alike?

  • Five coordinate neighbors were more similar than remaining models across all four benchmarks; ALFWorld edit-distance effect was 0.88 versus 0.40 for size and remained 0.60 across families.Coordinates exceeded family-based neighbors on ALFWorld, TextWorld, and MiniWoB++ except for first action on TextWorld, and tied on BFCL.Source: §3.3 · p. 6
  • Behavioral similarity decreases with coordinate-neighbor rank, returning to the overall mean around rank 30, while size-neighbor effects are nearly flat beyond rank one.The divergence position and profile of idiosyncrasies follow the same pattern.Source: §3.3 · p. 6
  • In ALFWorld, coordinate neighbors beat success-rate neighbors on all four behavioral aspects; after controlling for performance and size, edit-distance and 2-gram effects remained 0.61 and 0.68, including 0.31 each across families.In other environments, coordinate and success-rate neighbors were roughly on a par, with significant coordinate advantages limited to specified measures in TextWorld and MiniWoB++.Source: §3.3 · pp. 6–7

4 Composing Harness Effects in Agent Behavior Maps

The paper tests whether harness components can be represented as displacements whose sums predict the coordinates of unscored combinations. It evaluates this using a fixed three-slot harness design and compares predictions with measured compositions.

4.1 Harness interventions and setup

  • Harnesses vary context management, planning, and acting; each single-component change from BASE defines a displacement whose sum predicts a composition’s coordinates.Context management includes windowed, retrieve, or summarize; planning is none or replan; acting is direct or think.Source: §4.1 · p. 7
  • The experiment scores 12 configurations but uses only BASE and four single-component configurations to construct predictions for seven compositions across 12 LLMs.The 60 scored BASE and single-component agents yield composed coordinates for 84 combinations; measured composed coordinates are reserved for validation.Source: §4.1 · p. 7

4.2 Predicting coordinates of unscored compositions

  • Two-component compositions have median cosine 0.90 and median norm ratio 0.98; additivity varies by component and environment.Retrieve with planning or reasoning agrees best, while replan + think is weakest; three-component compositions and results on ALFWorld and SWE-Bench show weaker agreement.Source: §4.2 · p. 8

5 Related Work

The paper extends model-map approaches from static language models to sequential agents and relates its approach to policy comparisons and harness evaluation. Its distinction is measuring behavior from fixed trajectories without running the target agents.

5 Related Work

  • Prior language-model maps use log-likelihoods on fixed texts or prompt–response distributions, while policy comparisons use actions on fixed states.The paper extends language-model maps to closed-loop agents.Source: §5 · p. 8
  • Unlike performance prediction from cheaper benchmarks, which needs target-model scores, the paper’s coordinates need neither target scores nor environment runs.The paper characterizes fixed-trajectory measurement cost with a reference-mismatch term that accumulates over steps.Source: §5 · p. 8
  • Earlier harness studies run LLM–harness combinations; this work represents components as map displacements and tests their additivity without running composed agents.Prior behavior comparisons run agents and use aggregate statistics or classifiers; this work also relates map distances to trajectory KL and tests whether close pairs behave alike when run.Source: §5 · p. 8

6 Conclusion

The conclusion presents Agent Behavior Maps as a way to locate agents by behavior as well as performance, using teacher-forced scores on shared reference trajectories. It reports behavioral similarity among map neighbors and approximate composition of harness-component displacements, with limits on the guarantees and scope.

6 Conclusion

  • Agent Behavior Maps use teacher forcing on shared reference trajectories to locate agents without running them.The conclusion reports that coordinates predict success rates for unrun models and that map neighbors take similar action sequences when run.Source: §6 · p. 8
  • Harness-component displacement sums approximate coordinates of unscored combinations, but accuracy depends on the components and environment.The conclusion also notes that the theory decomposes the distance-to-trajectory-KL gap into four error sources and limits its guarantee to close pairs.Source: §6 · p. 8

Limitations

The limitations constrain the map’s quantitative interpretation, trajectory coverage, KL estimation, eligible models, behavioral sampling, and harness-additivity scope. The appendix also distinguishes grammar-constrained trajectory-KL scoring from later experimental scoring without the grammar mask.

Limitations

  • The trajectory-KL guarantee and behavioral correspondence apply to close pairs; behavioral correspondence fades beyond about the 30th neighbor.For different-LLM pairs, distance estimates exceed measured KL by a median factor of 7.5 despite matching its rank order; far-pair behavioral differences are not established.Source: Limitations · p. 9
  • Fixed reference trajectories cannot measure policy differences in regions the reference policy never visits, and the mismatch term is not determined by the likelihood matrix alone.A general theory of the reference policy is left for future work.Source: Limitations · p. 9
  • Trajectory-KL estimates use canonical token-sequence probabilities rather than action probabilities marginalized over tokenizations.Renormalization can resolve this where the candidate set is enumerable; otherwise it remains a limitation.Source: Limitations · p. 9
  • Models whose APIs do not return log-probabilities for given text cannot be mapped, limiting the study to open-weight models..Source: Limitations · p. 9
  • Behavior comparisons use one rollout per task, and the study does not compare run-to-run variation with neighbor behavioral distances.The text notes that behavior varies between runs.Source: Limitations · p. 9
  • The harness experiment covers 12 configurations of a three-slot design and 12 LLMs, with ALFWorld and SWE-Bench results covering six LLMs.Whether additivity holds for other scaffold designs is untested.Source: Limitations · p. 9

A.1 Grammar-constrained action probabilities

  • Grammar-constrained decoding renormalizes allowed tokens, and action probability sums the probabilities of token sequences that normalize to that action.The trajectory-KL comparison in §3.1 uses grammar-constrained scoring, whereas experiments from §3.2 onward score without the grammar mask.Source: §A.1 · p. 12

A.2 Proof of Theorem 1

The proof bounds the gap between trajectory KL and squared centered-coordinate distances by decomposing approximation, reference-distribution mismatch, centering, and sampling errors. It also explains how trajectory-length cross terms affect the distance and gives bounds for uncentered variants.

A.2 Proof of Theorem 1

  • For bounded log-ratios, local KL differs from half the expected squared log-ratio by a cubic-order remainder.The lemma also bounds the local KL between zero and ε^2/2.Source: §A.2 · pp. 12–13
  • Theorem 1 bounds the errors between trajectory KL and squared step or trajectory distances with explicit constants.The proof attributes the gaps to quadratic-approximation remainder, taking the quadratic term under the reference rather than on-policy distribution, and centering.Source: §A.2 · pp. 12–13

A.3 Proof of Corollary 1

  • The finite-sample corollary bounds estimated distances by combining simultaneous concentration of four reference-trajectory sample means with Theorem 1’s approximation error.Whole trajectories, not within-trajectory steps, are the independent sampling units.Source: §A.3 · pp. 13–14

A.4 Cross terms in the trajectory distance

  • The trajectory distance adds cross-step and variable-length terms to the step distance, and its cross term can be negative.The cross term has up to T(T−1)/2 summands per trajectory; consequently, the trajectory version does not always overestimate.Source: §A.4 · pp. 14–15

A.5 Uncentered variants

  • Under the theorem’s assumptions, uncentered step and trajectory variants also have error bounds, without the centered versions’ μ^2 terms.The two uncentered bounds are stated in Proposition 1.Source: §A.5 · p. 15

B.1 Setup of the two experiments

The experiments use a fixed BASE harness for the two §3 studies, while the harness-component study is separate. A robustness check compares masked and raw action likelihoods on shared reference steps.

B.1 Setup of the two experiments

  • Across six models with masked and raw scores on 3,412 shared reference steps, the grammar mask changed mean log-likelihood per action by −0.01 to +0.03 nats.Source: §B.1 · p. 15
  • The two experiments in §3 fix the BASE harness; the multi-harness experiment is reported separately in Table 11.The 100-model study uses 100 open-weight LLMs, and family labels serve as a baseline in §3.2–3.3.Source: §B.1 · p. 15, Table 4 · p. 16
  • Replacing raw scores with masked scores leaves the nearest neighbor unchanged for all ten tested LLMs and yields distance-rank correlations of at least 0.997.The five nearest neighbors remain unchanged for nine models.Source: §B.1 · p. 16

B.2 Trajectory KL: centering, step vs. trajectory, and sample size

The step-distance estimate tracks directly measured trajectory KL more closely than the trajectory-distance estimate in the eight-model comparison. Centering, distance definition, and reference-trajectory sample size affect agreement in distinct ways.

B.2 Trajectory KL: centering, step vs. trajectory, and sample size

  • Centering gives slightly higher KL correlation and lower RMSE, while no centering choice wins across KL agreement, performance prediction, and behavioral-neighbor measures.The paper reports centered coordinates in the main text and uncentered results in the appendix.Source: §B.2 · p. 16
  • The step version agrees better with KL in correlation and RMSE; both estimates overstate measured Ksym, by median factors of 7.5 for step and 74 for trajectory.The ratio of those factors is about 10, on the order of the mean reference trajectory length of 23.8 steps.Source: §B.2 · p. 16
  • On the 100-model map, centered step estimates correlate with Ksym at Pearson r = 0.86 and Spearman ρ = 0.88; the two embeddings’ 28 estimates have rank correlation 0.97.The comparison uses the same Ksym but a different reference distribution and scoring convention.Source: §B.2 · pp. 16–17
  • At N = 100, the step estimate’s median coefficient of variation is 0.074; at N = 3,200, it is 0.010, with corrected log–log slope −0.48.Despite lower resampling variation, the median estimate-to-Ksym ratio stays between 7.5 and 7.6 for step and 72 and 74 for trajectory across N.Source: §B.2 · pp. 17–18

B.3 Performance prediction: protocol and full results

The section specifies leave-one-model-out performance prediction and behavioral-distance analyses using shared ALFWorld-based coordinates. It also describes how behavior is measured, neighbors are selected, and statistical tests should be interpreted.

B.3 Performance prediction: protocol and full results

  • The four-benchmark average covers 200 tasks across ALFWorld, TextWorld, BFCL, and MiniWoB++, with 100 models evaluated and no missing values.The three-other-environment average covers 150 TextWorld, BFCL, and MiniWoB++ tasks.Source: §B.3 · p. 18
  • Predictors hold out one model at a time, tune hyperparameters by inner leave-one-out on the other 99 models, and clip predictions to [0, 1].Bootstrap intervals for differences from the size baseline use 2,000 model resamples.Source: §B.3 · p. 18
  • Adding mean log-likelihood to size barely changes predictions, while uncentered-coordinate predictions are about as good as centered-coordinate predictions.Using Spearman rather than Pearson correlation does not change the method ordering.Source: §B.3 · p. 18

B.4.1 Definitions

  • Behavior is measured with action-type frequency, normalized Levenshtein, action 2-gram Jaccard, and first-action disagreement distances.The appendix also adds divergence position, conditional disagreement, profile distance, repetition and exploration measures, invalid-action rate, and other task-specific rates.Source: §B.4.1 · pp. 18–19
  • Five nearest models are selected by coordinates or by size, family, mean log-likelihood, or benchmark success rate; paired Wilcoxon tests use models as units.No multiple-comparison correction is applied, so significance markers are descriptive guides; small differences outside ALFWorld are suggestive.Source: §B.4.1 · pp. 19–20

B.4.2 Robustness of the neighbor effect

Coordinate neighbors generally retain behavioral similarity under alternative distance measures, performance controls, and coordinate choices. Distance reliability varies by measure and benchmark, with early-step and success-pattern distances less reproducible, especially on BFCL.

B.4.2 Robustness of the neighbor effect

  • Divergence-position and conditional-disagreement measures follow the first-action pattern, but success-rate neighbors are closer on profile and success-pattern measures.Profile and success-pattern distances reflect failure frequency and are closely tied to success rate.Source: §B.4.2 · p. 20
  • After controlling for success rate and size, coordinate-neighbor effects remain on every behavioral aspect in ALFWorld and mainly on local and early-step measures elsewhere.The success-pattern control subtracts expectations from a null distribution that preserves model success rates and task difficulty.Source: §B.4.2 · p. 20
  • Excluding same-family models preserves coordinate-neighbor effects on all four benchmarks, while uncentered coordinates show weaker effects.The authors suggest overall likelihood level may influence neighbor selection in uncentered coordinates.Source: §B.4.2 · p. 20

B.4.3 Reliability of the behavioral distances

  • Composition, global, and local distance matrices correlate at 0.85 or higher across random task halves; early-step measures and success patterns are less reproducible, especially on 25-task BFCL.The profile and conditional-disagreement measures could not be split by task.Source: §B.4.3 · p. 20

C.1 Harness implementations, models, and references

The harness experiment evaluates centered agent coordinates and harness-component displacements using shared reference trajectories. Component displacements align with composed displacements across environments, but the agreement varies by environment and reconstruction error can remain substantial.

C.1 Harness implementations, models, and references

  • Harness inputs combine a shared fixed part with selected history, current-step planning or reasoning, and an action instruction; retrieval, summarization, replanning, and thinking alter these components.The history budget is 8,192 tokens and the full input is capped at 16,384 tokens; retrieval and summarization are skipped for short histories within the stated budget condition.Source: §C.1 · pp. 20–21
  • The 12-LLM-by-12-harness setup scores fixed reference actions by teacher forcing; τ-bench, SWE-Bench, and ALFWorld coordinates use 4,213, 3,916, and 2,299 common steps, respectively.ALFWorld and SWE-Bench use six of the twelve LLMs.Source: §C.1 · pp. 21–22, Table 11 · p. 22
  • On τ-bench, centering removes mean log-likelihood changes of +9.6 for retrieve, +3.5 for summarize, −4.2 for replan, and −6.2 for think nats per step; each sign is shared by at least 11 of 12 LLMs.Values are medians over LLMs; context management makes reference actions more natural, while planning and reasoning move the agent away from them.Source: §C.1 · p. 22

C.2 Additivity: full tables

  • 0.63 (ALFWorld) and 0.81 (SWE-Bench) are the median cosines for five two-component compositions; displacement directions agree more weakly there than on τ-bench, and ALFWorld relative reconstruction error reaches about 1.These medians are over LLMs; ALFWorld uses only 2,299 reference steps scored by all 72 agents.Source: Table 12 · p. 23, Figure 5 · p. 23, §C.2 · p. 23
  • Relative reconstruction error is 1 + r^-2 − 2c/r, so matching displacement norms still leaves error whenever the cosine c is below 1.Here r is the norm ratio and c is the cosine between the composed and actual displacement vectors.Source: §C.2 · p. 23

Related papers

Loading 2610.30014v1…