PaperlayerView exact revision on arXiv

Source-linked AI summary

Behaviour analysis for large language model-driven agents

Tianqiang Yan, Lizhen Qu

arXiv:2610.32087v1cs.AIcs.CL

TL;DR

Understanding agent behaviour requires relating trajectory activity to context and outcomes, while connecting learned relationships to identifiable actions. The paper proposes a framework linking behavioural discovery, trajectory analysis, and experiments; in ReBeCA, selected behaviour sets achieved better predictive fit than outcome-only and full-variable models.

Abstract

from arXiv · show

Understanding language-agent behaviour requires learning how actions are organised across a trajectory and how this organisation relates to context and outcomes. We propose a general framework that connects data-driven behavioural discovery with analysis and experimental investigation. Categories developed from recorded activity define what is measured. Analyses preserve the behaviours' positions in the process and express their relationships as interpretable hypotheses. Statistical assessment and tests in held-out contexts examine the strength and scope of the evidence. Experiments, including prompt steering, then test how behaviour and outcomes change under deliberately varied conditions. A study of iterative self-reflection illustrates how discovery, dependency learning and prompt comparisons can inform one another. We develop this approach for trajectories of varying length, event order and agent roles, and identify advances in coding, analysis and experimentation needed to support its wider use.

Learning behavioural variables from trajectories

Behavioural categories must be grounded in recorded activity and retain the context, position, and actors associated with each occurrence. Measurement choices also distinguish what happened from what was said and whether an opportunity to act existed.

Learning behavioural variables from trajectories

  • Diverse trajectories, outcome-blind category development, contrasting passages, and separate development records help refine definitions and assess coverage.Once rules are fixed, independent trajectories allow assessment of per-category errors and human disagreement.Source: Learning behavioural variables from… · p. 3
  • Connecting each coded occurrence to its record, occasion, position, and actors preserves the context needed to interpret behavioural measurements.The section also recommends recording triggers, tool returns, and external state when these specify the activity.Source: Learning behavioural variables from… · pp. 3–4
  • A codebook can distinguish statements, executed actions, and recorded state changes, using counts, duration, or latency when suited to the question.For example, a stated intention to verify, a verification call, a subsequent revision, and a correct answer are distinct observations.Source: Learning behavioural variables from… · p. 3
  • Event-anchored windows can make positions comparable across open-ended runs, but interpretation depends on how absent, repeated, overlapping, and asynchronous events are represented.Anchoring at the first occurrence yields one window per episode and confines analysis to that occasion.Source: Learning behavioural variables from… · p. 4
  • Resetting memory and external state supports repeated episodes without learning across runs, while same-history continuations require state restoration.The resulting evidence is scoped to the recorded generation conditions, which can also be affected by expectations of training or evaluation.Source: Learning behavioural variables from… · p. 4

Learning behavioural structure across a trajectory

Trajectory-aware analysis distinguishes marginal outcome associations from information that remains given other recorded behaviours. The selected variables and their interpretation depend on the measured variables, outcome, and modelling assumptions.

Learning behavioural structure across a trajectory

  • Indexing observations by round or event preserves where behavioural relationships arise, while joint modelling examines relationships informative given other recorded variables.Conditioning on a later revision can remove an earlier behaviour’s additional predictive contribution.Source: Learning behavioural structure across… · p. 4
  • 11 of 15 code-derived variables were individually associated with COMET outcome at unadjusted P < 0.05, while the selected set contained one such variable.Marginal association and conditional dependence therefore answer different behavioural questions.Source: Learning behavioural structure across… · p. 4
  • ReBeCA uses causal discovery with graph scoring and temporal restrictions; complexity control and process constraints are possible safeguards against redundant or spurious associations.Findings still depend on recorded variables and modelling assumptions and require independent assessment.Source: Learning behavioural structure across… · p. 4
  • Marginal association, conditional information, prediction on new episodes, and instruction comparisons establish different kinds of evidence.The instruction-comparison quantities distinguish performance change from baseline and the combined change relative to the sum of separate changes.Source: Learning behavioural structure across… · p. 5
  • Outcome definitions and added behavioural codes can change which earlier behaviours retain outcome information, so sensitivity analysis reveals dependence on representation.Relationships with explicit behavioural meaning can then inform experiments on instructions.Source: Learning behavioural structure across… · p. 5

Interpretability from measurement to experiment

Interpretable relationships connect statistical findings to identifiable activity that can be examined experimentally. The section also distinguishes what association, prediction, and instruction comparisons establish.

Interpretability from measurement to experiment

  • An interpretable relationship specifies the behaviours involved, when they occur, under what conditions, and with what supporting evidence.Explicit behavioural meanings connect the statistical finding to activity that can be examined experimentally.Source: Interpretability from measurement to… · p. 5
  • Relating defined behaviours to internal representations and computations could connect model mechanisms to processes observed across a task.The passage frames this as a potential connection between internal analyses and behavioural analyses.Source: Interpretability from measurement to… · p. 5
  • Marginal and conditional associations describe recorded relationships, prediction assesses information on new episodes, and instruction comparisons examine responses to changed prompts.Measuring behaviour under changed conditions connects experimental results to the motivating hypothesis.Source: Interpretability from measurement to… · p. 5

Testing behavioural hypotheses

Testing behavioural hypotheses requires assessing both how well relationships extend beyond their discovery setting and how agents respond to deliberately varied conditions. Recoding behaviours helps interpret experimental outcomes and shape subsequent tests.

Testing behavioural hypotheses

  • Separating selection from final assessment through independent data or nested resampling protects predictive comparisons from evaluation-data selection optimism.ReBeCA compares selected sets with outcome-only and all-variable models using cross-validated log-likelihood.Source: Testing behavioural hypotheses · p. 5
  • Assessment in contexts excluded from discovery, fitting, and model selection examines generalisation across tasks, environments, or generation conditions.Random splits instead address variation within the sampled setting; resampling and varied coding or scoring choices test stability and sensitivity.Source: Testing behavioural hypotheses · p. 6
  • Factorial prompt experiments compare single and combined instructions on matched new problems, while restored-state continuations can hold preceding interaction fixed.Estimates concern the tested prompt conditions and should account for uncertainty, repeated observations, and multiple comparisons.Source: Testing behavioural hypotheses · p. 6
  • Recoding selected and unselected behaviours reveals changes accompanying an instruction, including whether an inconclusive test elicited its target behaviour.These measurements can inform subsequent prompts, controls, and hypotheses.Source: Testing behavioural hypotheses · p. 6

Lessons from the ReBeCA prototype

The ReBeCA prototype links round-indexed behavioural measurements to prediction and prompt tests in iterative self-reflection. Its findings identify candidate behaviours, while model variation and measurement limits constrain interpretation.

Lessons from the ReBeCA prototype

  • Four Qwen3 models completed 417 mathematics and 400 translation episodes over five rounds, with a medoid response selected from the largest semantic cluster at each step.Outcomes were final-answer correctness in mathematics and translation-quality change under two model-based metrics.Source: Lessons from the ReBeCA prototype · p. 6
  • Across rounds, text changed in 77.7–100% of fully graded mathematics episodes, but correctness flipped in only 1.3–1.8%; initial and final accuracy were 75.2% and 76.4%.Four behaviour codes and two uncertainty indicators per round yielded 30 variables per episode; the automated coder matched human-majority labels on 89 of 100 instances.Source: Lessons from the ReBeCA prototype · p. 6
  • Selected mathematics and translation behaviour sets achieved better cross-validated predictive fit than both outcome-only and full-variable models; conditional associations averaged 29.8 and 16.0 percentage points for CO2 and Sp4.Selection and comparison reused the same cross-validation, and associations used selection data; cross-environment transfer remains untested.Source: Lessons from the ReBeCA prototype · p. 6, Figure 3 · p. 7
  • 59 of 200 AIME answers were correct at baseline, versus 64 and 62 under single instructions and 49 under their combination; baseline comparisons were non-significant after Holm correction.The model-specific responses differed: specificity yielded 33 of 50 correct for the 4B model versus 21 at baseline, but 18 for the 8B model.Source: Lessons from the ReBeCA prototype · p. 6, Figure 3 · p. 7
  • In the 4B model, specificity alone solved 33 problems versus 19 for the combined instruction, while the pooled interaction was −9 percentage points with a 95% interval of −19 to 1.The interval remains compatible with zero interaction, motivating further tests of the combination.Source: Lessons from the ReBeCA prototype · p. 7, Figure 3 · p. 7
  • Interpretation is bounded by steering instructions that targeted selected dimensions, pooled coding agreement, and codes combining the dimension evaluated in feedback with the dimension changed in revision.Transfer to other model families and less constrained tasks remains open; recoding steering trajectories and separating feedback from revision could clarify findings.Source: Lessons from the ReBeCA prototype · p. 7
  • Beyond self-reflection, proposed designs distinguish actions, opportunities and research questions, while the prototype connects behavioural discovery with experiments and exposes measurement questions for broader use.Table 1 describes proposed research designs and uses a codebook different from ReBeCA's self-reflection codebook.Source: Table 1 · p. 8, Lessons from the ReBeCA prototype · p. 8

Extending the framework to open-ended behaviour

Open-ended and collaborative trajectories require measurements that preserve event order, context and agent roles. The proposed extensions connect those measurements to behavioural comparisons and, where symbolic state information exists, planning rules.

Extending the framework to open-ended behaviour

  • For varying-length, multi-agent trajectories, tool returns and external state help establish events, while initiator, recipient, executor and verifier roles keep collaborative actions interpretable.Repeated events, absent opportunities and actions involving several agents affect what counts as a comparable observation.Source: Extending the framework to open-ended… · p. 8
  • Replanning after a first error versus repeated errors can be measured, jointly analysed given other variables, and tested with prompts while tracking the same behaviour in both studies.Recurrent and asynchronous events may require sequence or event-based models that preserve behavioural meanings.Source: Extending the framework to open-ended… · p. 8
  • When symbolic actions and states are available, Safe Action Model learning can supply preconditions and effects to constrain candidate dependencies before graph search.Hard exclusions require a justified impossibility rule; uncertain support can guide preferences, with rule-guided and unguided models compared on independent trajectories.Source: Extending the framework to open-ended… · p. 8

Building cumulative knowledge of agent behaviour

Cumulative knowledge depends on shared behavioural definitions, trajectory provenance and explicit reporting of evidence and scope. The framework also treats experiments as inputs to continued measurement and hypothesis refinement.

Building cumulative knowledge of agent behaviour

  • Cross-study comparisons depend on shared understanding of what was measured, how trajectories arose and which evidence supports conclusions.Reporting should specify agent conditions, trajectory and outcome definitions, coding, analyses, experiments, scope and provenance.Source: Building cumulative knowledge of agent… · p. 9
  • Cross-model coding can test whether behavioural definitions transfer, while per-code prevalence, human disagreement and error types identify where measurements need revision.Complete trajectory records and releases with generation conditions, tool returns, state changes and sampling rules support alternative analyses, subject to privacy and access constraints.Source: Building cumulative knowledge of agent… · p. 9
  • Behavioural measurement can connect findings across reinforcement learning, coordination failures, safety studies and verbal-reasoning monitoring, whose reliability requires assessment.Internal interventions offer another way to examine hypotheses, with appropriate controls informing interpretation.Source: Building cumulative knowledge of agent… · p. 9
  • Experiments can refine behavioural accounts by measuring targeted and accompanying changes, comparing prompt formulations and contexts, and informing further graphical-model learning.Adaptive designs could select tests that distinguish competing hypotheses, while reports specify which observations support, revise or reject hypotheses and which populations estimates describe.Source: Building cumulative knowledge of agent… · p. 9

Outlook

The framework links discovery, analysis and experiment through explicit behavioural meanings, and ReBeCA illustrates one implementation in self-reflection.

Outlook

  • Explicit behavioural meanings connect discovery, analysis and experiment, allowing findings to be tested and refined across contexts.ReBeCA illustrates this connection in self-reflection.Source: Outlook · p. 10
  • The broader empirical aim is to account for how agents respond to information, coordinate actions and reach outcomes.Source: Outlook · p. 10
  • ReBeCA is presented as one implementation of the framework, rather than the full scope of the broader empirical account.Source: Outlook · p. 10

Data availability

The underlying trajectories and instruction-test data are slated for public repository deposit upon publication; derived numerical summaries accompany the Perspective.

Data availability

  • The underlying trajectories and instruction-test data will be deposited in a public repository on publication.Derived numerical summaries accompany the Perspective; the study is described in the public ReBeCA preprint.Source: Data availability · p. 10

Supplementary Information

The supplementary information defines behavioural comparison quantities, predictive assessment, and verification procedures for the reported analyses.

Behavioural variables and comparison quantities

  • The quantities define episodes under specified conditions, independently coded outcomes, and behavioural variables indexed by round in fixed-round studies.Event-anchored studies require corresponding rules for recurring or absent events; codes may represent presence, frequency, or duration.Source: Supplementary Note 1 | Behavioural… · p. 17
  • The framework distinguishes outcome associations and conditional information from instruction-based performance and interaction comparisons.A(v) measures outcome differences for presence versus absence of a behaviour; D(v) measures added outcome information given other variables, while Δ and ι compare prompt conditions.Source: Supplementary Note 1 | Behavioural… · p. 17

Predictive assessment

  • Cross-validated log-likelihood assesses candidate variable sets on held-out data against empty-set and all-variable models.Higher values indicate better predictive fit under the specified estimator and fold construction; selected-set scores require independent or nested assessment when the same criterion selected the set.Source: Supplementary Note 2 | Predictive… · p. 17

Verification of supporting summaries

  • The mathematics archive contains 418 records, while Fig. 3b uses the 399 complete-output episodes with non-missing grades at every position.The two translation-metric exports contain the same 400 trajectories; the plot omits a legacy boxed-expression comparison that conflates answer changes with extraction failures and syntactic differences.Source: Supplementary Note 3 | Verification of… · p. 18
  • Instruction-test statistics use 50 problems, four models, and four conditions, with problem-level bootstrap intervals and multiplicity-corrected tests.Intervals use 20,000 bootstrap resamples; exact two-sided McNemar tests cover 24 condition-pair comparisons, and Cochran’s Q tests are corrected across four models.Source: Supplementary Note 3 | Verification of… · p. 18
Loading 2610.32087v1…