PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Cognitive Depth: Operational Criteria for Evaluating LLM Agents

Nijesh Upreti, Chris Sypherd, Vaishak Belle

arXiv:2610.04168v1cs.AI

TL;DR

Existing agentic benchmarks measure task completion but often do not identify which trajectory-level capacity is weak, and evaluations cover the five criteria unevenly. The paper proposes a five-criterion profile with perturbation and deployment-observation methods; existing evaluations measure C well on short prompts, M weakly in specific formats, and T and A sparsely.

  • Problem

    End-to-end agent benchmarks provide task-completion scores but leave the failing part of the agentic loop unclear.

    Source: §1 · p. 1

  • Method

    The paper defines a trajectory-level profile across five criteria and combines controlled perturbation, deployment observation, and mechanistic interpretability for measurement.

    Source: §1 · p. 2, §4.1 · p. 11, §4.2 · p. 11

  • Results

    Existing evaluations measure C well on short prompts, M weakly and mostly in specific task formats, and T and A only sparsely, while aggregate benchmarks measure task breadth.

    Source: §5 · p. 13

  • Takeaways & Limitations

    The framework treats single-call benchmark progress as potentially decoupled from trajectory-level progress, with the decoupling largest in agentic settings.

    Source: §8 · p. 18

    The profile hypotheses remain illustrative; direct testing requires matched agents, held-out perturbation sets, and repeated runs.

    Source: §9 · p. 19

Abstract

from arXiv · show

Agentic large language model (LLM) systems are commonly implemented as an LLM in a loop with Planning, Memory, Tools, and Control Flow. This application-focused view connects agentic LLM research with deployable systems and leaves open how such systems should be evaluated beyond end-to-end task success. Building on this view, we define agentic cognitive depth as a trajectory-level profile across five operational criteria. The profile contains context sensitivity ($C$), temporal continuity ($T$), multimodal coordination ($M$), adaptive interaction ($A$), and metacognitive monitoring ($Mc$). The first four criteria measure how well Control Flow, Memory, Tools, and Planning are used across a trajectory. The fifth measures whether the system monitors and regulates the full run. For each criterion, we give operational proxies and a perturbation procedure, then connect the profile to the agent's world model. We provide the structure needed to extend benchmarks such as GAIA, SWE-bench, WebArena, and TRIP-Bench with per-criterion diagnostics. Symbolic verifiers, structured memory, planner coupling, and tool constraints provide practical ways to build and test these capacities.

1 Introduction

Existing agent benchmarks report end-to-end completion but often leave unclear which part of an agentic loop failed. The paper proposes a five-criterion trajectory profile and a measurement methodology, while limiting its contribution to measurement rather than a full cognitive theory or empirical benchmark.

1 Introduction

  • Single-call scores and end-to-end task completion leave the failing part of an agentic loop unclear.Trajectory-level sequences add multiple points where aggregate scores can hide failure sources.Source: §1 · pp. 1–2
  • The proposed agentic cognitive-depth profile measures context sensitivity, temporal continuity, multimodal coordination, adaptive interaction, and metacognitive monitoring.The first four criteria map to Control Flow, Memory, Tools, and Planning; Mc adds explicit monitoring and control over the trajectory.Source: §1 · p. 2
  • The paper links the profile to the agent’s world model and proposes combining controlled perturbation, deployment observation, and mechanistic interpretability.Proposed design patterns include symbolic verification for Mc, structured memory for T, planner coupling for A, constrained tool routing for M, and difficulty-aware control for C.Source: §1 · p. 2
  • The work is limited to measurement; a broader cognitive theory and an empirical benchmark are left for future work.The authors hypothesize that agents with the same end-to-end score can have different profiles that predict different deployment failures.Source: §1 · p. 2

2 Agentic Cognitive Depth Criteria and Architecture

Agentic cognitive depth represents an observable task trajectory as a profile of five normalized behavioral scores rather than a single aggregate. The first three criteria connect trajectory behavior to control flow, memory, and coordination across modalities or tools.

2.1 The Profile

  • A task trajectory is represented by a five-score profile, with scores normalized to [0, 1] and reported as a tuple to preserve diagnostic differences between agents.The profile uses observable input and output behavior; it avoids claims about consciousness or general intelligence.Source: §2.1 · p. 3

2.2 Context Sensitivity (C) and Control Flow

  • Context sensitivity measures whether control flow changes appropriately with input context rather than following the same trajectory for every input.The control-flow decisions include which call to make, when to stop, how to handle errors, and how to manage context.Source: §2.2 · p. 4

2.3 Temporal Continuity (T ) and Memory

  • Temporal continuity measures whether an agent maintains and updates a coherent model of entities, agents, and states across extended interaction.Memory substrates include short-term context, retrieval-augmented context, and persistent long-term memory; the criterion tests recall, updates, and contradiction detection across a trajectory.Source: §2.3 · pp. 4–5

2.4 Multimodal Coordination (M) and Tools

  • Multimodal coordination requires performance on tasks using multiple modalities to exceed the best single-modality performance; in deployed agents, the criterion also covers coordinating tool channels.Redundant or contradictory tool calls, and calls that ignore prior results, are identified as low-M failures.Source: §2.4 · p. 5

2.5 Adaptive Interaction (A) and Planning

The section defines adaptive interaction as within-session strategy revision and metacognitive monitoring as regulation of the full run. It distinguishes these trajectory-level capacities from component presence alone and motivates the five criteria through recurring groups across agentic and cognitive-architecture literatures.

2.5 Adaptive Interaction (A) and Planning

  • Adaptive interaction measures whether an agent revises its strategy in response to within-session feedback rather than repeating a failed approach.The section treats switching strategies after identifying an error as model-based behavior, and ties the criterion to plan adherence and revision.Source: §2.5 · pp. 5–6

2.6 Metacognitive Monitoring (Mc) as an Architectural Gap

  • Metacognitive monitoring tracks reasoning and knowledge gaps, then regulates the full trajectory through compute allocation, uncertainty calibration, failure detection, and strategy selection.The authors note that local self-reflection and retry loops usually revise an answer without regulating the full run.Source: §2.6 · p. 6
  • M is horizontal integration among object-level modalities and tool channels, whereas Mc is a vertical layer that monitors and regulates that process.The section describes high M with low Mc as possible when an agent combines tools well but misses unreliable combinations.Source: §2.6 · p. 6
  • Evidence on current LLM metacognition is mixed: compositional self-prediction and transfer often fail, while confidence can correlate with correctness and internal signals can be partly monitored or steered.The passage identifies a remaining monitoring-without-control problem: reliability signals may not be used to choose actions.Source: §2.6 · pp. 6–7

2.7 The Mapping

  • The proposed Mc design places a monitor and controller above the object-level loop, while the criterion-to-component mapping evaluates how components are used, not merely whether they exist.An adequate memory store or broad tool set can still yield low T or M when the agent fails to reconcile state or coordinate results.Source: §2.6 · p. 7, §2.7 · p. 7, Figure 1 · p. 7

2.8 Why These Five Criteria

  • The five criteria reflect groups recurring across agentic frameworks and cognitive architectures: control, memory, tools or action, planning or reasoning, and reflection or metacognition.The authors select these groups because they recur across the literatures and can each be measured.Source: §2.8 · pp. 7–8, Table 1 · p. 8

3 The World-Model Substrate

The section treats a maintained world model as the substrate for trajectory-level criteria, then proposes task-step mental models that revise and write back information. It connects explicit externalisation to inspectable agent design, while predicting that explicit, maintainable models improve criterion profiles even when end-to-end scores match.

3 The World-Model Substrate

  • A world model stores information about the environment, tools, agent capabilities, and task that can guide, verify, or revise later actions.An externalised version gives developers an object to inspect when a criterion profile is low.Source: §3 · p. 8

3.1 What a World Model Contains

  • Table 2 organises seven overlapping world-model components and their current status; those components are often fragmented across prompts, weights, stores, tools, and controller code.The fragmentation makes components hard to inspect and update; the table also lists explicit symbolic forms for externalisation.Source: Table 2 · p. 9, §3.1 · p. 9, §3.2 · p. 9

3.3 Mental Model and Non-Monotonic Update

  • A persistent world model supplies a task-step mental model that observations can revise and write back, retracting contradicted conclusions while preserving session learning.The paper says current agents may paraphrase contradictory tool returns into agreement; explicit symbolic beliefs with AGM-style retraction are proposed as a direct way to handle the update problem.Source: §3.3 · pp. 9–10
  • The five criteria measure whether the agent loads the right world-model fragment, maintains it, integrates modalities, revises after failure, and monitors its own inconsistency.The authors predict explicit, maintainable world models will score higher on these criteria than implicit, fragmented models, even when end-to-end scores match.Source: §3.3 · p. 10

3.4 Externalising the World Model

  • Externalisation is a design recommendation independent of the measurements; in the proposed hybrid loop, symbolic components verify LLM-proposed actions against the world model before action.The agent then updates the model after acting.Source: §3.4 · p. 10

4 Measurement Methodology

The methodology combines controlled perturbation, observation during deployment, and mechanistic interpretability to profile the five criteria. The modes trade off control, ecological validity, and access to internal states, and their combination can reveal when benchmark scores overstate a criterion-level capability.

4.1 Perturbation Measurement

  • Perturbation holds a base task fixed, applies a controlled change, and compares behaviour with and without it to attribute differences to the edit.Criterion-specific scores are computed over perturbed tasks and normalised to [0, 1]; synthetic perturbations may diverge from deployment behaviour and can be costly to design.Source: §4.1 · p. 11

4.2 Dynamic Observation and Mechanistic Interpretability

  • Dynamic observation tracks real-workload behaviour over extended deployment, capturing drift, long-tail failures, learning curves, and calibration at population scale.It trades causal isolation for ecological validity and leaves causation unresolved; existing telemetry can make this mode add no perturbation cost.Source: §4.2 · p. 11
  • Mechanistic interpretability probes internal states, attention, and representations to measure criterion-specific signals, offering the most precision and explanatory insight but the least scalability.Examples include activation differences for context sensitivity and confidence-to-correctness signals for metacognitive monitoring.Source: §4.2 · p. 11

4.3 Combining the Three Modes

  • The recommended mode pairings are perturbation plus interpretability for C and M, perturbation plus dynamic observation for T and A, and all three modes for Mc.The complete profile synthesises perturbation, dynamic, and, where access permits, interpretability scores.Source: §4.3 · p. 12
  • Capability overstatement occurs when an aggregate benchmark score passes its threshold while a criterion score falls below its own threshold.A high aggregate score alongside a low criterion score is evidence that the benchmark score reflects an artefact more than the underlying capability.Source: §4.3 · p. 12

5 Existing Evaluation Methods and the Five Criteria

The section maps existing evaluation methods to the five criteria and finds that current methods leave full trajectory decomposition open. Aggregate task success and agentic completion provide useful coverage, but often leave criterion-level weaknesses entangled or unmeasured.

Existing evaluation methods

  • The mapping asks which criteria existing methods measure and which remain hidden inside aggregate scores, extending rather than replacing detailed surveys of agentic evaluation.The authors summarise method coverage qualitatively in Table 3.Source: §5 · p. 12, Table 3 · p. 13

Classical tests

  • Classical tests cover limited slices: the Turing Test captures short-conversation C and partial within-session T, while leaving M, cross-session T, A, and Mc outside its test.The Winograd Schema probes one aspect of C and is near-ceiling for frontier LLMs; the Chinese Room challenges what passing the Turing Test establishes.Source: §5 · p. 12

Achievement on fixed tasks

  • Fixed-task evaluations range from closed-domain mastery to interactive exploration, but their coverage remains narrow or incomplete across the five criteria.ARC-AGI-3 can test C, T, and A, while embodied evaluations measure M and within-task A in narrow simulated physical domains.Source: §5 · p. 12

Aggregate and agentic benchmarks

  • Aggregate benchmarks measure task competence, while agentic benchmarks capture C, T, M, and A implicitly through completion and leave criteria entangled; Mc remains implicit.TRIP-Bench, AMA-Bench, and Hell or High Water probe aspects of interaction, memory, or failure recovery, but still score end-to-end success.Source: §5 · p. 13

Coverage summary

  • The coverage summary rates C well on short prompts, M weakly and mostly in specific formats, and T and A sparsely; full five-criterion trajectory decomposition remains open.The Table 3 ratings are the authors’ qualitative reading, and per-criterion profiles are identified as needed for targeted improvement.Source: §5 · pp. 13–14, Table 3 · p. 13

6 Worked Cases

The worked cases illustrate how the five-criterion profile distinguishes architecture-level tendencies and localises failures that end-to-end outcomes leave unresolved. A coding trajectory shows interacting Memory, Planning, and Control Flow failures within one task.

Illustrative architecture profiles

  • Illustrative profiles assign low T and Mc to single-turn and ReAct-style agents, while memory-plus-planning agents can score medium-to-high from C to A but still low on Mc.The authors present these as illustrative estimates, not empirical measurements.Source: §6 · p. 14

Low-score cases

  • Example low scores include failing to short-circuit a trivial query for C, reverting an earlier edit for T, ignoring returned data structure for M, persisting after tool failure for A, and not using self-knowledge for Mc.Each example links a trajectory failure to a specific criterion.Source: §6 · p. 14

Coding trajectory

  • In the coding case, overwriting and reintroducing an edit signals T and A failures, while reporting success without rerunning tests signals C and A failures.Per-call evaluation could score individual calls as reasonable and end-to-end evaluation would record only task failure; the five criteria localise the failure to Memory, Planning, and Control Flow.Source: §6 · p. 14

7 Relation to Existing Frameworks

The paper positions its five trajectory-level measurements as a complement to existing cognitive and agentic frameworks, not as a full theory of cognition. It connects architectural components, benchmark evidence, and rationality accounts to diagnostic measurements. [p11, p13, p14, p20, p27]

7.1 Cognitive Science Traditions

  • The criteria make a narrower operational claim than cognitive-science traditions: they specify what to measure and can be used across different views of cognition.Enactivist and embodied approaches overlap especially with context sensitivity and multimodal coordination.Source: §7.1 · p. 15
  • Cognitive architectures specify system components, while this framework measures whether those components are used coherently; the two approaches are complementary.The Common Model of Cognition maps memory, perceptual-motor systems, goal selection and planning, and the decision cycle to Memory, Tools, Planning, and Control Flow.Source: §7.1 · p. 15
  • Multi-component memory theory grounds temporal continuity in multiple stores, while global-workspace theory grounds multimodal coordination in shared broadcasting across processors.The account also links persona consistency to memory bound to a “working self” and identifies performance beyond the best single processor as an integration signature.Source: §7.1 · p. 15

7.2 AI Evaluation Literature

  • HELM and BIG-bench can supply prompts for five-criterion evaluation when scenarios are organized by cross-task properties and scored per criterion.This responds to critiques that benchmark performance can overstate cognitive capacity.Source: §7.2 · p. 15

7.3 Concurrent Work on Agentic Failure Taxonomies

  • Unlike failure taxonomies built from observed errors, this framework derives criteria from architectural components and uses benchmark tasks plus perturbations for per-criterion measurements.Related taxonomies and benchmarks cover overlapping subsets of the criteria.Source: §7.3 · pp. 15–16

7.4 Neurosymbolic AI for Agentic Systems

  • A verifier coupled to a generator can implement metacognitive monitoring, while planner coupling, structured memory, and constrained tool routing map to other criteria.The proposed fit remains an empirical prediction because purely neural agents could also satisfy the criteria.Source: §7.4 · p. 16

7.5 The Metacognitive AI Literature

  • The paper places metacognitive monitoring alongside the other four criteria and proposes measuring all five with one perturbation method.The cited literature argues that metacognition matters but is often missing from agentic-AI development.Source: §7.5 · p. 16

7.6 Rationality Theory

  • The profile maps bounded resources and rationality concepts onto agentic criteria, including context and history management, desire attribution through adaptive interaction, and self-monitoring through metacognitive monitoring.The paper also maps context sensitivity to choosing between deliberation modes and relates adaptive interaction, temporal continuity, and metacognitive monitoring to active-inference concepts.Source: §7.6 · pp. 16–17

8 Implications

The implications are to report criterion profiles alongside aggregate scores and use them to guide evaluation and architectural improvement. The paper also cautions that benchmark progress may not track trajectory-level progress.

8 Implications

  • A criterion profile alongside aggregate scores better supports diagnosing deployment failures and comparing agents that have the same aggregate score but fail on different workloads.The proposed perturbation benchmarks pair a base task with per-criterion variants and report a profile alongside end-to-end performance.Source: §8 · p. 17
  • A low criterion score can narrow improvement work to a likely component group, but precise localization requires architectural knowledge and trace evidence.For example, low adaptive interaction points toward Planning, while low temporal continuity points toward Memory.Source: §8 · pp. 17–18
  • Single-call benchmark gains may be partly decoupled from trajectory-level progress, and the paper identifies this as an empirical question to test.A rising MMLU score with an unchanged five-criterion profile is one stated example.Source: §8 · p. 18

8.1 Building for Agentic Cognitive Depth

  • The proposed design maps context sensitivity, temporal continuity, multimodal coordination, adaptive interaction, and metacognitive monitoring to targeted architectural changes.The mappings include a difficulty-aware controller, explicit state tracking, tool coordination, feedback-driven revision, and a monitor-controller; the object-level capacities interact through the world model.Source: §8.1 · p. 18

9 Discussion, Limitations, and Future Work

The discussion frames agentic evaluation as a system-level, trajectory-level problem and positions the profile between raw logs and aggregate scores. It also identifies open measurement caveats and proposes held-out, repeated perturbation benchmarks tied to trace evidence.

Discussion

  • Evaluation should use held-out tasks, repeated runs, and logs that distinguish changes in the model from changes in the surrounding agent system.The cited benchmark-validity concerns include cost reporting, holdout quality, reproducibility, and benchmark overfitting.Source: §9 · p. 18
  • Trajectory-level benchmarks expose failures that single-call evaluations and aggregate success rates can hide, while the proposed profile summarizes them between raw logs and final scores.Examples include tool-agent-user interaction, multimodal desktop tasks, and software tasks where the agent-computer interface affects performance.Source: §9 · p. 19

Limitations

  • The framework remains a diagnostic abstraction: criteria can overlap, joint sufficiency is open, root-cause analysis needs step-level evidence, and scoring requires further specification.Its architectural mapping is abstract because deployed systems mix prompts, retrieval, tools, controllers, memory, and human input.Source: §9 · p. 19

Future Work

  • Future benchmarks should use held-out tasks, logged trajectories, repeated trials, confidence intervals, and per-criterion pass rates and costs, then connect low scores to traces and interventions.The proposed testbeds include GAIA, SWE-bench Pro, WebArena, τ-bench, OSWorld, TRIP-Bench, AMA-Bench, and ARC-AGI-3.Source: §9 · p. 19

10 Conclusion

The conclusion presents agentic cognitive depth as a diagnostic framework for observable trajectories, organized around five criteria mapped to agent components. It proposes testing whether profile differences among similarly scoring agents predict deployment failures.

10 Conclusion

  • The profile measures context sensitivity, temporal continuity, multimodal coordination, adaptive interaction, and metacognitive monitoring across agent trajectories.The first four criteria map to Control Flow, Memory, Tools, and Planning; metacognitive monitoring adds a meta-level monitor and controller.Source: §10 · pp. 19–20
  • A proposed next experiment applies five perturbation classes to held-out benchmark samples and reports each profile beside end-to-end scores.The falsifiable hypothesis is that agents with similar aggregate scores will separate on profiles and that those differences will predict deployment failures.Source: §10 · p. 20

Related papers

Loading 2610.04168v1…