Source-linked AI summary
Cognitive Models and AI Algorithms Provide Templates for Designing Language Agents
Ryan Liu, Dilip Arumugam, Cedegao E. Zhang, Sean Escola, Xaq Pitkow, Thomas L. Griffiths
TL;DR
Difficult tasks can exceed a single LLM, while the large space of multi-agent architectures makes effective design challenging. The paper formalizes agent templates and surveys language agents instantiated from cognitive models and AI algorithms. It concludes that these templates provide more interpretable structures for modular language-agent design.
Problem
Single LLMs cannot handle some difficult tasks, but combining multiple LLMs is challenging because the architecture space is large and practitioners often lack resources for exhaustive search.
Method
The paper defines agent templates as modular specifications and surveys existing language agents whose designs instantiate cognitive models or AI algorithms.
Results
The surveyed templates span modular designs such as decomposition, specialized-model orchestration, and policy iteration, demonstrating concrete cognitive-model and algorithmic structures in language agents.
Takeaways & Limitations
Cognitive models and AI algorithms provide reusable blueprints for developing effective and interpretable modular language agents.
Takeaways & Limitations
Many divide-and-conquer approaches predate modern reasoning models and mature long-running multi-turn agents.
Abstract
from arXiv · showhide
While contemporary large language models (LLMs) are increasingly capable in isolation, there are still many difficult problems that lie beyond the abilities of a single LLM. For such tasks, there is still uncertainty about how best to take many LLMs as parts and combine them into a greater whole. This position paper argues that potential blueprints for designing such modular language agents can be found in the existing literature on cognitive models and artificial intelligence (AI) algorithms. To make this point clear, we formalize the idea of an agent template that specifies roles for individual LLMs and how their functionalities should be composed. We then survey a variety of existing language agents in the literature and highlight their underlying templates derived directly from cognitive models or AI algorithms. By highlighting these designs, we aim to call attention to agent templates inspired by cognitive science and AI as a powerful tool for developing effective, interpretable language agents.
1. Introduction
Designing effective language agents is difficult because the architecture search space is large, while real-world practitioners often lack the data and resources for prolonged iteration. The paper argues that cognitive models and AI algorithms provide reusable templates for modular, interpretable agent design.
- Language-agent designs specify the roles LLMs should play and how they should interact, but the large architecture search space makes brute-force exploration infeasible.
- High-stakes practitioners often lack the data and resources needed to iterate extensively over possible agent designs, motivating effective design heuristics.
- The paper argues that cognitive models and existing AI algorithms provide templates for designing language agents.
- These templates organize distinct processes into carefully ordered, sequential operations for solving problems.
- The paper surveys agent architectures as instantiations of cognitive models or AI algorithms, spanning communication, planning, and efficient exploration.
- An explicit agent-template definition turns an agent’s goal into a pipeline for breaking down and completing tasks, increasing interpretability and leveraging previously successful strategies.
2. Background
Language agents address complex tasks but create a large architecture-design space that practitioners rarely search systematically. The paper situates its template approach among related modular frameworks and emphasizes concrete connections to cognitive models and AI algorithms.
- Language agents are LLM-based autonomous systems that follow language instructions across diverse tasks in real or simulated environments.
- Because complex agents contain many components and implementation choices, the number of possible architectures rises accordingly.
- Practitioners commonly rely on folk intuition, manual design, or chance rather than searching the full architecture space.
- Simulating human data is one response to limited architecture-search data, but such data may not represent an adequate distribution of human rewards.
- Related frameworks include agentic context engineering, compressor-predictor systems, CoALA, compound AI systems, and GPTSwarm.
- Unlike the paper’s concrete connections to cognitive models and AI algorithms, some nearby frameworks formulate agent design as an optimization problem or leave construction details for future work.
3. Defining templates for language agents
The paper formalizes an agent template as a directed acyclic graph of LLMs and tools that determines execution order and data flow. This formalism supports modular designs with parallel roots, dependency-ordered modules, and terminal outputs.
- The paper introduces a formal definition of a language-agent template to provide common structure for analyzing cognitive models and AI algorithms.
- A language model maps an input prompt to a distribution over possible output languages, while tools provide information or functionality whose outputs are textualized.
- An agent template is a directed acyclic graph G = (V, E) whose vertices are LLMs or tools and whose edges connect modules.
- Root vertices have no incoming edges, whereas terminal vertices have no outgoing edges and determine the template’s outputs.
- Root modules may execute in parallel, while each subsequent module waits until outputs from all its parent vertices are available.
- The graph is assumed to be acyclic and weakly connected to guarantee a sensible execution path.
- Templates can carry stateful information or metadata in addition to ordinary language inputs and outputs.
4. Templates based on cognitive models
The paper surveys cognitive-model-inspired language agents for communication, reasoning and planning, representation, and related reasoning practices. These designs translate cognitive processes into modular LLM pipelines, with reported gains in communication and planning applications.
- Templates based on cognitive models: Cognitive models are mechanistic hypotheses about mental processes, tested against human inputs, outputs, and collected behavioral data.
- Communication: Rational Speech Acts models communication as rational choice among utterances with utilities, using recursive inference about speakers’ and listeners’ beliefs.
- Communication: Liu et al.’s communication agent sequentially advises, generates utterances, profiles audiences, simulates reactions, and aggregates candidates within an RSA-inspired template.
- Communication: The RSA-based communication systems reported outperforming baselines and ablations according to human judgments, while another system significantly improved performance across a wide range of LLMs.
- Reasoning and planning: Cognitive reasoning work includes verbal protocols, and long-chain-of-thought reasoning models can be viewed as using a single reasoning-module template.
- Reasoning and planning: The Modular Agentic Planner uses six LLM modules for decomposition, action proposal, monitoring, prediction, evaluation, and coordination, inspired by prefrontal-cortex functions.
- Reasoning and planning: MAP combines these modules with tree search and achieves strong results on multi-step benchmarks including Tower of Hanoi and PlanBench.
- Representation: Programming-language representations use a reasoner to generate code and an interpreter to execute it, producing gains in mathematical, financial, symbolic, and interactive tasks.
5. Templates from AI Algorithms
The paper presents classic AI and reinforcement-learning algorithms as tested templates for decomposing, searching, evaluating, and improving language-agent behavior. These templates map algorithmic processes onto coordinated LLMs and tools, with reported benefits for reasoning, exploration, and decision-making.
- Templates from AI Algorithms: Classic AI algorithms provide tested solutions to problems faced by language agents and are organized here into classic search and reinforcement-learning templates.The analysis covers search, divide-and-conquer, policy iteration, posterior sampling, and information-directed sampling.
- 5.1.1. SEARCH: Search templates orchestrate language modules for node expansion and evaluation, interpreting chain-of-thought as a form of greedy best-first search.Tree of Thoughts makes this process explicit with separate generator and evaluator LLMs, while related work instantiates A∗ search with multimodal models.
- 5.1.1. SEARCH: Monte-Carlo Tree Search structures each iteration as selection, expansion, simulation, and backpropagation across a search tree.The algorithm supplies a sequential template for modeling search decisions and propagating value information.
- 5.1.2. DIVIDE AND CONQUER: Divide-and-conquer templates decompose problems into subproblems, solve them, and aggregate answers, often recursively.Least-to-Most uses decomposer and solver modules; Decomposed Prompting adds an aggregator and can incorporate tools, with applications spanning reasoning, code, image, and virtual-environment tasks.
- 5.1.2. DIVIDE AND CONQUER: HuggingGPT extends divide-and-conquer by assigning controller-generated subtasks to specialized models such as vision and speech systems.The controller summarizes the resulting responses, supporting multimodal tasks.
- 5.2.1. POLICY ITERATION: In-Context Policy Iteration implements policy iteration with policy, transition, and reward LLMs that estimate the environment and iteratively select actions.Across six small-state MDPs, ICPI performed as well as or better than tabular Q-learning, a rule-based trajectory policy, and a random policy.
- 5.2.2. POSTERIOR SAMPLING FOR RL: PSRL templates preserve efficient exploration in language agents, including difficult natural-language tasks such as Wordle without assuming that model-estimation data was already collected.The reported evidence uses cumulative regret curves and contrasts this setting with ICPI’s data assumption.
- 5.2.3. INFORMATION-DIRECTED SAMPLING: In a preliminary numerical Wordle study, IDS and PSRL language-agent templates preserved the expected ordering, while IDS more closely approximated the Bayes-optimal policy.The IDS evidence was limited to a single domain, leaving its broader effectiveness unresolved.
6. Discussion
The discussion argues that cognitive models and AI algorithms offer interpretable, reusable design templates rather than one universally best agent architecture. It recommends using these established templates as priors while investigating where new designs are warranted.
- Discussion: Template-based designs improve interpretability over arbitrary language-agent architectures and have already appeared in existing systems.The paper frames this as evidence supporting further exploration of the approach.
- 6.1. Alternative Views: A single best template is unlikely to suit all tasks because different problems call for different inductive biases.The paper invokes the No Free Lunch Theorem and emphasizes a framework containing multiple problem-template pairings.
- 6.1. Alternative Views: Even if LLMs represent a paradigm shift, existing cognitive and AI templates remain a prudent starting point for mapping agent designs and locating the frontier for genuinely novel templates.The paper notes that examined sequential-decision agents remain reinforcement-learning agents operating in Markov decision processes.
- 6.1. Alternative Views: Automatically discovered designs may sacrifice interpretability and prior knowledge about the relative performance of competing templates.The paper suggests cognitive-science and AI-inspired templates can function as priors for subsequent design evolution and discovery.
- 6.1. Alternative Views: Deep learning’s convergence on staple architectures illustrates how structured design manifolds can provide reliable starting points without guaranteeing optimality for every problem.The cited building blocks include convolutional layers, GRUs, LSTMs, residual blocks, and Transformers.
- 6.2. Call to action: The paper concludes that cognitive models and AI algorithms are effective templates, with many already applied to successful language-agent designs.It calls for researchers to investigate additional models and algorithms as sources of templates.
- 6.2. Call to action: Future templates may come from hypothesis generation, information theory, evolutionary algorithms, voting, mechanism design, game theory, and multi-agent research.The paper also suggests that language agents could contribute to richer or more general cognitive models.