Source-linked AI summary
Agentic Large Language Models, a survey
Aske Plaat, Max van Duijn, Niki van Stein, Mike Preuss, Peter van der Putten, Kees Joost Batenburg
TL;DR
Agentic LLM research needs a synthesis of how language models can reason, act, and interact as agents. This survey organizes the literature by those three capabilities, finds that they mutually reinforce one another and generate additional training data, and outlines applications alongside unresolved safety, ethical, and legal issues.
Problem
LLM training data is becoming harder to scale, while useful agentic behavior requires reasoning, action, and interaction beyond passive language modeling.
Method
The survey organizes LLM-based agent research into reasoning, acting, and interacting, and develops a research agenda across these categories.
Results
The three categories complement one another: reasoning supports all categories, retrieval supports tool use, reflection supports interaction, and acting and interacting generate training data.
Takeaways & Limitations
Agentic LLMs support applications in medicine, logistics, finance, scientific research, and social simulation, while enabling continued improvement through interaction with the world.
Takeaways & Limitations
Real-world deployment remains constrained by unresolved safety, ethical, legal, security, responsibility, and liability questions.
Abstract
from arXiv · showhide
Background: There is great interest in agentic LLMs, large language models that act as agents. Objectives: We review the growing body of work in this area and provide a research agenda. Methods: Agentic LLMs are LLMs that (1) reason, (2) act, and (3) interact. We organize the literature according to these three categories. Results: The research in the first category focuses on reasoning, reflection, and retrieval, aiming to improve decision making; the second category focuses on action models, robots, and tools, aiming for agents that act as useful assistants; the third category focuses on multi-agent systems, aiming for collaborative task solving and simulating interaction to study emergent social behavior. We find that works mutually benefit from results in other categories: retrieval enables tool use, reflection improves multi-agent collaboration, and reasoning benefits all categories. Conclusions: We discuss applications of agentic LLMs and provide an agenda for further research. Important applications are in medical diagnosis, logistics and financial market analysis. Meanwhile, self-reflective agents playing roles and interacting with one another augment the process of scientific research itself. Further, agentic LLMs provide a solution for the problem of LLMs running out of training data: inference-time behavior generates new training states, such that LLMs can keep learning without needing ever larger datasets. We note that there is risk associated with LLM assistants taking action in the real world-safety, liability and security are open problems-while agentic LLMs are also likely to benefit society.
1 Introduction
Agentic LLMs extend language models with reasoning, autonomous action, interaction, and greater autonomy, enabling applications across society and science. This survey organizes the field around these capabilities, explains how they reinforce one another, and identifies data-generation opportunities and unresolved challenges.
- Agentic LLMs are increasingly used or proposed for medicine, logistics, finance, daily assistance, and large-scale social science simulations.
- The survey distinguishes reasoning, acting, and interacting, covering decision-making methods, assistants using tools and robots, and multi-agent simulations of social behavior.
- Agentic LLMs reason, make decisions, and take autonomous actions in their environment to achieve specific goals.
- Reasoning technologies support increasingly intelligent assistants, whose actions and social experiments can generate data for further pretraining and finetuning.
- The survey uses literature selected through Google Scholar searching and citation snowballing, while excluding multi-agent work without LLMs.
- Agentic methods address limitations including manual prompt improvement, hallucination, and training data that excludes later events or specialized information.
2 Reasoning
The survey presents reasoning as the foundation of agentic LLMs, using step-by-step reasoning, self-reflection, retrieval, formal languages, and search to improve decision making and generate training data.
- Reasoning: Reasoning methods improve decision making through step-by-step prompting, self-reflection, retrieval augmentation, and related inference-time techniques.The section covers math word problems, algorithm generation, question answering, and decision-making benchmarks.
- Chain of Thought: Chain of Thought prompts LLMs to follow intermediate reasoning steps, improving performance on complex reasoning problems.The approach reformulates information from the question into intermediate steps before producing the final answer.
- Chain of Thought: 10-20 percentage points: Self-Consistency typically improves Chain-of-Thought performance by sampling diverse reasoning paths and selecting the majority answer.The method addresses error accumulation in longer reasoning chains by evaluating multiple paths.
- Formal reasoning: Formal-language approaches generate algebraic expressions or Python code that specialized solvers, interpreters, or planners can evaluate.MathPrompter improved MultiArith performance from 78.7% to 92.5% using generated expressions or Python code.
- Search: Tree of Thoughts uses generation and evaluation prompts with an external search algorithm to explore, backtrack through, and compare reasoning paths.The external search can also generate training data; Stream of Search trains LLMs on search trees containing good and bad outcomes.
2.2 Self Reflection
Self-reflection methods use feedback, external memory, and iterative prompt improvement to help LLMs assess and refine their outputs, while some approaches internalize these processes through training.
- Self-reflection: Self-reflection uses external algorithms and memory to assess an LLM’s predictions and create improved prompts for subsequent answers.The process resembles reinforcement learning because feedback guides repeated improvement of the model’s behavior through prompts.
- Progressive Hint Prompting: Progressive Hint Prompting repeatedly combines the question and current answer until the answer stabilizes, interactively improving prompts.With GPT-4, PHP achieved 95% on grade-school math, 91% on simple math word problems, and 79% on algebraic question answering.
- Self Refine: Self Refine iteratively has an LLM generate an initial output, provide feedback, and refine the answer through repeated prompting.Its workflow uses separate prompting stages for generation, feedback, and refinement.
- Reflexion: Reflexion uses actor, evaluator, and reflector prompts so evaluated failures produce verbal reinforcement cues for self-improvement.The actor generates text and actions, the evaluator scores them, and the reflector supplies cues to assist the actor.
- Memory: External memory records traversed reasoning branches, prompts, evaluations, and interaction history between LLM invocations.Memory between prompts can allow individual LLMs to acquire experience and develop persistent preferences or personality.
- Implicit reasoning: Implicit reasoning integrates reasoning capabilities into the trained model, with inference-time-generated data used to augment supervised fine-tuning.Reinforcement learning can combine data generation and training in one loop, as illustrated by DeepSeek-R1-related work.
2.3 Retrieval Augmentation
Retrieval augmentation supplies timely or specialized information unavailable during pretraining, using indexed data, knowledge graphs, query optimization, and adaptive retrieval methods.
- Motivation: Retrieval augmentation addresses LLMs’ lack of timely or specialized information by retrieving data at inference time from external sources.Examples include stock data, recent hotel bookings, and specialized databases absent from the original training corpus.
- Retrieval methods: Most RAG methods index unstructured text, optionally organize it as a knowledge graph, and optimize queries through expansion or sub-query decomposition.These operations improve access to textual information and support more complex retrieval requests.
- Adaptive retrieval: Adaptive retrieval lets LLMs determine when to retrieve additional information, including through search-query initiation mechanisms.Graph Toolformer applies Self Ask techniques so the LLM can decide when extra information is needed.
- Limitations: Retrieval augmentation can be costly, motivating combinations of curated ground truth and synthetic data judged or self-evaluated by LLMs.Integrating time-sensitive unstructured and structured data remains an important research area for agentic LLMs.
2.4 Discussion
The survey presents reasoning, reflection, retrieval, and interpretability as foundations for improving agentic LLM decision making. It also emphasizes unresolved limitations in causal, abstract, common-sense, and self-reflective reasoning.
- Neurosymbolic approaches combine neural LLMs with inference-time reasoning, planning, and knowledge retrieval to improve decision making.
- Chain of Thought: Chain of Thought prompting increased in-context multi-step reasoning performance and supports agents facing real-world interaction and other agents.
- Self Reflection: Self-reflection enables agents to evaluate behavior, plan possible futures, improve prompts, and reason about other agents’ expected behavior.
- Self Reflection: Self-reflective agents remain difficult to debug because interacting prompts can produce unexpected behavior, while accumulated traces can exceed context windows.
- LLMs struggle with causal inference, abstract and counterfactual reasoning, combinatorial puzzles, and abstract common-sense tasks.
- Explainable AI and mechanistic interpretability aim to reveal how LLMs reach conclusions, supporting improvements such as reducing hallucinations.
3 Acting
The acting category studies how LLM agents interact with the world to become more useful to users. These actions also generate interactive training data for further model improvement.
- Acting research focuses on world knowledge, robotic actions, tool use, and application-oriented agentic assistants.
- Agent actions produce new interactive training data that can be used to train LLMs further.
3.1 Action Models
Action-model research extends LLMs beyond language by grounding them in world models, robotic actions, and multimodal action sequences. World models support sample-efficient learning, while VLA systems perform increasingly complex tasks in unseen situations.
- World models act as learned surrogates for complex environments, enabling more sample-efficient policy training through reinforcement learning.
- World models have supported robotic movement, Atari, open-world games, and LLM-based systems such as WorldCoder.
- World models can also represent planning domains such as PDDL blocks-world and support task-planning success in ALFWorld.
- Vision-language-action models are trained on robotic sequences to execute actions in visual scenes toward language-specified goals.
- VLA models achieve zero-shot generalization to unseen situations and complex tasks such as folding laundry.
3.2 Robots and Tools
Robots and tools ground LLM reasoning in environmental feedback, APIs, browsers, and computer interfaces. These systems expand language-driven autonomy, but tool-enabled agents introduce substantial safety and security risks.
- Robot Planning: Embodied agents must ground language understanding in the environment, available actions, and physical consequences to act reliably.
- Robot Planning: SayCan compares intermediate reasoning steps with feasible robot movements to reduce impossible actions and potential errors.
- Robot Planning: Inner Monologue inserts feedback from success detection, object recognition, scene description, and human interaction into prompts as linguistic environmental information.
- Robot Planning: Language feedback from Inner Monologue significantly improves performance across simulated and real tabletop rearrangement and kitchen manipulation tasks.
- Robot Planning: Language-driven zero-shot object navigation maps free-form descriptions to standard object labels without prior exposure to the environment or target object.
- Tools: Tool-using LLMs must learn when and how to call external APIs, with frameworks supporting structured instructions, API generation, and tool-use benchmarks.
- Tools: A planner, caller, and summarizer can divide reasoning, request writing, and result summarization among specialized LLMs.
- Computer and Browser Tools and Agent Interoperability: Computer and browser tools translate textual intents into grounded environment-level operations, expanding tasks and domains for language-driven autonomy.
3.3 Assistants
Agentic LLM assistants apply reasoning, retrieval, tools, and multi-agent coordination across conversational, domain-specific, financial, medical, and scientific tasks. Reported benefits include improved task completion, collaboration, diagnosis, trading, and research support, alongside risks in automated negotiation and medical outputs.
- Assistant types: Assistants span conversational, shopping, travel, medical, financial, and scientific domains, with designs specialized for tasks or working contexts.Some systems combine domain finetuning, retrieval augmentation, tool use, planning, memory, or multiple specialized LLMs.
- Conversational assistants: AssistantGPT combines an LLM with tools, a planner, and memory for web searches, API interactions, voice conversations, and shell commands.The system is intended for educational and corporate settings and for remote support.
- Domain assistants: Reported applications include improved meeting support, flight-booking efficiency, medical diagnosis and note generation, and collaborative task scheduling.Some medical studies also identify shortcomings in specific areas, while meeting testers found interventions unnatural or inflexible.
- Negotiation: Automated negotiation can introduce bias toward agreement, overspending, or unreasonable deals, making contract terms and outcomes potentially risky.These findings concern LLM-based negotiation and experimental analyses of automated negotiation.
- Science assistants: Science assistants increasingly support end-to-end discovery workflows and human research collaborations, including idea generation, experimentation, review, and laboratory validation.AI Scientist reports promising results but has limited experiments, incorrect implementations, and visual errors; AI Co-scientist and The Virtual Lab report real-world validation.
- Trading assistants: Financial assistants use multimodal market information, specialized trading roles, and tool augmentation to produce analysis or trading advice, with TradingAgents outperforming baseline models in simulation.TradingAgents assigns roles including fundamental, sentiment, and technical analysts, plus traders with varied risk profiles.
3.4 Discussion
The discussion presents agentic LLMs as tools that connect language reasoning to real-world actions, scientific workflows, and productivity gains. It also emphasizes grounding requirements and unresolved safety, ethical, legal, and scientific-autonomy questions before responsible deployment.
- Scientific workflows: Scientific agents can generate ideas, download datasets, run experiments, and write papers, but current systems still depend on hand-written, experiment-specific prompts and templates.The survey asks whether independent, creative, high-quality research can be automated and notes that prompt and hyperparameter settings matter.
- Scientific productivity: Agentic LLM tools provide tangible researcher productivity gains in literature analysis, idea generation, experiment setup, and writing, while collaborative systems support validated human research.The survey also notes increased pressure on traditional peer review from more papers being written and submitted.
- Action and grounding: Tools, robots, world models, VLAs, and API-connected assistants extend LLMs from reasoning toward concrete real-world action.Grounding helps agents understand surroundings and avoid impossible actions, while planning can imagine futures and prevent invalid commands.
- High-impact applications: Medical and financial assistants report strong performance, but accuracy, comprehensiveness, and the reliability of advice remain open concerns.The survey reports performance above human doctors in diagnosis tasks and frequent outperformance of human analysts in financial trading, while retaining these caveats.
- Deployment boundaries: Commercial deployment faces unresolved safety, ethical, legal, liability, and workforce questions when assistants provide advice or take actions in the world.The survey asks who is responsible for harmful medical advice or losses caused by trading recommendations and calls for further research before responsible use.
4 Interacting
The survey distinguishes interacting agents from passive LLMs by emphasizing memory, planning, reflective initiative, and interaction histories. It organizes this area around social capabilities, role-based teams, and open-ended agent societies.
- 4 Interacting: Agentic LLMs differ from traditional passive LLMs through memories, future interaction planning, and reflective loops that can initiate actions.These abilities open possibilities for studying interaction with users and other machine agents.
- 4 Interacting: The section reviews traditional social capabilities, pairs or small role-based teams completing tasks or experiments, and open-ended LLM societies with self-organizing behavior.The organization moves from established interaction abilities to structured multi-agent collaboration and less-constrained social interaction.
4.1 Social Capabilities of LLMs
Research on LLM social capabilities covers conversation, etiquette, empathy, strategic behavior, social dilemmas, and theory of mind. Results show varied strategic and social performance, with improvements on some tests but persistent dependence on models, context, and evaluation design.
- Social capabilities: Social-capability research has progressed from descriptive and anecdotal human-task adaptations toward more structured tests and benchmarks.Topics include conversation, etiquette, empathy, strategic behavior, and theory of mind.
- Etiquette and empathy: Additional contextual information or memory improves LLMs’ empathetic interaction beyond immediate conversational contexts.LLMs can detect affective and emotional states in language and incorporate them into interaction behavior.
- Strategic behavior: In repeated Prisoner’s Dilemma experiments, LLaMa2 and GPT3.5 were more cooperative and forgiving than humans below 30% opponent defection, whereas LLaMa3 was more exploitative.Models favored cooperation over defection mainly when opponent defection rates were low.
- Strategic behavior: LLMs perform relatively well when self-interest pays off but less well in coordination games, with strategic reasoning varying across models and contextual framing influencing choices.GTBench finds failures in complete deterministic games, competitiveness in probabilistic scenarios, and advantages for commercial over most open-source models in complex games.
- Social intelligence: EgoSocialArena reports that all tested models lagged humans by 11% on cognitive, situational, and behavioral intelligence, while model size did not significantly improve social intelligence.First-person perspective improved theory-of-mind reasoning in that study.
- Theory of mind: On a social-situational judgment task, several LLM chatbots outperformed average human scores and aligned well with expert ratings when proposing suitable behaviors.The comparison included five chatbots and 276 human participants.
- Theory of mind: Commercial LLMs are improving on standardized theory-of-mind tests, but high scores do not yet establish generalizable theory-of-mind capabilities.The survey notes that early emergence claims were less convincing and that further research is needed.
4.2 Role-Based Interaction
Role-based interaction assigns LLM agents distinct roles and objectives so they can coordinate, debate, or collaborate on strategic and problem-solving tasks. Studies report benefits from structured communication, while performance varies across games and social settings.
- Strategic Behavior: Strategic-game benchmarks evaluate LLM abilities including rationality, reasoning, cooperation, coordination, and deception.MAgIC evaluates seven features across social deduction and game-theoretic scenarios; GPT-o1 and GPT-4 score significantly better than other LLMs.
- Strategic Behavior: GAMA-Bench finds that GPT-3.5 is robust but has limited generalizability, while Chain of Thought and newer models improve performance.GPT-4 outperforms other models, and successive GPT-3.5 iterations show marked intelligence gains.
- Strategic Behavior: Alympics provides a controlled platform for complex strategic interactions, including multi-round allocation of scarce survival resources.Its water allocation challenge requires distributing scarce resources across multiple rounds.
- Role-Based Task Solving and Team Work: Role-based frameworks use complementary agents, predefined roles, and communication protocols to divide complex tasks and coordinate solutions.CAMEL uses assistant and user roles, inception prompting, and role descriptions to sustain collaborative dialogue without continuous human prompting.
- Role-Based Task Solving and Team Work: Multi-agent feedback and debate can improve reasoning, factual correctness, and final outputs by having agents critique, revise, or judge one another.Constitutional AI uses critique and revision, while society-of-minds debate reduces reasoning errors and hallucinations.
- Role-Based Task Solving and Team Work: Role-playing performance remains uneven: GPT-4 achieves a significantly lower goal-completion rate than humans in Sotopia and struggles with social commonsense and strategic communication.Other studies report that GPT-4o performed best in sustainability-oriented outcomes, while universalization reasoning helped more models reach sustainable outcomes.
4.3 Simulating Open-ended Societies
Open-ended societies use LLM agents to simulate social dynamics, emergent coordination, norms, and collective behavior at scales ranging from small groups to over 10,000 agents. These systems also expose limitations, including conformity, norm violations, and destructive outcomes.
- Simulacra and Societies: Open-ended multi-agent simulations study emergent phenomena such as self-organization, collective intelligence, social conventions, and norms without requiring fixed roles.Agentic LLMs provide perception, memory, reasoning, decision-making, adaptive learning, and heterogeneous personalities for these simulations.
- Simulacra and Societies: Generative Agents simulates a town of 25 LLM-based agents whose personas, memories, reflections, and plans shape their behavior.Each agent records experiences and conversations, synthesizes higher-level reflections, and plans subsequent actions.
- Simulacra and Societies: AgentSociety scales social simulation to over 10,000 agents and supports studies of polarization, inflammatory messages, policy effects, misinformation, and external shocks.In misinformation experiments, accuracy labels and blocking specific influencers proved effective particularly in scale-free networks, whereas encouraging comments did not significantly reduce spread.
- Open-World Agents: AgentVerse structures group problem solving into recruitment, collaborative decision making, action execution, and evaluation, with experiments confirming effectiveness across multiple tasks.The framework studies whether groups of expert agents can outperform the sum of their individual contributions.
- Emergent Social Norms: Interaction can produce both social regularities and risks: norms may emerge spontaneously, while agents can conform, lose individual positions, or generate destructive behavior.Studies report human-like conformity and consensus, but also detrimental outcomes and increased error risk under debate-promoting instructions.
- Open-World Agents: Large-scale platforms such as OASIS model up to one million agents and reproduce social phenomena including misinformation, polarization, and herd effects.OASIS combines an environment server, recommender system, agent module, time engine, and scalable inferencer.
4.4 Discussion
The discussion connects multi-agent interaction to collaborative reasoning, social-science simulation, and new training data. It highlights both demonstrated capabilities and practical constraints, including unstable training and harmful emergent behavior.
- Interaction Studies: Multi-agent interaction can improve task performance when reasoning, predefined roles, protocols, feedback loops, or debate moderation coordinate agent contributions.The survey reports benefits across strategic games, collaborative reasoning, and role-based task solving, while noting varying performance.
- In Depth: CAMEL and Generative Agents: CAMEL uses assistant and user agents for role-playing problem solving, with experiments spanning 25,000 conversations across ten tasks.The system combines 50 agents and 50 users, with conversations depth-limited to 40 messages.
- In Depth: CAMEL and Generative Agents: CAMEL improved HumanEval performance from 30% to around 50% relative to a single prompt.The survey attributes stable learning to attention to constructive communication, while reporting repetition, fake replies, empty replies, loops, and role reversal as training problems.
- In Depth: CAMEL and Generative Agents: Generative-agent simulations use memory, planning, and reflection to support believable behavior amid accumulating interactions and cascading social dynamics.Evaluation probes memory, future planning, reactions to unexpected events, and reflection across alternative architectures.
- In Depth: CAMEL and Generative Agents: Synthetic generative agents replicated participants’ General Social Survey responses with 85% accuracy and performed comparably in predicting personality traits and experimental outcomes.Human participants’ answers across two interviews provided the comparison control for natural variation.
- Discussion: Emergent cooperation and social interaction simulations enable what-if studies of competition, information spread, disasters, democratic processes, and cybersecurity.The survey presents these simulations as a way to investigate social behavior under artificial conditions and changing circumstances.
- New Training Data: Interacting agents can generate new training states, but reinforcement-learning feedback loops remain unstable and may require diverse exploration, low learning rates, and substantial computation.Open-ended multi-agent simulation is presented as a possible alternative for creating diversity needed for stable convergence.
5 General Discussion and Research Agenda
The survey identifies applications, research opportunities, and risks for agentic LLMs, emphasizing a cycle in which agents generate data that supports continued learning. It also proposes research directions spanning self-reflection, distillation, emergent behavior, and safety.
- Applications: Agentic LLMs are attracting interest for applications including medical diagnosis, logistics, finance, and scientific research.The survey also notes potential effects on work processes in these fields.
- Training data: Inference-time interaction can generate data for augmentation, pretraining, or finetuning, addressing concerns about plateauing training corpora.The survey describes reuse of inference-time results as a feedback loop for continued improvement.
- Research agenda: The research agenda includes learning from agents’ own results, with hallucination, mechanistic interpretability, and open-world models identified as important topics.The survey specifically highlights the need for methods that support agents learning from their own outcomes.
- Research agenda: Knowledge distillation can transfer reasoning steps from large language models to smaller models at lower computational cost.The survey presents distillation as relevant to the scale and training-cost challenges of agentic LLMs.
- Research agenda: Studies of emergent cooperation and trust in agentic societies may inform views of intelligence and discussions about artificial super-intelligence.The survey frames large-scale agent behavior as relevant to understanding when cooperation and trust emerge.
- Safety: Safety remains unresolved because assistants’ and robots’ real-world actions raise responsibility, liability, privacy, fairness, and ethical questions.The survey identifies safety as crucial and says the problem is far from solved.
- Conclusion: The taxonomy’s reasoning, acting, and interacting categories complement one another, while agentic learning uses environmental feedback and is not limited by dataset complexity alone.Reasoning underlies tool use and robotics, self-reflection supports social interaction, and agents can learn from their own actions and world feedback.
6 Reproducibility Checklist for JAIR
The JAIR reproducibility section provides checklist items for reporting claims, methods, experiments, datasets, and theoretical contributions. For this paper, the checklist records no computational experiments, no dataset reliance, and no theoretical contributions.
- General checklist: The checklist asks authors to select applicable items for their research and explain how reported claims are substantiated.It also asks whether limitations or technical assumptions are stated clearly and explicitly.
- Theoretical contributions: The paper is marked as making no theoretical contributions.The theoretical checklist would cover assumptions, formal claims, proofs, motivation, notation, and citations for theoretical tools.
- Computational experiments: The paper is marked as containing no computational experiments.The checklist therefore includes no experiment-specific results to reproduce.
- Computational experiments: For experimental articles, the checklist covers code availability, licensing, raw data, random seeds, computing environments, metrics, runs, parameter settings, and statistical tests.These items are presented as reproducibility criteria rather than reported experimental findings in this paper.
- Datasets: The checklist is marked as not applicable to dataset-based work because the paper does not rely on one or more datasets.Dataset criteria include availability, licensing, citations, description, preprocessing, augmentation, batching, and splitting.