Exploring Autonomous Agents through the Lens of Large Language Models: A Review

1 Introduction

• Automation can leave some tasks requiring human intervention, while its productivity gains have also coincided with worker marginalization and greater wealth inequality.The introduction presents these as societal concerns alongside automation’s economic effects.Source: §1 · p. 3
• Prompt tuning selects or adjusts prompts to guide generated text, and the way people query LLMs can shape their responses.The introduction also connects LLM learning from data to research on human cognition.Source: §1 · p. 3
• LLM agents can support varied applications, but embodied robotic tasks challenge text-only models because they lack compatible visual perception.The passage describes combining LLMs with multimodal models as a solution for robotic tasks.Source: §1 · p. 4
• The review examines integrating LLMs into autonomous agents, including approaches and hurdles in frameworks such as AutoGPT and LangChain.It also discusses open-source models as a route to wider tool access, transparency, and innovation.Source: §1 · p. 4

2.1 Transformer Architecture

• Transformers use self-attention and parallel sentence processing, unlike sequential RNNs; Figure 1 presents the Transformer architecture.Encoder-decoder models generate outputs from encoded inputs, encoder-only models produce representations for prediction, and decoder-only models predict subsequent tokens.Source: §2.1 · p. 5, Figure 1 · p. 6

2.2 The Evolution of Large Language Models

• LLM development accelerated with deep learning and Transformers, which improved natural-language-task performance over the preceding RNN generation.The section also describes GPT-4, LLaMa 2, T5, BART, and RoBERTa and their differing capabilities.Source: §2.2 · pp. 5–6
• Model limitations vary: T5 can falter on out-of-distribution examples, BART is limited to 512-token sequences, and RoBERTa comparisons face cost and dataset constraints.The passage also notes the influence of hyperparameter choices on RoBERTa comparisons.Source: §2.2 · p. 7
• Vision-agent research includes AVIS, which uses an LLM to plan external-tool use and inspect outputs for visual question answering.The section also surveys vision-language applications in autonomous driving and intelligent transportation systems.Source: §2.2 · pp. 7–8
• Speech-agent work includes LLaSM for speech-and-language instructions and an LLaMA-7B system with a conformer encoder that outperformed monolingual baselines by 18%.The section also discusses experiential learning without parametric updates and calls for explainability in autonomous agents.Source: §2.2 · p. 8

3.1 All About Memory

• Transformer self-attention makes memory needs grow quadratically with input length; PagedAttention manages dynamic key-value caches, and MemGPT treats context windows as a constrained tier in a memory hierarchy. [47][48]The key-value cache for each request can be substantial and fluctuate in size; the hierarchy is intended to support continuity in nuanced contextual reasoning.Source: §3.1 · p. 9

3.2 From LLM with Autonomous Agents

• LLM agents can use tools to address task-execution limits that multimodality alone cannot solve, while LangChain combines tools and memory to select actions and retain context.LLMs may struggle with numerical or physical reasoning, and hardcoded action sequences restrict adaptability; LangChain agents use an LLM to decide which tools to invoke. [19]Source: §3.2 · p. 12
• Auto-GPT decomposes natural-language goals into subtasks and uses tools autonomously, but can fall into logic loops or “rabbit holes”; LangChain is a development library, not an autonomous agent. [20]Auto-GPT aims to pursue goals without user input, whereas LangChain provides interfaces and backend support for building LLM applications.Source: §3.2 · p. 13

3.3 The Art of Reasoning and Acting

• Direct prompting can produce hallucinations when LLMs lack recent data or domain knowledge, motivating reasoning steps and retrieval that grounds responses in external documents. [62]RAG retrieves contextual documents using a pre-trained neural retriever and dense vector index; retrieval instructions may be rephrased or expanded to improve performance.Source: §3.3 · pp. 14, 16, 17
• Chain-of-Thought and HuggingGPT support intermediate reasoning, while multipath methods, Tree of Thoughts, and Graph of Thoughts enable alternative reasoning paths or structured dependencies.HuggingGPT plans tasks and selects AI models for subtasks; MPR uses earlier answers as hints, and GoT represents thoughts as graph vertices connected by dependencies. [60][61][63][64][66]Source: §3.3 · pp. 14–15
• ReAct and Reflexion use environmental feedback to revise plans or internal world models, while human feedback informs alignment and retrieval-augmented generation grounds responses in external data.RLHF trains a reward model from human feedback; DPO instead optimizes a policy directly from binary preferences without fitting a reward model. RAG combines parametric model memory with a non-parametric document index. [67][68][69][70]Source: §3.3 · pp. 15–17

3.4 Prompting is all you need

• Automatic Reasoning and Tool Use improves on few-shot prompting and automatic Chain-of-Thought on unseen BigBench and MMLU tasks, while APE selects instructions using target-model evaluation scores. [73]Active-Prompt steers the model toward desired outputs, and Directional Stimulus Prompting adds fine-grained guidance and control. [74][75]Source: §3.4 · p. 18
• Prompt tuning adapts pretrained models by optimizing a soft prompt rather than model weights, and prefix-tuning learns 0.1% of parameters while matching full-data performance and outperforming fine-tuning in low-data settings.The passage also reports that prompt-based redeployment can reduce computing and energy use by at least 1,000 times; prefix-tuning extrapolates better to topics unseen during training. [77][78]Source: §3.4 · p. 19
• Diversity-Aware prompting clusters downstream data into homogeneous subsets and separately optimizes each subset’s prompt, mitigating optimization difficulty and improving performance.The strategy is initialized by a Meta-prompt and addresses dataset diversity and distribution shift.Source: §3.4 · pp. 19–20
• Pseudocode and code snippets [81] can clarify algorithm logic, while UML diagrams [82] can convey system architecture to multimodal models for code generation.The text also cites diagram-based code generation [83] and identifies Figure 4 as AlphaCode’s approach.Source: §3.4 · pp. 20–21, Figure 4 · p. 20
• For visual question answering, Img2Prompt [85] supplies image descriptions and self-constructed question-answer pairs, while retrieval-augmented visual prompts provide knowledge-base information to the LLM.Img2Prompt is described as LLM-agnostic; Figure 5 presents MedFuseNet’s VQA approach.Source: §3.4 · pp. 21–22, Figure 5 · p. 21
• For long conversations, summarizing or filtering prior dialogue, StreamingLLM [87], and ConversationChain memory help maintain context despite fixed context lengths and stateless defaults.In-context learning [84] also uses context to adapt responses to new tasks and situations.Source: §3.4 · pp. 21–22
• Prompting can support common-sense reasoning with generated knowledge [89] and enrich generation with retrieved external material; instructions can also specify formats, examples, step-by-step reasoning, or debate.The section also describes RAG as retrieving relevant context to enrich generated content.Source: §3.4 · pp. 22–23
• Constructive prompts can generate synthetic training and testing data when real-world data is scarce; diversity can be increased through bootstrapping, varied sources, and augmentation [91, 92].The passages describe evaluating dataset diversity and adjusting it as needed.Source: §3.4 · pp. 22–23
• Responsible prompting requires safeguards against prompt injections and harmful requests [93], including robust internal prompts, lower temperature, higher frequency penalty [94], and reasonable input-length limits.The text recommends regularly reviewing and updating safeguards.Source: §3.4 · pp. 23–24

• Evaluation checks whether agents perform as intended across foreseen and unforeseen conditions.Source: §4 · p. 24
• Evaluation helps developers identify performance issues, monitor agents, and make adjustments to improve effectiveness.Source: §4 · p. 24
• Demonstrated reliability and accuracy can foster user trust, especially for agents deployed in high-stakes contexts.Source: §4 · p. 24

4.1 Traditional Evaluation Frameworks

• Traditional frameworks assess specific tasks or objectives in controlled environments using benchmark tasks or datasets [109].Source: §4.1 · p. 24
• Simulation environments provide another way to test agent performance under varied conditions and scenarios [110].Source: §4.1 · p. 24
• Traditional frameworks may miss dynamic operating environments and complex agent-environment interactions.Source: §4.1 · p. 25
• Newer frameworks extend evaluation through development and operation as systems adapt, while addressing hierarchical integration and limited testing resources [108].Source: §4.1 · p. 25

4.2 AgentBench: An All-Encompassing Evaluation Framework

• AgentBench evaluates reasoning and decision-making in multi-turn, open-ended settings across eight environments, including five newly created environments and three recompiled from published datasets.The environments include operating systems, databases, knowledge graphs, digital card games, lateral-thinking puzzles, household tasks, web shopping, and web browsing.Source: §4.2 · p. 25

4.2.1 LLMs Performance in AgentBench

• Poor long-term reasoning, decision-making, and instruction-following were the primary hurdles identified across tests of 27 API-based and open-source LLMs.Top commercial models showed a significant performance gap over their open-source counterparts.Source: §4.2.1 · pp. 25–26

4.2.2 Enhancing Agent Performance

• The authors propose code training and high-quality multi-turn alignment data to enhance agent performance, and release datasets, environments, and an integrated evaluation package.Code training is proposed for environments such as OS and DB, while multi-turn alignment data is proposed to support conversational context and decisions.Source: §4.2.2 · p. 26

4.2.3 Comparison with Traditional Evaluation Frameworks

• Unlike isolated-task frameworks, AgentBench assesses LLM agents across diverse, dynamic environments and interactions.The review presents this broader setting as a way to evaluate autonomous-agent capabilities and identify areas for improvement.Source: §4.2.3 · p. 26

4.3 WebArena: A Novel Environment for Autonomous Agents

• WebArena[116] is a realistic, reproducible environment for language-guided agents, intended to bridge simplified synthetic tests and complex real-world scenarios.It includes functional websites across e-commerce, social forums, collaborative software development, and content management, plus tools and external knowledge bases.Source: §4.3 · pp. 26–27

4.3.1 Benchmark Tasks and Evaluation

• 14.41% end-to-end task success for the best GPT-4-based agent, compared with 78.24% for humans on WebArena tasks.The benchmark evaluates functional correctness on diverse, long-horizon tasks designed to emulate routine internet activities.Source: §4.3.1 · p. 27

4.3.2 Comparison with Traditional Approaches

• WebArena contrasts with synthetic agent-development settings by providing functional websites and tools in a setting that mirrors routine human internet tasks.The review says this supports broader evaluation of agent performance and capabilities.Source: §4.3.2 · pp. 27–28

4.4 ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

• ToolLLM[117] introduces a tool-use framework spanning data construction, model training, and evaluation to help LLMs use APIs for human instructions.The review identifies limited open-source tool use and instruction tuning focused on basic language tasks as the motivating gap.Source: §4.4 · p. 28

4.4.1 Data Construction, Model Training, and Evaluation

• ToolBench is automatically constructed with ChatGPT from 16,464 RESTful APIs across 49 categories, generated single- and multitool instructions, and annotated API-call solution paths.The authors also describe a depth-first-search decision-tree algorithm, ToolEval, and ToolLLaMA fine-tuned from LLaMA with a neural API retriever.Source: §4.4.1 · p. 28

4.4.2 Performance of ToolLLaMA

• ToolLLaMA demonstrated complex-instruction execution, generalization to unseen APIs, and performance comparable to ChatGPT, with strong zero-shot generalization on APIBench.Source: §4.4.2 · p. 29

4.4.3 Comparison with Traditional Approaches

• The review describes ToolLLM as diverging from traditional approaches by offering a realistic, reproducible setting for developing and testing agents.It says this setting supports more comprehensive evaluation and identification of current LLM challenges and limitations.Source: §4.4.3 · p. 29

• Human annotation assesses LLM outputs against criteria such as relevance, coherence, and fluency, but can be time-consuming, expensive, and subject to annotator bias.Source: §4.5 · p. 29
• The Turing Test gauges human-like conversational behavior, but offers limited insight into specific strengths and weaknesses and does not establish human-like comprehension.Source: §4.5 · p. 30
• User studies assess usability and satisfaction in real-world scenarios, while expert reviews provide nuanced assessments, especially in specialized domains.Source: §4.5 · p. 30

5.1 Multimodality: A Double-Edged Sword for LLM-Based Autonomous Agents

• Multimodal inputs strain computation, can produce inconsistent interpretations, and are difficult to learn from when comprehensive training datasets are scarce.These limitations can leave agents less adept at real-world scenarios involving intricate multimodal inputs.Source: §5.1 · pp. 30–31
• Efficient processing, diverse multimodal training data, careful feature design, and gradual task expansion with continuous testing are proposed to address multimodality challenges.The section links gradual scope expansion and refinement with maintaining agent capability and reliability as tasks grow more complex.Source: §5.1 · p. 31

5.2 Human Alignment in LLM-Based Autonomous Agents

• Misread instructions, training-data biases, and factually incorrect or semantically meaningless output can misalign agents with human expectations and undermine effectiveness or trust.The section also notes that such outputs can confuse users and contribute to misinformation.Source: §5.2 · pp. 31–32
• More representative training data, user feedback, and ongoing monitoring and evaluation are proposed to identify and address alignment problems.The section suggests using metrics that specifically measure alignment and regular review of agent actions.Source: §5.2 · p. 32

5.3 The Enigma of Hallucinations

• Hallucinations can produce misleading, illogical, inconsistent, biased, or inappropriate content, compromising agent efficiency and user trust.The section also connects fabricated information with user confusion and potentially misinformed decisions.Source: §5.3 · pp. 32–33
• Training-data improvements, user feedback, and continued monitoring and evaluation are proposed as ways to address hallucinations.The section mentions metrics for hallucination rates and regular review and adjustment of agent behavior.Source: §5.3 · p. 33

5.4 The Intricacies of Agent Ecosystems

• Complex agent ecosystems can be computationally demanding, produce inconsistent interpretations, and remain difficult to understand when comprehensive training datasets are scarce.These constraints affect agents’ handling of real-world scenarios with complex, multi-agent inputs.Source: §5.4 · p. 33
• The section proposes standardized communication, defined agent roles, modular design, diverse training data, continual adaptation, rigorous testing, explainability, bias mitigation, and human oversight.These practices are presented as ways to support information exchange, maintainability, trust, and ethical control across the ecosystem.Source: §5.4 · p. 34

• LLM-powered agents can comprehend and generate human-like text and are described as having potential across domains including customer service and healthcare.Source: §6 · p. 34
• Key challenges include multimodal performance, alignment with human ethics and values, hallucinations, and evaluation in complex real-world scenarios.The conclusion says conventional metrics such as accuracy or precision may not fully capture agent capabilities.Source: §6 · p. 35
• Prompting, reasoning, tools, and in-context learning are under exploration, while AgentBench, WebArena, and ToolLLM assess performance in complex scenarios.The conclusion anticipates agents assisting with tasks ranging from email responses to disease diagnosis.Source: §6 · p. 35