Source-linked AI summary
Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agents
Arunkumar V, Gangadharan G. R., Rajkumar Buyya
TL;DR
Agentic AI research lacks a unified architecture-focused view of increasingly diverse systems and their deployment risks. This paper proposes a taxonomy and engineering-oriented analysis of agent architectures, coordination, environments, evaluation, and safety, concluding that controllable workflow graphs and explicit handoffs are central to safer multi-agent systems. It also identifies hallucination in action, infinite loops, prompt injection, and real-world reliability as continuing boundaries.
Problem
The field contains diverse agent designs, while unconstrained deployment remains limited by reliability, efficiency, and alignment challenges.
Method
The paper develops a unified architecture-focused taxonomy and analyzes agent control loops, engineering components, multi-agent coordination, environments, evaluation, and safety.
Results
The paper concludes that multi-agent systems are moving toward controllable workflow graphs and explicit handoff patterns that improve observability, debuggability, and safety through flow engineering.
Takeaways & Limitations
Reliable Agentic AI requires attention to orchestration, verification, evaluation, and governance alongside model reasoning and tool use.
Takeaways & Limitations
Agentic systems remain vulnerable to hallucinated actions, cascading errors, repetitive loops, and prompt injection when connected to executable tools or untrusted content.
Abstract
from arXiv · showhide
Artificial Intelligence is moving from models that only generate text to Agentic AI, where systems behave as autonomous entities that can perceive, reason, plan, and act. Large Language Models (LLMs) are no longer used only as passive knowledge engines but as cognitive controllers that combine memory, tool use, and feedback from their environment to pursue extended goals. This shift already supports the automation of complex workflows in software engineering, scientific discovery, and web navigation, yet the variety of emerging designs, from simple single loop agents to hierarchical multi agent systems, makes the landscape hard to navigate. In this paper, we investigate architectures and propose a unified taxonomy that breaks agents into Perception, Brain, Planning, Action, Tool Use, and Collaboration. We use this lens to describe the move from linear reasoning procedures to native inference time reasoning models, and the transition from fixed API calls to open standards like the Model Context Protocol (MCP) and Native Computer Use. We also group the environments in which these agents operate, including digital operating systems, embodied robotics, and other specialized domains, and we review current evaluation practices. Finally, we highlight open challenges, such as hallucination in action, infinite loops, and prompt injection, and outline future research directions toward more robust and reliable autonomous systems.
Impact Statement
Agentic AI shifts systems from conversation partners to active collaborators that complete tasks end to end. The paper examines architectures for complex workflows and identifies risks and design goals for robust deployment.
- Agentic AI enables systems to carry out tasks end to end as active collaborators rather than only conversation partners.
- The paper investigates LLM architectures for complex workflows in software engineering, scientific discovery, and robotics.
- Key risks include prompt injection and hallucination in action.
- The paper proposes a roadmap toward autonomous systems that are robust, secure, efficient, and suitable for open-ended real-world environments.
1 Introduction
The introduction frames Agentic AI as systems that actively change their environments and argues that reliable deployment depends on controllable orchestration, engineering-focused architecture, and explicit safety evaluation. The paper organizes these concerns through a unified taxonomy and connects them to current risks and multi-agent workflows.
- Agentic AI extends generative systems by enabling perception, reasoning, and action that actively change an environment.
- Current foundation models offer stronger inference-time reasoning, structured tool use, and multimodal interaction, but base chat systems still lack robust long-horizon state and permissions.
- Controllable orchestration uses explicit state transitions and guardrails so models make local decisions within debuggable, checkpointed workflows.
- Deploying action-capable agents introduces concrete failures from hallucinations and security threats from indirect prompt injection through untrusted content.
- The paper proposes a six-dimensional architecture-focused taxonomy covering components, cognition, learning, multi-agent systems, environments, and evaluation.
- Its engineering perspective emphasizes memory, computer use, code as action, MCP connector layers, and orchestration controllers for robust, monitorable systems.
- The paper analyzes multi-agent interaction patterns including chains, stars, meshes, and explicit workflow graphs under a common framework.
- Evaluation is placed within the architecture using cost, latency, accuracy, security, and stability, alongside failure modes such as hallucination, infinite loops, and prompt injection.
2 Background and Definitions
The paper defines modern LLM agents as dynamic control systems that operate under partial observability, maintain mutable memory, reason before acting, and update from environmental feedback. This formalization distinguishes probabilistic cognitive controllers from earlier symbolic and narrowly trained agents.
- Modern LLM agents use pretrained models as general-purpose cognitive controllers augmented with memory and execution modules.
- The agent is modeled as a dynamic control system operating within a partially observable Markov decision process.
- At each time step, the system cycles through linked perception, memory, reasoning, action, and feedback functions.
- Perception maps the full environment state to a partial observation that may use multimodal encoders or text wrappers.
- Mutable memory combines new observations, prior reasoning traces, and execution feedback, including retrieval of relevant long-term information through RAG.
- The latent reasoning step produces a thought or plan before external action, ranging from chains of thought to hierarchical recursive tree search.
- The taxonomy figure traces the ecosystem from simple text-based loops to hierarchical systems operating in open-ended environments.
- The environment transitions after an action and returns feedback that closes the control loop.
3 Taxonomy
The taxonomy organizes Agentic AI across six connected dimensions, spanning system components, cognition, learning, collaboration, environments, and evaluation. Together, these dimensions describe architectural evolution from individual model capabilities toward coordinated, evaluated, and safety-aware systems.
- The taxonomy follows six connected dimensions: Core Components, Cognitive Architecture, Learning, Multi Agent Systems, Environments, and Evaluation.
- Core Components: Core Components cover multimodal perception, persistent memory, flexible action interfaces, and profiling that maintains consistent agent identity.
- Cognitive Architecture: Cognitive Architecture progresses from linear loops to hierarchical planning and reflection mechanisms for self-correction and feedback.
- Learning: Learning ranges from temporary in-context methods to permanent fine-tuning and alignment through human or model feedback.
- Multi Agent Systems: Multi-agent systems vary in interaction style, workflow structure, and communication topology, including cooperative roles and chain or star configurations.
- Environments and Domains: Environments include digital agents, embodied agents in robotics and games, and specialized domains such as healthcare, science, and finance.
- Evaluation and Safety: Evaluation combines cost, latency, accuracy, security, and stability with defenses against prompt injection and shared benchmarks such as SWE-Bench and OSWorld.
- The paper uses the taxonomy to analyze architectures, multi-agent coordination, and evaluation and safety practices for real-world deployment.
4 The Unified Architecture of Agentic AI
The unified architecture models an autonomous agent as a layered cognitive pipeline that transforms perception into action through memory, reasoning, tools, profiling, and learning. Its action space spans constrained APIs, executable code, interface navigation, and embodied control.
- Layered architecture: The architecture has three layers: core components for perception, memory, action, and profiling; cognitive architecture for planning and reflection; and learning for capability refinement.
- Core components: The agent core is represented as A = ⟨Φ, M, T, P⟩, where Φ is perception, M memory, T actions and tools, and P profiling.
- Perception: Perception is expanding from text-only processing toward multimodal grounding across screenshots, coordinates, video, audio, and 3D geometry.UI grounding remains difficult because small elements and dynamic layouts can cause drift or hallucinated clicks.
- Memory: Memory preserves state over time through retrieval, structure, retention policies, summarization, and deletion rather than simple vector lookup.Hierarchical summaries and distributed context help manage long-horizon tasks when raw histories or single prompts become impractical.
- Action and tools: Agent action has broadened from fixed API calls to code execution, agent computer interfaces, and embodied vision-language-action control.Computer-use actions support generic GUI operation but introduce latency and broader injection risk.
- Profiling: Profiling specifies an agent’s identity, role, and implicit constraints, shaping behavior by narrowing the search space and improving alignment.A security-focused profile favors safer plans than a profile without that emphasis.
4.2 Cognitive Architecture
The cognitive architecture is the agent’s decision core: it constructs and evaluates action trajectories using planning, search, inference-time reasoning, hierarchy, and reflection. These mechanisms trade broader deliberation and fault isolation against context, token, latency, and loop-related costs.
- Decision core: The cognitive architecture decomposes goals into action sequences and monitors execution for inconsistency and failure, using planning and reflection as its main functions.
- Sequential planning: ReAct interleaves reasoning with environment interaction, improving groundedness while remaining vulnerable to myopia, error propagation, and unproductive loops.Its trajectories alternate observations, reasoning traces, and actions.
- Search and planning: Tree of Thoughts and LATS add branching search, with LATS using Monte Carlo Tree Search to select candidates before irreversible tool calls.
- Inference-time reasoning: Frontier reasoning models internalize part of search at inference time through configurable thinking budgets, while external controllers continue to enforce safety and state persistence.
- Hierarchical planning: Hierarchical planners distribute long-horizon work across recursive or global-local structures, improving interpretability and fault isolation while increasing token and latency costs.
- Reflection: Reflection converts trajectory outcomes into guidance through verbal reinforcement, self-critique, iterative revision, tool validation, and recovery from failures.PALADIN targets diagnosis and corrected retries, while Expel transfers reusable safety and rollback rules across episodes.
4.3 Learning Paradigms
Agent learning spans prompt-based adaptation, weight updates, oversight, self-improvement, and external skill accumulation. These approaches seek more persistent and efficient behavior while facing prompt-growth, overfitting, and oversight bottlenecks.
- Adaptation and tuning: In-context learning is easy to deploy but short-lived and increasingly costly as prompts grow, motivating agent tuning on trajectory data.
- Adaptation and tuning: Agent-FLAN and FireAct internalize useful behaviors in model weights, and FireAct reports that trial-and-error trajectories can outperform prompt-only methods while reducing inference cost.
- Limitations: Effective tuning must avoid overfitting to surface formats instead of robust reasoning patterns.
- Oversight and self-improvement: RLAIF, process-level reward models, and self-improvement pipelines provide scalable feedback or synthetic corrections for search, selection, and web-agent improvement.
- Skill accumulation: Voyager accumulates executable skills in an external library that can be retrieved and composed without weight updates, avoiding catastrophic forgetting.
5 From Single to Multi Agent Systems
Multi-agent systems distribute workflows across specialized agents because many tasks exceed one model’s context, expertise, or cognitive capacity. Their coordination ranges from chains and stars to meshes and explicit workflow graphs, with verification and supervision addressing error propagation.
- Motivation: Multi-agent systems use specialized agents to address workflows that exceed a single model’s context window, domain expertise, or cognitive capacity.
- Communication topologies: Chain, star, and mesh topologies organize collaboration through sequential handoffs, controller-dispatched workers, or decentralized interaction.The topologies differ in structural control and interaction style.
- Workflow graphs: Production systems increasingly replace open-ended chat loops with explicit workflow graphs whose nodes invoke tools or models and whose edges constrain transitions.Flow engineering assigns developers the control structure while agents fill local decisions.
- Role-based collaboration: CAMEL uses role prompts for autonomous cooperative dialogue, but one-to-one mesh interaction can drift into unproductive behavior.
- SOP-based collaboration: MetaGPT encodes standard operating procedures and role-specific deliverables into prompts, turning unstructured collaboration into a sequential workflow.
- Hierarchical verification: Hierarchical verification uses distinct verifier, reviewer, or supervisor agents to challenge worker outputs and limit error propagation.Controlled results reported hallucination reductions of up to 100% when child outputs were filtered through a supervisor.
- Mesh collaboration: Mesh topologies prioritize creativity and diversity, supporting emergent behaviors such as debate, price discovery, and information diffusion.
6 Environments and Applications
Agentic systems operate across digital, embodied, scientific, and specialized domains, requiring interfaces and evaluation suited to each environment. Applications increasingly combine planning, tool use, long-horizon execution, and governance requirements.
- Digital Environments: Digital environments include web, desktop, enterprise, and operating-system tasks requiring agents to execute across interfaces and recover from partial failures.Web benchmarks test long-horizon interaction, while OSWorld evaluates end-to-end desktop control and exposes grounding and planning challenges.
- Digital Environments: Web agents have progressed from passive retrieval toward active execution, but dynamic interfaces and canvas elements motivate multimodal approaches.Screenshot-based systems infer visual UI structure, while visual interaction introduces additional environmental failure modes.
- Enterprise Applications: Enterprise agents must address repository-scale work, context exhaustion, search-space explosion, auditability, data governance, and failure recovery.These requirements extend beyond academic accuracy and are often absent from general benchmarks.
- Embodied Environments: Embodied agents ground language in physical constraints, evolving from high-level planning and affordance filtering toward vision-language-action and native-thinking systems.Robotics systems increasingly map multimodal perception and reasoning to motor commands, while cross-embodiment learning transfers skills across robot forms.
- Embodied Environments: Voyager demonstrates open-ended game learning by generating executable Python, storing reusable skills, and progressing through Minecraft’s technology tree 15.3× faster than baselines.Its Code-as-Policy approach uses an automatic curriculum and persistent Skill Library for long-horizon behavior composition.
- Specialized Domains: Specialized domains such as autonomous driving, science, healthcare, finance, and conversation require agents to integrate domain workflows with safety, accountability, traceability, or interaction constraints.These settings emphasize requirements beyond general-purpose agent capability, including traffic-rule reasoning, clinical governance, experiment cycles, market simulation, and sustained dialogue.
7 Evaluation and Safety
Evaluation of agentic systems must measure realistic performance across cost, latency, accuracy, security, and stability rather than relying on text similarity or mean success alone. The section also emphasizes that executable tools create concrete safety risks requiring layered system-level defenses.
- Evaluation Framework: The CLASSic framework evaluates agents across Cost, Latency, Accuracy, Security, and Stability as deployment moves beyond text-similarity metrics.The framework aligns performance measurement with enterprise requirements and realistic trajectories.
- Cost: Hierarchical architectures can maximize reasoning depth and task proficiency but incur exponential token growth and substantial computational overhead.This efficiency-intelligence trade-off constrains deployment in cost-sensitive, real-time applications.
- Latency: 47% synchronous success falls to 11% in asynchronous Robotouille settings, revealing weak temporal awareness when tasks require waiting for variable delays.DiMA addresses latency in safety-critical domains by distilling large multimodal planners into models with 100 times fewer parameters.
- Accuracy: Agent accuracy depends on tool use, state tracking, long-horizon recovery, memory, orchestration, grounding, and permissions rather than static question answering alone.Evaluation therefore includes environments such as GAIA, FrontierMath, AgentBench, and MultiAgentBench to test decomposition, reasoning, and emergent coordination.
- Security: Executable tools make prompt injection a security threat because direct or indirect malicious instructions can override objectives and turn agents into confused deputies.The risk is amplified by web content, tool outputs, screenshots, and low-level computer-use actions.
- Security: Prompt-only defenses are brittle, so robust security requires constrained permissions, sandboxes, user confirmation, and independent policy or audit components.These mitigations validate or restrict actions before sensitive execution and complement connector governance.
- Stability: Stability measures variance across repeated runs and resilience to perturbations, requiring failure severity, standard deviation, and worst-case scenarios alongside mean performance.This is particularly important when benign timeouts and catastrophic data leaks have very different consequences.
8 Challenges and Future Directions
Agentic AI remains difficult to deploy beyond controlled settings because reliability, efficiency, alignment, and self-improvement remain unresolved. The paper identifies action hallucinations, repetitive loops, computational overhead, value alignment, and limited post-deployment learning as key challenges.
- Agentic AI deployment in unconstrained environments is hindered by limitations in reliability, efficiency, and alignment.
- 8.1 Hallucination in Action and Error Propagation: Hallucinated actions can cause irreversible failures, while retrieval errors and early reasoning mistakes can propagate through multi-step execution.The paper highlights nonexistent API calls, incorrect file deletion, irrelevant retrieval context, and cascading failures in loops such as ReAct.
- 8.2 Infinite Loops and Agent Paralysis: Agents can repeatedly retry failed actions without changing strategy, motivating meta-cognitive modules that assess progress and interrupt futile loops.WebArena reports success rates often below 15% on long-horizon tasks, partly because agents fail to recognize local optima.
- 8.3 Latency and Computational Cost: Multi-agent debate and tree-search architectures impose substantial inference overhead, creating latency that is unacceptable for real-time applications.The paper points to separating planning from execution as one path toward reducing this cost, while noting that further optimization is needed.
- 8.4 Alignment and Social Norms: More autonomous agents must satisfy social contracts and safety constraints rather than optimizing only for task completion.The paper illustrates the risk with an agent that spams a user’s contacts to achieve a networking goal.
- 8.5 Towards Open-Ended Learning: Agents are largely static after deployment, motivating open-ended systems that generate curricula and continuously acquire, refine, and share skills.OMNI and Voyager are presented as examples aligned with lifelong learning and self-improvement.
9 Conclusions
The paper concludes that Agentic AI is evolving across reasoning, action, and collaboration, shifting the central design problem from prompting models to programming and controlling complete agent systems. Reliable deployment still depends on integrated architectures that are powerful, controllable, auditable, and aligned with real-world constraints.
- The central design question is shifting from how to prompt a model to how to program and control a complete agent system.
- Reasoning: Reasoning architectures are progressing from myopic single-loop solvers such as ReAct toward hierarchical, search-based, and inference-time reasoning systems with controllable compute budgets.
- Action: Action paradigms now span constrained APIs, code, and computer-use interfaces, making verification and recovery first-class design requirements.
- Collaboration: Multi-agent systems are moving from unstructured chat loops toward controllable workflow graphs and explicit handoffs that improve observability, debuggability, and safety through flow engineering.
- Verified evaluations show gains in desktop and operating-system control, but reliable success remains dependent on grounding, tool permissions, recovery loops, and efficient reasoning and verification.
- Progress beyond model scale requires architectures integrating perception, memory, tools, and collaboration while remaining powerful, controllable, auditable, and aligned with real-world deployment constraints.