PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Context Engineering for Large Language Models

Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, Shenghua Liu

arXiv:2507.13334v2cs.CL

TL;DR

Context Engineering addresses the need for a unified account of techniques that supply, process, and manage information for LLMs. This survey formalizes the field through a taxonomy and analysis of over 1400 papers, finding a comprehension-generation asymmetry and identifying evaluation as a major challenge.

  • Problem

    Context-engineering research is fragmented across specialized domains, obscuring connections among techniques and their underlying principles.

  • Method

    The survey formulates Context Engineering as an optimization problem and organizes its techniques into foundational Components and System Implementations.

  • Results

    The analysis of over 1400 research papers identifies a fundamental asymmetry: LLMs understand complex contexts remarkably well but struggle to generate equally sophisticated outputs.

  • Takeaways & Limitations

    The taxonomy provides a unified framework for integrating external knowledge, persistent memory, tools, and multi-agent coordination in context-aware AI systems.

  • Takeaways & Limitations

    Current evaluation methods inadequately capture complex reasoning, multi-step interactions, emergent behaviors, and longitudinal memory fidelity.

Abstract

from arXiv · show

The performance of Large Language Models (LLMs) is fundamentally determined by the contextual information provided during inference. This survey introduces Context Engineering, a formal discipline that transcends simple prompt design to encompass the systematic optimization of information payloads for LLMs. We present a comprehensive taxonomy decomposing Context Engineering into its foundational components and the sophisticated implementations that integrate them into intelligent systems. We first examine the foundational components: context retrieval and generation, context processing and context management. We then explore how these components are architecturally integrated to create sophisticated system implementations: retrieval-augmented generation (RAG), memory systems and tool-integrated reasoning, and multi-agent systems. Through this systematic analysis of over 1400 research papers, our survey not only establishes a technical roadmap for the field but also reveals a critical research gap: a fundamental asymmetry exists between model capabilities. While current models, augmented by advanced context engineering, demonstrate remarkable proficiency in understanding complex contexts, they exhibit pronounced limitations in generating equally sophisticated, long-form outputs. Addressing this gap is a defining priority for future research. Ultimately, this survey provides a unified framework for both researchers and engineers advancing context-aware AI.

1. Introduction

Context Engineering is presented as a unified discipline for organizing fragmented techniques that optimize the information payloads supplied to LLMs. The survey structures this field into foundational components and system implementations, then extends the taxonomy to evaluation and future directions.

  • Foundational components comprise Context Retrieval and Generation, Context Processing, and Context Management.
  • System implementations include advanced RAG, Memory Systems, Tool-Integrated Reasoning, and Multi-Agent Systems.
  • The field’s fragmented development obscures connections among techniques and creates barriers to understanding and effective use.
  • The survey introduces a structured taxonomy separating foundational Components from their integration into sophisticated System Implementations.
  • The survey also covers evaluation methodologies and future research directions alongside its taxonomy of components and implementations.

2. Related Work

Related work has surveyed many specialized areas of Context Engineering, including foundational capabilities, integrated systems, and evaluation. This survey positions those efforts within a horizontal taxonomy that makes their connections explicit.

  • Existing survey literature largely focuses on specific vertical domains within the broader field defined here as Context Engineering.
  • The proposed taxonomy connects foundational components with their integration into complex systems, bridging specialized research areas.
  • Foundational Components: Foundational-component surveys cover prompt engineering, external knowledge integration, long-context processing, self-improvement, and context management.
  • System Implementation: System-implementation literature addresses RAG, persistent Memory Systems, LLM-based Agents, Tool-Integrated Reasoning, and Multi-Agent Systems.
  • Evaluation: Evaluation surveys examine benchmarks and methodologies for both component-level and integrated system capabilities and performance.

3. Why Context Engineering?

Context Engineering extends prompt engineering into the design, management, and optimization of dynamic, structured information streams for LLMs. It formalizes context assembly and optimization while addressing the computational, reliability, and methodological challenges of modern context-aware systems.

  • Definition and scope: Context Engineering replaces the monolithic prompt view with a dynamic, structured stream of informational components.These components are sourced, filtered, formatted, and orchestrated for modern AI systems.
  • Definition and scope: The framework models context as components spanning instructions, external knowledge, tools, persistent memory, and dynamic system state.These components map to the survey’s core technical domains and system implementations.
  • Formal optimization: Context Engineering optimizes context-generating functions to maximize expected LLM output quality under a context-length constraint.The formulation includes task-specific generated context and the hard constraint |C| ≤ Lmax.
  • Formal optimization: Bayesian context optimization supports uncertainty handling, adaptive retrieval, and belief-state maintenance in multi-step reasoning.The formulation updates priors while maintaining beliefs over context.
  • Current challenges: Quadratic self-attention costs, repeated context processing, hallucinations, unfaithfulness, input sensitivity, and subjective prompt optimization constrain current systems.These limitations affect computational efficiency, commercial deployment, reliability, and methodological rigor.
  • Performance and adaptation: Context engineering improves performance through retrieval, structured prompting, few-shot examples, domain-specific methods, filtering, selection, and in-context adaptation.Reported examples include 18-fold text-navigation gains, 94% success rates, 9.90% BLEU-4 improvement, 175.96% exact-match improvement, and up to 9.8% code-generation improvement.

4. Foundational Components

Context Engineering organizes context optimization into foundational components that retrieve, process, and manage information, then integrates them into increasingly sophisticated systems. These components span prompt construction, retrieval, dynamic assembly, reasoning, and orchestration.

  • Context Engineering comprises Context Retrieval and Generation, Context Processing, and Context Management as its three foundational components.
  • Context Retrieval and Generation: Context Retrieval and Generation sources and constructs task-relevant information through prompt-based generation, external knowledge retrieval, and dynamic context assembly.
  • Context Retrieval and Generation: Prompt engineering combines task instructions, contextual information, input data, and output indicators, with the CLEAR Framework guiding prompt construction.
  • Context Retrieval and Generation: External retrieval and RAG combine parametric model knowledge with dynamically retrieved information, while newer systems add adaptive, hierarchical, graph-based, modular, or agentic retrieval.These approaches target current and domain-specific knowledge while supporting flexible retrieval and iterative planning.
  • Context Retrieval and Generation: Dynamic context assembly orchestrates acquired information into coherent, task-optimized contexts under model and resource constraints.Assembly mechanisms include template-based formatting, priority-based selection, adaptive composition, and multi-agent interaction control.

4.2. Context Processing

Context Processing transforms acquired information for efficient and useful model computation. The surveyed approaches address long-context costs, memory limits, self-refinement, and iterative error correction.

  • Context Processing optimizes acquired information through long-sequence processing, self-refinement, and integration of multimodal, relational, and structured representations.
  • Ultra-long sequence context processing: 122-fold more computation is required when Mistral-7B input grows from 4K to 128K tokens, while a 128K-token Llama 3.1 8B request can require up to 16GB.
  • Ultra-long sequence context processing: State-space models, linear attention, positional interpolation, and long-context architectures reduce computational or memory burdens while extending usable sequence lengths.Reported examples include up to 4000× speedup for linear attention and 2048K-token context windows for LongRoPE.
  • Efficient processing and memory management: Grouped-query, sparse, blockwise, selective, and memory-oriented attention methods improve long-context efficiency by reducing memory use, distributing computation, or selecting critical information.Sparse methods reported 92% of full-attention perplexity improvement with significant computation savings.
  • Self-refinement: Self-refinement uses feedback, self-evaluation, and repeated revision to improve outputs, with structured guidance and multi-aspect evaluation supporting more reliable correction.Extended reasoning and test-time scaling can improve reasoning even without adding new information.

Multimodal Context Integration

Multimodal Context Integration combines visual, audio, and other modality inputs with language representations. Its central challenges are deeper cross-modal fusion, modality bias, and limited mechanistic understanding.

  • Foundational techniques: Multimodal LLMs integrate vision, audio, and 3D data, commonly converting visual features into discrete tokens concatenated with text tokens.
  • Advanced integration strategies: Cross-modal attention and hierarchical processing support finer modality fusion and scalability for tasks involving lengthy multimodal inputs.
  • Challenges: Modality bias can make models favor textual inputs and produce multimodally ungrounded responses based on linguistic patterns rather than integrated sensory information.
  • Challenges: Effective multimodal reasoning requires combining modalities holistically, while limited mechanistic understanding of MLLMs remains a barrier to developing better architectures.

Advanced Contextual Capabilities and Future Directions

Advanced contextual capabilities extend LLMs beyond plain text by integrating structured knowledge and representations. These approaches improve grounding and reasoning while supporting richer applications over relational data.

  • In-Context and Long-Context Learning: In-context learning adapts models to new tasks without weight updates, but fixed context windows and image-token costs constrain many-shot multimodal learning.
  • Relational and Structured Context: Structured data integration addresses LLM difficulties with tables, databases, and knowledge graphs, where linearization can lose relationships and degrade performance.
  • Relational and Structured Context: Knowledge graph embeddings and hybrid graph-neural architectures encode entities, relationships, and textual context for integrated language-model reasoning.
  • Relational and Structured Context: Programming-language representations such as Python and SQL can outperform traditional natural-language representations on complex structured-data reasoning tasks.
  • Applications and Performance Enhancement: Knowledge graphs ground responses in verifiable facts, reducing hallucinations and improving factual accuracy, multi-hop reasoning, precision, and inference reliability.
  • Applications and Performance Enhancement: Structured knowledge supports healthcare decision-making, scientific hypothesis generation, business analytics, and robust natural-language interfaces for multimodal and personalized queries.Structured representations improved summarization performance by 40% and 14% across public datasets compared with unstructured memory approaches.

4.3. Context Management

Context management organizes, stores, compresses, and retrieves information to overcome finite context windows, stateless interactions, and long-context processing costs. These techniques support document understanding, extended reasoning, coherent multi-agent work, and persistent dialogue.

  • Fundamental Constraints: Finite context windows reduce efficacy on lengthy-document tasks and impose computational demands that hinder fast, high-throughput applications.Extending windows helps capture longer-range dependencies, but transformer processing becomes quadratically more expensive as sequence length increases.
  • Fundamental Constraints: LLMs exhibit “lost-in-the-middle” behavior, accessing information less reliably in middle positions than at context beginnings or ends.Extended chain-of-thought performance can degrade by as much as 73% compared with having no prior context.
  • Fundamental Constraints: Because LLMs process interactions independently and lack robust native state maintenance, explicit management systems are needed for coherent operation and failure recovery.This statelessness motivates mechanisms that preserve information across sequential exchanges.
  • Memory Architectures: Hierarchical and dynamic memory architectures manage fixed windows through paging, forgetting-based memory adjustment, episode pagination, memory gisting, and interactive lookup.MemGPT pages information between context and external storage, while MemoryBank adjusts memory strength according to time and significance.
  • Management Techniques: Context management combines architectural adaptations, semantic organization, retrieval mechanisms, caching, and compression to balance broad context with query-relevant detail.Examples include modified attention or positional encodings, interconnected semantic networks, bi-layer KV caches, and distributed attention across GPU clusters.
  • Applications: Effective context management enables whole-document analysis, multi-step reasoning, coherent multi-agent coordination, and persistent conversational interfaces.These applications depend on maintaining longer-range dependencies, synchronizing internal information, and preserving thread coherence across interactions.

5. System Implementations

System implementations integrate foundational context-engineering components into deployable intelligent architectures. The survey groups them into RAG, memory systems, tool-integrated reasoning, and multi-agent systems.

  • System Implementations: The survey presents four system-implementation categories: RAG, memory systems, tool-integrated reasoning, and multi-agent systems.Each category combines foundational components while addressing specific challenges in context utilization.

5.1. Retrieval-Augmented Generation

RAG integrates external knowledge with language-model generation through modular, agentic, and graph-enhanced architectures. These designs dynamically select, refine, and structure information for retrieval and reasoning.

  • RAG Overview: RAG connects parametric model knowledge with dynamic external information through modular, agentic, and graph-enhanced architectures.This provides access to current and domain-specific information beyond static training data.
  • Modular RAG Architectures: Modular RAG replaces linear retrieval-generation pipelines with hierarchical, reconfigurable components linked by routing, scheduling, and fusion mechanisms.Its hierarchy spans top-level stages, middle-level sub-modules, and bottom-level operational units.
  • Modular RAG Architectures: Modular RAG systems use adaptive search, multi-query fusion, source routing, query rewriting, and hybrid retrieval to improve retrieval accuracy and context relevance.Rewrite-Retrieve-Read and Generate-Read approaches exemplify these arrangements.
  • Modular RAG Architectures: Contemporary modular frameworks report improvements in retrieval accuracy and trustworthiness while supporting independently adjustable modules and domain-specific customization.FlashRAG provides 5 core modules and 16 subcomponents; KRAGEN combines knowledge graphs with vector databases for biomedical reasoning.
  • Agentic RAG: Agentic RAG embeds autonomous agents that use reflection, planning, tools, memory, and collaboration to adapt retrieval workflows to complex tasks.Retrieval becomes a dynamic operation involving continuous reasoning, cross-referencing, and iterative feedback.
  • Graph-Enhanced RAG: Graph-enhanced RAG represents entities and relationships explicitly, enabling structured pathway navigation and multi-hop reasoning across connected knowledge-graph nodes.These approaches use graph construction, graph neural networks, hierarchical indexing, pruning, and hybrid graph-vector retrieval to reduce noise and improve semantic understanding.
  • Applications: Dynamic RAG applications support real-time task allocation, grounded planning, ontology generation, and iterative improvement through self-memory mechanisms.DRAGON-AI combines textual and logical components for specialized dynamic retrieval environments.

5.2. Memory Systems

Memory systems provide persistent storage, retrieval, and use of information so LLMs can maintain context beyond stateless interactions. The survey covers memory classifications, architectures, access mechanisms, applications, and evaluation challenges.

  • Memory Systems: Memory systems extend LLMs beyond stateless pattern processing by supporting persistent information storage, retrieval, learning, adaptation, and long-term contextual understanding.Memory-enhanced agents combine immediate context awareness with longer-term knowledge application.
  • Memory Challenges: LLM memory remains constrained by inadequate structured storage and approximate vector similarity, which limit precise information handling.These limitations distinguish current memory mechanisms from more exact symbolic operations.
  • Memory Classification Frameworks: Memory is classified temporally into sensory, short-term, and long-term forms, and operationally into parametric, ephemeral activation, and plaintext memory.Short-term stores include key-value caches and hidden states, whereas long-term stores persist across interaction cycles.
  • Short-Term Memory Mechanisms: Short-term memory uses context windows and key-value caches for temporary token access, enabling in-context learning, few-shot learning, and task adaptation without parameter updates.Configurations include full, limited, and memory-less operation.
  • Long-Term Memory Implementations: Long-term memory methods use external storage, semantic organization, forgetting mechanisms, and hierarchical read-write architectures to address context limits and catastrophic forgetting.External memory retrieves relevant history without retaining all information inside the constrained context window.
  • Memory Access Patterns and Structures: Memory access and organization span sequential or random retrieval, textual stores, knowledge structures, hierarchical systems, and task-, temporal-, or semantic organization.Recognition, recall, and retention experiments assess accessibility duration and retrieval conditions.
  • Evaluation and Challenges: Evaluation is limited because most agents remain stateless and laboratory tests inadequately isolate memory performance or represent unpredictable real-world relevance changes.These constraints hinder assessment of genuine lifelong learning and separation of memory failures from reasoning failures.
  • Future Directions: Future directions include hybrid parametric and non-parametric memory, automated feedback, shared multi-agent memories, knowledge-graph integration, and domain-specific architectures.These directions target scalable evaluation, collaborative learning, and improved long-term knowledge evolution.

5.3. Tool-Integrated Reasoning

Tool-Integrated Reasoning extends LLMs from passive text generation to interactive problem solving by combining reasoning with external tools. The survey covers tool-use architectures, implementation methods, training data, self-improvement, and evaluation challenges.

  • Function Calling Mechanisms: Function calling converts generative models into interactive agents by producing structured outputs that invoke external tools and access current, domain-specific information.
  • Implementation Frameworks and Paradigms: Tool-use architectures evolved from Toolformer’s autonomous API learning and ReAct’s thought-action-observation cycle to multimodal and cross-domain systems.
  • Training Methodologies and Data Systems: Fine-tuning provides stable tool-use capabilities but requires substantial resources, whereas prompt engineering is flexible and efficient but unstable.
  • Implementation Frameworks and Paradigms: Tool-integrated agents perform intent recognition, function selection, parameter mapping, execution, and response generation across heterogeneous interfaces.
  • Tool-Integrated Reasoning enables LLMs to overcome outdated knowledge, calculation errors, and shallow reasoning through dynamic interaction with external resources.
  • Agent-Environment Interaction: 67.0% accuracy on AIME2024 after 400 training steps exceeded text-based RL baselines’ 40.0% accuracy with extensive training.
  • Evaluation: Current tool-agent evaluation includes real-world, multimodal benchmarks and exposes substantial gaps between system performance and human-level capabilities.

5.4. Multi-Agent Systems

Multi-Agent Systems coordinate autonomous agents through communication protocols, orchestration mechanisms, and context-aware collaboration. The survey traces interoperability standards, communication structures, orchestration paradigms, and persistent coordination challenges.

  • Multi-Agent Systems enable autonomous agents to collaborate on complex problems through communication, orchestration, and coordination across diverse architectures.
  • Communication Protocols: Agent communication evolved from KQML and FIPA ACL toward semantic, ontology-based protocols addressing interoperability across heterogeneous platforms.
  • Contemporary Protocol Ecosystem: MCP standardizes agent-environment interaction through JSON-RPC client-server interfaces, while A2A, ACP, and ANP extend peer, RESTful, and internet-scale interoperability.
  • Contemporary Protocol Ecosystem: A progressive layering strategy assigns MCP to tool access, ACP to message exchange, A2A to peer interaction, and ANP to network interoperability.
  • Communication Structures: Communication structures include hierarchical, decentralized, centralized, and shared-message architectures, with sequential exchanges and message-passing strategies.
  • Orchestration Mechanisms: Orchestration manages agent selection, context distribution, interaction flow, task progression, and deviations under different operational paradigms.
  • Context Management and Environmental Adaptation: Context-aware adaptation supports task decomposition and agent assignment, while persistent context maintenance and transactional integrity remain difficult.
  • Context Management and Environmental Adaptation: Automated orchestration improves overall performance because human users frequently struggle to select effectively among available agents.

6. Evaluation

Evaluating context-engineered systems requires both component-level diagnostics and end-to-end assessment because heterogeneous components can produce emergent or interfering behaviors. The survey reviews intrinsic, extrinsic, and system-level methods while identifying major benchmark and metric limitations.

  • Context-engineered systems require evaluation frameworks spanning component diagnostics, task performance, and overall robustness because their behaviors are dynamic and context-dependent.
  • Intrinsic Evaluation: Intrinsic evaluation tests isolated components such as prompt effectiveness, long-context retention, positional bias, reasoning coherence, and self-contextualization.
  • Intrinsic Evaluation: GPT-4 achieved approximately 20% improvement through iterative self-refinement processes involving frameworks such as Self-Refine, Reflexion, and N-CRITICS.
  • Extrinsic and System-Level Evaluation: Extrinsic evaluation measures end-to-end utility across question answering, reasoning, and real-world applications, while system-level evaluation captures synergy and interference among components.
  • Memory Systems Evaluation: Memory evaluation lacks standardized frameworks and must address stateless models, isolation problems, multi-session reasoning, temporal reasoning, and knowledge updates.
  • Real-World and Tool-Integrated Evaluation: GPT-4 completed less than 50% of GTA tasks, compared with 92% human performance, while advanced benchmarks test multi-turn and nested tool use.
  • Methodological Limitations: Static metrics such as BLEU, ROUGE, and perplexity inadequately capture reasoning chains, multi-step interactions, emergent behavior, and failure attribution.
  • Future Evaluation Directions: Future evaluation should combine dynamic, holistic, and longitudinal benchmarks with compositional generalization, autonomy, robustness, and socio-technical measures.

7. Future Directions and Open Challenges

Future Context Engineering research must develop unified theories and architectures that address the gap between strong comprehension and limited long-form generation. Open challenges also include multimodal and temporal reasoning, tool coordination, graph organization, safety, and human-AI collaboration.

  • Foundations and Theory: Context Engineering lacks unified theoretical foundations connecting disparate techniques and providing principled design guidance for systematic progress.
  • Comprehension-Generation Asymmetry: LLMs show a fundamental asymmetry between remarkable comprehension and pronounced limitations in sophisticated, long-form generation.
  • Long-Form Generation: Long-form generation requires planning mechanisms that preserve coherence, factual accuracy, and logical consistency across thousands of tokens.
  • Multimodal and Temporal Reasoning: Multimodal context engineering remains constrained by modality-specific encoders, limited cross-modal interaction, and difficulties tracking temporal dynamics during multi-step tasks.
  • Architectural Innovations: State space models such as LongMamba offer linear scaling and improved memory utilization for long sequences but require development to match transformers across diverse tasks.
  • Tool-Integrated Reasoning: GAIA reports 92% human accuracy versus 15% for advanced models, highlighting unresolved limitations in autonomous tool selection, parameter extraction, and multi-tool coordination.
  • Complex Context Organization: Graph-based context systems struggle with large structures, multi-hop consistency, and novel-topology generalization, while more precise specialized architectures increase complexity.
  • Intelligent Context Optimization: Self-refinement improves performance by approximately 20% for GPT-4, but autonomous evolution and meta-learning across diverse contexts remain open problems.

8. Conclusion

The survey formalizes Context Engineering through a unified taxonomy and analysis of over 1400 research papers. It identifies a comprehension-generation asymmetry in LLMs, alongside increasingly modular integration patterns and evaluation challenges for complex systems.

  • Conclusion: The survey establishes Context Engineering as a formal discipline for systematically designing, optimizing, and managing information payloads for LLMs.Its analysis connects external knowledge, persistent memory, and dynamic interaction with complex environments.
  • Conclusion: Its unified taxonomy organizes three foundational components alongside five system implementations that integrate them into sophisticated architectures.The components are Context Retrieval and Generation, Context Processing, and Context Management; implementations include RAG, Memory Systems, Tool-Integrated Reasoning, and Multi-Agent Systems.
  • Conclusion: The survey identifies a fundamental asymmetry: LLMs understand complex contexts remarkably well but remain limited in generating equally sophisticated outputs.It describes this comprehension-generation gap as one of the field’s most critical challenges.
  • Conclusion: Multiple techniques increasingly combine synergistically, while modular and compositional architectures support flexible adaptation across applications.These architectures are intended to maintain system coherence while accommodating diverse application requirements.
  • Conclusion: The field needs comprehensive evaluation frameworks for systems integrating multiple components, adaptive behaviors, and extended time horizons.The survey also calls for interdisciplinary collaboration as Context Engineering becomes more central to AI development.
Loading 2507.13334v2…