PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta Context Engineering via Agentic Skill Evolution

Haoran Ye, Xuning He, Vincent Arak, Haonan Dong, Guojie Song

arXiv:2601.21557v2cs.AIcs.NE

TL;DR

Existing Context Engineering methods rely on manually crafted harnesses that impose structural biases and restrict context optimization. MCE co-evolves CE skills and context artifacts through bi-level agentic optimization, achieving consistent gains across five domains and both offline and online settings. It also reports improved context adaptability, transferability, and efficiency.

  • Problem

    Manually crafted CE harnesses impose structural and optimization biases, restricting context optimization to a narrow design space.

  • Method

    MCE uses bi-level optimization in which a meta-agent evolves CE skills through agentic crossover and a base-agent optimizes context as files and code.

  • Results

    5.6–53.8% relative improvement over state-of-the-art CE methods was achieved across five domains, four LLMs, and offline and online settings.

  • Takeaways & Limitations

    MCE supports adaptable, transferable, and efficient context optimization while co-evolving the skills and artifacts used for CE.

  • Takeaways & Limitations

    MCE may be less advantageous for reasoning-intensive tasks and may struggle with very long, complex trajectories requiring fine-grained credit assignment.

Abstract

from arXiv · show

The operational efficacy of large language models relies heavily on their inference-time context. This has established Context Engineering (CE) as a formal discipline for optimizing these inputs. Current CE methods rely on manually crafted harnesses, such as rigid generation-reflection workflows and predefined context schemas. They impose structural biases and restrict context optimization to a narrow, intuition-bound design space. To address this, we introduce Meta Context Engineering (MCE), a bi-level framework that supersedes static CE heuristics by co-evolving CE skills and context artifacts. In MCE iterations, a meta-level agent refines engineering skills via agentic crossover, a deliberative search over the history of skills, their executions, and evaluations. A base-level agent executes these skills, learns from training rollouts, and optimizes context as flexible files and code. We evaluate MCE across five disparate domains under offline and online settings. MCE demonstrates consistent performance gains, achieving 5.6--53.8% relative improvement over state-of-the-art agentic CE methods (mean of 16.9%), while maintaining superior context adaptability, transferability, and efficiency in both context usage and training.

Meta Context Engineering

MCE uses a bi-level framework in which a meta-agent evolves skills while a base-agent optimizes context.

  • MCE is organized as a bi-level optimization framework.
  • The meta-agent drives skill evolution.
  • The base-agent manages context optimization.

1 Introduction

Context Engineering optimizes inference-time context, but manually designed harnesses constrain its representations and procedures. MCE addresses this by co-evolving CE skills and flexible context artifacts, and evaluations report gains across domains, models, and settings.

  • Context Engineering optimizes inference-time context to improve downstream utility and enable continuous self-improvement.
  • Manually crafted CE harnesses impose structural and optimization biases that restrict CE to a narrow design space.
  • MCE co-evolves CE skills and context artifacts through bi-level optimization, separating engineering strategy from the learned context artifact.
  • MCE gives the base-agent coding and file-system capabilities to instantiate and optimize context as flexible, programmatic artifacts.
  • MCE also showed superior context adaptability, transferability, and efficiency in context usage and training.
  • 5.6–53.8% relative improvement over state-of-the-art CE methods was achieved across five domains, four LLMs, and offline and online settings.The reported mean gain was 16.9%.

2 Background and Related Work

Prior CE and evolutionary approaches expose trade-offs in context design and optimization. MCE responds by combining highly agentic architectures, file-based tooling, and evolutionary optimization over integrated agent skills.

  • Agentic Context Engineering: Current CE methods use manually designed harnesses with trade-offs in episodic, list-based, and hierarchical context representations.
  • Motivation: No single agentic harness is universally optimal, motivating MCE’s dual-level optimization.
  • Motivation: Coding toolkits and file-system access provide flexible and verifiable foundations for general and vertical agents.
  • Motivation: MCE transitions from manually crafted workflows to agentic meta- and self-learning systems that manage skills and context through generated files and code.
  • Evolutionary Computation with LLMs: LLM-driven evolutionary computation expands optimization beyond fixed rules into unstructured search spaces and multiple abstraction levels.
  • Evolutionary Computation with LLMs: Agent skills integrate instructions, resources, and scripts while providing a modular interface that decouples optimization targets from agent architecture.
  • Evolutionary Computation with LLMs: Figure 2 presents the methodological overview of MCE.

3 Meta Context Engineering

MCE formulates context engineering as bi-level optimization that co-evolves executable skills and context artifacts. A meta-agent evolves skills from historical evidence, while a base-agent uses them to optimize file- and code-based context functions through rollout feedback and validation.

  • Problem formulation: A context function combines static components, such as knowledge bases and prompts, with dynamic operators that retrieve, filter, and compose them according to each query.Its file- and code-based representation permits arbitrary computational procedures without imposing a fixed context structure.
  • Problem formulation: MCE separates how context is represented and learned—the skill—from what is learned—the resulting context function—within a bi-level optimization framework.The inner optimization finds the best context function for a given skill, while the outer optimization selects the skill yielding maximal validation performance.
  • Meta-level: agentic skill evolution: Agentic crossover generates each new skill by inspecting the task specification and complete skill history, then selectively recombining successful and failed patterns.Skills may include natural-language methodology, executable scripts, structured templates, and validation protocols.
  • Evolved-skill properties: Evolved skills adjust autonomy, expressivity, granularity, and context verbosity to task complexity and model capacity.They can specify rigid workflows or delegate autonomy, support batch-level synthesis or instance-level updates, and use concise or detailed contexts as appropriate.
  • Base-level: fully agentic context optimization: The base-agent executes the current skill using a warm-start best context and training-rollout feedback to produce an updated context function.The workspace includes the skill folder, prior best context, rollout evaluations, and optional utilities for invoking other AI models.
  • Algorithmic orchestration: Each MCE iteration evolves a skill, performs context optimization, evaluates the resulting context on validation data, and retains the best-so-far solution.The procedure implements a history-informed (1 + 1)-ES in which one offspring context is compared with the current best.

4 Experiments

Across five domains and offline and online settings, MCE consistently outperforms baselines while adapting context length and improving context, transfer, and training efficiency. Its gains arise from flexible context optimization, batch-level processing, and evolving skills rather than fixed heuristics alone.

  • Domain adaptation: MCE improves domain adaptation substantially, with 89.1% average relative improvement over the base model across five benchmarks and 172.6% for Gemma3-4B.The gains are more pronounced for smaller models that may lack basic domain knowledge without context engineering.
  • Overall performance: MCE ranks first across all five benchmarks, achieving 89.1% average relative gain offline and 74.1% online, exceeding ACE’s 70.7% and 41.1%.The evaluation spans financial, chemistry, medicine, law, and AI safety tasks.
  • Domain adaptation: MCE-enhanced general models surpass domain-specific models, reaching 0.70 F1 versus 0.56 on LawBench and 0.80 versus 0.72 on Aegis2.0.These comparisons suggest learned context can substitute for expensive domain-specific fine-tuning.
  • Context adaptability: MCE adapts context length to task requirements, ranging from 1.5K to 86K tokens rather than imposing fixed brevity or verbosity.Effective FiNER contexts use 1.5K and 20K tokens, while LawBench and USPTO50k contexts reach 44K and 86K tokens.
  • Context efficiency: At comparable FiNER context lengths, MCE-S reaches 73% accuracy at ∼1.5K tokens versus ACE’s 65%, while MCE-L reaches 75% at 20K versus ACE’s 70% at 79K.The authors attribute this efficiency to global context restructuring and batch-level aggregation of rollout feedback.
  • Training efficiency: MCE reduces FiNER training duration to 1.9 hours versus ACE’s 25.8 hours and reaches 95% training accuracy in 450 rollouts versus ACE’s 2,169.Batch-level optimization enables direct data analysis or parallelized reflection and curation across rollouts.

5 Conclusion and Discussion

MCE reframes context engineering as a learnable agentic capability that co-evolves skills and context artifacts, outperforming prior methods while improving adaptability, efficiency, transferability, and training speed. Its main limitations arise on reasoning-intensive tasks and very long, complex trajectories.

  • Conclusion: MCE co-evolves CE skills and context artifacts through bi-level, fully agentic optimization rather than fixed workflows and schemas.The framework uses dual agents and represents context as files and code within an evolutionary process.
  • Conclusion: 5.6–53.8% relative improvement over SOTA methods, with a mean gain of 16.9%, demonstrates consistent superiority across five domains.The reported gains accompany superior context adaptability, efficiency, transferability, and substantial training speedups.
  • Limitations: MCE is particularly advantageous for domain knowledge acquisition and pattern matching, where learned skills capture data characteristics and organize domain-specific structures.The authors contrast this strength with reasoning-intensive tasks, where manually crafted iterative and reflective harnesses may already be well-suited.
  • Limitations: MCE may struggle with very long, complex trajectories requiring fine-grained credit assignment and detailed trajectory analysis.The authors attribute this constraint to the capabilities of the underlying agent model rather than the framework itself.
  • Future Work: Agentic skill evolution could generalize beyond context engineering to other agentic capabilities and task domains, including skill transfer and composition.Future work also proposes co-evolving context-utilization skills and studying emergent behaviors from skill composition.

A Experimental Details

The experiments span financial, chemistry, medicine, law, and AI safety benchmarks, using task-specific datasets, prompts, and evaluation procedures. The setup preserves standard splits where possible while applying computationally constrained sampling and stratification.

  • Benchmarks: Five benchmarks cover finance, chemistry, medicine, law, and AI safety to evaluate MCE across diverse domains.The experiments use original train/validation/test splits unless computational constraints require subsets, and process datasets in the same order across baselines.
  • Finance: FiNER maps financial entities to one of 139 US GAAP XBRL tags, requiring semantic disambiguation among similar long labels.The sampled subset focuses on debt- and credit-related tags, with 200/100/100 train/validation/test instances.
  • Prompting: The generator prompts impose structured output formats, including JSON fields for FiNER and retrosynthesis and tagged diagnosis output for Symptom2Disease.These templates pair task-specific context placeholders with domain instructions and final-answer formats.
  • Chemistry: USPTO-50k evaluates single-step retrosynthesis by asking models to predict precursor reactants from product molecules represented as SMILES.The sampled data contains 50 training, 30 validation, and 100 test instances, stratified across 10 reaction types.
  • Medicine: Symptom2Disease requires predicting one of 22 disease categories from natural-language patient symptoms, with exact diagnosis matching as the scoring criterion.The setup preserves a 212-sample test split and creates stratified 200-sample training and 50-sample validation splits.

C Learned Skills

The learned skills iteratively transform training errors into generalizable, structured context through reflection, abstraction, reasoning chains, and incremental refinement. The resulting workflow emphasizes semantic principles over memorized examples and evaluates success through validation improvement and reduced overfitting.

  • Skill Workflow: Three LLM calls convert prediction errors into generalizable reasoning principles, covering error analysis, rule generalization, and reasoning-chain creation.The process is organized across specialized context files such as reasoning chains, semantic principles, and tag references.
  • Skill Workflow: The skill loads existing context and training results, analyzes incorrect samples, then adds, updates, or merges insights before saving revised context.This workflow preserves useful knowledge while incrementally refining it rather than rebuilding from scratch.
  • Error Analysis: The error taxonomy targets superficial pattern matching, missing semantic distinctions, context blindness, and category confusion.Each category links a recurring reasoning failure to corrective guidance rather than memorizing an isolated answer.
  • Reasoning Chains: The refined context uses explicit decision chains that classify entity type, financial category, and the relevant aspect before selecting a tag.For example, facility capacity, debt-instrument value, and rate type are resolved through separate branching decisions.
  • Evaluation Criteria: The skill succeeds when context contains more principles than examples, explicit reasoning chains, anti-patterns, novel-example generalization, and improved validation accuracy.The stated criterion allows training accuracy to decrease slightly as overfitting is reduced.

D.1.1 Task Characteristics and Baseline Limitations

The tasks require dense domain knowledge, semantic disambiguation, and workflow-sensitive rules, exposing weaknesses in baselines that rely on demonstrations, brevity, or monolithic accumulation. MCE responds with structured, error-driven, workflow-oriented context that prioritizes generalizable decisions.

  • Task Characteristics: FiNER requires broad rule coverage and disambiguation logic because similar financial tags depend on nuanced contextual distinctions.The task bottleneck is mastering high-density long-tail rules rather than merely imitating training examples.
  • Baseline Limitations: ICL, MIPROv2, and GEPA struggle on FiNER because demonstrations or concise prompts fail to capture its extensive nuanced rules.ACE performs better by accumulating reflections and patterns in a playbook, while MCE develops structured hierarchical context files.
  • Skill Evolution: MCE’s learned context shifts from pattern extraction toward error-driven generalization after validation feedback exposes overfitting.The refined skill emphasizes abstract principles, semantic reasoning chains, and cross-example patterns.
  • Workflow Context: The context organizes facility classification around the facility name, separating revolving-credit capacity tags from term-loan debt-instrument tags.The decision chain maps revolving facilities to maximum borrowing capacity and term loans to debt-instrument face amount.
  • Task Characteristics: USPTO-50k challenges baselines with a large chemical reaction space, where generic instructions and brevity can omit rare reaction-specific rules.The evolved approach therefore emphasizes workflow-oriented, actionable context rather than encyclopedic coverage alone.
  • Context Organization: The learned contexts become structured knowledge systems with specialized files, decision processes, verification checks, and compressed error-pattern references.Examples include workflow.md, QUICK_REFERENCE.md, and context components organized by task-specific reasoning needs.

3. SUCCESS_PATTERNS.md: Validated patterns with 100% training accuracy

The context organizes reaction knowledge around workflow guidance, verification checkpoints, and documented success and error patterns. This organization supports systematic decisions rather than pattern memorization in a combinatorial reaction space.

  • Workflow organization: Workflow-oriented context includes reaction-specific guides and inline verification checkpoints for structured reaction analysis.The workflow includes heterocycle identification, pattern matching, and critical checks for oxygen presence and ester-versus-acid distinctions.
  • Error prevention: Error checks distinguish common reaction confusions, such as oxadiazole versus triazole and ester versus acid precursors for amide retrosynthesis.These checks explicitly identify the structural feature or precursor form that should be used.
  • Validated patterns: The context documents successful reaction patterns, including a Hantzsch thiazole synthesis example with its product, precursors, and key transformation.The example records a correct instance and identifies the thioacetamide-plus-α-halo-carbonyl route to a thiazole.
  • Implication: Combining error patterns with success patterns enables more robust generalization than error-only approaches and promotes systematic decision-making.The workflow-centric organization is presented as important for the combinatorial complexity of chemical reaction space.

D.3.1 Task Characteristics and Baseline Limitations

Symptom2Disease is a fine-grained classification task with minimal distributional shift, so pattern matching and preserving specific symptom distinctions matter more than abstraction. Accordingly, example-based methods outperform approaches that compress cases into general rules or accumulate noisy heuristics.

  • Task characteristics: Symptom2Disease maps natural-language symptoms to 22 disease categories with minimal train-test distributional shift and little need for deep reasoning.Test descriptions closely mirror training examples, making pattern matching central to classification.
  • Baseline limitations: GEPA and ACE underperform because abstraction can erase fine-grained symptom distinctions, while accumulated heuristics can introduce noise and propagate early errors.GEPA’s brevity bias overgeneralizes symptoms, whereas ACE’s monolithic playbook can override pretrained medical knowledge, especially online.
  • Example-based methods: Dynamic many-shot retrieval performs strongly because semantic similarity aligns with pragmatic similarity in symptom classification.MIPROv2 likewise benefits from preserving distributional features through example selection rather than compressing knowledge into abstract instructions.

D.3.2 Skill Evolution

For Symptom2Disease, skill evolution shifts MCE from broad context construction toward conservative, evidence-based refinement. The resulting system combines validated symptom patterns with selective discriminator retrieval and explicit stopping rules to avoid overfitting, noise, and irreducible ambiguity.

  • Initial skill: The initial skill uses Pattern Extraction, Profile Synthesis, and Error-Driven Refinement to organize symptom-diagnosis knowledge.It extracts diagnosis-specific patterns, synthesizes generalized profiles, and analyzes incorrect predictions and commonly confused diagnoses.
  • Conservative refinement: The optimal skill preserves a near-optimal 88% validation score and 1% gap, adding only discriminators that pass multi-gate validation without widening the train-validation gap.Its criteria require repeated evidence, low generalization risk, and high confidence for novel cases.
  • Stopping rules: Explicit stop criteria preserve the current state when validated patterns are scarce, training accuracy exceeds 90%, the train-validation gap exceeds 1.5%, or ambiguous errors dominate.The skill treats further refinement as counterproductive when the architecture is already near-optimal or remaining errors are likely ambiguous.
  • Discriminator design: The final context prioritizes decisive discriminators that resolve specific confusion patterns instead of accumulating encyclopedic information.Examples include distinguishing painful impetigo vesicles from itchy, widespread chicken-pox vesicles and separating asthma from pneumonia by symptom essence.
  • Overall lesson: For minimal-shift tasks, the evolved strategy is conservative: preserve validated patterns, require strong evidence for additions, and stop when errors are irreducible ambiguities.This reflects the learned principle that knowing when to stop is as important as knowing what to add.
  • Retrieval evolution: MCE evolves retrieval into a 1,440-line prioritized routing system that selects relevant context subsets for different symptom combinations.The decision-tree-like design uses critical rules, surgical fixes, semantic matching, error-pattern rules, and a full-guide fallback to reduce irrelevant context and interference.

D.4.1 Task Characteristics and Baseline Limitations

LawBench criminal-charge prediction requires long-context detail sensitivity, precise legal reasoning, and high-precision classification. In this setting, ACE’s accumulated playbook can create interference, whereas GEPA’s concise globally optimized instruction can generalize across case types.

  • Task characteristics: LawBench combines detailed case descriptions, legally precise distinctions, and high-precision charge classification.Correct predictions depend on legal elements such as whether force or threat was used when distinguishing theft from robbery.
  • Baseline limitations: ACE’s monolithic accumulation becomes a liability because retrieved historical bullets can overwhelm the current case with tangential patterns.The resulting context interference is especially problematic when subtle legal facts determine the correct charge.
  • Baseline comparison: GEPA’s brevity bias becomes advantageous by producing a globally optimized instruction that structures legal reasoning and generalizes across case types without overfitting to examples.The prompt is optimized across the dataset rather than built from accumulating case-specific patterns.

D.4.2 Skill Evolution

The skill evolves from pattern-based charge learning toward structural case decomposition and processing-stage error diagnosis. Iterations also reduce context size and adapt decision rules and retrieval to model-specific failure modes.

  • Pattern-based learning caused overfitting because it matched surface patterns rather than requiring deeper case analysis.
  • 88KB contexts correlated with a 14% train-validation gap, motivating reduction to approximately 30KB across 11 specialized files.
  • The evolved legal-classification context uses structural decomposition instead of accumulating charge-specific patterns.The framework processes cases by reading, decomposing, matching, and validating.
  • Errors are categorized by processing stage, including comprehension, act extraction, independence, charge selection, naming, and multi-charge detection.This shifts diagnosis from which charges were confused to how the error occurred.
  • For small-model safety classification, the skill emphasizes high-precision rules, safe exclusions, and decision rules over examples.The evolved context contains 12 specialized files organized around violation categories and safe exclusions.
  • A retrieval precision gate with early safe returns prevents clearly benign prompts from receiving violation-related context, reducing over-classification.
Loading 2601.21557v2…