PaperlayerView exact revision on arXiv

Source-linked AI summary

EEVEE: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents

Weixian Xu, Shilong Liu, Mengdi Wang

arXiv:2606.11182v1cs.LGcs.AI

TL;DR

Heterogeneous task streams create cross-dataset interference that single-prompt adaptation struggles to manage. EEVEE routes inputs to specialized prompts and co-evolves routing and prompting, improving average multi-benchmark scores by 10.38 and 24.32 points over Qwen3-4B-Instruct and DeepSeek-V3.2.

  • Problem

    Heterogeneous task streams expose cross-dataset interference, as updates for one domain can harm task-specific behavior in another.

  • Method

    EEVEE partitions inputs into task clusters, routes them to specialized prompts, and jointly improves routing and prompting through interleaved co-evolution phases.

  • Results

    10.38 and 24.32 points: EEVEE improves average multi-benchmark scores over Qwen3-4B-Instruct and DeepSeek-V3.2, respectively.

  • Takeaways & Limitations

    EEVEE provides a practical self-improving approach for heterogeneous real-world agent workloads, with strong gains in mixed-dataset settings.

  • Takeaways & Limitations

    EEVEE relies on ground-truth or rule-based labels and a prepared adaptation set, rather than learning fully from a completely online stream.

Abstract

from arXiv · show

In this paper, we propose EEVEE, the first multi-dataset test-time prompt learning framework for LLM agents, enabling test-time prompt learning under real-world task streams. Existing methods are largely designed for single-dataset settings, while real-world applications require models to handle heterogeneous input streams drawn from multiple datasets, domains, and task distributions, limiting their practical applicability. To mitigate cross-dataset interference, EEVEE introduces a router that partitions incoming inputs into task clusters and assigns them to suitable prompt configurations. This design is optimized via a router-prompt co-evolution strategy, which employs interleaved router and prompt learning phases to address their mutual dependency. Experiments across multiple datasets demonstrate that the framework improves robustness under heterogeneous data streams while maintaining single-benchmark learning capability and efficiency. Specifically, EEVEE improves average multi-benchmark scores by 10.38 and 24.32 points over Qwen3-4B-Instruct and DeepSeek-V3.2, surpassing SOTA methods GEPA and ACE by up to 37.2% and 48.2%.

1. Introduction

EEVEE addresses heterogeneous real-world task streams by routing inputs to task-specific prompt configurations and jointly adapting the router and prompts. Across multiple benchmarks, it improves performance, retention, and efficiency over foundation-model and test-time prompt-learning baselines.

  • Motivation: Test-time prompt learning adapts prompts to new inputs, distribution shifts, and failure modes without updating model weights.This lightweight mechanism supports self-improving agents that refine behavior through environmental interaction.
  • Problem: Existing methods mainly target single-dataset adaptation, whereas real-world streams combine heterogeneous domains, task formats, and capability mixtures.This regime is formalized as multi-dataset test-time prompt learning.
  • Problem: Cross-dataset interference can make updates for one domain harm another, causing negative retention on previously learned tasks.GEPA and ACE accumulate negative retention as additional benchmarks enter the adaptation stream.
  • Method: EEVEE routes incoming inputs into task clusters and assigns each cluster a suitable prompt configuration to reduce destructive interference.The router preserves prompt-based adaptation while avoiding a single adaptation path for all inputs.
  • Method: EEVEE uses router-prompt co-evolution, interleaving router and prompt learning phases so routing decisions and prompt updates improve together.The approach addresses their mutual dependency because routing determines prompt training data, while prompt behavior influences useful routing policies.
  • Results: 10.38 and 24.32 points are EEVEE’s average-score improvements over Qwen3-4B-Instruct and DeepSeek-V3.2, respectively.EEVEE also improves by up to 37.2% and 48.2% over GEPA and ACE, respectively, and reaches +41.53 cumulative retention gain after all tasks are introduced.

2. Methods

EEVEE handles heterogeneous test-time streams with a fixed target model, a router, and specialized prompts that co-evolve to reduce cross-task interference. Its three-stage procedure initializes diverse prompts, alternates router and prompt evolution, and then performs larger-budget prompt learning after routing stabilizes.

  • Framework: EEVEE maintains K specialized prompts and a router that assigns each input to a prompt while keeping the target model fixed.Inference computes z = R(x; P) and answers with M(x; p_z), preserving prompt-based adaptation across heterogeneous inputs.
  • Co-evolution: Router-prompt co-evolution alternates router updates with prompt updates because routing determines prompt data and prompt quality affects routing decisions.Router evolution fixes prompts while searching for a router; prompt evolution then fixes that router and updates each slot on routed data.
  • Router evolution: Router candidates are mutated and evaluated by downstream prompt correctness, with analysis of cases where another slot succeeds guiding reflective router improvements.Candidates are scored using downstream accuracy, consistency, and balance, with weights annealed toward downstream accuracy; validation admission requires improvement over the phase baseline.
  • Prompt evolution: Prompt evolution independently and in parallel updates each non-empty routed slot through mutation, reflection, validation scoring, and Pareto-front retention.The Pareto frontier preserves complementary prompts, while the empty-prompt floor removes ineffective edits.
  • Three-stage training: EEVEE uses initialization, exploration, and convergence: it first builds diverse prompt slots, alternates lightweight router-prompt search, then fixes the stable router for larger-budget prompt learning.Initialization uses Pareto-front prompts with complementary coverage; convergence reroutes training and validation data before optimizing prompts within each slot.

3. Experiments

EEVEE is evaluated on heterogeneous four-benchmark streams, where it improves average performance and benefits from learned routing with interleaved router–prompt optimization. Additional experiments show strong single-benchmark and generalization results, positive retention as task mixtures grow, and substantially lower token use than ACE.

  • Main results: 51.75 average score: EEVEE improves Qwen3-4B-Instruct over its target-model baseline by 10.38 points and outperforms GEPA and ACE by 14.02 and 16.83 points.The evaluation covers GPQA Diamond, Formula, TheoremQA, and HumanEval, with scores averaged over three runs.
  • Ablation: 51.75 average score: the full method exceeds the default router (43.58) by 8.17 points, the manual router (37.18) by 14.57, and no co-evolution (42.88) by 8.87.The ablation concludes that EEVEE needs both learned routing and interleaved router–prompt optimization.
  • Scaling across task mixtures: +41.53 cumulative retention: EEVEE remains positive as all four tasks are added, whereas GEPA and ACE end below zero after the full mixture.This indicates the advantage of router-conditioned prompt learning emerges as benchmark mixtures grow.
  • Generalization: 54.10 average: prompts learned on Qwen3-4B-Instruct raise DeepSeek-V3.2 from 39.75, while held-out-task performance rises on MBPP from 69.29 to 70.42.On MMLU-Pro, EEVEE decreases from 70.74 to 68.92, a 1.82-point drop.
  • Efficiency: 4.32k total tokens per example: EEVEE stays close to GEPA’s 3.47k and far below ACE’s 21.30k across the four benchmarks.Average input-token usage is 3.00k for EEVEE, compared with 2.44k for GEPA and 20.35k for ACE.

4. Related Work

Related work spans prompt-learning methods that evolved from fixed-objective soft prompts and triggers to feedback-driven optimization, and self-improving agents that use feedback, memory, or evolutionary loops. Recent reflective methods and adaptive playbooks are especially close to EEVEE’s setting.

  • Prompt learning: Prompt learning progressed from optimizing soft prompts, prefixes, or discrete triggers for fixed objectives to black-box and population-based optimization.Later methods use the model, scores, or textual feedback to optimize prompts and programs.
  • Prompt learning: Recent reflective methods include GEPA’s natural-language reflection and Pareto-front selection, alongside ACE’s adaptive-playbook treatment of context.These methods are identified as closest to the paper’s work in the supplied passage.
  • Self-improving agents: Self-improving agents extend prompt learning into feedback loops using natural-language feedback, verbal memory, longer-lived memories, or skill libraries.Examples include Self-Refine, Reflexion, generative agents, and Voyager.
  • Self-improving agents: Evolutionary discovery agents apply related improvement loops to scientific and algorithmic search through code-candidate evolution and adaptive search control.The passage presents these systems as evidence that histories can drive improvement.

5. Conclusion

EEVEE is a multi-dataset test-time prompt learning framework for heterogeneous task streams that reduces cross-dataset interference through router-conditioned prompts and router-prompt co-evolution. Experiments indicate strong mixed-dataset gains, transfer benefits, and practical value when feedback becomes reusable procedures, output contracts, or task-solving strategies.

  • Conclusion: EEVEE addresses heterogeneous task streams by maintaining router-conditioned prompts specialized for compatible task behaviors.Different inputs are assigned to prompts intended to reduce cross-dataset interference.
  • Conclusion: Its three-stage router-prompt co-evolution procedure initializes prompts, explores coupled updates, and refines prompts under a stable router.The procedure addresses mutual dependence between the router and prompts.
  • Conclusion: Experiments show strong gains over prompt-learning baselines in mixed-dataset settings, with benefits increasing as more tasks are introduced.The evaluation also reports reasonable held-out and cross-model transfer.
  • Conclusion: Case studies suggest prompt learning is most useful when feedback becomes reusable procedures, output contracts, or task-solving strategies.Overall, EEVEE is presented as both a practical self-improving method and an empirical lens for real-world test-time prompt learning.

6. Limitations and Social Impact

EEVEE improves multi-dataset prompt learning but retains limitations in reproducibility and adaptation. Its stochastic search prevents guaranteed exact performance reproduction, while its feedback loop still depends on labels and a prepared adaptation set.

  • EEVEE cannot guarantee exact performance reproduction across runs because stochastic search may produce different routers and prompt sets.This limitation is shared with other LLM-based evolutionary procedures.
  • Its feedback loop relies on ground-truth or rule-based labels to accumulate task knowledge, so it is not fully reflection-only.
  • EEVEE still needs a prepared adaptation set rather than a completely online stream.

A. Case Study Details · A.1. Learned Prompt Excerpts

The appendix presents representative learned prompts from diagnostic retests across Qwen3-4B-Instruct and DeepSeek-V3.2 runs. The excerpts show specialized slots enforcing code execution, scientific reasoning, and formula-specific numerical-output policies.

  • A. Case Study Details: The diagnostic retest compares the empty prompt with final router and prompt sets from six completed Eevee runs, spanning three Qwen3-4B-Instruct and three DeepSeek-V3.2 runs.Raw logs contain router decisions, answer calls, and per-example flip bundles; the appendix reproduces representative excerpts from one run of each model.
  • A.1. Learned Prompt Excerpts: Qwen3-4B-Instruct’s code-oriented slot learned a task-execution policy for incomplete Python functions that preserves interfaces, infers continuations, covers edge cases, and avoids extra text.The policy emphasizes producing only the precise output expected by the task, without explanations, markdown, comments, or formatting.
  • A.1. Learned Prompt Excerpts: The Qwen3-4B-Instruct code prompt distinguishes function writing, mathematical or logical problem solving, validation checks, and structured-data processing.It directs the model to parse input types, constraints, edge cases, boundary values, examples, ordering, and special requirements.
  • A.1. Learned Prompt Excerpts: The code slot’s output contract requires only the function body when a task requests only the continuation.This requirement operationalizes the slot’s emphasis on exact execution-compatible output.
  • A.1. Learned Prompt Excerpts: The Qwen3-4B-Instruct science slot routes many GPQA Diamond examples to systematic physical and mathematical reasoning, while lacking task-specific knowledge for every domain.Its instructions cover analytical reasoning, physical modeling, combinatorial mathematics, scientific computation, and exact constraints or known facts.
  • A.1. Learned Prompt Excerpts: For multiple-choice science questions, the learned prompt selects one correct option and formats the response as "Answer: (X)".For physical or astronomical problems, it invokes laws including Stefan-Boltzmann, Kepler, Doppler shift, blackbody radiation, and Newtonian gravity.
  • A.1. Learned Prompt Excerpts: DeepSeek-V3.2’s formula slot learned a stricter numerical-output policy aligned with Formula benchmark inputs containing explicit formulas and requiring compact numeric answers.The prompt specifies computing from the provided formula and data, then outputting only the numeric result with exactly two decimal places.
  • A.1. Learned Prompt Excerpts: The formula procedure identifies variables and values, converts percentages to decimals, substitutes them, preserves financial rates as decimals, rounds to two places, and emits only the resulting number.The required output excludes words, units, labels, currency symbols, and percentage signs.

A.2. Representative Raw Outputs

Representative outputs show learned prompts improving formula execution and HumanEval code completion by enforcing scale, formatting, and executable contracts. On GPQA Diamond, stronger explicit reasoning still fails when grounded in an incorrect physical prior about composition and density.

  • Formula: The learned response applies operating cash flow minus capital expenditure at the correct dollar scale and emits a strict numeric answer.The empty response flips the sign while retaining a million-scale decimal.
  • Formula: The Formula example pairs correct formula application with unit-scale discipline and a parseable final representation.
  • HumanEval: The learned HumanEval response preserves summing even elements at odd indices while satisfying the executable function-completion contract.The empty response captures the expression but does not produce an executable function body.
  • HumanEval: The HumanEval example combines implementation logic with the executable contract expected by the evaluator.
  • GPQA Diamond: On GPQA Diamond, the learned response performs an explicit density comparison but incorrectly treats equal composition as implying equal density.The error reflects a stronger generic reasoning pattern grounded in the wrong physical prior: self-compression changes the rocky-planet mass-radius relation.

B. Reproducibility and Experimental Details

The experiments evaluate EEVEE across heterogeneous benchmark streams using specified model, evolution, execution, and reproducibility settings. They report token-use efficiency while noting that stochastic router and prompt evolution can produce varying routers and prompts across runs.

  • Benchmark scope: The main suite comprises GPQA Diamond, Formula, TheoremQA, and HumanEval, with MBPP and MMLU-Pro reserved for held-out generalization and FiNER and IFBench for single-benchmark diagnostics.Each benchmark is capped at 500 examples and split into train and test partitions.
  • Model configuration: The main target models are Qwen3-4B-Instruct and DeepSeek-V3.2 in non-thinking mode, using model-specific sampling and generation-length settings.Qwen uses temperature 0.7, top-p 0.8, and 16,384-token maximum generation; DeepSeek uses temperature 1.0, top-p 0.95, and 8,192 tokens with thinking disabled.
  • Evolution settings: Each main run retains four bootstrap prompts, while router–prompt co-evolution uses 150 mini-steps, windows of 3, and a 0.005 phase-switch threshold.Evolution settings also include a 10-candidate-step bootstrap budget and router-score weights of 0.6/0.2/0.2 during evolution.
  • Efficiency: 4.32k total tokens per test example are used by Eevee on average, compared with 3.47k for GEPA and 21.30k for ACE.This comparison reports average total tokens per test example across the evaluated methods.
  • Reproducibility scope: Exact routers and prompt texts may vary across runs because router and prompt evolution are stochastic, so reproduction requires matching the adaptation protocol, endpoints or comparable checkpoints, splits, settings, and final-test evaluation.The paper is empirical and does not present formal theoretical results or proofs.

B.1. Hyperparameter Robustness

EEVEE remains stable across the tested router-score and prompt-search hyperparameter perturbations on Qwen3-4B-Instruct. Eight configuration-level macro averages span 5.92 points without configuration collapse, and every configuration improves over its corresponding initial-empty baseline.

  • Evaluation setup: Eight configurations vary router-score annealing, consistency/balance weights, prompt-search budget, minibatch size, and temporary prompt-pool size.Each configuration is evaluated using three independent trials, averaged first within configuration and then compared across configurations.
  • Robustness results: 45.05 to 50.97: configuration-level macro averages span 5.92 points with a sample standard deviation of 1.73 points.Scores are percentages, and each configuration-level result averages three independent runs across four benchmarks.
  • Robustness results: Every configuration improves over its corresponding initial-empty baseline in macro average, with no configuration collapse observed.Individual benchmarks can vary more than the aggregate average.

B.2. Main-Result Variation Across Runs · C. Ethics, Assets, and LLM Usage

EEVEE’s main average score is stable across three runs, although individual benchmark scores can vary because stochastic router evolution discovers different routing policies. The paper reports no new human-subject data or dataset, releases implementation assets, describes responsible-use limitations, and details LLM roles in the method and manuscript process.

  • B.2. Main-Result Variation Across Runs: 1.62 points on Qwen3-4B-Instruct and 1.08 points on DeepSeek-V3.2 are the standard deviations of EEVEE’s main average score across three independent runs.Table 7 reports the mean and sample standard deviation of the main average score used in Table 1.
  • B.2. Main-Result Variation Across Runs: Individual benchmark scores can vary more noticeably because stochastic router evolution discovers different routing policies across runs.Different routing policies allocate examples to different prompt slots.
  • C. Ethics, Assets, and LLM Usage: The experiments use public benchmarks and model API calls without collecting new human-subject data, running crowdsourcing studies, or introducing personal-information datasets.The paper explicitly states that no new human-subject data or personal-information dataset was introduced.
  • C. Ethics, Assets, and LLM Usage: Public benchmarks, provider-served or public model checkpoints, and published baselines including GEPA and ACE support the experiments.The paper states that code, configuration files, reproduction scripts, and asset metadata are released in the official repository, and that the work introduces a method rather than a new dataset.
  • C. Ethics, Assets, and LLM Usage: EEVEE can reduce the need to maintain one prompt-learning run per task family, but noisy, incomplete, or distribution-shifted feedback may reinforce incorrect heuristics.Adapted prompts should be validated on held-out data, and benchmark gains should not be interpreted as deployment reliability guarantees.
  • C. Ethics, Assets, and LLM Usage: LLMs serve as the target model and as multiple prompt- and router-learning components, including researchers, reflectors, selectors, reasoners, and evaluators where applicable.The paper also used GPT-5.4 once to write a fixed manual router for an ablation study.
  • C. Ethics, Assets, and LLM Usage: LLMs were also used for manuscript language editing and formatting, without changing the scientific claims.This use is distinguished from the LLM roles that are core components of the method.
Loading 2606.11182v1…