GuideEvolve: Self-Evolving Agent Framework with Expert Guidance and 3Ts Mechanisms

1 Introduction

• The original framework had 4% unexpected evolution terminations and 14% optimization failures, while its Bedrock replication frequently exhausted tokens and required architectural adjustments.Bedrock response-formatting requirements also necessitated fundamental changes.Source: §1 · p. 1
• GuideEvolve combines task-specific LLM expert feedback with Think Fast and Slow, Throwback, and Termination by choice mechanisms.The expert provides analysis and recommendations; the agent uses them to devise solver improvements, while the 3Ts support cognitive optimization, recovery, and endpoint decisions.Source: §1 · p. 1
• The framework targets AWS Service360 service-improvement and retail-strategy recommendations, emphasizing convergence, stability, resource management, and error handling.The paper presents these as practical application targets rather than benchmark findings.Source: §1 · p. 1

2 Background

• Self-evolving agents aim to improve problem-solving through iterative strategy refinement, drawing on work in reliable behavior, planning, reasoning, memory, and state tracking.Examples include Constitutional AI [1], Plan-and-Solve [11], ReAct [15], GraphRAG ([3]), and IRCoT [10].Source: §2 · pp. 1–2
• Gödel Agent [16] and Darwin Gödel Machine [17] support recursive structural modification, while AgentSquare, AlphaEvolve [8], ADAS [4], and AFlow [18] optimize more constrained components or designs.The cited approaches span modular-component optimization, algorithm-specific evolution, and higher-level agent-system design.Source: §2 · p. 2
• Existing approaches may lack efficient evaluation and degrade solution quality; Gödel Agent [16] reported 4% unexpected termination and 14% optimization failure.The passage identifies stability and consistent performance as especially important in enterprise environments.Source: §2 · p. 2

3 GuideEvolve Framework

• A self-evolving agent coordinates a solver for task execution and an expert for performance evaluation, separating task expertise from evolution control.The expert evaluates without access to solver implementation details, while the agent retains broader visibility of environment, history, and solver logic.Source: §3 · p. 2
• The stateless expert assesses strategically sampled success and failure cases using task questions and solver responses, without access to evolution state.Its isolation is intended to support objective assessment and reduce the agent's context burden.Source: §3 · p. 2
• The expert supplies performance metrics, strengths and weaknesses, and actionable case-study insights; the agent synthesizes them with its broader context into solver-logic changes.This division lets the expert focus on assessment while the agent maintains the evolution trajectory and directs modifications.Source: §3 · p. 2

4 Understanding Guided Evolution Through an Optimization Lens

• GuideEvolve is modeled as optimizing solver logic x to maximize expected task performance, while the optimization lens is explicitly an analogy rather than literal gradient descent.The baseline resembles zeroth-order search; expert guidance is likened to first-order directional information [16].Source: §4 · p. 2

4.1 Baseline Evolution as Zeroth-Order Search

• In the baseline, the agent evaluates performance and chooses update directions itself, with sampling noise, context bias, and bloated processing context contributing to inconsistent trajectories and possible suboptimal solutions.The described context bias arises when raw task-solver interactions are mixed with evolution history.Source: §4.1 · p. 3

4.2 Guided Evolution as First-Order Approximation

• The expert processes task-specific performance data in isolation and produces structured strengths, weaknesses, and recommendations for the agent's guided solver updates.The design aims to remove context bias in observations and reduce context load by excluding raw evaluation data from the agent's history.Source: §4.2 · p. 3

4.3 Predicted Benefits from the Analogy

• The analogy predicts cleaner signals may benefit MMLU more than capability-constrained MGSM, while variance reduction is expected from less context bias and processing load.The conceptual account expects evaluation-sampling variance to remain unchanged because both approaches use identical sampling strategies.Source: §4.3 · pp. 3–4

4.4 Empirical Support and Scope of the Analogy

• The reported 25-59% variance reduction and 50% fewer required iterations are consistent with the predicted stability and informed-search benefits.The paper describes the optimization lens as intuition, not formal proof.Source: §4.4 · p. 4

3Ts

• The 3Ts are the framework’s three self-governing mechanisms for autonomous and efficient evolution control.They comprise Think fast and slow, Throwback, and Termination by Choice.Source: §5 · p. 4

Think, Fast and Slow

• Think Fast and Slow uses fast execution for multiple low-risk actions and deliberate step-by-step planning for resource-intensive or error-prone operations.Expert evaluation begins only after confirmed successful modifications to the solver’s logic.Source: §5 · p. 4

Throwback

• Throwback preserves the solver’s previous best-performing state and performance history, enabling rollback, strategic state examination, and context-driven recall.The agent can examine historical information without a strict hierarchy between states.Source: §5 · p. 4

Termination By Choice

• Termination by Choice lets agents stop autonomously, while validation checks require at least five trials to prevent premature termination.Some agents attempt to stop early, while others continue beyond suggested conditions.Source: §5 · p. 4

Resource Management System

• The implementation addresses token limitations and response completeness through a resource-management system.The paper frames these platform constraints as opportunities for optimization.Source: §6 · p. 4

Token Budget Optimization

• The system prioritizes essential operations and reserves token budget primarily for tool calls, while keeping system messages immutable.Tool calls are identified as the critical mechanism for action execution.Source: §6 · pp. 4–5
• A dynamic sliding window adjusts output token limits to input-context size while managing action history.The approach is intended to optimize token use and ensure tool-call generation under varying context loads.Source: §6 · p. 5

Response Validation and Context Management

• When responses are truncated, progressive context reduction removes older action history first, expands reduction if needed, and retains at least one recent action message.JSON validation and tool-call verification are used to detect response completion.Source: §6 · p. 5

Resource Management System

• The integrated resource-management approach, coupled with throwback, supports operational continuity while minimizing information loss.It balances immediate operational needs with long-term evolution stability.Source: §6 · p. 5

7 Experiments Results

• Experiments use 800 test samples and six trials per task, with 20 training samples for validation; the reported benchmark results use Configuration A.Configuration B is reduced-scale for visualizations and ablations, and its higher accuracies should not be compared with Configuration A benchmarks.Source: §7 · p. 5

7.1 Performance Analysis

• 89.8% accuracy (±1.5%) on MGSM is comparable to Gödel-free (90.6% ±2.0%), while 89.1% accuracy (±0.9%) on MMLU exceeds Gödel-free (87.9% ±2.2%).GuideEvolve also reports 25% lower variance on MGSM and 59% lower variance on MMLU.Source: Table 2 · p. 6, §7.1 · p. 6
• 100% evolution success across both tasks and 15.1 MGSM and 13.4 MMLU iterations versus Gödel's 30 indicate robust convergence with roughly half the evolving effort.Throwback supports recovery from suboptimal paths, while termination by choice halts evolution at a performance plateau.Source: Table 3 · p. 6, §7.1 · pp. 6–7

7.2 Evolution Traces and Discussions

• In one MGSM trace, test accuracy rose from 0.9 to 0.97 after throwback at iteration 9, and the agent terminated at iteration 18.The trace also shows 100% validation accuracy coinciding with lower test accuracy, limiting expert guidance in that overfitting case.Source: §7.2 · p. 7, Figure 3 · p. 7
• Across both tasks, self-modification and evaluation were the most frequent actions, followed by throwback and self-termination.Evaluation uses task-expert guidance to assess modifications.Source: Figure 4 · p. 7, §7.2 · p. 7
• Three MGSM agents starting at 0.9 followed different improvement paths, reaching 0.96 within eight iterations or 0.97 after 18 or 19 iterations.One agent recovered from a temporary accuracy drop to 0.Source: §7.2 · pp. 7–8

7.3 Ablation Study

• In a baseline retaining all other features, removing task expert guidance and throwback mechanisms drops success rate by 20%, decreases accuracy by 17%, and increases iterations by 8.9.Without task expert guidance, the agent struggles with self-evaluation; without throwback functionality, it has difficulty recovering from performance deterioration, and frequent context overflow can lead to fatal errors.Source: §7.3 · p. 8

7.4 Limitations and Threats to Validity

• The evaluation covers two benchmark tasks, lacks full cross-model validation, and adds expert-module LLM-call costs whose production cost-benefit remains unanalysed.Generalization to open-ended business applications with multifaceted success criteria remains future work.Source: §7.4 · p. 8

8 Future Work

• Future work proposes co-adaptive evolution of solver and expert components, meta-evaluation of guidance and solver performance, and integration with Strands Agents.It also identifies business applications as a direction requiring evaluation metrics for complex scenarios and problem decomposition that preserves coherent evolution strategies.Source: §8 · p. 8

A Goal Prompt

• The goal prompt highlights the guidance and 3T mechanisms used in the self-evolving agent framework.The highlighted material is shown in red.Source: A Goal Prompt · p. 9

B Illustration of Termination by Choice

• Figure 6 illustrates the agent autonomously deciding when to terminate its evolution process.Source: B Illustration of Termination by Choice · p. 9

• Up to 0.97 accuracy appears in three MGSM trial trajectories using 5 training and 100 test samples, so the plot illustrates evolution rather than benchmark performance.The paper attributes the higher observed accuracies to the smaller test scale.Source: C Accuracy Progression by Independent… · p. 9

D Examples of Best Policies Found by GuideEvolve

• GuideEvolve discovered different solver strategies for MGSM and MMLU without explicit descriptions of the candidate models’ capabilities.SONNET_V36 and SONNET_V37 identify Claude 3.5 Sonnet v2 and Claude 3.7 Sonnet, respectively.Source: D Examples of Best Policies Found by… · p. 9
• The MGSM solver generates responses at three temperature settings and uses majority voting, with a Bengali-mathematics fallback that translates questions before reasoning.The strategy improves on the initial solver’s single-response approach.Source: D Examples of Best Policies Found by… · p. 9
• For MMLU, the discovered solver uses question length as a complexity proxy, selecting Claude 3.7 Sonnet for longer questions and Claude 3.5 Sonnet v2 at temperature 0.8 for simpler ones.Source: D Examples of Best Policies Found by… · p. 9

E An Example of Task Expert’s Performance Report

• A MGSM task expert report analyzes solver strengths, weaknesses, and failure cases, then provides actionable recommendations for improvement.Source: E An Example of Task Expert’s… · p. 9, Figure 10 · p. 14