PaperlayerView exact revision on arXiv

Source-linked AI summary

Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Framework

Krishna Aswani, Huilin Lu, Pranav Patankar, Priya Dhalwani, Iris Tan, Jayant Ganeshmohan, Simon Lacasse

arXiv:2410.06328v2cs.CLcs.AIcs.LG

TL;DR

CoT and Self-Discover rely on fixed reasoning modules, while Auto-Evolve generates task-specific modules and iteratively refines a reasoning plan. On BBH, it consistently outperforms comparison prompting strategies across four models, including an average 7% improvement over CoT.

  • Problem

    CoT and Self-Discover rely on fixed seed reasoning modules, limiting their coverage across diverse tasks.

    Source: §1 · p. 1

  • Method

    Auto-Evolve dynamically generates task-specific reasoning modules and iteratively refines its reasoning structure into an actionable plan.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Results

    Across four models on BBH, Auto-Evolve improves performance over CoT by an average of 7%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors identify dynamic reasoning modules and explicit structures as a potential route toward more interpretable and transparent AI systems.

    Source: §6 · p. 8

    Auto-Evolve’s reasoning structures can become overly complex and add overhead on simpler tasks, so the authors suggest making iterative refinement optional for those tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Recent advancements in prompt engineering strategies, such as Chain-of-Thought (CoT) and Self-Discover, have demonstrated significant potential in improving the reasoning abilities of Large Language Models (LLMs). However, these state-of-the-art (SOTA) prompting strategies rely on single or fixed set of static seed reasoning modules like "think step by step" or "break down this problem" intended to simulate human approach to problem-solving. This constraint limits the flexibility of models in tackling diverse problems effectively. In this paper, we introduce Auto-Evolve, a novel framework that enables LLMs to self-create dynamic reasoning modules and downstream action plan, resulting in significant improvements over current SOTA methods. We evaluate Auto-Evolve on the challenging BigBench-Hard (BBH) dataset with Claude 2.0, Claude 3 Sonnet, Mistral Large, and GPT 4, where it consistently outperforms the SOTA prompt strategies. Auto-Evolve outperforms CoT by up to 10.4% and on an average by 7% across these four models. Our framework introduces two innovations: a) Auto-Evolve dynamically generates reasoning modules for each task while aligning with human reasoning paradigm, thus eliminating the need for predefined templates. b) We introduce an iterative refinement component, that incrementally refines instruction guidance for LLMs and helps boost performance by average 2.8% compared to doing it in a single step.

1 Introduction

Existing prompting strategies rely on fixed reasoning modules, limiting coverage across diverse tasks. Auto-Evolve generates task-specific modules and refines its reasoning plan, outperforming comparison prompting methods on BBH across four models.

1 Introduction

  • CoT and Self-Discover rely on fixed seed modules, and a subset of Self-Discover’s modules dominates usage, limiting coverage across diverse tasks.The examples include prompts such as “think step by step” and “break down this problem.”Source: §1 · p. 1
  • Auto-Evolve creates custom reasoning modules for each task and iteratively refines reasoning structures to meet task-specific requirements.The structures are JSON instruction guidance for solving tasks step by step.Source: §1 · pp. 1–2
  • The framework has three components: a generator creates relevant modules, an initializer composes an initial plan, and an evolver refines it over multiple steps.The evolver’s iterative refinement is described as improving the plan.Source: §1 · p. 2
  • Auto-Evolve improves performance over CoT by an average of 7% across four models on Big Bench Hard.It also improves over Direct Prompt by 12.8% and Self-Discover by 4% on average.Source: §1 · p. 2

2 Related work

Related work develops prompting methods that elicit or structure intermediate reasoning. Auto-Evolve is positioned as a prompting approach among these methods.

2 Related work

  • Prompting methods guide reasoning without changing model parameters, whereas fine-tuning updates parameters using task- or domain-relevant data.Fine-tuning can be computationally expensive; Auto-Evolve is closely related to prompting methods.Source: §2 · p. 2
  • CoT elicits intermediate reasoning, while follow-up methods target rationale selection, consistency, structure, or prior planning.Self-Discover selects and adapts reasoning modules into a coherent structure and is reported to outperform CoT, Self-Consistency, and Plan-and-Solve on various benchmarks.Source: §2 · pp. 2–3

3 Auto-Evolve Framework

Auto-Evolve uses task examples and meta-prompts to build task-specific reasoning modules and a JSON reasoning plan, then iteratively refines that plan. Its four framework parts move from task-level setup to an evolving actionable structure.

3 Auto-Evolve Framework

  • Auto-Evolve has two stages: Stage 1 creates task-related modules and a JSON structure from task examples and meta-prompts; Stage 2 applies the finalized structure to individual instances.Stage 1 runs once per task category, and Stage 2 asks the model to follow the instructions step by step.Source: §3 · p. 3

3.1 Reasoning Module Generator (GENERATE)

  • GENERATE uses a model and meta-prompt with a few unlabeled task examples to create task-specific reasoning modules.Unlike Self-Discover’s predetermined set of 39 modules, GENERATE creates modules dynamically.Source: §3.1 · p. 3

3.2 Reasoning structure initializer (IMPLEMENT)

  • IMPLEMENT uses the first generated module, unlabeled task examples, an example action plan, and a meta-prompt to create an initial key-value reasoning plan.The initial plan provides the starting point for subsequent refinement.Source: §3.2 · p. 3

3.3 Reasoning structure evolver (REFINE)

  • REFINE incorporates additional generated modules into the existing plan, replacing it with the new structure for the next iteration.The process aims to address gaps in the initial plan, such as missing moral, intentional, or counterfactual analysis in reasoning QA.Source: §3.3 · p. 4, Figure 2 · p. 4

4 Experiments

The experiments evaluate Auto-Evolve on BBH using four language models and compare it with Direct Prompting, CoT, and Self-Discover. The setup averages three runs to address output non-determinism.

4.1 Datasets

  • BBH comprises 23 complex reasoning tasks and 5,511 instances, with accuracy used as the evaluation metric.The tasks span algorithmic and arithmetic reasoning, natural language understanding, world knowledge, and multilingual knowledge and reasoning.Source: §4.1 · p. 4

4.2 Models

  • The evaluation uses Claude 2.0, Claude 3 Sonnet, Mistral Large, and GPT-4, with three runs averaged because outputs were non-deterministic even at temperature zero.The passage identifies Tables 1–2 and Figures 4–5 as presenting comparisons with other prompt strategies.Source: §4.2 · p. 4

4.3 Baselines

  • The baselines are Direct Prompting, CoT, and Self-Discover, which respectively omit intermediate reasoning, prompt step-by-step reasoning, or offer thinking styles to guide a reasoning path.For Self-Discover and Auto-Evolve, variation in reasoning-module generation can produce disparities in downstream structures.Source: §4.3 · p. 5, §4.4 · p. 5

4.4 Experiments setup and evaluation

  • Auto-Evolve receives dynamically generated task-specific modules and structures, while two unlabeled task instances are randomly selected as examples during their generation.Answers are extracted programmatically when possible, with annotators manually evaluating responses that cannot be extracted automatically.Source: §4.4 · p. 5

5 Results and Discussion

Auto-Evolve improves BBH performance over CoT and Self-Discover across evaluated models, with task-level gains and category-wide improvements. The results also examine inference efficiency, refinement, generated reasoning structures, and transfer to open-source models.

5.1 Performance

  • Across 23 BBH tasks, Auto-Evolve improves over CoT and Self-Discover by 8.1% and 2.7% with Mistral Large; gains also appear across Claude 2.0, Claude 3 Sonnet, and GPT-4.Reported gains over CoT and Self-Discover are 10.4% and 6.7% for Claude 2.0, 2.5% and 3.1% for Claude 3 Sonnet, and 6.3% and 2.6% for GPT-4.Source: Table 1 · p. 5, §5.1 · pp. 5–6, Table 2 · p. 6

5.2 Efficiency

  • Auto-Evolve uses 1 call each for GENERATE and IMPLEMENT and 4–5 REFINE calls on average, then needs only 1 call per data point once the structure is defined.It achieves similar or better performance than Self-Consistency and Majority Voting with 10–40 times fewer inference calls.Source: §5.2 · p. 6

5.4 Ablation

  • Adding REFINE raises performance from 62.6% with GENERATE + IMPLEMENT alone to 65.4%, a 2.8% boost; REFINE is recommended selectively for complex tasks.Across four Claude 2.0 tasks, Auto-Evolve averages 7.25% improvement over CoT and 4.75% over Self-Discover; the authors report stronger gains on tasks such as Navigate, Arithmetic, and Date Understanding.Source: §5.4 · pp. 6–7, Figure 6 · p. 7

5.3 Themes: Improvement across categories

  • Auto-Evolve improves performance across all four BBH task categories, with the most notable gains in the Algorithm category.The authors suggest its complex reasoning structures are particularly effective for these tasks.Source: §5.3 · p. 6, Figure 5 · p. 7

5.5.1 Deep Diving into Auto-Evolve Reasoning Modules

  • For Boolean Expressions and Disambiguation QA, Auto-Evolve generates task-specific modules, while Self-Discover relies on more generic or less complex modules.Examples include identifying logical operators and tracking noun phrases to resolve pronouns; the authors attribute the specificity to dynamic generation without predefined seeds.Source: §5.5.1 · pp. 7–8

5.5.2 Deep Diving into Auto-Evolve Reasoning Structures

  • On Hyperbaton with GPT-4, Auto-Evolve generates modules for linguistic analysis and adjective-order rules, while Self-Discover’s plan misses the adjective-ordering requirement.The reported Auto-Evolve structure derives the conventional English order of adjectives and produces the correct answer.Source: §5.5.2 · p. 8

5.6 Transferability and Generalizability to OpenSource Models

  • With Auto-Evolve, Llama 3.1 70B reaches 72.0% on disambiguation QA, versus 56.8% for Self-Discover and 60.4% for CoT; smaller models also improve when guided by structures from the larger model.Llama 3.1 8B reaches 62.4% versus 45.2% with direct prompting and 54.4% with CoT; the authors note that smaller models struggle to generate complex plans independently.Source: §5.6 · p. 8

6 Conclusion and Future Work

Auto-Evolve dynamically generates task-specific reasoning structures and integrates iterative refinement, with reported gains over CoT across four models. The authors also identify potential interpretability benefits and plan to explore feedback mechanisms.

6 Conclusion and Future Work

  • Auto-Evolve surpassed CoT by up to 10.4% and averaged a 6.8% gain across GPT-4, Claude 2.0, Claude 3 Sonnet, and Mistral Large.The framework also transfers reasoning structures from larger models to smaller ones such as Llama 3.1 8B.Source: §6 · p. 8
  • Auto-Evolve dynamically generates task-specific reasoning structures, eliminating static seed modules and integrating iterative refinement.The conclusion presents this combination as enabling more effective reasoning across diverse problem domains.Source: §6 · p. 8
  • The authors see potential for more interpretable and transparent AI systems and plan to explore feedback mechanisms for refining reasoning structures.They identify JSON reasoning structures as pivotal to solving tasks effectively.Source: §6 · p. 8

Limitations

The authors identify limitations involving smaller models’ ability to generate reasoning structures, unnecessary complexity on simpler tasks, and nondeterministic outputs. These constraints motivate structure transfer, optional refinement, and repeated trials.

Limitations

  • Smaller models such as Llama 3.1 8B struggle to generate sophisticated reasoning structures autonomously, while structures from larger models can improve their performance.The authors identify optimizing transfer across model sizes and architectures as future work.Source: Limitations · p. 9
  • Cyclic incorporation of insights from multiple modules can make reasoning structures unnecessarily complex and add overhead on simpler tasks.The authors suggest first incorporating all modules in one step and making iterative refinement optional for tasks not solved that way.Source: Limitations · p. 9
  • Even at temperature zero, slight variations in generated modules led to disparities in downstream structures and outputs, so the authors averaged multiple trials at added computational cost.The experiments ran three trials and averaged results to ensure robustness.Source: Limitations · p. 9

Ethics Statement

The ethics statement warns that generated reasoning modules may reproduce or amplify training-data biases. The appendix also presents the meta-prompts used for Auto-Evolve’s components.

Ethics Statement

  • Generated reasoning modules and structures may propagate or amplify biases in the underlying model’s training data; the authors recommend human feedback or other guardrails.The stated concern includes cultural, societal, or linguistic biases in training data.Source: Ethics Statement · p. 9

A Auto-Evolve Prompt details

B Performance on BBH dataset

On BBH, Auto-Evolve exceeded human average performance on many tasks for three evaluated models. A Hyperbaton example contrasts Self-Discover’s decomposition with Auto-Evolve’s task-specific linguistic analysis.

B Performance on BBH dataset

  • Mistral Large with Auto-Evolve outperformed human average performance on 19 of 23 BBH tasks; Claude 2.0 and Claude 3 Sonnet did so on 11 of 23 each.Table 3 reports per-task performance for these models and four prompt strategies.Source: B Performance on BBH dataset · p. 11, Table 3 · p. 11

C Analyzing Reasoning Processes

  • On Hyperbaton, Auto-Evolve’s pattern and comparative analyses identified deviations from adjective-order rules and yielded the correct answer.Self-Discover’s modules instead emphasized simplification and decomposition, including breaking sentences into adjectives and simplifying grammatical rules.Source: C Analyzing Reasoning Processes · pp. 11–12, Figure 8 · p. 12

D Reasoning Module Analysis

The analysis finds that Self-Discover uses only a few of its seed modules on BBH, while Auto-Evolve generates detailed, task-specific modules and structures its guidance for Hyperbaton.

D Reasoning Module Analysis

  • On BBH, Self-Discover uses only a few of 39 seed modules; the authors suggest model bias or seed similarity may contribute, and argue fixed human seeds constrain flexibility versus task-tailored modules.The analysis and its Figure 9 concern Claude 2.0 on BBH.Source: D Reasoning Module Analysis · pp. 12–13, Figure 9 · p. 12

E Auto-Evolve Reasoning Module Comparison

F Auto-Evolve Reasoning Structure Comparison

  • On Hyperbaton, Auto-Evolve’s task-specific keys and sub-keys guide linguistic analysis, pattern recognition, and comparison, and the example reaches the correct answer using grammatical rules.The passage contrasts this structured guidance with Self-Discover’s broad, generic analysis.Source: F Auto-Evolve Reasoning Structure… · p. 13, Figure 11 · p. 14

G Auto-Evolve Performance Comparison

On BBH, Auto-Evolve outperforms prompting baselines on most tasks for GPT-4 and Claude 2.0. Its efficiency comparison reports fewer inference calls while maintaining the highest or on-par accuracy on two tasks.

G Auto-Evolve Performance Comparison

H Efficiency Comparison

  • For a 250-question task, Auto-Evolve uses ≈256 calls, versus 250 for Direct Prompt and CoT, 253 for Self-Discover, 2,500 for CoT+Self-Consistency, and 10K for reward-model voting requiring gold labels.Auto-Evolve’s total includes 6∼7 calls for iterative refinement plus one call per instance.Source: H Efficiency Comparison · pp. 14–15
  • Across Movie Recommendation and Geometric Shapes, Auto-Evolve uses the fewest inference calls while maintaining the highest or on-par accuracy.The comparison uses other methods’ data from Zhou et al., 2024.Source: Figure 14 · p. 15

Related papers

Loading 2410.06328v2…