PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Large Language Models Really Improve by Self-critiquing Their Own Plans?

Karthik Valmeekam, Matthew Marquez, Subbarao Kambhampati

arXiv:2310.08118v1cs.AI

TL;DR

This paper asks whether LLMs can reliably critique their own candidate solutions in iterative planning. It evaluates an LLM-based planning system and finds self-critiquing underperforms use of an external sound verifier, with false positives and little benefit from more detailed feedback.

  • Problem

    The paper investigates whether LLMs can effectively verify and critique their own candidate generations in iterative reasoning, focusing on planning.

    Source: §1 · p. 1

  • Method

    The authors evaluate GPT-4 as both generator and verifier in iterative classical planning, comparing self-critiquing with other plan-generation approaches and assessing verification and feedback levels.

    Source: §1 · p. 2, §5.1 · p. 4

  • Results

    Self-critiquing performs worse than using an external sound verifier, LLM verification produces many false positives, and binary versus detailed feedback has little impact on plan-generation performance.

    Source: §1 · p. 2, §6 · p. 5

  • Takeaways & Limitations

    The findings provide preliminary evidence to question LLMs as verifiers in iterative self-critiquing for planning.

    Source: §1 · p. 2

    Future work is planned to test more instances, domains, and prompting methods, including chain-of-thought.

    Source: §6 · p. 5

Abstract

from arXiv · show

There have been widespread claims about Large Language Models (LLMs) being able to successfully verify or self-critique their candidate solutions in reasoning problems in an iterative mode. Intrigued by those claims, in this paper we set out to investigate the verification/self-critiquing abilities of large language models in the context of planning. We evaluate a planning system that employs LLMs for both plan generation and verification. We assess the verifier LLM's performance against ground-truth verification, the impact of self-critiquing on plan generation, and the influence of varying feedback levels on system performance. Using GPT-4, a state-of-the-art LLM, for both generation and verification, our findings reveal that self-critiquing appears to diminish plan generation performance, especially when compared to systems with external, sound verifiers and the LLM verifiers in that system produce a notable number of false positives, compromising the system's reliability. Additionally, the nature of feedback, whether binary or detailed, showed minimal impact on plan generation. Collectively, our results cast doubt on the effectiveness of LLMs in a self-critiquing, iterative framework for planning tasks.

1 Introduction

The introduction tests whether iterative LLM self-critiquing can improve planning, amid optimism about LLMs verifying their own candidate solutions. Its findings question this approach: performance is worse than with an external sound verifier, and LLM verification produces false positives.

1 Introduction

  • Prior optimism about self-critiquing followed systematic doubts about LLM reasoning, including in planning [12] and arithmetic or logic [3].Source: §1 · p. 1
  • The paper systematically investigates iterative self-critiquing for classical planning, using GPT-4 for both plan generation and verification.The system evaluates verifier performance against ground-truth verification, self-critiquing’s effect on plan generation, and the effect of feedback levels.Source: §1 · p. 2
  • Self-critiquing degrades plan-generation performance relative to an external, sound verifier, while false positives from the LLM verifier undermine reliability.Binary versus detailed feedback has little impact on plan-generation performance.Source: §1 · p. 2
  • The authors characterize their study as preliminary evidence questioning LLM verifiers in iterative self-critiquing for planning.Source: §1 · p. 2

2 Related Work

Related work moves from systematic scrutiny of LLM reasoning toward proposals for iterative self-verification, while this paper examines those abilities in planning. Other multi-LLM approaches also depend on LLM verification, the capability this work scrutinizes.

2 Related Work

  • Proposals to improve reasoning have included having LLMs verify candidate solutions and iterate [10] [7] [6] [14], following systematic studies that questioned reasoning performance [12] [3].Source: §2 · p. 2
  • This work systematically investigates iterative self-critiquing specifically in planning.Source: §2 · p. 2
  • Debate [2] and cross-examination [1] also use multiple LLMs to generate and verify solutions, relying on LLM verification abilities.Source: §2 · p. 2

3 Background

The paper studies classical planning problems represented in PDDL, where a plan is a sequence of domain actions that reaches the goal from the initial state. Figure 1 is identified as the overall evaluation architecture.

3 Background

  • A plan is a sequence of domain actions that, when executed from the initial state, satisfies the goal conditions.Source: §3 · p. 2
  • In PDDL, a planning problem includes a domain, initial state, and goal state; the domain defines predicates and actions.Source: §3 · p. 2
  • Domain actions specify preconditions for applicability and effects that modify the state after execution.Source: §3 · p. 2
  • Figure 1 is captioned as the overall evaluation architecture.Source: Figure 1 · p. 3

4 Methodology

The methodology uses an LLM generator and verifier in an iterative planning loop, with GPT-4 as the default model. Prompts use one-shot generation and zero-shot verification.

4.1 The LLM+LLM planning system

  • The LLM+LLM system iterates candidate plans until the verifier approves one or the 15-iteration threshold is exceeded.The verifier provides feedback when it judges a plan incorrect; the system differs from the approach in [12] by using an LLM rather than external sound verifier VAL.Source: §4.1 · p. 3
  • GPT-4 is the default LLM for the system’s plan generation and verification.Source: §4.1 · p. 3

4.2 Prompt generation

  • The prompt generator converts PDDL domain and instance files into natural-language prompts for the LLMs.Source: §4.2 · p. 3
  • Generator prompts are one-shot, using a domain description, an example instance and plan, then the query instance; verifier prompts are zero-shot.Example instances are randomly selected, and the verifier receives the domain description, query instance, and candidate plan.Source: §4.2 · p. 3

5 Evaluation and Analysis

On Blocksworld, LLM+LLM backprompting slightly outperformed generation without backprompting but fell short of LLM+VAL; verifier false positives and feedback-level comparisons further qualified performance.

5 Evaluation and Analysis

  • Evaluation used GPT-4 at temperature 0 on 100 random Blocksworld instances, with VAL providing ground-truth plan verification.The verifier comparison also used VAL as the sound verifier.Source: §5 · p. 3, §5.2 · p. 4

5.1 Effect of self-critiquing on plan generation

  • LLM+LLM backprompting slightly outperformed the nonbackprompting baseline but underperformed LLM+VAL on Blocksworld.The authors suggest verifier type-1 errors contributed to the gap versus LLM+VAL; the slight gain over the nonbackprompting method may also reflect extra generation opportunities.Source: §5.1 · p. 4, Table 1 · p. 4

5.2 Analysis on the self-critique verifier

  • The verifier correctly classified 61 of 100 instances but labeled 38 invalid plans as valid.The evaluation compared its binary judgments with ground-truth classifications from VAL.Source: §5.2 · p. 4, Table 2 · p. 4

5.3 Effect of the levels of feedback on plan generation

  • Detailed feedback did not significantly improve plan-generation performance over accurate binary feedback when the LLM had ample opportunities to generate a plan.The compared feedback levels included binary feedback, the first inexecutable action and unmet preconditions, and open conditions with unmet goals.Source: §5.3 · pp. 4–5, Table 3 · p. 5

6 Conclusion and Future Work

The paper investigates iterative self-critiquing in classical planning and reports that it was ineffective in its Blocksworld evaluations. The authors plan broader experiments across instances, domains, and prompting methods.

6 Conclusion and Future Work

  • Blocksworld evaluations found self-critiquing ineffective for planning, with verifier false positives undermining reliability and feedback detail having little impact.The authors suggest the core issue lies in binary verification capabilities rather than feedback granularity.Source: §6 · p. 5
  • The authors plan more extensive experiments across instance counts, domains, and prompting methods such as chain-of-thought.Source: §6 · p. 5

Related papers

Loading 2310.08118v1…