Source-linked AI summary
Can Large Language Models Reason and Plan?
Subbarao Kambhampati
TL;DR
The paper asks whether LLMs perform principled reasoning and planning rather than retrieve patterns from training data. It tests this by evaluating GPT models on planning tasks, including obfuscated domains, and advocates external verification; the results do not show autonomous executable planning.
Problem
The central question is whether LLMs perform principled reasoning rather than produce correct-looking answers through memory and pattern finding.
Method
The paper evaluates GPT models on planning instances and reduces approximate retrieval by obfuscating action and object names, while examining fine-tuning, prompting, and external verification.
Results
GPT4 reached 30% empirical accuracy in Blocks World, but its performance plummeted under obfuscation, and the studies showed no autonomous generation of executable plans.
Takeaways & Limitations
LLMs can generate useful candidate solutions and approximate domain knowledge, but these outputs should be checked by external verifiers or expert humans in LLM-Modulo frameworks.
Takeaways & Limitations
Human-in-the-loop prompting can produce a Clever Hans effect, because humans who know which solutions are correct may steer the LLM and bear responsibility for accuracy.
Abstract
from arXiv · showhide
While humans sometimes do show the capability of correcting their own erroneous guesses with self-critiquing, there seems to be no basis for that assumption in the case of LLMs.