PaperlayerView exact revision on arXiv

Source-linked AI summary

Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving

Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy Lillicrap, Danilo Rezende, Yoshua Bengio, Michael Mozer, Sanjeev Arora

arXiv:2405.12205v1cs.AIcs.LGstat.ML

TL;DR

The paper asks whether LLMs have knowledge of the skills they use to solve math problems, and whether that knowledge can improve problem solving. It uses an LLM to create clustered skill labels and retrieves matching solved examples at inference; the paper reports gains on math tasks, including an 11.6% improvement over CoT on MATH.

  • Problem

    The paper asks whether LLMs have metacognitive knowledge of their reasoning and learning-relevant processes, including a more detailed catalog of the skills they use for math than broad dataset labels provide.

    Source: §1 · pp. 1–2

  • Method

    A powerful LLM labels math training questions with skills, clusters similar labels into coarser categories, and supplies skill-matched solved examples to an LLM solving a new question.

    Source: §1 · pp. 2–3

  • Results

    The method improves performance across mathematical datasets and transfers to weaker LLMs; on MATH, it improves 11.6% over CoT.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    The paper presents LLM-assigned skill labels as actionable in-context guidance for mathematical problem solving.

    Source: §1 · p. 3, §5 · p. 9

    The framework is evaluated in math; application to other kinds of problem solving is left for future work.

    Source: §1 · p. 3

Abstract

from arXiv · show

Metacognitive knowledge refers to humans' intuitive knowledge of their own thinking and reasoning processes. Today's best LLMs clearly possess some reasoning processes. The paper gives evidence that they also have metacognitive knowledge, including ability to name skills and procedures to apply given a task. We explore this primarily in context of math reasoning, developing a prompt-guided interaction procedure to get a powerful LLM to assign sensible skill labels to math questions, followed by having it perform semantic clustering to obtain coarser families of skill labels. These coarse skill labels look interpretable to humans. To validate that these skill labels are meaningful and relevant to the LLM's reasoning processes we perform the following experiments. (a) We ask GPT-4 to assign skill labels to training questions in math datasets GSM8K and MATH. (b) When using an LLM to solve the test questions, we present it with the full list of skill labels and ask it to identify the skill needed. Then it is presented with randomly selected exemplar solved questions associated with that skill label. This improves accuracy on GSM8k and MATH for several strong LLMs, including code-assisted models. The methodology presented is domain-agnostic, even though this article applies it to math problems.

1 Introduction

The paper asks whether LLMs have metacognitive knowledge of the skills they use in mathematical reasoning and whether that knowledge can improve problem solving. It builds a repository of clustered skill labels and solved examples, reports performance gains, and leaves application beyond math for future work.

1 Introduction

  • The paper asks whether LLMs possess metacognitive knowledge and whether such knowledge can be used to improve their capabilities.Here, metacognitive knowledge concerns knowledge of one’s cognitive processes and learning-relevant properties of information.Source: §1 · p. 1
  • The targeted knowledge is an LLM’s catalog of skills for solving math questions, more fine-grained than broad labels such as algebra or probability.The paper frames this as a way to understand the skills from the LLM’s viewpoint.Source: §1 · p. 2
  • A powerful LLM labels training questions with fine-grained skills, clusters similar labels into broader families, and reclassifies examples for a skill-exemplar repository.At inference, the tested LLM selects a skill and receives solved examples associated with it.Source: §1 · p. 2, Figure 1 · p. 2
  • The repository’s skill-matched examples are presented as an addition to prompting methods such as Chain-of-Thought.The paper reports that skill-exemplars improve problem solving on the same dataset.Source: §1 · p. 3
  • The study evaluates the approach in math; applying it to other kinds of problem solving is left for future work.The authors suggest the method may apply more broadly but do not evaluate that scope here.Source: §1 · p. 3
  • On MATH, the skill-based approach improves 11.6% over CoT and code-based solutions improve 7.52% over the PAL baseline.The paper also reports improved in-context performance for weaker LLMs and transfer from the GSM8K repository to other math datasets.Source: §1 · p. 3

2 Related Works

Related work covers latent-skill inference in human learning and skill induction in machine learning, distinguishing the paper’s black-box LLM approach from parameter-defined skills.

2 Related Works

  • In human learning, statistical methods infer latent skills from data and use them to forecast student learning more accurately.Source: §2 · p. 3
  • Machine-learning studies of skill induction define skills in terms of model parameters.Source: §2 · p. 3
  • Unlike those parameter-based studies, this paper uses a powerful LLM as a black box to identify and consolidate skills.Source: §2 · p. 3

3 Automated Skill Discovery

The method discovers fine-grained math skills with a powerful LLM, clusters them into broader labels, and uses skill-matched solved examples at inference. The paper reports that the repository can also help weaker models and integrate with existing prompting approaches.

3.2 Inference at test time

  • At test time, the LLM selects a skill from the repository and receives randomly selected exemplars for that skill as few-shot prompts.The GSM8K repository is also applied to other math word-problem datasets.Source: §3.2 · p. 4

3.3 Skills from strong LLMs improve weaker LLMs

  • Skill labels from a stronger LLM are reported to improve weaker models, and skill-specific exemplars can be combined with CoT, PAL, and self-consistency prompting.The repository’s skill labels are described as human-interpretable and useful to other LLMs, including Mixtral.Source: §3.3 · pp. 4–5
  • The MATH table caption reports that Skill-Based CoT outperforms all other methods across topics, with experiments using GPT-4-0613.Source: Table 2 · p. 5

4 Experiments

The experiments evaluate skill-based exemplars with text and program prompts, then test transfer across models and datasets. Skill-based prompting improves results across these settings, while analysis identifies limits in applying secondary skills and interpreting specific problem properties.

4 Experiments

  • Evaluation spans text-based CoT, program-based PAL, and transfer across models and datasets, with random, topic-based, complexity-based, and retrieval baselines.The main experiments use GPT-4-0613; transfer to another model is tested with Mixtral 8x7B.Source: §4 · pp. 5–6

4.1 Text-based Prompts

  • On GSM8K, Skill-Based outperforms CoT, Random, and Retrieval-RSD; adding self-consistency further improves its performance.Table 3 covers results for GPT-3.5 Turbo and GPT-4-0613.Source: Table 3 · p. 6, §4.1 · p. 6

4.2 Program-Aided Prompts

  • On 500 MATH test examples, skill-based program prompting improves over PAL by 7.52%, despite using only one code-based exemplar.Table 4 reports program-aided results across MATH topics.Source: Table 4 · p. 7, §4.2 · p. 7

4.3 Transfer of Skill Exemplars

  • On Mixtral 8x7B, Skill-Based without self-consistency matches CoT with self-consistency, and with self-consistency exceeds it by 4.0%.The transfer experiment uses GPT-4-0613 skill labels and exemplars.Source: Table 5 · p. 7, §4.3 · p. 8

4.1 Text-based Prompts

  • On MATH, Skill-Based CoT beats standard CoT by 11.6%; skill knowledge also improves text- and program-based prompting and transfers across models and datasets.The broader cross-setting pattern is reported across the experiments.Source: §4 · p. 5, §4.1 · p. 7

4.3 Transfer of Skill Exemplars

  • Across six other word-problem datasets, GSM8K-derived skills outperform CoT throughout and achieve the highest accuracy in four cases.Skill-Based also beats CoT + PAL in four of the six datasets.Source: Table 6 · p. 8, §4.3 · p. 8

4.4 Analysis

  • Skill-Based prompting can still fail to apply the primary skill or to handle secondary skills and specific problem properties.Examples include choosing the wrong input for a logarithm and miscounting a hexagon’s diagonals.Source: §4.4 · p. 9
  • Analysis tracks main-skill, secondary-skill, and calculation errors, which can co-occur; Skill-Based prompting raises skill success and reduces secondary-skill and calculation errors.Examples contrast topic-based mistakes about trigonometry and negative cubes with correct skill-based reasoning.Source: §4.4 · p. 9

5 Discussion and Conclusion

The paper presents a framework that extracts LLM-estimated math skills and uses a repository of skill-labeled question-answer examples to guide solving unseen problems. Experiments report enhancements across diverse mathematical datasets and models, while the discussion identifies limits in representing compound skills and dependence on advanced models.

5 Discussion and Conclusion

  • The repository yields substantial empirical enhancements across mathematical datasets, and its benefits transfer to weaker LLMs.The paper primarily studies in-context learning and identifies fine-tuning as a future direction.Source: §5 · pp. 9–10
  • The framework extracts LLM-estimated skills to categorize math questions and builds a repository of skill-labeled question-answer pairs for guiding solutions to unseen problems.The paper reports that this approach improves in-context learning across grade-level and competition-level math challenges.Source: §5 · p. 9
  • Assigning one named skill per question can miss the secondary skills that compound math problems often require.The paper suggests hierarchies of skills and multiple labels per datapoint as a possible richer approach.Source: §5 · p. 10
  • The current framework relies on advanced models such as GPT-4, while extending the methodology to fine-tuning remains future work.The paper notes that skill discovery improved GPT-4 in-context learning and suggests fine-tuning GPT-4 may raise its capabilities.Source: §5 · p. 10

7 List of Skills

This section lists the skills in the GSM8K and MATH skill exemplar repositories after clustering.

7 List of Skills

  • The section lists the clustered skills that make up the skill exemplar repositories for GSM8K and MATH.The passage does not enumerate the skill labels.Source: §7 · p. 14

8 Prompts Used for Skill Labelling and Skill Clustering

The section identifies prompts for labeling training and test examples and for clustering the labeled skills.

8 Prompts Used for Skill Labelling and Skill Clustering

  • Figure 2 presents separate prompts for labeling training examples, clustering skills, and labeling test examples.The training-labeling, clustering, and test-labeling prompts appear at left, middle, and right, respectively.Source: §8 · p. 14

9 Related Works: Prompting Strategies

The section describes existing prompting approaches and contrasts fixed or complexity-based example selection with the paper’s skill-relevance-based selection. The skill-exemplar approach can be integrated with several prompting methods.

9 Related Works: Prompting Strategies

  • CoT provides math questions with detailed reasoning chains, while PAL prompts code-based solutions using code-based examples.The section also describes self-consistency as selecting the most frequent answer and progressive hint prompting as feedback-based refinement.Source: §9 · pp. 14–15
  • Figure 2 shows distinct prompts for training-set skill labeling, skill clustering, and test-set skill labeling.These are shown at the figure’s left, middle, and right, respectively.Source: Figure 2 · p. 15
  • Skill exemplars dynamically select in-context examples by relevance and can integrate with CoT, PAL, and self-consistency prompting.The approach replaces conventional in-context examples with examples selected from the Skill Exemplar Repository.Source: §9 · p. 15
  • Existing prompting methods use fixed in-context examples, while ComplexCoT selects examples with higher reasoning-chain complexity.The proposed approach instead selects examples based on relevance rather than complexity.Source: §9 · p. 15

10 Experimental Details

The experiments cover grade-school word problems and competition-level MATH, examining per-skill accuracy and the construction of skill-exemplar repositories.

10.2 Grade Level Math Word Problems

10.1 Description of Datasets

  • The evaluated datasets span grade-school word problems and competition-level MATH, with MATH covering seven mathematical topics.The listed word-problem datasets include SVAMP, ASDIV, Single EQ, Single OP, AddSub, and MultiArith.Source: §10.1 · pp. 15–16

10.2 Grade Level Math Word Problems

  • The Skill-Based approach outperformed Random on 11/18 GSM8K skills.Figure 3 compares per-skill accuracies for the two approaches.Source: §10.2 · p. 16, Figure 3 · p. 17

10.3 MATH Dataset: Competition Level Math Problems

  • For MATH, clustering reduced hundreds of fine-grained labels per topic to 11–23 coarse skills per topic.Before clustering, the topic-level counts ranged from 406 to 877 skill labels; afterward, they ranged from 11 to 23.Source: §10.3 · p. 16

10.4 Comparing Skill Annotation Models

The section compares models for skill annotation and clustering, then describes adapting PAL prompts to include skill-matched exemplars.

10.4 Comparing Skill Annotation Models

  • Llama-2 70B was excluded as a skill annotator because it could not provide sensible skill names and struggled with the prompt instructions.Source: §10.4 · p. 17
  • GPT-4, GPT-3.5, and Mixtral produced skill labels, but Mixtral failed to cluster them sensibly, grouping all skills together.GPT-4 was more descriptive and sometimes more accurate in labeling than the other models.Source: §10.4 · p. 17
  • Using GPT-4-created skill repositories yielded higher MATH performance than using GPT-3.5 repositories, indicating higher-quality GPT-4 annotations.The comparison is presented in Table 12.Source: §10.4 · pp. 17–18, Table 12 · p. 18

10.5 Program Assisted Language Models

  • The PAL prompt adaptation replaces three text-solution exemplars with same-skill repository examples while retaining a code-solution exemplar.The displayed format includes three question-and-text-solution pairs followed by a question-and-code-solution pair.Source: §10.5 · p. 18

10.6 Examples where the Model makes Mistakes despite Receiving Skill Exemplars

The section examines mistakes despite skill exemplars, describes GPT-4 error categorization, and compares Skill-Based and Topic-Based approaches on error-related metrics.

10.6 Examples where the Model makes Mistakes despite Receiving Skill Exemplars

  • Table 13 presents cases where models answered incorrectly despite receiving skill-based exemplars relevant to the problem.The table marks correct reasoning steps in blue and errors in red.Source: §10.6 · p. 18, Table 13 · p. 19

10.7 Prompting GPT 4 to classify errors

  • The GPT-4 classification prompt categorizes examples as skill, secondary-skill, or calculation errors.The prompt is shown in Figure 5.Source: §10.7 · p. 18, Figure 5 · p. 23

10.8 Performance on Ablation Metrics

  • Skill-Based prompting achieved higher skill success rates and lower secondary-skill and calculation errors than the Topic-Based approach.Figure 4 compares skill success rate, secondary skill success rate, and calculation success rate.Source: §10.8 · p. 19, Figure 4 · p. 22
Loading 2405.12205v1…