MoT: Memory-of-Thought Enables ChatGPT to Self-Improve

Motivation

• Conventional improvement methods rely on annotated data, retrieval of annotated examples, or fine-tuning, which can require substantial data and computation.Fine-tuning may also reduce general abilities and is infeasible for improving models available only through inference APIs.Source: §1 · pp. 1–2

Contribution

• MoT lets LLMs self-improve through pre-thinking and memory recall without annotated datasets or parameter updates.The model generates thoughts on unlabeled data, stores high-confidence thoughts externally, and recalls relevant memory at test time.Source: §1 · p. 2

Method

• MoT uses Few-Shot-CoT with multiple-path decoding and answer-entropy filtering for pre-thinking, then uses LLM-retrieval to recall reasoning-relevant memories.The proposed retriever is intended to capture deeper logical connections than semantic embeddings.Source: §1 · p. 2

Results

• Experiments show MoT improves ChatGPT in arithmetic, commonsense, factual, and natural language inference tasks, with component and cross-model analyses supporting its generality.The authors also release code and generated reasoning paths to facilitate further research.Source: §1 · p. 2

Few-Shot-CoT

• Few-Shot-CoT prompts an LLM with question, rationale, and answer demonstrations to generate a rationale leading to the final answer.Its demonstrations are provided as question/rationale/answer pairs.Source: §2 · p. 2

Zero-Shot-CoT

• Zero-Shot-CoT elicits reasoning by appending “Let’s think step by step” after the test question.Answer triggers such as “Therefore, the answer is” are used to extract the final answer from the generated reasoning path.Source: §2 · p. 3

Answer Extraction

• Few-Shot-CoT decodes a reasoning sequence from demonstrations and parses its final answer using the trigger “The answer is”.Each demonstration d_i contains an input, rationale, and answer.Source: §2 · p. 3

3 Method

• MoT first has a frozen LLM think over an unlabeled dataset, retain high-confidence reasoning paths as external memory, and recall relevant memory for each test question.The framework avoids dependence on labeled datasets or costly parameter fine-tuning and can operate through an inference API.Source: §3 · p. 3

3.1.1 Let LLM Think before Test Stage

• For each unlabeled example, the model samples multiple reasoning paths, uses majority voting to select the most consistent answer, and stores one corresponding question/rationale/answer path.The authors retain one path for each example to limit memory size.Source: §3.1.1 · p. 3

3.1.2 High-Confidence Thought Filtering

• Answer-entropy removes thoughts above a threshold because lower-uncertainty thoughts are more likely correct; stricter filtering leaves more accurate thoughts and slightly higher accuracy than Max-P at equal filter-out ratios.Figure 4 compares accuracy with filter-out ratio after multiple-path decoding on DROP; answer-entropy yields slightly higher F1-score than Max-P under the same ratio.Source: §3.1.2 · p. 4, Figure 4 · p. 4

3.2 Recalling

• LLM-retrieval first filters semantically irrelevant memories, then selects helpful memories from diverse clusters using the LLM itself.The method uses semantic embeddings and cosine similarity to form candidates, retrieves one memory per cluster, and uses the LLM to choose among candidates.Source: §3.2 · pp. 4–5

3.3 Inference

• The retrieved memories are supplied to Few-Shot-CoT inference, while semantic pre-filtering and question-only candidates reduce the number of LLM calls and inference cost.The authors identify exploring alternative implementations, including LLM-based uncertainty filtering, as future work.Source: §3.2 · p. 5, §3.3 · p. 5

4.2 Main Results

• MoT exceeds Few-Shot-CoT and Zero-Shot-CoT by 3.7 and 9.1 points respectively, and surpasses Zero-Shot-CoT on every dataset.These results are reported for ChatGPT across the evaluated task families.Source: §4.2 · p. 6, Table 1 · p. 6
• Relevant memory and explicit reasoning are both essential for MoT to achieve consistent improvements across extensive datasets.The no-rationale and no-thinking variants perform similarly overall, while both remain stronger than standard zero-shot and few-shot baselines on several datasets.Source: §4.2 · p. 6
• MoT with gold filtering outperforms standard MoT, indicating potential gains from stronger CoT and verification methods.The gold variant uses labels to exclude incorrect memory.Source: §4.1 · p. 5, §4.2 · p. 6

4.3 Analyses

• MoT consistently outperforms Zero-Shot-CoT and Few-Shot-CoT across decoding temperatures and sampling times, although gains slightly diminish with more samples.This evaluates MoT under self-consistency on OBQA, BoolQ, and WikiQA.Source: §4.3 · pp. 6–7, Table 2 · p. 6
• LLM-retrieval further improves memory-based performance beyond semantic embedders, indicating that LLMs better capture complicated reasoning logic than embeddings.MPNet alone already improves over Few-Shot-CoT, while LLM-retrieval performs best among the compared retrieval methods.Source: §4.3 · p. 7, Table 3 · p. 7
• Filtering is necessary because incorrect memory can degrade reasoning, while most filtering thresholds still improve over baselines.The filtering analysis varies answer-entropy thresholds and finds general insensitivity to the threshold hyper-parameter.Source: §4.3 · p. 7, Figure 5 · p. 7
• MoT still outperforms both CoT baselines with only 10 percent of the original memory pool.This supports usability when unlabeled examples or available external memory space are limited.Source: §4.3 · p. 7, Figure 6 · p. 7
• MoT produces consistent improvements with Zero-Shot-CoT and Plan-and-Solve, with further gains when the more advanced Plan-and-Solve method is used.These results show transferability across CoT methods, including settings without manual CoT demonstrations.Source: §4.3 · p. 8, Table 4 · p. 8

Model Augmentation by LLM-generated Data

• Prior model-augmentation methods use LLM-generated datasets, reasoning paths, or instruction data, typically with fine-tuning to improve smaller or instruction-following models.These include ZeroGen, ProGen, ZeroGen+, and methods that generate reasoning or instruction data.Source: §5 · p. 8
• MoT differs from generated-data approaches by improving the LLM through memory-of-thoughts without parameter updates and while remaining compatible with API-accessing LLMs.The related work also characterizes some automatic CoT-demonstration and knowledge-base methods as specialized cases of MoT.Source: §5 · p. 8

Demonstration Retrieval for LLM

• Demonstration-retrieval methods usually retrieve input/output examples from annotated datasets for in-context learning, sometimes using trained retrievers or LM feedback.Prior systems target tasks such as translation, knowledge-based question answering, dialogue state tracking, and semantic parsing.Source: §5 · p. 8
• MoT applies demonstration retrieval to complicated reasoning without annotated datasets or parameter updates, and is presented as the first such exploration in this setting.Its memory includes rationales rather than only input/output pairs.Source: §5 · p. 8

6 Conclusion

• MoT lets LLMs self-improve through Memory-of-Thought without annotated datasets or parameter updates.Source: §6 · p. 8
• MoT significantly improves ChatGPT on arithmetic, commonsense, factual, and natural language inference tasks.Source: §6 · p. 8
• Further analyses show that each component contributes critically and that MoT improves consistently across various CoT methods and LLMs.Source: §6 · p. 9

Limitations

• Answer-entropy filtering can leave erroneous thoughts in memory, motivating future work on stronger false-thought filtering.The authors specifically identify residual mistakes after uncertainty filtering as an unresolved limitation.Source: Limitations · p. 10
• Retrieved memory is currently concatenated with the test input rather than used to verify the test question’s reasoning path.More sophisticated memory-use strategies are left for future work.Source: Limitations · p. 10
• Experiments use a specific unlabeled dataset without internet or external-tool access, while generated memories may amplify language-model bias.The paper frames this as a safe initial setting and identifies debiasing as future work.Source: Limitations · p. 10

A The Example of LLM-Retrieval

• The paper points readers to an LLM-retrieval example in Table 6.Source: A The Example of LLM-Retrieval · p. 15

Overview

• The benchmark covers four task families: arithmetic reasoning, natural-language inference, commonsense reasoning, and factual reasoning.These families include ten datasets overall, including AQuA, DROP, ANLI subsets, OBQA, ComV, BoolQ, FactCK, and WikiQA.Source: B Dataset Details · p. 15

Split, Evaluation Metric and Statistics

• Evaluation uses official test sets for most datasets, validation data for BoolQ, and manually created train/test splits for FactCK and WikiQA.FactCK and WikiQA use training questions as the unlabeled pre-thinking data because released splits are unavailable.Source: B Dataset Details · p. 15
• Accuracy evaluates classification and multiple-choice tasks, while F1-score evaluates abstractive question answering such as DROP and WikiQA.For DROP, scores take the maximum over multiple annotated answers.Source: B Dataset Details · p. 15
• Larger evaluation datasets are randomly reduced to 1,000 examples because of budget limits, and dataset statistics appear in Table 7.Source: B Dataset Details · p. 15

C Performance on Different LLMs

• MoT consistently outperforms baselines on Text-Davinci-002 and Text-Davinci-003.Source: C Performance on Different LLMs · p. 15
• The experiments evaluate MoT’s generality across two different language models.Source: C Performance on Different LLMs · p. 15
• The reported results indicate that MoT’s effectiveness does not rely on one specific evaluated language model.The authors connect this pattern to potential improvements with stronger future models.Source: C Performance on Different LLMs · p. 15

D The Impact of Demonstration Quantity

• The comparison examines MoT and Few-Shot-CoT under varying demonstration quantities, as presented in Figure 7.Source: D The Impact of Demonstration Quantity · p. 15
• MoT consistently outperforms Few-Shot-CoT across varying numbers of demonstrations, showing stable performance in this comparison.Source: D The Impact of Demonstration Quantity · p. 15
• With one or two demonstrations, MoT outperforms Few-Shot-CoT with four demonstrations on OBQA and DROP.Source: D The Impact of Demonstration Quantity · p. 15

• The main implementation uses gpt-3.5-turbo-0301, while text-davinci-002/003 experiments show consistent improvements.Source: E Implementation Details · p. 16
• The pre-thinking stage uses temperature T = 1.2 and n = 16 sampled reasoning paths, while test-time outputs use greedy decoding.Source: E Implementation Details · p. 16
• The filtering threshold is τ = 0.3 and each memory cluster contributes 10 candidates, with exploratory settings showing similar performance.Source: E Implementation Details · p. 16
• SBERT semantic filtering is used for memory recall, and MoT is run separately on each dataset rather than recalling memory across datasets.Source: E Implementation Details · p. 16
• Few-shot CoT demonstrations come from prior work when available and are otherwise generated by ChatGPT from randomly selected training questions.Source: E Implementation Details · p. 16

• MoT is presented as a first step toward LLM self-improvement through a memory mechanism.Source: F Ethics Statement · p. 16
• The experiments use a specific unlabeled dataset, and the LLM cannot access the internet or control external tools.Source: F Ethics Statement · p. 16
• The authors characterize this restricted setting as safe enough to avoid serious impact and unrecoverable societal consequences.Source: F Ethics Statement · p. 16