Source-linked AI summary
Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models
Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, Bin Cui
TL;DR
Existing reasoning methods depend on designed exemplars or costly multi-query exploration, and do not derive reusable high-level guidance from completed tasks. BoT stores and reuses distilled thought-templates through a dynamically updated meta-buffer; across 10 reasoning-intensive tasks, it reports gains over previous SOTA methods at a fraction of multi-query cost.
Problem
Single-query methods often require task-specific assumptions or exemplars, while multi-query methods are computationally intensive; both lack general high-level guidance distilled from prior tasks.
Source: §1 · pp. 1–2Method
BoT stores high-level thought-templates distilled across tasks in a meta-buffer, retrieves and instantiates a relevant template for each problem, and dynamically updates the buffer.
Source: §1 · p. 2Results
Across 10 challenging reasoning-intensive tasks, BoT improves over previous SOTA methods by 11% on Game of 24, 20% on Geometric Shapes, and 51% on Checkmate-in-One, using 12% of multi-query methods’ average cost.
Source: §1 · p. 2Takeaways & Limitations
The authors conclude that BoT demonstrates SOTA performance on 10 challenging tasks and offers promising prospects for future research and application.
Source: §7 · p. 12BoT’s enhancements are limited on problems requiring human-like creativity; weaker-model initialization may also yield suboptimal thought-templates.
Source: §7 · p. 12
Abstract
from arXiv · showhide
We introduce Buffer of Thoughts (BoT), a novel and versatile thought-augmented reasoning approach for enhancing accuracy, efficiency and robustness of large language models (LLMs). Specifically, we propose meta-buffer to store a series of informative high-level thoughts, namely thought-template, distilled from the problem-solving processes across various tasks. Then for each problem, we retrieve a relevant thought-template and adaptively instantiate it with specific reasoning structures to conduct efficient reasoning. To guarantee the scalability and stability, we further propose buffer-manager to dynamically update the meta-buffer, thus enhancing the capacity of meta-buffer as more tasks are solved. We conduct extensive experiments on 10 challenging reasoning-intensive tasks, and achieve significant performance improvements over previous SOTA methods: 11% on Game of 24, 20% on Geometric Shapes and 51% on Checkmate-in-One. Further analysis demonstrate the superior generalization ability and model robustness of our BoT, while requiring only 12% of the cost of multi-query prompting methods (e.g., tree/graph of thoughts) on average. Notably, we find that our Llama3-8B+BoT has the potential to surpass Llama3-70B model. Our project is available at: https://github.com/YangLing0818/buffer-of-thought-llm
1 Introduction
Buffer of Thoughts (BoT) addresses the limits of existing reasoning prompts by reusing high-level thoughts distilled from prior problem-solving. It combines retrieval and task-specific instantiation with a manager that expands the thought library as more tasks are solved.
1 Introduction
- Single-query methods such as CoT and few-shot prompting contrast with multi-query methods such as ToT and GoT, but both have limitations and lack reusable high-level guidance distilled across tasks.Single-query methods depend on designed prompts or relevant exemplars; multi-query methods are computationally intensive.Source: §1 · pp. 1–2
- BoT stores cross-task thought-templates in a meta-buffer, retrieves a relevant template for each problem, and instantiates it with task-specific reasoning structure.The templates are distilled from different problem-solving processes and can be shared across tasks.Source: §1 · p. 2
- A buffer-manager dynamically updates the meta-buffer and distills templates from solutions, increasing its capacity as more tasks are solved.Source: §1 · p. 2
- The authors attribute accuracy gains to adapting shared templates rather than building reasoning structures from scratch, and associate retrieval followed by instantiation with greater robustness.They also describe directly leveraging historical reasoning structures as a route to improved efficiency over complex multi-query processes.Source: §1 · p. 2
- BoT improves over previous SOTA methods by 11% on Game of 24, 20% on Geometric Shapes, and 51% on Checkmate-in-One, at 12% of multi-query methods’ average cost.The experiments cover 10 challenging reasoning-intensive tasks.Source: §1 · p. 2
2 Related Work and Discussions
The related-work discussion distinguishes BoT from retrieval-augmented language models, prompting-based reasoning, and analogical reasoning by focusing on reusable high-level thoughts for general reasoning tasks.
Retrieval-Augmented Language Models
- Retrieval-augmented LMs query external databases to mitigate hallucination and improve output quality [20] [21] [22]; BoT instead stores high-level thoughts rather than specific instances.The paper frames its meta-buffer as a retrieval database intended to address varied LLM reasoning tasks.Source: §2 · p. 3
Prompt-based Reasoning with Large Language Models
- CoT and related prompts decompose questions into subtasks, while ToT and GoT explore nonlinear reasoning paths; these approaches can demand more resources or manual, task-specific prompts.Meta-prompting methods [15] [40] can require considerable context and do not leverage historical guidelines or thoughts.Source: §2 · p. 3
Analogical Reasoning
- Analogical Prompting and Thought Propagation generate analogous problems, risking noise and error accumulation; Thought-Retriever uses past thoughts but targets textual comprehension and generation.The discussion identifies a remaining need for a higher-level analogical approach to general complex reasoning.Source: §2 · p. 3
3 Buffer of Thoughts
BoT distills task information, retrieves and instantiates reusable thought-templates, then uses a buffer-manager to distill and update the meta-buffer as problem-solving proceeds.
3 Buffer of Thoughts
- BoT processes a task by distilling its information, retrieving and instantiating a relevant thought-template, then summarizing the solution to expand the meta-buffer.The buffer-manager distills high-level thoughts from problem-solving processes and dynamically updates the meta-buffer.Source: §3 · p. 3, §3.3 · p. 5
3.1 Problem Distiller
- The problem distiller extracts essential parameters, variables, objectives, and constraints, then reorganizes and translates them into simpler high-level structures for later retrieval.The distiller separates task-information extraction and comprehension from final reasoning.Source: §3.1 · p. 4
3.2 Thought-Augmented Reasoning with Meta Buffer
- BoT retrieves a template by embedding similarity between the distilled task and template descriptions, then adapts it to the task; new tasks use one of three coarse-grained templates.The authors recommend a similarity threshold δ of 0.5–0.7 for determining whether a task is new.Source: §3.2 · p. 5
3.3 Buffer Manager
- The buffer-manager distills reusable templates by summarizing core challenges, general solution steps, and a broadly applicable answer template, then updates the meta-buffer to limit redundancy.It compares a new template description with existing ones; when the buffer already has the necessary knowledge, it does not update.Source: §3.3 · pp. 5–6
4 Experiments
BoT is evaluated across diverse reasoning tasks and prompting baselines, with reported gains in accuracy, efficiency, and robustness. Its strongest reported accuracy improvements are on complex reasoning tasks, while its inference time is lower than conventional multi-query methods.
4 Experiments
- The evaluation covers mathematical, algorithmic, knowledge-intensive, multilingual, programming, and creative tasks, comparing standard, single-query, and multi-query prompting methods.The 10 tasks include Game of 24, three BBH tasks, three BIG-Bench tasks, P3, MGSM, and sonnet writing.Source: §4 · pp. 6–7
4.1 BoT Achieves Better Accuracy, Efficiency and Robustness
- 23% on Game of 24, 20% on Geometric Shapes, and 51% on Checkmate-in-One are BoT’s accuracy improvements over Meta-prompting.The comparison reports consistent outperformance across challenging benchmarks, especially Game of 24 and Checkmate-in-One.Source: §4.1 · p. 7, Table 1 · p. 7
- BoT requires only 12% of the cost of multi-query methods on average and has reasoning time comparable to single-query methods across tasks.Figure 3 compares logarithmic inference time across benchmarks; the paper attributes lower time to retrieving and instantiating templates rather than constructing reasoning structures from scratch.Source: §4.1 · p. 7, Figure 3 · p. 8
- BoT’s average success rate exceeds the second-best method by 10% across tasks in repeated evaluations of randomly sampled examples.Success rate is the average accuracy over 10 evaluations of a 1,000-example test subset.Source: §4.1 · p. 8, Figure 4 · p. 8
5 Model Analysis
The model analysis examines template distributions, component timing, and performance across model sizes. It reports improvements for smaller and larger models using BoT, with Llama3-8B+BoT potentially surpassing Llama3-70B alone.
Distribution Analysis of Thought-Templates
- MGSM’s more diverse scenarios yield more thought-templates, while simpler tasks such as Checkmate-in-One and Penguins yield more fixed, task-specific templates.The analysis samples 100 tasks from each of six benchmarks and counts templates after updating the meta-buffer from scratch.Source: §5 · pp. 8–9, Figure 5 · p. 9
Distribution Analysis of Time Cost
- Task distillation and template retrieval take relatively little time, while instantiated reasoning takes longer; overall, component time costs are described as relatively balanced.The timing analysis measures average costs for each BoT component across tasks.Source: §5 · p. 9, Figure 5 · p. 9
Better Trade-off between Model Size and Performance
- BoT substantially improves accuracy for Llama3-8B and Llama3-70B on Game of 24, word list sorting, and Checkmate-in-One.Llama3-8B+BoT has the potential to surpass the single Llama3-70B model.Source: §5 · p. 9, Figure 6 · p. 9
6 Ablation Study
The ablations report that problem-distiller, meta-buffer, and buffer-manager each contribute to BoT’s performance. The buffer-manager’s benefits increase over successive rounds as the meta-buffer expands.
Impact of Problem-Distiller
- Disabling problem-distiller reduces accuracy more on complex tasks such as Game of 24 and Checkmate-in-One than on simpler tasks.The paper attributes this difference to the greater difficulty of extracting key information and constraints from complex problems.Source: §6 · p. 10, Figure 7 · p. 10
Impact of Meta-Buffer
- Disabling meta-buffer causes a noticeable performance decline for Llama3-70B and GPT-4, particularly on Game of 24 and Checkmate-in-One.The ablation identifies a larger impact on benchmarks requiring complex reasoning.Source: §6 · p. 10, Figure 8 · p. 10
Impact of Buffer-Manager
- With buffer-manager, accuracy steadily improves across four rounds while reasoning time improves; without it, accuracy shows no upward trend.Each round samples 50 questions from each benchmark, and later rounds use templates obtained from previously solved problems.Source: §6 · p. 11, Figure 9 · p. 11, Figure 10 · p. 11
7 Discussion
The discussion identifies limits for creative problems and for template quality when the meta-buffer is initialized using a weaker model. It also outlines future directions involving external resources and optimization of template distillation.
Limitations and Future Directions
- BoT’s enhancements are limited on problems requiring human-like creativity, which may not rely on a specific thought-template.Source: §7 · p. 12
- A weaker model used to initialize the meta-buffer may produce suboptimal thought-templates because of limited reasoning and instruction-following ability.Source: §7 · p. 12
- Future directions include integrating external resources into an open-domain system and making thought-template distillation optimizable.The paper suggests optimization may improve template quality for more complex tasks.Source: §7 · p. 12
Conclusion
- The conclusion describes BoT as using accumulated prior-task experiences and dynamically distilled templates to raise LLM reasoning capacity over time.Source: §7 · p. 12
A.1 Detailed Thought-Templates
The appendix illustrates thought-templates across six task categories, showing how each organizes task-specific steps toward a solution.
A.1.1 Text Comprehension
- For text comprehension, the template structures tabular data, integrates updates from natural language, compares the relevant attribute, and selects the matching answer.The example uses penguin attributes such as age and weight.Source: §A.1.1 · p. 16
A.1.2 Creative Language Generation
- For creative language generation, the sonnet template plans required words and rhyme, drafts the ABAB CDCD EFEF GG sections, then reviews coherence and form.It identifies a theme before drafting quatrains and a closing couplet.Source: §A.1.2 · p. 16
A.1.3 Common Sense Reasoning
- For common-sense reasoning, the date template handles ordinary day increments, month ends, and the year boundary, accounting for February’s leap-year variation.At year end, it advances to January 1 of the following year.Source: §A.1.3 · p. 17
A.1.4 Mathematical Reasoning
- For mathematical reasoning, the quadratic template computes D = b^2 − 4ac, classifies roots by D, then computes real or complex roots as appropriate.It distinguishes two real roots, a repeated real root, and two complex roots.Source: §A.1.4 · p. 17
A.1.5 Code Programming
- For code programming, the 24-game template permutes input numbers, generates operation combinations, attempts solutions, formats results, and provides a no-solution fallback.The example uses [1, 7, 10, 3] as input and 24 as the target.Source: §A.1.5 · p. 19
A.1.6 Application Scheduling
- For application scheduling, the chess template applies SAN moves to a board, tests legal moves for checkmate, and returns the mating move in SAN.It initializes a board and processes the supplied move sequence before checking candidate moves.Source: §A.1.6 · p. 20
A.2 Prompt for Problem Distiller
The Problem Distiller extracts essential information from a user query and reformats it to support task resolution.
A.2 Prompt for Problem Distiller
- The distiller extracts key values, identifies the objective and constraints, and extends the task into a meta-problem that accommodates input and output variations.It includes the original query’s key information as an example for solving the extended problem.Source: §A.2 · p. 21
A.3 Prompt for Instantiated Reasoning
The instantiated-reasoning prompt selects a reasoning structure suited to the task and applies a relevant thought-template when one is available.
A.3 Prompt for Instantiated Reasoning
- The prompt distinguishes prompt-based, procedure-based, and programming-based structures, associating them with different task types.It links prompt-based reasoning to common-sense and scheduling tasks, procedure-based reasoning to creative and text tasks, and programming-based reasoning to math and code tasks.Source: §A.3 · p. 21
- It asks the model to identify a suitable domain expert, choose a structure from the distilled problem information, and follow a provided thought-template.Source: §A.3 · p. 21